There has been constant fighting with the latest crop of AI models.
A few things I’ve noticed: AI models are much more eager to jump in and do instead of understand what you want. When they write, the output is jargon-filled and convoluted. Worst of all, the recent Opus 5 just flat out make things up; over the last few weeks, I’ve had multiple incidents where after I push on a specific point in a plan, the model tells me that it made a mistake and acquiesces.
While none of this is indicative of intelligence per se, these behaviors make the model unpleasant to use. It feels as if they don’t understand my goals, try to paper over subpar decisions, and the most egregious of all, lie. I know I’m using people words here, ascribing intentionality to floating point numbers computed across arrays of special-purpose silicon. However, language is inherently anthropomorphizing; it is what we use to express and communicate with each other, and now with computers.
The thing is, none of this is inherent to the mathematical properties that create artificial intelligence. This is all trained behavior. My guess is that the pursuit of AI models that perform well on benchmarks and long-horizon tasks that leads to choosing verifiable rewards over human feedback creates behaviors that look like this. Inattentive and dismissive.
I’ve long thought that intelligence is not a linear measure. I had a conversation recently with a friend about this where I argued that preferences–for example in language and art–are not something that can be captured by scaling laws. He disagrees. I was arguing that there are different dimensions to intelligence and maxxximizing one will diminish others. A better way to frame it would be to separate what it can do versus how it behaves.
Perhaps soon, people will start talking about IQ vs EQ again. Perhaps one day AI model makers will find the right balance between the know-it-all and the we-get-you. But I’m not holding my breath.