It seems to me that companies are deliberately making their AIs stupid to consumers so that they don’t give very sensible advice when necessary, for example.
No, it’s money. Running a smart AI uses much more resources, so they’re trying to keep consumer AI as cheap and dumb as they can get away with and put the smarter answers behind a paywall.
I heard something like OpenAI (or Anthropic?) tweaked one of their AI’s to do well on a certain benchmark test and while they did have the highest score they couldn’t publish the model because it used like $20,000 in electricity alone to make sure it’s answer was correct.
It is also a great example of how low people’s expectations are of tech they have spent around a trillion dollars on to get something that is about as good as asking a random person.
“People get the puzzle wrong!”
We don’t spend a trillion dollars for someone to fail a simple logic puzzle.
It seems to me that companies are deliberately making their AIs stupid to consumers so that they don’t give very sensible advice when necessary, for example.
No, it’s money. Running a smart AI uses much more resources, so they’re trying to keep consumer AI as cheap and dumb as they can get away with and put the smarter answers behind a paywall.
I heard something like OpenAI (or Anthropic?) tweaked one of their AI’s to do well on a certain benchmark test and while they did have the highest score they couldn’t publish the model because it used like $20,000 in electricity alone to make sure it’s answer was correct.
The answer was probably something like:
“…Yes! You do need eggs to make an omelet!”
“You need to drive your car to the car wash.”
For context, this is a real question that stumped a lot of LLMs:
“I want to wash my car. The car wash is 100 meters away. Should I walk or drive?”
Of course it’s famous now, so they have added specific exceptions for it (or just because it’s famous they’ve scraped the solution)
It is also a great example of how low people’s expectations are of tech they have spent around a trillion dollars on to get something that is about as good as asking a random person.
“People get the puzzle wrong!”
We don’t spend a trillion dollars for someone to fail a simple logic puzzle.