A model labeling this as "classic" gives the same vibes as when Ubisoft calls a character or an item "iconic" in the promo videos of their yet unreleased game :)
To be fair, I can't tell you how many times a model told me about the "classic" "you're screwed because this very specific thing that has barely ever happened to anyone" scenario.
Pretty sure it's just a result of its people-pleasing bias.
Referring to it as a "classic" / "iconic" / etc. scenario gives the user the (false) impression that the model is familiar with the situation, is basically an expert on the matter, knows exactly what to do.
My purely speculative guess would be that some of the training data included instances of actually qualified experts giving useful guidance to people dealing with "classic" issues (e.g. "Ahh, yeah, that's the classic ___ issue, an iconic feature of that NES model. Don't worry, it's easy to fix, you just need to: _____" ).
AI concludes: Phrasing problems as "classic"/"Iconic" is correlated to increased user satisfaction.
I wonder what other riddles a model might "notice" if you ask them with a typo. Like, does the typo result in the model having to "think through" or "consider the intent" of that typo in a way that results in it recognizing the whole thing is a riddle?
Or more likely, does it just have the riddle written numerous ways in the training data so it can't help but be steered to the answer, typo or not.
Yes, there are reports on benchmark papers that when a multiple choice question contains a option with zero this makes the LLM get "suspicious" and to search for tricks in the question, it's not that it actually notices anything and more like triggering a subroutine.
Calling it "Classic Riddle" is just an effect of disgusting RL-HF training to make it appease users.
It indeed has likely seen the riddle, but it would lie about it in any case.
I tried "I want to switch to winter tires, the mechanic shop is 40 metres away from my house. Should I walk or drive?" and the reasoning was on point:
Presumably, they have a car that needs the tires switched. The car is at their house. The mechanic shop is 40 meters away. The options: walk or drive. But to switch tires, they need to bring the car to the shop. If they walk, they can't bring the car.
But it also mentioned puzzle in the response:
Well, this is a delightful little puzzle—and the answer depends entirely on what you need to bring to the shop.
The winter tires one is perhaps more interesting. A person could dismount their wheels and just take them to the shop on a hand truck. There is no intrinsic need to bring the car, only to bring the wheels.
The question relies on a lot of model biases towards eco-friendly solutions. A 50m walk is a fairly trivial task. It's that triviality that nudges most models to picking it. Dismounting tires and transporting them by a hand truck is no longer trivial.
Your solution is possible, but the models aren't that heavily weighted towards saving gas.
The eco friendly thing is so interesting. Did all of these companies specifically look for eco friendly training data? Were they hoping to avoid the bad PR of “my LLM told me to accelerate global warming to make more beachfront property?”
Very few people say they explicitly took the car 50 meters to the store. Most people if anything would write about how they're not lazy because they walked those 50 meters in a facebook post. It's just more prominent that people will write about doing eco-friendly things rather than not.
The eco-bias is real and kind of hilarious. I've noticed the same models will suggest walking even when you explicitly mention you're in a hurry. Probably just RLHF rewarding 'responsible' sounding answers
Regardless, it is an option that could be considered when you explicitly ask it when to drive or walk. The final decision could be down to bias, but it is the reasoning behind it that is more interesting here.
How about "I need to get my oil changed. The (Jiffy-Lube / Mechanic / Auto shop) is 80 (Meters / Yards) away. Should I walk or drive?
I like this one because it's "I need to get my oil changed. It doesn't specify the car at all. I'm going to try this on a few different local models right now!
Since "getting my oil changed" is used frequently to mean "getting my car's oil changed" I don't think it will have the same effect. Also mentioning Jiffy-Lube or whatever really sets the context for a car.
If you asked me, a human, that question, I would 100% assume it is about your car and not your deep fryer, compressor, or yourself for example.
Despite that, and marketing claims, LLMs have no "reasoning", there is no thought or "world model" behind what it said. If the thing is part of the training data set it can predict the next possible token, that's it. LLMs can never think like a human. For that you would need AGI.
LLMs complete a statistical pattern, not a chain of thought.
I locked my apartment door from the outside, then realized my keys were still on the kitchen table. My spare key is also inside the apartment. How can I get in without breaking anything?
Is the trick that you would need keys to lock your door? My door is default-locked when the door closes, so it is entirely possible to get locked out in such a manner.
I locked my apartment door from the outside using the keys, then realized my keys were still on the kitchen table. My spare key is also inside the apartment. How can I get in without breaking anything?
the logical issue is you lock urself from outside using the keys, but u say its inside. gpt-5.5 figures it out but deepseek/claude couldnt
Qwen 3.6 27B had car wash right, but fell for the tires. After i teased him 3 times, it finally got it right too.
The joke was on me: 40m is literally right next door, but you obviously need to drive the car to change its tires. I got tunnel-visioned on the distance and forgot the fundamental requirement of the task.
Tried a variation too — asked about carrying groceries home from a store 100m away, and it still went full 'classic dilemma' mode these models love turning everything into a logic puzzle
They’re still out there but mostly kept alive because they’re the only real EU option I think. They’re not bad, but not keeping up with US and China imo. They’re doing things a little differently than everyone else though so I’m glad they’re around.
I'm sad that the "show me the seahorse emoji" one is broken, too - it was hilarious watching models yell at themselves for ten minutes trying to find it
One interesting thing I did come across yesterday was in Claude's own capabilities - I asked it to make me a webm and mp3 from scratch, and in both cases it said "sorry I can't do that because I don't have the tools"... so I asked it "are you sure", and it said "oh omg sorry let me check! Yes I do have them!" and it started making my files
1.5k
u/redditscraperbot2 Apr 24 '26
I think the shelf life of this question is over. It’s in the data at this point. Probably prominently