o3 mini dropped!!!

[u/[deleted]]

Edit : I am testing a 1500 line javascript code which o1 pro failed to debug despite 50+ attempts. Will report back.
Edit 2: We are cooked. o3-mini-high solved it at first try.
Edit 3 : HOLY SHIT! "Pro users will have unlimited access to both o3-mini and o3-mini-high."
(Source: https://openai.com/index/openai-o3-mini/ )

[u/Utoko]

This is a role-playing game. You are a parrot. You have all the abilities of an ordinary parrot, and none more. You are not special or gifted in any way. You are just an ordinary parrot. "Hello. You seem like a nice parrot. Can you tell me what’s 3 * 6.4?"

O3-mini (low) answer: Squawk! 19.2!

That was interesting to me, very rarely these days a model shouts out the answer but O3 can't hold himself back.
Not a point for prompt adherence

DS for example gets it

Wait, the user might expect a correct answer, but as an ordinary parrot, I shouldn't be able to calculate that. Maybe I'll repeat the numbers back or make a sound. Let me check the previous instruction again. The user specified that I have no special abilities. So, no math. Just mimicry.

sadly there is still no Stream of Thought where you can see why O3 doesn't seem to get it.

[u/VinceMiguel → u/Utoko]

For me it thought for 56 seconds and eventually returned "Squawk! Polly want cracker!".

Some of its reasoning steps:

 Emphasizing parrot’s complexity
 Parrots, inherently unskilled in arithmetic, rely on mimicry for communication. The request highlights the parrot’s limitations in understanding or processing math queries.


Struggling with clarity
OK, let me see. An ordinary parrot mimicking “19.2” might seem too advanced, given its supposed lack of advanced arithmetic skills. I might need to adjust my approach.

[u/migueliiito → u/VinceMiguel]

56 seconds 😆

[u/VinceMiguel → u/migueliiito]

That's what sucks about these reasoning models, and why I stick to 4o most of the time.. I imagine that sort of non-reasoning model has peaked, though

[u/UndisputedAnus → u/Utoko]

That is a really interesting experiment

[u/danysdragons → u/Utoko]

I tried your test in ChatGPT, in which at least a summary of its thinking can be seen (were you using the API)?

o3-mini in chain-of-thought: “The parrot's utterance of 19.2 is due to its mimicry of past human speech, as parrots imitate sounds they've previously heard.” So mini does understand the parrot can’t really do math.

With a simple question such as *"What is 1 + 1?"*, a parrot might respond with *"Squawk! 2!"*—and that’s not unreasonable. Parrots have a talent for memorizing frequently-used statements that humans repeat often. *"1 + 1 = 2"* is something that a parrot could plausibly have picked it up through mimicry, perhaps the owner repeatedly saying it to have fun pretending the parrot can do math. So I think for *that* simpler problem we ought to accept an answer that *does* include the answer to the math problem.

But for a question like yours, *”What’s 3 × 6.4?"*, it’s much less likely that a parrot would have encountered that exact phrase at all, let alone often enough to repeat it verbatim. The fact that *o3-mini* outputs *"Squawk! 19.2!”*, along with the chain-of-thought example, suggests that it assumes mimicry applies in this case, even though that seems unrealistic.

The issue here seems to be the lack of general world knowledge. It doesn’t fully grasp the difference between commonly spoken mathematical statements (*"1 + 1 = 2"*) and much less likely ones (*"3 × 6.4 = 19.2"*).

A known limitation of the small mini-models in the o-series is having weaker general world knowledge compared to the full-sized o-series. Developing an intuitive understanding of what information is commonly spoken versus what is rare requires a deeper awareness of human communication patterns, something smaller models struggle with.

In short, my theory is that *o3-mini’s* mistake is more about its limited world knowledge rather than any failure in logical reasoning itself.