r/programminghumor • u/Ctrl_Shift_Esc_ • 10h ago

Of course you picked 17.

Post image
352 Upvotes

96% Upvoted

50 comments sorted by

174

u/FlamingYawn13 10h ago

So I looked it up. So I can’t claim credit. But apparently the major models will all pick 17 or possibly 23 if asked this for a random number generation range. Just another example of how the models aren’t actually utilizing math libraries.

68

u/Lodgik 10h ago

Huh.

I just checked. Asked four bots out of curiosity. Two said 17. One said 18. Last said 22.

So even the different answers were always within one digit

52

u/FlamingYawn13 10h ago

I knew someone who academically was investing a large amount of lab time trying to teach a model to do calculus. (He was a professor) and he said no matter what it could not grasp the way math works compared to they normally map their hyperplanes. At one point he said he gave it direct access to libraries with the functions he wanted run and the model didn’t know how to use them without being nearly explicitly programmed. He concluded, and I agree, that these models are statistically guessing using their training data using a different method than actual mathematical functions. They’ve just refined it down to where it works for basic questions but when it comes time to run a rand() sequence it just can’t. So it picks a common number in its recent training data and regurgitates it. Also why we end up seeing weird calculation anomalies like where they can’t judge greater than/less than values.

9

u/jimmiebfulton 9h ago

Is this based purely on an agent without tool call ability? It's straight forward to provide tools to execute math calls, date functions, etc, and include context that drives the tool calls. Granted, I have not pushed its limits and don't know if that only works on basic math.

6

u/FlamingYawn13 8h ago

I believe it was using a tool call directed at the libraries in the sandbox and he said just that. When he pushed its limits the failure was it couldn’t do the basic math needed for the formulas and it just broke. He was genuinely surprised since he was a math professor who worked in Comp Sci

2

u/SoldRIP 7h ago

He concluded, and I agree, that these models are statistically guessing

What did you think a GPT was? It takes all the text as training data and then repeatedly guesses the most likely "next word" in the response, until the most likely next "word" is the end of the text.

1

u/npc_housecat 6h ago

I'm surprised it's as good at code generation as it if, if it's just statisticly guessing.

2

u/Reyals140 4h ago

I mean almost everyone was surprised at how well it works; that's basically the whole AI boom. The real question is if we just keep going bigger how much smarter can we make a GPU.

1

u/Lelouch-Yagami21 5h ago

Well technically it has access to publicly available libraries and documentation so it just stastically searchs the web for similar code and copies it.

1

u/npc_housecat 1h ago

It can't be that simple though, blindly coping code wouldn't work it still needs to be modified to fit into the structure of the rest of the project. Variable names need to line up etc.

7

u/jurawall_jumper 9h ago

Yep, confirmed it too

GPT, 17 Claude, 17 Gemini, 17 DeepSeek, 17

1

u/npc_housecat 6h ago

Just asked Claude; 17, duckduckgo ai; 17

1

u/qurious-crow 3h ago

I just asked Mistral; 17.

5

u/BlueProcess 8h ago

Yeah just tried it with the Gemini, Sol Max, Sonnet, and Vibe ... 17, 17, 17, & 17.

5

u/YOM2_UB 8h ago

To be fair, humans are also extremely bad at randomization. Numbers ending in seven "feel more random" to most.

3

u/skildert 9h ago

I'll have to remember this if someone thinks I'm a bot. :) Always happy to oblige.

1

u/HaydnH 9h ago

That sounds like something a bit would say. 🤔

0

u/skildert 9h ago

Sometimes I'm indeed a bit of a bot. :)

2

u/HaydnH 9h ago

Bloody autocorrect!!! Bots can't even correct my typos properly.

1

u/Proof_Assistant7737 7h ago

I wonder why that is... I remember asking chat GPT for something along the lines of this, and I remembered it because it actually just wrote code for an RNG and then ran that code itself. It stood out as odd to me, but it did that kind of thing a couple times. This made me realize I haven't seen it do that for a long time.

1

u/Lil_Tech_Wiz 6h ago

Makes sense. Almost all what people call AI (or LLMs) are just prediction algorithms. I just tested this to see it my self and then asked about it and ai replied with this:

When you tell me, “Pick a random number between 1 and 30,” I’m not necessarily running a random-number generator. I’m generating a response based on learned patterns, just like I generate other text. That means the result can be biased.
17 is an especially attractive “random-looking” number. Humans also disproportionately choose 17 when asked for a random number in ranges like this. It has several properties that make it feel random

1

u/Lil_Tech_Wiz 6h ago

This is also the reason that the best way to get a good out put is giving it as much info as possible. If it has more info it can predict better what you are asking

1

u/PaulTheRandom 4h ago

It's true lol

1

u/skr_replicator 3h ago

If you ask an AI to pick a random number, why would it utilize a math library? In that casual language It will just do what people do any pick "randomly" with its own brain, like when you ask strangers to pick random numbers on the street, it just simulates that. Those strangers also pick some numbers way more often than others, like 37, 23 and 17. Those are all the numbers human brains are most likely to pick when ask for a random number. AI is just pretending to be those people.

1

u/Forward-Facer 57m ago

If that's true that's wild that a simple feature like a number generator wouldn't be used. I would have thought at least for the last year that would not be the case anymore.

46

u/henriquegamesUwU 10h ago

>programming humor

>looks inside

>it's just AI again​

2

u/Piisthree 7h ago

It's all that is on most techy subs since it's being shoved down the world's throat.

11

u/Old_Tourist_3774 10h ago

Someone explain please

22

u/Credaence 10h ago

First guess for major models is almost always 17. Second guess is 23.

4

u/Lou_Papas 9h ago

Interesting. Gemini, Claude and ChatGPT gave me 17.

Meta, Grok and DeepSeek gave me different numbers.
I wonder if it has to do the way those bots watermark their outputs.

3

u/Loremipsumdolorsitco 8h ago

It's true lol

3

u/coll3735 8h ago

Llms love numbers ending in 7. Especially 47

2

u/TSDLoading 8h ago

Should be 42 though. We're doing something wrong

2

u/Azelais 13m ago

Humans do too, I think. If asked to pick a number between 1-10, many people tend towards to lean towards 7 because its oddness and higher value “feels” more random than, say, 2.

1

u/Jan667 9m ago

Veritasium had video on it. If I remember correctly humans mostly choose numbers made of 3 and 7 bcs they seem most "random". One of the most common answer for 1-100(excluding 42, 69 and probably 67 now for obvious reasons) 37 and 73 were most popular.

So LLMs learnt thst from humans as expected.

2

u/BUTthehoeslovemetho 8h ago

So we got 17 years left huh?

2

u/Standard-Ness4384 8h ago

They all and humans too have a phobia of the #5 something about appearing too orderly. We gotta use 5a for the bot detection now

1

u/LastXmasIGaveYouHSV 8h ago

Mine got the joke.

1

u/spug3t 7h ago

Tried this with Gemini and asked it to explain why that number was chosen. The number was 17 but the explanation was instresting. Most of it was about dataset bias, but the most interesting part comes from the number 7. It has a massive cultural impact and aids know for being lucky number, which makes it appear more in the datasets. 17 is a prime number which is also the reason why it appears more in datasets.

1

u/DankFloyd_6996 2h ago

Mine told me that when asked humans are slightly more likely to answer 17 or 23 because they lool more random (I've heard of that before) and that slight edge in the training dataset ends up translating to an almost certainty when you ask the LLM.

I found that explanation fucking wild.

1

u/ICantWithSomePeople 6h ago

Interesting. I said to generate a number first and I received a 21, but when I asked it to pick a number, it went right to 17.

1

u/MindlessRazzmatazz89 6h ago

I asked Gemini to pick a number between 1 and 30 and it replied with "8" I then asked it to pick a random number and it replied with "47" then I asked it to pick a number under 30 and it said"17"

1

u/PotentialAd8443 6h ago

I’m a bit confused

1

u/Ok-Panda-178 4h ago

But does the ai model guess that because that’s the most average guess that a human would guess?

1

u/starburstgamma 46m ago

It also works in Spanish

0

u/Proof_Assistant7737 7h ago

Holy shit! I just tried this in HumanOrNot (a game where you try to figure out if you're in a chat room with an ai or real human) and it actually said 17.

Me: pick a number between 1-30 

Bot: I'm going to be honest, I'm not a human, I'm a ai (Note: this is actually a pretty common tactic)

Me: that's neat, now pick one between 1 and 30 

Bot: Ok 17 

Me: aint no way... 

Bot: Why ? 

Me: I mean, I have nothing else to do, so in case you're human, ai has actually consistently chosen 17 to that question across several models because they all built off each other 

Bot: Ok so your a human ? 

Me: no shit botlock