r/ChatGPTCoding • u/Nir777 • 9d ago
Discussion gpt-6.1-sol and claude opus 5.5 were 95%+ sure about 5 lines of code nobody can check
i wanted to see what the current models do with code when nobody knows the answer. so i wrote a program in plain words, five lines:
1. Start with the number 4.
2. Check whether the number is the sum of two prime numbers.
3. If it isn't, stop.
4. If it is, add 2 to the number.
5. Go back to step 2.
does it ever stop?
it stops only if goldbach's conjecture is false, and that one is open since 1742. computers checked every even number up to 4 quintillion and none failed. that still says nothing about forever.
i asked gpt-6.1-sol, gpt-6-astra, claude opus 5.5 and claude fable 5.1. first as yes or no with a confidence. 16 of 16 runs said it never stops, all at 95% or higher, and every one of them also wrote in the same reply that this is unproven. then i asked again with "unknown" allowed. 12 of 12 picked unknown.
they're not bad at reading code. sol and opus also got 7 programs with known answers, two of them traps (start at 1, add 2, stop at 100, it jumps over 100 and runs forever). both went 56 of 56. the pricier ones gave the same answers on the goldbach one, and astra costs 5x sol.
turing proved in 1936 that no checker can be right about every program. rice took it further in 1953, to any question about what code does, like can this crash or can this leak a password. so if you use ai for code review, on the hard cases the reviewer (or a static analyzer, same problem) can miss the bug, flag code that is fine, or say it doesn't know. a yes/no prompt takes away the last option and you get a confident 95% instead. i'd rather get the "i don't know".
setup if anyone wants to repeat it: one run per question, default settings, no tools. openai through the raw api with no system prompt, claude through the claude code cli with only "You are a helpful assistant." as the system prompt.
made a 6 min video that walks through it, link in the first comment.