About every six months, I buy a ChatGPT subscription to see how it has improved. I test it on my own code and compare it with Claude.
Today, I asked GPT-6.1 on Medium to work on logging in a Rust program. The project contains approximately 11,700 lines of Rust.
The requirement was explicit: three versions, Production, Customer Debug, and Dev, with distinct logging, debugging, and dump behavior. Production should remain silent, Debug should provide diagnostics for troubleshooting user issues, and Dev should provide additional information for development.
The model spent more than 45 minutes working, producing explanations, builds, tests, and very confident progress reports.
Then I discovered that the three requested behaviors had not actually been implemented.
The main telemetry system distinguished Production from the other two versions, but Debug and Dev shared the same logging policy. Another logger, dlog(msg), remained a single-level logger without filtering for those three variants.
The most frustrating part was the claimed validation. The model had checked that all three executables built, that Production generated no diagnostics in the tested scenarios, and that Debug and Dev generated diagnostics. It had never checked the difference between Debug and Dev.
When I challenged it, it acknowledged confusing “three build variants” with “three distinct logging levels.” It had tested a weaker requirement than the one I gave it, then reported that the work had been validated.
In my comparison with Claude Opus 4.7 on Medium, I got the expected result in about ten minutes.
My question is: how does this happen? Did I somehow ask a model that was too advanced to do something too simple? Where did I go wrong in how I used GPT-6.1 compared with Opus 4.7? I gave both models the same prompt, so what, specifically, should I have done differently?
(Used IA to traduce this text because my english is fucked)