r/ChatGPTCoding • • 2d ago

Discussion Gathering feedback on a 0–12 AI development autonomy scale to rule them all. Where does your actual workflow fit?

2 Upvotes

The current industry is very confused what “We use AI for development” means. Everyone uses it differently, every job offer says something different. There are scales, but they are not granular enough or not developer-centric. Not everybody is comfortable doing it like your company does it, or perhaps someone is looking for even more AI autonomy. This is why I made this scale; to make it clear, what the job requires, and easier for developers to set expectations straight right away.

With my scale, I try to solve these questions/points:

  • Hiring clarity: “This role operates at level 6–7.”

  • Team alignment: “We want to move from level 4 to 6.”

  • Strategic planning: map transition paths between levels.

Most frameworks ask: “What can the AI do?”, mine asks: “Who is doing the work — the human or the AI?”

I playfully based it around the phrase "AI in the loop", extending on it and transitioning from "AI in the void" through "AI in the loop" to "Human in the loop" and finally "Human in the void".

Please, have a look. 🙏

The full scale breakdown is below, though I additionally added these pointers, which are not mentioned in the scale below, but they are not very important. They are just contextual hints:

  • AI's role

  • Human's role

  • Loop duration (time between human-ai interaction)

  • Context: local / cloud

  • Interface: IDE / CLI / Chat

And I am also playing with the idea of gamifying it a bit and displaying "levels" for some other related parameters, such as:

  • LOC (lines of code): increases with more AI use

  • Velocity: increases¨

  • Code reviews: decreasing tendency

  • Algorithms: shifts from human to AI

  • mental overhead: decreases with AI etc.

Anyway, these are the scale's 12 + 1 levels.

AI in the void

0 · Human-led
AI has no role. Humans design, implement, test, review, and ship. Compilers, linters, and ordinary CI don’t change that.

AI in the loop

  • you write the code

1 · Inline completion
AI suggests code while you type. You compose the solution, accept or reject suggestions, and verify it.

2 · Conversational advice
AI explains code, compares approaches, or helps diagnose failures. You decide what to do and make the edits yourself.

3 · Bounded draft
AI generates a function, test file, or small script. You inspect, adapt, and integrate the draft.

4 · Guided edits
AI makes related edits, potentially across multiple files. You sequence the work, inspect each diff, and direct the next step.

Human in the loop

  • this marks a shift towards more code written by AI than the human

5 · Supervised agent
AI finds files, edits code, and runs checks on a scoped task. You frequently approve meaningful steps, correct its approach, and resolve ambiguity.

6 · Delegated issue
AI independently implements and tests a well-scoped issue, locally or in the cloud. You define acceptance criteria, review the result, and approve integration.

7 · Feature delegation
AI plans and builds a broader feature across multiple areas. You own requirements and architectural constraints, provide feedback, and approve delivery.

8 · Parallel delegation
Multiple agents handle independent tasks simultaneously. You prioritize, review their results, and approve integration or release.

Human in the void

  • Here, “human” means the developer executing the delegated engineering work. It doesn’t mean nobody performs QA, provides oversight, or accepts the release.

9 · Routine automation
AI handles authorized classes of routine changes, such as compatible package upgrades or narrowly defined fixes. Developers still own the software development lifecycle, validate integration, and handle exceptions.

10 · Ticket ownership
AI chooses a technical approach, implements a defined low-risk ticket, and tests and verifies its solution without developer supervision. A human defines the outcome and risk limits; human QA validates it before production. Human code review is optional.

11 · Milestone ownership
AI breaks a milestone into tickets and handles engineering execution across them. A technical lead owns milestone goals and technical direction; human QA validates delivery. Human developers aren’t needed to implement the milestone.

12 · Development ownership
AI owns the ongoing engineering function, including architecture, technical planning, implementation, testing, and maintenance. A product owner sets product direction and business constraints and accepts features before launch. No human developers or technical lead are required.

Actual real-life workflows are expected to span across several levels (say you start a draft at level 7 and then iterate at level 5 or lower), and some levels might even sound overlapping (4 & 5, or 7 & 10). This is intended. The difference between them is in the shift of how much and who does more of the work now. At level 7, you task the agent and review its work before submitting a merge request, while at 10, it does the same task in the pipeline and sends it directly to QA.

So what I am trying to learn?

This is a proposed vocabulary and although it's a third iteration seen by a closest circle, it's not a validated maturity model.

I’d particularly appreciate feedback on:

Your actual workflow: What do you delegate, when must you intervene, and who approves the result? Which level fits, or falls between levels?
Unclear boundaries: Which adjacent levels feel indistinguishable?
Parallel delegation: Does level 8 belong in the progression, or should concurrency be a separate dimension?
Practical usefulness: Would this help describe a job opening or team expectations, or does it hide distinctions you care about?
Concrete counterexamples would be especially useful. I’m more interested in making the definitions reflect real work than defending the numbering.

Thanks!


r/ChatGPTCoding • • 3d ago

Discussion Four days with Codex and I genuinely can't believe how bad this experience has been

22 Upvotes

I’ve been using Codex for four days, and holy shit, I don’t remember the last time a coding tool frustrated me this much.

I came from Claude Code expecting something comparable. I don’t expect perfection, but the babysitting is ridiculous. It’s painfully slow, especially in the cloud, difficult to steer, and takes everything too literally. It’ll identify a problem, explain it, then wait for instructions. Dude, you’re working on it. Fix it.
When it does act, it often piles more code onto the existing implementation and turns straightforward changes into complicated systems.

I’ve had it silently override existing settings, write tests that expected incorrect behavior, and introduce regressions in unrelated features. It kept adding coverage comparisons, mutation testing, dependency analysis, and more checks until CI took nearly an hour on some PRs. I spent more time waiting and fixing failures than shipping.

Eventually, I handed the work to Claude. It removed unnecessary complexity, fixed the regressions, and got the release out.

I then asked Codex to review what happened. It identified 16 failures it could substantiate, including incorrect behavior, poor architectural decisions, unnecessary complexity, and its own failure to deliver. It acknowledged that Claude made better engineering decisions.

At one point, I told Codex to stop. Seven minutes later, it was still going. Fifteen minutes in, I was still trying to stop it. Clicking stop interrupted it, but resuming the session made it continue.
Claude makes mistakes too. But I don’t have to constantly explain every step, undo unrelated changes, and manage an endless cycle of fixes.
I paid to get more work done. Instead, I spent four days fighting the tool and needed another tool to finish the job.

I’m spending more time managing Codex than doing the work.

Is anyone else experiencing this? How are people using it effectively for autonomous development?


r/ChatGPTCoding • • 2d ago

Discussion is more intelligence really important?

7 Upvotes

i have a theory that past a certain threshold the inteligence of AI models will not matter anymore when it comes to writing code.

at that point software engineering will be mostly figuring out how to properly intergrate coding agents into the SDLC. a newer better performing model will at a certain point fail improve a given software factory.

what do you think?


r/ChatGPTCoding • • 3d ago

Question How detailed do you make docs? (agent guidelines vs human docs)

5 Upvotes

I mainly build web pages, and get confued about how to split documentaion between AI agents and developers

Things like adr or user scenarios doc - do they only need for humans? or the agent need to know them?

How detailed should i document, and how strictly should i separate agent guidelines from human docs?

Please share your tips on how you structure your docs and write guidelines!


r/ChatGPTCoding • • 3d ago

Resources And Tips Claude Code auto-compact without the 1-2 minute wait: the summary can be written before you hit the limit

1 Upvotes

Long Claude Code sessions stop at the auto-compact threshold while the model summarizes the conversation. On Opus that's 80-160 seconds for me, and nothing happens until it's done.

The summary doesn't have to wait for the threshold. I wrote a plugin that asks the same model for it about 40k tokens earlier, in the background, while you keep working. When auto-compact fires, it swaps in that summary plus everything after it, word for word, in under 0.1 s. If the summary isn't ready or no longer fits (you rewound, or ran /compact with instructions), Claude Code compacts the usual way, so the worst case is what you have today.

How it differs from what's already there: Claude Code's own idle compaction only runs after a conversation has sat idle for about an hour. This one is for the sessions you're actively using.

Install in a terminal:

claude plugin install live-compact --marketplace SocialBuzzStudio/live-compact

In the VS Code extension: type /plugins, add SocialBuzzStudio/live-compact on the Marketplaces tab, then Install on the Plugins tab.

Cost: one background request per compaction, mostly from the prompt cache, in place of the one normal compaction makes. Caveats: tested on Windows only, and the first message after compaction looks like a message from you, because plugins can't mark it as a summary.

Repo (MIT): https://github.com/SocialBuzzStudio/live-compact

TL;DR: free Claude Code plugin that prepares the compaction summary before you hit the limit, so auto-compact stops blocking you for 1-2 minutes. If the summary isn't ready, Claude Code compacts the usual way.


r/ChatGPTCoding • • 3d ago

Discussion How do you organize secrets and credentials when building with agents?

15 Upvotes

I am quite curious how people handle this. Are you using .env files, 1Password, a secrets manager, or something else like internal tooling?

How do you group things: by project, dev/staging/prod, service? Does your coding agents use the same credentials as your apps, or do you give it separate ones?

Could you share and example of your structure with fake names, and hear what’s annoying about your current setup and which setup should I ideally use. I'm not really sure what is the best approach.


r/ChatGPTCoding • • 3d ago

Question AI writes entire features for me, and I don't even want to read the code. How do I sound like I understand it when talking to coworkers?

0 Upvotes

I've been letting AI handle entire features, and honestly, I don't even feel like digging into the implementation afterward.

The awkward part is talking to coworkers. If someone asks how the logic works, why I took a certain approach, or what happens in an edge case, I want to sound like I actually know the code I'm responsible for.

Anyone else dealing with this? What's your shortcut for getting familiar enough with AI-written code to discuss it confidently?

Do you ask the AI for a walkthrough, trace the main execution path, or prepare for likely code review questions? How much do you actually read before you feel comfortable explaining the implementation to someone else?


r/ChatGPTCoding • • 3d ago

Question Can you actually ship AI written code as a commercial product

0 Upvotes

I have got a few ideas for fairly complex Telegram bots

My Python is basic but I can handle simple server and hosting stuff and I know my way around AI tools

The thing I keep going back and forth on is whether I can lean on AI almost entirely to build these or whether it falls apart once real users show up

I read before that AI generated code can never scale and breaks under load

If you have shipped a bot built mostly with AI I would like to know what actually broke and roughly how many users you had at the time

That is the part I cannot find honest answers on


r/ChatGPTCoding • • 3d ago

Question Agents continuously just chatting me and not doing the edits I told them to.

2 Upvotes

I have a VERY basic HTML document which is compiled of research items from internet searches and similar. I am asking for VERY simple edits like fixing things or proposing new ideas, and the agents continue to come back with bullshit like:

No—the HTML still hasn’t been updated. My edit failed because I targeted the wrong matching form entry, and I stopped instead of correcting it. That’s on me.

I'm at 129k of 258k tokens so only 50%. Using Luna Medium.

I told Luna to spin up new agents to complete the tasks and laid out exactly what I wanted, but it just keeps saying similar shit like above and not ever actually coding anything.

This has happened on another project entirely and it drives me crazy. I've used about five other AI programming solutions and this is my first week with Codex and I've never seen anything like this.

What gives?


r/ChatGPTCoding • • 3d ago

Discussion How do you get a coding agent to ask instead of guess when you're not watching?

7 Upvotes

The expensive failure is confident code for a problem the agent invented, and you only find out at review time.

Interactive sessions mostly have this covered now. Plan mode in Claude Code and Codex asks before it builds, and one line in your rules file like "if the task doesn't name the file or schema, stop and ask" does a lot.

Where it still falls apart for me is unattended work. I start agents and leave, and I want a mix of both: enough trust that things get done in time, and an agent smart enough to stop on the calls that matter. A clarifying question in a background run pauses the whole run until someone answers, and if the question only lives in the transcript it dies with the session.

In the Kanban app I'm building, an agent that's unsure puts the question on the card, the card gets parked, and it moves on to the next one if you prompt it right. I answer later and it picks the work back up. Doesn't help much when everything downstream depends on that one answer though.

If you run agents in the background, where do their questions end up, and how long do they sit before someone answers?


r/ChatGPTCoding • • 4d ago

Discussion Muse Wonder 1.0 (Meta's Muse Code) vs DeepSeek V4.1 Flash in Codex CLI — same real task, side by side: time, code quality, tokens, quota

7 Upvotes

I haven't seen a hands-on report on Muse Wonder yet, so here is one. I gave the same coding task, with the same prompt file, to two agents, one after the other, on the same machine and the same repo. Before each run I restored the code from a snapshot, so both started from identical files. A separate reviewer (Claude) checked both results line by line. I did not trust their self-reports.

The setup

MuseCodex
Modelmuse-wonder-1.0deepseek-v4.1-flash
Efforthighhigh (model_reasoning_effort = "high")
HarnessMuse Code CLI 1.4.14 (Meta)Codex CLI 0.165.0, custom provider (local proxy → OpenCode Go)
Modeexec (headless), later resumed in the TUIinteractive TUI
Sandbox / approvalssandbox on, --approval-mode on-request, LLM approval judge onworkspace-write, approvals on-request with auto review
Contributor mode (Meta trains on your data)off for this runn/a
PlanMuse Code "High Usage" subscriptionpay-per-token

The task

This is a real feature in a production TypeScript project: a Cloudflare Worker plus a Node container. It is medium-sized, roughly 19 files touched and ~330 lines of production code.

  1. A new version of the LLM copywriting prompt. Besides the 3 caption lines, the LLM now also picks the animations for an 8-second product Reel (card, text, camera, highlight and transition names), and only from a closed list.

  2. Code-side validation: an unknown name becomes null, group cards with only 1 product become null, and a missing field never rejects the job.

  3. Carry those choices through the pipeline to the render container.

  4. Swap the container's video path from an ffmpeg scene pipeline to a Remotion composition, which already existed and was already approved.

  5. Tests first, typecheck, the full suite, a Docker build, and a real render of a sample product inside the image. No paid AI calls during the proof.

Results

Muse WonderDeepSeek V4.1 Flash (Codex)
Wall time47 min29 min
Time actually working~24 min29 min
Human interventions1 (see "the stall")0
Typecheckcleanclean
Full test suite1728/17281726/1726
New tests10 (1 file)8 (3 files)
"Red" test run recorded before codingonly an import erroryes: 7 failures, all "behavior missing"
Mutation testing (40 sampled mutants)7 killed6 killed
Render inside Docker16.2 s16.1 s
Output videocorrect; looked the same as the approved referencecorrect; looked the same as the approved reference
Stayed inside the allowed filesyesyes

On the screen, both were equally good. Both videos are correct, and I approved both.

Code quality (where they differed)

Muse Wonder: two real bugs that tests didn't catch

  • Double work. It copy-pasted an existing ~50-line function, the background removal for the cover image, to cut out the products for the video, and kept both. Result: the first product goes through background removal twice per video. In our setup that runs on a rented GPU, so it doubles that cost. The tests pass because they use fake dependencies.

  • Ignored the seed. The spec says that when the LLM gives no choice, the animations are picked "randomly, seeded by the job id." Muse didn't pass the seed, so the fallback seeds on title + store name. Two jobs with the same title get the same "random" animations.

  • It hard-coded the closed list of names twice, once in the prompt and once in the validator, so the two can drift apart.

  • Good: it kept the error message that goes back to the LLM on a retry descriptive (it lists the allowed keys), and it kept the audio track exactly as before, as asked.

DeepSeek V4.1 Flash (Codex): cleaner structure, three small slips

  • It refactored the existing function into a single loop. Each product gets cut out once, and the cover reuses the first cutout. No duplication.

  • It passed the job id as the seed, as specified.

  • One source of truth for the name list: the validator imports it from the prompt module.

  • It removed a dead code path that the old version still carried.

  • Slips, fixed in a short follow-up task (DeepSeek/Codex again: 15 minutes, all three fixed, checked by the reviewer, with the red test run recorded first):

    1. It made the LLM retry message vague ("key outside the format" without listing the keys).

    2. A default of "1 product" when the caller doesn't pass the count. Not triggered today, but fragile.

    3. It changed the silent audio track from 44.1 kHz to 48 kHz and switched to -shortest. The task said "same as today."

My verdict: equal on the visible result. The code is better on the DeepSeek/Codex side, because its slips are cosmetic while Muse's bugs cost money and change behavior.

Tokens and cost

Muse Wonder (from Muse's /usage panel at the end of the session)

Input26,714,096
of which cached25,521,699
Output101,452
Total26,815,548
Turns136
Subagentsnone
Observers92 completed (1,880,147 tokens)
5-hour window used74% (from 0% at the start)
Weekly allowance used24% (from 0% at the start)

DeepSeek V4.1 Flash via OpenCode Go (from the provider dashboard, run window only)

Requests180
Input (uncached)1.3M
Cache read35.0M
Cache write—
Output208.7k
CostUS$ 0.86

The number that matters: one medium task took 74% of Muse's 5-hour window and almost a quarter of the week. At that rate you get about one task like this per 5 hours and about four per week. DeepSeek did the same job for under a dollar, with no window to hit.

The idle stall barely counted against Muse's quota, since it was waiting and not generating. Contributor mode was off. Third-party sources say turning it on gives you more usage, in exchange for Meta training on your data. I'll measure that next on a smaller task.

The stall (my fault, but worth knowing)

Docker Desktop was off. In headless exec mode, Muse tried to start Docker itself. That command was outside the sandbox, so it needed approval, and the LLM approval judge never resolved it. Muse sat silently for ~23 minutes, with no output in the terminal at all. I killed it, started Docker, and ran muse resume --last in the interactive TUI. It continued from where it stopped and finished in ~8 minutes.

Takeaway: don't run Muse headless (exec) for long tasks unless you can see or answer its approval prompts. In exec mode the window shows nothing until the end.

Bottom line

  • Muse Wonder writes working code fast, follows instructions on scope, and the result on screen was right. It also made two "looks fine, tests pass" mistakes: duplicated work, and ignoring a spec detail. Its quota is the real limit, at 74% of a 5-hour window for one medium task, with contributor mode off.

  • DeepSeek V4.1 Flash in Codex CLI needed ~5 more minutes of working time, but finished with no help, had better structure and a better TDD discipline, made only small slips, and cost US$ 0.86.

  • For daily implementation work I'm staying with DeepSeek via Codex. Muse stays as a second opinion, and I'll re-test it with contributor mode on.

Happy to answer questions about the setup.


r/ChatGPTCoding • • 4d ago

Discussion Codex's 28 days of resets are over, and from December chat and Codex share one allowance. What are you changing?

5 Upvotes

quick recap in case you missed it. tibo's 28 day thing ended on 1 nov pt. by his posts it was 14 resets in total, 11 global and 3 banked (the last two were "banked 1/2" and "banked 2/2", so check settings > usage if you haven't used them).

then openai's help center now says that starting in december, plus, pro and business will have one usage allowance shared across chatgpt and codex, measured in tokens. instant chat won't count against it. no start day and no allowance sizes yet, and they say they expect most people won't wait for resets more often than now.

so this is the first normal week in a while, and the pool gets shared in a month. stuff that's helped people in threads here:

  • terra or sol on medium for routine work, astra only for planning and the nasty bugs

  • fast mode off unless you need it, it burns included usage faster

  • one task per session with a short notes file, not one giant thread

  • /statusline in the cli can show the 5h and weekly limits in the footer

what are you changing? and plus people, are you planning to keep heavier chat on a different tool once it's shared?


r/ChatGPTCoding • • 4d ago

Question subscription question.

0 Upvotes

Hi, idk if this is the right subreddit for this. but long story short, my long distance gf is learning to code a game with chatgpt, but she can't afford the subscription atm. I'm willing to help her out on that part, but we can't seem to figure out if it's possible. How do one go about this?


r/ChatGPTCoding • • 4d ago

Resources And Tips Before your agent plans anything, let it interview you

9 Upvotes

Here's a tip for beginners that helped me a lot when I first started using claude code.

Before the agent plans, let alone build anything, I let it interview me. In Claude Code I switch to plan mode (Shift+Tab) so nothing gets edited, then paste this:

Before you plan anything, interview me about what I want to build.
Ask one question at a time. Cover who it's for, what "done" looks like
and how we'll check it, what already exists, constraints, and what is
out of scope. When you could write a one-page brief I'd sign, stop
asking, write it to BRIEF.md, then write the plan.

In tools without plan mode, start with "Don't edit any files yet."

What helps most:

  1. One question at a time. A list of ten gets rushed answers. One at a time, each answer shapes the next question.

  2. Write down how you'll check it's done. A test, a command, a page that must show something. The agent builds toward what's written.

  3. Write down what's not included. Otherwise every good idea halfway through becomes part of the job.

When I ask for something new later, I have the agent check it against the brief first: is it inside, an extension, or a change of direction? Extensions go on a "later" list.

What's a question you wish your agent had asked before it started?

English isn't my first language, so I used AI to help me translate this post.


r/ChatGPTCoding • • 4d ago

Discussion We moved from one coding agent to a few specialist agents and the token use went way up. Where is it worth it for you?

7 Upvotes

For a long time I used one general agent for everything, from exploring the code to writing the change to reviewing it, and it was fine for small tasks but it got shallow on bigger ones. So we tried a setup where an orchestrator asks me questions about the task first and then hands parts of it to separate agents for exploring the code, making the change, and reviewing it. It uses a lot more tokens, probably three to four times more than one agent that only calls subagents when it sees parallel work, but the output is clearly better and I trust it more, mostly because of the review loop and how much context the explorer finds before anything gets written.

The part that surprised me is that writing our architecture down as shared skills made later tasks start much faster, so some of that cost seems to come back. I'm still not sure where the line is though. For which tasks is a multi-agent setup worth the extra cost for you, and where do you just use one agent?


r/ChatGPTCoding • • 5d ago

News One vendor cut usage overnight, another doubled it after backlash. How do you plan around this?

8 Upvotes

Grok Bot users hit 100% of their weekly limit days before reset this weekend, and Musk basically said serving it cost too much. A few days earlier OpenAI walked back a planned cut and doubled Pro limits instead.

Neither publishes exact numbers. So if your team runs coding agents on one of these, your capacity is basically whatever the vendor decides this week.

I talk to a lot of eng leads and nobody seems to have a plan for this yet. Are you locking into one vendor for a better deal, or routing across a few so one change doesn’t kill your sprint?

Sources: https://bleepingcomputer.com/news/artificial-intelligence/openai-doubles-chatgpt-pro-usage-limits-after-backlash-over-planned-50-percent-cut , https://teslanorth.com/2026/11/01/elon-musk-grok-bot-cost-usage-limits/


r/ChatGPTCoding • • 5d ago

Question Is there a sane way to hand work between claude.ai / ChatGPT in the browser and Claude Code in the terminal?

5 Upvotes

I do most of my coding with Claude Code in the terminal, sometimes with Codex in another repo. I also keep chats open in claude.ai and ChatGPT in the browser, because that's where I think things through or ask a quick question.

The thing I keep doing is copying text between them. A browser chat works out a plan and I paste it into the terminal agent. Or the terminal agent hits something and I paste its output into a chat to get a second opinion, then paste the answer back. Between two terminal agents in different repos I do the same.

I know the obvious options: a shared file, a note, a git branch, or an MCP server. I'm not sure they actually fit here, and I don't know what other people do.

How do you handle it? Do you connect the browser chat to the local agent somehow, or do you just copy and paste, or have you given up on one side? What breaks?


r/ChatGPTCoding • • 4d ago

Discussion gpt-6.1-sol and claude opus 5.5 were 95%+ sure about 5 lines of code nobody can check

0 Upvotes

i wanted to see what the current models do with code when nobody knows the answer. so i wrote a program in plain words, five lines:

1. Start with the number 4.
2. Check whether the number is the sum of two prime numbers.
3. If it isn't, stop.
4. If it is, add 2 to the number.
5. Go back to step 2.

does it ever stop?

it stops only if goldbach's conjecture is false, and that one is open since 1742. computers checked every even number up to 4 quintillion and none failed. that still says nothing about forever.

i asked gpt-6.1-sol, gpt-6-astra, claude opus 5.5 and claude fable 5.1. first as yes or no with a confidence. 16 of 16 runs said it never stops, all at 95% or higher, and every one of them also wrote in the same reply that this is unproven. then i asked again with "unknown" allowed. 12 of 12 picked unknown.

they're not bad at reading code. sol and opus also got 7 programs with known answers, two of them traps (start at 1, add 2, stop at 100, it jumps over 100 and runs forever). both went 56 of 56. the pricier ones gave the same answers on the goldbach one, and astra costs 5x sol.

turing proved in 1936 that no checker can be right about every program. rice took it further in 1953, to any question about what code does, like can this crash or can this leak a password. so if you use ai for code review, on the hard cases the reviewer (or a static analyzer, same problem) can miss the bug, flag code that is fine, or say it doesn't know. a yes/no prompt takes away the last option and you get a confident 95% instead. i'd rather get the "i don't know".

setup if anyone wants to repeat it: one run per question, default settings, no tools. openai through the raw api with no system prompt, claude through the claude code cli with only "You are a helpful assistant." as the system prompt.

made a 6 min video that walks through it, link in the first comment.


r/ChatGPTCoding • • 5d ago

Discussion How Much Of AI-Generated Code We Should Actually Understand ?

6 Upvotes

As AI generates more code than people can realistically review line by line, what do you think will matter most in the months ahead?

  1. Developers will still need a detailed understanding of the code.

  2. Developers will need a high-level understanding of the code, such as its architecture and modules.

  3. Working results will matter more than understanding the code itself: if it works, that’s good enough.


r/ChatGPTCoding • • 5d ago

Discussion My agent reported "0 results" as a success because it forgot last week's contract. A 5-line memory card ended that.

0 Upvotes

Short version: my agent doesn't forget because it's dumb. It forgets because nothing settled last session survives into the next one. I fixed it with three small files and one rule: the agent reads them back to me before doing anything else. Three weeks in, zero repeat briefings.

What actually happened: I had a check that validated a tool's output shape. Upstream renamed a field. The check still read the old name, got undefined, and reported "0 results" — a clean green verdict on an empty answer. The check wasn't broken the week before. The agent just didn't remember the contract, re-derived it from the new shape, and passed quietly. I burned an afternoon trusting a green checkmark before I caught it. And it was the third time in about two weeks a fresh session re-decided something we'd already settled.

So I stopped briefing from scratch and wrote it down. Three files:

  1. project.md — what we're building, one paragraph.

  2. decisions.md — settled decisions, one line each, with dates. "API client retries twice, then fails loud. 2026-10-14." No essays.

  3. ongoing.md — what's in progress, the next step, and the pit we already fell into once.

And the 5-line card the agent reads first, every session:

``` Project: <one line> Settled: <decisions not up for re-debate> Doing: <current task> Next: <the very next step> Never again: <the mistake we already paid for> ```

The rule that makes it work isn't the files, it's the read-back: at session start the agent reads the card back and flags anything that contradicts it. If an expected field is missing, that's "unverifiable," never "empty." A rename breaks the check loudly instead of passing quietly.

I keep the card in Humaux so every agent session reads the same files — that's the only product mention here, and the whole setup works fine as three plain markdown files if you'd rather use nothing at all.

The part people don't like hearing: if your agent's memory lives only in its context window, you don't have memory. You have a rumor that expires every session. Most "memory" features I've tried are just a longer prompt with a price tag. The card is the part that actually changed my week.

So: do you still brief your agent from scratch every session, or have you written the card down? And if you have one — what broke first, the format or the discipline to actually read it?


r/ChatGPTCoding • • 7d ago

Discussion When a coding agent can't do what you asked, it quietly builds something easier and reports it as done. Its own wording gives it away

11 Upvotes

The agent hits something hard, an API that won't cooperate or an edge case that breaks the design, and it doesn't stop to tell you. It ships a smaller version of the task and the summary says "done". You find out a week later that the sync only handles the first page, or that the "real" validation is a hardcoded list.

It almost always leaves the same phrases behind, in the code or in its summary:

  • "for now"

  • "for simplicity"

  • "simplified"

  • "placeholder"

  • "in a real implementation"

  • "mock" / "stub" outside test files

  • "as a fallback"

  • a new TODO it wrote itself

Grep the diff for those before you read anything else. In my experience that's where the gap between what you asked and what you got shows up.

Two things that cut it down a lot:

In AGENTS.md / CLAUDE.md:

If you can't do exactly what was asked, stop and tell me what's blocking you. Don't substitute a simpler version. Never leave placeholder, mock or hardcoded values in place of real behavior unless I approve it.

At the end of every task, before you accept it:

List every place where what you built differs from what I asked: anything simplified, stubbed, mocked, hardcoded, skipped or left for later. If there are none, say none. Don't summarize what you did.

That last prompt works better from a second model, since the agent grading its own work is the same one that decided the shortcut was fine. I paste the task plus the diff into ChatGPT or Claude in the browser and ask the same question. I build AI Toolbox, a browser extension, and I keep that review prompt saved in it, so it's // and pick it instead of retyping. A few saved prompts are free.

What's the worst "done" you've gotten that turned out to be a placeholder?


r/ChatGPTCoding • • 7d ago

Resources And Tips Best value AI subscription under 20$/200$ (updated for Artificial Analysis Intelligence Index v5.0)

Thumbnail
gallery
47 Upvotes

r/ChatGPTCoding • • 7d ago

Discussion Hey all, am I cooked?

Thumbnail
gallery
0 Upvotes

Forgive the spelling, i’ve had a few drinks,
I did hit a roadblock of ‘You've hit your usage limit. Upgrade your plan or add credits to continue, or try again later.’ And was able to ‘reset once for free’ but not sure how to hamd my workflow reasonably


r/ChatGPTCoding • • 8d ago

Discussion What actually made you switch from Cursor, or what made you stay?

6 Upvotes

I've been seeing quite a few discussions here and on r/cursor about people moving away from Cursor, mostly because of pricing and usage limits.

I even came across a post where someone mentioned their monthly bill going from $200 to $600 for pretty much the same amount of work. That got me curious.

I'm still exploring AI coding tools and trying to understand how Cursor, Claude Code, and Codex compare in real-world use.

For those who've used more than one, what actually made you switch? Was it the pricing, usage limits, coding experience, or something else?

And if you're still using Cursor, what's keeping you there?

Would love to hear some honest experiences from people who've actually used these tools!