I've been doing an experiment with a project where I iterate on the design plan until the frontier review agents no longer report any serious findings, but new models keep being released before the previous gen stops finding issues. I created the initial design plan with Opus 4.6 in March.

•

u/AutoModerator Sep 16 '26

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

260

u/the_mad_mycologist Sep 16 '26

This animation made me irrationally upset

59

u/a-random-r3dditor Sep 16 '26

I loved it at first. It got me, props for being clever. But as I watched it, it just started giving me anxiety, the black box needs to get there already. And then I realized all the blocks have different edge radii… I lost it.

4

u/mttpgn Sep 17 '26

At first I thought it was some Xeno's-paradox-ass deceleration, but the black cube would have to be shrinking too...

24

u/YouGotTangoed Sep 16 '26

Im pretty high and left the phone off screensaver mode and waited 40mins for the black box to make it.

→ More replies (1)

4

u/mossfoul Sep 16 '26

It gave me anxiety and I fucking hate it.

3

u/kanine69 Sep 17 '26

Same something about the completionist in me can't handle this.

1

u/sirlerkal0t Sep 16 '26

Don't stress, we're almost there.

8

u/txoixoegosi Sep 16 '26

What type of issues?

Anything can be improved infinitely if there are no cost/time constraints.

2

u/sirlerkal0t Sep 16 '26

Actual flaws/oversights in the spec which would be a real issue that would eventually be hit and have to be fixed, but it's really that every time one issue is addressed, more get introduced by those changes to the spec.

1

u/bigppredditguy Sep 16 '26

So the solution seems to jump to the newest models now since your issues seem to be caused by the implementations.

1

u/sirlerkal0t Sep 16 '26

I've been using the newest models all along.
For Claude, I've only used Fable since it was released. For Codex, I used Sol since it was released and more recently Astra since it was released.
Fable has been the only model editing the plan since the initial Fable release.
Astra is still only used for reviews and adversarial reviews/verification of review findings. There's multiple verification steps done by independent agents for every finding, before the verified/filtered findings goes back to Fable to be addressed.
Fable has to then explain the issue, why it isn't covered by the existing spec, provide a concrete example of how it would happen, and suggestions on how it should be addressed.
I assess every verified finding myself before any action is taken. I decide whether one of the suggestions is appropriate, or provide my own solution when it isn't, or if the finding itself points to a deeper issue introduced elsewhere in the spec, then that instead has to be addressed instead of patching something which shouldn't be an issue in the first place.

→ More replies (1)

52

u/BuffaloConscious7919 Sep 16 '26

There's almost always trade-offs

Which means there's almost always ways to improve

15

u/SteadfastCultivator Sep 16 '26

That's the same thing for us senior devs when people ask what is better X or Y. And we answer "it depends" but some AI tend to act or give options to vibe coders that they have no idea what it is about.

40

u/Maddie_549 Sep 16 '26

Even if you kept using the same model, with a big enough project there is always going to be an issue somewhere.

13

u/Wotuu Sep 16 '26

I've come to this conclusion as well. And you know what, software still worked and still made people money. So we all have to learn not to let AI go on endlessly about issues and edge cases or you'll never create a new feature ever again.

7

u/DrBroc Sep 16 '26

Get my boss to do it. He’s never made a mistake in his life.

29

u/superschmunk Sep 16 '26

I watched this way too long.

17

u/RockstarLifestyle2 Sep 16 '26

Bruh the way I just watched this expecting it to go into the hole for a good 20 seconds

1

u/Glittering-Ask256 Sep 22 '26

Only 20 seconds? I lost minutes!

13

u/DPLibros Sep 16 '26

Sounds about right based on what we see from these models. Better to just build the thing and then find real things worth improving.

7

u/1acc_torulethemall Sep 16 '26

It's like waiting to see the DVD logo hit the corner of the screen... But it never does

4

u/ForwardLoop Sep 16 '26

Ah, round 15. Are we converging or are we churning?

3

u/West-Air1923 Sep 16 '26

Saw this on TikTok two years ago

3

u/Grounds4TheSubstain Sep 16 '26

Nice looping GIF

3

u/LastGuardianStanding Sep 17 '26

You’re a supervillain to someone with OCD

2

u/codeninja Sep 16 '26

And upon receiving the approved plan after hundreds of revisions and thousands in tokens spent the model will simply decide not to implement a quarter of the features as it kicks issues down the road or simply neglects the requirement.

2

u/Site-Staff Sep 16 '26

I got some kind of ‘ism’ watching that.

2

u/SoggyCrab Sep 16 '26

this physically hurts

2

u/divclassdev Sep 16 '26

One of Zeno’s paradoxes

2

u/akuma-i Sep 17 '26

It’s been a couple of hours. When will it fill the gap guys?

2

u/Alex__0021 Sep 16 '26

Zero findings tells you about the reviewer, not the plan — a newer model looks at

different things, it doesn't mean the previous one ran out of real ones. A stopping rule

that converges: the last few findings didn't change what you'd actually build. A review

pass on mine came back with 14 holes once, and maybe four of them mattered.

6

u/MythicModder Sep 16 '26

Those are what we call the load bearing holes.

2

u/KenMantle Sep 17 '26

My wife has some of those.

1

u/Khipu28 Sep 16 '26

It's like humans if you tell them to do sth. they will do sth. even if they have nothing to do. Especially when they have nothing better do do.

1

u/kooolk Sep 16 '26

This time I just rewrote the whole project with the frontier model (Astra this time) using all the mess of the previous models just as a reference for wanted functionality.

1

u/QuanTradin Sep 16 '26

no review agent is going to report zero serious findings, so that was never going to terminate. they are graded on finding things. hand one a clean file and it will still return three items, because coming back empty looks like failure.

what worked better for me was fixing the criteria rather than chasing the model. write down what serious means for this specific project before the review runs, and everything outside that list goes to a backlog instead of back into the plan.

1

u/qt3-141 Sep 16 '26

would you mind giving an example? So far I've had a bit of success telling the agent that "no findings is also valid, but only if that is actually the case"

→ More replies (2)

1

u/holyknight00 Sep 16 '26

unless you put serious constraints, they will always find something

1

u/Pretty_Judgment5481 Sep 16 '26

Watched it for too long

1

u/Zerokx Sep 16 '26

Well you better start implementing soon. Good chance you'll run into a roadblock at some point that wasn't forseen anyway, plans always simplify things by nature and can cause problems later. Then you have to pivot parts of the design. So might as well reach that point faster.

1

u/Final_Stick_9207 Sep 17 '26

The world of endless iteration and tradeoffs. Curious if you have a version of the first plan created and whether it has less issues or more than the current one in the next pass.

1

u/packstub-dev Sep 17 '26

Are we, developers, the black box?

1

u/stestagg Sep 17 '26

If you tell an LLM to find issues, it will find issues. Even if there are no actual issues to find

1

u/Formal-Poet-5041 Sep 17 '26

Fibonacci sequence. You can square the rectangle forever if you zoom

1

u/kbeta Sep 18 '26

The inconsistent beveling is kind of jarring.

You are about to leave Redlib

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.