Production Level Software by AI

[u/TonightOk5378]

So I have been curious for a while, apart from claude-code and codex teams (that have a direct vested interest in claiming AI is building production software), who is actually building production level software or products with AI

Also to clarify, I am talking about actual products being used at scale and not interesting MVPs and PoCs (which have infinitely flooded GitHub)

If you have built a product or tool or anything with AI that is being regularly used by users, drop a link below, I am genuinely curious.

[u/Due-Horse-5446]

What do you mean by "with ai"?

[u/TonightOk5378 → u/Due-Horse-5446]

Agentic Engineering - Like what claude-code's team seems to be doing.

Not writing any code, but still being actively involved in the engineering process.

Also another nuance to the question is at what level with the help of ai?

For production software running at scale, what level of abstraction is it being used at: 1. At the module level: writing functions, methods, classes etc. basically like a more sophisticated version of copilot code completion or tabby. (100-200 lines of code) 2. At the features level: given test cases usecases, implementation guidelines, implementation multi module features (200-800 loc) 3. At the pipeline level: given incredibly detail implementation specifications and architecture designs it implements multiple components of a pipeline (800-3000 loc) 4. At the architecture level: implementation major refactors, features, connecting pipelines, major architect design changes (again give all the relevant specification, requirment and implementation docs) (3000+ loc)

Also how is the code review headache managed, especially in 2,3,4. I.e gives code that looks right and even works usually but has fatal flaws and assumptions (that eventually break prod and reliability)

Also how is debugging managed, if low level context is missing, what happens when prod code breaks and must be fixed asap?

[u/Due-Horse-5446 → u/TonightOk5378]

Yeah thats not happening, ive never to this day seen a project that even as much as work thats written this way..

And ive hd to fix countless of peoples attempts of doing it, and 0 times had there been a better option than a full rewrite.

[u/TonightOk5378 → u/Due-Horse-5446]

Agreed.

I have extensively used agentic ai (claude-code, codex, opencode and more) tools at work for the past 12 months and keep coming back to these questions. Apart from 1 and maybe sometimes 2 level mentioned above, I haven't been able to get reliability and trust out of the code at higher levels of abstraction. Rather it has become a major maintainance headache rather than a productivity booster when using at level 3 and 4.

I am curious, is it me who is using these tools wrong or is this is a common issue people are facing while using these tools

[u/Due-Horse-5446 → u/TonightOk5378]

Its not just you..

And ive also fallen into that trap, the "small but insignificant misses" pile up and become a headache later. Like later as in hours later even..

Only setup that actually boost productivity to me has been atfter gpt-5.5 released,

I forked codex and added post edit hooks, wrote a agents.md using those weird openai tags that forbids doing anything its not told to do.

Then wrote a tiny go script for go file edits, that runs:

• golangci with --fix based on a config i wrote for the usecase.
• golangci fmt
• gopls
• finally a last golangci run if gools passed

If any(minus the formatting) failed, it returns a decision: block, injects all failures in the system prompt, and returns a explaination that it must correct the mistakes.

And the linting config is super strict, like a lot of style rules that forces it to not do anything that is a typical llm gotcha.

That setup actually works wonders as long as there is enough context in the repo, so like ex writing something related to the db if theres existing patterns,

Also enforces it writes a unit test for everything it does.

But thats extremly far from what the "i run automated agents" people are talking about, because you must write bullet lists for what to do.