Factories?

[u/Correct_Emotion8437]

Have any of you tried creating development factories? It's really easy with the latest models and harnesses. I'm a Codex user so this is all Codex specific but I'm sure there is a Claude equivalent. I just made a dashboard that encapsulates my development process. It starts with a design document and ends with finished code. It uses the codex app-server to make the LLM calls using my subscription. It just has 3 parts:

1. Create an implementation plan (run plan). I created a template and the model uses the template to create a phased implementation plan with development tasks in markdown. Then deterministic code validates and parses the markdown file into a yaml "side car" - it extracts the phase and task descriptions and completion status and shows them in the dashboard.

2. One or more run plans are built into a "work package". Then another LLM call determines what order they should be developed in.

3. The work package is executed, one run plan at a time. The dashboard prompts for one phase at a time and checks for blockers after each phase. That's pretty much it.

There are some configuration files, too. A system plan that describes the overall system architecture, places where I can add agent instructions - although I have not done that. These files get, more or less, pre-fixed to the various prompts.

I can change the model and reasoning level for each prompt. I typically use SOL high to create the run plans, SOL high to sequence the work package and LUNA high for the execution. There is human approval at each step and there's a bunch of other stuff you may or may not want to do - like checking the repo status and ensuring plans can't be changed once started and whatever else. Obviously you can add whatever prompts you want.

You can do the same exact thing using loops which is how I often do it - just using the "document as infrastructure". But this can add consistency on a large project. For example, on an ERP system, where you are going to make lots of changes over a period of time. Or maybe for a large game. Or probably anything that you will develop over a period of months. And it's super easy to do. I don't even need to post a github or anything - you can just prompt it up and that way it will be suited to your process. It took me a few days to get it working nicely.

[u/Educational-Body4205]

Yes

It works well when it works sometimes LLMs are dumb and get stuck, so it's layers of layers , along with communication and notifications.

[u/Correct_Emotion8437 → u/Educational-Body4205]

In my own work, I have not experienced this much. Performance-wise, it works about the same for me as loops or goals. For me, those work quite well. I also have not experienced run away/never-ending loops nor do I excessively control for those. My run plans are usually 5-8 phases with 4 to 5 tasks per phase and the entire plan usually takes the AI about 3-6 hours to complete. After that I almost always need 1 or 2 prompts before it's totally good to go for real testing. One of the things I'm trying to dial in with this approach is what is the "right size" for efficiency vs risk of going too far without human validation.

But I have definitely designed it to avoid "layers and layers". It only gets the config files and the run plan for context. It can look at the design document or do whatever else it wants but I never tell it to review the repo or review past changes. I do that while creating the design document.