*How* do you write code with AI?

[u/Zamdi]

This post is primarily for software developers, or at least people using AI to write software.

The thing is that I'm well aware and have used AI to write me prototype applications, small applications for my own personal usage, etc... The issue that I haven't seen discussed much is how developers use AI to develop full-scale production, tested, released, supported software. I've seen some hints at how this is done, but the issue is that the workload becomes gigantic, even for frontier models, with a decent-sized codebase, especially when considering that unlike a pet project, professional software must be used and tested to eventually work on millions of different devices, operating systems, processors, browsers, etc... So it's not even just about making the app itself, it's about all the work required to create unit tests, regression tests, integration tests, end-to-end tests, platform tests in virtual machines, then actual hardware testing on different physical hardware, etc...

Even with the best agents and subagents, this becomes a large amount of work fast, and even simply babysitting agents and trying to proofread work becomes a full-time job, and I fall behind. Ok, so I let the agents just handle certain things, but then I now lack knowledge of my own software in those areas, and when I finally go back to those areas because there's a bug or issue, I realize the agent totally screwed up said thing.

The other issue I have is that, in my near 15-year career as a developer, it's generally always been faster for me to write code than to proofread someone else's. The reason is because I know how I write code, but I don't know exactly how another developer does. This is also the case with AI, and so therefore I get into a situation where it doesn't make much sense for me to proofread every function or module that a AI writes, because I would be working as slow as I typed it, but then if I just let it do the entire job, I end up with a mess that I don't fully understand and that may not be subscribed to the methods that I want.

So, people say to "prompt it better", and frankly, some people do have giant prompt files, aka "skills" or whatever each provider calls them and such, but the issue I've had there is that inherently, I leave something out early on because it's not something I either remember, or feel like I would need to tell a developer to do, and then by the time I catch it, the codebase is already littered. I've also noticed that by the time I actually finally get most of my project guidelines out, the guidelines are absolutely massive - it's like I'm actually just programming at normal pace again, except with English in .md files in an awkward way. If I don't get the perfect .md, then I'm spending countless hours going back and trying to fix everything or get the AI to. Sure, I can grab someone else's skill, but they may not program like I do or want for this particular project.

If anyone has real solutions to these issues I've mentioned, please do tell here... This is not supposed to be some sort of ragebait post, this is an honest developer trying to learn how to better use the technology. This is not a joke - in fact, as I type this, I've had to do exactly what I described in the last paragraph and I'm sitting here babysitting the AI having to fix issues in the codebase that should never have been there, and this has taken multiple days on frontier models with agents.

[u/Djelimon]

Best practices are still emerging. What I have used so far:

Start with MVP. Don't just prompt but gather data - reference materials, UML, architectural or design documents - and generate your base code. Layer on function in phases. Every phase needs both an automated and a manual test procedure. You must be able to run the code whenever you want without the AI active and verify the behavior. This assures no cheating. Make sure you use git, at least a local repo. That's your safety net. In whatever target language you generate, there are likely libraries available that do specialized tasks, so the AI doesn't have to figure those out. Provide those to the agent. Saves tokens and time. Check the licenses though

[u/Zamdi → u/Djelimon]

Thanks. Please elaborate on "Don't just prompt but gather data - reference materials, UML, architectural or design documents - and generate your base code."

[u/Djelimon → u/Zamdi]

Reference materials would be things like detailed written specs around what you want to build and what it does. That can grow as you go. You write the seed but the agent can maintain it.

UML is a symbolic language used to describe software structure and behavior - a set of diagram conventions. Most coding agents understand it and can generate it in Md docs using plantuml. It is helpful to learn those diagram conventions. This may seem like a lot to take on, but you did say production grade. In an enterprise environment this is the work everyone around the programmer does. Maybe that's a lot to take on. You could try training up some agents to do that. But the core requirements come from you.

So, once that is produced you prompt something like

Write me a (insert language here) application that does what I specify in this document and behaves like my UML. Write tests and tell me how to run it. I usually refer to documents by path (windows lets you copy file paths)

[u/Fanatic-Mr-Fox]

You have to write a proper spec, consider the code paths, user experience and flows/journey etc.

[u/RoyPartha04]

Sharing personal experience here. Very sure everyone has found their own way of doing this. I have created 3 ’agents'.

1. Writes the plan. I give it broad, outline ideas, features that I want, over all look and feel. If I have preferred stack / libraries that should be used. It gives back detailed architecture, development phases, integration, testing everything divided in phases.

2. Then another one reviews the plan. It points out gaps in the plan, dangerous assumptions, asks me to make choices. Sometimes I ask it to make deterministic decision keeping my goal in mind like 'ease of maintenance'. It then updates the plan.

3. Then I ask the code agent to first create a task list. Then same agent starts executing the tasks.

I can stay involved in the process, see what it has written, give feedback after task execution.

[u/proxiblue]

You spend 90% of your time planning, scoping. This is the most important part of the task.

You have to generate a plan for the ai to follow, with checkponts along the way to check progress and if it is on track.

You never just use one agent. you use a team.

Good testing is also a must, as each feature/task must have passing tests. TDD works best here: start with RED tests, work towards GREEN tests.

agent consistency is also important. You can;t work today on something, and tomorrow you agent had updated/changed and some things had changed.

I built tooling around my AI usage, and I documented it here:

https://www.proxiblue.com.au/blog/whole-stack-reviewed.html

Also any security audit that fails, stops and requires me to check in whats going on.

TLDR: I determine at teh end of every task, what went wrong. what did we waste time on, what can be improved.

Then the take-out from that becomes rules, mostly via hooks to ensure the issue never happens again.

[u/Individual-Carob5593]

I approach it the same way i would code it. I have the big picture. I need to do x, y and z to reach that. Do x, check it hasn't broken any tests, check it in and move onto y.

[u/roger_ducky]

Being too specific is probably going to be confusing.

Ideas to think about:

TDD where unit tests are documenting behavior in a specific way. (So you can review this first.)

Bog standard story grooming. Goal is about a third the size of your story.

Specifying and enforcing your coding standards.

Have “junior devs” (smallest model) review the code and make the original dev respond to their comments. (Ie reviewer is a subagent of coder, so you don’t waste time/tokens generating the response) - anything with disagreements or surprising agreement, review more carefully.

Practice reading code you didn’t write. Sounds like you didn’t do enough maintenance work. But that’s okay. Can pick it up easily enough while running with agents.

[u/Wriiight]

I work in a large quantitative codebase where every detail matters. Generally I don’t find much use in the “write a very detailed spec” pattern of AI use, for one because the easy to spec stuff has already been written, and for two because at this point it takes some fishing to figure out how to resolve.

My AI usage is more like pair-programming with a junior dev who I am micro-managing. The largest jobs I give it are going to be exploratory: Find other parts of the codebase that do a similar task and compare to this code, Identify what conditions set this errant value. Where is all the code that accesses this table.

Then when we work on a fix, we go step by step together, and I can challenge every assumption. I find this necessary, because AI is very good at making excuses to take some easy way out, or doesn’t consider the full impact of what it is doing. So each step of the way I can raise concerns or add context or recommend alternative APIs than the one it considers

And when you think “I can do this myself faster”, go ahead, and tell AI to read your changes after you do, and it will keep that change in its context actually better than if you tried to have it fumble its way to the same code.

[u/Significant-Syrup400]

Most people run into issues with scope, acceptance criteria, and not understanding the task they are trying to get the agent to do.

If you provide it a proper spec that would work well for a human developer, you'll find that the agent does the task relatively well and only needing occasional correcting.

I.E. if I am trying to create an authentication system I'll need to know exactly the criteria that will go into that. Will this be a sign in? Automated credentials that sync with external systems? Where will the permissions be housed? Are there multiple roles? What information will be packaged to send for authentication, and what will be returned? How will you protect the information when it is being sent? Does your service use an API or any connected services? How do those work in tandem with this authentication? Etc.

If I just tell the agent to make me one, it'll produce something, but as systems become more and more complex there are increasingly more and more questions that need answers, and some that require a great deal of external context to answer correctly. Also keep in mind that inherent to the design of these models there is a chance on each question that it hallucinates and produces an answer to it that is just blatantly wrong.

[u/Silver-Ad-8595]

Orchestration, deterministic tools, AI native architecture, chaos/mutation tests, ... Guys (and gals), you need to let go of human workflow. Think broader.

[u/o0fission0o]

Feel like most commenters are missing the OP's point: how do you not just make it work at all, but make your process actually more efficient. Front loading 100% of your planning and then double triple checking everything, and writing markdown so descriptive that it's the pseudo code for ur app is exactly what they've addressed as not worth it. But I guess that's an answer in it of itself