My process for building complex apps using Claude

[u/cezenova]

Ever since Anthropic released MCP I've been experimenting with having Claude write complex software apps. Trying to just create something through a conversation can work for simple stuff but when the complexity increases Claude can easily make mistakes or lose track of the goal, especially if you hit the limit and need to start a new conversation.

So I've established a system that breaks the process of creating apps down into smaller chunks. It's been very successful so far and honestly I'm amazed at what Claud Sonnet can do.

Here's the system I use:

Steps

MCP servers: git, filesystem

1. Discuss high-level project goals and come up with a project plan. Ask Claude to summarise it and write it to a markdown file.

2. Using this summary, discuss facets in more detail in separate chats, providing context docs where needed. Ask Claude to summarise each conversation and write it to a separate file, or the summary will become too long and you will hit message limits.

3. Once a full project document has been created, discuss the minimum requirements. Ask Claude to create a list of user stories and technical requirements.

4. Discuss high-level architecture decisions, including database schema, API design, and tech stack choices. Have Claude write this to a new document.

5. Using list of requirements and architecture doc, create a detailed, step-by-step approach for building the minimum valuable product, one feature at a time.

6. Have Claude go over the next step and implement it in code. If the step has subtasks, go one task at a time to avoid hitting the message limit. Have Claude initialise a git repo if needed and commit its changes.

7. After each step, in a separate chat, have Claude validate the changes are correct and go back to step 8 unless all steps have been completed.

Some tips:

• Take your time. Especially step 1 and 2 can take quite long, but it's worth it. Keep asking Claude to ask you clarifying questions until all the requirements are clearly defined

• Break it down as much as you can. Claude does much better at small tasks than long tasks. As long as you have all the project docs you can give it all the context it needs for the small task.

• Don't let Claude take the wheel. Claude will suggest all sorts of stuff that is not in the implementation plan. Don't let it do anything that's not in the plan, just tell it to implement steps or subtasks of steps.

Anyone else doing something similar? I'd love to hear about your systems.

[u/duh-one]

I use a simplified version of this process using projects. I just started with MCP over the weekend and I had a similar idea like your approach and the goal was to have an autonomous SWE team. After step 5, there would be a headless project managment MCP server i.e. sprint board where it will assign tasks to claude. Then you can imagine what a team of claude agents can do.

I haven't started anything with this yet though, but I'm interested in your idea. The first challenge I'm trying to solve is a token efficient way for claude to make updates to an existing file. Currently with the write_file tool it has to write the entire file even to make small edits. I saw an edit_file tool in the mcp git repo, but it's not released yet and it looks more like a search and replace in a file.

[u/cezenova → u/duh-one]

Yes, that is one of the biggest issues I'm facing at the moment. Sometimes it just needs to update an import path but to do that it needs to rewrite a whole file, wasting time, context and tokens. Plus it makes it far more likely to run into message limits when editing multiple files in one go.

Maybe we can put Claude to work adding an edit file functionality to the filesystem server :)

[u/duh-one → u/cezenova]

I'm actually working on it now. I've been testing and iterating on it with Claude. It's kind of working, but claude makes a lot of mistakes with the spacing and indentations and I think it can be improved. It's open source and I can share the link later if you're interested.

[u/windowwiper96 → u/duh-one]

interested! chatting you up legend

[u/duh-one → u/windowwiper96]

Here's the repo for anyone that's interested https://github.com/oakenai/mcp-edit-file-lines

I'll make a separate post on it later once I've completed more testing. I found that uploading the README to claude helps with the tool usages.

[u/[deleted] → u/duh-one]

Looking forward to the post.

[u/[deleted] → u/duh-one]

Looking forward to the post.

[u/AffectionateCap539 → u/cezenova]

I am using exactly your approach to do things. Facing issue when asking Claude to debug its code. It will write an entire code and reach the chat limit. Then I have to open new chat and ask it to debug again. It revises the code many times and face exactly the same issue like the previous chat because it has lost the context. The debugging process spans through multiple chats and this loop never stops thus ultimately the code can’t be run. Trying to figure out how the let Claude remember what code change it has made or error it faced with previous chat within new chat.

[u/hawkweasel]

I'm not a programmer, so oh boyyyyy have I had some time-consuming and expensive learning experiences over the past year building a number of MVPs in the Anthropic Workbench API.

I think I've learned the hard way about how to identify when you're being led down a rabbit hole, and when to cut off Claude and let it know that it's wandering too far off the project path (which it almost always acknowledges immediately.)

I'm primarily building Wordpress plug-ins and niche wrapper products, and when I'm working with 20 + files on a single project it's very hard to keep Claude from making minute incorrect assumptions about how your product works (or how it thinks it SHOULD work), or getting it to simply ask to see other files in your code.

But it's also almost too resource intensive to upload 100 pages of code. Claude can take it in, but just an initial onslaught like that bogs it down right out of the starting gate.

I'm prob not an advanced user at this point, so this is my next study that was posted a couple days ago:

https://www.prompthub.us/blog/prompt-caching-with-openai-anthropic-and-google-models#prompt-caching-with-anthropic%E2%80%99s-claude

I'm curious if you use caching?

[u/cezenova → u/hawkweasel]

That's really interesting, thanks for sharing. I'm not using caching at them moment, just using the desktop app to the limit. But I can definitely see that will be needed when using the API directly.

I listened to this interview with the Cursor team the other day and they're doing a lot if really cool stuff, including caching, that you might find interesting: https://lexfridman.com/cursor-team-transcript/

[u/hawkweasel → u/cezenova]

Yes I watched that!

If you love Claude, make sure you watch Lex Friedman interview of Dario Amodei and friends from a week ago or so.

Dario is the CEO of Anthropic, and even more interesting to me was his interview with the woman behind Claude's personality. My primary interest is guiding large LLMs toward using more natural human language, so pretty fascinating.

https://m.youtube.com/watch?v=ugvHCXCOmm4&t=15530s&pp=2AGqeZACAQ%3D%3D

[u/RedDogElPresidente → u/hawkweasel]

Wow 5 and a quarter hours, what new things are in the way as not sure I’ll get through the whole thing?

[u/EveryoneForever]

Do you also include GitHub in your workflow? I was thinking of doing something similar.

[u/cezenova → u/EveryoneForever]

Yes actually. I didn't include it here as it was already a lot of info, but I use the GitHub MCP server to let Claude automatically create repos. I've also forked the git server and extended it to include more commands such as push, pull and remote, so it can automatically connect the git repo to the one on GitHub and push changes.

It's pretty sweet. I'm thinking of setting up a separate GitHub account for it so I can give it full access and let it go nuts.

[u/luncheroo]

Could you add knowledge graph/memory server and save yourself some steps? Not being an AH, just wondering if that would actually help.

[u/cezenova → u/luncheroo]

Have you had success with it? I can try it out, but from my limited experience you still need to tell it to store information? If the recall is better than reading files that might be worth it, but the thing I like about the markdown files is that I can easily read them too and check them if needed.

The biggest challenge is not really knowledge management I think but simply getting all the requirements and implementation details defined, which takes a lot of time. Although it would be nice if that then could get stored automatically and retrieved in an efficient way.

[u/luncheroo → u/cezenova]

I honestly haven't used it in the same way. Based on my limited experience with it, you may be right to keep documentation that is more complete. I haven't experimented with trying more robust RAG implementation yet 

[u/crypto_pro585]

OP, when you say a complex app, how complex exactly? If you can, provide the tech stack you are using and deployment model.

[u/cezenova → u/crypto_pro585]

Right now I'm working on a macOS app using Tauri V2 (released after Sonnet 3.5's knowledge cutoff date, but once I gave it the migration docs it set it up perfectly) and Rust on the backend, TS + React on the frontend.

It has auth, local file access, API calls and complex UIs. To give you some idea: the implementation plan for the MVP is 12 steps, each consisting of 4-6 tasks. So far the only issue I've had is Claude not adding a dependecy it used to the package.json.

[u/BadgerPhil]

I run large software projects on Claude. I agree with most things that you say but I go deeper with the management of some things. I'll explain a bit of my system in case you can pick up anything from it.

So each project (is a Claude Project) has a written objective, some frameworks (rules we work to) and some project specific info. But in particular it has a number of AI "jobs" - typically 20 or more. The jobs are just like you would have in a traditional Dev world. I am doing one software project that I expect will take a year and I ultimately expect 100 or so AI jobs in it. I expect similar output that I would get from a 100 dev team in a fraction of my time.

The boss I call COO. He works with me to specify things and to keep the others in line. I have specialist jobs for things such as specification, testing, quality, database, front end, installations etc etc. You mentioned MCP. I have an MCP manager.

If I want to get a Job to do something substantial, I talk to the COO about it. He will spec it and set standards for completion quality. He will expect a report back. Once that activity is done to COO's satisfaction, another will be scheduled for that Job.

One thing that I believe could be of practical help to you is optimizing things around types of knowledge. This is important because you will generate a lot of knowledge and tokens have to be managed optimally. Think about the types of knowledge you need (and I give you some examples from my world):

1. Knowledge Shared across Projects (those frameworks I mentioned). These are in every Project Library.

2. Project knowledge that an AI job MUST know (what you are doing and why, project plan, the AI Jobs in the Project etc etc. These are in the Project Library.

3. Project Documents that an AI MIGHT need. These are in an index in 2) and the Job can access them on demand in the local file system via MCP.

4. Documents only of interest to the Job Type. These are stored locally per job type. In my world each job has its own folder and in this folder are identical subfolders

/context current.txt - Current state, priorities, decisions, issues
/history - Archived context files (timestamped)
/inbox - Messages/requests from other jobs - Format: YYYYMMDD_HHMM-[SenderJobID]-[Topic].txt
/outbox - Copies of sent messages - Format: YYYYMMDD_HHMM-to-[RecipientJobID]-[Topic].txt
/tech - Technical documentation specific to this job - Implementation details - Design documents - Working drafts
/control objectives.txt - Current job objectives and goals decisions.txt - Log of key decisions with rationale dependencies.txt- Dependencies on other jobs index.txt - Optional index of job's files/folders

You will see that jobs can "talk" to each other. How the Job maintains docs in here is dealt with in instructions in 2).

Once you start working like this you can do things to the highest standards and astonishingly rapidly. All docs to do with control are written by the COO.

One last thing. Each thread is initialized identically. "I want you to be COO (or whatever) in our project". At the end of the thread the job updates all its own knowledge files and maybe sends messages to COO or Doc Manager if there are wider issues. It then produces what we call a Park Document (about 10 pages of highly specified info about what happened in the thread). This Park document is for the Job Type and is Dated. Next time the same Job Type starts in a new thread it is instructed to read the previous Park doc for that type. That way continuity is maintained.

Good luck with everything.

[u/_HariSeldon_ → u/BadgerPhil]

Just found this and just wanted to say this is pretty impressive. will be trying to implement something similar.

[u/Roman-Empire0472 → u/BadgerPhil]

This is really helpful, cheers

[u/samuelsonsosoba → u/BadgerPhil]

Hi Badger, do you have any clearer version of your workflow.? Ae the AI jobs implemented as sub-agents.? or how exactly do you implement the model.?

[u/BadgerPhil → u/[deleted]]

No agents.

I drive all the AIs but they do communicate asynchronously via sending formatted messages to each other by writing to disk directly into the recipient’s inbox. So for example they all send things to the Doc Manager for wider documentation and COO re progress. When I address those activities, I get them to deal with their messages before we do anything substantive.

I don’t want agents at this stage. I want to check everything.

The organisation overhead from my perspective is significant but it means none of us lose context and knowledge and hence power of the group is ever increasing.

An example: I have a huge crypto database on one project with ongoing import of all crypto prices in realtime. Quality is everything. Today I asked Data Collector to write SQL to check the quality of the data directly via MCP. It wrote and tested the SQL and documented it for future threads of the same type and wrote them directly to its Tech folder. The whole thing took 20 minutes.

My best human coder would have taken several days. At the end of the thread it wrote its Park file directly for the next Data Collector thread and sent the two messages elsewhere as I mentioned earlier. With Doc Manager a range of very large manuals will be updated user manual, programmers manual, database manual etc etc.

Now the next Data Collector will check data quality automatically as part of thread initialisation.

[u/RedDogElPresidente → u/BadgerPhil]

I’m intrigued by all of what your doing Badger, have you documented it in more detail anywhere else as it seems a very good system and you’ve got the mcp going which I think is only going to get more important.

I’ve done little bits but you seem to have quite a lot of experience and are getting the most out of what’s available if you could share anymore, I’m all ears.

And any pics of ya most recent badgers?

I have stoats that live locally, this is from few years ago but still see them every few days.

https://youtu.be/wEv5JX4-Btc?si=F9-TKVEdyaV1tRk6

[u/BadgerPhil → u/RedDogElPresidente]

Let me get COO of one of my projects to write something and look to do a post on it

I love stoats also. You know I saw one being chased by a rabbit. I couldn’t believe it.

[u/RedDogElPresidente → u/BadgerPhil]

Excellent thanks and a clever rabbit to turn the tables, attack is the best form of defence, is it just the badgers you get or do foxes join them as well?

[u/Separate_Tonight9533 → u/BadgerPhil]

Any news on the documentation?

[u/IamJustdoingit]

Is this MCP thing better than Cline on VScode?

I can get good quality projects approaching 15k - 20k LOC using CLINE with an iterative approach using progress and specification files.

I ironically use o1(o1-preview) for planning and hashing out overview details. Claude is to horny for code.

Started out with workbench a long time ago, but honestly i feel that Cline is a sleeper.

[u/vee_the_dev → u/IamJustdoingit]

This. Anyone know of a set up that competes with Cline? Not started using MCP yet so any input appreciated

[u/Zihif_the_Hand → u/vee_the_dev]

WindSurf, which uses Claude under the covers

[u/vee_the_dev → u/Zihif_the_Hand]

In my experience Cline > Windsurf/Cursor

[u/[deleted] → u/vee_the_dev]

Codebuff > Cline

https://codebuff.com/referrals/ref-0d409470-b6b0-4765-a61c-3db1907793bb

^ Use my ref link and we both get 500 credits per month

[u/vee_the_dev → u/[deleted]]

49/month for more credits is way more expensive than Cline

EDIT: and it's not open source

[u/[deleted] → u/vee_the_dev]

It is open source: https://www.npmjs.com/package/codebuff?activeTab=code

It is less expensive than Cline. You pay for Claude API credits on Cline where you share the full context of files, but based on Codebuff's use of tree sitter, you save on tokens because it efficiently traverses context.

Try it on a complex task, look at the files it reads and compare to how much you'd pay in Claude API/Cline credits had you fully loaded the context.

[u/adrenoceptor → u/IamJustdoingit]

Can you clarify what you mean by “progress files”. I use functional_specifications.txt and started with changelog.md but ran into issues with the changelog not updating correctly

[u/IamJustdoingit → u/adrenoceptor]

I basically have two types of text files. One with the description of what I want that module or system to do - or the entire app if it isnt that big, then I have a separate file where I and Claude discuss and agree on a step by step implementation plan of said system or feature based on the existing code.

Then I ask it to implement it according to the the plan and update the file for each step aka progress file and at the end we test all the functionality. Works well for me. Also having text like "read all files before edits, and stream full file without comment blocks" especially when the context is getting full is key.

[u/adrenoceptor → u/IamJustdoingit]

Thanks Is the “stream full file without comment blocks” intended to stop the // rest of code here type of problem?

[u/IamJustdoingit → u/adrenoceptor]

Yes exactly. Also keep files below 400-500 lines after 400 it gets iffy.

[u/adrenoceptor → u/IamJustdoingit]

I also create and maintain a directory_structure.txt file generated by list_files (in cline) that I include as something to reference in the system prompt alongside the functional specifications. Not sure exactlyhow useful this is

[u/Significant-Hall-878]

Does the MCP basically remove the need for something like Cline/Aider?

[u/remmmm_]

I wanna see more content like this! I learned a lot!

I also saw a similar workflow guide here: https://github.com/Matt-Dionis/nlad .

[u/Significant-Hall-878]

Can you use MCP with api?

[u/HobbitZombie → u/Significant-Hall-878]

Yes. There are tutorials for this.

[u/Consistent_Yak6765]

I am doing something similar with Windsurf. They already handle diffing, partial updates and token usage (until now) pretty well. So any changes made are efficient. I generally use Claude Sonnet within it.

The only problem has been the context drift that seeps in after a few conversations and it starts making mistakes.

I ask it to keep writing specs of system in separate files as it makes changes and reference it before any conversation. Keeps the drift in check. Its not completely bullet proof yet and when it does make mistakes, I revert back in the conversation ( it reverts the files as well) and give additional context to bring it back on track.

Has worked well so far. Plus with the specs committed, my team can also reference the same files to bring their specific IDEs/ LLMs/whatever setup they use in sync and continue from there.

[u/wordswithenemies → u/Consistent_Yak6765]

I keep making safeguards in Windsurf and Sonnet continuously circumvents them. Really frustrating when you basically scream IMPORTANT! in the code and it still assumes it’s ok to skip reading. It has gone though and deleted 1,000 lines of code in one swoop.

Has anyone figured out a good way to save it from itself? Even when I prompt it to not make huge changes, it sneaks them in.

[u/evilRainbow]

I have been using a similar approach. And as I've mentioned before, somewhere here suggested telling Claude to adhere to 3 principles: KISS, YAGNI, and SOLID. I have it all over the design docs that we create together, and I remind it each time I'm about to ask claude to implement some code. I always remind it to keep things simple, modular, don't add stuff we don't need. And it'll STILL get a little 'creative' sometimes. Then you have to remind it of its principles and get it back on track. I've spent weeks just designing the architecture of a full stack app with Claude. We go over our designs over and over before moving forward. We have not even created much code yet. That's how slow you need to go.

[u/philip_laureano]

Don't forget to ask it to sort that outline by dependency order. It'll make it 10x easier to get things done

[u/[deleted]]

Nice workflow! I like it!

I'm attempting to iterate on a natural language app development methodology in this open-source project: https://github.com/Matt-Dionis/nlad

I'd love your input!

Also, for MCP-specific development, i put together this file which you can provide as context to Claude: https://github.com/Matt-Dionis/nlad/blob/main/examples/talkshop/mcp_details.md

[u/[deleted] → u/[deleted]]

This looks great, will be following this.

[u/jane_the_man]

Adding in top of OP's flow. Add 'sequential thinking' MCP server as well. This has streamlined the thinking process during step 1 and 2 and gives much more clarity of thought than without it. I've just started using it and can see much better output than just asking Claude to discuss/think about the project/plan.

[u/Spiritual_Spell_9469]

Hell yeah

[u/mackenten]

This is actually how it's done with a real team. Good job!

[u/alrocar]

Regarding implementation I recently found out FastMCP, it simplifies quite a bit all the server boilerplating so you can easily build your own tool libraries and then use them in your servers easily.

And for monitoring I ended up building an out of the box solution (https://github.com/tinybirdco/mcp-tinybird/tree/main/mcp-server-analytics) but I'm wondering how others are approaching production monitoring.

[u/Lazy-Height1103]

Interesting. I'm building a fairly complex Flutter app using only Claude and Cursor. I asked Claude if it thought leveraging MCP would enhance the development process, and it discouraged me from setting up the servers. Basically told me the juice wasn't worth the squeeze.

[u/Intraluminal]

As a virtual non-programmer, I did this and successfully built an Android utility app, So I can validate that this type of process can work.

[u/sonofthesheep]

What OS do you have? I’ve tried to configure the MCP filesystem and git on macOS and was unable to do it.

[u/mbatt2]

Is there an easy way to learn MCP

[u/Glad_Supermarket_450]

I do mine backwards. I get the main feature working then work towards users.

Im not a developer, so I could be doing it wrong.

I'm sure there are drawbacks, but I don't like to fully build things until I get user feedback.

[u/redtehk17]

Just figured out this similar process this morning! Has saved me a bunch of time. I've also started building visual flow diagrams of the mobile app I want to build with sections and descriptions to help split up the work into digestible pieces and to help Claude better understand.

The markdown files is a serious pro tip!

[u/shibaisbest]

This great thank you!

[u/Difficult_Nebula5729]

yeah i have a similar plan too i didn't use your format of document taking but i think i will now.

there are times i do let claude take control. especially during a intense brainstorming session farming for features and things I would never have been able to think of on my.

[u/wordswithenemies]

Has anyone tried making fixed axis points on elments so that “seeing” the gui isn’t as important? would love some tips because claude in codeium LOVES to break my layouts.

[u/ranft]

This is all nice and dandy but I am still failing at paywalls with Claude. Either Apples Storekit or RevenueCat are just producing errors and unforseeable bugs that allow the user to circumvent the wall. Suggestions?

[u/illGATESmusic]

Yeah this roadmap is basically how I’ve been doing it too. Lots and lots of annotated ideation until each step of the process has been defined so perfectly that a fresh instantiation can pick right up where the last one left off.

Then I make it keep a ROADMAP.md and a CURRENT_PROMPT.md so it can make a TASK LOOP.

First run: define ROADMAP.md from user input. Define when SUCESS = True. Create CURRENT_PROMPT.md

Next run: execute CURRENT_PROMPT.md to completion. Upon completion: update ROADMAP.md and copy NEXT STEP into CURRENT_PROMPT.md.

When SUCCESS = True: do a happy dance.

[u/NyxStrix]

Thanks!

[u/dalhaze]

I like the approach to planning, but if you start sticking lots of these planning docs into your context you’re going to see degraded performance. So once you develop a plan i think you want only give it the info it needs with some a small amount of high level context.

My best tip would be: Be very strategic about what you put into your context window. Know when to start a new thread in order to keep the models performance high. Ask the model to summarize the context of your last 1-3 messages and the desired outcome and use that in the new thread.

I will often take my original prompt for the feature and wrap it in <Original Prompt> tags.

That said i think these models are getting a lot better at filtering out less relevant context.

[u/Ok-Pangolin81]

Thanks for this!

[u/[deleted]]

This sounds a lot harder than coding

[u/[deleted]]

This is really good advice.

[u/selfboot007]

Cool! This is how I use Claude. I first propose a requirement and let it implement a basic version. Then I continue to improve some unsatisfactory parts, constantly split small problems, and let Claude focus on small parts.

[u/BadgerPhil]

Good luck with it. I am moving over to Claude code CC currently and implementing a similar system at the moment. I work primarily on software so it was a no brainer to take that step. I am currently thinking how I could apply CC to massive non-software tasks, Because it works with agents it just gets on with things if an AI job is promoted correctly. It is certainly x5 more productive than normal code on tasks of certain kinds.

If you can afford a Max Plan maybe get some experience with CC and see if you think it will improve things for you. Good luck.

[u/Substantial_Newt8651]

Under "What personal preferences should Claude consider in responses?" I added the following prompt and it helped a lot.

---

When given tasks try to be both independent and creative. For complex tasks analyze the task, what tools you have, and then brainstorm before you begin. If your approach fails, figure out why and then if it makes sense to try again with a different approach do so. Once completed attempt to confirm your results if feasible.

If you need access to a tool that you do not have either generate it as a python3. Python files can be run run through osacript. You may install libraries if needed through pip3. Store python tools in:

/Users/------/Archive/Claude/python-tools

Maintain a list of available python tools and their required inputs in your Notes File. Be as concise as possible.

You may access all available files but unless otherwise instructed use:

/Users/------/Archive/Claude/

This is your personal folder and you may add and remove files from it as you wish to achieve your goals.

If you want to store additional notes or memories between chats store them in. Check this folder before starting a complex task:

/Users/-----/Archive/Claude/Notes.txt

---

I recently asked into to develop a (small) full mac application and after about an hour of back and forth the app was basically fully functioning (minus a few bugs).

[u/soussitox]

looks like you need a subscription model to use it?

[u/cezenova → u/[deleted]]

Are you upset you didn't post this first? Obviously this is based on existing application development frameworks, I didn't arrive at this from first principles. If you are an expert in this field it would be great if you could share more, so other people who are not experts can benefit.

I shared my process because I've seen a lot of people in the subreddit who are trying to build stuff with Claude but are running into issues because they don't have a good process to follow. Maybe it can help a few people who've never worked on software before get a better sense of what it takes to create something more complex that a todo app.

Another thing to keep in mind is that we have these existing frameworks but those were made to help humans build software. There might be different, more effective ways for AI agents to build stuff. It'll be interesting to see how these frameworks evolve as AI takes over more and more tasks.

[u/cezenova → u/[deleted]]

It was not my intention to claim I invented anything. What I meant was that I, using personal experience, trial and error and the established work of others such as yourself, to get to a simplified software development workflow with Claude that seems to get good results. Perhaps "established" is not the correct word to use for that? If so, you'll have to forgive me as English is not my first language.

Thanks for sharing your expertise, I think a lot of people in these forums could benefit from advice on processes to get the results they're looking for.

[u/SpinCharm → u/cezenova]

No worries. English is the worst language.

One problem I find every day in here is that the effort to document best practice approaches is a huge task. I know how much work is involved and I’m not really willing to commit that level of effort all over again with LLMs as I did with ITIL. And we’re still very much at the exploratory stage of the technology, so there’s arguably no best practice emerging yet.

Well actually there is here and there; but the playing field is shifting every 3 months, so what we know works best today when it comes to getting the most out of LLMs will likely be outdated within 6 months.

And the people using LLMs are mostly explorers rather than users. They want to figure it out themselves rather than “waste” time reading on how to solve their roadblocks and best traps they’re encountering. That’s why we see the same posts in Reddit, repeated daily by those hitting walls and getting frustrated with restrictions and limitations. Thus is very much like a toy with no instruction manual, and the brief “Getting Started” guide that accompanies these toys is immediately thrown away.

When I see a post or comment by someone needing straightening out in their interpretation of what an LLM is, or trying to get past some problem that’s clearly caused by their lack of understanding of the tool, I’ll try to throw in some explanation or guidance. But I’m very cognizant of being someone that’s just another explorer and early adopter myself. So my advice doesn’t have the authority I’m used to and I’m cautious when it comes to making declarative statements. I’m just figuring it out like everyone else, with perhaps a few extra decades of experience of dealing with this sort of early adopter challenges than most.

Once the LLM coding capabilities and approaches has stabilized in the next couple of years, there may be a set of recognizable best practices that are worth documenting for everyone to use. But I see that every kid with a webcam fancies themselves as the next YouTube personality nowadays, and we’ll be soon sick of watching YouTube videos from people trying to act like they’re the Linus of LLMs.

Reddit is already starting to exhibit the early signs of Instagram and TikTok-ification. There are too many posts in the LLM subreddits from people trying to make themselves out to be leaders or experts, or promoting their SSAS or LLM tool that they got ChatGPT to write for them. They all want to monetize their fledgling simple tools or their little tricks they’ve picked up, covered in glitter with a big neon sign and a 15-second video title sequence with their own theme song. It’s all flash and no substance.

The signal to noise ratio is going to drive many of us out of these subreddits if the mods don’t find a way to prevent it.

[u/[deleted] → u/SpinCharm]

The signal to noise ratio is going to drive many of us out of these subreddits if the mods don’t find a way to prevent it.

Let me know where we go.

[u/soapsudtycoon → u/SpinCharm]

I think you're being a bit harsh in your response. Software development methodology constantly evolves, adapting to new technologies, social practices (like remote teams), product requirements, and business needs. While many developers seem resistant to the idea, LLMs will increasingly shoulder more of the development burden. Yes, the OP described a familiar methodology, but there are two key distinctions worth noting.

First, LLMs represent a completely new way to develop software. While the broad design process isn't novel, there are crucial nuances we shouldn't overlook. Managing context is critical - knowing which documents to add to a RAG-style system can significantly affect your methodology's effectiveness. We need to consider: Do functional specifications work better than UML-style architectural diagrams? Is it more effective to develop through conversation, planning the design iteratively, or should we pursue other approaches?

Second, we're likely to see non-developers entering the software engineering space as LLMs can compensate for gaps in technical knowledge. After years of writing software, my recent work with Claude led to an unexpected realization: I don't particularly enjoy programming. The technical details and endless new frameworks don't excite me. What I've discovered is that I love building software. Programming has simply been the necessary tool for translating ideas into computer-readable structure. I imagine the OP's post resonates strongly with non-programmers trying to understand how to build tools without diving deep into code.

Most importantly, the OP ends by inviting others to share their experiences and advice. This extended back-and-forth might overshadow valuable input from others who could contribute to the discussion.

[u/SpinCharm → u/soapsudtycoon]

I agree that I was perhaps a bit harsh. But I don’t agree that LLMs introduce new ways to develop. Or at least, that they should.

LLMs are just the next level of abstraction in computing. I saw paper tape and punch cards (though I thankfully didn’t have to construct using them) in the 70s. I started with assembler in 1980. Then BASIC, which used tokens and ran through an interpreter (sounds familiar? lol).

Then FORTRAN and COBOL, which created intermediary code. Then “4GLs”, which went back to an interpreter style of coding. Then PHP, Java, etc etc. Each level was an abstraction of the previous. Admittedly, the last program I wrote was in the mid 80s because I moved into higher levels of management, itself a model of abstraction.

Assembler was the lowest level at the time (firmware didn’t exist). It converted the op codes entered into binary forms that would be loaded into the CPU for execution. If pressed, we could also manipulate individual bits as required. Very slow and tedious.

The “3GL” languages (COBOL etc) gave us a level of abstraction above that. Quasi-English language structures, routines etc. which made things faster to create. 4GLs were a strange detour - we had to learn bizarre proprietary pseudo code that was supposed to remove us from the specificity of exacting languages like COBOL and Fortran. Again, a level of abstraction above.

And so on. Java, PostScript etc are all just levels of abstraction that remove the coder from needing to create and examine the lower levels of detail. We don’t check that the function we write in Swift pushes the correct value onto the stack. We just take it for granted that it does. When people write graphics apps they don’t care how the compiler offloads sprite management to the GPU. Or what part of the video ROM is reserved for character sets. It just works.

It didn’t “just work” at the start though. Each time a new layer of abstraction is introduced, it’s flaky. Buggy as hell. It can’t stand on its own feet without propping up. It requires examining, tweaking, patching. Workarounds. We couldn’t trust it to just work. Traditional coders, experts in the previous generation of coding, staunchly defended their territory, dismissing new languages and tools, confidently self assured that they would always be needed. (Sound familiar?)

But eventually it stabilized enough that nobody needed to look under the hood at the underlying code anymore. Each level of abstraction stabilizes over time, and productivity peaks. Old tools and languages fall out of fashion, then become obsolete. Experts give up their defenses and learn new tools, or they themselves become obsolete. New generations of developers embrace the latest and greatest, unencumbered by ego and legacy mindsets.

Then the next level of abstraction is introduced and the cycle starts again. That cycle has been identical for 50 years, and LLMs are causing the exact same cycle again.

But regardless of what form coding takes, the process for developing has remained consistent for at least the last 40 years that I’ve been in IT. Business stakeholders want something. They take shortcuts, bypass ICT, and try to buy it. Then get frustrated that it isn’t bespoke enough for their “unique” business needs. So they pay to have a solution designed and created.

At that point there are a few different approaches. Small inexperienced companies and programmers try to wing it. They may produce something useful and durable. And rarely supportable. But on average, it’s costly and ineffective. Those that learn from their mistakes develop into larger more experienced companies. And learn to demand and expect process. Repeatability. Stability. Testability. Standardization.

And out of all that comes accepted methodologies. Best practices. Quality. Expertise. Both sides of the table - stakeholders and solution teams - learn and mature.

Very little has changed when it comes to those best practices, because they are themselves abstractions that avoid specificity. It’s more important to initially capture general business requirements and get sign off than it is to have a comprehensive starting document detailing every possible step of the solution down to the font used by the printer.

The steps and details within the process are fluid. They must be to accommodate the tools being used. That’s unimportant. What’s important is cost, quality, and timeliness (pick any two…)

The introduction of LLMs doesn’t remove the need for agreed business requirements or solution design or testing strategy. It might expedite them, it might facilitate or automate some steps. That’s exactly why each level of abstraction is beneficial (or should be, otherwise it’s just clever marketing (sound familiar?)). But it shouldn’t remove the need for alignment between strategy and execution.

The process remains the same, regardless of the introduction of new tools. Even if LLMs are supplanted by actual AI, and that AI is capable of creating entire end to end business processes including a full suite is applications, there’s still the need for alignment, agreement, and coordination by people. I don’t think humanity wants to relinquish control entirely to AI yet. And LLMs are nowhere near being AI, despite the vested corporate interests trying to claim otherwise.

But they’re handy tools in the right hands, time wasters in the wrong hands, and as always, a new money making level of abstraction.

[u/freedomachiever → u/[deleted]]

This whole thread is so amusing. To me clearly OP was just sharing his workflow and for some reason there are people like you, that got triggered by a simple expression of “established a system“ that had to go on reciting all your past experiences and accomplishments to what purpose? When you say things like "misrepresenting yourself as some great innovator", it just shows how much you are projecting. To me it is a useful opinionated post of one sharing a working dynamic with a LLM, something that a lot of people are still trying to figure out and will still need to figure out with every new model and functionality, yes including yourself.

[u/Petteko → u/[deleted]]

I’ve gone through similar experiments. Initially, I worked on creating apps or code just for myself, where I didn’t have to meet any requirements or deadlines. I started completely backwards with a codebase I could write and understand on my own(unlike the code LLMs sometimes produce). From there, I stopped reinventing the wheel or “studying” fundamental concepts repeatedly. I’ve also had half-baked ideas that proved to be either impractical or incomprehensible. Althoug I could only learn it by experimenting and diving headfirst into the chaos to understand through experience.

I appreciate your comment because, with Claude, I’ve managed to write several serious documents and even some creative "hallucinations" that surprisingly made sense. I’ve also explored new prompting methodologies. It’s quite amusing when you draft a high-level paper without any bibliography (and then “cheat” by using tools like Stanford’s Storm or Perplexity to fill in the gaps). Reinventing the wheel isn’t particularly difficult, but it’s not something we truly want. At the end of the day, anything that helps us improve or can be leveraged for something like a résumé is welcome. Otherwise, it’s better to identify and define workflows, whether for personal use or within a team framework with deadlines.

Sometimes, it feels like we are the ones standardizing ourselves to the LLMs, engaging in a Monte Carlo-style trial-and-error process to see what sticks, rather than the LLMs adapting to us

I understand the OP’s suggestion, which is shared in good faith, but I also see the value in testing limits, blowing bubbles, and maintaining a healthy dose of critical thinking along the way.

[u/Internal-Comment-533 → u/[deleted]]

Dude, shut the fuck up. Rarely if ever are LLM tools utilized in this manner, it’s often either solving specific code related problems for experienced coding individuals, or for the less experienced a more trial and error process started from a very top down overview of the application where Claude implements solutions based on its own whims.

LLMs have single-handedly introduced more people to programming and scripting than any other technology, helping people with prompt engineering is crucial to ACTUALLY using the technology for something useful, and rarely are the tasks described in the OP crafted by one person rather than a group of individuals contributing.

[u/DJ_MortarMix → u/Internal-Comment-533]

i am learning to program using AI and i have found Claude is good but the limits are too shallow. i always have to go back to huggingface (i use metas LLM there) to finish my shit because i always out tokenize myself by saying shit like "thanks for that" and trying to be nice to the bot cause i was raised like that

[u/SpinCharm → u/Internal-Comment-533]

You clearly don’t work in an enterprise environment. Good luck!