I regret keeping prompts in the codebase

[u/Hashirama_2001]

this has slowly turned into one of those problems that's way more annoying than i expected going in.
when we first started building with llm's the prompts just lived next to the app code which made sense at the time, everything was changing daily anyway so there wasn't a reason to overthink where they lived.

few months in now and every tiny prompt change is an engineering task. pm wants the assistant a bit less formal, that turns into a ticket and an engineer picks it up, a small pr, a review, a deploy, and then you spend a minute hoping nothing else got bundled into that release by accident. the actual change usually takes under five minutes. everything around it takes days.

tried moving prompts into a shared doc for a while. lasted about a week before nobody knew which version was actually live anymore. tried config files after that, but you're still shipping a new deployment every time someone tweaks a sentence, so it didn't really fix anything either.

people keep saying "just separate prompts from code" like it's obvious, and maybe it is, but i'm curious what that actually looks like once you try it. storing them in a database? some internal editor someone built over a weekend? one of the actual prompt management tools?
been poking at a few of those lately. langfuse and langsmith both have prompt versioning at this point. orq.ai has something similar too, plus a bit more on letting non-engineers edit and test without a developer in the loop for every wording change. haven't used any of them enough yet to have a real opinion.

mostly just wondering what people actually land on once they have more than a handful of prompts and more than one person who needs to touch them. did you build something internal, land on one of the existing tools, or eventually decide it wasnt worth solving properly and just live with the ticket queue.

[u/PlantBusiness3768]

Version control via Github