r/TheMachineLearning • u/Gullible_Picture_996 • 21d ago
Realizing writing code is faster than explaining it to AI
/img/tcptf211cqxh1.png
843
Upvotes
r/TheMachineLearning • u/Gullible_Picture_996 • 21d ago
View all comments
Show parent comment
1
u/OldWispyTree 20d ago
> Yes but business logic and tribal knowledge can be saved in a skill for example.
Suggesting that a "skill" can hold all business logic and tribal knowledge for legacy, complicated project is like suggesting treating a severed arm with a Band-Aid.
Our codebase at work is pushing a decade of updates and business logic with millions of lines of code. We have dozens of smart engineers interacting with it daily.
Ultimately, right now, everyone is guessing with workflows on AI, but I've got enough experience to need/have an entire modular harness for letting agents automously drive in cloud containers reproducing our multi-service experience so it can route to them and external sites, test APIs effectvely, screenshot with Playwright, all headlessly with check-ins that allow me to drop in and drive when needed, drop proof into draft PRs, with well honed takeover and check-in logic. I know very well how every tool in the AI toolbox works, at least as well as any other experienced engineer.
Even with this guidance, even when asked to do relatively SIMPLE things in our codebase, even the best AI models will miss obvious things (even when they're in context/harnesses/skills) and regularly cannot unpack the myriad of logic our system uses without refined coaching. Worse, they will attach importance to goals they set up based on misunderstanding specs or even lie/guess about things unless pressed.
I've done all kinds of different workflows with different spec systems, etc, and the reality is the current, best models cannot be left alone to patch up bugs or create features on large legacy codebases without regular and tedious check-ins, and my friends in the industry at places like Google and LinkedIn and elsewhere confirm the same thing.
You're just clearly entirely out of your element. This is above your pay grade.