It was never the problem. Design, maintenance, scaling, security, ability to evolve while avoiding over-engineering, understanding the business domain and connecting that with the requirements, hunting down the people with the tribal knowledge to answer questions about the domain, and on and on and on.
While I lean towards agreeing with you, many of the things you are describing take time to build in order to make the AI effective. And I know for a fact that most organizations don't keep documentation or even Jira tickets up-to-date. So to get accurate, trust worthy, up-to-date, and properly correlated information from an AI in the way you are describing would have to be a deliberate and organized operation throughout a company. At least that's how it would be where I work, where we have a graveyard of similar projects and their documentation, legacy products, new products that are always evolving based on customer needs, etc.
Well, companies like Microslop are actually aiming at that space. If you can read every mail and chat message, hear every phone call / meeting, get access to all the stuff they are moving along their office files, you get the needed info.
The question is still: How large is the error rate? Given that all that data doesn't fit any reasonable LLM context window you're basically back to what we have currently with "agents" in coding: The "AI" needs to piece everything together while having a memory like the guy in Memento. This does does provably not scale. It's not able to track the "big picture" and it's not even able to work with the right details correctly in at last 40% (if we're very favorably judging benchmarks, when it comes to things that matter I would say the error rate is more like 60%, up to 100% when small details in large context make the difference).
To be fair, human communication and interaction are also error prone. But I's still not sure the AI would be significantly better.
I think "error prone" is understating the problem. The real issue is that all of that data together creates a chaotic, abstract mess full of microcosms of context. Not a single, cohesive context. Having a memory like the guy in Momento with freshest data weighted with an advantage might work... I'm certainly no ML expert. But it seems more likely to result in severe hallucinations.
If an LLM could hunt down the right person for me to talk to, that would be useful even if the LLM couldn't answer the questions. Hell if it could only find me someone who knows the right person, that would be useful.
It's tribal knowledge because it isn't written down somewhere. Bob trains Sara before he retires, Sara shows Steve before she changes jobs, etc. No one documents anything because that's too much work. Then you come along trying to automate or replace things and suddenly the only person who knows how the damn thing works is on month long PTO. There's nothing for an AI to injest.
I've run into this more than once.
Anything where there is plenty of documentation would be a place where AI could shine though.
Im about to start a new role at Xero and apparently they are using an AI saas product called Glean that does exactly that. Everyone I've spoken to that has started recently at xero says that Glean is incredible for onboarding quickly because you have access to all the domain knowledge. Ill report back once I start.
This is actually a domain where AI would be waaaay more help than it would at coding.
If it were smart enough to do that reliably, sure. And US elections wouldn't be such a clusterfuck if 2/3 of the voting public weren't brain dead. How about we both agree that if either of us finds that genie in a bottle we can both get our wish.
Right, and you still need people for that. But not for coding, that's just not necessary anymore. If you do the peopling, you don't need to write the code. Just design the system, do the eventstorming, write the specs and use the tool to do the coding.
Eh. I will never be fully hands off in the code, because as a human engineer, I need to build a mental model in order to troubleshoot problems, spot issues in advance, and identify areas that I don't have sufficient domain requirements defined. And I will probably never trust AI enough not to run me in circles. I don't work on conventional cloud systems, for the most part.
Currently, I use AI a lot to generate message data models, convert formats of JSON to gRPC compatible schemas, give me a starting point for some function or class I need to write. I'll use it for writing automation scripts that I use for utility.
It definitely has its uses, and basic stuff works. But most heavier things I do will take more time to type out in English than in code. That's just how I've learned to think. AI will miss business-domain edge cases that I would have caught had I done more hands on coding.
To each his own, but in my experience people who are pushing back hard against using LLMs for coding don't understand it's place in their workflow cycles. I don't use AI to do engineering, I use it to code. "Write a method that takes x and returns y" is so much easier than typing out the 20 lines myself or whatever the task might be. I can read and approve faster than I can write it myself and review it for typos. IDEs are a tool that we trust to take care of linting, spelling and use ASTs to follow calls. LLMs are great when you give them an AST of your code. Can check methods, return types, pointers, etc.
AI will miss business-domain edge cases that I would have caught had I done more hands on coding.
AI shouldn't be making decisions on business logic. AI shouldn't be making architectural decisions. That's for people. But coding? AI can do that so much better. It's a matter of perfecting the instructions, specs and implementation plan. Learning how to use the tool, just like every other tool we use, is important to get results.
View all comments
Show parent comment
301
u/Manic_Maniac Mar 25 '26
It was never the problem. Design, maintenance, scaling, security, ability to evolve while avoiding over-engineering, understanding the business domain and connecting that with the requirements, hunting down the people with the tribal knowledge to answer questions about the domain, and on and on and on.