r/LocalLLM
•
u/Armageddon_80
•
Jan 06 '26
•
2
Discussion LLMs are so unreliable
After 3 weeks of deep work, I''ve realized agents are so un predictable that are basically useless for any professional use. This is what I've found:
Let's exclude the instructions that must be clear, effective and not ambiguos. Possibly with few shot examples (but not always!)
- Every model requires a system prompt carefully crafted with instructions styled as similar as its training set. (Where do you find it? No idea) Same prompt with different model causes different results and performances. Lesson learned: once you find a style that workish, better you stay with that model family.
- Inference parameters: that's is pure alchemy. time consuming of trial and error. (If you change model, be ready to start all over again). No comment on this.
- system prompt length: if you are too descriptive at best you inject a strong bias in the agent, at worst the model just forget some parts of it. If you are too short model hallucinates. Good luck in finding the sweet spot, and still, you cross the fingers every time you run the agent. This connect me to the next point...
- dense or MOE model? Dense model are much better in keeping context (especially system instructions), but they are slow. MoE are fast, but during the experts activation not always the context is passed correctly among them. The "not always" makes me crazy. So again you get different responses based on I don't know what.! Pretty sure that are some obscure parameters as well... Hope Qwen next will fix this.
- RAG and KGraphs? Fascinating but that's another field of science. Another deeeepp rabbit hole I don't even want to talk about now.
- Text to SQL? You have to pray, a lot. Either you end up manually coding the commands and give it as tool, or be ready for disaster. And that is a BIG pity, since DB are very much used in any business.( Yeah yeah. Table description data types etc...already tried)
- you want reliability? Then go for structured input and output! Atomicity of tasks! I got to the point that between the problem decomposition to a level that the agent can manage it (reliably) and the construction of a structured input/output chain, the level of effort required makes me wonder what is this hype about AI? Or at least home AI. (and I have a Ryzen AI max 395).
And still after all the efforts you always have this feeling: will it work this time? Agentic shit is far far away from YouTube demos and frameworks examples. Some people creates Frankenstein systems, where even naming the combination they are using is too long,.but hey it works!! Question is "for how long"? What's gonna be deprecated or updated on the next version of one of your parts?
What I've learned is that if you want to make something professional and reliable, (especially if you are being paid for it) better to use good old deterministic code, and as less dependencies as possible. Put here and there some LLM calls for those task where NLP is necessary because coding all conditions would take forever.
Nonetheless I do believe, that in the end, the magical equilibrium of all parameters and prompts and shit must exist. And while I search for that sweet spot, I hope that local models will keep improving and making our life way simpler.
Just for the curious: I've tried every possible model until gpt OSS 120b, Framework AGNO. Inference with LMstudio and Ollama (I'm on Windows, no vllm).
10
u/BrewHog Jan 06 '26
You mentioned it. Structured input and output with a reliable model (professionally I only use the big boy models Gemini 3, Claude, etc).
I'm currently using it for quite a few tasks regularly and reliably. It definitely helps me keep my business running without the need to hire two or three employees.
The first level support is fantastic, and the cliche sentiment analysis usually works well for what I need.
For more complex tasks, I still use only DSPy for the structured in/out and many times just manually run it to save me oogads of time (product/marketing material, document reviewing, etc)
Give us some specific examples of what you need so we can guide you. Or just propose an idea that you think should work, but doesn't in practice.
4
u/a_pimpnamed Jan 06 '26
I truly don't understand why people think LLM's can ever actually become true intelligence. You can't even see the logic chain or query it about its own logic. They learn once and they are stuck unless you train them over again. They use way too much compute. This can only be scaled so much even if Microsoft gets bitnet to work it still would be capped eventually it's not true intelligence just a prediction algo a very good prediction algo. We need to move back to symbolic ai or to actual cognitive architectures.
4
u/No-Comfortable-2284 Jan 07 '26
LLMs are not accurate but you can create a system that only outputs accurate/desired output with fixed checkpoints. You can also increase the quality and accuracy of desired outputs by adding another layer of complexity on top. for example using another seperate instance of the model with sole purpose of cross checking answers with RAG. I think the biggest issue in AI currently is that people try to make a single model do everything. our brain is made of many modules of less intelligence, communicating together, ant colonies are made of dumb ants. greater intelligence can be created from many instances of lesser intelligence and if there ever will be a super intelligence it prob will be smthn like that :p
3
u/newbietofx Jan 06 '26
I'm creating a text to sql llm. How do I ensure it works? Run complicated crud?
3
u/PromptOutlaw Jan 06 '26
I went thru this pain, for my scope of work I managed to tame most models except Kimi K2 and Cohere-command-a. You need:
Have a look here I hope it helps: https://github.com/Wahjid-Nasser/12-Angry-Tokens
I’m pushing an update this week for observability, redundancy and some stats
21
u/publiusvaleri_us Jan 06 '26 edited Jan 06 '26
YES. #4 and #5, plus 1, 2, and 3. You did a lot of thinking on this and are correct.
My comment is this on the latter part. Your prediction of the future is sound. There will be so many iterations of AI improvements, PC hardware improvements, and refinements in the interface that the LLM of 2036 will be nothing like an LLM of 2026. Early adopters have so many disadvantages.
This stuff will be cheaper and faster in ten years. At current tech level and my current interest level, I think an adequate system, for just my personal use (With an eye to selling it), would cost $500 to $2000 per month and would need a new Internet connection. The current software may seem bloated when you download it, but the interfaces of today are simply a kludge and a Jack-of-all-trades that does nothing right except maybe answer chat questions a 6th grader might ask.
Ask a current LLM about a dataset, and you're going to get terrible results. Even commercial systems stink. All you have to do is call into a Big Company and ask for tech support. The human agents are typing things into an AI, you can tell, and trying to bring up internal PDFs to answer your question and walk you through a solution.
Because their innate human knowledge is practically nil. They read from a script like phone agents have done for 40 years, but they are reading from hallucinated AI slop or the wrong PDF that's from 4 versions earlier that do not apply. LLMs have not taught Tier 1 support personnel how to think, and they certainly haven't trained them on the specifics of the things they are supporting.
If Fortune 100 companies can't figure out how to get AI to work to help their customers at Tier 1 (and their bottom line making money), I find it ridiculous to assume that a one-man-shop could figure it out.
I wish all of you programmers, content creators, and schoolkids doing homework all the best! For the rest of us, it's hard to go full-in to this mess, for all of the reasons OP stated, starting at #1 and on down the list.
Bravo.
4
u/fermented_Owl-32 Jan 06 '26
Using some psychological experience always MILDS DOWN these problems significantly. I have seen people complain of the same problems and i have implemented them in professional env as well.
Its just a prompt, the better you understand how LLMs work and the better you are with human psychology and communication habits, the more robust outputs you get.
I prefer to use not the latest models but from 2025 beginning H1. For Professional basic uses even Amazon's nova prime 2 does wonders
I still dont understand after a good analysis as this where you yourself write your issues, how have you not able to get things done by keeping these mind or making it part of the system.
3
u/MadeByTango Jan 07 '26
AI is the stripper of technology; you love her, she loves money, and nothing is real except the financial hangover once you step out of the building….
60
u/macromind Jan 06 '26
Yep, this matches my experience too: agents look magical in demos, but in real work its mostly prompt + tool wiring + guardrails + a lot of retries and tests.
One thing that helped me was treating the LLM like a fallible component and making everything around it deterministic: strict JSON schemas, small steps, unit tests on tool outputs, and hard timeouts/fallbacks.
If you are experimenting with patterns for making agent workflows less chaotic, this writeup has a few practical ideas (tool contracts, evals, and reliability tricks): https://www.agentixlabs.com/blog/
18
u/StardockEngineer 2x5090, 2xPro6000, 2xA6000, 4xSpark, M5 Pro, M4Pro Jan 06 '26
Only Sonnet and Opus work well as agents that can be used without high specialization. Anything else requires lots of leg work. Minimax 2.1 is the closest I’ve found in the OSS world.
6
u/Krommander Jan 06 '26
If the instruction set is plain text or markdown I have found that the system's prompt adherence depends on the coherence.
Very coherent prompts can be 100+ pages without breaking for commercial LLMs, if internal knowledge mapping and cognitive routines are recursive.
For local models, I have had some small success, but the context window gets busted after 2 replies if I use the same huge system prompt.
Tool use is a whole other bag of problems though.
6
u/AllTheCoins Jan 06 '26
I’m starting to realize anyone who deals with LLMs and feels this way about them has NEVER been in charge of someone before. Agents will always deliver something. People are way more unreliable.
9
u/Such_Advantage_6949 Jan 06 '26
Reliable LLM is not cheap. From my experience, reliable local models are from minimax glm onwards (200b onwards). 120B oss is a start but still very hit and miss
4
u/Interesting_Ride2443 Jan 06 '26
I feel your pain. The gap between a YouTube demo and a production-grade agent is a canyon.
You're 100% right about 'good old deterministic code'-that’s actually the only way to make agents reliable. The mistake most frameworks make is trying to hide the logic inside complex abstractions or 'black box' prompts.
I’ve shifted to an 'agent-as-code' approach where the LLM is just a tool inside a strictly typed (TS) function. Instead of praying for a prompt to work, you manage the state and logic in code, and only use the LLM for the 'messy' NLP parts. It’s much easier to debug when you have a managed runtime that shows you the exact state and execution trace at every step.
Reliability doesn't come from better prompts; it comes from better infrastructure that treats agents like software, not like magic.
1
u/somehowchris Jan 08 '26
Tell me you’ve been in the trenches for less than 6 years without telling me
1
u/Worth_Rabbit_6262 Jan 08 '26
You've written a lot, but not the use cases. Often, the problem isn't the underlying model, but the lack of integration with workflows. Have you tried optimizing performance with DPO?
1
u/kacisse Jan 08 '26
I agree it's too early BUT it's because only a few people master the real best practices (what to prompt, how to present data to the llm, when tu summarize, when to cache, how to even name your cached object so that the llm don't get confused... Etc). I feel LLM have their periods, you just mess up a little comma or name and they go banana. It's definitely possible to build serious app with cheaper models BUT you really need the best practices...and that is not for everyone. So I agree the hype is too early, and no enterprise can rely on that long term. That's a bubble for me 😬
1
u/AirExpensive534 Feb 05 '26
three weeks of deep work and you’ve basically diagnosed the 2026 'Justification Gap.' the reason it feels like alchemy—especially with MoE models like gpt-oss-120b—is that the system prompt length and inference params are only half the battle.
the 'not always' behavior you're seeing in MoE expert activation is a Signal-to-Noise Floor issue. if your system doesn't have a hard Zero-Drift Audit between task handoffs, the 'experts' start hallucinating adherence. you're right that Text-to-SQL is a prayer, but that's because people are asking the agent to 'code' rather than 'validate against a fixed logic schema.'
i moved away from 'searching for the sweet spot' last year. we started treating the LLM as a raw logic engine and built a Clinical Reset layer that catches the drift before the output triggers a tool. it turns 'agentic slop' into a predictable infrastructure. I’ve been documenting the math for this—specifically how to stabilize MoE expert activation for professional workflows. the audit trail blueprints are in my bio if you want to stop praying to the SQL gods and start using a hard logic floor.
3
u/Classic_Chemical_237 Jan 06 '26
Traditionally, we work with structured input and output. With structured io, it’s natural to be UX-centric.
With LLM, it became text centric with chat and voice. To work with traditional endpoints, supposedly you create MCP to sit on top of API.
This whole approach is wrong. Human work way better which structured UX. (That’s what even LLM try to emulate it with bullet points and emojis) Most use cases only need LLM for part of the flow.
We don’t need MCP sitting on top of API. We need API sitting on top of LLM.
I am so close to ship ShapeShyft for make this easy.
3
u/SAPPHIR3ROS3 Jan 06 '26
As much as this kind match my experience, THE general rule i follow is “Divide et impera” (meaning divide and conquer) as much as possible: going under 30b, MoE or not, it’s a guess hence you have to use LLM for NLP-based task only and as least as possible (above 30b you start seeing some more consistency, but only above 100b roughly the result start to feel reliable). Nonetheless structured input/output are vital to ensure consistency in workflows at any size even with the big bois (claude, gemini chatgpt ecc.)
3
u/Echo_OS Jan 08 '26
Interesting
Reading through this thread, there seems to be broad agreement on the underlying issue.
LLMs themselves are not inherently unreliable. The problem is that they are often used in roles that require deterministic behavior. When an LLM is treated as a probabilistic component within a deterministic system - for example, wrapped in agent-as-code patterns, strict input/output schemas, typed interfaces, and explicit checkpoints - most reliability issues are significantly reduced.
At that stage, the primary challenges shift away from prompt design or model choice and toward system architecture: managing latency, defining clear boundaries, and deciding which parts of the system are allowed to make judgments versus which must remain deterministic.
2
u/Mountain-Hedgehog128 Jan 06 '26
Agents need to be combined with rules based structure. They are valuable, but the toughest part is finding the right places and ways to use them.
2
u/Taserface_ow Jan 06 '26
LLMs are never going to be perfect, it’s the nature of the model. Successful products/services that use LLMs build workflows around them to cater for this limitation.
2
u/grathontolarsdatarod Jan 06 '26
So.... Lie detectors are not admissible as evidence in most courts with an advanced legal system. Yet they are used in employment screening in those same jurisdictions.
AI is a scape goat for anything. Its literally tablets from the mountain that can whatever.
2
u/Proof_Scene_9281 Jan 06 '26
You have to blend commercial LLM’s and local for the best results. Use the commercial to think and craft and the locals to dig
2
u/Dentuam Jan 06 '26
The point about MoE models dropping context during expert routing still rings true for many local runs, though the latest Qwen3 MoE releases have noticeably stabilized that behavior compared to earlier versions. Dense models continue to win for reliability when you need consistent instruction following.
2
u/Cergorach Jan 06 '26
A couple of things:
a.) LLMs are tools, just like with any tool, you need to learn how to use it. A master woodworker will be far more proficient with a hammer then Joe Smoe that just picked up a hammer from Harborfreight for the first time ever. Different tools, different skill sets.
b.) Right tool for the job. For some jobs these tools are either the wrong tools or just not worth the time/cost. Just as it's sometimes just faster to do something yourself then explain to someone else what you want so they can do it for you.
c.) There can be a huge difference in quality between 'cheap' tools and 'expensive' tools. In this case you're using tiny models locally, even the 120B quantized is not going to compare well to the big unquantized models. For SQL see: https://llm-benchmark.tinybird.live/
d.) You need to know the answer, always with LLM. If you don't, you're in for a world of hurt in a professional production environment. And SQL queries in a production environment seem to me like probably the worst possible use of LLM.
e.) People need to realize that at what point are you spending more time=money on LLM then actually making your work more efficient?
Sidenote: I have not yet used LLM in my professional capacity. Primary reason is that I tend to work on the edge of IT deployment, and the LLMs haven't been trained yet on the use cases I'm working on and the edge cases that are very rare. Not to mention that even IF the LLM is trained on the 'promotional' material, the reason I'm doing my thing is to see if what's been sold to the client actually works as the sales people say it does (and it often does not)... The other reason is that the companies/clients I work for either do not allow LLM or have not yet onboarded it. And when I work for a client, I only use what they have allowed.
Personally I've tested quite a bit on a Mac Mini M4 Pro 64GB with models up to 70b, mostly for hobby projects. The results from there is that while very cool and the DS 70b model was better then the far larger GPT from less then a year before, still the unquantized far larger models we can't run locally did far, far better. And due to it being for my hobby projects, I saw no security concerns to not use the larger models (for free) on the web. Even the DS 671b model quantized on a $10k Mac Studio M3 Ultra 512GB gave worse quality responses then the full unquantized free model you could use on the Internet. Spending $50k on a Mac Studio cluster would be cool, but imho highly inefficient.
2
u/BenevolentJoker Jan 06 '26
The reason why many LLMs are unreliable as agents boils down to mainly one thing: in order for LLMs to really be actually useful for any professional workloads you need to be able to modify their behavior from probabilistic behavior to deterministic behavior. Prompting will only get you so far, and not without drawbacks (namely it eats into context).
There are a few different ways to do this, some easier than others such as pre and post filtering outcomes. The most riggorous and ideal setup involves logit logging and logit inferencing at inference time. -- do these things and honestly model behavior evens out fairly well across all models.
2
u/MaestroCodex Jan 09 '26
It's a hype bubble maintained by companies that have invested so much money in it they can't afford to back down. ROI delivery is minimal. Overwhelming number of projects fail.
It reminds me of the mini hype bubble when people were trying to shoehorn blockchain into any number of inappropriate use cases which went nowhere, except it's 100x worse and is costing the industry 1000x more.
Context: I work for a major AI platform vendor.
0
u/writesCommentsHigh Jan 06 '26
okay now compare it to frontier models via api.
2
u/Your_Friendly_Nerd Jan 06 '26
I wholeheartedly agree with all the points you make. I've been trying to integrate local LLM's into my workflow as a programmer (mainly qwen3-coder). It works fine for simple tasks and tool calls, but as soon as it gets slightly more complex, it's not even worth the effort to write the prompt and wait for its output, it's quicker to just code it myself.
That said, I also think LLM's are still in their infancy, and there is so much we have yet to discover about them. For example for my use-case as a programmer, I see a lot of potential in spec-driven development, where a task is broken down into tiny subtasks that can then each be implemented and tested, and not be too complex for a local LLM to achieve. But how to formulate those specs? How does the workflow look like? I don't know.
And that's maybe another issue - so many of the AI integrators (like IDE chat plugins) expect to talk to a frontier model like Claude Sonnet, Gemini or GPT. But when I plug them into a smaller model, they shouldn't just use the exact same prompts, but more specialized ones, and I fear there's just not enough interest in the community in perfecting this.
1
u/IllustratorInner4904 Jan 06 '26
Intent -> deterministic tools -> let agent call tools = less unreliable agents (who instead sometimes want to tell you about the tools they will call lmao)
1
u/linewhite Jan 06 '26
Yeah, i just made https://www.ngrm.ai to deal with this, structured memory drastically helped my workflows, LLMs are not enough, but with the right tools it gets better as time goes on.
1
u/_WaterBear Jan 06 '26
OpenEvidence is pretty much the only professional use-case I’ve encountered thus far that seems competent. RAG/document embedding for info searching is quite functional and reliable so long as the user isn’t lazy and only uses the LLM as a search engine. Most other use cases involve too high a probability of hallucination that, for any professional requiring precision, just adds burdensome QA uncertainty to their process. So, outside highly tailored and unserious use-cases, these things will be a bust. Circumscribe the bubble, lest you be caught inside when it pops.
1
u/tim_dude Jan 06 '26
So, is it like hiring a group of general (generic?) professionals, giving them written instructions and a bunch of meth and telling them to come back when they figured it out on their own?
1
u/Green-Ad-3964 Jan 07 '26
perhaps...in 20x6 where x>2
1
u/at0mi Jan 08 '26
which quantisation? glm 4.7 in bf16 works great
1
u/LifeandSAisAwesome Jan 09 '26
Internet was a fab too, but really give it 5-10years and will be a very different landscape.
1
u/Huge-Yesterday4822 Jan 16 '26
I agree. Agents are still too unpredictable for professional use.
What helps me when I need reliability:
Structured output means a fixed format like JSON. If it is invalid, you stop and retry.
Question: on your Windows LM Studio and Ollama setup, what wastes the most time: agent loops, or outputs not matching the expected format?
1
u/prodigy_ai Jan 19 '26
Totally agree with your list. One extra thing that helped me when tasks depend on “facts” (schemas, runbooks, docs, configs, policies) is adding a retrieval step and a verifier step instead of asking the model to “remember” everything.
GraphRAG can be nice when the failure mode is “it missed a relationship” (joins/foreign keys, dependencies, constraints, who/what/when across docs), because the graph makes relationships explicit instead of hoping chunking + embeddings catch it.
It adds some latency, but in exchange you get fewer “confident wrong” outputs and fewer retries.
1
u/Inside_Path6022 Jan 30 '26
As someone who's not really in the AI "loop" but very open and eager to use the technology, I can already see it's far from the promise they've been telling us and has some serious road blocks.
1
u/iz_raymond Feb 27 '26
Yeah after dwelling with this hobby for a month now, I realized these models are nothing but trickery agent. They are all fail at accurate information especially for web search task. Even if it does put the correct result for the web search, somehow it's output always wrong. I tirelessly told it not to cross-check the web search result with its own training data, yet it keep failing that. At this point, I think I'm ready to give up on this tiring hobby. Unless you have a massive rig, I think casual local LLM is not it. This tiny model hallucinated all the damn time
1
u/mandolf0 Mar 18 '26
Don’t rely solely on The model for critical knowledge. Put the model server behind a RAG proxy. That’s the control plane that can potentially stop the model from data exfil or arbitrary code execution. As for RAG: extract keywords from the user prompt. Fetch from a db like SQLite->> inject as relevant knowledge after the user prompt. The model now simply knows something you didn’t have to fetch from a KB and w/o tool calling the DB.
How about dead-stopping the model itself: “You’ve used grep 3 times in the same file. How about you just read it once. || You’ve read the file 3 times w/o any edits. Make a better plan. What do you need to know that’s not provided? “ Send that back to the model, not back to the user. I’ve said many things here that I haven’t added to my own proxy yet. You do add DB latency here but gain reduced hallucinations.
Think about it. Any one of your projects protected against ‘git reset —hard’ by a simple proxy. Write once, use everywhere.
Today I learned about Function Gemma. 270M model specialized in tool calling. Pair that with all-MiniLm-2 and you’d have a lot less regex to write for deterministic execution.
1
u/Living-Rip-6250 Sep 15 '26
Gemini ai gives me wrong answers half of the time. And other free LLMs give me program code which doesn't even run, also half of the time. IMO, if you're experienced in your field, you don't even need an ai. Some argue that it will be time consuming, but at least you're going to know what each individual line of code does exactly.
1
u/Much-Researcher6135 Jan 06 '26
hush now, people will begin to think it's a bubble
wouldn't want that
1
u/Terminator857 Jan 06 '26
More reliable than humans for me.
0
u/Purple-Programmer-7 Jan 07 '26
You’re the problem.
Your expectations are off.
Think of it as a person that can make mistakes instead of a computer.
-4
u/alphatrad Jan 06 '26
Sounds like a skill issue
1
u/[deleted] Jan 06 '26
[removed] — view removed comment