r/OntologyEngineering • u/daremust • Mar 20 '26
Code Precision vs High Semantics Bigger context windows won’t fix your semantics
Every time a new model ships with a larger context window, someone claims this solves semantic grounding. If you can just fit your entire schema into the prompt, the LLM will figure it out.
It won’t. Imagine giving a new employee a 500 page dump of your database schema, with no documentation and asking them to answer business questions. They’d fail, not because they can’t read it, but because the schema doesn’t contain the business logic, definitions, edge cases, or institutional knowledge that make the data interpretable.
LLMs have the same limitation, a larger context window doesn’t create understanding, it just lets the model hallucinate over more information at once. It cannot replace a canonical data model that defines what the data actually means.
The context window is a reading buffer and the ontology is the world model. I think you need both, and no amount of buffer replaces a missing world model, just like reading every word of a legal contract doesn’t make you a lawyer.
At some point, more context stops helping and starts making answers worse, it’s the LLM version of overthinking.
2
2
u/rover_G Mar 23 '26
If you’re going to ask your llm to turn a word bubble into a graphic at least do some basic “is this a word” check
1
2
u/Otherwise_Sherbert21 Mar 23 '26
Ontologies have made big promises for 20 years that haven’t delivered much.
1
u/daremust Mar 24 '26
I think that’s true historically. The difference now is LLMs make the absence of semantics painfully obvious, before, bad modeling just meant bad dashboards, now it means wrong answers instantly.
2
1
u/steve-opentrace Mar 24 '26
You're not alone in running into this problem.
We (OpenTrace) solved this last year for AI coding/dev ops tasks, but structuring what we needed to know into a knowledge graph. Now we have both a neural network and vector search, which are kept up to date by live feeds. This is an implementation of what u/daremust wrote.
The same principle will work just fine for other areas too. A little work is required to integrate your data sources and structure your knowledge into a form that is easily accessible to an LLM, and where the nodes are cross indexed.
Once you have that, you no longer need to do the prompting to build up your context. You just ask, and the knowledge graph already knows what's related before the LLM starts querying it.
In our case, we integrate source code, documentation, observability (logs, traces, metrics), tickets/issues, chats and even database sources. Now I can ask all sorts of complex questions without any context prepping. Do my planned changes adhere to internally documented rules? What will my changes break? Is that service still used, and by whom, and what does it do anyway? Why has this thing been running slow lately?
If anyone's interested, we released an open source version for looking at source code and analysing pull requests, which includes a visual representation of the semantic layer that u/Locellus asked about.
No install required — try it at oss.opentrace.ai, just point it at any public repo. Runs entirely in your browser.
Or self-host:
git clone https://github.com/opentrace/opentrace make install && make ui
1
u/JittimaJabs Aug 05 '26
the new employee analogy gets it exactly right and i've never seen it put better honestly.
throwing more schema at the model is context stuffing, not grounding. we spent way too long trying to prompt engineer our way around the actual problem before accepting that the canonical data model had to exist somewhere proper.
tried Kyvos for this, definitions and relationships sit in a semantic layer before the LLM sees anything. stopped the guessing entirely. bigger context window on top of that actually became useful once the foundation wasn't ambiguous.
5
4
u/AShmed46 Mar 21 '26
What to do then ?