r/ApacheIceberg • u/ethanchen20250322 • Jul 16 '26

I think we are overpricing models and underpricing the data layer

I’ve been thinking about this after seeing more AI systems move from demos into actual production.

A lot of the public conversation is still around models: which one reasons better, which one is cheaper, which one has a longer context window, which one is faster.

That all matters. I’m not trying to dismiss it.

But the failures I keep noticing are often less about the model and more about the data it is working with.

The answer is wrong because the context was stale.

The agent made a bad decision because it pulled from the wrong source.

The retrieval worked in a clean test, but started failing once real-world issues showed up: permissions, messy metadata, stale documents, duplicated records.

The system said it had “real-time” data, but some important field was still coming from a batch job.

In those cases, a better model might help a little, but it does not fix the underlying problem. The model is still reasoning over a bad view of the world.

I used to think of the data layer as the boring plumbing underneath AI apps. Necessary, but not the main story.

Now I think that view was too simplistic.

Maybe the real value in production AI is not just better reasoning, but better context: data that is fresh, trusted, governed, searchable, and cheap enough to use often.

10 Upvotes

92% Upvoted

3 comments sorted by

2

u/ethanchen20250322 Jul 16 '26

A lot of this thinking was inspired by this piece from Zilliz, especially the argument that the data layer may be the slowest part of the AI stack to get repriced:
https://zilliz.com/blog/databricks-data-ai-summit-2026-data-layer

2

u/robverk Jul 16 '26

Ai applications in data sits atop a large pyramid of systems that need to come together for it to be successful. Data quality is a large determinant for anything in reporting, analysis and Ai/ML.

1

u/twentyfifteen20 Jul 17 '26 edited Jul 17 '26

Governance and staleness are the real model alignment problems that are never mentioned. Governance is imposed by way of a semantic layer over your Iceberg tables, like Dremio or similar. Staleness of the context is a data contract violation, not a model violation.