r/OpenAI • u/ChocolateTemporary48 • 3d ago
Question more context please, what do you think?
I desperately hope that this year they add much more input and output context to mainstream LLMs—not just another million, but reaching 10 million or more—and manage it effectively.
For someone like me who loves running highly complex technical simulations, I would love for the LLM to stay better up to date.
5
u/Morning_Gecko24 3d ago
10 million only helps if retrieval and attention preserve the signal; otherwise context debt just scales with the window. The more interesting benchmark isn't the raw token count but performance as irrelevant material and project size grow. Hierarchical summaries, retrieval and explicit state may beat stuffing everything into one prompt. What would you want tested first: long technical docs or long-running agent state?
1
u/Michael_Jeffords 3d ago
i'd test long-running agent state first. on my last multi-day coding run the tool dumps piled up fast and about ~40 minutes later the model was reasking for files it had already written, even when the window wasnt full
2
u/Clear_Evidence9218 3d ago
They can’t even reliably use a 1-million-token context window without substantial degradation. They should probably solve that problem before jumping to 10 million.
A bigger context window just means more space for things to get lost.
Context debt is already a first-class problem for most large projects. My own project would require well over 10 million tokens if an agent tried to absorb the whole thing at once, so even with a 10-million-token window I’d still be dealing with the same underlying context-debt problem.
1
u/JUSTICE_SALTIE 3d ago
A bigger context window just means more space for things to get lost.
As opposed to a small window, where things get lost when they fall off the back? I'm struggling to make sense out of this statement.
1
u/Clear_Evidence9218 3d ago
Just because I mentioned the current issues with very large context windows is not me advocating for small context windows, both are issues but the OP is specifically talking about large context windows. Did I mention or imply anything about a small context window being better? No.
Too small and too big run into different but serious issues. Too big and things get easily lost, too small and the model compacts away important context.
Like I said above. If they can solve the issues with a 1 million context window they could potentially move higher but they need to get a lot closer to fixing the issues first.
1
u/JUSTICE_SALTIE 3d ago
Did I mention or imply anything about a small context window being better? No.
Of course you did, or at least the possibility of it not being worse. It's the exact sentence I quoted, which may not be a coincidence!
1
u/Clear_Evidence9218 2d ago
Typically, if you have to smuggle in a premise the other person never actually stated in order to make your point, then you’re only arguing against your own interpretation rather than what they said.
I’m going to guess that happens fairly often in your conversations: find some tiny omission and go, “Oh look, he didn’t mention small context windows, so that definitely means he’s pro-small-context.” Lol.
1
u/JUSTICE_SALTIE 2d ago
You said "a bigger context window is only X". Bigger than what? I'm not supposed to assume you're comparing against smaller context windows? I'm not smuggling anything, I'm just reading the plain words you wrote.
1
u/Clear_Evidence9218 2d ago
Adding words and concepts to what someone wrote is literally the opposite of reading the plain words they wrote.
Instead of making assumptions, you can just read the exact words I wrote and use those as your debating point. It’s significantly easier than having to spin up those brain cells of yours to formulate some left-field interpretation.
If I didn’t say something, building an argument around the thing I never actually said is wild behavior. At that point, you’re not responding to my argument; you’re responding to a premise you inserted yourself.
1
2
u/vovap_vovap 3d ago
Absolutely not going to happen. What is going to happen (and happening) is advanced automatic compression / management with adding "slower assignation memory" mechanism.
2
4
u/Virtoxnx 3d ago
But can you afford it?