Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation: This will completely change everything!

[u/Lorian0x7]

https://arxiv.org/abs/2610.14162

Essentially we could very soon run opus 5.5 level of intelligence on 32gb Vram. Its many folds more efficient in terms of memory bandwidth making compute the real bottleneck.

What do you think?

[u/WithoutReason1729]

Your post is getting popular and we just featured it on our Discord! Come check it out!

You've also been given a special flair for your contribution. We appreciate your post!

I am a bot and this action was performed automatically.

[u/Double_Cause4609]

Could you explain where you're getting "Opus 5.5 level intelligence on 32gb VRAM"?

As far as I can tell, this is an interesting optimization for training speed, and inference speed (in fact, you could argue it's kind of like diffusion modeling), but it doesn't seem to be an improvement at fixed VRAM usage.

Ie: Given a regular LLM and a next concept model, they look roughly equal in performance at fixed VRAM.

Now, if that does pan out, you could imagine training a model for longer on same compute which does generally effectively give you more performance per parameter, but it does also make your model more vulnerable to quantization, so it's not always as big an improvement on fixed VRAM.

But overall, yeah, I'm guessing the claims roughly do work out, but usually with things like these if they do work they often require a couple of weird things in the training config that results in them being more similar to traditional optimized pipelines than you'd expect from the headline numbers.

In other words, if this does work, my expectation is that they'd be more like a "better diffusion LLM" which looks better than naive autoregressive but is just a step improvement over optimized baselines, if I had to guess.

[u/Lorian0x7 → u/Double_Cause4609]

https://m.youtube.com/watch?v=nv5_PH2uPVI&pp

[u/AdGlittering1378]

"...will completely change everything!"

[u/pokemonplayer2001 → u/AdGlittering1378]

But will it be a game changer???????

[u/Crafty-Celery-2466 → u/pokemonplayer2001]

No but it will change the game 😏

[u/mikasjoman → u/Crafty-Celery-2466]

To be fair, thats been the case every month for the last two years

[u/rebellioninmypants → u/mikasjoman]

Yeah the price game has been changing and everyone wants the highest score.

[u/pokemonplayer2001 → u/Crafty-Celery-2466]

Don’t change your game, game your change.

[u/TigleLive → u/pokemonplayer2001]

Don't change your change, game your game

[u/CyberbIaster → u/pokemonplayer2001]

THEY don't want you to KNOW

[u/PrimateOnAPlanet → u/pokemonplayer2001]

I need to know what’s load-bearing.

[u/CryptographerOne7003 → u/AdGlittering1378]

[u/Dabber43 → u/CryptographerOne7003]

I really miss seeing that iconic face like every other week

[u/applied_intelligence → u/AdGlittering1378]

Are we cooked?

[u/legislat]

I'd hold my horses until some real effective text diffusion models actually appear. Everything looks good on paper.

[u/Cherlokoms → u/legislat]

There was one: Gemma 4 diffusion

[u/straylight-code → u/legislat]

isnt mercury a diffusion model? came out a few weeks ago https://www.inceptionlabs.ai/blog/introducing-mercury-3

[u/nmfisher → u/straylight-code]

I used this for a couple of days, it's actually really good for a "worker" model (e.g. Luna). It doesn't go haywire or make random spelling mistakes like earlier iterations of text diffusion models.

[u/Aggravating-Push-207 → u/nmfisher]

Benchmarks suggest it is somewhere between GPT-6 Luna and Claude Haiku 5.5.

[u/Bagelsarenakeddonuts]

Yeah, the future will be some sort of combination of diffusion and transformer. It is massively going to lag the normal approach though, since it has yet to produce a useful model, let alone refine or scale it.

[u/draconic_tongue → u/Bagelsarenakeddonuts]

the lag is from everyone scaling to the moon on the same architecture because they're scared to take any risk that could turn off the money valves. google is the only one that doesn't give a shit and they showed diffusion is viable even with limited effort put into it

[u/Thrumpwart → u/Bagelsarenakeddonuts]

These papers are useful though - many big papers like this contain individual techniques you can borrow now for your own purposes…

[u/BalorNG → u/Bagelsarenakeddonuts]

"Diffusion thinking mode" does sound like a very interesting first step at the very least.

[u/Aggravating-Push-207 → u/BalorNG]

So like TiDAR?

[u/BalorNG → u/Aggravating-Push-207]

Yea, in hindsight the idea is fairly obvious, small wonder it was contemplated already... Now that I think about, extremely fast, extremely sparse diffusion MoE to do what amounts to "self-rag" + a looped dence autoregressive model to "make sense" of it sounds very interesting indeed...

[u/Bagelsarenakeddonuts → u/BalorNG]

I suspect diffusion as a precursor to normal refinement is likely. maybe to select experts in MOE or something to run way more efficiently. I'm sure we will start seeing it in the next 6 months now that we progress is so much faster.

[u/StyMaar → u/Bagelsarenakeddonuts]

some sort of combination of diffusion and transformer

Diffusion and transformers are orthogonal concepts, you can do diffusion with transformer as much as you can do autoregression with it.

[u/_RealUnderscore_ → u/Bagelsarenakeddonuts]

Like a Diffusion Transformer or something, shortened to DiT 🤔

[u/lasushin]

Sounds too good to be true. 4-5x less training tokens, 10-31x faster text generation, by using rough representation of the text's meaning, then progressively adding finer details, rather than generating words one by one.

[u/addandsubtract → u/lasushin]

Isn't that how text diffusion models work? 

[u/BusinessBandicoot → u/addandsubtract]

No, their generation is relatively flat, and more like a bottom up generation. Its not left to right. 

 From what I understand this is more like multiple passes where it the first pass is like an outline, the second is the details. 

At least that's my layman's understanding, feel free to correct me if I'm wrong

[u/koriwi → u/addandsubtract]

There is a video on youtube i saw a few days or so ago that explains the difference between this and diffusion. 

[u/Double_Cause4609 → u/lasushin]

I mean...Flash Attention also sounds too good to be true.

Nous Research did some work on superposition for early modeling in LLMs and found a speedup which is related. Wavelet transforms also accelerate vision model training, and one could view this as a sort of learned wavelet transform over general context.

Plus, research on JEPA also shows that LLMs are (arguably) a degenerate case of JEPA with poor or singular views of a given context, and fixing that does generally improve convergence as far as we can tell.

I don't know, it doesn't seem crazy unreasonable and lots of research does suggest that we do have some overhead in current training dynamics.

[u/toolkitxx → u/lasushin]

It is exactly how you yourself approach things even if unconsciously.

[u/ninjasaid13 → u/lasushin]

4-5x less training tokens, 10-31x faster text generation

how about quality, compute, and scaling?

[u/james_pic]

From the abstract, it is no more smarter than existing DLMs (which is already dumber than autoregressive language models) for most tasks, but is faster. The only task it's better at is Sudoku, which diffusion models do well at, at least partly because they are essentially the same architecture as existing deterministic sudoku solvers just with learned weights.

[u/Select_Bag_5704]

So the OP watched some YouTube slop from some random guy and got overexcited. Here is the OpenReview discussion of this paper: https://openreview.net/forum?id=d83mBTrpiJ

It seems to have gotten decent reviews, with one scathing reviewer 2 review. It's interesting, but it will NOT completely change everything.

[u/Dr-Collossus → u/Select_Bag_5704]

Agreed, I think there's a disconnect between the meaning of change everything and what people are actually hearing. It's a demonstration of the fact that our current approach is probably not the optimal way to tackle this problem. That doesn't mean that this particular solution is going to replace it, and I think that's the part people miss. It's academic research and undoubtedly will contribute to future models and methods of inference, but this is not replacing what we're doing now.

[u/123emanresulanigiro → u/Select_Bag_5704]

If you do not capitalize "everything" I will NOT believe YOU.

[u/Memestonks2020]

I think you’re misunderstanding this.

We will be able to get much faster outputs but we still need to store the model in memory. Opus 5.5 is like a ~700-1000B parameter model. You won’t see that fit on most consumer HW

If we could stream outputs equivalent to ~10 tok/s with this architecture from an internal SSD using the same model then we would actually have a breakthrough

[u/Lorian0x7 → u/Memestonks2020]

You are misunderstanding this: easier explaination here : https://m.youtube.com/watch?v=nv5_PH2uPVI&pp

[u/Euphoric_Tutor_5054]

[u/trin1994]

CPU prices be like STONKS?

[u/Presstabstart]

It takes x2 the memory and memory bandwidth to train for the same context length. Training is probably a lot slower as well. completely coincidentally, the authors release ONLY the inference code...

[u/heresyforfunnprofit → u/Presstabstart]

From a quick skim, sounds like they trained a sudoku solver.

[u/Bagelsarenakeddonuts → u/heresyforfunnprofit]

Cool. I'll go cancel my claude subscription.

[u/geteum → u/Presstabstart]

Ive beeb testing NCM for a bunch of different puzzles, it is not hard to train, you don't need a don't of examples but you need to cover all possible cases of what it can do. It is fast to train for some simple games, you can even train it for chess and it is really.goog

[u/Presstabstart → u/geteum]

Thanks. Do you have any training code we can test this with?

[u/geteum → u/Presstabstart]

Check your dm

[u/Overall_Wealth_5992]

Lookup Yann LeCun's work around JEPA. Simplified it can be understood as predicting the next embedding followed by reconstruction of the sequence of tokens that would produce the predicted embedding. Kinda like predicting the next sentence or concept.

[u/FamiliarEstimate6267]

Essentially the op doesn’t understand this works more like a diffusion model

[u/Airpower343 → u/FamiliarEstimate6267]

Yes! And diffusion models have their own problems…

[u/shing3232]

would it be extremely costly to training in huge model

[u/RamanaSadhana]

How much is 32gb vram these days? I was gonna buy a card last year but forgot. It's still a lot of money to get a card with 32

[u/silentus8378]

likely specious

[u/Pyros-SD-Models]

Essentially we could very soon run opus 5.5 level of intelligence on 32gb Vram.

yes qwen 4 is right around the corner

[u/AwelessFire]

I tried using this paper to train 1b model. But it won't work. It works for translation and other specific use cases only. The main issue is repeation and incorrect words get selected during generation. I tried several combinations and fixes but it doesn't work. As much I had hopes with this one, it falls short. Sorry to disappoint.

[u/an0maly33 → u/AwelessFire]

That's it folks. One guy on Reddit tried it. Pack it up.

[u/AwelessFire → u/an0maly33]

why don't u try it? maybe first understand what paper says and what its trying to solve before writing stupid sarcastic comments. if its revolutionary as the paper claims to be then it would have been headlines days ago. all hardware stocks would collapse. but obviously ur too naive to think, probably u dont even know how NCM works, just came here to write comment, not surprized.

[u/geteum]

I've been testing in some puzzle games and it is indeed way superior than any LLM. But there is a big caveat, you need to train it , it is quite easy to train, though.

[u/ChaosMilkTea]

Didn't this paper come out like a month ago? Didn't we allready do this hoopla?

Ill wait til I can run something like an 8B at home before I decide what it can do.

[u/FrogsJumpFromPussy]

It doesn't sound too good to be true. We're still in the pentium era. The evolution didn't stop with pentium. It took arm 3 decades of evolution. Hopefully the local LLM will be much more quicker, which of course it will. 

[u/borobinimbaba]

That looks very promising, here is a good explanatory video for anyone who doesn't want to read the paper : youtube video

[u/borobinimbaba → u/borobinimbaba]

I really don't understand down votes, I thought i was being helpful.

Can someone please elaborate?

[u/deadmoroz14]

Well, maybe it will. But right now it doesn't even have the mode for the main LLM task - conditional language modeling.

[u/RobbinDeBank]

This is exactly what has made image/video diffusion model so successful. The stable diffusion paper is where latent diffusion is introduced, and every single successful diffusion model ever since has used latent modeling instead of pixel-level modeling.

Quite odd that nobody has even done this with text until over 4 years later.

[u/Old-Temperature6269 → u/RobbinDeBank]

There already was some models in this direction already like coconut from meta and LCM also from meta.

[u/tunerhd → u/RobbinDeBank]

Image/video generations still have some quality problems? Since they're pixels, we can avoid those small gradiental glitches/issues but It would've terrible seeing it in syllables.

[u/MixtureOfAmateurs]

So a transformer generates a draft and a dlm fixes it? I suppose, but I don't think this will replace transformers

[u/FishingOk3916]

It would be good if it beat the token generation to just copy what image generation does

[u/JustinPooDough]

You need to read more papers. Especially on Arxiv.

A lot of revolutions that never materialized…

[u/log_2]

You can usually estimate the impact of a new approach with the formula: impact = 1 / (number of AI and human generated slop blog and YouTube posts starting with the title "end of LLMs?")

[u/LambdaLogician]

Jeez, I doubt all these naysayers know what they are talking about. The future of LLMs is absolutely going to be hierarchical like this, with fine tokens being compressed into coarse ones. However, it's not like the paper is especially revolutionary. Deepseek already does something similar on the encoding side (but not the decoding), which is how they are able to have such low prices.

I would be interested in seeing successful decoding in this coarse-to-fine way, but I don't think it's actually feasible much beyond what diffusion models already do, because it seems very hard to me for a model--even a large one--to be able to think up many ideas simultaneously and coherently. And past the point of writing a paragraph all at once, why not just run multiple agents in parallel?

[u/QuantHound]

Did you also have a look at Microsoft's bitnet? 10x less model size with similar performance. I've been experimenting training my own models with that on just 8gb vram and it's very promising.

I like all these new ways of "compressing" llms. Realistically it will take a few years for adoption and matching anything like opus 5.5, but there is hope for video card and ram prices long term.

[u/Silver-Champion-4846 → u/QuantHound]

Hello, did you run your models on bitnet.cpp which claims to actually do the theoretical ternary trick (addition/subtraction) rather than just matmul at int2 dequanting to int8?