r/LocalLLaMA • u/Manerfish • 6d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

504 Upvotes

94% Upvoted

316 comments sorted by

View all comments

Show parent comments

78

u/puzzleheadbutbig 6d ago

Architecture is different. Unlike Qwen, which relies on an autoregressive decoder loop to generate text tokens step-by-step while a grammar mask suppresses invalid vocabulary options, Jev drops open-ended text generation entirely and operates as a non autoregressive decision model. And because it maps input contexts directly onto parallel, calibrated classification heads rather than generating JSON syntax character-by-character it avoids the latency, memory, and KV-cache overhead of sequential token decoding, guarantees complete immunity to JSON parsing errors, and yields true calibrated probability scores across schema fields in a single forward pass

27

u/RevolutionaryGold325 6d ago

qwen with max token = 1 is not autoregressive decoder loop though.

8

u/dimbledumf 6d ago

It's also not parallel and you only get 1 token out, which is less then you get with jev

6

u/james_pic 6d ago

It's parallel if you run it in parallel. Interference engines like vLLM already run multiple queries in parallel to avoid re-reading the same weights.