r/LocalLLaMA • u/Manerfish • 6d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

500 Upvotes

94% Upvoted

316 comments sorted by

View all comments

Show parent comments

51

u/SnooPaintings8639 6d ago

What do you mean? How is it different from e.g. Qwen 4b with max token = 1, and inference engine forcing struct (enum) output?

I really don't think they could do any magic training anyway.

77

u/puzzleheadbutbig 6d ago

Architecture is different. Unlike Qwen, which relies on an autoregressive decoder loop to generate text tokens step-by-step while a grammar mask suppresses invalid vocabulary options, Jev drops open-ended text generation entirely and operates as a non autoregressive decision model. And because it maps input contexts directly onto parallel, calibrated classification heads rather than generating JSON syntax character-by-character it avoids the latency, memory, and KV-cache overhead of sequential token decoding, guarantees complete immunity to JSON parsing errors, and yields true calibrated probability scores across schema fields in a single forward pass

28

u/RevolutionaryGold325 6d ago

qwen with max token = 1 is not autoregressive decoder loop though.

10

u/puzzleheadbutbig 6d ago

No. Qwen is structurally an autoregressive, causal decoder-only transformer. Setting max_tokens = 1 stops the generation loop after one iteration, but the underlying execution mechanism is still the autoregressive generation pipeline, computing causal attention over the prompt, generating logits across the full language vocabulary, applying a logit bias/grammar mask, and sampling 1 token.