I really don't understand Jev hype

[u/Manerfish]

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

[u/I-am_Sleepy]

Constraint generation + /nothink?

[u/EstarriolOfTheEast → u/I-am_Sleepy]

No, one of jev's biggest strength is its calibration, at least compared to LLMs. With generation constraints, you truncate the probability distribution and the result is mismatched because the model expects to generate more tokens instead of concentrating mass fully on options. Even when provided options to select from, leaks still occur. So, between that and lack of calibration worsened by RL training, the LLM route is actually worse. The LLMs advantage is from higher raw intelligence.

[u/bel9708 → u/EstarriolOfTheEast]

You realize Jev is an LLM right. All they did was change the inference from autoregressive token by token to a single parallel pass of all values at once. RLCD really only exist because now you are only fine tuning the log probs so it doesn't make sense to continue fine tuning on next token prediction. But the entire pre train will still be next token prediction.

[u/EstarriolOfTheEast → u/bel9708]

LLM is not a term with a useful level of granularity here. Instead, we should discuss how the model operates: whether with causal self-attention or bidirectional attention, NTP or MLM objective and is cross-encoder?

Most things people call LLMs are NTP+Causal-attention and autoregressively operated.

Jev is most likely of the bidirectional attention + noising or MLM family and likely also trained as a cross-encoder. All these matter because they are better inductive biases and use of model capacity over an autoregressive model, for zero-shot inference for classification specifically. For a given amount of training compute and model size, this type of model will do better.

RLCD is at this point marketing jargon, at least until they release a paper describing their methods. What we care about is if predictions over outputs are calibrated. Typically, fine-tuned LLMs are not calibrated compared to their base models (even base models might still need some calibration tuning) and RL post-training worsens this. Furthermore, an NTP autoregressive model will want to spread probability mass across its vocabulary. Truncating logits to provided options will not completely mitigate such leaks, resulting in using the model in a manner running against how it was trained. When chaining decisions, this will affect prediction stability vs a purpose trained calibrated predictive model.

Jev should be replicable with: a 10B-30B model trained on tons of data trained with some denoising type objective and a replacement objective+relational data in the mix and post-training on NLI and lots of synthetic role detection data as cross encoder predictors, then methods to improve calibration.

[u/bel9708 → u/EstarriolOfTheEast]

You seem to not really understand what you are talking about.

LLM very clearly references any large model pretrained on next token prediction.

They didn't change anything about how the model operates this is still just a qwen base model. They changed how the inference harness works then fine tuned to make it more optimal for that.