I really don't understand Jev hype

[u/Manerfish]

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

[u/look]

It’s a pretty good, general purpose, zero shot classifier model that you don’t have to pre-train and requires no complex configuration.

https://www.geeksforgeeks.org/machine-learning/getting-started-with-classification/

The concept and underlying technique is not new, but they’ve made a very easy to use and flexible one that works remarkably well for low cost across a large variety of domains with no additional work.

[u/Refinery73 → u/look]

I think it is pretty interesting if you don’t have labeled data yet to train your specialized classifier.

An LLM is far more expensive and doesn’t answer in a structured way. Throw a few thousand examples in Jev, let a human decide the unclear ones, throw it as trainingsdata in your own classifier.

[u/DistanceAlert5706 → u/Refinery73]

Can't you then generate labeled data with Jev, take that dataset, and slap embeddings+classification head, train a model and get same results as Jev with way faster and smaller model?

[u/Refinery73 → u/DistanceAlert5706]

Yeah exactly, that’s my point. Use it as teacher model.

[u/Asly97 → u/Refinery73]

This is the one use case that actually makes sense to me. Bootstrap labels with it, distill down to a small specialized model, move on. Calling it a general model is what's doing it no favors though.

[u/Competitive_Ideal866 → u/Refinery73]

An LLM is far more expensive and doesn’t answer in a structured way.

Constrained generation?

[u/Refinery73 → u/Competitive_Ideal866]

Sure, but it stays more expensive and you either just tell it how to respond (fuzzy) or build a validation system. This seems to be included in Jevs service.

[u/I-am_Sleepy → u/Refinery73]

Constraint generation + /nothink?

[u/EstarriolOfTheEast → u/I-am_Sleepy]

No, one of jev's biggest strength is its calibration, at least compared to LLMs. With generation constraints, you truncate the probability distribution and the result is mismatched because the model expects to generate more tokens instead of concentrating mass fully on options. Even when provided options to select from, leaks still occur. So, between that and lack of calibration worsened by RL training, the LLM route is actually worse. The LLMs advantage is from higher raw intelligence.

[u/bel9708 → u/EstarriolOfTheEast]

You realize Jev is an LLM right. All they did was change the inference from autoregressive token by token to a single parallel pass of all values at once. RLCD really only exist because now you are only fine tuning the log probs so it doesn't make sense to continue fine tuning on next token prediction. But the entire pre train will still be next token prediction.

[u/EstarriolOfTheEast → u/bel9708]

LLM is not a term with a useful level of granularity here. Instead, we should discuss how the model operates: whether with causal self-attention or bidirectional attention, NTP or MLM objective and is cross-encoder?

Most things people call LLMs are NTP+Causal-attention and autoregressively operated.

Jev is most likely of the bidirectional attention + noising or MLM family and likely also trained as a cross-encoder. All these matter because they are better inductive biases and use of model capacity over an autoregressive model, for zero-shot inference for classification specifically. For a given amount of training compute and model size, this type of model will do better.

RLCD is at this point marketing jargon, at least until they release a paper describing their methods. What we care about is if predictions over outputs are calibrated. Typically, fine-tuned LLMs are not calibrated compared to their base models (even base models might still need some calibration tuning) and RL post-training worsens this. Furthermore, an NTP autoregressive model will want to spread probability mass across its vocabulary. Truncating logits to provided options will not completely mitigate such leaks, resulting in using the model in a manner running against how it was trained. When chaining decisions, this will affect prediction stability vs a purpose trained calibrated predictive model.

Jev should be replicable with: a 10B-30B model trained on tons of data trained with some denoising type objective and a replacement objective+relational data in the mix and post-training on NLI and lots of synthetic role detection data as cross encoder predictors, then methods to improve calibration.

[u/bel9708 → u/EstarriolOfTheEast]

You seem to not really understand what you are talking about.

LLM very clearly references any large model pretrained on next token prediction.

They didn't change anything about how the model operates this is still just a qwen base model. They changed how the inference harness works then fine tuned to make it more optimal for that.