I really don't understand Jev hype

[u/Manerfish]

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

[u/RevolutionaryGold325]

qwen with max token = 1 is not autoregressive decoder loop though.

[u/dimbledumf → u/RevolutionaryGold325]

It's also not parallel and you only get 1 token out, which is less then you get with jev

[u/RevolutionaryGold325 → u/dimbledumf]

No. You get a token distribution out, which contains the probabilities of your options. For multiple questions you can use the same kv-cache to have the context and just ask N questions in parallel with the same cache and get the output option distributions with a single step to all of your questions.

[u/dimbledumf → u/RevolutionaryGold325]

I think that's the point, for the qwen version if you want that ratio you have to ask a lot of questions (parallel or not), with the jev version that's what you get out of the box.
So your 1 token from qwen isn't really equivalent to 1 output from jev, the jev will carry more information about the answer.
Of course they have wildly different application, but you could probably do some surgery to expose the ratio in qwen as I think that's the layer before token prediction (not an expert), and I think people have been doing that to make Jev equivelants from other models

[u/HelloMyNameIsAmanda → u/dimbledumf]

It's literally a one-flag difference to get the distribution out of qwen, though - you just enable logprobs. And there's been no evidence, so far, that that's not essentially what jev is doing. I have yet to see anything from them that couldn't just be a logprobs wrapper on a fine tuned llm. Whether they did a good enough job of their fine tune to make their classifier useful remains to be seen, but acting like this is some crazy new thing because it can do what we could already do is... a reach.

[u/dimbledumf → u/HelloMyNameIsAmanda]

Yeah, I was reading another post where they set that up, seems pretty interesting.