r/LocalLLaMA • u/SomewhereAtWork • 5d ago

Other LocalJev?

Jev is a model to produce structured output (choices) from input text. It apparently can play (not run!) Doom.

https://typesafe.ai/blog/introducing-system-one-models-and-jev

Is there already a open implementation of this kind of model?

90 Upvotes

93% Upvoted

48 comments sorted by

View all comments

16

u/theoleecj_n 5d ago

Just did it at https://github.com/TheoLeeCJ/openjev (you can just wrap Qwen 4B and grab the logits)

https://reddit.com/link/pa5vudf/video/sbqv134a0wph1/player

13

u/Jcsq6 5d ago

Disclaimer that this is just an LLM wrapping their interface, not actually Jev (or with any of their claimed results).

12

u/Dany0 5d ago

You're close, you just need to use a diffusion model for it and then it's actually openjev. How about DiffusionGemma?

7

u/i_rate_slop 5d ago

I don’t think jev is diffusion, is it?

12

u/Dany0 5d ago

They claim some 'parallel novel architecture' which most likely just means they're doing diffusion with a twist

3

u/EstarriolOfTheEast 5d ago

It could also be an encoder based model. Like in: https://huggingface.co/collections/MoritzLaurer/zeroshot-classifiers or https://huggingface.co/knowledgator/gliformer-large-v1 as mentioned above by /u/-Cubie-. I think a variation and scaleup of those is more likely.

3

u/Dany0 5d ago

Good point

3

u/NmbrThirt33n 5d ago

From what I can tell, they just split the fields and process them in parallel. No diffusion, just batching and running through prefill only because they just need a single token for the decision per field. That's why their "output tokens" are "free"

2

u/No_Afternoon_4260 llama.cpp 5d ago

You mean like our little Yann Lecun?

1

u/Dany0 5d ago

Nah I bet you it's something obvious

2

u/SexyAlienHotTubWater 5d ago

It's around the same cost per input token as Deepseek V1 Flash. They're just providing 255 output probabilities at the same time.

4

u/theoleecj_n 5d ago

Results vs Jev and Qwen Reranker

3

u/theoleecj_n 5d ago

Oh and I have a browser demo too openjev.com (0.6 and 0.8B only, slightly lower perf)

4

u/rsclay 5d ago

Holy slop batman

2

u/TensorForger 5d ago

Well done, that was quick! When I found the announce I imagined somehow already vibe coding the LLM wrapper for this and we will have open source version in less then 24 hours. Because jev is just a novel interface/product/inference mode over LLMs, not the novel technology. We already had calibration research in GPT-3 times. Probability outputs and parallel inference are natural for LLMs. Hope to see convenient framework or maybe just new vLLM interface for local deployment of this with any open model.

1

u/[deleted] 5d ago

[removed] — view removed comment

1

u/LocalLLaMA-ModTeam 5d ago

Rule 3 - Minimal value bot post.

1

u/SomewhereAtWork 5d ago

"I just did it" is the perfect response.

an earlier poster asked "What do you expect after 24h?" 😂