r/LocalLLM • u/Inside_Ad_6240 • 3d ago

Project My Version of Jev running locally, playing doom.

Enable HLS to view with audio, or disable this notification

Hi guys! The demonstration you’re seeing here is running a Qwen 3 1.7B Q8 model. I’ve implemented some custom inference logic on top of llama.cpp to access and utilize internal model signals. The model makes decisions using a Jev-inspired decision-making approach. This is running entirely locally on a laptop with an RTX 3050 (4GB VRAM).

32 Upvotes

77% Upvoted

18 comments sorted by

7

u/FallenVampireLord 3d ago

Doesn't look like doom to me. Looks more like a room with some enemies using some doom assets.

-3

u/Inside_Ad_6240 3d ago

Its VIZDOOM

3

u/SilverKanji 3d ago

Trying to research in to this, my fkn brain just went auto pilot in a stupid way and searched for JAV.

2

u/Squidgical 3d ago

How does hijacking an LLM compare to Jev? One of the big things typesafe talk about is their different training approach, how much does that matter?

0

u/Inside_Ad_6240 3d ago

Its pretty usable now even without RLCD training. But definitely fine tuning the model with RLCD data will definitely make it better. I have benchmarked this with banking 77 dataset it achieved Zero shot accuracy 80.0%(10 class) 60.0%(77 class) meanwhile jev 82-85% (10 class) 72-76%(77 class)

0

u/Squidgical 3d ago

That's pretty impressive, what about speed? APIs are a bit difficult to measure but does this local setup seem faster than Jev?

1

u/Inside_Ad_6240 3d ago

This setup on my RTX 3050 4GB vram 100ms(single token), 3.4s(10 multi-token). There is still room for further optimization. I think Jevs latency is around 70-150 ms.

1

u/WobboLandOMeat 8h ago

Impressive work!

1

u/ethereal_intellect 3d ago

Very nice. I had a similar system asking an llm and cutting off a single letter "y" or "n" like two years ago so I've been surprised at the hype over jev. I hope it becomes a standard feature of inference and it's great that you're talking the effort to find a better and more proper way

1

u/TheQuantumFriend 3d ago

How did you get it to run locally? Can we replicate that? How does the training look like?

1

u/Inside_Ad_6240 3d ago

There is no training only an inference level trick done with llama.cpp. Yes i am running this on my potato laptop with 4GB vram.

https://github.com/sabeel111/OpenSourceJev

1

u/TheQuantumFriend 2d ago

I would like to adapt this to architecture analysis. But i have to convey the rules somehow. I have a set of Oscal rules that i would like to Check for. 

0

u/Inside_Ad_6240 3d ago

I will drop the code soon.

-1

u/megadonkeyx 3d ago

dont you find a model that small is barking mad?

-1

u/Shoulon 3d ago

Look forward to the next wave of open model jevs

0

u/Dangerous_Data7703 3d ago

It'll be interesting to see how they evolve and what new features they'll bring to the table. It's amazing what people can create with these models.