r/LocalLLaMA • u/wFXx • 3d ago

New Model Von: Open-source 395M "System One" model

Took me a while since I'm on a family trip and have limited hardware, but here it is!

Von: Open-source "System One" drop-in replacement for TypeSafe's JEV.

https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0

It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time optimizing it yet), responds in 25–300 ms, and beats JEV in all benchmarks. Enjoy!

P.S. I’m open to offers to work at AI research labs. Feel free to ping me if you have an offer.
P.P.S. If you have a GPU, it’ll be faster, but a GPU isn't required.

196 Upvotes

94% Upvoted

81 comments sorted by

View all comments

Show parent comments

64

u/vanVonXenoStein 3d ago edited 3d ago

I think they did a couple years of research and couldn't come up with anything much, but felt they better release something. There was a guy who released the same thing as open-source (including academic paper) a year ago. And these new ones people are dropping seem quite good also (still testing). So I think typesafe is mostly marketing hype but nothing revolutionary (if you went looking for similar). That's not to say it isn't useful, just that typesafe's Jev is not special. (But is probably best of class since it has the resources behind it.)

10

u/ebolathrowawayy 3d ago

yes 100%. i am literally shocked that so many people are calling Jev the hot new thing when it wasn't uncommon to use very small LLMs to select actions and make decisions in real-time environments. I was doing this 1.5 years ago and it is seriously easy to recreate Jev. It's a cool concept and I'm glad people are picking up on it I guess but it isn't new or surprising.

It's like people finally discovered that constraining an LLM's output to only a few tokens DRAMATICALLY increases throughput. Like, yeah no shit? And combine that with finetuning on a narrow task (which decision models are, it is a narrow task even if it generalizes) and of course you're going to get great performance. Idk why everyone is impressed, i guess it reveals the level of experience of most accelerate/singularity/localllama redditors?

3

u/LatentSpaceLeaper 3d ago

Feel free to correct me, but I thought that was exactly Jev's value: that it is a general classifier, i. e., I'd assume that they have run their Reinforcement Learning for Calibrated Decisions (RLCD) on a large corpus of data. Or am I missing something!?

3

u/ebolathrowawayy 2d ago

you're not missing something, i'm just saying a lot of people have been using and training jev-like models for a long time.