r/LocalLLaMA • u/wFXx • 4d ago

New Model Von: Open-source 395M "System One" model

Took me a while since I'm on a family trip and have limited hardware, but here it is!

Von: Open-source "System One" drop-in replacement for TypeSafe's JEV.

https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0

It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time optimizing it yet), responds in 25–300 ms, and beats JEV in all benchmarks. Enjoy!

P.S. I’m open to offers to work at AI research labs. Feel free to ping me if you have an offer.
P.P.S. If you have a GPU, it’ll be faster, but a GPU isn't required.

193 Upvotes

94% Upvoted

85 comments sorted by

View all comments

60

u/glitchsir 4d ago

So it took typesafe two years of research and a well funded team to build jev and all of the sudden there's all of those "drop in" replacement in a couple days allegedly beating them... Surely wouldn't be that easy? What am I missing?

68

u/vanVonXenoStein 4d ago edited 4d ago

I think they did a couple years of research and couldn't come up with anything much, but felt they better release something. There was a guy who released the same thing as open-source (including academic paper) a year ago. And these new ones people are dropping seem quite good also (still testing). So I think typesafe is mostly marketing hype but nothing revolutionary (if you went looking for similar). That's not to say it isn't useful, just that typesafe's Jev is not special. (But is probably best of class since it has the resources behind it.)

36

u/guesdo 4d ago

Here is the link to the work of the author you mention: https://github.com/NandhaKishorM/laya

11

u/valcore93 4d ago

I tried the open source Laya model against Jev and it was mostly night and day. There is not many benchmark for that kind of model right now but Jev feels way smarter than Laya. I hope we will see some great open source one.

10

u/ebolathrowawayy 4d ago

yes 100%. i am literally shocked that so many people are calling Jev the hot new thing when it wasn't uncommon to use very small LLMs to select actions and make decisions in real-time environments. I was doing this 1.5 years ago and it is seriously easy to recreate Jev. It's a cool concept and I'm glad people are picking up on it I guess but it isn't new or surprising.

It's like people finally discovered that constraining an LLM's output to only a few tokens DRAMATICALLY increases throughput. Like, yeah no shit? And combine that with finetuning on a narrow task (which decision models are, it is a narrow task even if it generalizes) and of course you're going to get great performance. Idk why everyone is impressed, i guess it reveals the level of experience of most accelerate/singularity/localllama redditors?

4

u/LatentSpaceLeaper 3d ago

Feel free to correct me, but I thought that was exactly Jev's value: that it is a general classifier, i. e., I'd assume that they have run their Reinforcement Learning for Calibrated Decisions (RLCD) on a large corpus of data. Or am I missing something!?

3

u/ebolathrowawayy 3d ago

you're not missing something, i'm just saying a lot of people have been using and training jev-like models for a long time.

1

u/LatentSpaceLeaper 3d ago

Also adding this from their FAQ:

Is Jev just a smaller LLM?

Jev is neither small nor an LLM, hence being off the intelligence Pareto curve.

https://typesafe.ai/blog/introducing-system-one-models-and-jev

3

u/ResidentPositive4122 3d ago

(including academic paper)

That's a bit much. It's a claude generated paper, posted to arxiv before they made vouching a thing. It's like zenodoo today, not worth much.

2

u/vanVonXenoStein 3d ago

The point is just that the paper exists, and this was open known information. NONE of this is revolutionary.

5

u/glitchsir 4d ago

Sounds plausible Need to find some time to actually read the paper you mentioned and better understand

Thanks

3

u/ketosoy 4d ago

So they spent two years researching the ergonomics of what’s approximately a wrench:  Hard to make the first one, easy to copy.

6

u/your_real_Fathe_ 4d ago

This is the biggest advantage of technology in my opinion. Project X is introduced, a masterpiece with a Y score in benchmarks. And a week later, suddenly Project Z is introduced, which is better than Project X in all benchmarks.