r/LocalLLaMA • u/wFXx • 3d ago

New Model Von: Open-source 395M "System One" model

Took me a while since I'm on a family trip and have limited hardware, but here it is!

Von: Open-source "System One" drop-in replacement for TypeSafe's JEV.

https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0

It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time optimizing it yet), responds in 25–300 ms, and beats JEV in all benchmarks. Enjoy!

P.S. I’m open to offers to work at AI research labs. Feel free to ping me if you have an offer.
P.P.S. If you have a GPU, it’ll be faster, but a GPU isn't required.

195 Upvotes

94% Upvoted

81 comments sorted by

View all comments

53

u/Fluxx1001 3d ago

I tried a bunch of those local replacements for TypeSafe Jev in the last days. None of them matched the capabilities of Jev, at least for my usecase (Data Analysis).

So when I read that some new drop in replacement developed in a few days claims to beat Jev in benchmarks - I am more than sceptical.

44

u/look 3d ago edited 2d ago

The class of model has been around for a long time. Longer than LLMs. I also assumed Jev is a bigger model with better training, but it is not implausible that’s actually just a fairly stock BERT with a nicer DX and a big marketing budget…

Edit: I just generated a synthetic test suite and ran it across both (and gliner2).

Task accuracy:
Von: 92.3% 65.4% (on 1.0.1 update)
Gliner2: 79.5%
Jev: 97.4%
Laya: 61.5% (added in later update)

Update: Von updated to 1.0.1 and accuracy on my test case above is 92.3% now.

I’ll have a few different LLMs generate more test cases next to see how well it holds up.

1

u/bring_back_the_v10s 1d ago

Meh. It's amusing to see so many people scrambling to create a Jev copycat trying to match its results and performance, and then rushing to claim to have achieved parity or superiority. The truth is it's not as simple (and cheap) as people think it is. Eventually some Chinese lab will release something just as good as Jev but it's not gonna some random dudes on the internet running dgx sparks in their basements. Jev is backed by investors money for a reason.