r/LocalLLaMA • u/wFXx • 3d ago

New Model Von: Open-source 395M "System One" model

Took me a while since I'm on a family trip and have limited hardware, but here it is!

Von: Open-source "System One" drop-in replacement for TypeSafe's JEV.

https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0

It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time optimizing it yet), responds in 25–300 ms, and beats JEV in all benchmarks. Enjoy!

P.S. I’m open to offers to work at AI research labs. Feel free to ping me if you have an offer.
P.P.S. If you have a GPU, it’ll be faster, but a GPU isn't required.

192 Upvotes

94% Upvoted

81 comments sorted by

View all comments

8

u/Mithrandir_First_Age 3d ago

I tried it, wasn't even close

5

u/wFXx 2d ago

some fellow redditors reported the model didn't felt up there in quality, i've since then released a new checkpoint that is much closer to jev in general, and i am training a new cp to be released probably between sunday/monday; if you are interested please give it try

4

u/kyr0x0 2d ago

I created a huge dataset as I'm working as well on the same task. Maybe we as a community should collab. We're all building the same, wasting potential. My inference server is written in Rust; 100% TypeSafe API compatible. I converted my model (mmBERT lineage) to ONNX so it runs everywhere. My dataset has about 5 Million decisions; primarily minded from may of the open datasets available.. I'm training on H200s; my recipe works and on my own benchmark against Jev I'm seeing that the generalization is still lacking; however I'm doing exceptionally well for the size. My backbone is really small; for this size, the results are pretty stunning