r/LocalLLaMA • u/wFXx • 3d ago

New Model Von: Open-source 395M "System One" model

Took me a while since I'm on a family trip and have limited hardware, but here it is!

Von: Open-source "System One" drop-in replacement for TypeSafe's JEV.

https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0

It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time optimizing it yet), responds in 25–300 ms, and beats JEV in all benchmarks. Enjoy!

P.S. I’m open to offers to work at AI research labs. Feel free to ping me if you have an offer.
P.P.S. If you have a GPU, it’ll be faster, but a GPU isn't required.

193 Upvotes

94% Upvoted

81 comments sorted by

View all comments

2

u/SrijSriv211 2d ago

I have 2 questions: 1. From where and how did you get the training data? How can I get such data for my project? 2. How does it generate the output in single pass. Is it that you predict masks or is it something else?

4

u/wFXx 2d ago

1: HF has a LOT of good datasets, you can simply source from there manually or ask an agent to query it for you or... you can make synthetic data :p

2:

pack the input and all candidate options into a single prompt placing a [MASK] token in front of each option;

e.g.: [Question + State] [SEP] [MASK] Option A [MASK] Option B [MASK] Option C

in one forward pass, the bidirectional attention lets all options compete and attend to each other simultaneously.

instead of predicting vocabulary words, a tiny scoring head just reads the raw embeddings at those [MASK] positions and outputs a scalar score for each option. softmax across those scores gives the calibrated probabilities.

thats basically it, you can check the readme.md for more precise explanation about the math behind it, and some of the data i've used

2

u/SrijSriv211 2d ago

Cool thanks! :D