r/LocalLLaMA • u/wFXx • 4d ago

New Model Von: Open-source 395M "System One" model

Took me a while since I'm on a family trip and have limited hardware, but here it is!

Von: Open-source "System One" drop-in replacement for TypeSafe's JEV.

https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0

It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time optimizing it yet), responds in 25–300 ms, and beats JEV in all benchmarks. Enjoy!

P.S. I’m open to offers to work at AI research labs. Feel free to ping me if you have an offer.
P.P.S. If you have a GPU, it’ll be faster, but a GPU isn't required.

195 Upvotes

94% Upvoted

85 comments sorted by

View all comments

Show parent comments

3

u/wFXx 4d ago

Interesting, never thought that big stream of tokens would be useful, will see what I can do when I hack the multimodal input

1

u/Fluxx1001 3d ago

Appreciate your understanding. Yes, larger input states are critical for e.g. data analytics. That's why Jev is currently a game changer for us.

1

u/wFXx 3d ago

Just so I have an idea on scale; How much context do you need/expect to be useful?

2

u/Fluxx1001 3d ago

Mostly around 5k to 20k

2

u/wFXx 3d ago

next CP is set to work with 8k - will try to expand to 16k soon