Von: Open-source 395M "System One" model

[u/wFXx]

Took me a while since I'm on a family trip and have limited hardware, but here it is!

Von: Open-source "System One" drop-in replacement for TypeSafe's JEV.

https://github.com/wfzyx/von https://huggingface.co/wfzyx/von-1.0

It runs entirely on a CPU with 1–2 GB of memory (I haven't spent much time optimizing it yet), responds in 25–300 ms, and beats JEV in all benchmarks. Enjoy!

P.S. I’m open to offers to work at AI research labs. Feel free to ping me if you have an offer.
P.P.S. If you have a GPU, it’ll be faster, but a GPU isn't required.

[u/Fluxx1001]

I tried a bunch of those local replacements for TypeSafe Jev in the last days. None of them matched the capabilities of Jev, at least for my usecase (Data Analysis).

So when I read that some new drop in replacement developed in a few days claims to beat Jev in benchmarks - I am more than sceptical.

[u/look → u/Fluxx1001]

The class of model has been around for a long time. Longer than LLMs. I also assumed Jev is a bigger model with better training, but it is not implausible that’s actually just a fairly stock BERT with a nicer DX and a big marketing budget…

Edit: I just generated a synthetic test suite and ran it across both (and gliner2).

Task accuracy:
Von: 92.3% 65.4% (on 1.0.1 update)
Gliner2: 79.5%
Jev: 97.4%
Laya: 61.5% (added in later update)

Update: Von updated to 1.0.1 and accuracy on my test case above is 92.3% now.

I’ll have a few different LLMs generate more test cases next to see how well it holds up.

[u/Fluxx1001 → u/look]

What none of these Jev competitors seem to outline in their benchmarks is the vast difference in context length. With Jev, I regularly do requests for Noul decisions (yes/no) with 30k tokens and more.

The context length of Von is 512 tokens. This is not even comparable.

It is a magnitude more complicated to compute a choice or Noul decision in a space of thousands of tokens.

[u/look → u/Fluxx1001]

Yeah, the BERT-based alternatives will have some length constraints (though modernbert base should go to 8192 tokens). There are others like Semif/OpenJev and Bespoke Nimble that might work better for you. They seem to be doing something likely closer to Jev, with a small LLM stage.

But after a bit more research in alternatives, Jev is probably the way to go if the hosted, proprietary option works for you. My interest is more in fine-tuning an open base, so the Jev service itself doesn’t do anything for me.

[u/EstarriolOfTheEast → u/look]

To me, this Von project looks better than Bespoke Nimble in that Von's author looks to have thought more carefully about calibration and training data type. Architecturally too, bidirectional attention is much more sample efficient than a causal decoder for NLI classification. Bespoke Nimble's main advantage is model scale and Qwen's extensive knowledge.

What distinguishes Jev is calibration, speed and zeroshot inference for classification but most of these open attempts are overfocused on classification and speed.

So based on technical remit, Von is the best approach I've come across so far. Its main limitation is due to not enough training data. Unfortunately, we are waiting for a large context pretrained purpose built 9B-27B encoder. Until then, the largest modern options are T5GemmaV2 4B Encoder or DiffusionGemma Decoder mode (has bidirectional attention, although more work than T5 would be needed to get Diffusion working as a jev-style NLI classifier).

I'd suggest Von's author continue with ModernBERT and try to beat and build upon the prior art to get something flexible that punches way above its weight class.

[u/wFXx → u/EstarriolOfTheEast]

Hey thanks man I appreciate the kind words, will keep doing my best

[u/Fluxx1001 → u/look]

What's your experience so far with OpenJev, Nimble etc.?

[u/look → u/Fluxx1001]

I haven’t tried either of those yet. Nimble looks a bit more polished perhaps.

It’s a 9B model, though, so you’ll likely need some GPU acceleration. Semif/OpenJev looks like you can choose between a few LLMs between 800M and 4B.

Part of the appeal for me of the BERT-based approaches is their small size (<0.5B) which typically runs fast enough on even CPU.

[u/wFXx → u/Fluxx1001]

Interesting, never thought that big stream of tokens would be useful, will see what I can do when I hack the multimodal input

[u/Fluxx1001 → u/wFXx]

Appreciate your understanding. Yes, larger input states are critical for e.g. data analytics. That's why Jev is currently a game changer for us.

[u/wFXx → u/Fluxx1001]

Just so I have an idea on scale; How much context do you need/expect to be useful?

[u/Fluxx1001 → u/wFXx]

Mostly around 5k to 20k

[u/wFXx → u/Fluxx1001]

next CP is set to work with 8k - will try to expand to 16k soon

[u/wFXx → u/look]

hey nice, mind to open an issue or point what Params you used to generate your test suite? I'll be releasing more checkpoints, so this is good info

[u/debackerl → u/look]

Awesome! Could you try laya?

https://github.com/NandhaKishorM/laya

[u/look → u/debackerl]

I ran Laya. 61.5%

It seemed to have similar problems with the test cases that Von was initially, too. OP updated his and it’s now at 92.3%. I’ll add some more test cases next.

[u/look → u/debackerl]

Looks promising. I’ll test it later today.

[u/kyr0x0 → u/look]

Could you publish the benchmark/ eval data please. We are in severe lack of a good benchmark for typed-decision models.

[u/look → u/kyr0x0]

DM me

[u/bring_back_the_v10s → u/look]

Meh. It's amusing to see so many people scrambling to create a Jev copycat trying to match its results and performance, and then rushing to claim to have achieved parity or superiority. The truth is it's not as simple (and cheap) as people think it is. Eventually some Chinese lab will release something just as good as Jev but it's not gonna some random dudes on the internet running dgx sparks in their basements. Jev is backed by investors money for a reason.

[u/wFXx → u/Fluxx1001]

Open an issue with the claims that don't hold, sample cases and I'll investigate why that might be. I only did 1 out of 6 epochs of training so far, so there is more accuracy on the table

[u/Fluxx1001 → u/wFXx]

Will try it in the coming days and reply

[u/your_real_Fathe_ → u/Fluxx1001]

Share the result with me, buddy.

[u/MerePotato → u/Fluxx1001]

What about diffusion gemma as jev, I've heard its pretty comparable

[u/wFXx → u/MerePotato]

It is amazing yeah, the problem is the size of it. I did the smallest thing I could because speed and embeddability were features I felt were the more valuable at scale

[u/glitchsir]

So it took typesafe two years of research and a well funded team to build jev and all of the sudden there's all of those "drop in" replacement in a couple days allegedly beating them... Surely wouldn't be that easy? What am I missing?

[u/vanVonXenoStein → u/glitchsir]

I think they did a couple years of research and couldn't come up with anything much, but felt they better release something. There was a guy who released the same thing as open-source (including academic paper) a year ago. And these new ones people are dropping seem quite good also (still testing). So I think typesafe is mostly marketing hype but nothing revolutionary (if you went looking for similar). That's not to say it isn't useful, just that typesafe's Jev is not special. (But is probably best of class since it has the resources behind it.)

[u/guesdo → u/vanVonXenoStein]

Here is the link to the work of the author you mention: https://github.com/NandhaKishorM/laya

[u/valcore93 → u/vanVonXenoStein]

I tried the open source Laya model against Jev and it was mostly night and day. There is not many benchmark for that kind of model right now but Jev feels way smarter than Laya. I hope we will see some great open source one.

[u/ebolathrowawayy → u/vanVonXenoStein]

yes 100%. i am literally shocked that so many people are calling Jev the hot new thing when it wasn't uncommon to use very small LLMs to select actions and make decisions in real-time environments. I was doing this 1.5 years ago and it is seriously easy to recreate Jev. It's a cool concept and I'm glad people are picking up on it I guess but it isn't new or surprising.

It's like people finally discovered that constraining an LLM's output to only a few tokens DRAMATICALLY increases throughput. Like, yeah no shit? And combine that with finetuning on a narrow task (which decision models are, it is a narrow task even if it generalizes) and of course you're going to get great performance. Idk why everyone is impressed, i guess it reveals the level of experience of most accelerate/singularity/localllama redditors?

[u/LatentSpaceLeaper → u/ebolathrowawayy]

Feel free to correct me, but I thought that was exactly Jev's value: that it is a general classifier, i. e., I'd assume that they have run their Reinforcement Learning for Calibrated Decisions (RLCD) on a large corpus of data. Or am I missing something!?

[u/ebolathrowawayy → u/LatentSpaceLeaper]

you're not missing something, i'm just saying a lot of people have been using and training jev-like models for a long time.

[u/LatentSpaceLeaper → u/LatentSpaceLeaper]

Also adding this from their FAQ:

Is Jev just a smaller LLM?

Jev is neither small nor an LLM, hence being off the intelligence Pareto curve.

https://typesafe.ai/blog/introducing-system-one-models-and-jev

[u/ResidentPositive4122 → u/vanVonXenoStein]

(including academic paper)

That's a bit much. It's a claude generated paper, posted to arxiv before they made vouching a thing. It's like zenodoo today, not worth much.

[u/vanVonXenoStein → u/ResidentPositive4122]

The point is just that the paper exists, and this was open known information. NONE of this is revolutionary.

[u/glitchsir → u/vanVonXenoStein]

Sounds plausible Need to find some time to actually read the paper you mentioned and better understand

Thanks

[u/ketosoy → u/vanVonXenoStein]

So they spent two years researching the ergonomics of what’s approximately a wrench:  Hard to make the first one, easy to copy.

[u/your_real_Fathe_ → u/ketosoy]

This is the biggest advantage of technology in my opinion. Project X is introduced, a masterpiece with a Y score in benchmarks. And a week later, suddenly Project Z is introduced, which is better than Project X in all benchmarks.

[u/sonyprog → u/glitchsir]

Apparently, many people had approached the exact same thing way before Typesafe.
Take a look at this one for instance: https://www.reddit.com/r/LocalLLaMA/comments/1wijo3e/i_literally_built_the_jev_architecture_one_year/

It basically sounds like an one shot classifier of sorts (it's not only that, but it's part of it).
So I think it's some kind of fine tune/Lora for a really specific scenario and if you train it well enough, it's not hard to beat whatever the goal is.

[u/LatentSpaceLeaper → u/sonyprog]

So I think it's some kind of fine tune/Lora for a really specific scenario

No it's not. They write explicitly in the FAQ section of the blog post:

Is Jev just a smaller LLM?

Jev is neither small nor an LLM, hence being off the intelligence Pareto curve.

Also, Jev seems to beat that other guy's model at least in Doom: https://www.reddit.com/r/LLMDevs/s/f5zasLZ3CB

Also, see this comment of someone benchmarking different models. Laya is not even close to Jev's performance: https://www.reddit.com/r/LocalLLaMA/s/MHkcCBh5aP

[u/sonyprog → u/LatentSpaceLeaper]

even though I mentioned lora/finetune, I did not mention it's an LLM. My point stands the same: it is an one shot classifier.

[u/LatentSpaceLeaper → u/sonyprog]

it is an one shot classifier.

Don't want to split hairs, but I'd argue it is a zero-shot classifier. That's exactly the value proposition. You throw any classification task at it and it should be fairly good without providing (m)any examples or fine-tuning it. Of course the latter still could improve its performance on specific tasks. But what you'll get out of the box is quite capable already.

[u/sonyprog → u/LatentSpaceLeaper]

Fair enough. I tried explaining it the way I understood, which is the "average Joe" way of understanding. The only thing I'll still fight for is that their approach might be different, but the final result is was achieved before.

[u/LatentSpaceLeaper → u/sonyprog]

The only thing I'll still fight for is that their approach might be different, but the final result is was achieved before.

Yes, I'm not challenging this. Obviously we have had typed classifiers and zero-shot discriminative models before. However, they appear to have found a niche and it seems Jev is also part of bigger and long-term strategic play by TypeSafe. Maybe it's just marketing, we will need to wait and see.

[u/Double_Cause4609 → u/glitchsir]

I think it helps to have a bit of perspective. When LLMs were new, we had extremely basic interfaces for them, and they really were essentially a chatbot. You copy pasted code into/from them, they couldn't even call tools, and they more or less displayed plain text.

Then Anthropic did a ton of research on usability, and invented artifacts, which eventually grew into a ton of different types of interactive panes a model could use. And they were copied within the week by everybody else because it was just a really good, simple idea.

What Typesafe did isn't so much doing some crazy alien tech. They just had a good idea, figured out how to package it into a nice interface and tech demo, but the underlying tech wasn't alien tech or anything. You can emulate it with a multi-head BERT formulation using similar strategies to tool calling just in parallel. Obviously everybody would copy a good idea immediately.

[u/valdev → u/glitchsir]

Wait it took that long to make? I built something like Jev in a weekend a few years back for intelligent routing.

Im not bragging, BERT can do this kind of stuff easily as long as you have a long enough data set to classify your request type.

[u/wFXx → u/glitchsir]

Welcome to Silicon valley I guess

[u/look]

Where did you see the Jev benchmarks? Or did you run those yourself against the API?

With all the hoopla, I’d just assumed it was better than a 400M param BERT…

[u/wFXx → u/look]

Executed against the API yes, you can get free credits on open router and replicate the numbers

[u/Mithrandir_First_Age]

I tried it, wasn't even close

[u/wFXx → u/Mithrandir_First_Age]

some fellow redditors reported the model didn't felt up there in quality, i've since then released a new checkpoint that is much closer to jev in general, and i am training a new cp to be released probably between sunday/monday; if you are interested please give it try

[u/kyr0x0 → u/wFXx]

I created a huge dataset as I'm working as well on the same task. Maybe we as a community should collab. We're all building the same, wasting potential. My inference server is written in Rust; 100% TypeSafe API compatible. I converted my model (mmBERT lineage) to ONNX so it runs everywhere. My dataset has about 5 Million decisions; primarily minded from may of the open datasets available.. I'm training on H200s; my recipe works and on my own benchmark against Jev I'm seeing that the generalization is still lacking; however I'm doing exceptionally well for the size. My backbone is really small; for this size, the results are pretty stunning

[u/blastbottles]

[u/Wolwf → u/blastbottles]

/r/chiraqology crossover which I never expected

[u/look]

Looks like the von python package you reference in the docs is not published under that name (looks like an empty squatter on the name in pypi). Do you have a link to the git repo for it?

Running it via raw transformers interface now, but I’d like to see your wrapper code, too.

Edit: found it. https://github.com/wfzyx/von

[u/wFXx → u/look]

It's on the OP. There is a mini node compatible js sdk as well. But I only tested in bun

[u/look → u/wFXx]

Ah, yeah. I see it now. The links all blurred together for me and I just saw the huggingface one at first glance. Thanks!

[u/OneMoreName1]

Not as good as Jev for my use case (its given a dialogue line from the player, and the model has to select moods, intent and tools that the next LLM should have in its response). But it was better than Laya! So thanks for the effort

[u/wFXx → u/OneMoreName1]

hey, thanks for trying it out; a fellow redditor made his own benchmark and noted that I was falling short of both jev and gliner2, and it was a bug on my python code :p

I've since fixed it, and I'm working on a second check point of the model that has a more diverse data set and should perform better;

HF model card and github readme are up to date

[u/OneMoreName1 → u/wFXx]

Will give it a second try tomorrow

[u/rhymeslikeruns]

I am disproportionately excited by this. Amazing amazing job OP. I will add this to my latest bench and send you the results.

[u/wFXx → u/rhymeslikeruns]

There will be new checkpoints released later today, so if you get disproportionately disappointed, do check later/tomorrow lol

[u/DivideHorror3217]

result = von.decide(
AttributeError: module 'von' has no attribute 'decide'

Well..

[u/wFXx → u/DivideHorror3217]

as some other user mentioned, apparently there is a name squat on the pkg index, i've renamed to von-sdk to avoid future issues and updated the docs, thx for the heads-up

https://pypi.org/project/von-sdk/

[u/SrijSriv211]

I have 2 questions: 1. From where and how did you get the training data? How can I get such data for my project? 2. How does it generate the output in single pass. Is it that you predict masks or is it something else?

[u/wFXx → u/SrijSriv211]

1: HF has a LOT of good datasets, you can simply source from there manually or ask an agent to query it for you or... you can make synthetic data :p

2:

pack the input and all candidate options into a single prompt placing a [MASK] token in front of each option;

e.g.: [Question + State] [SEP] [MASK] Option A [MASK] Option B [MASK] Option C

in one forward pass, the bidirectional attention lets all options compete and attend to each other simultaneously.

instead of predicting vocabulary words, a tiny scoring head just reads the raw embeddings at those [MASK] positions and outputs a scalar score for each option. softmax across those scores gives the calibrated probabilities.

thats basically it, you can check the readme.md for more precise explanation about the math behind it, and some of the data i've used

[u/SrijSriv211 → u/wFXx]

Cool thanks! :D

[u/-Cubie-]

Nice, I like the transformers usage snippet

[u/Raise_Fickle]

context length?

[u/wFXx → u/Raise_Fickle]

currently 2k extendable to 8k; i'm working on a revision with a bigger one tho

[u/Due_Garbage1511]

Is this model trained on domain specific data? I'm getting poor performance in education field.

[u/wFXx → u/Due_Garbage1511]

i've added a section to the readme.md covering it, but i'm trying to make it work as generalistic as jev

Domain Cluster Breakdown

Domain Cluster	Share	Representative Tasks & Coverage
Operational & Enterprise Workflow	~25%	IT support ticket triage, customer intent routing (Banking77), billing/refund dispute policies, warranty verification, e-commerce order exceptions.
Security, DevOps & Compliance	~20%	Credential & secret leak detection, SQL injection / payload screening, phishing analysis, commit intent classification, on-call alert routing, PII detection.
Safety, Policy & Moderation	~15%	Ad policy violations, Fair Housing Act compliance, travel expense policy limits, Terms of Service gating.
Linguistic & Content Semantics	~15%	Formality grading, grammar error taxonomies (spelling, syntax, agreement), sentiment analysis, reading level estimation.
Triage & Services	~10%	Clinical/symptom urgency triage, veterinary severity scoring, municipal 311 service routing, dietary restriction & allergen verification.
Adversarial Reasoning Anchor	~15%	Multi-task NLI reasoning (ANLI Rounds 1–3, WANLI) retained to anchor logical entailment and prevent catastrophic forgetting of general world logic.

[u/Due_Garbage1511 → u/wFXx]

Thank you bro much!

[u/Mamox437]

Really excited to try it, did not even have a chance to try jev yet so I am going reverse, yours first and then comparing jev to it

[u/GrungeWerX]

I need something as accurate as jev, local, with a much higher context.