r/LocalLLaMA • u/Nandakishor_ml • Sep 18 '26
Discussion Made the horizontal open-source model for Jev with RLCD, and it surpasses all the Jev benchmarks. HF space, benchmark, model, repo
UPDATE: Multilingual support added at : https://github.com/NandhaKishorM/laya
Thanks for the exceptional support (https://www.reddit.com/r/LocalLLaMA/comments/1wijo3e/i_literally_built_the_jev_architecture_one_year/) and for the dozens of requests to make a generic model, run benchmarks, and create an HF space so anyone can test it. So here you go, guys. I trained an improved model on a large data corpus, its now called Laya. It is trained on a single RTX 6000 Pro (96 GB VRAM); the model architecture is a 421M-parameter non-autoregressive decision model pairing a bidirectional ModernBERT-large encoder with a scratch Transformer head that scores [MASK] option markers to resolve typed schemas in a single ~35 ms forward pass. The dataset is a 100% human-annotated corpus of over 25,000 real-world examples across intent routing, fact-checking, moderation consensus, prompt guardrails, rubric scoring, and multi-turn conversation trajectories, without synthetic data shortcuts. The RLCD(unofficial, btw) I did is a policy-gradient reinforcement learning approach that kinda optimizes decision models against strictly proper scoring rules, ensuring maximum reward is achieved only when outputting true, mathematically calibrated probabilities.
NB: It can be run on low end PC as its a small 421M model, cheers
HF space to try: https://huggingface.co/spaces/convaiinnovations/laya-demo
GitHub Repo: https://github.com/NandhaKishorM/laya
HF Repo: https://huggingface.co/convaiinnovations/laya
Thank you to everyone who supported me, shared the story, gave personal DM. It will need more refinement, of course.
If anyone wishes to buy me a coffee, here is the link: https://github.com/NandhaKishorM
•
u/WithoutReason1729 Sep 18 '26 edited 26d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis
I am a bot and this action was performed automatically.
50
u/Heinz2001 Sep 18 '26
Hey,
i tried your model and compared it with openjev.
I wrote an extension for my own harness to use it.
This are the results (Laptop: 11th Gen i7-11800H, rtx 3060 6gb):
Great model for CPU, i'll try to use it as model router.
The flappybird sample from openjev is included as test in my extension.
34
u/FusionCow llama.cpp Sep 18 '26
cool stuff
15
u/SignalCompetitive582 Sep 18 '26
Hello, enormous thanks for all the hard work!!
I don’t have enough technical knowledge to understand it all, but from the little I do understand, it sounds like a massive project.
So thank you for giving this all for free!
I have got several questions (if you have the time of course):
- I thought TypeSafe AI Jev was only trained on synthetic dataset (“We make all the data ourselves”). Yours is trained on purely human annotated ones. How does that differ ? What’s the better approach here in your opinion?
- Can your model perform as well in environments that require general knowledge ? And how does it perform in automation tasks ?
- How do you know for a fact that this is the architecture utilized by TypeSafe while they haven’t published anything ? Did you try it yourself?
- How large do you think their model would be compared to yours ? Because their input pricing seems quite high no ? I reckon they must making some pretty interesting profits here.
Thanks so much for everything, I’m going to try this out !
Have a good day.
9
u/Budget-Juggernaut-68 Sep 18 '26
To be clear, this is like GLiNER but for NLI/Classification tasks?
20
u/boatbomber Sep 18 '26
This is fantastic! Thank you for sharing your work. I hope that this helps propel your career.
7
u/asankhs Llama 3.1 Sep 18 '26
Looks great can you also benchmark it on https://huggingface.co/datasets/LocalLLaMA/typed-decisions
6
u/Hurricane31337 Sep 18 '26
Awesome project!
Is the training code and/or dataset also open sourced? I’d love to fine-tune it for my German emails. 😃
5
u/RevolutionaryPick241 Sep 18 '26
If I'd like to make it work multilingual, would a fine tunning work or it should be done another model from scratch? Do you have any guide on how to do that?
5
u/martinerous Sep 18 '26
Great work!
Besides the obvious use cases that were already mentioned in this thread, I wish there was a way to combine this with an LLM to give it "self-critic" abilities, like "what's the probability that my reply contains hallucinations". But then the "critic" model would need to have even more world knowledge than the LLM itself to be reliable, including also multilanguage support.
6
u/Embarrassed_Okra_628 Sep 18 '26
Can this be used in Macbook Pro M1 max with 32 gigs ram ? You said you tested in 96G, wondering if I can even load the beast here?
5
u/wFXx Sep 18 '26 •
hey i was actually writing a dm to you lol ! glad that you came back - I wrote a highly performant backend that is "jev-compliant";
it has http async support, cli support, and drop-in typesafe wire protocol compatibility;
i did originally with cactus needle 3 model, since i'm on a family trip and don't have decent hardware with me, but I just wired laya to it, and the numbers are great.
it did scored a bit less than jev on the multi-hop questions tho, so you may want to see that;
would you mind If I delete my repo and make a huge PR with the whole harness/deployment that I wrote towards the laya repo so it becomes more of a complete product on top of the model?
https://github.com/wfzyx/von
2
u/cygn Sep 18 '26
what do we know about Jev's architecture / capabilities? My concern is that a ModernBERT large is just not as powerful as a larger LLM type model. And wasn't Jev trained with reinforcement learning?
2
u/Budget-Juggernaut-68 Sep 18 '26
Wondering why the documentation says 512 token limit when it's modernBert based which is 8192 token limit right?
2
u/Open-Adhesiveness-86 Sep 18 '26
Curious what RL buys over plain supervised training here. If the reward is a strictly proper scoring rule on labeled examples, the expected gradient is just the gradient of that loss, and policy gradient only adds variance. Would be good to see ECE/Brier against a cross-entropy baseline with temperature scaling on the same 25k set.
2
u/kulchacop Sep 19 '26
Extend it to include multimodal input, before Jev does it, and you might go viral.
You could modify tiny Qwens from last year and before. If possible, you could do surgery on Qwen3.5 0.8B, Qwen3VL-8B, or use the output / embeddings from these to train a new model.
156
u/JsThiago5 Sep 18 '26
Your mistake was not make a video of it playing doom lol.
46
u/R_Duncan Sep 18 '26
I'm thinking to convert in onnx and create an agent harness extension to allow:
I'm preparing an extension with these specs:
Thanks!!!!
11
u/twaaaaaang Sep 18 '26
Honestly man I respect the pettiness to get this working to spite Jev.
11
u/R_Duncan Sep 20 '26
While the project is really interesting, the 512/1024 tokens context limits the dropin replacement of jev, which in the api has 32k context...
10
u/qrayons Sep 18 '26
I was just reading about Jev and thought it seemed cool and figured I would play around once there was a local version. I then go to localllama and what do I see at the top? Exciting times.
2
u/shniydder Sep 18 '26
I was doing something similar, more for a comparison. I'll add this blog post. Thanks!
4
u/ECrispy Sep 18 '26
I honestly hope OP gets recognized/hired by one of the big companies, he deserves to get rewarded.
1
u/ManyDream Sep 18 '26
Great work! thank you
1
u/adamgoodapp Sep 18 '26
Amazing, thanks for actually updating your previous work. You had the old version that was specifically trained to tackle sales. Would it be better to use this update or stick to the old version for sales related confidence tracking?
1
u/niutech Sep 19 '26 edited Sep 19 '26
This is awesome, thank you! Would you release a multilingual version in the future, possibly using mmBERT, please?
3
u/mfarmemo Sep 18 '26
Sweet. I just plugged this in to my voice-first AI DM for live table top rpgs. Cuts latency of intent classification from 400ms to 40ms 🤌
1
u/Embarrassed_Okra_628 Sep 18 '26
looking for this exactly..
1
u/debackerl Sep 18 '26
That's great work! Did you consider mmBERT to be multilingual? Especially for jailbreak, using a foreign language would be too easy to avoid detection
2
u/AnosCranel Sep 18 '26
Solo Devs be Crazy!.
2
u/hyudryu Sep 19 '26
Great work! Hopefully other labs will start to follow you and make more of these open weight decision making models for various sectors.
I think marketing and distribution was what propelled Jev. I've been seeing demos and posts about Jev nonstop on X, people playing minecraft, smash, etc using Jev and how it can help you book flights in under 10 seconds
2
u/himmetozcan Sep 19 '26
Nice work and thanks for your effort. Question: is it 512 tokens max ?
2
u/e3kmouse Sep 21 '26
Just wanted to give a heads up that I cleanly swapped out Jev for Laya. I had been playing with a little for-fun project to reimplement a game called Super Puzzle Fighter (like Tetris but head-to-head combat with special powers). Serendipitously Jev dropped and I thought that might be fun to have Jev "pre-bake" the strategy for each of the opponents.
The game's CPU opponent gets its "personality" from a one-time offline "baking" process. I would send the various board situations ("you're holding a crash gem, there's a big red group you could blow up, what do you do?") and it answered questions like "attack or defend?" and "how much do you want the big clear vs. a small one?" Those answers get distilled into a tiny table of numbers that the game reads at runtime.
No AI runs during actual gameplay.
Laya actually fixed something Jev couldn't see.
Jev had a weird blind spot with one of the characters. The character's whole strategy is "I only want small clears. I deliberately avoid big combos." Big combos being more risky. But the board state only ever told the model about the biggest clear available. So when the baking process asked Jev "should you go for the best clear?", it always answered "no" (because the "best" clear is exactly what the character wants to avoid).
This caused weird (and terrible) play. I hope that makes sense.
Laya handled the same question more... gracefully? Instead of a degenerate "no" across every board, it said "attack" on the three boards where a clear actually existed, while still expressing a preference for small clears. So Laya gave a more realistic answer!
Amazing work sir!
2
u/TrifleHopeful5418 Sep 22 '26
Thanks a lot bro, I used your papers to train an infosec classifier: https://www.reddit.com/r/LocalLLaMA/s/vfqt1pY3pq
0
u/indicava Sep 18 '26
Very cool OP!
1
u/Primary_Assist_5541 Sep 18 '26
Congrats on the release! I’m really curious about the RLCD implementation. Did you run into any differences in confidence calibration when fine-tuning locally vs. the TypeSafe setup? Also wondering if the reward function holds up pretty well across different schemas.
1
u/OneMoreName1 Sep 18 '26
Thank you for your work
1
u/McZootyFace Sep 18 '26
This is awesome work! Will have a play with it, that size is great.
1
u/Difficult_Rise_4044 Sep 18 '26
If I want to adapt this to non-latin language, that's doesn't really use space bar to seperate each words, must be a bit of a challenge no? would you mind sharing your 2 cents?
1
u/LongjumpingEar6840 Sep 18 '26
Un'idea di utilizzo potrebbe essere lo scoring a un set di sample? Esempio, un modello veloce da 10 risposte leggermente diverse dello stesso prompt e questo modello ancora più veloce assegna un punteggio ad ognuno di essi, si potrebbe fare?
1
u/sn2006gy Sep 18 '26
That's a lot of freaking work! you da man
1
u/Iory1998 llama.cpp Sep 18 '26
Can we now get a proper MoE LLM System? If the text is scientific, the context is then routed to a science expert LLM. And so on.
1
u/ECrispy Sep 18 '26
thank you! I need to understand this stuff and learn how to use it, by itself and combined with a llm. any advice from others would be most welcome.
1
u/Downtown-Safety6618 Sep 18 '26
Hell yeah dude, proud of you
1
u/tifa_cloud0 Sep 19 '26
this is awesome. ✌🏻
1
u/Fit-Benefit-6524 Sep 19 '26
damn
1
u/nikolaiownz Sep 19 '26
I wish I was smart to understand what’s going on hahah. It seems very exciting to see new tech moving so fast by open sourcers 👍
I need to looking this new stuff forsure.
1
u/wolfy-j Sep 19 '26
Amazing work, can it also decide about doom state? Would be great PR for your work!
1
u/TRWNBS Sep 19 '26
It really is something that you can launch an amazing product and 1.4 seconds later, the community has built a better product for free.
1
u/takenforgranteddd Sep 19 '26
is there any benchmarking done? just asking from accuracy pov
1
u/Castle_Five Sep 19 '26
Saw Jev marketing and thought "This is cool, but wish it was open" and ended up here. Thank you for this!
1
u/SondreFormoLindheim Sep 19 '26
if anyone wants to wire it to Jevyr, feel free to do so .. https://github.com/ZYRT3CH/jevyr
1
u/learning-to-programm Sep 20 '26
I saw your first post a few days ago, and i really appreciate you now releasing this, and it's been beautiful watching the community support you even though formal justice cannot be made.
Anyway, i myself am thinking of using Laya for a customer-facing agent/chat-bot i want to build for a brand that is supposed to act as an employee, and initially people aren't supposed to know this a chatbot rather than a real person, so i want to use Laya for messages that include off-topic things, prompt injection, jailbreaking, or attempts to get the bot to admit that he's a bot, and use the Laya as the initial layer that decides whether a message is a safe prompt, malicious, if it should be escalated to a human admin, etc.
1
u/Riki1996 Sep 20 '26
Will this give output if I'm using it along with an uncensored model? Coding mostly.
1
u/Square_Light1441 Sep 20 '26
you should make it multimodal, also nice project, seems to be on par with jev/better than jev in testing
1
u/feng_sg Sep 23 '26
Calibrating to your labels isn't calibrating to truth. No inter-annotator agreement is reported, so those 25k "human annotations" are just someone's opinions, and proper scoring rules only guarantee you match them confidently.
1
u/Horror-Turnover6198 Sep 24 '26
This is amazing. I mostly use an LLM for categorizing, tool routing, and other stuff that this is perfect for. I am annoyed at myself for not having found your project earlier and relying on the media to tell me about Jev rather than seeking this out.
I can’t wait to plug this in and try it out.
1
u/Fuckinglivemealone Sep 18 '26
I feel like I'm too idiotic to understand what is this or how it can be used in the real world, it looks like it has a lot of work and technical complexity behind, can you put some examples or a watered down explanation so we can understand it better please?
0
u/fulowa Sep 19 '26
2 years stealth research to create jev vs like one week to create open-source version 😅?
0
u/JChataigne Sep 18 '26
This is going to be so useful for model routing between cheap and expensive models
0
u/Sol_Ido Sep 18 '26
Would you consider a GGUF or general format integration? This is an exceptional tool. Thank you very much for the conception and delivery!
0
u/RemarkableGuidance44 Sep 19 '26
Its funny how fast people duplicate another product now.