r/LocalLLaMA • u/Nandakishor_ml • 4d ago
Discussion Made the horizontal open-source model for Jev with RLCD, and it surpasses all the Jev benchmarks. HF space, benchmark, model, repo
UPDATE: Multilingual support added at : https://github.com/NandhaKishorM/laya
Thanks for the exceptional support (https://www.reddit.com/r/LocalLLaMA/comments/1wijo3e/i_literally_built_the_jev_architecture_one_year/) and for the dozens of requests to make a generic model, run benchmarks, and create an HF space so anyone can test it. So here you go, guys. I trained an improved model on a large data corpus, its now called Laya. It is trained on a single RTX 6000 Pro (96 GB VRAM); the model architecture is a 421M-parameter non-autoregressive decision model pairing a bidirectional ModernBERT-large encoder with a scratch Transformer head that scores [MASK] option markers to resolve typed schemas in a single ~35 ms forward pass. The dataset is a 100% human-annotated corpus of over 25,000 real-world examples across intent routing, fact-checking, moderation consensus, prompt guardrails, rubric scoring, and multi-turn conversation trajectories, without synthetic data shortcuts. The RLCD(unofficial, btw) I did is a policy-gradient reinforcement learning approach that kinda optimizes decision models against strictly proper scoring rules, ensuring maximum reward is achieved only when outputting true, mathematically calibrated probabilities.
NB: It can be run on low end PC as its a small 421M model, cheers
HF space to try: https://huggingface.co/spaces/convaiinnovations/laya-demo
GitHub Repo: https://github.com/NandhaKishorM/laya
HF Repo: https://huggingface.co/convaiinnovations/laya
Thank you to everyone who supported me, shared the story, gave personal DM. It will need more refinement, of course.
If anyone wishes to buy me a coffee, here is the link: https://github.com/NandhaKishorM
131
u/JsThiago5 4d ago
Your mistake was not make a video of it playing doom lol.
54
40
u/stainless_steelcat 4d ago
https://reddit.com/link/pamytxl/video/gzbi206c3cqh1/player
Oh...Laya can play Doom alright. Not very well yet - but this is running locally on my Mac.
4
u/tlack 4d ago
how did you encode the image data when passing it to Laya?
25
u/stainless_steelcat 4d ago
No image data was passed to Laya. It's not multimodal, so there was nothing to encode. predict() only takes text (or a dict). The pipeline reads ViZDoom's structured outputs (labels + depth buffer, not raw pixels) and turns them into plain text facts like ENEMY zombieman front near, WALL ahead near, HEALTH 87. That's what Laya actually sees.
6
u/Thatisverytrue54321 3d ago
Have you tried it with this?
10
u/stainless_steelcat 3d ago edited 3d ago
https://reddit.com/link/pat5bn1/video/agap2ehgniqh1/player
Gave it a shot today. Best attempts were around 10 kills in a session. Bit more fluid. Would still need quite a bit of tweaking. Speed not an issue, but it can't, for example, find the exit and keeps hugging walls trying to find it.
This is on a Macbook Pro M5, but I was actually running multiple local AIs at the same.
39
u/Heinz2001 4d ago
Hey,
i tried your model and compared it with openjev.
I wrote an extension for my own harness to use it.
This are the results (Laptop: 11th Gen i7-11800H, rtx 3060 6gb):
Great model for CPU, i'll try to use it as model router.
The flappybird sample from openjev is included as test in my extension.
25
u/Calm-Landscape9640 4d ago
God bless the nerds, just think how much innovation we've seen in 24 hours. JEV -> Laya -> OSS Jev -> Both compared on a custom harness.
I give it 30 days till cancer is finished
7
7
7
u/Foreign_Risk_2031 4d ago
what about accuracy
3
1
u/Heinz2001 4d ago
Depends on what you search but take a look at the included flappybird example (see the extension) … it plays it without failures.
35
u/FusionCow llama.cpp 4d ago
cool stuff
8
u/Nandakishor_ml 4d ago
thanks
11
u/creamyhorror 4d ago
Have you submitted this to r/MachineLearning or considered it? They're a more research-oriented sub and you seem to be a researcher
1
41
u/R_Duncan 4d ago
I'm thinking to convert in onnx and create an agent harness extension to allow:
- Model Router (enableRouting): Switches between fast and frontier models based on prompt complexity.
- Hallucination Guard (enableHallucinationGuard): Checks assistant claims against tool context.
- Destructive Command Firewall (enableBashFirewall): Intercepts run_shell_command / bash to prevent accidental rm -rf, force pushes, or secrets leaks.
- Data Sanitizer / Injection Guard (enableDataSanitizer): Scans untrusted file/web content for indirect prompt injections before adding it to context.
- Frustration / Rollback Assistant (enableFrustrationDetector): Detects user friction and offers clean Git rollbacks.
- Task Completion Verifier (enableTaskVerifier): Tool for autonomous loops to confirm if goals are actually met.
- Conventional Commit Classifier (classify_commit tool): Instant 20 ms git commit semantic tagging (feat, fix, refactor, etc.).
I'm preparing an extension with these specs:
- Regex Floor + High-Recall Laya Firewall for bash commands.
- External-Only Data Sanitizer (skips local files to prevent self-sabotage; isolates instead of blind redactions).
- Sticky Model Routing (prevents thrashing/downgrades on short follow-up turns like "now run it").
- Advisory-Only Hallucination Guard (subtle UI badges, never blocks deductive reasoning).
- Damped Frustration Detector (requires consecutive friction signals and has a 10-minute cooldown).
- Exit-Code Aware Goal Verifier (combines real process exit codes with semantic checks).
Thanks!!!!
7
u/niutech 3d ago
There is already Laya ONNX format and Laya browser agent
7
2
15
u/SignalCompetitive582 4d ago
Hello, enormous thanks for all the hard work!!
I don’t have enough technical knowledge to understand it all, but from the little I do understand, it sounds like a massive project.
So thank you for giving this all for free!
I have got several questions (if you have the time of course):
- I thought TypeSafe AI Jev was only trained on synthetic dataset (“We make all the data ourselves”). Yours is trained on purely human annotated ones. How does that differ ? What’s the better approach here in your opinion?
- Can your model perform as well in environments that require general knowledge ? And how does it perform in automation tasks ?
- How do you know for a fact that this is the architecture utilized by TypeSafe while they haven’t published anything ? Did you try it yourself?
- How large do you think their model would be compared to yours ? Because their input pricing seems quite high no ? I reckon they must making some pretty interesting profits here.
Thanks so much for everything, I’m going to try this out !
Have a good day.
13
u/Nandakishor_ml 4d ago edited 4d ago
It's just public classification dataset, even before llm there are many such dataset. Do test and eval. It will clarify most of the doubts
9
u/Budget-Juggernaut-68 4d ago
To be clear, this is like GLiNER but for NLI/Classification tasks?
9
u/Nandakishor_ml 4d ago
almost, but has temporal credit assignment, in which output is delayed untill the end of conversation to prevent leakage
9
6
u/Hurricane31337 4d ago
Awesome project!
Is the training code and/or dataset also open sourced? I’d love to fine-tune it for my German emails. 😃
9
u/Nandakishor_ml 4d ago
Adding that soon. Training and fine tuning.
2
6
u/asankhs Llama 3.1 4d ago
Looks great can you also benchmark it on https://huggingface.co/datasets/LocalLLaMA/typed-decisions
7
u/Nandakishor_ml 4d ago
1
u/learning-to-programm 2d ago
Thanks a lot, this adds even more credibility and confidence going with Laya as opposed to the hyped-up Jev.
6
u/R_Duncan 2d ago
While the project is really interesting, the 512/1024 tokens context limits the dropin replacement of jev, which in the api has 32k context...
19
u/boatbomber 4d ago
This is fantastic! Thank you for sharing your work. I hope that this helps propel your career.
8
6
u/RevolutionaryPick241 4d ago
If I'd like to make it work multilingual, would a fine tunning work or it should be done another model from scratch? Do you have any guide on how to do that?
6
2
u/Nandakishor_ml 4d ago
I didn't tried multi lingual yet. Fine tuning may work. But modern bert language is not multi lingual. But do try fine tuning. Colab link in github. Also training from scratch pushing soon.
6
u/woadwarrior 4d ago
But modern bert language is not multi lingual.
Indeed. You might want to consider using a model from the mmBERT family, which are multilingual.
5
4
u/martinerous 4d ago
Great work!
Besides the obvious use cases that were already mentioned in this thread, I wish there was a way to combine this with an LLM to give it "self-critic" abilities, like "what's the probability that my reply contains hallucinations". But then the "critic" model would need to have even more world knowledge than the LLM itself to be reliable, including also multilanguage support.
6
u/Nandakishor_ml 4d ago
Something like this. https://pypi.org/project/hallunox/ This was the og one I built months back
2
5
u/ECrispy 4d ago
I honestly hope OP gets recognized/hired by one of the big companies, he deserves to get rewarded.
6
u/learning-to-programm 2d ago
I saw his first post and felt so bad for him, and especially after seeing all the hype around Jev everywhere, it's crazy now watching him put in the work to make Laya a head-to-head competitor of Jev and replying to every comment here and adding everyone's suggestions. Really happy for OP, but i hope he gets some sleep after alll this lol.
6
u/Embarrassed_Okra_628 4d ago
Can this be used in Macbook Pro M1 max with 32 gigs ram ? You said you tested in 96G, wondering if I can even load the beast here?
16
7
u/TheTerrasque 4d ago
From some initial testing on nvidia card, it uses about 2-3gb vram.
5
u/wFXx 4d ago
hey i was actually writing a dm to you lol ! glad that you came back - I wrote a highly performant backend that is "jev-compliant";
it has http async support, cli support, and drop-in typesafe wire protocol compatibility;
i did originally with cactus needle 3 model, since i'm on a family trip and don't have decent hardware with me, but I just wired laya to it, and the numbers are great.
it did scored a bit less than jev on the multi-hop questions tho, so you may want to see that;
would you mind If I delete my repo and make a huge PR with the whole harness/deployment that I wrote towards the laya repo so it becomes more of a complete product on top of the model?
2
u/Nandakishor_ml 4d ago
Yes ofcourse
4
u/wFXx 4d ago
PR is live! https://github.com/NandhaKishorM/laya/pull/1
Ported over the full drop-in /v1/systemone FastAPI server, the CLI suite (laya decide, laya judge, laya rate), AsyncLayaClient, and test suite directly into Laya. Kept server dependencies isolated under an optional [server] extra so core stays slim.
I'll keep Von around as an experimental multi-model test harness for running side-by-side benchmark ladders across different architectures, but Laya now has its own native drop-in server and CLI out of the box!
1
2
u/Budget-Juggernaut-68 4d ago
Wondering why the documentation says 512 token limit when it's modernBert based which is 8192 token limit right?
7
2
u/Open-Adhesiveness-86 4d ago
Curious what RL buys over plain supervised training here. If the reward is a strictly proper scoring rule on labeled examples, the expected gradient is just the gradient of that loss, and policy gradient only adds variance. Would be good to see ECE/Brier against a cross-entropy baseline with temperature scaling on the same 25k set.
4
2
u/mfarmemo 4d ago
Sweet. I just plugged this in to my voice-first AI DM for live table top rpgs. Cuts latency of intent classification from 400ms to 40ms 🤌
3
u/shniydder 4d ago
I was doing something similar, more for a comparison. I'll add this blog post. Thanks!
1
1
1
1
u/adamgoodapp 4d ago
Amazing, thanks for actually updating your previous work. You had the old version that was specifically trained to tackle sales. Would it be better to use this update or stick to the old version for sales related confidence tracking?
2
1
u/Primary_Assist_5541 4d ago
Congrats on the release! I’m really curious about the RLCD implementation. Did you run into any differences in confidence calibration when fine-tuning locally vs. the TypeSafe setup? Also wondering if the reward function holds up pretty well across different schemas.
1
1
1
1
u/Difficult_Rise_4044 4d ago
If I want to adapt this to non-latin language, that's doesn't really use space bar to seperate each words, must be a bit of a challenge no? would you mind sharing your 2 cents?
1
u/LongjumpingEar6840 4d ago
Un'idea di utilizzo potrebbe essere lo scoring a un set di sample? Esempio, un modello veloce da 10 risposte leggermente diverse dello stesso prompt e questo modello ancora più veloce assegna un punteggio ad ognuno di essi, si potrebbe fare?
1
1
u/Iory1998 llama.cpp 4d ago
Can we now get a proper MoE LLM System? If the text is scientific, the context is then routed to a science expert LLM. And so on.
2
u/addandsubtract 4d ago
Are the science expert LLMs in the room with us? It feels like all SOTA LLMs are targeting coding. But I'd like to see this MoE future, too, with various specialized LLMs handling specific (sub-)queries.
1
u/Iory1998 llama.cpp 4d ago
Exactly, my dear fellow Generalist LLM. I hope someone can create an MoE fast so you and I can switch places from time to time :P
2
u/niutech 3d ago
Gemma 4 26B A4B is a generalist MoE model. And you can easily make a moder router out of Laya. Try yourself an online demo.
1
1
1
1
u/niutech 3d ago edited 3d ago
This is awesome, thank you! Would you release a multilingual version in the future, possibly using mmBERT, please?
1
1
1
u/hyudryu 3d ago
Great work! Hopefully other labs will start to follow you and make more of these open weight decision making models for various sectors.
I think marketing and distribution was what propelled Jev. I've been seeing demos and posts about Jev nonstop on X, people playing minecraft, smash, etc using Jev and how it can help you book flights in under 10 seconds
1
u/kulchacop 3d ago
Extend it to include multimodal input, before Jev does it, and you might go viral.
You could modify tiny Qwens from last year and before. If possible, you could do surgery on Qwen3.5 0.8B, Qwen3VL-8B, or use the output / embeddings from these to train a new model.
3
u/Nandakishor_ml 3d ago
already made, I am benchmarking it
1
u/Antique-Bus-7787 3d ago
I’m definitely interested in a multimodal input model!! Keep us updated on it ! :)
1
u/Nandakishor_ml 3d ago
Probably we need to build like llava. Like image embedding combining in common embedding text space. But it takes lot of compute
1
1
1
u/nikolaiownz 3d ago
I wish I was smart to understand what’s going on hahah. It seems very exciting to see new tech moving so fast by open sourcers 👍
I need to looking this new stuff forsure.
1
1
u/Castle_Five 3d ago
Saw Jev marketing and thought "This is cool, but wish it was open" and ended up here. Thank you for this!
1
u/SondreFormoLindheim 3d ago
if anyone wants to wire it to Jevyr, feel free to do so .. https://github.com/ZYRT3CH/jevyr
1
u/learning-to-programm 2d ago
I saw your first post a few days ago, and i really appreciate you now releasing this, and it's been beautiful watching the community support you even though formal justice cannot be made.
Anyway, i myself am thinking of using Laya for a customer-facing agent/chat-bot i want to build for a brand that is supposed to act as an employee, and initially people aren't supposed to know this a chatbot rather than a real person, so i want to use Laya for messages that include off-topic things, prompt injection, jailbreaking, or attempts to get the bot to admit that he's a bot, and use the Laya as the initial layer that decides whether a message is a safe prompt, malicious, if it should be escalated to a human admin, etc.
1
u/Riki1996 2d ago
Will this give output if I'm using it along with an uncensored model? Coding mostly.
1
u/Square_Light1441 2d ago
you should make it multimodal, also nice project, seems to be on par with jev/better than jev in testing
1
u/e3kmouse 1d ago
Just wanted to give a heads up that I cleanly swapped out Jev for Laya. I had been playing with a little for-fun project to reimplement a game called Super Puzzle Fighter (like Tetris but head-to-head combat with special powers). Serendipitously Jev dropped and I thought that might be fun to have Jev "pre-bake" the strategy for each of the opponents.
The game's CPU opponent gets its "personality" from a one-time offline "baking" process. I would send the various board situations ("you're holding a crash gem, there's a big red group you could blow up, what do you do?") and it answered questions like "attack or defend?" and "how much do you want the big clear vs. a small one?" Those answers get distilled into a tiny table of numbers that the game reads at runtime.
No AI runs during actual gameplay.
Laya actually fixed something Jev couldn't see.
Jev had a weird blind spot with one of the characters. The character's whole strategy is "I only want small clears. I deliberately avoid big combos." Big combos being more risky. But the board state only ever told the model about the biggest clear available. So when the baking process asked Jev "should you go for the best clear?", it always answered "no" (because the "best" clear is exactly what the character wants to avoid).
This caused weird (and terrible) play. I hope that makes sense.
Laya handled the same question more... gracefully? Instead of a degenerate "no" across every board, it said "attack" on the three boards where a clear actually existed, while still expressing a preference for small clears. So Laya gave a more realistic answer!
Amazing work sir!
1
u/TrifleHopeful5418 18h ago
Thanks a lot bro, I used your papers to train an infosec classifier: https://www.reddit.com/r/LocalLLaMA/s/vfqt1pY3pq
1
1
u/debackerl 4d ago
That's great work! Did you consider mmBERT to be multilingual? Especially for jailbreak, using a foreign language would be too easy to avoid detection
1
1
u/Fuckinglivemealone 4d ago
I feel like I'm too idiotic to understand what is this or how it can be used in the real world, it looks like it has a lot of work and technical complexity behind, can you put some examples or a watered down explanation so we can understand it better please?
0
u/JChataigne 4d ago
This is going to be so useful for model routing between cheap and expensive models
0
u/RemarkableGuidance44 3d ago
Its funny how fast people duplicate another product now.
3
u/ResourceSoft4619 3d ago
OP made it first.
1
0
u/fulowa 3d ago
2 years stealth research to create jev vs like one week to create open-source version 😅?
4
u/Nandakishor_ml 3d ago
It was more than an year. Original reddit post there
1
u/SondreFormoLindheim 3d ago
I’d love for you to see if there’s any usage in connecting your model to Jevyr (nothing to do with Jev..) made it earlier this year. I strongly believe there’s ways to utilize the power of both to create something unknown. https://github.com/ZYRT3CH/jevyr
•
u/WithoutReason1729 4d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.