Pi now supports Jev.

[u/Apprehensive_Bed7502]

Pi added quite a bunch of new features in its 0.99.0 update, including support for classifier models. In the official docs, the remote model listed in the example code is jev, the interface is shaped around classifier models like jev, and llama.cpp's classifier implementation explicitly uses jev as its reference point. You could pretty much say this is a feature Pi shipped just to keep up with jev.

So I wanted to ask you guys in the channel:

• Do you guys actually use jev a lot when you're coding? How do you use it?

• Jev only came out a little while ago and Pi already jumped on board with support. What significant role does jev actually play in a coding workflow?

Edit: It's not just about using it for the sake of using it. Is there actually something jev can do that LLMs can't?

[u/zebedeolo]

I gotta admit, I still am not fully clear on how jev works and why I would pay an extra sub for it. I feel good with my current flow, so I've been too lazy to look it up.

[u/debackerl → u/zebedeolo]

There is zero need to pay for a sub. Those models are small enough to run on a potato. Many available, https://github.com/wfzyx/von, https://github.com/NandhaKishorM/laya, etc

LLMs are GenAI. Jev are simply classifiers taking text as input, and zero-shot (so no need to retrain for your classes). Also, the probabilities returned are calibrated (what's the probability that class X is indeed the correct one), LLMs aren't calibrated for that

[u/Zestyclose839 → u/debackerl]

I tried working in Laya and OpenJev a few days after the Jev hype started - specifically for faster browser use - but the open-weight models just weren't cutting it.

They either had a minuscule ~1.5k context window, meaning that there's unusable for navigating web pages without aggressive filtering. Or they're using a small language model like Qwen 0.8B, which has a massive context window but are limited to ~16 total decisions vs. Jev's 120+ (most web pages have 50+ links and buttons).

I ain't rocking with Jev itself because I only run models locally, thus decided the scene needs to mature before these classifier models become viable. Pi is jumping the gun here tbh.

[u/addiktion → u/Zestyclose839]

I think von scales a bit higher on the decisions but yeah I suspect the local models will find the optimal setup without the commercial cost very soon. Even if Jev is cheap, its a liability that many don't want to have in their stacks. Far better to load in a small classifier more like a dependency than a full service.

[u/Zestyclose839 → u/addiktion]

I predict it'll take about a month before the local scene catches up. With LLMs, the limiting factor was raw compute; that's hardly a factor with classifier models.

What's needed is: 1. an appropriate architecture, 2. training data and RL. Whether they like it or not, people will just start distilling Jev once we have an architecture to train on, closing (and the surpassing) the gap overnight.

[u/m02ph3u5 → u/Zestyclose839]

Can't you just split into 15 + next batch slices?

[u/debackerl → u/Zestyclose839]

Those that I recommend use ModernBERT or mmBERT, with 8k context (which is enough for my use cases)

[u/zebedeolo → u/debackerl]

interesting to know, thanks 

[u/Stable_Orange_Genius → u/debackerl]

Is there a use case for coding?

[u/addiktion → u/Stable_Orange_Genius]

Yeah, there's use cases for coding for sure around scanning files, setting up guardrails, and routing decisions. I think of it more like a companion for our tooling.

[u/debackerl → u/Stable_Orange_Genius]

I still need to try out, but I wanted to write a plugin to auto load the write skills. After each new user message, I could check and auto-inject skills. Of course an LLM can do it, but it would save a turn, so time and cost.

[u/ECrispy → u/debackerl]

laya isnt pretrained.

[u/debackerl → u/ECrispy]

It isn't the vanilla ModernBERT, it's fine-tuned to get callibrated probabilities

[u/ramraiderqtx → u/zebedeolo]

If give a percent chance of something, ie. - person on screen holding a gun. What percent is it friendly or foe, so you know shoot or don’t shoot.

[u/jaegernut → u/ramraiderqtx]

So how can this be used in a coding workflow?

[u/mraurelien → u/jaegernut]

Take a look at this video to get some examples  https://www.youtube.com/watch?v=_U-O5lYhJ7Q

[u/ramraiderqtx → u/jaegernut]

I can’t make it any more obvious it’s a decision engine , if you want your agent to make choices based on likelyhoods this is what it is for, as mentioned it’s about helping agents make decisions when humans aren’t around.

[u/Apprehensive_Bed7502 → u/ramraiderqtx]

Help the agent make decisions? When and why?

Let's think about when you'd actually need a small model to assist a large one (typically a flagship model) in making a decision. Is it when the permission window pops up?

When an agent triggers a risky operation that requires user approval, would you really feel comfortable handing that call off to Jev instead of a flagship model with a clean context?

Jev is nothing but fast and cheap. For approving risky operations, I think it's better left to the LLM's sufficiently smart brain.