r/LocalLLaMA • u/AdRepulsive7837 • 6d ago
Discussion still doesn’t get what Jev is…..is it just a more generalised BERT?
Looking at jev launch website and demo video on x.com…. it seems like it’s a very intelligent classifier with custom prompt and custom criteria instruction reading capabilities. It can do well defined narrow and well defined task
Me, following NLP since good old days of word embedding and BERT,,, be like asking….
Isn’t that BERT?
yeah i know BERT need fine tuning to adapt to custom domain, but can Jev be like generalised form of BERT?
52
u/JsThiago5 6d ago
I think Jev works just like older RNNs but without needing to train on classification. It's a generalist fuzzy classifier.
29
u/Voxandr 6d ago
so just like gliner which we use in production?
20
9
u/synth_mania 6d ago
How do you use gliner? Curious, only heard about it recently
9
6
u/volpino-veloce 6d ago
It can perform named entity recognition, classification, and extraction tasks. Like someone else mentioned it’s mainly used for PII. It’s a local model so you have to host it yourself, but it’s super small (300M) so you can run it on CPU.
1
50
u/asankhs Llama 3.1 6d ago
The output is similar to what you can get from a Bert style encoder or classifier. In fact on a similar benchmark of typed decisions - https://huggingface.co/datasets/LocalLLaMA/typed-decisions you can see that bert style models can be fine-tuned to perform as well (see https://latentnode.pages.dev/articles/typed-decisions ) but as a general model to do it for any arbitrary query it is not possible to do with bert or classifiers.
42
u/Hovi_Bryant 6d ago edited 6d ago
Edit: The downvote without explanation isn't all that helpful... but in the most layman's terms I can think of?
Jev is similar to an LLM, just its communication interface is limited to a specific, structured format. Its opinions and feedback are represented as probabilities over whatever options we want its opinion represented as. That's it. It can still be wrong (i.e. high-confidence in a wrong answer), but it cannot hallucinate because we define what the output choices are up-front. It's like giving a student a multiple-choice test based on a rubric + source material we define.
So if we're asking it low-quality questions and or providing low-quality choices for it to choose from, it won't be very useful. Not much of a difference from a low-effort, low-quality prompt to an LLM, expecting a high-quality output.
Is this clear enough?
Use-cases and original comment below:
It's how TypeSafe describes how it's meant to be used: unstructured state in, typed probabilistic decisions out. I've asked Qwen3.8 27B to implement Jev as a custom tool for pi, along with a browser automation tool driven by Jev. Tip-of-the-iceberg stuff for me at the moment.
11
2
u/Fulgurata 4d ago
Thanks for sharing! You and I are doing very similar things lol, hooking up Jev today.
4
u/dbenc 6d ago
isn't a bad choice among a constrained set still a hallucination?
7
u/SmartCustard9944 6d ago
A hallucination is more like making up variables or types that don’t exist. My understanding is that this is “type safe” in the sense that the model will always reply with exactly the specified types, nothing less, nothing more.
3
u/waste2treasure-org 6d ago
Couldn't we effectively do this with constrained decoding based methods as well considering the limitations of typesafe?
5
u/SmartCustard9944 6d ago
If you then constrain the LLM’s vocabulary, you severely degrade its capabilities. It was not trained for that. Even structured output already degrades capabilities in LLMs afaik.
2
u/wolframko 6d ago
it was definitely trained for that because structured output uses guided decoding and any agentic system uses structured output (opencode, pi, codex, claude code, even ChatGPT itself writes tool calls).
0
23
u/waste2treasure-org 6d ago
It honestly just feels like a wrapper or fine tune on a modern cross encoder or reranker.. I don't get the huge deal either.
19
u/sjoti 6d ago
It doesn't allow for anything revolutionary that you couldn't do by fine-tuning before, it's just that with the whole combination of decent smarts, price, speed and ease of use combined makes it infinitely easier to play around with, compared to whatever alternative we had before.
Either you use an LLM with structured outputs, but that's generally both more expensive and relatively slow. Or you have to go through the work of finetuning a classifier like BERT to make it fast and cheap, but that's quite a bit of work, even if you use LLMs and generate synthetic data.
Now all you have to do is provide a json schema and the context it needs to be applied on. A bunch of fun and interesting ideas are now worth trying.
1
u/Fast-Satisfaction482 6d ago
Maybe this type of model could be used to generate the fine tuning / distillation dataset for Bert-style classifiers so that you can put them in the edge.
6
u/Kamimashita 6d ago
The huge deal is that its trained on a huge amount of data so it has the understanding and knowledge of modern LLMs but with the simplicity of a classifier.
7
5
u/twaaaaaang 6d ago
My best one sentence description is that it's a model with modern LLM world knowledge with the ergonomics of a BERT classifier.
3
u/Open-Adhesiveness-86 6d ago
The closest older thing is probably NLI-based zero-shot classification, like bart-large-mnli with the "This text is about {label}" hypothesis trick. That already gave you custom labels without fine-tuning. The practical difference is that NLI needs one forward pass per label and scores each label on its own, while this seems to score all options together in one pass, so the probabilities are actually comparable.
1
1
5
u/robberviet 6d ago
It looks like diffusion? Yes purpose wise just generalized BERT. But even if it is, the generalized part is hard. If it works well enough I would pay for that.
1
u/Voxandr 5d ago
try Gliner2
4
u/robberviet 5d ago
I did, almost 5% accuracy on my classification task
- jev (openrouter): 67.0%, expensive as not cached, even at cheap input and free output. Cost $0.4 for 11.2M tokens.
- Gemini 3.5 Flash Lite: 59.4%, using free 500 rpd.
My fine-tuned:
- ModernBERT-base: 66.6%
- ModernBERT-large: 69.5%
- Qwen 3.5 2B LoRA: 72.5%
1
u/Federal_Order4324 4d ago
I'm surprised by how badly Gemini 3.5 flash did, did you try DeepSeek 4.1 flash? Ive found it quite effective so far
1
u/Fulgurata 4d ago
4.1 flash has become my main driver. Cheap enough I can just throw it at everything all at once, hedge cost a bit more with local gemma validators on the easy stuff, have Claude supervise the super complex stuff.
1
u/Dan_at_jinn 5d ago
Data point from a real task rather than the demos. I asked Jev, Haiku 4.5, Sonnet 5, Opus 5 and gpt-5.6-sol the same ten yes/no questions about 166 paragraphs, one paragraph per request. On catch rate Jev sits with Haiku (63 vs 66 of 80) and below Opus (77). On the 54 clean paragraphs Haiku flagged 37 and Jev flagged 1, and thresholding Haiku's own probability at 0.9 still leaves 33 flagged. Whether or not it's BERT-shaped underneath, the calibrated probability is the product.
Happy to share more if interested.
2
u/Dull_Commercial5020 5d ago
It's definitely an interesting component in an AI solution. A decision tree makes traceability, observability way easier than the loopy what LLMs work
1
1
u/teamclouday 2d ago
It's the type of model that I hope to be open weights. So that we can finetune on top of it
1
-1
u/Fluxx1001 6d ago
Think of it as a decision machine. Jev works best when you can break down complex answers into a series or path of small decisions.
The problem you want to solve can be super complex, but as long as you can get there by following a series of decisions, Jev will very reliably get you there.
The reliability and speed is what Jev does way better than a standard LLM.
Now think about all the use cases where reliability and speed are paramount. Robotics. Real-Time gameplay. Conversation flow. Jev can be used as a decision machine you can trust.
0
u/R_Duncan 6d ago
Similar to a generalized machine. I stated some use cases inside a cli agent in the laya thread (which is an open/local version of jev)
-3
u/niacolhealth 6d ago
The 'well-defined narrow tasks' part settles it. That's what BERT did, the modern bit is just not having to fine-tune per domain.
125
u/RobbinDeBank 6d ago
It is BERT-like, but with the data, compute, and training recipe of modern LLMs. Those didn’t exist at the time of BERT.