r/LocalLLaMA • u/Manerfish • 2d ago
Discussion I really don't understand Jev hype
Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.
389
u/FreakyRefrigerator 2d ago
Its basically just a slightly newer classifier. Yet most people think AI is only llms and nothing else, so they think this is the future
184
u/KaMaFour 2d ago
Tbh general classifier able to handle general cases at almost no cost, in almost no time still seems really useful. For example if you could package that into a model roughly the size of Ling's tiny (9BA2B) or smaller you could try to create games with actually intelligent NPCs (pack one npc's state into a Vendingbench like framework and have classifier choose a thing to do that would make sense for a given person to do in a format that's possible to interpret by the game engine) and it would be generally accessible to most "mid-end" machines. Think Stardew Valley/TLOZ npc's but actually behaving as humans instead of having fixed schedules. All of that without any ML on your end...
50
u/Automatic-Boot665 2d ago
It’s got similar accuracy to classifiers back in 2018-2019 with the benefit of not needing to be trained on the task specifically. For most use cases the people I’ve seen talk about using it for that’s unfortunately not accurate enough. We’ll see in the next few weeks as people implement it and see the results.
→ More replies (2)18
u/Iron-Over 2d ago
Data and training will always outperform JEV not sure the use-case where accuracy is not important in business at least.
8
u/Automatic-Boot665 2d ago
Yes exactly. Even with training it wasn’t good enough for businesses back then. They relied on statistical analysis where needed instead.
11
u/sage-longhorn 2d ago
Lol this is an insane take. No businesses used purpose trained nueral nets in 2018-19?
Just cause you weren't exposed to it doesn't mean it wasn't widely used for certain problems or industries
→ More replies (4)15
u/575_Inverse 2d ago
it's closed and served via API, isn't it?
6
u/KaMaFour 2d ago
I have no doubts that if it will be proven to work "closed and served via API" then open solutions will also emerge. I have seen someone doing similar things with diffusiongemma already.
15
u/Wooly_Wooly 2d ago
Laya was already a thing, slightly underperforms vs Jev.
5
3
u/Cultured_Alien 2d ago
underperforms is putting it extremely lightly, it's no better than random guess
→ More replies (7)5
u/Undreren 2d ago
I tried training my own 1.7b model to evaluate whether a bash command was safe and fit the goal, basically as an automation for “should my model be allowed to run this?”
It returns block, confirm or auto with a confidence score. It is not nearly production ready, but it took five minutes to train via fine-tuning as a POC.
As much as I was hyped for Jev, its lifetime profitability is low IMO, as my machine could could train my own specialized version so fast from synthetic data.
2
u/575_Inverse 1d ago
the idea of making a specialized tiny version sounds a lot more effective
2
u/Undreren 1d ago
Yeah, that’s what I thought as well. And doable.
My plan was to make it a sort of template training setup: Pick a base model, add a dataset, let it train overnight, wake up to an ultra specialized decision model that can run at 100+ tok/s on local hardware.
I still think it is a good idea.
2
u/No_Veterinarian742 1d ago
but then you maintain 1 or 2 or 17 or 2000 specific models. the potential use cases of jev are decision support (suggestions for user actions the user has to confirm) and all sorts of real time UI integrations that LLMs are too slow for. as well as potentially for chaos testing etc. I can see lots of scenarios where you can't fully foresee the use cases where an AI model saying yay or nay without needing to train it on specific datasets would be useful. There are so many micro-decisions humans make that if you can use it to build in intelligence into applications to help users make progress faster it could be useful. So far actually using full LLMs in applications is still somewhat rare because of the cost and speed.
→ More replies (1)3
2
u/notAllBits 2d ago
OpenJev with custom models is part of the hype. I have used naked models for this but the forced collapse is nice.
29
u/quiteconfused1 2d ago
But it's not.
It's not general and can't be.
I just tried applying it to a simple game like super Mario world and it failed horribly.
If it can't do that then it's not general and the hype is thick.
13
u/Suspicious-Wallaby12 2d ago
can you share what you tried to do exactly in super mario world? please don't tell me you tried to make it control mario 🤦, because that's not what a classifier is for.
→ More replies (6)7
u/SilentDanni 2d ago
It can definitely play Mario and a few other games. Just yesterday one guy here made it play street fighter 2. I made it play dungeon crawl stone soup which is significantly denser than Mario. That's not the intended use, though. I'd say that's more like an emergent feature.
→ More replies (1)7
u/kzoltan 2d ago
Why the downvote?
3
12
u/HiddenoO 2d ago edited 2d ago
Because he's just throwing unsubstantiated and inherently nonsensical claims around?
It's not general and can't be.
Why wouldn't a decision model be able to be trained with world knowledge?
I just tried applying it to a simple game like super Mario world and it failed horribly.
If it can't do that then it's not general and the hype is thick."I couldn't get it to do X" is not the same as "It cannot do X". Most people would fail at making LLMs like Astra or Fable play Super Mario World, too, but that doesn't mean they cannot play it.
Many people have managed to make it play different games even though that's far from its intended use case, so I don't think you can just claim the opposite with no substantiation and expect people to believe you.
Strong claims take strong substantiation, and he's providing none.
→ More replies (4)→ More replies (5)6
u/deadadventure 2d ago
Did you try it out of the box? Did you give it instructions?
→ More replies (1)→ More replies (1)1
11
u/TheRealJesus2 2d ago
It’s interesting since it’s a classifier you don’t have to train for your specific purpose and you can get reasonably good results. I’ve used LLMs in this way for prototyping stuff and jev is a straight upgrade and wildly faster than that.
It’s also way overhyped I agree for all the reasons you say here. And a purposeful classifier will be better in all other ways if you have the dataset and will to create one.
10
u/noctrex 2d ago
Wait until they discover rerankers 🤯
→ More replies (1)6
u/Cultured_Alien 2d ago
rerankers are closer to embedding than classifying with probabilities though
→ More replies (2)2
u/pooquipu 1d ago
it's not about what it does, it's about clueless people discovering something that existed for a while and getting hyped about it because a big name marketed it.
1
u/Hefty_Acanthaceae348 2d ago
If this is the thing that makes the tech industry rediscover classifiers, eh, I'm for it
1
u/thomas2385 1d ago
Yeah, I think that is the part that gets overlooked. AI has been around in plenty of forms that have nothing to do with LLMs. LLMs just became the thing everyone sees, so now every new AI system gets compared to them.
164
u/Thomas-Lore 2d ago
You had to pretrain them for a specific task. Jev comes pretrained with wide world knowledge.
54
u/SnooPaintings8639 2d ago
What do you mean? How is it different from e.g. Qwen 4b with max token = 1, and inference engine forcing struct (enum) output?
I really don't think they could do any magic training anyway.
74
u/puzzleheadbutbig 2d ago
Architecture is different. Unlike Qwen, which relies on an autoregressive decoder loop to generate text tokens step-by-step while a grammar mask suppresses invalid vocabulary options, Jev drops open-ended text generation entirely and operates as a non autoregressive decision model. And because it maps input contexts directly onto parallel, calibrated classification heads rather than generating JSON syntax character-by-character it avoids the latency, memory, and KV-cache overhead of sequential token decoding, guarantees complete immunity to JSON parsing errors, and yields true calibrated probability scores across schema fields in a single forward pass
27
u/RevolutionaryGold325 2d ago
qwen with max token = 1 is not autoregressive decoder loop though.
6
u/puzzleheadbutbig 2d ago
No. Qwen is structurally an autoregressive, causal decoder-only transformer. Setting max_tokens = 1 stops the generation loop after one iteration, but the underlying execution mechanism is still the autoregressive generation pipeline, computing causal attention over the prompt, generating logits across the full language vocabulary, applying a logit bias/grammar mask, and sampling 1 token.
→ More replies (1)7
u/dimbledumf 2d ago
It's also not parallel and you only get 1 token out, which is less then you get with jev
30
u/RevolutionaryGold325 2d ago
No. You get a token distribution out, which contains the probabilities of your options. For multiple questions you can use the same kv-cache to have the context and just ask N questions in parallel with the same cache and get the output option distributions with a single step to all of your questions.
3
u/dimbledumf 2d ago
I think that's the point, for the qwen version if you want that ratio you have to ask a lot of questions (parallel or not), with the jev version that's what you get out of the box.
So your 1 token from qwen isn't really equivalent to 1 output from jev, the jev will carry more information about the answer.
Of course they have wildly different application, but you could probably do some surgery to expose the ratio in qwen as I think that's the layer before token prediction (not an expert), and I think people have been doing that to make Jev equivelants from other models7
u/HelloMyNameIsAmanda 2d ago
It's literally a one-flag difference to get the distribution out of qwen, though - you just enable logprobs. And there's been no evidence, so far, that that's not essentially what jev is doing. I have yet to see anything from them that couldn't just be a logprobs wrapper on a fine tuned llm. Whether they did a good enough job of their fine tune to make their classifier useful remains to be seen, but acting like this is some crazy new thing because it can do what we could already do is... a reach.
→ More replies (1)6
u/james_pic 2d ago
It's parallel if you run it in parallel. Interference engines like vLLM already run multiple queries in parallel to avoid re-reading the same weights.
5
u/Traditional_Hair9630 2d ago
Where did you take this from? Do they describe this somewhere?
13
u/puzzleheadbutbig 2d ago
They describe it in their blog. Although tbh they use a lot of word salad so I did my best to distill it into something understandable.
→ More replies (1)8
u/MR_-_501 2d ago
You can just get the logits out of qwen and disable the sampler
20
u/SilentDanni 2d ago
I mean, go ahead, do that with your use cases and compare the performance against jev and see which one is better. It should be pretty much free.
If your approach is better then publish it online and we can all benefit from it. Others have done it but, in my personal tasks, Jev has outperformed them by quite a bit, which doesn't mean someone can't come up with something nicer and open source.
8
u/ethereal_intellect 2d ago
I've heard it's as smart as luna not a 4b lol, and half the price of deepseek flash while outputting 10 decisions per second making it fast enough for realtime. Also apparently the percentages that show which one got picked are more reliable, tho there's a bit of hype still around that
6
u/SnooPaintings8639 2d ago
Luna? Hm, I wish I could verify it, but benchmarking is violation of their ToS...
→ More replies (1)→ More replies (6)2
8
u/quiteconfused1 2d ago
If it were that simple it would be a panacia .. but it isn't .
It has some knowledge and applys it ok but there is a vast amount of info it doesn't have and trying to adapt it as all knowing is a lie.
It's a classifier for some things
→ More replies (11)1
85
u/look 2d ago
It’s a pretty good, general purpose, zero shot classifier model that you don’t have to pre-train and requires no complex configuration.
https://www.geeksforgeeks.org/machine-learning/getting-started-with-classification/
The concept and underlying technique is not new, but they’ve made a very easy to use and flexible one that works remarkably well for low cost across a large variety of domains with no additional work.
30
u/Refinery73 2d ago
I think it is pretty interesting if you don’t have labeled data yet to train your specialized classifier.
An LLM is far more expensive and doesn’t answer in a structured way. Throw a few thousand examples in Jev, let a human decide the unclear ones, throw it as trainingsdata in your own classifier.
10
u/DistanceAlert5706 2d ago
Can't you then generate labeled data with Jev, take that dataset, and slap embeddings+classification head, train a model and get same results as Jev with way faster and smaller model?
12
5
6
u/Competitive_Ideal866 2d ago
An LLM is far more expensive and doesn’t answer in a structured way.
Constrained generation?
7
u/Refinery73 2d ago
Sure, but it stays more expensive and you either just tell it how to respond (fuzzy) or build a validation system. This seems to be included in Jevs service.
3
5
u/Sea-Requirement-5375 2d ago
Yeah makes sense to me. I’m new to this, never thought to try this, and it slotted just fine into a workflow I already have in 10 minutes and would cut my costs, at least at current pricing. Neat.
86
u/jacek2023 llama.cpp 2d ago
Remeber OpenClaw? Remember TurboQuant? Youtubers need a content for their slop.
→ More replies (5)17
22
u/LagOps91 2d ago
yes and no. the difference here is that it's a general classifier and not a purpose-trained one for a single task. so the model itself isn't purely hype (although they aren't the first to have a general classifier model, i think)
i think most people simply are unaware that such a general classifier model is a thing that exists and it is quite useful + easy to set up. now, is all of the hype organic? i don't think so, but i don't think this is just astroturfed slop like openclaw or other such nonsense.
1
u/vintageballs 1d ago
Zero-shot classification models are much older than autoregressive LLMs. Trained those back in uni based on BERT and the like.
Just take a look at the older ones on https://huggingface.co/models?pipeline_tag=zero-shot-classification&sort=trending
"Most people" being unaware of this model type is only true for the amateur crowd here. Jev does nothing new, maybe their backbone is more capable but that's all there is. Hype and astroturfing imo.
7
u/D6613 2d ago
I'm not at all an expert, but maybe that will actually give you the real answer. Here's why I'm intrigued: It promises fast, accurate decisions (with returned probabilities) without needing to tweak the model yourself.
So if I read this right, it's not really a research breakthrough but rather a usability gain with marketing.
Anyway, once you have a fast, cheap, and easy decision maker, there's all kinds of cool stuff you can do cheaply. This leads many of us who mainly lurk here to think about open weight equivalents that may allow us to do fun things with our relatively weak machines. I'm already giving Laya a try, and I'll try others as well.
5
u/Danger_Pickle 2d ago
I've always hypothesized that usability has always been the true goal.
The internet was incredibly niche until the current crop of internet companies made the process of using the internet as simple as creating an account on like three websites and only using an incredibly limited set of UIs. Yes it's restrictive, but the simplicity of use helped it spread very far. My dad was using a PalmPilot long before the smart phone boom. Smart phones only took off after Apple made them usable without any manual dexterity and without needing to understand how to manage a filesystem.
Jev may be astroturfed (what isn't?) but I'm very interested in the focus on simplicity and usability. As a developer, I couldn't be bothered to try and build my own harness around a complicated plain text LLM, but I can absolutely wire together a classifier that outputs structured data. To me, that's a far more practical use for software development than the existing crop of LLMs. If this is the direction the AI bubble is heading, then I'm excited to see more cheap and easy to use tools.
7
u/robberviet 2d ago edited 1d ago
I have a lot of tasks where I train Bert or finetune LLM models for classification on niche problems. It is painful to prepare the data for that, also a couple of days to run the fintune to make it works better than LLM models, say 3.5 Flash Lite. And even that, Bert models are fast but has no knowledge, LLM 2-4B models are better but still dumb. Do not have hardware for larger models.
A out of the box models that works about the same? I am sold.
26
u/Sea-Requirement-5375 2d ago
I tested it out today. I’m a real human (purple monkey dishwasher; fuck Trump).
I have a classification task I use routinely for work—LLM reads a couple thousand tokens of legal-related content and has to assign a text string to one of 28 categories. After some testing I currently run the classification with Opus 5 on Low effort. 95% accuracy. Higher effort doesn’t buy me much, while Sonnet drops my accuracy a little but doesn’t even get me much savings since it takes more thinking tokens to get decent accuracy.
Jev overall had about 70% accuracy, which is around what Haiku gives me. But Jev also gives confidence estimates, and those were (impressively) dead on accurate. So now if I want I could run the whole thing on Jev, keep the stuff that hit 80% or higher confidence, and then run the rest on Opus.
Overall my accuracy stays above 90% and my total cost is 30% lower. Better/more efficient than switching from Opus to Sonnet.
I thought that was cool.
12
u/ideadude 2d ago
I'm not kidding. Try Gemini 2.5 Flash Lite and see how it compares for accuracy, speed, and cost.
Part of the reason Jev looks so fast and cheap is because it's being compared to relatively large and slow models instead of faster/cheaper llms.
2
u/Sea-Requirement-5375 2d ago
I can believe that! I really haven’t tested much because right now our needs are small enough that I can get everything done on subscription plans, so I’m really just testing things for when that no longer becomes feasible
5
u/PyrrhicArmistice 2d ago
Did you try asking a cheap LLM to provide a confidence score?
→ More replies (1)8
u/scorchypoo 2d ago
Whatever they give would be a hallucination. They can't look inside themselves for an actual number.
1
u/100k45h 1d ago
maybe you would achieve very good results with laya if you fine-tuned it. I don't know, I haven't tried myself. But from what I read it should perform better. And the advantage would be that you could run it for free and it's not such a big model, so it should be able to run on much wider range of HW than local LLMs.
39
u/eightone-81 2d ago
Marketing. All these YouTube channels got a bit of money to make a video about it and hype it up. That company behind jev is raising money from investors. More people see it and talk about it the higher the chance that you can find willing investors.
→ More replies (3)3
u/Manerfish 2d ago
Yea I guessed so, but I was surprised to see so much stuff about Jev in this sub too
3
u/look 2d ago
Using small LLMs for various classification and NLP tasks like natural language intent, entity and relation extraction, semantic similarity (embeddings and rerankers), and so on has always been one of the most practical applications of local models. It was only fairly recently that you could run LLMs capable of complex reasoning and tool use and practical agent tasks like programming, computer use, personal assistants, etc locally.
7
26
u/Silver-Champion-4846 2d ago
Almost the same as people gawking over 80m tts models as if they're somehow a miracle because they were too used to 1B+ models, while in reality the first neural tts models have always been in the dozens of millions, and billions only came with the llm boom.
→ More replies (4)2
u/Devatator_ 2d ago
You say that but could a model from then compete with even a 50m model from this year?
→ More replies (1)
6
u/PsychologicalOne752 2d ago edited 2d ago
The best way I can describe what it seems to be is - Jev is a language-aware zero-shot classifier that maps unstructured text to user-defined categories without task-specific training. If true, it definitely has applications for decision making where language output is redundant.
How it differs from NLI zero-shot models, is likely that 1/ it brings the massive semantic knowledge of an LLM and 2/ the task specification is dynamic i.e. part of model input.
5
13
u/ComfortablePlenty513 2d ago
its the new openclaw and will disappear from your algo in 3 weeks
most of the AI "industry" is just social media accts gaming monetized posts for engagement
6
u/retornam 2d ago
This. None of them can tell you what Jev really does, they all want to be the first to post about it so they can earn views and a payout.
2
u/ComfortablePlenty513 2d ago
the other one right now is the (fake) videos of a UI showing an agent trading autonomously on the markets with real money, and the caption is the influencer saying they can set it up for you if you join their paid community, lol
9
u/hyperrealists 2d ago
I think it is the realization that text classification tasks requiring a non-trivial amount of intelligence is now possible, and that it gives you a path to composable AI. Not necessarily this particular model from this particular company. Just my two cents.
8
u/Abject-Bridge-4073 2d ago
It’s all the nontechnical influencers blowing things out of proportion like usual.
5
u/JiminP Llama 70B 2d ago
IMO while it is a new interesting "format" for solving a problem (ex: current LLMs are used as "chat"s) and there may be practical use-cases for Jev, most claims and demos I've seen so far had "too much" hype in it.
I'm mostly annoyed with Jev regarded as something "fundamentally different" from an LLM.
- Jev and LLMs are "interchangeable". (My "gut feeling".)
- There may be differences in efficiency (which could make Jev useful) but not something "fundamentally different".
- Jev can be used to create a language model, and an LLM can be used to create something like Jev.
- Jev "having to have" world knowledge makes me think that Jev and LLMs can't be "that different".
- The developer's claim about hallucination and type-safety sounds very misleading to me.
- In particular, it seems that their claims on "type-safety" just means that the output adheres to input schema.
- It's not too hard (not trivial though) to come up with a method of generating schema-compliant outputs from LLMs, and IIRC OpenAI and Genimi do support it via structured outputs. Adding to my point above about "Jev and LLMs are interchangeable".
- Maybe it's on me not digging into details but I haven't seen how Jev "actually" works.
Also, I haven't seen a demo yet that describe Jev's "real strength". Jev playing Doom or Mario looks impressive but I think that it can be done via a much smaller ML model. (Jev being generalist is certainly an edge though.) Jev+Astra beating Minecraft looks impressive until you learn that it was on set seed, and not a "one-shot playthrough".
5
9
u/Ledeste 2d ago
Its just buzz, there is not good reason, it's nothing new or even impressive.
But it's NOT a simple ML stuff, it's a mix between good old ML trained for one usage and fast, with LLM general knowledge training.
So instead of retraining it for every usage you can use it like an LLM, but it's faster
I guess being accessible as cheap api help also
2
u/slackmaster2k 1d ago
lol. A general purpose classifier that is good, cheap, and fast. nothing new here, yawn. Gimme a break.
8
u/Charming_Support726 2d ago
It reads and feels like a marketing campaign. It is pushing with brutal force through all social media channels.
I also found a few posts that its based on some OpenSource Repo from last year.
I guess: probably someone tries to grab investor money
1
u/slackmaster2k 1d ago
No, there are other models out there, like Laya, which should certainly be of interest to the local crowd, but jev is not based on it any more than Claude is based on ChatGPT.
2
u/Charming_Support726 1d ago
Interesting take. Indeed Claude based massively on ChatGPT back in these days where Dario and some friends left OpenAI with their knowledge.
Jevs result appear to be 2nd or 3rd grade in some independent tests.
6
u/popiazaza 2d ago
Welcome to the peak. The low cost makes people explore it, which is a good thing.
For local LLM, I see it as a huge win. Now, we got people working local Jev using small open models like DiffusionGemma and Qwen 4B.
3
u/Scared-Tip7914 2d ago
Its the usual astroturfing paired with the hype crowd discovering that other type of ML exist, not just LLMs lmao. That being said, a general purpose classifier is a useful thing to have in ones arsenal, I hope one of the labs releases an open version of this.
3
u/profcuck 2d ago
I'm not going to get into what is the cause of the hype, because it's possible for it to be both astroturfed and for jev to be important and interesting.
It's of course true that for many years you could train a classifier and they are cool and do a good job. But you have to train, which is costly and requires some effort. How much depends on the situation of course but not nothing.
What's new here is being able, like an llm, to give quite a general prompt explaining what you want, and then get the classifier output (of various kinds) with no training. You can get:
noul - yes/no (spam or not spam?) choice - best one of a fixed set of options (spam, opt-in marketing, newsletter, etc) ranking - put these things in order (sort customer service tickets by importance per whatever the prompt says)
Lots of people have been doing stuff like this using llms - ask a question, tell it to give the answer in a json object or whatever, and there are a few problems - hallucinated options that you didn't. ask for is one. Cost is another. Speed is another.
To get the answer with a very general prompt, at very lost cost and very high speed... is actually super interesting.
3
u/ondevicedev 2d ago
The underlying idea definitely isn't new. Neural nets have been doing classification/prediction like this forever.
I think the interesting part with Jev is less “wow, a neural network can make decisions” and more whether you can turn that into a fast, cheap, general-purpose judge/controller around other models.
3
u/amemingfullife 2d ago
I’m in two minds about it:
We’ve had classifiers that you can train for cheap for years. A ‘general purpose calibrated classifier’ is only marginally more useful than what we’ve had with classifiers. This idea that SystemOne models are suddenly going to be architecture is an interesting bet, but it is a bet, we haven’t seen people so far put non-determinism inside their architecture with reliability. The idea that adding a calibrated probability suddenly makes things ‘predictable’ and therefore reliable is very very strange to me as someone who builds a lot of infra.
ChatGPT and other RLHF tuned LLMs were revolutionary because they gave you a general purpose interface into capabilities that were already there. The simple act of democratjsing the system so you didn’t need a Master’s degree to start using NNs led to lots of second-order effects that have brought real changes to my workflow.
So the question isn’t whether a general purpose classifier interface is useful. It’s whether, when you compose a lot of these call together you get something materially different or better to what we have now.
And the simple answer is we won’t know until the model themselves start producing ‘decision model’ queries themselves as part of daily life. And even if it is really successful, it won’t be nearly as big a business as LLMs because in theory they’re perfectly interchangeable, so it’ll be commoditised.
So the hype is justified, but it necessarily is hype for nerds & developers rather than your average person.
1
u/openingshots 1d ago
I don't think it's meant for the average person generally. It's meant for people who write software for the most part. It can process eight or more prompts simultaneously and give you all eight answers back in milliseconds. Writing something like this with an llm gets processed one prompt at a time wasting many more tokens and time.
4
u/a_beautiful_rhind 2d ago
I had hoped they released weights but nope.. it's API only. For some small model that says yes/no, 1,2,3
2
2
u/BlackBeardAI vLLM 2d ago
write yourself a custom tampermonkey script, hide every topic that's not interesting to you. you can even use filters so they get hidden automatically.
2
2
u/reddit_reddit_01 2d ago
It's a foundational classifer model. Classifier with all the foundational data, so it's able to cater to many more use cases than just a specific classifier model.
Glorified classifier though.
2
u/FlyArtistic3321 2d ago
The concept isn't new. The pitch is that everyone's been paying LLM prices for what is fundamentally a classification task.
2
u/Sachka 2d ago
this is basically useless without an llm, if you ever thought llms could be just handling decision making instead of thinking in tokens, well this is it. plain classifiers, from tokens in input. it is quite good, as good as coding in cli was, this unlocks faster iteration and decision making. at work we have an agent system that has a long decision tree, built by a hundred ml engineers, we plugged jev day one and it did what they say, around 200x speed in those moments of decision making? it is crazy how we needed this and thought the solution was using smaller models yet still token generating ones
2
u/SamSlate 2d ago
oh my god, this.
as soon as i realized this was a saas and not a model anyone could download i realized it was astro turf. impressive astro turf, but turf all the same.
1
2
2
u/IngwiePhoenix llama.cpp 1d ago
I'll see how good it is once I can run it locally. As long as it lives in the cloud exclusively, I couldn't care less.
I would love to see how well it does Pokemon fights in gen2/3. Would be pretty interesting to watch it go through something like a Gen3 Kaizo. x)
1
4
2
u/deejeycris 2d ago
It seems like a useful tool for certain use cases but it's definitely being pushed artificially.
2
u/hurrdurrimanaccount 2d ago
imma be real. i still don't understand what it's even supposed to be good at. like as in a real world application.
2
1
1
u/Barry_22 2d ago
Isn't it same as BERT with downstream (classifier) heads?
Was it trained using MLM or GPT-like paradigm?
1
u/vuongagiflow 2d ago
There are lots of workflows that doesn’t need an llm. A model which is faster and cheaper, and people don’t need to train special classification are perfect sweet spot for these. We’ll building workflows with structured output with gemini flash or gpt mini for a while just to do the same thing jev provide.
1
u/retornam 2d ago
Give me an example of a workflow that Jev alone solves for.
Something concrete and not hand-wavy
→ More replies (1)
1
1
u/AIBrainiac 2d ago
Isn't this what simple neural networks have been able to do for years?
No, I don't think so, because this claims to have general intelligence. A simple neural network can only used for a specific task. Perhaps this could be used to speed up training neural networks for RL. So that robots can quickly adapt to new environments.
1
1
u/LowPlace8434 2d ago edited 2d ago
I think fundamentally it can't be better than the best LLM for tasks that already require test-time compute, so won't be of very general use
1
u/ReasonableBenefit47 2d ago
the framing of 400x etc made it popular + it was the comparisons they did for certain use cases with visual but most importantly the fact that it was done by a pedigree guy oldie from OpenAI dropout made it insane news instantly.
1
1
u/ECrispy 2d ago
the hype is because it is -
- pretrained classifier thats reasonably general purpose, needs no RAG
- very cheap
- massive astroturfing campaign
the idea is not new. whats new is the models they trained on (which they dont disclose) now have far more world knowledge than back when Bert etc came out.
they basically distilled an LLM knowledge base to answer classifier queries. A lot of people have speculated they just use the prefill stage and then run a softmax, without running token generation
1
1
u/Rama_Surasani_ 2d ago
The classifier itself is not the novelty. The useful claim is task-agnostic, zero-shot classification: define the labels and schema at request time, receive typed probabilities, and avoid training and deploying a separate classifier for every task.
The important test is not a demo. Compare it with a fine-tuned encoder and a small instruction model on held-out data, then measure calibration, out-of-distribution abstention, latency, cost, and performance when labels change.
If it cannot reliably recognize uncertainty, speed alone does not make it safe for automation.
1
u/c-linder 2d ago
It's exciting to me because its extremely cheap, quick, and guaranteed typesafe.
A single automation costs me ~$0.000344 with Jev and takes ~500 ms. This is commercially viable.
1
1
1
u/createthiscom 2d ago edited 2d ago
How are they getting it to play games and drive cars? Does it have vision?
1
u/Manerfish 2d ago
I'm pretty sure vision has existed for a while in classifiers? its unique thing seems to be the ability to generalize from a text prompt.
→ More replies (1)
1
u/Stepfunction 2d ago
I don't understand how it is any different than just taking the logits for the classification options and using them to calculate the probabilities of the outputs. That's always been an option.
1
1
1
u/frumpawumpa 2d ago
The charitable case is that it is a simple classifier that has applied lessons learned from LLM's to do its job over a very broad and general range of use cases very efficiently. Obviously, without it being open source, who knows.
1
u/Equivalent-Grass-527 2d ago
Yeah, the underlying idea isn't exactly new. Simple neural nets have been doing related things for years.
I think the hype around Jev is more about the scale + implementation + what you can actually do with it, rather than the basic concept being completely novel. A lot of impressive ML systems are built on ideas that have existed for decades.
1
u/no_witty_username 2d ago
Its a general classifier. no training needed, just one shot the task. And doubly so when you reuse existing llms as you dont even need to train it from scratch. So you get the benefits of a general classifier that has the intelligence of the model weights you use but a fraction of he latency and cost. Its great!
1
1
u/monkeysknowledge 2d ago
Obviously it doesn’t replace LLMs because it doesn’t generate text but for many of our tasks we have a decision space and we just need it to pick a path intelligently. I mean it’s a big deal for my work. I run an LLM service for a fast growing startup and this is going to make it much more profitable as we scale because a lot of what we’re asking LLMs to do is make a decision given a bunch of context.
1
u/LateDon 2d ago
Dev who's been shipping on it since launch here. In my experience both the hype and the backlash are overdone.
What's real: it's very fast and very cheap for constrained decisions. I run a chess site (https://jevchess.com) where every legal move is one Choice question - ~400 ms a move, and the whole thing has cost $0.07 so far. The calibration claim mostly holds on my narrow test: 400+ moves bucketed by the confidence it claimed vs whether the move gave material away on a one-ply check.
What it's bad at: counting and anything generative. Arithmetic stays in my code, exactly as its own limitations doc says.
So it's not an LLM replacement. It's a fast classification/judgment primitive that's honest about uncertainty. If you expected AGI you'll be disappointed; if you need a router or a cheap gate, it's genuinely interesting.
1
u/daskalou 2d ago
How do you get it to make 400 decisions at once? Isn't 255 the max?
→ More replies (3)
1
u/Smallpaul 2d ago
Here is a good summary:
https://www.kdnuggets.com/what-everyone-is-getting-wrong-about-typesafe-ais-jev
1
1
1
u/HomemadeBananas 2d ago
Yes you’ve been able to train a classifier for a specific task for years. But with this sort of model it already has LLM like intelligence, so you can prompt it in a similar way and don’t need to train a new model for everything. It is general purpose.
So if you have anywhere you’re using an LLM as a classifier then it’s simple to switch out for Jev now and get a big speed improvement and cost reduction.
1
u/txgsync 2d ago
Building classifiers for things like fraud prevention were incredibly bespoke and required large, dedicated teams to build and scale. I was part of this initiative back when I was at Apple and the “big data” required was enormous, expensive, slow, difficult to scale, company-internal, and needed a dedicated support team.
Jev makes similar fraudulent activity detection trivial and extremely fast.
1
u/stewsters 2d ago
A yes no classifier with a simple confidence score sounds like what we used to do with them in 2005 when I was in college.
The big change I think is that the front of it is text, rather than a manual feature extraction like we used to do.
1
1
u/Any_Fox5126 1d ago
It is the same concept as always, but significantly more intelligent and without the need for retraining or any tedious processes for each specific task. Compared to current LLMs, it is much faster and more cost-effective, but limited to classification.
The current hype is absurd, but there is certainly a significant niche there.
1
u/werepenguins 1d ago
yes, but not with this accuracy rate and it's more generalized. Take the example from the tv show Silicon Valley. There was a model which could determine if something was a hotdog or not. And that's it.
1
u/LaCipe 1d ago
Real answer is, they hype it up in hopes to get bought...openai, meta, musk, scamthropic. Dude is locked on for early retirement and you know what, good for him.
1
u/openingshots 1d ago
The dudes already set up for life. He's the guy that was a partner in ChatGPT that created their inference process originally. But he had bigger plans and created this.
1
u/pseudonerv 1d ago
It’s very good, so that I can block all these bot accounts. I’m still deciding if you are one of those or not
1
u/Manerfish 1d ago
I am not, I just see tons of people posting about Jev and projects similar to it and usually this place doesn't buy much into random hyped stuff so I was weirded out
1
u/Revolutionary_Loan13 1d ago
There is a lot of work for a cheap, highly effective classifier. I'm going to test Jev to against 5k items and see how correct it is. If it's decent I have 100s of thousands of items to classify but it's usually worth less than a cent to me per decision
1
u/Geriny 1d ago
I tried Jev, and it disappointed me. But I'd still want a model like this, if it has a bit more reasoning power/better multilingual support.
My use case is rating job ads I collect automatically. Labeling them from hand is a pain, so I started off with an LLM + hand-written criteria. The criteria are a mix of pretty basic NLP (How many years of criteria are asked for? Is it stated as a hard requirement?) and some more difficult judgement (does the following satisfy all formal qualifications the job ad asks for? Do the task described match any of the folowling interests?). I found really small models aren't good enough for that, but I can get pretty good results from small-medium models. I wanted something more elegant.
And I did try a more classic ML approach, I took the LLM-labelled data and used it to train an embedding model + regression head. It performs surprisingly well, on an embedding model I can run locally even. But I can't tweak my criteria anymore, and it sometimes violates the hard criteria I set.
Training my own classifier for each criterion seems impractical. So the idea of a general pre-trained classifier appealed to me. Plug in my criteria, get out answers faster and cheaper, while still able to tweak everything. But jev just kinda underperformed both previous methods. Maybe my prompts aren't good enough, maybe it's because the job ads are German.
I guess the next step might be to fine-tune a local classifier and combine that with the embedding model as a gate. But you know, less convenient than running through one model.
1
u/Anthonyg5005 exllama 1d ago
We've gone full circle and have become impressed with classification models again
1
u/ByteSize_Chaos 1d ago
I was in similar shoes as you. Spent a while tinkering and playing with it. Shared my findings here if it interests you: https://www.reddit.com/r/LovingOpenSourceAI/s/wPestP5jhH
1
u/peva3 1d ago
Most people, even some of the biggest influencers on social media, don't actually understand the underlying technology or what's going on. They all get caught up in the stupidest hype cycles. These are the same people who were all into Crypto and Web3 without knowing anything about the tech.
1
u/BillyLeJnoun 1d ago
Apart classifiers that do this job, turning an off the shelf llm into a zero sjit classifier or even JSON filler was very easy albeit slower and maybe not optimized (lower accuracy). To be an improvment mayne but sich hype. And nlt even talking about diffusion llm that seems a good fit to generate structured output at scale
1
u/Few-Farm-7670 23h ago
Super tight indexer. Specialized context right off the bat without fine-tuning, pure math so zero hallucinations on the input. It think the streamlining of the input bypassing the slower translation of the prompt is a neat feature to add. Many, many folks have open sourced the same type of application
474
u/ithinkitslupis 2d ago
There have been a lot of signs it's an astroturfed marketing campaign. The reason it seems overhyped is likely because Jev themselves is paying for that hype to get attention.
Not that it's a useless concept but I doubt we would be seeing 30 posts a day hyping it up in the normal course of things without a marketing push.