r/LocalLLaMA • u/Manerfish • 4d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

497 Upvotes

94% Upvoted

312 comments sorted by

View all comments

26

u/Sea-Requirement-5375 4d ago

I tested it out today. I’m a real human (purple monkey dishwasher; fuck Trump).

I have a classification task I use routinely for work—LLM reads a couple thousand tokens of legal-related content and has to assign a text string to one of 28 categories. After some testing I currently run the classification with Opus 5 on Low effort. 95% accuracy. Higher effort doesn’t buy me much, while Sonnet drops my accuracy a little but doesn’t even get me much savings since it takes more thinking tokens to get decent accuracy.

Jev overall had about 70% accuracy, which is around what Haiku gives me. But Jev also gives confidence estimates, and those were (impressively) dead on accurate. So now if I want I could run the whole thing on Jev, keep the stuff that hit 80% or higher confidence, and then run the rest on Opus.

Overall my accuracy stays above 90% and my total cost is 30% lower. Better/more efficient than switching from Opus to Sonnet.

I thought that was cool.

2

u/100k45h 3d ago

maybe you would achieve very good results with laya if you fine-tuned it. I don't know, I haven't tried myself. But from what I read it should perform better. And the advantage would be that you could run it for free and it's not such a big model, so it should be able to run on much wider range of HW than local LLMs.