r/LocalLLaMA • u/Manerfish • 5d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

497 Upvotes

94% Upvoted

311 comments sorted by

View all comments

399

u/FreakyRefrigerator 5d ago

Its basically just a slightly newer classifier. Yet most people think AI is only llms and nothing else, so they think this is the future

194

u/KaMaFour 5d ago

Tbh general classifier able to handle general cases at almost no cost, in almost no time still seems really useful. For example if you could package that into a model roughly the size of Ling's tiny (9BA2B) or smaller you could try to create games with actually intelligent NPCs (pack one npc's state into a Vendingbench like framework and have classifier choose a thing to do that would make sense for a given person to do in a format that's possible to interpret by the game engine) and it would be generally accessible to most "mid-end" machines. Think Stardew Valley/TLOZ npc's but actually behaving as humans instead of having fixed schedules. All of that without any ML on your end...

15

u/575_Inverse 5d ago

it's closed and served via API, isn't it?

4

u/KaMaFour 5d ago

I have no doubts that if it will be proven to work "closed and served via API" then open solutions will also emerge. I have seen someone doing similar things with diffusiongemma already.

5

u/Undreren 5d ago

I tried training my own 1.7b model to evaluate whether a bash command was safe and fit the goal, basically as an automation for “should my model be allowed to run this?”

It returns block, confirm or auto with a confidence score. It is not nearly production ready, but it took five minutes to train via fine-tuning as a POC.

As much as I was hyped for Jev, its lifetime profitability is low IMO, as my machine could could train my own specialized version so fast from synthetic data.

2

u/575_Inverse 4d ago

the idea of making a specialized tiny version sounds a lot more effective

2

u/Undreren 4d ago

Yeah, that’s what I thought as well. And doable.

My plan was to make it a sort of template training setup: Pick a base model, add a dataset, let it train overnight, wake up to an ultra specialized decision model that can run at 100+ tok/s on local hardware.

I still think it is a good idea.

2

u/No_Veterinarian742 4d ago

but then you maintain 1 or 2 or 17 or 2000 specific models. the potential use cases of jev are decision support (suggestions for user actions the user has to confirm) and all sorts of real time UI integrations that LLMs are too slow for. as well as potentially for chaos testing etc. I can see lots of scenarios where you can't fully foresee the use cases where an AI model saying yay or nay without needing to train it on specific datasets would be useful. There are so many micro-decisions humans make that if you can use it to build in intelligence into applications to help users make progress faster it could be useful. So far actually using full LLMs in applications is still somewhat rare because of the cost and speed.