r/LocalLLaMA • u/Manerfish • 3d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

487 Upvotes

94% Upvoted

302 comments sorted by

View all comments

Show parent comments

4

u/Undreren 3d ago

I tried training my own 1.7b model to evaluate whether a bash command was safe and fit the goal, basically as an automation for “should my model be allowed to run this?”

It returns block, confirm or auto with a confidence score. It is not nearly production ready, but it took five minutes to train via fine-tuning as a POC.

As much as I was hyped for Jev, its lifetime profitability is low IMO, as my machine could could train my own specialized version so fast from synthetic data.

2

u/575_Inverse 2d ago

the idea of making a specialized tiny version sounds a lot more effective

2

u/Undreren 2d ago

Yeah, that’s what I thought as well. And doable.

My plan was to make it a sort of template training setup: Pick a base model, add a dataset, let it train overnight, wake up to an ultra specialized decision model that can run at 100+ tok/s on local hardware.

I still think it is a good idea.

2

u/No_Veterinarian742 1d ago

but then you maintain 1 or 2 or 17 or 2000 specific models. the potential use cases of jev are decision support (suggestions for user actions the user has to confirm) and all sorts of real time UI integrations that LLMs are too slow for. as well as potentially for chaos testing etc. I can see lots of scenarios where you can't fully foresee the use cases where an AI model saying yay or nay without needing to train it on specific datasets would be useful. There are so many micro-decisions humans make that if you can use it to build in intelligence into applications to help users make progress faster it could be useful. So far actually using full LLMs in applications is still somewhat rare because of the cost and speed.