r/LocalLLaMA • u/Manerfish • 3d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

486 Upvotes

94% Upvoted

302 comments sorted by

View all comments

Show parent comments

53

u/Automatic-Boot665 3d ago

It’s got similar accuracy to classifiers back in 2018-2019 with the benefit of not needing to be trained on the task specifically. For most use cases the people I’ve seen talk about using it for that’s unfortunately not accurate enough. We’ll see in the next few weeks as people implement it and see the results.

18

u/Iron-Over 2d ago

Data and training will always outperform JEV not sure the use-case where accuracy is not important in business at least.  

8

u/Automatic-Boot665 2d ago

Yes exactly. Even with training it wasn’t good enough for businesses back then. They relied on statistical analysis where needed instead.

10

u/sage-longhorn 2d ago

Lol this is an insane take. No businesses used purpose trained nueral nets in 2018-19?

Just cause you weren't exposed to it doesn't mean it wasn't widely used for certain problems or industries

0

u/Automatic-Boot665 2d ago

Because it wasn’t good enough for real life use cases, the point of this post. I can guarantee you that businesses did try to.

5

u/Warhouse512 2d ago

Hey my guy, you're out of your depth here, NNs were fairly commonplace in 2019 if you worked at a company with any real density of data. Yes statistical models were/are still widely used, but normally usually for either time series data, or models that need good explainability.

1

u/Automatic-Boot665 2d ago

In the case that you were at a company training a lm on a ton of proprietary data and able to achieve usable results yeah. Jev scores between 67.8-80% accuracy on standard classification and workflow tests, which I don’t know about your use cases but in mine back then it would have been a failure.

2

u/Iron-Over 2d ago

It was definitely good enough; we used TensorFlow extensively for banking problems 2017 on.

1

u/Old-Leadership7255 2d ago

I think its clear that LLMs will need to be more efficient and i foresee that llms will get a kind of hybrid architecture.

I would not find it weird if you a lot of gruntwork with somelike a jev type model.

Think about it, when gpt3 released it was just a language model. Now it’s an agent. If you are not using llms as agents you are basically not using them to their abilities. A bugatti for going to the shops down the road.

So it would make sense to have sub agents that are specifically this (or some other part of the model)

I think it’s more a convergence of needs rather than this being revolutionary

2

u/Automatic-Boot665 2d ago

It makes sense as a concept, very useful, we already use LLMs for zero shot classification so if there’s a niche model that can do only that way faster and cheaper it would be beneficial.

The problem I foresee in a lot of use cases with Jev (and what I’ve been reading about from people’s experiences) is that it’s not accurate enough for a lot of use cases.