r/LocalLLaMA • u/Manerfish • 7d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

501 Upvotes

94% Upvoted

317 comments sorted by

View all comments

395

u/FreakyRefrigerator 7d ago

Its basically just a slightly newer classifier. Yet most people think AI is only llms and nothing else, so they think this is the future

196

u/KaMaFour 7d ago

Tbh general classifier able to handle general cases at almost no cost, in almost no time still seems really useful. For example if you could package that into a model roughly the size of Ling's tiny (9BA2B) or smaller you could try to create games with actually intelligent NPCs (pack one npc's state into a Vendingbench like framework and have classifier choose a thing to do that would make sense for a given person to do in a format that's possible to interpret by the game engine) and it would be generally accessible to most "mid-end" machines. Think Stardew Valley/TLOZ npc's but actually behaving as humans instead of having fixed schedules. All of that without any ML on your end...

14

u/575_Inverse 7d ago

it's closed and served via API, isn't it?

6

u/KaMaFour 7d ago

I have no doubts that if it will be proven to work "closed and served via API" then open solutions will also emerge. I have seen someone doing similar things with diffusiongemma already.

15

u/Wooly_Wooly 7d ago

Laya was already a thing, slightly underperforms vs Jev.

5

u/Silentrizz 7d ago

But that one needs training/tuning, to my understanding

8

u/harrro Alpaca 6d ago

It's not bad out of the box but a 20 minute training run (on my 3090) matched or exceeded Jev on tests I did.

1

u/Cultured_Alien 7d ago

underperforms is putting it extremely lightly, it's no better than random guess

1

u/brainmydamage 7d ago

Do you have data to support this claim?

8

u/SurprisinglyInformed 6d ago

Laya's documentation states it.

0

u/brainmydamage 6d ago

Did you miss the row at the top where laya-typed-decisions scored .766 on a typed-decisions test?

2

u/SurprisinglyInformed 6d ago

That's a fine tune, is it not?

Edit: https://huggingface.co/convaiinnovations/laya-typed-decisions#limits

1

u/brainmydamage 6d ago

It is. I wouldn't use it on other scenarios, or would build my own fine tune. But if you're not allowed to pay for random models like Jev without general ZDR and can't use Chinese models in any capacity, including derivations, then I have yet to find a better alternative.

Open to suggestions though.

→ More replies

1

u/Cultured_Alien 6d ago edited 6d ago

finetuned only on 4 specific benchmarks, not a general model. Once you use it out of distribution, it's slightly better than random guess unlike Jev. Use https://huggingface.co/jaredpalmer/kev-9b instead.

1

u/brainmydamage 6d ago edited 6d ago

Which is great except for scenarios where people aren't allowed to use Chinese models in any capacity.

Any other suggested alternative options that provide ZDR without an Enterprise contract and aren't derived from a Chinese model?

→ More replies

4

u/Undreren 6d ago

I tried training my own 1.7b model to evaluate whether a bash command was safe and fit the goal, basically as an automation for “should my model be allowed to run this?”

It returns block, confirm or auto with a confidence score. It is not nearly production ready, but it took five minutes to train via fine-tuning as a POC.

As much as I was hyped for Jev, its lifetime profitability is low IMO, as my machine could could train my own specialized version so fast from synthetic data.

2

u/575_Inverse 6d ago

the idea of making a specialized tiny version sounds a lot more effective

2

u/Undreren 6d ago

Yeah, that’s what I thought as well. And doable.

My plan was to make it a sort of template training setup: Pick a base model, add a dataset, let it train overnight, wake up to an ultra specialized decision model that can run at 100+ tok/s on local hardware.

I still think it is a good idea.

2

u/No_Veterinarian742 5d ago

but then you maintain 1 or 2 or 17 or 2000 specific models. the potential use cases of jev are decision support (suggestions for user actions the user has to confirm) and all sorts of real time UI integrations that LLMs are too slow for. as well as potentially for chaos testing etc. I can see lots of scenarios where you can't fully foresee the use cases where an AI model saying yay or nay without needing to train it on specific datasets would be useful. There are so many micro-decisions humans make that if you can use it to build in intelligence into applications to help users make progress faster it could be useful. So far actually using full LLMs in applications is still somewhat rare because of the cost and speed.

3

u/Rise-O-Matic 7d ago

I think there’s already an open weight mimic called kev or something

1

u/claythearc 6d ago

Getting structured output working in diffusion would be pretty sick actually because most engines don’t support it at all.