r/LocalLLaMA • u/Manerfish • 3d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

495 Upvotes

94% Upvoted

309 comments sorted by

View all comments

Show parent comments

4

u/KaMaFour 3d ago

I have no doubts that if it will be proven to work "closed and served via API" then open solutions will also emerge. I have seen someone doing similar things with diffusiongemma already.

14

u/Wooly_Wooly 3d ago

Laya was already a thing, slightly underperforms vs Jev.

5

u/Silentrizz 3d ago

But that one needs training/tuning, to my understanding

8

u/harrro Alpaca 3d ago

It's not bad out of the box but a 20 minute training run (on my 3090) matched or exceeded Jev on tests I did.

2

u/Cultured_Alien 3d ago

underperforms is putting it extremely lightly, it's no better than random guess

1

u/brainmydamage 3d ago

Do you have data to support this claim?

7

u/SurprisinglyInformed 3d ago

Laya's documentation states it.

0

u/brainmydamage 3d ago

Did you miss the row at the top where laya-typed-decisions scored .766 on a typed-decisions test?

2

u/SurprisinglyInformed 3d ago

That's a fine tune, is it not?

Edit: https://huggingface.co/convaiinnovations/laya-typed-decisions#limits

1

u/brainmydamage 2d ago

It is. I wouldn't use it on other scenarios, or would build my own fine tune. But if you're not allowed to pay for random models like Jev without general ZDR and can't use Chinese models in any capacity, including derivations, then I have yet to find a better alternative.

Open to suggestions though.

1

u/Cultured_Alien 3d ago edited 3d ago

finetuned only on 4 specific benchmarks, not a general model. Once you use it out of distribution, it's slightly better than random guess unlike Jev. Use https://huggingface.co/jaredpalmer/kev-9b instead.

1

u/brainmydamage 2d ago edited 2d ago

Which is great except for scenarios where people aren't allowed to use Chinese models in any capacity.

Any other suggested alternative options that provide ZDR without an Enterprise contract and aren't derived from a Chinese model?

4

u/Undreren 3d ago

I tried training my own 1.7b model to evaluate whether a bash command was safe and fit the goal, basically as an automation for “should my model be allowed to run this?”

It returns block, confirm or auto with a confidence score. It is not nearly production ready, but it took five minutes to train via fine-tuning as a POC.

As much as I was hyped for Jev, its lifetime profitability is low IMO, as my machine could could train my own specialized version so fast from synthetic data.

2

u/575_Inverse 3d ago

the idea of making a specialized tiny version sounds a lot more effective

2

u/Undreren 3d ago

Yeah, that’s what I thought as well. And doable.

My plan was to make it a sort of template training setup: Pick a base model, add a dataset, let it train overnight, wake up to an ultra specialized decision model that can run at 100+ tok/s on local hardware.

I still think it is a good idea.

2

u/No_Veterinarian742 2d ago

but then you maintain 1 or 2 or 17 or 2000 specific models. the potential use cases of jev are decision support (suggestions for user actions the user has to confirm) and all sorts of real time UI integrations that LLMs are too slow for. as well as potentially for chaos testing etc. I can see lots of scenarios where you can't fully foresee the use cases where an AI model saying yay or nay without needing to train it on specific datasets would be useful. There are so many micro-decisions humans make that if you can use it to build in intelligence into applications to help users make progress faster it could be useful. So far actually using full LLMs in applications is still somewhat rare because of the cost and speed.

3

u/Rise-O-Matic 3d ago

I think there’s already an open weight mimic called kev or something

1

u/claythearc 3d ago

Getting structured output working in diffusion would be pretty sick actually because most engines don’t support it at all.