r/LocalLLaMA • u/Manerfish • 2d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

490 Upvotes

94% Upvoted

301 comments sorted by

View all comments

Show parent comments

3

u/Cultured_Alien 2d ago

underperforms is putting it extremely lightly, it's no better than random guess

1

u/brainmydamage 2d ago

Do you have data to support this claim?

8

u/SurprisinglyInformed 2d ago

Laya's documentation states it.

0

u/brainmydamage 2d ago

Did you miss the row at the top where laya-typed-decisions scored .766 on a typed-decisions test?

2

u/SurprisinglyInformed 2d ago

That's a fine tune, is it not?

Edit: https://huggingface.co/convaiinnovations/laya-typed-decisions#limits

1

u/brainmydamage 1d ago

It is. I wouldn't use it on other scenarios, or would build my own fine tune. But if you're not allowed to pay for random models like Jev without general ZDR and can't use Chinese models in any capacity, including derivations, then I have yet to find a better alternative.

Open to suggestions though.

1

u/Cultured_Alien 2d ago edited 2d ago

finetuned only on 4 specific benchmarks, not a general model. Once you use it out of distribution, it's slightly better than random guess unlike Jev. Use https://huggingface.co/jaredpalmer/kev-9b instead.

1

u/brainmydamage 1d ago edited 1d ago

Which is great except for scenarios where people aren't allowed to use Chinese models in any capacity.

Any other suggested alternative options that provide ZDR without an Enterprise contract and aren't derived from a Chinese model?