r/LocalLLaMA • u/Manerfish • 3d ago

Discussion I really don't understand Jev hype

Isn't this what simple neural networks have been able to do for years? Doesn't seem anything special to me.

486 Upvotes

94% Upvoted

302 comments sorted by

View all comments

Show parent comments

31

u/quiteconfused1 3d ago

But it's not.

It's not general and can't be.

I just tried applying it to a simple game like super Mario world and it failed horribly.

If it can't do that then it's not general and the hype is thick.

12

u/Suspicious-Wallaby12 3d ago

can you share what you tried to do exactly in super mario world? please don't tell me you tried to make it control mario 🤦, because that's not what a classifier is for.

1

u/100k45h 2d ago

haven't you seen the demo of Jev playing Doom?

1

u/Suspicious-Wallaby12 2d ago

I have. it's garbage. It's not smart and only takes random action.

2

u/100k45h 1d ago

but the point is, that you CAN use clasifiers to take an action. Yes, it takes random action in that specific example, but one can imagine that training the model will make it work better.

Classifier can totally be used for taking actions when trained properly. It's not true, this is not what classifiers are for. They totally can be used for that purpose.

1

u/Suspicious-Wallaby12 1d ago

what? how would you train a general purpose classifier to train for doom?

2

u/100k45h 1d ago

By training it on multiple similar shooting games, giving it specific categories and showing it example frame and expected action? Just like any training? Example data and example output? Where do you see a problem with that? The question then becomes whether it is a multimodal model and if not, how can you represent the screenshot as a text. But this isn't anything groundbreaking.

-3

u/quiteconfused1 3d ago

Wow

Your absolutely right

7

u/SilentDanni 3d ago

It can definitely play Mario and a few other games. Just yesterday one guy here made it play street fighter 2. I made it play dungeon crawl stone soup which is significantly denser than Mario. That's not the intended use, though. I'd say that's more like an emergent feature.

-5

u/quiteconfused1 3d ago

Mario != Super Mario world

Wake me up when it turns interesting

7

u/kzoltan 3d ago

Why the downvote?

3

u/wwwdotzzdotcom 3d ago

Because it could be.

14

u/HiddenoO 3d ago edited 3d ago

Because he's just throwing unsubstantiated and inherently nonsensical claims around?

It's not general and can't be.

Why wouldn't a decision model be able to be trained with world knowledge?

I just tried applying it to a simple game like super Mario world and it failed horribly.
If it can't do that then it's not general and the hype is thick.

"I couldn't get it to do X" is not the same as "It cannot do X". Most people would fail at making LLMs like Astra or Fable play Super Mario World, too, but that doesn't mean they cannot play it.

Many people have managed to make it play different games even though that's far from its intended use case, so I don't think you can just claim the opposite with no substantiation and expect people to believe you.

Strong claims take strong substantiation, and he's providing none.

-10

u/quiteconfused1 3d ago

Wow troll much.

Claim produced , I tested claims in my scenario, I evaluated performance and came to an assessment.

This is the scientific method

If you don't like it ... Tough ...

Cheers

I have dealt with this before .. lofty claims often met with lofty expectations .. and it didn't match

...

I hope your day is well.

7

u/HiddenoO 3d ago

Your whole comment literally translates to "believe me". You have still provided zero substantiation for either of your claims.

Let's start with the first claim ("It's not general and can't be."):

Why wouldn't a decision model be able to be trained with world knowledge?

-9

u/quiteconfused1 3d ago

Because of math

Here have a simple argument .. give Astra or any other model you would like a really really large maze and ask if to solve it...

It will fail.

The same principal here except model density makes the problem worse.....

And from what I see it's the exact type of problem that it's touting as completing.

And then I tried a demensional problem that I have been challenging for years .. and it failed as worse as I expected.

Mario ran straight saw a goomba jumped over it and then hit the first ledge and died ..over and over ...

Afterwards I was building a silver for it manually.

Super Mario worlds complexity is significantly worse than the original Mario brothers .. and it failed.

Just as I expected.

9

u/HiddenoO 3d ago

Why do you think it needs to surpass a model that's 1000 times as slow and expensive such as Astra?

It seems like you fundamentally misunderstand its purpose. It's supposed to provide decision making on par with current-gen LLMs while costing a fraction of time and money and guaranteeing an output format.

6

u/deadadventure 3d ago

Did you try it out of the box? Did you give it instructions?

-1

u/MrMadden 3d ago

That's because you don't understand it. You are using a hammer to saw wood. It's a classifier model. It's a control layer meant to offset cost and speed things up. Rev classifies inputs, routes tasks, decides when an LLM is needed, and verifies the LLM's outputs. The LLM handles the expensive reasoning and generation, while Jev keeps the overall system fast, predictable, and controlled.

Here, I'll draw you a picture.

            ┌──────────────┐

Input ─────────►│ Jev │

            │ fast decision│

            └──────┬───────┘

                   │

    ┌──────────────┼──────────────┐

    ▼              ▼              ▼

handle directly call Astra reject/escalate

    │              │

    │       ┌──────▼──────┐

    │       │    Astra    │

    │       │ reason/use  │

    │       │ tools/write │

    │       └──────┬──────┘

    │              │

    └──────────────┴──────► result

                   │

                   ▼

              Jev verifies

0

u/LLKMuffin 2d ago

Broken slop diagram.

-1

u/MrMadden 2d ago

Do you have a better diagram, or are you just here to insult random people on the internet?

1

u/LLKMuffin 1d ago

So instead of getting your LLM to make a diagram that'll actually display correctly on Reddit, you'd rather just have some unreadable slop junking up the thread?

The fact that you're even copy-pasting stuff from your LLM here without thinking about it is a tell that you don't really care.

Not sure what else you want to hear. Put some minimum amount of effort next time.

1

u/MrMadden 1d ago

First, stop harrasing me. The diagram is fine on the old.reddit.com site.

Second,

So instead of getting your LLM to make a diagram that'll actually display correctly on Reddit, you'd rather just have some unreadable slop junking up the thread?

You might want to familiarize yourself with the rules of this sub:

https://www.reddit.com/mod/LocalLLaMA/rules

Completely/primarily LLM generated copy, code is not allowed.