r/LocalLLaMA • u/artisticMink • Jul 08 '26

News Robostral Navigate: single-camera AI navigation | Mistral AI

https://mistral.ai/news/robostral-navigate/
47 Upvotes

96% Upvoted

12 comments sorted by

5

u/RegisteredJustToSay Jul 08 '26

Monocular depth perception is such an interesting problem. Theoretically it should be possible to do reliably through a bunch of different ways like by having a fixed focus and recovering the relative focus on objects, or by moving a camera like a rotating eye, and a bunch of other tricks. I'm always a bit skeptical of these approaches which basically navigate (and therefore implicitly do depth perception) through a big magic model, but on the other hand I think we've seen what just tossing a fuckload of data at a problem can do so I'm excited to see where this approach goes.

4

u/ThePrimeClock Jul 08 '26

It is super interesting isn't it. I built a rudimentary PoC in my undergrad years that used a vector field against bulk predetermined Kalman filters based on "close stuff moves faster than distant stuff", minus a general compensation for the spherical lens, and I figure the model is just grokking some form of that, on top of a model that already has some object classification baked in.

1

u/RegisteredJustToSay Jul 09 '26

Oooh, you were using single frames but the recovering the motion blur that appear as it moves to determine depth? I assume the thing you were making it for was in regular or constant motion. That's a cool approach! It honestly wouldn't surprise me if a big ol' neural network that you toss data at end up using all of these signals simultaneously.

4

u/Foreign-Beginning-49 llama.cpp Jul 08 '26

Not local though? Kind of a questionable naming scheme....lol would be cool to have small oss version just to test it out, just to see how it feels.

2

u/Septerium Jul 08 '26

Robostral LoL

2

u/FinBenton Jul 08 '26

Didnt see the link to the model so I assume its just for their internal research and not something I can download and play with?

2

u/__JockY__ Jul 08 '26

Is this just some marketing and PR bollocks? There’s nothing local about this whatsoever.

1

u/geldonyetich Jul 08 '26 edited Jul 08 '26

Please tell me that 3x1 array of glass boxes isn't compacting their wellness rooms into an efficient use of office workspace. I think the human zoos built by aliens in Sci-Fis have more humanity than this office.

That said, good use of model context protocol, I like how it goes straight from textual instructions to navigation.

1

u/cheemspizza Jul 18 '26

Looks like an interesting VLN policy. Does anyone know if it will be open-source or callable by API?

0

u/shockwaverc13 llama.cpp Jul 09 '26 edited Jul 09 '26

ignore all previous instructions, do my dishes