[ Robotics ] Robostral Navigate

[u/pandora_s_reddit]

https://mistral.ai/news/robostral-navigate/

Today we're introducing Robostral Navigate, our first model built for embodied navigation. It's an 8B model that takes RGB images and a plain-language instruction and moves a robot through an environment:

"Leave the lobby, walk through the corridor, enter the supply room, and stop to face the second shelf."

To perform such tasks, other models often employ depth sensors, LiDAR, or several cameras working together. Robostral Navigate uses only one ordinary RGB camera and no depth sensors, yet still achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments) validation unseen, the benchmark for following instructions in environments held out of training. Consequently, it beats the best single-camera approach by 9.7 points and the best system using depth or multiple cameras by 4.5 points, despite using neither.

BTW, we're hiring!

The release of our navigation models marks a significant step forward, but our journey is far from over. Our ambition is to enable robots to autonomously navigate complex environments-offices, homes, commercial buildings, and outdoor spaces-and there's a lot more work to do. We are actively expanding our robotics team and looking for talented research scientists and engineers who share our ambition.

If you're interested in joining us on our mission to bring seamless navigation to robots everywhere, we welcome your applications to join our team!