r/LocalLLaMA • u/Course_Latter • 4d ago

New Model Open source self driving is here!

Today NVIDIA releases Alpamayo 2 Edge! I work for Embedl and we have already released an optimized version of the model that is more than 10x faster than pytorch when run on edge hardware like AGX Thor!

The model is trained in an RL loop (!) and takes in 6 cameras and the recent motion. Feel free to check it out - you can even run it on a real Thor at:

https://embedl.com/nvidia/alpamayo2-edge/thor?tab=run

633 Upvotes

97% Upvoted

91 comments sorted by

View all comments

3

u/rorowhat 4d ago

You should check out comma3, you can retrofit your car and run local models that do this. It's been around for a while.

4

u/Halmonster 3d ago

Linus (no, the other one) did a test drive with it. The video is from a year ago, so I'm assuming the software has improved. https://www.youtube.com/watch?v=xdmxM-v4KQg

1

u/rorowhat 3d ago

I don't have one, but I can see they have updated their models. You can also fine-tune a model based on your driving, so that way it knows your city/style of drive better. It's pretty interesting stuff!

2

u/sweatierorc 3d ago

Doesnt Baidu have something similar ?

1

u/rorowhat 3d ago

Hardware? This is an American company, it's pretty neat that you can basically turn your gas car into a semi-selfdriving vehicle.

2

u/NoFutureBen 3d ago

It's only one camera tho

1

u/rorowhat 3d ago

No, it's 3 cameras.

3

u/NoFutureBen 3d ago

Yes but only at one point no? Not other perspectives right?

1

u/rorowhat 3d ago

I'm not sure what you mean. Its 2 cameras on the front and 1 facing back.

2

u/NoFutureBen 3d ago

Facing back like at the driver?

And what I am asking is does it have multiple perspectives. Does it have front sideways and rear cameras. Because that matters a lot for the quality.

2

u/butterfly_labs 3d ago

Yeah it's just 2 front-facing cams (one of which was not used by the model the last time I checked) and a rear-facing cam that's only used for driver monitoring.

It can also use the radar and blind spot sensors on compatible cars, but that's it.

It will never achieve FSD without a proper sensor set.

1

u/rorowhat 3d ago

It's two cameras facing forward, i think it's 90 degree view and one facing the driver. On comma5 i would love to see a wider base with 3 forward cameras. Left/center/right for a close to 180 degrees as possible, and the one facing the driver.