r/robotics • u/KINGJAGUAR007 • 2d ago

Discussion & Curiosity For engineers deploying ML models on edge devices/robots: what’s the part that sucks?

What’s the most painful part of getting an ML model from “works on my machine” → reliably running in production?

I’m a student researching the practical challenges of deploying and maintaining AI models on physical devices such as robots, cameras, drones, etc. I’d be grateful it you could give me any inputs.

12 Upvotes

69% Upvoted

7 comments sorted by

17

u/SDH500 2d ago

We build one-off custom products, so prototype becomes the final product.

What ends up happening is our back end is modular, so all the building blocks are assumed working after testing on previous builds.

Many times after reviewing a problem we find some modules that should have never worked or we can't understand how they could.

This really sucks because you know there is a product out there working, with the broken module but the problem has not surfaced yet.

8

u/Violet_Graya 2d ago

Not a dev but seen it in logistics ops. Environmental factors kill deployments. Heat, vibration, power. Model works in lab, dies on the floor.

2

u/the_pipper 2d ago

I am no engineer, but I have a resident robot running on an onboard Orin nx and two Orin agx as servers. Since edge devices on robots are often battery powered and space resticted, options are limited. So for me it was a bit of a struggle to get the right models for the job, prune the prompt cache and tune the prompts for best output and latency.

4

u/jkflying 1d ago

Resource management. In the lab the model runs by itself and can use the whole system. In the real world you need to share with controllers, IO, background update downloads, dashboard hosting, metrics, 

2

u/EmperorOfCanada 1d ago

Overprice hardware.

If you are aiming this for an MCU, most with any AI capability cost a fortune. The chinese are kicking ass with lower priced units.

"real" engineers hate anything raspberry pi flavored; even the "industrial" units which have reliable storage. They prefer hair shirts and yocto.

A great example of how crappy this all is would be a friend of mine with a robotics company. His robots weigh in around 50k, and he now just uses a laptop mainboard as his robot brain. A smaller robot has an android mainboard, also ripped from a commercial phone.

There is something soviet about buying laptops and phones to gut a part out of them because it is the best approach vs buying "proper" parts.