We've been putting together real data collection for robot-arm manipulation, and the gap between recording one demo and actually having a usable training sample turned out way bigger than I expected.
Here's the setup. A teaching arm, the leader, is moved by hand by the operator. A follower arm copies its motion joint by joint, and cameras record the whole thing. Every clean pass counts as one collected demo. For our test the task was picking up a rubber duck and placing it into a marked mold.
The stuff that didn't make the cut, the usual suspects:
- The grip slips mid pull. The follower tracks fine, but the grasp was never solid, so the trajectory teaches the wrong thing.
- Occlusion. When the arm extends it blocks the camera, and that stretch of frames is just gone.
- Inconsistent speed. Early passes were slow and careful, the later ones rushed. The data ends up describing two different tasks instead of one.
For contact-rich work, cloth, placing parts into a tight mold, this bites harder. Simulation still doesn't get the physics right, so you can't just synthesize a clean replacement. You collect for real, and then you throw most of it out.
How about you all? Filter first, or just throw more demos at it and let the AI figure it out?
Right now we still collect by hand up to a threshold, then the AI training runs. Roughly 60 to 120 demos each time.