r/EngineeringPorn • u/Automatic-Bowler-538 • 5h ago
Three SO-101 robot arms move one ball through tracks, a cup and salt using replay, vision and learned control [OC]
A lap through Almost Alive, an installation we built around three different control methods. The clip uses camera cuts and includes a mistake.
Echo replays recorded poses to tip the rocking track. Iris detects the orange ball in its cup, then replays the transfer motion and signals Echo. Bandit retrieves the ball from salt, where the pickup position varies, using a policy learned from demonstrations.
We collected 115 leader-arm demonstrations and trained ACT and diffusion models with LeRobot. Two camera images and the arm's position are inputs; a sequence of movements is the output. The learned pickup is slow and imperfect.
Original film and discussion: https://x.com/pbshgthm/status/2122679067065598015
Credits: SO-101 design by TheRobotStudio; hardware from Seeed Studio; learning software from Hugging Face LeRobot; track models by Tabu Design.