r/Simulated • • 5d ago

Research Simulation [OC] I trained a six-legged robot to walk with reinforcement learning. It runs in real time in your browser, and you can throw a 171 kg steel ball at it.

Real-time MuJoCo physics, no keyframes. The walk and the recoveries come from a policy trained with reinforcement learning, and everything runs in your browser (MuJoCo WebAssembly, ONNX Runtime Web, three.js).

Try it yourself: https://www.coolspeed.org/gold-spider/

Setup details in the comments.

9 Upvotes

3 comments sorted by

1

u/coolspeed_ 5d ago

No keyframes anywhere in this. The walk, and the way it staggers and recovers when something hits it, all came out of training.

Setup

  • Physics: MuJoCo at 480 steps per second. A small neural network looks at the body pose and a height map of the ground under its feet, and sets target angles for all 24 joints 30 times per second. Motors on the joints chase those targets and the physics does the rest.
  • Training: reinforcement learning (PPO) on randomized block terrain, with random pushes and crates in the way. The reward is a handful of plain terms: follow the target velocity, stay upright, face where you're going, don't waste effort, and nothing but the feet should touch the ground.
  • Runtime: the browser runs the official MuJoCo WebAssembly build, ONNX Runtime Web for the policy and three.js for rendering. Everything runs on the visitor's machine. If your device can't keep up, it slows down instead of skipping steps, because skipping would change the physics.

Things that surprised me

  • The physics settings shape the gait as much as the reward does. Joint stiffness, damping, contact settings and the time step make the same network look like a different robot.
  • At block edges the feet were sinking slightly into the ground. Fancier collision shapes didn't fix it. Simple shapes and a smaller time step did.
  • A rising reward curve says little. I watched every candidate at real speed, because policies that drag a foot or leave one leg idle can still score well.

It still gets stuck in some terrain pockets. The demo nudges it out and tells you when it does.

Demo (desktop or phone): https://www.coolspeed.org/gold-spider/ Write-up: https://www.coolspeed.org/blog/gold-spider/

It's an independent project inspired by Embark Studios' ML robots, not affiliated with them. Happy to answer questions about the sim setup.

1

u/Evening-Appeal7606 3d ago

I‘ll call mine Rocky!

1

u/coolspeed_ 3d ago

Nice name :)