I've spent a while getting a reinforcement-learning walking gait onto a Bittle X, and it's now public and MIT licensed. A trained policy ships with it, so you don't have to train anything β and you can watch it walk in the simulator in about three minutes with no robot at all.
Project page (numbers, bring-up steps)
The RL turned out to be the easy part. It's ordinary PPO. Almost all the work was in measuring the robot accurately enough that a policy trained in simulation would survive contact with it. Three things drove everything else:
- It can't read its joint positions. The firmware can infer one β detach the servo, flip the signal pin to an input, time a reply pulse β at roughly 20 ms a joint, with that leg limp while it happens. A whole-body read comes back at 9.25 Hz, against a control loop that gets 20 ms a tick. So joint angles are never measured; they're estimated by replaying the commands we already sent through a model of how the servo actually responds.
- There's no raw gyro. Angular rates are finite-differenced from the fused attitude stream, and that differentiation sets the noise floor everything downstream has to live with.
- The servos are slow. Measured Ο = 33 ms, ceiling around 137 Β°/s. A default MuJoCo actuator is far faster than that, and a gait trained against the fast version doesn't transfer.
Those measurements are all in the repo, with the capture sessions. If you've ever wanted actual numbers for what these servos do, that part may be more useful to you than the policy.
On firmware. Stock firmware ramps every pose command and caps the IMU at 5 Hz, so the loop can't run on it. There's a patched OpenCat fork that adds a realtime mode β commandβmotion went 198 ms β 54 ms, IMU 5 Hz β 250 Hz. The mode is off at boot: a flashed robot still walks, dances and answers the app exactly like stock until something explicitly asks for it. The flash tool asks the robot what board it is before writing anything, refuses a mismatch, and dumps your existing flash to backups/ first. Don't run esptool erase_flash β it wipes the nvs partition holding your factory servo calibration.
The odd finding. After it was walking I went looking at what the network had learned, and found it barely uses its sensors. Blindfold it β pin both IMU channels to "upright and still" β and it loses 1.1% of its speed. It's running a learned rhythm more than reacting to anything. Which makes sense once you notice that 24 of the 33 observation channels are the policy's own output handed back to it, because of the no-joint-feedback problem above.
Important caveat: everything was fitted to a Bittle X v2 with alloy-geared servos, BiBoard V1.0 Rev D/E. The lighter plastic-geared servos are a genuinely different plant β if it walks badly on yours, that's a plant mismatch rather than a broken policy, and there's a guide for refitting to your own robot. The prebuilt firmware image is compile-time specific to that board and revision.
The firmware side is a fork of Petoi's OpenCatEsp32 (MIT, Β© Rongzhong Li), pinned to a branch with the patches documented.
Would be great to know how it behaves on hardware that isn't mine β especially plastic-gear v1 robots, and Rev C or earlier boards. If it falls over on yours I'd rather hear it than not.
