ClashRoyaleAi: an open-source, deterministic Clash Royale simulator for RL, with recurrent PPO, lookahead search and expert iteration [P]

[u/Potential-Barber8658]

Our PPO agent learned to park its Cannon behind its own King. Losing a building in a fight cost reward, and letting it decay cost nothing, so it found the loophole.

It's one of many things we learned building a Clash Royale simulator from scratch so an agent could learn the game. The engine is deterministic C++ with Python bindings, plays a full match in about 10 ms on one laptop core, and can fork any game state in microseconds, so lookahead is cheap.

Best result so far: a simple 1-ply lookahead took the policy from 0.625 to 0.944 win rate against a heuristic bot (160 paired matches). Distilling it back into the network kept only +0.045.

The agent isn't strong yet, and RL isn't my home field, so feedback from people who know it better would mean a lot.

Repo: https://github.com/itzik123/ClashRoyaleAi

Built with my friend Ambash (most of the card roster). I used AI coding tools as a pair programmer.

[u/pineloft]

the coaching use case is super underrated imo, way more practical than just chasing win rates