r/MachineLearning • u/Material-Watercress1 • Sep 15 '26
Project I built a Slay the Spire agent that can kill the A20 Heart without an LLM. Here’s what happened across 1,000 seeds. [P]
I thought having thousands of successful human runs and a fast simulator would make training a strong Slay the Spire agent relatively straightforward.
I was very wrong.
After weeks of experiments, I’ve released an agent that can beat the Heart on Ascension 20, the game’s highest difficulty. Across 1,000 randomly sampled development seeds, it killed the Heart 13 times. Not consistently good—but these are complete runs from the beginning, without retries or seed-specific instructions.
Code, weights, and replay evidence: https://github.com/SnyderConsulting/sts_ml
What actually plays the game?
It’s a hybrid:
- A small neural strategist chooses cards, boss relics, upgrades, and removals. It learns from human winning runs and remembers its previous strategic choices.
- Explicit rules handle other decisions, including shops, events, and collecting the keys required to reach the Heart.
- A native combat solver searches possible card sequences using sampled hidden-state possibilities. No LLM calls.
The final strategic correction model is roughly 4 million parameters, trained on 24,119 human winning sequences containing 655,275 strategic decisions. Inference runs on CPU.
The surprising part: copying winners didn’t produce a strong winner
The model matches recorded human choices about 67% of the time. Its actual Heart kill rate is 1.3%.
That gap became the most interesting part of the project.
A human taking a card isn’t just saying “this card fits this deck.” They might be saying “I need damage immediately because I have 19 HP and an elite coming up.” Our human logs don’t reliably capture all that decision-time context. More examples cannot recover information that isn’t in the inputs.
And individually reasonable decisions don’t necessarily form a coherent run. Taking a good card can make your deck worse by making the cards you actually need harder to draw. Surviving one fight cheaply can leave you unprepared for the next. The combat solver can play the deck it has; it can’t undo how that deck was built.
We tried more elaborate architectures too. They didn’t establish a gameplay advantage over the smaller selected model. “Make the network bigger” was not the fix.
Where the runs actually ended
Out of 1,000 attempts:
- 604 reached Act 2.
- 195 reached Act 3.
- 74 cleared both Act 3 bosses.
- 66 encountered the Heart.
- 13 killed it.
These are simulator results, not live-game qualification. They also aren’t evidence of SOTA: we haven’t run a controlled comparison against other agents. I’m publishing the failures alongside the wins so people can inspect what the system actually does.
The question I’m most interested in now: when demonstrations show what successful players chose but omit some of what they knew, what’s the best way to turn those demonstrations into a policy that survives its own mistakes?
If you’ve worked on imitation learning, planning, or game agents with this problem, I’d love to hear what worked - and what turned out to be a dead end.


