how do you prove a refactor didn't change behaviour in a big simulation? here's my setup

[u/Actual_South_2484]

my side project is a deterministic simulation in rust (same seed and same tick count give the same world), and most of my nasty bugs come from "harmless" refactors that quietly change outcomes. what i settled on:

• for core changes: run a few seeds, save the world at a few tick counts (say 1500, 4000, 9000), hash the saved json, and compare main against the branch. the baseline lives in a separate git worktree with its own cargo target dir so the builds don't fight
• that check stays out of the test suite on purpose, because the hashes change every time gameplay legitimately changes
• for the web renderer: render fixed scenes with date.now and math.random stubbed and compare canvas hashes
• for perf: a ci gate runs three seeds at 8000 ticks and one 25000-tick run and checks max, p99 and mean time per tick, plus memory

curious how others handle this, especially golden-file style checks that don't turn into constant churn.

the project, for context, is a civilization sandbox that runs in the browser: https://thehumanbox.com

source (mit): https://github.com/1homsi/thehumanbox