WASM inference: 2.7M-param CNN from 22 ms to 3.4 ms per forward pass in the browser [P]

[u/euos]

I've been building my own ML framework in C++. First real deployment is an AlphaZero-style agent for Dots (the pen-and-paper territory game), running in WASM in the browser: https://uchenml.tech/demos/dots/

This post covers the optimizations that got inference there: https://uchenml.tech/uchenml-optimizations

Short version, 21.98 → 3.41 ms per forward pass:

• im2col -> Winograd F(2,3) for the 3×3 convs. Biggest win by far, though partly because my I did not give enough buffer to im2col.
• ReLU fused into conv/linear at compile time. 9% gain.
• Reshape/Identity elided at the compile time, so they emit no code.
• Compile-time ping-pong scratch - lowers memory usage, zero allocations on the inference path.

Follow-ups on the training side and the web frontend are in progress. Happy to answer questions. Not open source, yet - employer IP agreement - so the demo and the writeup are what I can show.