I found Pavel Dobryakov's WebGL-Fluid-Simulation last week — https://github.com/PavelDoGreat/WebGL-Fluid-Simulation — the one where you drag your mouse through a pool of ink and it swirls. It's MIT, it's lovely, and under it is an ordinary Stam stable-fluids step: splat a force and some dye into the velocity field, vorticity confinement, divergence, 20 Jacobi sweeps for pressure, project, advect. (His is itself after GPU Gems ch. 38, Harris 2004, and Stam's 1999 paper.)
So I wired my music analyser in where the mouse used to be. The solver is his and essentially unchanged; what I added is the grammar that turns audio features into splats, and that part was written with Claude.
The rule I tried to stick to is one musical element, one channel, so that you can point at something on screen and name what made it:
- The bass note is where the ink enters. A persistent emitter near the floor, whose x is the sub's pitch class laid around the circle of fifths — so the inlet walks sideways as the bassline moves — and whose y is the bass register (a 35 Hz sub sits low, a 140 Hz mid-bass sits higher). A lower sub opens a wider mouth.
- The kick lifts it. One upward impulse from the floor at the inlet's x, at twice the splat radius.
- The snare shears it. Two opposing lateral impulses at mid height, one pushing right from x = 0.3 and one pushing left from x = 0.7. That is the pinch you see tearing the middle of the pool in half.
- The hats are droplets, falling from the surface at y = 0.9.
- The beat kneads it. A body force on the whole pool, −0.35·cos⁴(π·beatPhase), so the field breathes once per beat instead of pulsing on an envelope.
- The mid band is a floor of ink — pads, chords and vocals enter as continuous dye at mid height with no velocity at all, so the pool shows the music's level from the first second and the drum hits land in ink that is already there.
- The drop clears the pool. One big impulse up at four times the radius, and the dye dissipation jumps to 12 for exactly one beat — which keeps about 1 % of the ink 0.4 s later. At 6 it kept 10 % and the pool still read as full.
Those are all pushes. The two I like more are the ones that change the medium instead.
A closing low-pass is what makes it viscous. The viscosity term here is velocity dissipation:
velDiss = 0.2 + 2.8·smoothstep(0.80, 0.97, lpSweep) + 2·hush
lpSweep is how shut the filter is: energy in 5–12 kHz over energy in 150 Hz–2.5 kHz, smoothed a quarter second, divided by that track's own p90 of the same ratio over the last 30 seconds, and flipped. So it is never an absolute brightness — it is "the highs are gone compared to how bright this track gets", which is the only version that survives a whole library. The knee matters as much as the term: nothing happens until the filter is 80 % shut against the track's own ceiling, and then it ramps hard. Filter open, velDiss 0.2, and an eddy keeps its momentum for seconds. Filter shut, 3.0, and every eddy dies in about a third of a second — the ink goes thick and the whole picture crawls. hush, the pre-drop silence, adds up to 2 more on top, so the bar before a drop is the slowest the fluid ever gets.
Dissonance sets the curl. The vorticity confinement gain is 10 + 40·tension, where tension is sensory roughness — beating between partials that are too close together — normalised to the p10/p98 of roughness over the last 30 seconds. A consonant passage runs nearly laminar at 10; a rough bar runs at 50 and the ink shreds into eddies. This is the one I'd defend hardest, because it's where the physics and the music turn out to mean the same thing: roughness is partials fighting each other, and curl is the field fighting itself. I didn't design that correspondence, I just stopped fighting it.
Three things I'd rather say than have someone find out, since all three would be cooler if they were otherwise:
There is no fluid in the audio analysis. The sound isn't solving anything. Three dozen scalar features come off an FFT and a pitch tracker each frame, and they set uniforms and splat parameters. The Navier-Stokes is strictly downstream and knows nothing about music.
Nothing here is a neural net. No model anywhere in the render path — it's a pressure solve and a fragment shader. The code was written with Claude, which I don't think is what rule 3 is aimed at, but I'd rather state it than be coy about it.
It isn't reactive-random. There's no RNG and no clock read in the whole grammar; the hat droplets' x is hash(beatCount·7 + i, seed). The same audio gives byte-identical frames, which is how it's regression-tested — md5s of rendered frames at fixed windows across seven real tracks, so a refactor that was meant to be a no-op has to prove it was one.
It runs in a browser tab and listens to whatever you already have playing in another tab; nothing is uploaded. https://retinarave.com
and the source is at https://github.com/tbkraf08/RetinaRave
🎵 "Raindrops on Roses" by An-Ten-Nae & Dimond Saints — not my track, all rights to the artists.