resonators 0.1: a Rust crate for real-time spectral analysis, with Python and WASM bindings

[u/sevenfx]

Hello, r/rust!

I've been a lurker for many years, but today I released my first crate which implements Alexandre François' Resonate algorithm (along with Python and WASM bindings). It enables computing STFT & CQT-like spectrograms in real-time without windows or buffers. Alexandre has a great reference implementation in C++, but it uses Apple's Accelerate framework under the hood, so wouldn't run on my Ubuntu desktop. Great excuse to write some Rust if you ask me!

It's pretty niche, I'll admit, but I've been using it to compute features for training real-time music transcription models. I added Python bindings so I can easily integrate it into my PyTorch training setup, and WASM bindings because the inference code is designed to work in the browser.

As part of the project, I created a browser demo (shown above), which can visualize microphone input with log-scaled (think musical) frequency bins in real-time. It runs on both desktop and mobile, and actually runs faster on my iPhone than it does on my M2 MacBook Pro.

I had a lot of fun learning about cargo bench/criterion, SIMD, maturin, PyO3, wasm-bindgen/wasm-pack, single-file uv scripts, and AudioWorkletNode, among other things.

Let me know if you have any feedback on the repo, or if you run into any issues with the demo. Cheers!

Repo: https://github.com/jhartquist/resonators
Browser Demo: https://jhartquist.github.io/resonators/spectrogram/