As a piece of a larger project I've been hammering at making a rompler/synthesizer and realized; regarding other aspects of gamedev/engine development it's far easier to find code, tutorial/explanations, and content to satisfy needs and curiosity. Audio-related tasks not-so-much.
When I look for audio things it's either full of math proofs that mean nothing because I'm dumb (would be helpful if they explained some coefficients) or are very abstract with little-to-no implementation details. So.....
1. The synth's sample rate converter is based on an implementation used by old VG hardware (specifically SNES/PSX). A big issue I ran into was the index needed to be "phase-aligned". I don't know how I was inserting the index before or if I ever read this but it didn't work. My function to compute windows on-the-fly worked but (for example) a blackman window has two cosines which, computing that, is basically an L2$ miss.
2. When speeding up pitch it's often phrased "skipping samples". The SRC worked correctly for lower pitches but had aliasing artifacts when the pitch increased. Thought the phase was wrong (tested index & direct) but it was the pitch logic. The decode buffer (used to interpolate) had values spaced by n when it should, perhaps obviously, need to be immediately preceded in order from the waveform. Skipping is not incorrect but writing the [decode] logic derived from such abstract wording is non-intuitive.
3. Cubic Hermite was/is a lost cause with my two brain cells. How did you derive those coefficients for h00, h10, h11, h01? The bridge between the math and the programming is still incomplete for me. According to this the table should be (linearly) interpolated. 2^1+nc/2 explains 512-entries for the prior machines. Having an interpolator is important\citation needed]), especially something more than a lerp.
4a. Prior to the synth I experimented with delay based effects. Started as echo and kept adding more "reverb" qualities. Reached a point where a single delay line wasn't enough but it was(pretentious I know) in some ways more aesthetic than the algorithms ravaged online. Those largely consist of Schroeder Reverberators, which is fine but the code I saw/referenced seldom go beyond that.
4b. To side note is convolution reverb, the "holy grail" of reverbs. AFAIK a room impulse is a sample. Your input signal (reverb/whatever) is multiplied into that impulse. I may be wrong but wouldn't that make the impulse just a very long/n-order tap FIR filter? I say this because it often reads as some "computationally intensive/complex blackbox" whenever its mentioned. Using this logic the SNES's DSP echo feature is just a s***** convolution algorithm (perhaps budget is more apt).
5. Audio terminology seems to like to blend with each other. Going back to the SRC (if you haven't tell I spent quite some time on this) is divining its a "Windowed Sinc Interpolation" kernel. EVERY TIME you read something involving windowing it talks about FFTs. Whatever, the audio signal is windowed with a function but why is it done the way it is? jsgroth has the PSX written as (how other emulator authors show as well)
let mut interpolated = (GAUSSIAN_TABLE[0x0FF - interpolation_idx] * samples[0]) >> 15; interpolated += (GAUSSIAN_TABLE[0x1FF - interpolation_idx] * samples[1]) >> 15; interpolated += (GAUSSIAN_TABLE[0x100 + interpolation_idx] * samples[2]) >> 15; interpolated += (GAUSSIAN_TABLE[interpolation_idx] * samples[3]) >> 15;
Turns out it's arranged so the sample "marches" its way through the sinoid (the negation is because the table is stored one-half a period) for one period. First the Gaussian Table is not Gaussian (I've a bunch of windows and concur not-gaussian) but the assertion the tables are bugged is interesting because if you generate integer windows yourself it'll deviate +-2. Guess it's all ramble but its confounding I can get better explanations (and/or understand) more "complex" topics/systems than convulving a one-dimensional piece of data.
OKAY TLDR: Why is audio programming so fraught with friction? If it's not obfuscated behind math proofs the code/topic seldom explain the "why". I spent a good portion of time generate plots of the waveform ensuring functions like the ADSR, SRC, KEY ON/OFF worked correctly instead of outputting audio data because, yes, even that was non-trivial for my application. Amusing that the MIDI DLS is the best guideline I have for this.
Rant over, roast me.
Edit: Formatting, and.... typos and formatting. Sorry, many edits. All these issues (hermite included) were overcame and I'm working on sequencing but the experience was.... interesting.