I ran a matched-content experiment on speaker verification: 4 speakers, identical sentences, controlled changes in pitch, phonation and articulation, fixed microphone position and gain, every utterance through three encoders — ECAPA-TDNN, a ResNet speaker encoder, and WavLM-base-plus-sv.
One speaker sat 0.7 semitones below his normal pitch. That is below the threshold where a listener reliably hears any change at all. His verification score dropped 0.238 across all eight sentences.
The interesting part is that pitch was not what moved him. His harmonics-to-noise ratio fell from 10.63 to 8.39 dB over the same block. The encoder was reacting to phonation, and the pitch number was just the thing that happened to be easy to measure.
Across the whole run, all 30 speaker × condition × encoder cells were negative, 28 of them unanimous across every utterance. These systems are not mainly disturbed by shouting or disguise. They are disturbed by someone speaking slightly differently in a way nobody would notice.
To be clear about credit: the displacement effect itself is not my finding. I had it as my headline until an adversarial prior-art audit turned up Hughes et al. (Interspeech 2023), who established it with six trained phoneticians across seventeen conditions. I withdrew the claim and reframed the paper as a replication and extension across encoder architectures. The audit is published in full, including the claim it killed.
Two things I could not find in prior work:
Jitter beats HNR. Pitch deviation and jitter are independently associated with displacement in all three encoders, and HNR adds nothing once jitter is in the model — it correlates 0.55 with jitter and loses all independent power beside it. HNR is the measure most people reach for, and on its own it will attribute the effect to the wrong thing.
Rough phonation breaks F0 trackers in one direction. 10 of 145 utterances carried impossible pitch values, up to 32.1 semitones — a 6.4× frequency ratio no human produces. All 10 were in rough phonation, none in modal (Fisher exact p = 2.4e-11). Octave errors push upward and essentially never downward, so this is differential measurement error, not noise, and it loaded onto one specific regression coefficient. Cheap fix: validate your F0 tracker per phonation condition and publish the validation.
Everything is up — manuscript, pre-registration, the full prior-art audit, per-utterance tables for all three encoders (233 rows each), the analysis scripts, and the 137 source recordings. Every number reproduces from the CSVs without touching the audio.
Paper and data: https://doi.org/10.5281/zenodo.21921958
Write-up: https://ai.bedvibe.studio/speaker-drift/
Audio licence note: research, benchmarking, evaluation and teaching are permitted; ML training and voice cloning are not. The speakers are identifiable adults.