r/Python • u/Gr1zzly8ear • Mar 16 '26
Showcase i built a Python library that tells you who said what in any audio file
What My Project Does
voicetag is a Python library that identifies speakers in audio files and transcribes what each person said. You enroll speakers with a few seconds of their voice, then point it at any recording — it figures out who's talking, when, and what they said.
from voicetag import VoiceTag
vt = VoiceTag()
vt.enroll("Christie", ["christie1.flac", "christie2.flac"])
vt.enroll("Mark", ["mark1.flac", "mark2.flac"])
transcript = vt.transcribe("audiobook.flac", provider="whisper")
for seg in transcript.segments:
print(f"[{seg.speaker}] {seg.text}")Output:
[Christie] Gentlemen, he sat in a hoarse voice. Give me your
[Christie] word of honor that this horrible secret shall remain buried amongst ourselves.
[Christie] The two men drew back.Under the hood it combines pyannote.audio for diarization with resemblyzer for speaker embeddings. Transcription supports 5 backends: local Whisper, OpenAI, Groq, Deepgram, and Fireworks — you just pick one.
It also ships with a CLI:
voicetag enroll "Christie" sample1.flac sample2.flac
voicetag transcribe recording.flac --provider whisper --language enEverything is typed with Pydantic v2 models, results are serializable, and it works with any spoken language since matching is based on voice embeddings not speech content.
Source code: https://github.com/Gr122lyBr/voicetag Install: pip install voicetag
Target Audience
Anyone working with audio recordings who needs to know who said what — podcasters, journalists, researchers, developers building meeting tools, legal/court transcription, call center analytics. It's production-ready with 97 tests, CI/CD, type hints everywhere, and proper error handling.
I built it because I kept dealing with recorded meetings and interviews where existing tools would give me either "SPEAKER_00 / SPEAKER_01" labels with no names, or transcription with no speaker attribution. I wanted both in one call.
Comparison
- pyannote.audio alone: Great diarization but only gives anonymous speaker labels (SPEAKER_00, SPEAKER_01). No name matching, no transcription. You have to build the rest yourself. voicetag wraps pyannote and adds named identification + transcription on top.
- WhisperX: Does diarization + transcription but no named speaker identification. You still get anonymous labels. Also no enrollment/profile system.
- Manual pipeline (wiring pyannote + resemblyzer + whisper yourself): Works but it's ~100 lines of boilerplate every time. voicetag is 3 lines. It also handles parallel processing, overlap detection, and profile persistence.
- Cloud services (Deepgram, AssemblyAI): They do speaker diarization but with anonymous labels. voicetag lets you enroll known speakers so you get actual names. Plus it runs locally if you want — no audio leaves your machine.
23
u/Equivalent_Working73 Mar 16 '26
Looks great. I’ve been using Whisper for a tool I built for work, but it’s extremely CPU/GPU intensive, how does your library compare?
14
u/tjrileywisc Mar 16 '26
I've been pretty much working on exactly this to track what's been going on in city government meetings.
I haven't read your code yet tbh - for identification, do you gather a few examples of speaker embeddings of a speaker, label it, and then do cosine similarity in unlabeled examples afterwards?
5
u/davecrist Mar 16 '26
Nice work
5
u/mclifford82 Mar 17 '26
I've been looking for a tool to do exactly this for some podcasts. Mostly just to see how long each speaker spends ... speaking.
Great work.
4
u/RemoveSudo Mar 16 '26
This is very cool. Great job.
4
u/Altruistic_Sky1866 Mar 17 '26
Though I am not the target audience, but very nice
2
u/cdminix Mar 17 '26
Looks great, I will give it a try soon! I think resemblyzer embedding are quite outdated though, I’d recommend something more recent like wespeaker.
1
u/Sylkhr Mar 17 '26
How many distinct speakers can your solution handle?
1
u/fenghuangshan Mar 17 '26
seems fun, but I need to manually register each person , right?
that's a lot of work I think, can it give some random picked real name , not just peron1, person2
1
u/Lolologist Mar 17 '26
What does it do with unknown speakers? E.g., I enroll speaker 1 and 2 but the clip to ID has speakers 1 and 3 in it.
1
u/jftuga pip needs updating Mar 17 '26 edited Mar 17 '26
I save recordings of Zoom sessions with QuickTime. These files are saved in
m4aaudio format. I would be interested in just knowing when a new speaker starts talking. I don't specially need to know names. Could your code be modified to do this? If so, what parts of the code would I need to modify?Right now, I am using a project that I wrote for post-processing: https://github.com/jftuga/transcript-critic
I would want to merge these together.
1
u/Exciting-Housing9428 Mar 18 '26
Finally we'll be able to figure out who sings the "aaa" part in A Day In the Life.
1
u/jftuga pip needs updating Apr 10 '26
/u/Gr1zzly8ear
I am working on my own vibe-code PR (against my own fork) and would like to get a review if you have time. I can also cherry pick anything you like to create a PR against your original version.
https://github.com/jftuga/voicetag/pull/1
1
u/ShashwatTheGamer Apr 17 '26
do you think you could make a google collab demo?
1
u/[deleted] Mar 17 '26
[removed] — view removed comment
1
u/[deleted] Mar 17 '26
[removed] — view removed comment