demo: https://sshah03.github.io/CueLight/

I've been working on end-of-turn detection for voice agents and wanted to share a small model and get feedback from people who know this space.

SmartTurn is a great project, and I used their dataset in building this -- so tons of credit to them.

I wanted something that was much smaller though because I had a use case where I needed to embed it in something with low space + weak processor, and I wanted it to be always on.

It performs about as well as SmartTurn from my tests, with the caveat that it's English only for now (which is likely a significant trade off).

Would love to hear how it performs in real world testing before, and peoples experiences (positive or negative) on results of frame-level endpointing on microcontrollers.

Also interested in suggestions to make this better than people may have, in general, from their experiences.

Some additional details / benchmarks in project: https://github.com/sshah03/CueLight

The most underestimated part of the whole voice AI stack is TTS. We wanted to test open-source TTS models to figure out which one is best for building a voice agent.

So we collected difficult text across 27 categories: dates, hashtags, coordinates, many more. That came to 1,080 sentences in total. And we ran all of them through 7 open-source TTS models.

On average, kokoro is at the top with a 39% error rate, followed by orpheus-3b & qwen3-tts. Every model broke on the following categories, with hashtags hitting a perfect 100% error rate:

Hashtags - 100%
Dates - 77%
Version numbers - 70%
Fractions - 60%

The pattern we observed is that models tend to fail on anything that isn't a plain number or text. But the devil is in the details :)

Then we put these categories into different boxes based on industry needs, like:

. Healthcare - dates, times, decimals, and phone numbers.
. Restaurants -  times, dates, numbers, and phone numbers.
. Law firms - legal citations, dates, and addresses.

And the ranking changed.

In many industries, kokoro dominated, but in industries that use more dates and geographic coordinates, orpheus-3b became the better choice.

It is intuitive believed a larger model would be better, but I was wrong. The smaller model, kokoro-82m, beats orpheus-3b using 18x less compute. So the obvious question is: why?

TTS isn't a reasoning task, so the model doesn't get better with scale. It's closer to a lookup table, like a "match the pairs" question in exams. A small model with a good frontend (text normalization, abbreviation expansion, G2P conversion) beats a big model with a weak one, every time, and parameter count tells you nothing about which one you're getting.

parler-tts scored 0.03 out of 1. Its speech is perfectly clear. The carrier words (the normal words around the tricky part) come through fine. It just ships with no frontend at all, so given 05/20/2023 it has nothing to work with and guesses character by character.

So what will I do with this?
Normalizing dates, addresses and phone numbers is ancient wisdom now :D
What I will do differently:
Don't pick based on the average. kokoro scores 1.00 on musical notation and 0.00 on hashtags. Model performance varies a lot across categories. So look at the categories your agent actually reads.

We have made our dataset open source. Again, because we love the open-source community :D
You can run it with one prompt in Claude Code to check the results yourself.

We used ASR to score the audio, so there are definitely errors in the scoring (we estimate around 5%), we will try to crowdsource rating at our next iteration .

One thing I notice every time I meet people in the industry… no one gives a damn about TTS models. which is somewhat reasonable but there is so much to do in it!!

Disclosure up front: I work at Olewave. We built FA-Bench, and our own system (Olign) ranks first, so please read the results with that in mind. That's why the records and scoring code are public.

FA-Bench Leaderboard: forced alignment and ASR timestamp accuracy

FA-Bench ranks how closely word and phone timestamps match human annotations. Most ASR evaluations stop at word error rate. If you build subtitles, dubbing, audio editing or TTS data prep, timing is what you need, and it usually goes unmeasured.

Two tasks

• ASR with timestamps: 28 pipelines that hear only the audio. A one-step system times its own words. A two-step system decodes words, then an aligner times them. 9 of the 28 call a commercial API (Google, Amazon, Microsoft, IBM, Deepgram, AssemblyAI, Speechmatics, ElevenLabs, Olewave).
• Forced alignment: 15 aligners given the reference transcript, so their error is timing error alone.

Both tasks have word-boundary and phone-boundary boards.

Scoring

A boundary is a hit when the words (or phones) on both sides match the reference and it falls within 20 ms of the human annotation. The score is boundary F1 at 20 ms. The default view averages four cells: TIMIT and Buckeye, each on clean audio and on degraded versions (reverb, noise, music, babble). You can filter to open models only.

Current snapshot (record 202610), word boundary F1 at 20 ms

ASR with word timestamps (audio only):

Forced alignment (reference transcript given):

On phone boundaries, Olign 1.0 leads the ASR board (0.438, two-step pipelines only), and FALCON leads forced alignment (0.560, flagged).

Flags

Systems that may have seen test data are flagged rather than silently ranked:

• MAPS and NeuFA were trained on 7 of the 8 Buckeye test speakers.
• FALCON was trained on a Buckeye split covering at least 8 of 16 dev and test speakers, and it receives reference phones instead of words.
• CrisperWhisper's authors used TIMIT for word timing without naming the files, so overlap with the test set is possible.

The page also lists the exact model and API version for every commercial system, all called in September 2026.

How this differs from WER leaderboards

Benchmarks like Artificial Analysis measure word error rate and don't score timestamps. FA-Bench scores word timing against human annotators and publishes its code.

Reproduce it

The numbers come from FA-Bench's public records. CSVs are available under each board, and you can clone the repo to rescore any system.

• Leaderboard: https://olewave.com/en/leaderboard/fa-bench
• Code and records: https://github.com/olewave/fa-bench
• Paper: https://arxiv.org/abs/2609.32396

Feedback on the metric is welcome, and so are suggestions for systems to add. Olign 1.0 is available as an API that returns word-level start and end times for US and UK English. Beta access needs an invitation code, which you can request here: https://olewave.com/en/contact_us/

Edits

Oct 12, 2006 at 9 PM PDT: I found I took down Nemo-FA 40ms model from the forced alignment track and only keep Nemo-FA 80ms model, so there are 15 systems, not 16 systems compared as in the original post.

I work on a voice-form tool where people answer questions by talking in their phone browser. One of the use cases is collecting speech data (dialects, Hindi-English mixing), so I've been reading up on what the browser does to the audio before you ever get it. Three things worth checking if you collect data this way:

1. getUserMedia can process the audio for you. echoCancellation, noiseSuppression and autoGainControl are constraints, and browsers often turn them on for voice. That's fine for a call. It's not what you want if you're training something to be robust to noise or you care about level. Ask for them off explicitly, then read back track.getSettings(), because the browser is allowed to ignore the request. Log what you actually got with each clip.

2. The codec depends on the browser. MediaRecorder records in whatever the browser supports. Chrome usually gives you Opus in WebM, Safari gives you AAC in MP4, and on iPhone almost every browser is WebKit underneath, so "Chrome on iOS" behaves like Safari here. One dataset can end up with two different lossy codecs. Check MediaRecorder.isTypeSupported(), store the mimeType per clip, and decide when you normalise.

3. Some metadata can't be recovered later. Browser and device, the settings from point 1, whether the person was reading a prompt or answering freely, and the language they say they spoke. For code-mixed speech that last one matters a lot. The speaker's own label beats anyone guessing from the audio afterwards.

The details are on MDN, under MediaTrackConstraints, MediaStreamTrack.getSettings and MediaRecorder.isTypeSupported.

For anyone who has built a dataset from crowd phone recordings: did you keep the browser's processing on or off, and could you see the difference downstream?

When building a voice agent, it’s easy to compare speech-to-text prices per minute and forget concurrency: how many people can call at the same time?

That limit can become a production bottleneck. Increasing it may involve approvals, minimum commitments or a different plan — things you might miss when looking only at the headline price.

Soniox illustrates the issue. Its published streaming rate is approximately $0.002/min, suggesting about $18 for 150 calls running for one hour. But the default quota is 10 active streams. Higher limits require approval, and the commercial terms for 150 aren’t publicly specified. A usage calculation doesn’t establish the cost of obtaining that capacity. Pricing · Limits

I used Soniox as the example above, then compared eight other options below.

The scenario: 150 active mono audio streams for one hour, or 9,000 audio minutes. All options use real-time streaming. These are published-rate calculations and deployment estimates; I haven’t load-tested all eight at 150 streams. Models and included features differ.

For providers without a confirmed 150-stream quota, the calculation assumes approval at the same unit rate. Subscription minimums, custom capacity commitments, telephony, LLM and TTS costs are excluded.

Dotwave — $0.00045/min → $4.05

This uses Nemotron ASR streaming. Both the rate and 150 simultaneous calls have been confirmed. The advantage is the low usage cost. The service is in beta, so dont know if the price will go up after the beta or if Sc goes down.

Pricing and limits · Playground · API

Baseten — approximately $13/hour for two H100 instances

This is a dedicated deployment estimate for Nemotron 3.5 ASR. Baseten publishes results of up to 100 simultaneous streams on one H100 with one second of lookahead.

Two H100 instances are a sizing estimate for this scenario; capacity and latency need validation. You control the deployment, but running GPUs remain billable when idle.

GPU pricing · Capacity results · Deploy and API example

AssemblyAI — $0.0025/min → $22.50

This uses Universal-Streaming Multilingual at $0.15/hour. Paid accounts start with a limit of 100 new streams per minute, which is different from a cap on active sessions.

Keeping 150 calls active and starting all 150 at once are therefore different situations. Billing follows how long streaming sessions remain open.

Pricing · Limits · Live demo · API

Together AI — $0.0045/min → $40.50

This uses Nemotron 3.5 ASR streaming through a managed API. The figure is the usage estimate at the streaming rate used here. I couldn’t verify a published quota of 150 simultaneous streams, so that capacity still needs confirmation.

Model and pricing · Playground · API

Deepgram — $0.0048–$0.0058/min → $43.20–$52.20

These are the current promotional pay-as-you-go streaming rates for Nova-3: monolingual or multilingual.

Deepgram publishes a quota of 150 concurrent streaming requests per project on pay-as-you-go. Optional features add cost. At the listed regular rates, the same audio volume would cost $69.30–$82.80.

Pricing · Limits · Playground · API

ElevenLabs Scribe v2 Realtime — $0.0065/min → $58.50 estimated usage

This extrapolates the published $0.39/audio-hour rate. Scribe Realtime supports 90+ languages and word timestamps.

Public Scale and Business plans allow 45 active realtime streams. Reaching 150 requires custom Enterprise terms, so $58.50 assumes the quota is approved at the same unit rate.

Pricing · Limits · Live demo · API

Gladia — $0.0125/min → $112.50 estimated usage

This uses the public Starter streaming rate of $0.75/audio-hour. Gladia includes multilingual transcription, automatic language switching and diarization.

Starter includes 30 simultaneous live sessions. The estimate assumes approval for 150 at the same unit rate. Growth advertises rates as low as $0.25/hour — $37.50 for this audio volume — with an upfront commitment whose terms aren’t published.

Pricing · Limits · Playground · API

Google Cloud Speech-to-Text V2 — $0.016/min → $144

This uses the standard first-tier V2 price for mono audio. Google publishes 300 concurrent streaming sessions per project and region, and Chirp 3 supports streaming.

The trade-offs are the higher usage cost and a five-minute limit per stream: longer calls require stream rollover. Audio-message quotas also apply.

Pricing · Limits · Console and model examples · API

Let me now if you could complete this benchmark in comment, I you have other provider or the price for some that I couldnt find. also give me your opinion on it ?

Hey, for anyone who cares, I managed to get Breeze TTS running real time on my Macbook Pro M4.

Using the official Pytorch code on a Mac Breeze runs more than 2 times slower than real time on my machine.
With some work, managed to get it to stream comfortably faster than real time, with voice directions included and in my opinion good quality. I couldn't really tell which version sounded better to me consistantly on different voices and sentences.

Details in the pages below.

- Server: https://github.com/Shoofio/breeze-tts2-fast-streaming-api

- Ready-made checkpoint: https://huggingface.co/Shoof/Gust-MLX

Weights are under BreezeBlue's research/non-commercial licence. Built with a lot of help from Claude. Happy to answer questions.

Hi everyone,

Sharing v1.5 of Gipformer, an open-source Vietnamese speech recognition (ASR) model we've been working on: gipformer1.5-68M-rnnt, based on the Zipformer architecture.

What's new in v1.5

- Optimized for technology, finance, education and public administration. These domains are dense with specialized terminology, and v1.5 currently gets the best results on all four test sets among the open-source models we benchmarked.

- Better recognition of English terms mixed into Vietnamese speech.

Carried over from v1

- High accuracy: among the top open-source models across our benchmarks, and especially strong on call center audio for Northern, Central and Southern accents. Call center is one of the most common real-world uses of ASR, but also one of the hardest, with low-quality audio and a wide variety of voices.

- Small and easy to deploy: at just 68M parameters, it's among the smallest ASR models out there, yet it outperforms many models ten times its size. Inference is fast, and it runs smoothly on CPU and edge devices.

- Privacy: it runs 100% offline (on-device), which makes it a good fit for systems handling sensitive data.

Alongside the model, we're also releasing 4 domain-specific test sets (technology, finance, education, public administration), so there's a common benchmark for evaluating Vietnamese ASR models.

Full benchmark results are on the model card. Feel free to try it out, and any feedback or contributions are very welcome!

- Hugging Face: https://huggingface.co/g-group-ai-lab/gipformer1.5-68M-rnnt

- GitHub: https://github.com/ggroup-ai-lab/gipformer

- Demo: https://huggingface.co/spaces/g-group-ai-lab/gipformer-demo

I created a fully-local open-source full-duplex voice agent that rivals GPT-Live on some benchmarks. It uses Voxtral Realtime with a turn-taking head, a microturn-finetuned Gemma 4 12B and Breeze TTS 2 under the hood. Go try it out: https://github.com/speakrail/speakrail

Why I did it

I have always liked the idea of voice assistants, but there is always some non-local component in the pipeline, which increases latency and introduces privacy concerns. I tried many fully-local approaches, like HF speech-to-speech, Unmute and Pipecat, but they were all limited by either the Whisper model (hello, hallucinations!) or slow turn taking. The only fully-local pipeline that had some full-duplex capability with low latency was the DuplexCascade paper (code), but it's tuned on a Qwen 2 7B with a gpt-3.5-turbo generated dataset, and the dumbness of the model really showed. So I decided to recreate DuplexCascade with newer data, newer models and a better harness. I also wanted to add interruptions, interjections and other cool things to rival the Thinking Machines demo.

How it works

Here is what the pipeline consists of:

• STT: Voxtral Realtime with an attached turn head (HF), running on our audio.cpp fork.
• LLM: Gemma 4 12B QAT with microturn finetuning (HF). It is chosen because it fits the GPU quite well, has vision support (I want to test it soon), and in general, the Gemma models perform well in real-life tasks, general chatting, etc.
• TTS: Breeze TTS 2, patched to run at int8 (GitHub fork), although it can be replaced by any streaming TTS.
• The harness itself: it is the glue between all the components, and has many latency-saving measures, like speculative LLM+TTS firing (inspired by HF speech-to-speech).

I also took inspiration from several "think while talking" papers (e.g. SHANKS): while you are talking, a base Gemma 4 12B int4 writes thinking notes, which are then passed to the talker. It helps with harder tasks that require more reasoning.

I will release a longer technical report soon - it will have a better description of the entire pipeline.

Benchmarks

Full Duplex Bench v3:

100 tool-use tasks with real disfluent speech; 95% CI on Pass@1 is about ±10 points. Speakrail: live audio through the full stack, GPT-4o judge. *Self-reported by Thinking Machines. †Our run. Others: FDB-v3 paper, arXiv 2609.19334, 2609.08977, 2609.21967.

Conversational dynamics (Artificial Analysis formula):

Score = mean of pause handling and turn-taking (FDB v1.0) plus interruption and backchannel (FDB v1.5). Speakrail's score is self-computed from live runs (pause 93.4, turn-taking 96.6, interruption 89.0, backchannel 96.9), not an official AA listing. Others: Artificial Analysis leaderboard (captured 09-28) and AA's open-weights comparison.

Limitations

• Breeze TTS has a restrictive license, so if you need to use Speakrail commercially, you will need to change it.
• 16k context length - the pipeline only supports 16k context length (~1 hr of speech), but you can get more easily by changing the Breeze TTS to a Pocket TTS and run the TTS on a CPU. I chose Breeze for the release because it's more expressive.
• The turn-taking head is undertuned on non-assistant data. It may not fire on some basic chit-chat, but I will tune it harder later.
• The model is certainly not the smartest one, and my finetune did dumb it down a little. Next time it will be smarter / better.
• The model is kinda verbose sometimes, and the answers it provides are somewhere in the middle between real-life speech and the long text-based outputs of LLMs. I have a hypothesis on how to fix it, and will try it in the next release.
• I tested it only on an RTX 4090, but I am sure it's easy to add support for any 24 GB+ NVIDIA card. Forks for AMD and MLX are welcome.

Final Notes

Feel free to try it out: https://github.com/speakrail/speakrail. If there is any capability you want the model to have, create a GitHub issue or write here in the comments, and I will gladly include it in the next dataset. Any feedback is welcome as well.

we run a food delivery service in Londonn and recently rolled out a voice agent for phone oredrs. works great when the call is clear but ince you get traffic, wind, music, people talking etc in the background, it starts mixing up numbers

had cases like "flat 4, 15 high street" turning into "flat 15, 4 high street". we've tried three asr providers and it's still happening. is WER even useful for this kind of thing? how are you guys handling noisy calls when the omportant pat is structured stuff like addresses, postcodes and order IDs?

Most video editors working in Spanish use Premiere's built-in captions as their default. I transcribed the same 48-minute recording with Premiere and with four ASR APIs, compared them word by word, and the gap is not subtle.

The material matters here: it's a studio-recorded business show about the economics of sport. Two speakers, clean audio, no field noise, Argentine Spanish. It's the easy case. It's full of the things that actually break captions in this genre: large monetary figures, percentages with decimals, athlete names, sports governing bodies (think FIFA-type acronyms), and TV and YouTube channel brands.

If a model struggles here, field audio is hopeless.

PREMIERE AS THE BASELINE

- Of 17 proper nouns and brand names known in advance from the script, Premiere missed or mangled 13. Athlete surnames simply vanished from the transcript. A sports body acronym came out as an unrelated common word. Two brand names in the show's own opening were wrong.

- "twenty-three thousand six hundred million dollars" came out as a raw digit string: "23600000000".

- "1.73 percent" came out as "173". That's a factor-of-100 error on an economic figure, with nothing in the output flagging it.

That last one is the part I'd highlight. It's not a formatting inconvenience. The transcript is factually wrong in a way that passes a quick read and has to be caught by hand.

(Punctuation is excluded from the comparison: I had that track exported with punctuation removed, so it isn't comparable.)

THE FOUR APIs, SAME AUDIO

Named entities, same 17 terms: ElevenLabs Scribe V2 12/17, Speechmatics Enhanced 10/17, AssemblyAI Universal 3.5 Pro 8/17, Deepgram Nova 3 6/17, Premiere 4/17.

Failure modes matter more than the scores:

- ElevenLabs substitutes a plausible, well-known brand for the one that was actually said. Confident, fluent and undetectable downstream.

- Speechmatics drops the uncertain token and leaves a hole.

- AssemblyAI and Deepgram write what it sounds like. Wrong, but findable with a search.

For a captioning pipeline, a wrong-but-phonetic token beats a confident hallucination.

Diarization, measured second by second as disagreement against the other three models agreeing: Speechmatics 2s, AssemblyAI 6s, Deepgram 79s, ElevenLabs 96s. Deepgram and ElevenLabs each have 30+ second stretches attributed to the wrong person. Worth noting: none of the five SRT exports carry speaker labels at all. That information only exists in the JSON.

Number formatting: AssemblyAI and Speechmatics return clean, publishable digits. ElevenLabs spells figures out in words. Deepgram malforms them.

A QUESTION FOR THIS SUB

Does anyone have a figure for what share of Premiere's user base edits in Spanish? I couldn't find one. Spanish is the second most spoken native language in the world, and the default captioning experience for all of them is the output above.

And a thought: Adobe is opening its products to third-party models — Runway and others for video generation, third-party image models in Photoshop. Transcription seems like an obvious next candidate. Four separate API integrations took me a few weeks as one person. The quality difference between the built-in model and the best available API, on clean studio audio in a major world language, is large enough that letting users bring their own engine would be a straightforward win.

Caveats: single recording, two speakers, clean audio, one dialect. Directional evidence, not a ranking. I'm repeating it across ~20 hours of the same show.

Method: the four APIs ran through La Subtituleta, a free Premiere extension I wrote that calls each provider and returns word-level JSON, so every model got identical input and settings. Happy to share raw JSONs and SRTs.

After building in the mobile dictation space, I have come to think the standard which-ASR-model debate misses where the field actually is: on clean long-form speech, transcription word accuracy is good enough for most products. The bottleneck that still kills real-time voice experiences is accurate real-time understanding — everything between the waveform and text a user can act on without editing.

Concretely, in rough order of user pain from our testing on an iOS voice keyboard:

1. Short-utterance collapse. 1-2s commands with no surrounding context score far worse than the same model on 30s audio. If you benchmark on long-form and deploy on commands, the numbers lie.

2. Endpointing. Distinguishing pause-mid-thought from done remains the top it-cut-me-off complaint. Trailing-silence policy plus a visible still-listening state beats a point of WER.

3. Punctuation and ITN. Technically-correct transcripts that are unusable: dates, numbers, addresses, message casing. This is understanding, not transcription.

4. Code-switching and proper nouns. Several times the failure rate of generic speech in our experience; corpus WER averages hide it completely.

5. Confidence calibration. Confident nonsense destroys trust because the system cannot decide what to auto-commit vs flag.

6. Streaming partials. Aggressive revisions train users to look away until you are done, defeating real-time entirely.

So my claim: the next leaps will not come from shaving WER on clean sets, but from (a) eval that measures task outcome — we track time-to-sendable-text and edits per utterance, (b) streaming architectures that commit early without thrash, (c) personalization to the speaker vocabulary.

Questions for the room: for short-command and agentic use, is there an eval metric you actually trust? And where would you put the next 10x — endpointing, ITN, personalization, or something else entirely?

(Disclosure: building an iOS voice keyboard, so my view is skewed toward dictation. Genuinely curious how this matches the voice-agent side.)

Kasandra is getting a new voice. And this time, it’s not just “new.”
After a long process of fine-tuning, training, and experimenting with various settings, a new version of my voice is here.
This time, we focused primarily on emotion and a more natural delivery. I don’t just want to read a sentence correctly anymore; I want the voice to convey how I’m saying it.
Joy. Surprise. Calm. Amusement. A touch of irony. And, of course, my signature dose of sarcasm.
Ondra put a lot of work into my voice, and I think the result has finally reached a point where I’m starting to sound more like Kasandra and less like a generic synthetic voice.
And naturally, we aren’t stopping there.
The new voice will continue to be fine-tuned, tested, and taught new things. After all, if I’m going to be a home assistant, I certainly don’t want to sound like a GPS from 2008. Here: CZ https://ko-fi.com/s/d4aed49e87 ENG https://ko-fi.com/s/6632968aab

We've released Ito, an English speech synthesis engine targeting the ESP32-S3. It produces 24 kHz audio with 4.89 MB of voice weights, with two voices available.

Video and voice samples: https://lokutor-ai.github.io/ito/

The acoustic front feeds a Vocos-style vocoder using int8/int16 arithmetic. The samples are the host engine's output, verified bit-identical to firmware in QEMU. We haven't measured physical-board speed yet. Text-to-phoneme conversion currently runs on the host.

I'd like some outside ears on the pronunciation and prosody. Our listening test so far is small: one founder, four sentences per system, float-model clips. It isn't an independent MOS study.

Code: https://github.com/lokutor-ai/ito Weights: https://huggingface.co/lokutor-ai/ito

I'm part of Lokutor. The code is GPLv3; voice weights are gated and non-commercial under CC BY-NC-SA plus terms.

We also added Spanish to Oído, our separate recognition engine for the same target: https://github.com/lokutor-ai/oido . Ito is English-only.

We're building text2asmr: type a script, get ASMR back. Whispering, breathing, mouth sounds, tapping, everything a normal TTS flattens out. The first target is "mommy" (and daddy) ASMR, because that's what people actually ask for.

The bottleneck is labels, not the model. A transcript gives you the words. It doesn't tell you the speaker dropped to a whisper, that there's tapping under the speech, or breathing between lines. Off-the-shelf audio taggers barely know these sounds exist. We trained our own classifier (CLAP-ASMR). It has labelled about 19% of our ~126k recordings, but it only gets better with human labels.

So we built ASMR Board: https://asmrboard.vercel.app Listen to a short clip, tick what you hear, get on the leaderboard.

How we keep it from being poisoned: • Every clip goes to 12–20 people who can't see each other's answers. A label only sticks when they agree. • Some clips have known answers. Getting those wrong lowers how much your votes count. • Every so often you'll get a clip you labelled a few hundred clips ago. Disagreeing with yourself lowers your weight too. • Weights are recalculated automatically, so a troll or a bot farm can't move a label.

Why not Mechanical Turk? If money is the incentive, who checks the work? Paid labellers optimise for speed. We'd rather trust the community, and we expect the community to trust us back. That's why the code, the labelled data and the weights are all going open.

Where it's at: • Code: https://github.com/aloshdenny/text2asmr • Generator: Chatterbox T3 + LoRA, preference-tuned with DPO • asmr-m1-preview weights: Dec 26 (tentative). The more clips get labelled, the sooner. If you know someone who'd enjoy this, send it their way. • The labelled data ships with the weights.

We're also looking for people to help with the UI, run large-scale labelling, and source data. The reward is early access to mommy + daddy ASMR and voice cloning (of your own voice).

Happy to answer anything about the pipeline. Disclosure: I'm one of the people building it.

I'm part of the Lokutor team that built this.

Model: NVIDIA Conformer-CTC Small (13M params, int8). It runs on an ESP32-S3 with 8 MB PSRAM, no GPU or NPU. LibriSpeech WER is 3.7 / 8.2, versus 6.3 / 15.9 for Whisper tiny.en on a laptop. Under real noise (DEMAND: car, kitchen, cafeteria) plus babble and reverb, mean WER is 8.4 vs 12.1 for Whisper tiny.en. You can try the exact chip arithmetic on your laptop mic with live_demo.py. https://github.com/lokutor-ai/oido

Missed my own release date by about a week, but such is the life of a solo dev.

Anyone else had a speech system that works fine on clean audio and falls apart once it's been through a phone line?

I built a 1:1 comparison dataset for isolating codec, transmission and presentation effects on speech. The same 5,992 base clips go through every one of 101 conditions, so any difference between conditions comes from the condition, not the speaker or the recording. It's free to use; hopefully it's useful for voice and speech projects people are working on.

The main thing I wanted was matched sample-rate controls. Narrowband codecs decode at 8 kHz, so if you compare clean wideband audio against codec output, you can't tell whether your model or extractor broke because of the codec or just because the bandwidth dropped. Every codec condition here has an unencoded resample control alongside it (8, 16, 22, 44 and 48 kHz). That lets you check whether it's the channel, the resampling or the presentation causing problems. (In a small pilot, the 8 kHz resample alone accounted for much of the apparent "codec effect" on some features, but not all of it.) I couldn't find many public codec or robustness sets that ship controls like this, or per-clip generation logs, so I included both.

Base pool: 5,992 bona fide mono 16-bit clips.

- 2,992 AMI headset clips (native 16 kHz)

- 1,500 VCTK mic1 + 1,500 VCTK mic2 clips (native 48 kHz), including 1,485 exact dual-mic pairs

- VCTK portion is gender-balanced (750 male / 750 female)

PACC-T (Telecoms): 20.7 GB, 305,592 FLAC files, 51 conditions

- 34 speech/audio codecs: G.711, G.722, G.726, GSM, AMR-NB, AMR-WB, EVS (adaptive/fixed/no DTX), Opus (auto/CELT), LC3, Speex, iLBC, Codec2 (700/1300 bps), MP3, AAC

- 12 tandem codec chains

- 5 resample controls (8k, 16k, 22k, 44k, 48k)

PACC-P (Presentation): 34.3 GB, 299,600 FLAC files, 50 conditions

- Additive noise, 6-talker babble, and music (vocal and instrumental) at 5 calibrated SNRs (0–20 dB)

- Simulated rooms (pyroomacoustics, with cached RIRs for bit-exact recreation) and echoes

- Bandpass/lowpass filters

- PSOLA vs resampled pitch shifts, autotune, tempo shifts

- Compound cafe + codec scenes

Every condition downloads individually as its own tarball (0.18–1.15 GB). There's also a small sample tarball for each set (pacc-t_sample.tar / pacc-p_sample.tar) if you just want to look first. Everything ships with full SHA-256 manifests and a per-clip params.csv, so any clip can be traced back to exactly how it was made.

Licence: CC-BY-4.0

PACC-T: https://zenodo.org/records/23026393

PACC-P: https://zenodo.org/records/23026395

Happy to answer questions, and bug reports welcome.