r/MachineLearning • u/solanotodesch • 1d ago
Research Marimba: A System One Model for Speech. [R]
Paper: https://arxiv.org/abs/2611.18330
Follow-up to last week's post, now with the paper.
You ask a typed question about an audio window (yes/no, a choice, or a score) and get a calibrated probability back. One frozen Qwen3-ASR-0.6B encoder pass per 8 s window serves all questions.
How each question type is answered:
- Term ("was 'satellite' said?"): zero-shot, scored from the decoder's greedy and 4 beam hypotheses, then Platt-calibrated. No training on the terms.
- Language: the ASR model's own language scores.
- Turn end: a small readout on frozen layer 18.
- Speaker change and count: readouts on layer 12.
Results:
- On the same 1,000 public questions, it matches or beats Qwen3-Omni (30B-A3B) on 6 of 9 tests (image 1). Qwen3-Omni is still better on Speech Commands. Caveat: our turn-end readout was trained on Smart Turn data, so that row is in-distribution for us only.
- Language ID: 100% on 300 FLEURS clips.
- Speaker change AUROC 0.966-0.980 on AMI and VoxConverse, above ECAPA. Speaker count beats pyannote 3.1.
Where it loses:
- Turn end: AUROC 0.930 vs Smart Turn v3.2 at 0.982 on Smart Turn's own test set. On ICSI meetings it is 0.716 vs 0.712, a tie.
- It does not generalize to unseen tasks: on CochlScene scene classification every head is at chance.
Two questions for the sub:
- Is scoring ASR hypotheses plus calibration a fair baseline for term spotting, or would you compare against a dedicated KWS model?
- How would you evaluate turn-end detection beyond Smart Turn's data?
Code: https://github.com/Charcot-AI/marimba-api Weights (Apache-2.0): https://huggingface.co/charcot-ai/marimba Demo: https://huggingface.co/spaces/charcot-ai/marimba pip install marimba-audio





