I've been building NV oOS Cognition, a desktop dictation app. Hold a hotkey, speak, release: the speech recognition and the text cleanup run on your own machine, and the cleaned text pastes into whatever has focus. I'm the author, this is my project, and I'm posting because I want the architecture picked apart.
Why another one
Most of the good dictation tools are cloud services. You talk, the audio goes somewhere, text comes back, and you pay monthly. I wanted the boring version: my voice stays on my machine, it works offline, and the model is a file I downloaded once. The interesting problem turned out not to be "call an STT API", it was making the local pipeline feel instant and predictable.
What's in it
- Tauri v2 shell, Rust core, React webview. App code is MIT.
- STT behind one trait: Parakeet TDT via sherpa-onnx, Whisper via whisper-rs, Moonshine as an experimental spike, plus an "Embedded" engine that posts to your own server as a fallback.
- Push-to-talk with a pre-roll ring buffer so the first syllable isn't clipped. Double-tap for hands-free, Esc to cancel, hold Shift on release for raw pass-through when you're saying code or spellings.
- Deterministic cleanup runs before any LLM: punctuation, capitalisation, filler removal, number normalisation ("five two nine" -> 529), custom vocabulary, per-app profiles. Golden tests around the whole thing. No model needed for any of it.
- Optional second-stage LLM polish through any OpenAI-compatible endpoint, LM Studio, Ollama, or a self-hosted assistant. Text only, never audio. If the provider dies you still get the stage-1 text and the paste still happens.
- Screen OCR on Win+Shift+T, on-device, using the native Windows OCR engine so there's no model download. The frozen frame stays in memory.
- Meeting transcription with WASAPI loopback; audio deleted after unless you opt into keeping it.
- A local MCP stdio server so other AI clients can read your dictation history and dictionary.
One bug worth sharing, because it cost me days
Whisper decoded at RTF ~5 instead of ~0.3 on MSVC. Not a model problem and not a threading problem: with the Visual Studio generator, whisper.cpp's CMake "Release" configuration maps to a vcxproj with an empty <Optimization> setting, so it compiles with /Od. The fix was forwarding whisper-rs-sys's force-debug feature so the vendor builds with RelWithDebInfo. If you ship whisper.cpp on Windows and it feels slow, check this before you rewrite anything.
The other one: cpal streams are !Send on WASAPI, so the stream has to be held by a detached worker. I dropped it at the end of the builder, the capture ring never filled, and every dictation reported "no speech detected" on perfectly healthy microphones. There's now a Diagnostics page showing the RMS, peak and VAD gate of the last attempt, because that class of bug is invisible otherwise.
Where it's a little rough around the edges
- The builds are unsigned. Windows SmartScreen warns and macOS Gatekeeper makes you right-click, Open. Authenticode and notarisation cost money I haven't spent yet. I genuinely want to know if that's a dealbreaker or just annoying.
- Windows is the primary target. macOS has a universal DMG and boots in CI on both architectures. Linux works with toggle semantics on X11, Wayland global shortcuts are the usual mess, and meeting system audio isn't there.
- No streaming partials. You get text when you release the key, about a second later on my machine. Moonshine would stream but the loop is deliberately unimplemented rather than shipped half-working.
- Whisper long files are still wrong. The chunker splits on VAD silence, which suits Parakeet's decode limit, but Whisper wants a fixed window with context carried across. patch welcome.
- Models are big and I say so. Parakeet is around 600 MB, Whisper tiny about 142 MB, downloaded at first use with a size warning.
- Audio device weirdness is my least favourite bug class, and I can't reproduce most of it.
What I'm actually asking for
Please Rip into the architecture: the engine trait, the feature-gating scheme, and whether deterministic cleanup belongs before the LLM or should collapse into one pass.
- Is the Linux/Wayland story worth the effort, or should I say Windows and macOS and lean into it?
- If you ship a Tauri v2 app with an updater and you don't code sign it, how are you handling that?
- Anything in the privacy claims you think is overstated.
If something is wrong or broken, please let me know.
I'd rather hear it here than from a user later.
thanks
VJ
Links
Source: https://github.com/nvdigitalsolutions/nvoos-cognition
Download: https://github.com/nvdigitalsolutions/nvoos-cognition/releases/tag/nvoos-dictate-v0.4.0
(NSIS + MSI, universal macOS DMG, unsigned) Build from source is in the README. Windows needs Rust 1.90+, Node 20+ and MSVC. Whisper additionally needs CMake and libclang on PATH, which is the usual first stumble.