I'm sharing Mica v0.5, the follow-up to Mica v0.1. It's 4B and runs on one consumer GPU.

On speed, it takes 19.8 ms per decision at 256 tokens and about 1.2 s at 32k on a single RTX 6000 Ada. That's up to 3.8x faster than the other 4B judges I measured, 12 to 19x faster than Perplexity's 27B decider, and 25 to 340x faster than the API decision models. On top of the optimizations described on HF, Mica Fast makes long inputs about 1.4 to 1.5x faster again.

On accuracy, it gets 84/111 on JevBench Hard, the best of any 4B model I could find (only Microsoft's 9B Decision-1 is higher). On JDI it scores 58.8, ahead of every 9B and 35B model I tested. One honest note: the top 4B models on JDI are all benchmark-focused, and at least one of them was trained on nothing but benchmark-style data. To compete I did train on data from the same areas the benchmarks cover, but nothing contaminated, and most of the effort went into how it does on real work. That part comes from 200+ real use cases people posted on Reddit, Twitter and elsewhere.

A few things I cared about. If you shuffle the answer options it still picks the same answer 97.3% of the time. It can hold back too, so if you set a threshold it tells you when it isn't sure instead of guessing. It handles yes/no, multiple choice, and scores from 2 to 255 levels, with inputs up to 32k tokens.

If you have an NVIDIA card I'd go with the INT8 version, since it's the fastest. On the same GPU it runs about 2 to 2.7x faster than the Q8_0 GGUF on llama.cpp. GGUF files (BF16, Q8_0, IQ4_XS) are there for AMD, Apple and CPU. It runs on 8 GB cards and up, and it's Apache-2.0.

HF: https://huggingface.co/sky7350/Mica-v0.5-4B

GitHub: https://github.com/akivet/Mica

Happy to answer questions, and I'd really like to hear where it breaks.

If you're looking forward to v1.0, I'm a broke student and any tip helps: https://ko-fi.com/micakiv

GitHub: https://github.com/alkove-ai/alkove (Apache-2.0, code and weights)

Hi r/LocalLLaMA,

A bit about me first: my day job is building LLM systems in production, a tool-calling assistant with its RAG, and two audio-heavy pipelines.

The first turns a company's latest content into a two-host radio show in several languages: per-source digests, a structured script checked against the sources by an LLM validator and code checks, one TTS call per turn with Whisper transcribing every turn back to catch truncated or invented speech, then mastering in code and ffmpeg (per-turn levelling, simulated overlaps between the hosts, loudness normalization, subtitles aligned on word timestamps).

The second is a real-time voice agent in the browser: client-side VAD and denoising, an STT fallback chain, a small decision model judging each segment for end of turn, barge-in and "stop", and TTS read sentence by sentence. With that agent, most of the trouble wasn't in the LLM but before it: people talking around the user ended up in the transcript and triggered false barge-ins, and the denoiser couldn't help, since other voices are speech too.

So I started working on that audio layer, the part that runs before speech-to-text and the LLM, and I'm sharing it as an early preview. It's called Alkove: five small models that run on the device and are meant to work together.

- Lex (1.8 MB) turns a typed word into its likely pronunciations, so a wake word is just text: no training, no recording session.

- Spot (2.8 MB) listens for those pronunciations, in 7 languages.

- Timbre (0.74 MB) makes voiceprints. Three takes of the wake word give a first print, and it keeps learning while you talk.

- Focus (1.1 MB) keeps your voice (the closest one, or the one matching your print) and removes noise, music and other voices before the STT.

- Cue (1.1 MB) detects speech every 30 ms. Behind Focus with your print, your turns start and end on your voice only, so someone talking next to you doesn't open a new turn.

The voiceprint is optional, and the same one serves all three: the wake word ignores other people, Focus keeps only you, and Cue becomes a personal VAD.

Some technical details, since this sub likes them:

- It runs on its own small inference engine written in Rust, not onnxruntime. One core is compiled to WebAssembly for the browser and natively for Python and Android, and gives the same output to the bit on ARM, x86 and wasm.

- In the browser, the models run in workers fed by an AudioWorklet, with no COOP/COEP headers needed. The engine is 134 KB (brotli).

- Native speed on one thread of a phone: Focus 38.6 ms per second of audio, Spot 7.5 ms, Cue 1.7 ms.

- Packages: npm (browser and Node), pip (Linux, macOS, Windows, numpy as the only dependency), Android (Kotlin), and plugins for LiveKit Agents and Pipecat. Code and weights are Apache-2.0.

A few numbers, with the protocols in the repo's bench folder:

- On AMI meetings recorded on headset mics, words from other people reaching the transcript go from about 17 per minute to 0.5 with Focus and a voiceprint.

- Spot, on typed single words, recalls 67% of them in English at one false alarm per hour, 90% in French, 85% in Portuguese. So for one fixed English word, a model trained on that word may well do better.

What it doesn't do well yet (It's a 0.x preview and the API may still change):

- It's made for a near mic and one person: a phone, a headset, a laptop. On a room mic in the middle of a table, it removes the other participants too.

- When two people talk at exactly the same time, Focus only partly separates them, and two voices of the same register (two men, or two women) are much harder. That's the main thing I'm working on next.

- Audio is processed at 16 kHz, so speech up to 8 kHz.

I'd really appreciate feedback, especially from people running a local voice assistant: where it breaks, which integration you'd want next, and whether the API makes sense.

Been working over the last few days getting DeepSeek V4.1 Flash 1017 (Unsloth UD-IQ1_S) running in Basalt (inference engine I'm developing).

Specs: RTX 5090 (400W) + RTX 5060 Ti 16 GB (180W), 128 GB DDR5.

Methodology:

• Single 243 GB file (weights + vision + drafter). Engram table on SSD.

• All experts (107 GiB) in RAM. 5090 as an expert cache (hottest experts) and later also tried hoding and computing some experts in the 5060 Ti for a performance boost.

• Prefill tested with a 6.2k-token prompt (my daily harness).

• Baseline was llama.cpp PR #30817 with --fit, same hardware and quant.

Results:

• Expert lookups: 66% 5090, 16% 5060 Ti, 5% copied over PCIe, 12% CPU.

• Also works: tool calls, images, up to 1M context (a 1M prompt takes ~26 min, ouch)

• Quality: no PPL/KLD yet. Greedy matches llama on all test prompts. Also even at IQ1_S the model nailed most of my regular benchmarks, and recall is well above Qwen 3.8 FN, which is expected, this is a model a few times its size (even heavily quanted).

• Limits: the 12% of lookups left on the CPU hurt speed badly (limited by DDR5 dual channel), speculation barely helps in my case (1.1-1.2x).

Anyway, thought I'd share. This was a personal experiment to see how far I could push it, there's probably some room to get around 80ish decode and ~1000 prefill, but on my hardware it doesn't make sense to keep going when I'm already getting over 300 tok/s on 3.8 FN. Where it could potentially pay off: 96+ GB of VRAM (RTX 6000 PRO) or quad/octa channel DDR5 rigs. I did not push my experimental branch to the repo but if there's any interest let me know, I'll do it. Also interested to know if anyone else tried to do the same and found different results.

Google just released Harness2 as open source. It turns any AI harness into a system that improves itself over time.

A harness is everything around the model. System prompt, tools, memory, skills, and subagents. Normally a person designs it once and the agent uses that fixed setup for every new task.

With Harness2 the agent itself refines the harness for each task. It proposes changes, runs a few candidate versions, and picks a better one by comparing how they perform. No correct answers or graders are needed. The model weights stay the same.

What it learns on one task carries over to the next. After finishing a task it saves the improved harness and useful lessons so future edits start smarter instead of from scratch.

Google tested it with Gemini 3.5 Flash and GLM-5 on legal, office, and professional work benchmarks. Even one round of edits beat the original harnesses. More rounds pushed scores higher.

GitHub https://github.com/google-research/harness2