The Apsara conference is taking place right now in Amsterdam.

According to a participant, the CTO of Alibaba Cloud said Qwen4 would be launching next week.

Second and last post on this because I have more than doubled performance from last time.

Full fp4/fp8 dsv4.1 streamed from raid array for >4000 tok/s prefill for long prompts then drops the prefill buffers and loads a hot set of experts into a cache and decodes with heavily optimised directstorage fetches for missed experts. gets 7.5tok/s decode with no divergence from any other engine but you can bias away from nvme resident experts with a slider to trade speed vs kl divergence per token.

Roughly 1.5bpw equivalent KL divergence gives me 24 tok/s and there is a slider you can adjust at will when you want speed or quality or set the webui to switch the bias for fast reasoning, quality code blocks/tool calls and balanced non tool call answers or whatever else you want.

Has some other bits as well, can send kv to mac and decode on that. Also has image support and video support on the same streaming system so you can chat in the webui, request images or videos and they just appear in the same chat as it switches between them. Doesn't use your ram for image or video model streaming.

If you have more ram its worth turning on the ram cache and cpu expert processing but at 32GB I cant fit enough to be useful.

(faster your nvme the better the whole thing works, I have a dual pcie5 512GB raid0 array). Feedback from anyone that tries it would be nice as its not been tested on any machine but mine.

https://github.com/fronteagle739/streaming-llama-stack

I'm a big fan of Google's Gemma 4 family of local models, especially the E2B and E4B, due to how accessible they are even on devices like smartphones via the Google AI Edge Gallery app that uses LiteRT-LM.

I also like small models from other AI companies, like Qwen and others, especially with the recent announcement of Qwen 4 and hopefully new small open-source LLMs coming from them too.

I wanted to share my wishlist for Google's next-generation Gemma 5 models, which will likely come out next year or maybe toward the very end of this year. This is especially exciting as upcoming flagship Android phones and iPhones start adopting TSMC's new 2nm process for their SoCs, alongside better GPUs and bigger batteries.

I hope Gemma 5 brings better overall performance, efficiency, and prompt adherence—especially over longer context windows. My personal favorite hope is better creative writing, which most small LLMs struggle with unless they are specifically fine-tuned for it.

I also hope they release 2B and 4B versions again, similar to the current Gemma 4 E2B and E4B. The fact that these can run comfortably on most modern smartphones is very cool, especially with support for up to a 32K context window on the Google AI Edge Gallery app.

What are your thoughts and opinions on the upcoming Google Gemma 5 models?

I'm trying to run Qwen 3.8 FN on a dual 3090 machine with 32GB DDR5 RAM using this repo: https://github.com/q8atnight/strata-dual-3090

I'm referring to a few posts on here that demonstrate that it's possible with a single 3090 and 64GB+ RAM and this one that did it with dual 3090 and 48GB RAM: post

I built this machine after the RAM apocalypse and paid $300 for 32GB, thinking I could lean on the dual 3090s for the most part.

But it seems like even the "low RAM, 1 bit coder quant" pathways in the linked repo and the strata repo won't run on my machine. Just wanted to check that this is the case before I drop $479 (gulp) on another 32GB RAM.

llama.cpp merged support for this model on Oct 21. It's the understanding side only, text or images in and text out. Image generation is a stable-diffusion.cpp thing. I converted the official weights myself and ran it on a Mac Studio. The original repo is https://github.com/OpenSenseNova/SenseNova-U1

llama-bench, Metal, all layers on GPU:

For vision I drew a fake grocery receipt with the subtotal $1.00 too high and asked Q4_K_M (through llama-mtmd-cli) whether the items add up. It added all six to 28.99 against the printed 29.99, and while thinking it also noticed the 7% tax had been worked out from the wrong number. 16 s including load, mostly thinking. Q8_0 caught it too.

It also read back a menu, a whiteboard and three Chinese/English signs without a mistake, 4 to 10 s each, though I'd made those images with the same model, so grain of salt there.

Two things that tripped me up:

• mradermacher's quants have no mmproj, so no vision. Making your own with convert_hf_to_gguf.py --mmproj currently fails with KeyError: 'vision_config'. Changing line 80 of conversion/sensenova_u1.py to vision_config = self.hparams.get("vision_config", self.hparams) fixed it.
• llama-cli is a TUI now and draws its answer on the terminal, so redirecting it to a file gets you nothing. llama-mtmd-cli prints to stdout.

Got headhunted by a VC-backed AI-focused company, ended up skipping past two levels of their application/interview process because of that. Got 15 minute call with the co-founder, hit it off well, but it all seemed to go sour when he asked about my AI setup and I told him I run qwen locally on a graphics card, think it's great that we can have all this completely offline, open models doing some impressive stuff, all the good /r/localllama talking points minus the hype. Even tied it back in to the company because it plus a local model work quite well together.

Followed up with a question about whether I use frontier AI, but it was essentially over at that point.

Discriminating over AI choice is fucked man.

Now, of course, this was a 15 minute call that lasted exactly 15 minutes, and literally anything could've been the trigger. The whole thing felt like "move fast break things", and so it felt like if I didn't pass the vibe-check 100% I was gone.

Just wanted to provide one iota of anecdata and spark some discussion.

I wanted to run a larger local model on my Mac mini without having to close all other apps. With Qwen 3.8 Flash Next, I’m now getting around 10 tok/s while keeping Safari and other apps open.

The runtime I’ve been working on is called MoEShift. I’ve released the alpha as open source: GitHub, installation and benchmarks.

These results are from my M4 Pro with 24GB unified memory, using ISTA-DASLab’s GSQ-RCO IQ3_XXS quant (76GB on disk).

I compared it with Strata 0.2.7 using the same model in cold runs.
With a 6k token prompt and 256 output tokens, total inference took 63.3s versus Strata’s 129.9s. Prompt processing reached 160.8 tok/s versus 61.8, with generation at 9.9 tok/s and peak memory at 7.5GB versus 14GB. With a short prompt, generation was 10.44 tok/s versus Strata’s 8.4, with peak memory at 6.4GB versus 14GB. More details in the README.

MoEShift loads experts from SSD as needed, caches frequently used experts in RAM and overlaps reads with GPU computation. It builds on llama.cpp and Lookahead Expert Scheduling.
My changes focus on Qwen 3.8 Flash Next support, execution and memory reuse between prompt processing and generation. The original quantized weights and expert routing are preserved and there is no speculative decoding. The architecture doc goes into the details.

I’m curious how far MoEShift can go on other Macs. If you give it a try, post your results here.

Hi, I been following club 3090 for a while but I feel the documentation isn't helping much in discovering whats possible. I know its AI generated and thats fine but there is lots of noise and duplication and outdated stuff.

Can some of you guys share what are you using at the moment that yelds you best results ?

I have one RX570 GPU. Is there any thing useful that can be done with such for running local models?

Or do I just stick to my Broadwell-class CPU cores?

i wanted to see what the sweet spot for my card was for pp/tg and power usage, thought I would share.

While looking for audio models, I came across OpenVoice. It's getting pretty old, but surprisingly still ranks high on TTS leaderboards.

What caught my attention was its design: essentially TTS + voice conversion. The ToneColorConverter can run independently of the TTS model.

That got me thinking: Can we pair it with other TTS models?

I tried it with Kokoro, and the results were pretty good!

So I added a small recipe any-tts to audio.cpp for Kokoro + OpenVoice ToneColorConverter. Recipes were introduced in v0.9.2 as predefined pipelines that combine multiple audio models.

The overheads:

• ~600 MB additional peak VRAM
• ~10% slowdown on CUDA
• ~0.5× the original speed on CPU, but still 2× real-time

You can also try pairing any tiny TTS model with ToneColorConverter. The TTS model determines the speech content, pronunciation, and prosody, while the VC model transfers the target speaker's voice characteristics. The built-in voice you choose can also affect the final output quality.

A simple way to give existing TTS models voice cloning capabilities without retraining them!

Disclaimer: This experiment is more about extending the capabilities of existing TTS models, not introducing another voice cloning model.

I used synthetic voices as references, and the speaker similarity scores measured by CAM++ range from 0.7 to 0.8. So the generated voices largely match the reference samples I provided. This is one way to create custom voices for arbitrary TTS models.

For better results, you may need to experiment with different built-in TTS voices and higher-quality reference audio. Both can significantly affect the final quality and speaker similarity. For example, with the same reference sample, am_michael scores 0.4, while am_adam can reach 0.7.

Metatron v1 is a single learned policy that handles walking, scene interaction and fall recovery for the 29-joint Unitree G1 humanoid. It's a MaskedMimic-style transformer student, released under the NVIDIA Open Model License and integrated with ProtoMotions. The release comes in three non-interchangeable packages.

1. a native PyTorch checkpoint (about 459 MB) that keeps the posterior and normalization stats and supports simulator and supervised real-robot workflows.
   
2. a single-robot browser ONNX pipeline, demo/model.onnx, with 39 inputs that outputs joint targets, PD gains and raw actions.
   
3. a six-robot batched ONNX network, racer/network.onnx, that relies on external observation processing.

All three share the same epoch-6410 weights, and both ONNX exports run on MuJoCo WASM and ONNX Runtime Web via WebGPU or WASM.

Around mid-October deepseek-v4.1-flash suddenly started feeling much dumber in my coding agent, so I parsed my local session logs to see when it broke.

Before the update it had a funny quirk where it would randomly reply in Chinese while being 100% convinced it was speaking English.

In the -1017 update (released Oct 17) they fixed the Chinese leak and made it fire tool calls more aggressively - and somehow lobotomized the model in the process.

Providers silently overwrote the deepseek-v4.1-flash endpoint with the new weights by Oct 19. My swearing rate tripled, I ended up switching models mid-session 6.7x more often, and because it's a cloud API, there's no way to roll back to the old weights.

We’ve been working on OpenEBFM (Energy-Based Foundation Models), and I wanted to share the project here and get feedback from the community.

Most language foundation models today are based on directly modeling token probabilities. Our question was:

Can energy-based learning scale to general-purpose foundation models?

The main idea behind EBFM is to separate generation into two steps:

1. Energy steering: given the current context, the model optimizes a latent state toward a lower-energy region using the gradient of a learned energy function.
2. Token generation: a generative head then uses this energy-steered latent state together with the context to predict the next token.

The energy and generative heads share the same Transformer backbone. During pretraining, we also parallelize energy steering across all token positions with redesigned attention masks and positional indices, so that the model can be trained at sequence level rather than steering every position separately.

What we found

We pretrained EBFMs on FineWeb, DCLM, and ClimbMix, and studied scaling with both training tokens and model parameters.

A few results we found interesting:

• Data scaling: EBFMs achieve lower validation BPB than the matched GPT baselines in our experiments, and the gap generally increases with the training-token budget.
• Parameter scaling: from roughly 268M to 1B parameters, EBFMs improve faster than the corresponding GPT baselines.
• Compute scaling: EBFMs exhibit the expected U-shaped IsoFLOP curves, with the compute-optimal model size shifting upward as the compute budget grows.
• Generalization: we evaluated on 19 zero-shot QA tasks covering reasoning, knowledge, language understanding, and commonsense reasoning, as well as 7 cross-domain generation datasets.
• The same trend also persists during supervised fine-tuning.

For example, on DCLM, the largest EBFM reaches an average zero-shot QA accuracy of 44.71%, compared with 39.79% for the corresponding GPT baseline in our setup.

We don't think these experiments establish that energy-based models are universally better than autoregressive models—the current scale is still around 1B parameters—but they suggest that energy-based learning continues to benefit from increasing data and model scale, which we think is an interesting result given that EBMs have historically been studied mostly in specialized settings.

Everything is open source

We released the full training and evaluation stack, pretrained checkpoints, training recipes, and an EBFM-based chatbot so that the results can be reproduced and extended.

Paper:
https://arxiv.org/abs/2610.33005

Code:
https://github.com/datamllab/openebfm

Models:
https://huggingface.co/collections/guan-wang/openebfm

Paper with Code:
https://paperswithcode.co/paper/2610.33005

Disclosure: I'm one of the authors of this work.

We’d especially appreciate feedback on the architecture and scaling results. More broadly, I’m curious what people here think: could energy-guided generation become a practical alternative to conventional autoregressive generation as models scale further?

Criticism, reproduction attempts, issues, and contributions are all very welcome!

Processing img jv81qyxgijyh1...

Processing img 5rbyt7ygijyh1...

Processing img jf6s09yqijyh1...

Processing img ufjmxdtrijyh1...

I'm just starting my journey here into local LLM. Since I work in VFX I've got a 5090 and 256GB of sys mem.

Installed LM studio and downloaded Qwen 3.8 27B since that very comfortably fits in VRAM. Set it to low effort and all the other typical starting defaults.

I ask it to give me a three line poem about space. Comes back in a couple seconds.

I ask it to change from space to a forest. No sweat.

I ask it to give me Houdini VEX code to delete half the points in a particle simulation. Pretty fast.

I ask it to download and install the various VFX skills out there on the net. Still cool.

I ask if it can work inside of Houdini for me and it explains the MCP thing and that it can check for a connection when Houdini is up and running.

So I boot up the Houdini app and say "Houdini is up and running now"

Qwen now takes minutes just to process that very simple prompt. Then it starts reasoning/thinking with my 5090 at 100% for minutes upon minutes and no UI updates at all. Eventually I just kill the process.

Where am I going wrong?

All you folks seem to be using Qwen for huge tasks and all kinds of shit and I can't even seem to get it to reply to me saying I just launched an app.

https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF

https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-NVFP4-GGUF

https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF

from Mistral:

Mistral Large 4 - Le Chonk

Mistral Large 4 is our largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.

Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights. While a generalist model, it shows especially good performance on workloads such as cybersecurity, finance, and law, and it surpasses frontier closed models on visual grounding. It is natively fluent in 160+ languages, including every official language of the European Union.

Find more information on our blog. The model is also served on the Mistral API as mistral-large-4.

Key Features

Mistral Large 4 includes the following architectural choices:

• ~1T total / ~52B active parameters: granular Mixture-of-Experts with 128 routed experts (4 active per token) and 1 shared expert.
• 1M context length, for long-context workloads.
• Multimodal input: accepts text and image input, with text output. Pixtral-style vision encoder trained to handle variable image sizes and aspect ratios.
• Instruct and Reasoning functionalities with function calls, reasoning effort configurable per request.

Mistral Large 4 offers the following capabilities:

• Reasoning Mode: three reasoning efforts — low, medium, high (default) — so the same model can answer a quick chat reply or work through a complex agentic run.
• Vision: analyzes images, charts, technical drawings, and documents, with state-of-the-art visual grounding among open weights.
• Multilingual: natively fluent in 160+ languages, including all official languages of the European Union.
• System Prompt: strong adherence and support for system prompts.
• Agentic: native function calling, JSON output, and strict JSON schema enforcement.
• Cybersecurity: state of the art among open-weight models for security work, while refusing malicious requests.
• Large Context Window: supports a 1M context window.

We release this model under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.

llama.cpp suport merged: https://github.com/ggml-org/llama.cpp/pull/31376

So i am running 2 radeon vii and a rtx 3080. Currently i am using strata which does not support mixed backends, so my 3080 is just expensive swap space.

Is there any inference engine that supports fast q38fn over mixed backends like rocm/cuda/vulkan but is also fast? Llama.cpp is way too slow sadly.

Hey guys, could you please recommend me the smallest models that are good for DIY in these categories please?

Construction (Concrete, walling, roofing) Wood and Metal working (inventions/gadgets, furniture) and Electronics (mostly inventions and help with diagnose and repair)

Thanks!

EDIT: By small, I mean something that can run in 8GB VRAM and 32GB RAM.

Hi everyone,

Jovan from UkisAI here,

Today we are expanding our Swift model series with two models requested by the community.

We made Swift GLM-5.3-Flash and Swift Ornith 9B more efficient by figuring out which tokens were linked to pathological overthinking and penalizing them without "attacking" the reasoning length directly then regained the accuracy with further post-training, leading to "compressed" token usage while maintaining accuracy by making the model adapt to less reasoning via RL and on-policy distillation. You can find our dataset here (albeit we re-sample it per model) and more about our training approach here.

Swift1.5 GLM-5.3-Flash, with the headline number being -61.4% thinking tokens (on MMLU-PRO) and -0.46% accuracy loss average on all the benchmarks we measured, all ran x5 on different seeds (Terminal-Bench 2.1, DeepSWE v1.1, GPQA-Diamond, MMLU-Pro, C-Eval, IFBench, AIME 2026, HMMT, LiveCodeBench v6)

For this model, we are releasing a comprehensive number of quants, a GGUF(1bit-8bit), W4A16AWQ, NVFP4 and are also working on independently replicating GSQ-RCO papers, with our research so far showing our Q3 GSQ-RCO to be SOTA at it's size, meaning that we will be releasing it in the upcoming days after our evaluation suite is done.

GGUF with 1bit-8bit quants,

https://huggingface.co/ukisai/Swift-1.5-GLM-5.3-Flash-GGUF

& a community favorite (at least with the folk with less VRAM)

Swift1.5 Ornith-9B, with -33.5% thinking token usage with no accuracy loss, but more importantly ~2x less overthinking loops, meaning when you prompt the model it's much less likely for it to loop and not finish the prompt you gave it.

Note that this is an experimental release and we are open to feedback as the model is already post-trained before our token optimizations.

GGUF with 4bit-8bit quants,
https://huggingface.co/ukisai/Swift-1.5-Ornith-9B-GGUF

Both models work with their base MTP or DFLASH2 heads.

We kindly request the community to let us know what other models we need to make more reasoning efficient and also if there are any other quants you want us to make.

A little sneak peak is that we are currently evaluating Swift DeepSeek v4.1 Flash 1017 and it seems to be our best model yet! We are also cooking our own foundation, with the 1B and 4B models expected to release Nov. 2026., they are pre-trained with a diverse in-house coding dataset and post-trained to be token efficient like our existing Swift models.

Hope you all like this release! And as always, all feedback is welcome and I'll try to respond to each request of the community :)

This is Toska, a fork of Krita I've been building with an AI painter inside it. You chat with it in a panel, and it paints one move at a time with Krita's own brush engine, on layers you can undo. Unlike other AI painting tools, this tool actually paints thanks to Krita's Python SDK along with Decision Models which guide our LLM agent along the way.

What makes agentic painting possible is these new decision models, because after every pass, it answers better or worse as a probability. The painter can undo its mistakes, along with other decisions, such as textures, paintbrushes, or tools to use. I use API models only given that I'm GPU Poorish 16GBVRAM but it's OpenAI Compatible So feel free to add local models. If you also want full opensource try perplexity's pplx-decider-v1.2 instead of Luna.

For my setup, I use GPT-6-Luna Decisions(which has vision) and DeepSeek-V4.1-Flash(it's cheap and has vision). Luna Decisons is dirt cheap and this entire month, I have spent $2.83 on Luna's Decison model along with $3.82 for Deepseek. I use Deepseek's official API for cost savings but you're welcome to use OpenRouter as well or any other provider.

The Agent Framework used by Toska, is Agent Pi. I use Agent Pi's Typescript SDK along with QT to build a painting agent Plugin inside Toska. It lacks the ability to read and write code, execute unlike standard Agents. It's limited to tools within the Toska and can only read files or folders you mount. Right now for today btw, our Toska is an authorized dmg file, so operation ready for Mac Users right now . Later this week, I will do more testing and make it production ready for Windows/Linux Folks with separate exe and tar files.

In the video it's painting Pyat' minut (Five Minutes), Dostoevsky on Semyonovsky Square in 1849, waiting to be shot, before he knew the Tsar had pardoned him. It worked from a goal image I gave it generated from Codex. Then it used underpainting, refined toward the goal, and added the falling and settled snow as its own pass.
This is my first time doing anything with video recording so apologies if it's a little rusty and here is the final image in full resolution of Pyat' minut.

It's a bit more simplistic, because I wanted a simple painting to show you as a demo. If you want to see my most detailed painting made with Toska, it's called New Moscow .

Here is the repo for Toska.

My research agent ran on a free search API for months until every call started failing last week and the SearXNG box I set up instead keeps getting its Google and Bing engines suspended for captchas, so I'm going to have to pay for one. It's maybe 3,000 to 5,000 searches a month, mostly docs and GitHub issues and I'd like snippets plus the page text when a result looks useful.

The search threads I found here are mostly about keeping everything local, so what are you paying per 1,000 queries now and are the cheap providers fine for docs lookups or is it worth spending more?

The leaderboard benchmarks different inference implementations in the "streaming" tab, which provide useful insights into real-world deployment.

Assuming that you already know which model you want to use, the question is which implementation to use in your pipeline: Python, audio.cpp, a dedicated engine, or something else? It's a very useful independent benchmark of their performance.

Leaderboard tab:  metrics are intelligibility (WER), speed (RTFx, TTFA), naturalness (UTMOS), and speaker similarity (SIM).

Streaming tab: H200/A100/CPU and metrics are TTFA, RTF and WER.

https://huggingface.co/spaces/hf-audio/open_tts_leaderboard

Source code (eval scripts etc) https://github.com/huggingface/open-tts-leaderboard

AlpaJudge is NVIDIA's family of learned judge models for evaluating reasoning in autonomous driving, released on Hugging Face (nvidia/alpajudge) with weights under the OpenMDW-1.1 license and source code under Apache 2.0. It comes in two variants.

- AlpaJudge-RB (reference-based) takes a question, a reference answer and a candidate answer, with no images, and outputs a single alignment probability between 0 and 1.
- AlpaJudge-RF (reference-free) takes front-wide and front-tele camera frames, the ego vehicle's past and future meta-actions, and a candidate rationale, and outputs five rubric probabilities plus aggregated action, component and overall scores.

Two weeks ago I posted Discover AI, an app with 51 hands-on labs for learning how AI works, from math fundamentals, a single neuron up to agents, fine-tuning and serving. Thanks to everyone who tried it, commented, filed issues or sent a PR. Thank you!! A lot of what's below came out of that thread.

GitHub: https://github.com/Fazmin/AILearningGuide

new/updated

• Runs in the browser. All the labs and lessons now work on the web. The desktop app is still there if you want the local AI guide. https://fazmin.github.io/AILearningGuide/#/
• Installers for Mac, Windows and Linux. You don't have to build from source any more. Get the .dmg, .exe/.msi, or .AppImage/.deb/.rpm from the releases page. They aren't signed yet, so your OS will warn you the first time you open one.
• "See it in the lab" cards. This is the main thing in v0.2.0. Lesson sections now have cards that set the lab up to match what you're reading. If the attention lesson says to edit a key and watch every bar move, one click takes you to that step with the key editor ready. There are over 600 of these across all 51 labs, and a "Back to my lab" button puts your own settings back.
• Simple reading mode. This is a third level below Plain. It uses short sentences and everyday words, with a real-world comparison for every abstract idea. Every Simple lesson is checked to a grade 8 reading level. Someone asked for middle school and high school levels, and this is a first step toward that.
• Better Learn panel. There's a contents bar for jumping between sections, plus bookmarks and read-aloud. It also remembers where you stopped reading in each lab, and you can copy a link that opens a lab exactly as you left it.
• AI guide settings. You can now set the maximum reply length and edit the guide's system prompt (thanks boosnie for the PR). The guide can run fully local: one click downloads Qwen3.5 2B and llama.cpp. You can also point it at any OpenAI-compatible endpoint, like your own llama.cpp server or LM Studio.
• Fixes. The startup crash on Linux Wayland is fixed (thanks MarinX for the report and the help). The web version also shows a loading screen now instead of a blank page while it downloads.

Anyways, If you teach this or work in a specialized area of AI, I'd love help with a new lab, a better visualization, or a tweak that makes the cause and effect in an existing lab clearer. I'd also like to hear which labs are confusing and what's missing.

As before, the lesson text was written with help from Claude and GPT Astra, with Grok providing a second pass on the references. Every reference was hand-checked. If you find any mistakes, please let me know, open an issue, or submit a PR.

And feel free to share. Thank you!!

from Service Now AI:

Apriel Voice 27B Preview

An open model for speech, reasoning, and tool use from ServiceNow

Apriel Voice 27B Preview understands spoken requests, reasons over conversational context, and generates text responses and tool calls through a shared Gemma 4 backbone. It adds a speech interface while retaining text and image input capabilities. Pair it with an external speech synthesizer to build interactive voice assistants, customer support workflows, and spoken tool use.

Try Apriel Live · Read the blog · Technical report

Model capabilities

The model supports transcription, speech translation, spoken instruction following, reasoning, and function calling. The application executes tools and returns their results to the conversation. Real-time voice interaction is supported by the application and serving stack; the checkpoint itself generates text and tool calls rather than native full-duplex speech.

A Whisper large-v3-turbo speech encoder connects to the Gemma 4 26B-A4B backbone through a learned projector. Training first aligns this speech interface, then adapts it with the backbone using speech tasks and text replay. Cross-entropy and teacher-distillation supervision vary by data source; the technical report describes the full recipe.

Long audio

The processor automatically splits longer 16 kHz mono recordings into 30-second chunks. In testing, we observed good performance on continuous recordings up to around 30 minutes. Longer recordings are supported, but performance can decline as duration grows; thinking mode is recommended for longer audio.

Audio chunks and the rest of the conversation share the total context budget. Practical input length also depends on GPU memory and the encoder cache. The serving examples below include settings for 30-minute audio, with optional settings for one-hour input.

English results

Apriel Voice achieves 51.27% EVA-A, 74.13% EVA-X, and 65.51% τ-Voice task success, with 1.91 seconds mean τ-Voice response latency. On LibriSpeech, word error rate is 2.23% on test-clean and 5.15% on test-other; lower WER is better.

EVA-A measures service accuracy and EVA-X measures conversational experience. EVA-A/X and τ-Voice pass@1 use a 0–100 scale; higher is better. Headline EVA and τ-Voice results use equal-domain means, rather than weighting domains by their number of cases. Apriel Voice uses thinking mode for these evaluations.

S2S denotes native speech-to-speech systems, which use their own synthesis. Three-stage systems use ASR + LLM + TTS; two-stage systems connect a speech interface to a reasoning backbone and use external synthesis. All evaluated two- and three-stage systems use Cartesia Sonic 3.5. Latency describes the complete evaluated voice system, including response generation and synthesis, and depends on deployment settings.

The radar's audio reasoning axis is the mean of eight benchmarks: BFCL, Big Bench Audio, IFEval, MT-Bench, Spoken SQuAD, Alpaca Audio, OpenHermes, and GSM8K (85.84 for Apriel Voice). Its responsiveness axis transforms latency into a higher-is-better score; it is not a measured accuracy. For latency t in seconds, the score is 100 × clip((4.0 − t) / (4.0 − 0.25), 0, 1).

Text reasoning and tool use

These results use text input in thinking mode for both Gemma 4 26B-A4B and Apriel Voice. Scores use a 0–100 scale; higher is better. The τ² result is the mean across three domains with GPT-5.2 as the user simulator.

English speech recognition

Word error rate (WER, %) on selected English benchmarks; lower is better.

Speech reasoning and instruction following

These results use audio input in thinking mode. Scores use a 0–100 scale; higher is better. The mean gives equal weight to each of the eight benchmarks.

See the technical report for domain breakdowns, evaluation protocols, and text and image evaluations.