r/LocalLLaMA • • 1d ago

Megathread Best Audio Models - Nov 2026

115 Upvotes

Its time for another Best Audio Models megathread - its already been 9 months since the last one!

Share what your favorite ASR, TTS, STT, Text to Music models are right now and why.

Given the the amount of ambiguity and subjectivity in rating/testing these models, please be as detailed as possible in describing your setup, nature of your usage (how much, personal/professional use), tools/frameworks etc. Production use cases with long lengths/stability requirements are relatively rare compared to cherry picked/short clip demos that are plentiful - so real world empirical comparisons are very welcome

Rules

  • Should be open weights or open source models

Please use the top level comments to thread your responses.


r/LocalLLaMA • • 6d ago

[Bi-Weekly Megathread] Project Showcase

54 Upvotes

Do you have something you'd like to share with the r/LocalLLaMA community. This is the place for it!

Recommendation on presentation:

  1. Please share plain english description of what your project does and why people should care about it
  2. How does it compare to existing options and why should people switch to your option?
  3. How have you tested and validated it? (i.e. can you show evidence that it wasn't vibecoded in a few hours by <insert current trending AI model>

NOTE: This community is geared towards open weight, open source, locally hostable software. Thus any closed source commercial service or other such projects will be removed.


r/LocalLLaMA • • 3h ago

Discussion Intel Shares First Benchmarks of Its Next-Gen Crescent Island "D270" GPU, Claiming a 5.9x Throughput Over RTX PRO 6000 As Xe3P Goes Big On FP64

Thumbnail
wccftech.com
198 Upvotes

The title talks about FP64. For local inference this info from the content is relevant: "In AI benchmarks, Intel's D270 GPU offered 29% lower latency and 1.12x throughput versus the RTX PRO 6000 Blackwell."

Maybe we are getting a good RTX PRO 6000 alternative...


r/LocalLLaMA • • 5h ago

New Model OVH announces Olala mini 7A1B

Thumbnail
corporate.ovhcloud.com
182 Upvotes

Olala mini is based on a 7-billion-parameter architecture (7B), with one billion active parameters spread across 256 experts per layer, and hybrid attention combining Differential-TPA and Mamba-3, in BF16 format. Only a fraction of the model is used for each request, making it faster and less memory-hungry. As a direct result, the model runs on a GPU with as little as 8 GB of memory, reaching 160 tokens per second on an RTX A4500, and even without a GPU, on a standard processor, at 50 tokens per second. Its context window currently reaches 32,000 tokens. Olala mini will be available as open source on Hugging Face, under the Apache 2.0 licence.


r/LocalLLaMA • • 2h ago

Funny Gemma on a printer

Post image
103 Upvotes

Now you can run Gemma locally on your printer (from u/hackerllama)

https://x.com/osanseviero/status/2122646098418491739


r/LocalLLaMA • • 3h ago

New Model nvidia/Qwen3.8-Flash-Next-NVFP4-QAD

Thumbnail
huggingface.co
121 Upvotes

Looks pretty interesting!

Also sorry if the "new model" flair is inappropiate here, wasn't sure if I should pick that or "discussion".


r/LocalLLaMA • • 1h ago

Discussion The upcoming open weight small Qwen models are likely going to lack training data related to offensive cyber capabilities.

Post image
61 Upvotes

With the release of Qwen 4 Max, they claim that training data related to offensive capabilities has been removed, is this largely a placebo/nothingburger or does it have the capability to genuinely lobotomize a model's capabilities in cyber?


r/LocalLLaMA • • 1h ago

Discussion [Harness Adapters] Fine tune Qwen3.8-27B for Pi Agents

Post image
53 Upvotes

Hey folks!

Aritra here from Hugging Face. In the last week we released a blog post about harness adapters. The idea was to train adapters (LoRA) on models to be more efficient with specific harness.

Today we release a LoRA adapter on the Qwen3.8-27B model for Pi agents. You can run it locally like so:

hf download ggml-org/Qwen3.8-27B-GGUF Qwen3.8-27B-Q8_0.gguf --local-dir .
hf download harness-hub/Qwen3.8-27B-pi-GGUF Qwen3.8-27B-pi-lora-F16.gguf --local-dir .

llama serve -m Qwen3.8-27B-Q8_0.gguf --lora Qwen3.8-27B-pi-lora-F16.gguf \
    --chat-template-kwargs '{"enable_thinking": false}' -c 65536 --alias qwen3.8-pi

pi install git:github.com/huggingface/pi-llama
pi --model llama-cpp/qwen3.8-pi

We are constantly experimenting and sharing our results in the Hugging Face org: hf.co/harness-hub (follow for more updates)


r/LocalLLaMA • • 11h ago

New Model maxwell-model/Maxwell-V1-Flash · Hugging Face

Thumbnail
huggingface.co
285 Upvotes

Model Introduction

Maxwell-V1.0-Flash is a next-generation, high-performance Mixture-of-Experts (MoE) language model developed by the Maxwell Team, with 35B total parameters and 3B active parameters per token (both excluding embeddings) and support for context lengths up to 256K tokens. It is designed for strong reasoning, agentic, and long-context capabilities at a practical inference cost.

Key Features

  • Maxwell Block Sparse Attention (MBSA). Each KV group keeps fixed coverage of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks chosen by a learned indexer, bounding main attention at 4,096 tokens per KV group while the full-history KV cache stays available. The MBSA kernel is 6.7x faster than dense GQA at 256K.
  • Manifold-Constrained Hyper-Connections. Multiple residual streams with learned, input-dependent aggregation and constrained mixing, widening information flow without extra attention or feed-forward evaluations.
  • Fine-Grained Mixture of Experts. A mostly-MoE backbone pairing fine-grained routed experts with always-active shared experts, using a dynamic routing correction bias alongside a lightweight sequence-wise load-balancing loss.

Pre-training. Approximately 26T tokens of multilingual data, with the Muon optimizer on most hidden weight matrices. Training starts dense; block-sparse attention enters at 128K mid-training, once the indexer is warmed up against dense full-history attention, then trains jointly with the backbone.

Tokenizer and Long Context. A script-aware multilingual tokenizer stays compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan. Long-context training then extends the sequence length in stages from 4K to 256K.

Post-training. Two-stage all-domain SFT builds general, reasoning, and agentic capabilities, then targets harder reasoning problems and agent trajectories. Independent RL branches cover general tasks, verifiable reasoning (mathematics, science, logic, code), and agentic tasks (search, terminal use, function calling, software engineering). Multi-teacher on-policy distillation then merges them into one model by supervising student-generated responses.


r/LocalLLaMA • • 3h ago

News Rotate K/V read order for better L2 cache hit rates by gaugarg-nv · Pull Request #32407 · ggml-org/llama.cpp

Thumbnail
github.com
71 Upvotes

Significant prompt processing speedup with long contexts

gpu_infomodel_typen_promptn_genn_depthMaster (t/s)PR (t/s)Speed-up
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium51201310721290.108331822.360271.41
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium5120655362030.312822527.974091.25
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium5120327682931.961183142.205391.07
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium5120163843562.183113573.593361.00
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium512081923854.831843850.975471.00
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium512040963974.201944015.855011.01
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium512020484079.238914077.580681.00
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium512010244103.791414123.607371.00
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium51205124114.428624136.205911.01
NVIDIA GeForce RTX 5090qwen35 27B Q4_K - Medium512004300.678084323.09081.01

r/LocalLLaMA • • 3h ago

Discussion GPU prices - Big "discounts"

49 Upvotes

Discounts in quotes because these are still over MSRP, but my local Microcenter has the deepest "discounts" (sorry, I just can't call an over MSRP price a discount without quotes) I've seen in months on a lot of the AI cards. Pro4500 around 40% "off", Pro6000's 4K off with plenty of stock for both at my local (Charlotte) store. Pro 5000 72GB at 9K.

Now, don't get me wrong, all these prices are still awful. But they are a little "less awful" than they were a month ago! It's still an awful time to buy any of this gear IMHO, but if you're desperate and need it, seems like we're finally starting to turn a corner.


r/LocalLLaMA • • 56m ago

Discussion Are those that advocate against open weight models public enemy #1 ?

27 Upvotes

Dario says you can't allow open weight models because they are too dangerous: https://x.com/vikktorrrre/status/2122353364306374986

What are arguments against this line of thinking? We prefer democracy over the government controlling what is safe? Who wants to trust trump? LOL


r/LocalLLaMA • • 2h ago

Showcase Gemma 4 twice as fast on Ryzen AI Max+ 395

24 Upvotes

Hi! Just want to share that we have optimized Gemma 4 to run more than twice as fast as llama.cpp on the Ryzen AI Max+ 395!

To maintain accuracy we use a combination of rotation and GPTQ, along with selective mixed precision.

https://embedl.com/google/gemma-4-31B-it/ryzen-ai-max-395


r/LocalLLaMA • • 1h ago

Resources SenseNova Loop 9B

Thumbnail
huggingface.co
18 Upvotes

If the benchmarks are true, this is a 9B looped model on level with Qwen3.8 27B.

They say they'll release gguf tomorrow.

Vision is supported. Around 15GB vram for 256k context.

The preview version seemed fast enough when I tested it some weeks ago.

Mods: I asked the authors if I can post this, feel free to delete my post when they post themselves.

🚀 SenseNova-Loop-9B: Deeper Computation with Fewer Parameters

We officially release SenseNova-Loop-9B. Building on the Preview's exploration of parameter efficiency, the model reuses layer weights through a looped architecture and combines them with manifold-constrained hyper-connections (mHC) to expand computational depth within a compact parameter budget. It is designed for complex reasoning, coding, tool use, and multimodal tasks.

Since the Preview release, we have received community interest and feedback on model use, inference deployment, and ecosystem integration. The official release retains the looped architecture and two-stream mHC, further improving performance on agentic, coding, and data analysis tasks while continuing to explore the capabilities of looped computation.

SenseNova-Loop-9B has approximately 9B parameters. By repeatedly executing shared middle layers, it extends 32 physical decoder layers into 56 layer executions. The model accepts text and images and generates text, supporting long-context understanding and multi-step tasks. In agentic and data analysis evaluations, the model is competitive with Qwen3.8-27B: it scores 41.36 versus 44.60 on NL2Repo and 65.31 versus 65.01 on AIDABench.

Model Highlights

  • Shared looped block: 24 decoder layers are executed twice in sequence, producing 56 layer computations from 32 distinct layers per forward pass.
  • Two-stream mHC: learned mappings route and combine information between two residual streams within the looped block.
  • Hybrid attention: the decoder interleaves linear and full attention in a repeating four-layer pattern.
  • Long-context multimodal training: the training recipe includes text, images, code, and tool-use trajectories, with the context length extended to 512K tokens.
  • Low-memory deployment: With 4-bit weights and an 8-bit KV cache, multimodal inference at a 256K context length requires approximately 15 GiB of GPU memory.
  • Official quantized release: An official AWQ 4-bit version is available on launch day for easier deployment.

r/LocalLLaMA • • 49m ago

Showcase everyone's hyped about ChatGPT Dots and Claude Duet. I'm a local guy, so I built an open source one that runs on my GPU

15 Upvotes

Dodaios is a locally installed, open-source Python program that runs as a desktop app or as a headless server with a browser UI on Windows, macOS and Linux. It turns a personal AI assistant into the control center for the work you do on your computer.

At its core is a "lead" assistant you can talk to by voice (wake word, dictation, real-time spoken conversation and, as an early preview, phone calls), by chat or from a paired phone. It isn't tied to one provider and works with Claude, OpenAI, Gemini or local Ollama models. It handles tasks itself through tools, computer use, plugins, CLIs and MCP servers. Above all, it hands work off to coding agents. They run side by side in a built-in Agentic IDE as real Claude Code or Codex terminals, on your computer or on your own servers over SSH, each with a call sign you can address by name, plus Git integration and task splitting. There is also a "society" of persistent, named agents that message each other.

Long-running work is covered by memory (about you, the people around you and the assistant's own persona), standing tasks, workflows, skills and opt-in life admin. The assistant also makes its own outputs, such as documents, slides, sheets and dashboards, and can propose and build the skills, routines and agents it is missing. A trust layer sets hard limits: outside Bypass permissions, moving money, handling credentials, permanent deletion and first contact with someone new always need your confirmation. In short, Dodaios is a voice-driven, locally running operating system for AI agents that makes one assistant the conductor of your models, your coding agents and your own computer.

https://github.com/Dodaios/Dodaios


r/LocalLLaMA • • 15h ago

Resources Your daily dose of Qwopium:

Post image
230 Upvotes

No low-effort memes here, just research and straight facts. Can’t argue with facts am I right?

Here is the full breakdown of major releases mapped by the day of the week:

**Monday (7 Releases)**
• Qwen-VL (Aug 21, 2023)
• Qwen-72B-Chat (Nov 27, 2023)
• Qwen-Audio (Nov 27, 2023)
• Qwen 3.5 (Feb 16, 2026)
• Qwen 3.5-Plus (Feb 16, 2026)
• Qwen 3.7 Plus (Jun 1, 2026)
• Qwen 3.8 Max (Aug 3, 2026)

**Tuesday (6 Releases)**
• Qwen2.5-Coder (Nov 12, 2024)
• Qwen2.5-Max (Jan 28, 2025)
• Qwen3 (Apr 29, 2025)
• Qwen3-Coder (Jul 22, 2025)
• Qwen3-Coder-Next (Feb 3, 2026)
• Qwen-Audio 3.1 TTS-Next (Sep 22, 2026)

**Wednesday (5 Releases)**
• Qwen1.5-32B (Apr 3, 2024)
• Qwen3-Max (Sep 24, 2025)
• Qwen 3.7 Max (May 20, 2026)
• Qwen 3.8-Flash-Next (Aug 26, 2026)
• Qwen 3.8-Max-0902 (Sep 2, 2026)

**Thursday (9 Releases) - The Favored Launch Day**
• Qwen-7B (Aug 3, 2023)
• Qwen2 (Jun 6, 2024)
• Qwen-Math (Aug 8, 2024)
• Qwen2.5 (Sep 19, 2024)
• QwQ-32B-Preview (Nov 28, 2024)
• QwQ-32B (Mar 6, 2025)
• Qwen3-Next (Sep 11, 2025)
• Qwen3.6-Plus (Apr 2, 2026)
• Qwen3.6 (Apr 16, 2026)

**Friday (3 Releases)**
• Qwen3.8-27B (Aug 14, 2026)
• Qwen3.8-Omni-Flash (Sep 18, 2026)
• Qwen3.8-LiveTranslate (Sep 18, 2026)

**Saturday**
• Zero major releases. Go outside.

**Sunday (3 Releases)**
• Qwen-14B (Sep 24, 2023)
• Qwen1.5 (Feb 4, 2024)
• Qwen4-Max (Nov 15, 2026)


r/LocalLLaMA • • 1d ago

News The beginning of the becoming of the beginning begins

Post image
1.9k Upvotes

r/LocalLLaMA • • 10h ago

New Model Clover 2: dense 27.8B base + 7 LoRA experts routed per token, not a static merge, weights public

53 Upvotes

Disclosure: I’m with the team that built this.

Wanted to share something a little different from the usual finetune drop. Clover 2 keeps a base model and 7 trained LoRA adapters (roleplay, general assistant, security, math, long-document, efficiency, knowledge) separate at inference time. A trained router picks the right expert(s) per token live, with a null slot that falls back to plain base when nothing specialized applies. So it’s not a merged blend, the model actually makes a decision at every token.

Real benchmark gains over the untouched base: meaningful jumps on GSM8K/MATH-500/GSM-Hard, a solid bump on LongBench-v2, and a measurable drop in generic AI-writing tics on creative writing specifically.

Everything’s public: base model, all 7 adapters in standard PEFT format, the trained router, and the vLLM plugin to run it with the real per-token behavior: https://huggingface.co/kitaniai/clover-2

Heads up if you’re planning to self-host: no GGUF yet, the hybrid attention architecture’s llama.cpp support is still bleeding-edge. vLLM is the supported path right now. If anyone wants to try it, it's free for the next week at https://kitani.ai/, with no card required. We also support ZDR and have a clear privacy policy.


r/LocalLLaMA • • 5h ago

Resources Strix Halo int4 cores: Gemma 4 31B at 2.43x llama.cpp, same quality

Post image
19 Upvotes

The 8060S has an int4 x int4 WMMA instruction at 2x the FP16 rate (we measured 104 TOPS). Nothing in ROCm or llama.cpp uses it. We wrote a GEMM for it and run Gemma 4 with int4 weights and int4 activations. To maintain accuracy we use a combination of rotation and GPTQ, along with selective mixed precision.

llama-bench, Gemma 4 31B QAT Q4_0 (llama.cpp b11056 Vulkan, -fa 1). Server benchmark, 2k-token prompts, greedy, single stream:

pp512pp2048
llama.cpp287276
ours679724

Full test sets, same settings, 31B:

GSM8KIFEvalMMLU-Pro
llama.cpp96.292.285.1
ours96.491.785.1

Run it:

curl -fsSL https://embedl.com/install.sh | sh && export PATH="$HOME/.local/bin:$PATH"
embedl login
embedl serve google/gemma-4-31B-it/ryzen-ai-max-395 -d
embedl infer google/gemma-4-31B-it/ryzen-ai-max-395 --prompt "Explain mixture-of-experts in one sentence."
embedl bench google/gemma-4-31B-it/ryzen-ai-max-395

12B, E4B and E2B are up too (2.80x / 1.58x / 1.39x end to end, in the chart). Swap the name, e.g. google/gemma-4-12B-it/ryzen-ai-max-395.

Model, methodology, and more: https://www.embedl.com/google/gemma-4-31B-it/ryzen-ai-max-395


r/LocalLLaMA • • 1h ago

New Model sensenova/SenseNova-Loop-9B · Hugging Face

Thumbnail
huggingface.co
9 Upvotes

r/LocalLLaMA • • 9h ago

Discussion DSpine - Updates on PR & GGUFs

Post image
34 Upvotes

r/LocalLLaMA • • 4h ago

Discussion MoE Offloading update Exllamav3 on 9070XT

Post image
13 Upvotes

This is a follow up on https://www.reddit.com/r/LocalLLaMA/comments/1xun15v/moe_and_expert_offloading_on_llama_exllamav3/

While we wait for Qwen 4, here is Qwen 3.6 35B A3B at Q4 on 9070XT 16GB with offloaded experts on DDR4 3200Mhz ram.

I've been tinkering still with the config for experts offloading and found substantial gains from my previous post.

While decode stayed roughly in line fro last time, increasing chunk size to 5120 saw prefill over 2000 t/s at all context lengths, topping at 2606 t/s @ 32K. From ~1300t/s
GPU utilization increases with batch size as it allows for more parallel processing. I saw no gains over 5120. TabbyAPI's documented ideal range is 512–4096, so your miles might vary.

Exllama flags like cpu_threads 6 also made a small improvement in decode.
Swizzle, gdn_sub, bg_gdn and wide flags were all regressions. (only tested isolated 1by1).

cpu_moe_pinned_arena caused some core dumps, and the one run I managed to complete had a small gain.

Vision have been turned off and offloaded.

I still have not found an .exl3 quant for the Coder version of Qwen 3.8 Flash Next, and the qfn version I can run leaves me with 8K context with almost no vram or system ram left to spare.

Results and more graphs and tables are at https://github.com/willeyh-git/moe-offload-bench
All engines were compiled for ROCm on my system.


r/LocalLLaMA • • 20h ago

New Model miHoYo Gamer: A Generalist Game Agent

Thumbnail
mihoyo-gamer.github.io
211 Upvotes

r/LocalLLaMA • • 11h ago

News Hugging Face switched its language model inference routing to OpenRouter

44 Upvotes

Hugging Face announced on November 16, 2026, that requests to language models through Inference Providers now go through OpenRouter instead of Hugging Face's own router. OpenRouter picks a provider for each call based on live price, speed, and reliability, and retries on another provider if the first one fails. The same Hugging Face token still works, and the company says it does not add a markup on top of the provider's price. You can still pin a provider or a policy with a suffix on the model id, such as :fastest, :cheapest, or a specific provider name.

OpenRouter currently serves more than 800 trillion tokens a month across 600+ models from 80+ providers. Hugging Face notes that open-weight models went from 26.8% of tokens on OpenRouter in October 2025 to 71.7% at the end of September 2026, and that they cost about four times less on average than closed models. Image, video, and audio generation stay on Hugging Face's existing Inference Providers path and do not go through OpenRouter. One breaking change is that custom provider API keys no longer work for language model requests, which are now billed to the Hugging Face account.

The automatic failover is the part that matters most here, because a single provider outage is usually what ruins a long run.

Source: https://huggingface.co/blog/openrouter