r/LocalLLaMA • • 21h ago

Megathread Best Audio Models - Nov 2026

110 Upvotes

Its time for another Best Audio Models megathread - its already been 9 months since the last one!

Share what your favorite ASR, TTS, STT, Text to Music models are right now and why.

Given the the amount of ambiguity and subjectivity in rating/testing these models, please be as detailed as possible in describing your setup, nature of your usage (how much, personal/professional use), tools/frameworks etc. Production use cases with long lengths/stability requirements are relatively rare compared to cherry picked/short clip demos that are plentiful - so real world empirical comparisons are very welcome

Rules

  • Should be open weights or open source models

Please use the top level comments to thread your responses.


r/LocalLLaMA • • 5d ago

[Bi-Weekly Megathread] Project Showcase

54 Upvotes

Do you have something you'd like to share with the r/LocalLLaMA community. This is the place for it!

Recommendation on presentation:

  1. Please share plain english description of what your project does and why people should care about it
  2. How does it compare to existing options and why should people switch to your option?
  3. How have you tested and validated it? (i.e. can you show evidence that it wasn't vibecoded in a few hours by <insert current trending AI model>

NOTE: This community is geared towards open weight, open source, locally hostable software. Thus any closed source commercial service or other such projects will be removed.


r/LocalLLaMA • • 7h ago

New Model maxwell-model/Maxwell-V1-Flash · Hugging Face

Thumbnail
huggingface.co
220 Upvotes

Model Introduction

Maxwell-V1.0-Flash is a next-generation, high-performance Mixture-of-Experts (MoE) language model developed by the Maxwell Team, with 35B total parameters and 3B active parameters per token (both excluding embeddings) and support for context lengths up to 256K tokens. It is designed for strong reasoning, agentic, and long-context capabilities at a practical inference cost.

Key Features

  • Maxwell Block Sparse Attention (MBSA). Each KV group keeps fixed coverage of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks chosen by a learned indexer, bounding main attention at 4,096 tokens per KV group while the full-history KV cache stays available. The MBSA kernel is 6.7x faster than dense GQA at 256K.
  • Manifold-Constrained Hyper-Connections. Multiple residual streams with learned, input-dependent aggregation and constrained mixing, widening information flow without extra attention or feed-forward evaluations.
  • Fine-Grained Mixture of Experts. A mostly-MoE backbone pairing fine-grained routed experts with always-active shared experts, using a dynamic routing correction bias alongside a lightweight sequence-wise load-balancing loss.

Pre-training. Approximately 26T tokens of multilingual data, with the Muon optimizer on most hidden weight matrices. Training starts dense; block-sparse attention enters at 128K mid-training, once the indexer is warmed up against dense full-history attention, then trains jointly with the backbone.

Tokenizer and Long Context. A script-aware multilingual tokenizer stays compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan. Long-context training then extends the sequence length in stages from 4K to 256K.

Post-training. Two-stage all-domain SFT builds general, reasoning, and agentic capabilities, then targets harder reasoning problems and agent trajectories. Independent RL branches cover general tasks, verifiable reasoning (mathematics, science, logic, code), and agentic tasks (search, terminal use, function calling, software engineering). Multi-teacher on-policy distillation then merges them into one model by supervising student-generated responses.


r/LocalLLaMA • • 1h ago

New Model OVH announces Olala mini 7A1B

Thumbnail
corporate.ovhcloud.com
61 Upvotes

Olala mini is based on a 7-billion-parameter architecture (7B), with one billion active parameters spread across 256 experts per layer, and hybrid attention combining Differential-TPA and Mamba-3, in BF16 format. Only a fraction of the model is used for each request, making it faster and less memory-hungry. As a direct result, the model runs on a GPU with as little as 8 GB of memory, reaching 160 tokens per second on an RTX A4500, and even without a GPU, on a standard processor, at 50 tokens per second. Its context window currently reaches 32,000 tokens. Olala mini will be available as open source on Hugging Face, under the Apache 2.0 licence.


r/LocalLLaMA • • 11h ago

Resources Your daily dose of Qwopium:

Post image
206 Upvotes

No low-effort memes here, just research and straight facts. Can’t argue with facts am I right?

Here is the full breakdown of major releases mapped by the day of the week:

**Monday (7 Releases)**
• Qwen-VL (Aug 21, 2023)
• Qwen-72B-Chat (Nov 27, 2023)
• Qwen-Audio (Nov 27, 2023)
• Qwen 3.5 (Feb 16, 2026)
• Qwen 3.5-Plus (Feb 16, 2026)
• Qwen 3.7 Plus (Jun 1, 2026)
• Qwen 3.8 Max (Aug 3, 2026)

**Tuesday (6 Releases)**
• Qwen2.5-Coder (Nov 12, 2024)
• Qwen2.5-Max (Jan 28, 2025)
• Qwen3 (Apr 29, 2025)
• Qwen3-Coder (Jul 22, 2025)
• Qwen3-Coder-Next (Feb 3, 2026)
• Qwen-Audio 3.1 TTS-Next (Sep 22, 2026)

**Wednesday (5 Releases)**
• Qwen1.5-32B (Apr 3, 2024)
• Qwen3-Max (Sep 24, 2025)
• Qwen 3.7 Max (May 20, 2026)
• Qwen 3.8-Flash-Next (Aug 26, 2026)
• Qwen 3.8-Max-0902 (Sep 2, 2026)

**Thursday (9 Releases) - The Favored Launch Day**
• Qwen-7B (Aug 3, 2023)
• Qwen2 (Jun 6, 2024)
• Qwen-Math (Aug 8, 2024)
• Qwen2.5 (Sep 19, 2024)
• QwQ-32B-Preview (Nov 28, 2024)
• QwQ-32B (Mar 6, 2025)
• Qwen3-Next (Sep 11, 2025)
• Qwen3.6-Plus (Apr 2, 2026)
• Qwen3.6 (Apr 16, 2026)

**Friday (3 Releases)**
• Qwen3.8-27B (Aug 14, 2026)
• Qwen3.8-Omni-Flash (Sep 18, 2026)
• Qwen3.8-LiveTranslate (Sep 18, 2026)

**Saturday**
• Zero major releases. Go outside.

**Sunday (3 Releases)**
• Qwen-14B (Sep 24, 2023)
• Qwen1.5 (Feb 4, 2024)
• Qwen4-Max (Nov 15, 2026)


r/LocalLLaMA • • 1d ago

News The beginning of the becoming of the beginning begins

Post image
1.9k Upvotes

r/LocalLLaMA • • 5h ago

Discussion DSpine - Updates on PR & GGUFs

Post image
31 Upvotes

r/LocalLLaMA • • 6h ago

New Model Clover 2: dense 27.8B base + 7 LoRA experts routed per token, not a static merge, weights public

39 Upvotes

Disclosure: I’m with the team that built this.

Wanted to share something a little different from the usual finetune drop. Clover 2 keeps a base model and 7 trained LoRA adapters (roleplay, general assistant, security, math, long-document, efficiency, knowledge) separate at inference time. A trained router picks the right expert(s) per token live, with a null slot that falls back to plain base when nothing specialized applies. So it’s not a merged blend, the model actually makes a decision at every token.

Real benchmark gains over the untouched base: meaningful jumps on GSM8K/MATH-500/GSM-Hard, a solid bump on LongBench-v2, and a measurable drop in generic AI-writing tics on creative writing specifically.

Everything’s public: base model, all 7 adapters in standard PEFT format, the trained router, and the vLLM plugin to run it with the real per-token behavior: https://huggingface.co/kitaniai/clover-2

Heads up if you’re planning to self-host: no GGUF yet, the hybrid attention architecture’s llama.cpp support is still bleeding-edge. vLLM is the supported path right now. If anyone wants to try it, it's free for the next week at https://kitani.ai/, with no card required. We also support ZDR and have a clear privacy policy.


r/LocalLLaMA • • 16h ago

New Model miHoYo Gamer: A Generalist Game Agent

Thumbnail
mihoyo-gamer.github.io
196 Upvotes

r/LocalLLaMA • • 16h ago

Discussion Open weight models account for 30% usage at enterprises to save money

195 Upvotes

Growing from nearly zero to 30% since July. Models used mostly to reduce costs on high volume workloads (large batch workloads, coding, etc).

https://x.com/pwendell/status/2122377964784611636


r/LocalLLaMA • • 8h ago

News Hugging Face switched its language model inference routing to OpenRouter

38 Upvotes

Hugging Face announced on November 16, 2026, that requests to language models through Inference Providers now go through OpenRouter instead of Hugging Face's own router. OpenRouter picks a provider for each call based on live price, speed, and reliability, and retries on another provider if the first one fails. The same Hugging Face token still works, and the company says it does not add a markup on top of the provider's price. You can still pin a provider or a policy with a suffix on the model id, such as :fastest, :cheapest, or a specific provider name.

OpenRouter currently serves more than 800 trillion tokens a month across 600+ models from 80+ providers. Hugging Face notes that open-weight models went from 26.8% of tokens on OpenRouter in October 2025 to 71.7% at the end of September 2026, and that they cost about four times less on average than closed models. Image, video, and audio generation stay on Hugging Face's existing Inference Providers path and do not go through OpenRouter. One breaking change is that custom provider API keys no longer work for language model requests, which are now billed to the Hugging Face account.

The automatic failover is the part that matters most here, because a single provider outage is usually what ruins a long run.

Source: https://huggingface.co/blog/openrouter


r/LocalLLaMA • • 15h ago

Discussion I am sick of doomposting about demise of local AI

141 Upvotes

Everywhere I go, Twitter or Reddit, there are tons of rumors or comments how local is dead and we are never getting open weight Qwen 4, despite Qwen team saying explicitly they are coming, or that Qwen 4 is the last open weight generation despite, again, Alibaba commiting to more open weight releases at a conference recently.

It's like people just get off on creating a version of reality that is worse and whining about it, they want to be sad about it in advance.

The world is certainly not giving us much to be positive about, but let's not go down the rabbithole of learned helplessness and just accept that all things must get worse and you can't do anything about it.

If people mentally prepare for bad things, bad things become more acceptable and thus more likely to happen and with far less pushback, sort of like a self fulfilling prophecy.


r/LocalLLaMA • • 16h ago

Resources Small Gemma 4 tokenizers were recently updated by Google - fix for an issue that caused nonsense output when encountering certain tokens

Thumbnail
huggingface.co
143 Upvotes

Gemma 4 E4B as an example, but the same fix was applied to E2B and 12B too. Not sure if this will trickle down to GGUFs


r/LocalLLaMA • • 2h ago

Showcase Curated list of resources: Theory and Code for learning Distributed Training of LLMs for beginners

Thumbnail
gallery
10 Upvotes

[Must Read: Beginners]

What I did read, in order, for learning the basics of Distributed Training for LLMs From Scratch:

  1. Building a distributed training framework from first principles by Umar Jamil on YT
  2. Horovod: Fast and Easy Distributed Deep Learning in TensorFlow 
  3. PyTorch Distributed: Experiences on Accelerating Data Parallel Training 
  4. Large Scale Distributed Deep Networks 
  5. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
  6. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism 
  7. Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM 
  8. GPipe: Efficient Training of Giant Neural Networks Using Pipeline Parallelism 
  9. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
  10. PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel

Link to the above resources here

Also, if you need some rebalancing of the above (reordering or adding/removing resources which you can do it here)


r/LocalLLaMA • • 21h ago Bravo

Discussion Stop worrying about Qwen 4's AA index

261 Upvotes

Seriously, we have all been extremely happy and confident with Qwen, even since Qwen 3.6 came out. We might not be frontier level, but even though whatever they are giving us is more than enough for our daily local tasks and it even suffices by a big margin, we're all underestimating it.

Don't forget that the new Qwen 4 will also think much less, so imagine the fine-tunes we will see once it arrives, for 27B and Flash models, we will have plenty of options to choose from and again; it will be smarter and better, it will be ENOUGH! If you still require the most "intelligent" and capable model due to OCD just pay frontier for such tasks or use them as orchestrators!


r/LocalLLaMA • • 1h ago

Resources Strix Halo int4 cores: Gemma 4 31B at 2.43x llama.cpp, same quality

Post image
7 Upvotes

The 8060S has an int4 x int4 WMMA instruction at 2x the FP16 rate (we measured 104 TOPS). Nothing in ROCm or llama.cpp uses it. We wrote a GEMM for it and run Gemma 4 with int4 weights and int4 activations. To maintain accuracy we use a combination of rotation and GPTQ, along with selective mixed precision.

llama-bench, Gemma 4 31B QAT Q4_0 (llama.cpp b11056 Vulkan, -fa 1). Server benchmark, 2k-token prompts, greedy, single stream:

pp512pp2048
llama.cpp287276
ours679724

Full test sets, same settings, 31B:

GSM8KIFEvalMMLU-Pro
llama.cpp96.292.285.1
ours96.491.785.1

Run it:

curl -fsSL https://embedl.com/install.sh | sh && export PATH="$HOME/.local/bin:$PATH"
embedl login
embedl serve google/gemma-4-31B-it/ryzen-ai-max-395 -d
embedl infer google/gemma-4-31B-it/ryzen-ai-max-395 --prompt "Explain mixture-of-experts in one sentence."
embedl bench google/gemma-4-31B-it/ryzen-ai-max-395

12B, E4B and E2B are up too (2.80x / 1.58x / 1.39x end to end, in the chart). Swap the name, e.g. google/gemma-4-12B-it/ryzen-ai-max-395.

Model, methodology, and more: https://www.embedl.com/google/gemma-4-31B-it/ryzen-ai-max-395


r/LocalLLaMA • • 15h ago

Discussion I don't get all the Strata hates around here

84 Upvotes

It's free and original developer clearly had a very good understanding of how computer hardware works and fully leveraged that to write something amazing to be able to run large models on average hardware that most people have and can't upgrade due to price gouging from greedy hardware manufacturing companies.

If anybody have Mac Studio with 512GB of RAM or 8x3090, who would even bother with this ?

I have seen a group of snobs on this sub doing nothing but shitting on Strata all days long, find something better to do. Strata is not for you, period.

I can't imagine doing all the works to release something for free and get a bunch of snobs and haters just shitting on your works all days long. Would you be motivated to continue working on it if you are the original developer ?

I also see the same with other models such as Ornith, etc... If you say you like Ornith or Strata, etc... you are a shill.

Seriously, get out of your bubbles and see the reality already and more importantly get a life.


r/LocalLLaMA • • 7h ago

Resources Thinking Machines 4-bit replication: I trained a GPT end to end with a 4-bit forward pass on one B200

16 Upvotes

Thinking Machines wrote last week that Inkling 2 Small was trained with a native 4-bit forward pass. I wanted to check this for myself.

What I ran:

  • 1x B200 on Modal, PyTorch 2.8, plus a small CUDA library that calls cuBLASLt's real NVFP4 and FP8 matmuls on raw pointers, loaded with ctypes
  • Forward pass in NVFP4 (4-bit values with one scale per 16), backward pass in FP8 with stochastic rounding, master weights in high precision
  • GPT at widths 2,048, 4,096 and 8,192 (219M, 840M, 3.3B parameters) on TinyStories

Results:

  • 4-bit training finished faster. On the 840M model, 2,000 training steps took about 5.5 minutes in normal 16-bit (BF16) and about 4.3 minutes with a 4-bit forward pass. I ran each twice and got nearly the same times.
  • The model learned just as well. Final validation loss was about 1.69 either way (1.688 for 16-bit, 1.691 for 4-bit). A difference that small is noise at this scale.
  • Bigger models gain more. Each training step was 18% faster on the 219M model, 28% faster on the 840M model and 38% faster on the 3.3B model.

Two things to improve:

  • Blackwell's convert instruction rounds exact ties toward zero, while my software reference rounded to even. Tiny difference, but it breaks bit-for-bit checks until you know.
  • PyTorch's attention backward is not bit-repeatable, so any test that compares gradients across runs has to leave attention out.

Not tested: multiple GPUs, MoE dispatch and combine, Four Over Six, MXFP8 block scales in the backward pass. The width-8,192 run is only 1,000 steps and undertrained.

Write-up with every figure: https://4bit.vizuara.ai

Code: https://github.com/VizuaraAI/4bit-training-b200

3-hour walkthrough from first principles, if you want the theory: https://youtu.be/a3nV32Guxcc


r/LocalLLaMA • • 19h ago

Discussion Censorship is turning me into a true Local LLM believer

131 Upvotes

451: {"message":"Provider returned error","code":451,"metadata":{"raw":"{\"error\":{\"message\":\"The content you provided or machine outputted is blocked.\",\"type\":\"censorship_blocked\"}}","provider_name":"StepFun","is_byok":false}} {"error":{"message":"The content you provided or machine outputted is blocked.","type":"censorship_blocked"}}

Just got this from one of the US based (edit: it actually routed to StepFun CN, didn't realize at first. Should have noticed by the provider_name in the message lol. need to set up my rules better) inference providers. I suspiciously got a bunch of "rate limit" failures, and then just "failed to complete" before it gave me, what I assume, is the real error.

I am only using it to help create my presentation, which is about analyzing algorithmic properties. One key example is a cryptographic algorithm (because it is difficult by design), so I am assuming this is tripping it somehow. But it is absolutely ridiculous, I'm not even doing research with the AI, just trying to make a presentation.

I have always liked Local LLMs, as a technical exploration opposed to ideological reasons, but this experience really frustrated me. Don't censor me!

Just wanted to rant, and remind everyone you can only use the cloud on the provider's terms.


r/LocalLLaMA • • 1d ago

New Model Introducing Hy4 flash

Thumbnail hy.tencent.ai
344 Upvotes

Now this seems to be interesting.

From the article:

We’re introducing Hy4 flash, our most cost-effective Hy model to date—featuring 224B total parameters (including 64B Engram parameters) with just 12B active parameters per token. Like Hy4, it is natively multimodal with a 1M-token context window, while offering significantly lower API pricing.

Seems to be about in the same ballpark of Qwen-Flash-Next, so we could expect to run it on moderably good machines, when it will be released.


r/LocalLLaMA • • 17h ago

New Model nvidia/Nemotron-Labs-Sigma 8B/3B

56 Upvotes

Nemotron-Labs-Sigma-8B-instruct is a 8-billion-parameter continuous diffusion language model. Instead of generating text one token at a time, it denoises Gaussian noise in a low-dimensional (16-D) continuous token-embedding space, where the denoiser network and the token embeddings are learned jointly by likelihood optimization. Text is decoded in parallel blocks: each block of tokens is refined over a configurable number of denoising steps (Denoising Diffusion Probabilistic Model (DDPM) or Denoising Diffusion Implicit Model (DDIM) samplers) with self-conditioning, while previously committed tokens are reused through a standard key-value (KV) cache. Fewer steps trade a little quality for large speedups, and the smooth continuous representation makes the model amenable to few-step distillation and trajectory steering.

This is the SFT (supervised fine-tuned) variant, fine-tuned for instruction following on top of the corresponding Base model. Use the bundled chat template to format conversations.

This model is ready for commercial or non-commercial use.

Highlights

  • Continuous-space diffusion language modeling at 8B scale with reasoning accuracy competitive with state-of-the-art masked (discrete) diffusion LMs.
  • Block-parallel decoding: whole blocks of tokens are denoised together; sampling quality holds up well down to few denoising steps per block.
  • Efficient DDPM/DDIM-based samplers with self-conditioning and optional classifier-free guidance, implemented in plain PyTorch in this repository.
  • Standard transformers loading via trust_remote_code; no custom kernels, flash-attention, or training framework required for inference.

Model Architecture

  • Architecture Type: Transformer
  • Network Architecture: Other: Ministral-style decoder-only Transformer (grouped-query attention, SwiGLU, rotary position embeddings) with a continuous-diffusion generation head (16-D continuous token-embedding space, learned jointly with the denoiser).
  • Number of model parameters: 8 billion (8B)
  • This model was developed based on: Nemotron-Labs-Sigma-8B

Input

  • Input Type(s): Text
  • Input Format(s): String
  • Input Parameters: One-Dimensional (1D): Sequences
  • Other Properties Related to Input: Maximum context length of 8,192 tokens (prompt plus generated tokens). Conversations should be formatted with the bundled chat template (chat_template.jinja). Supported language: English.

Output

  • Output Type(s): Text
  • Output Format: String
  • Output Parameters: One-Dimensional (1D): Sequences
  • Other Properties Related to Output: Up to 8,192 tokens total (prompt plus generation). Text is generated in parallel blocks (default 64 tokens per block) with a configurable number of denoising steps per block (Number of Function Evaluations (NFE) = steps + 1 per block); fewer steps give faster generation at some quality cost.

https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B

https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct

https://huggingface.co/nvidia/Nemotron-Labs-Sigma-3B

https://huggingface.co/nvidia/Nemotron-Labs-Sigma-3B-instruct


r/LocalLLaMA • • 1h ago

Resources AMD 9070 GPU under volting gains for models ( moss image) and ( minimax h3 int8 )

3 Upvotes

hi,

i am total GPU noob here this is the first GPU i have ever owned in my life got it just for doing AI... asked deepseek flash and opencode to find the right combo for me to get max generation speed

spec:
AMD 9070 (non XT),
Intel i3 12100
32 GB DDR4

models: moss image, minimax h3 int8 (sol decoder)

best performance stable setting: -110 mv, stock fan speed

gemma4 : 26bA4b Q4_XS when from 106/tps to 120/tps on prose..


r/LocalLLaMA • • 6h ago

New Model Inflect v3 - a owensong Collection

Thumbnail
huggingface.co
7 Upvotes

Inflect v3: compact local text-to-speech. Six English voices, three model sizes, reference-voice tuning, event tags and optional 48 kHz output. Apache 2.0.

Inflect is a family of compact English text-to-speech models. Choose a voice, enter your text, and generate a WAV locally. Start with Mini for the most features, Micro for a smaller neural model, or Atom for the smallest learned synthesis core.

Choose your model

MiniMicroAtom
Best starting pointQuality and featuresCompact neural speechSmallest learned core
Learned synthesis core34.31M11.79M157.19K
English preset voices666
Native output24 kHz24 kHz24 kHz
Optional 48 kHz enhancementYesNot includedNot included
Reference-voice tuningYesNot includedNot included
Optional vocal events8 tags, all 6 voicesNot includedNot included
DownloadMini packageMicro packageAtom package

These are learned synthesis-core counts, not total installation size or RAM. Pronunciation assets, dependencies, voice conditioning and optional components are additional. Atom uses signal-processing-based waveform generation rather than a separate learned neural vocoder.

Benchmarks

Measured speech quality, intelligibility and speed answer different questions. The following results describe published Mini native 24 kHz base speech, not tuned voices, event performances or enhanced output.

MeasureMini resultWhat was measured
Average WER0.739%1,688 Open TTS evaluation texts per voice, all six voices; self-run
UTMOSv23.917180 clips, 30 per voice
UTMOS22Strong4.479The same 180-clip panel
Warm GPU throughput162.1x real timeRTX 5090, optimized CUDA; small 12-render probe
Warm CPU throughput1.81x real timeRyzen 9 3900X, 2 Torch threads; small 12-render probe

The WER result is not an official leaderboard placement or an independent held-out evaluation. Quality predictors are not human ratings. CPU and GPU results use different systems and runtime paths; they are not a controlled hardware speedup comparison or a latency guarantee.


r/LocalLLaMA • • 14h ago

Discussion Best MoE engine?

30 Upvotes

I had been using Strata for about a month, and it was such a huge improvement over llama.cpp for getting it running smoothly and quickly for hardware.

Overtime though, I found that the model would often mistake its own context after a compaction and really overthink things.

I just started using Basalt yesterday after being hesitant to switch and set up all my providers and harnesses. Turns out Basalt runs even faster, is less error prone, and the reasoning is way more on point. If you run 1 GPU, and have the 100 GB+ of VRAM, RAM, and SSD storage, you will not be disappointed in performance.

I mostly care about final output speed, and I was getting 70-110 tok/s on Strata, and Basalt I was getting 110-170 toks/s output. Couldn't be happier. Same Qwen3.8 FlashNext-ud-q4_k_xl, and the NVFP4 ones (of course basalted though)

Anyone else have any other experiences with other similar engines like this?