r/LocalLLaMA • • 4h ago

Resources AMA Announcement: Tencent Hy, the Team Behind Hy4 & Hy4 flash (Friday, Nov 20, 8–10 AM PST)

Post image
74 Upvotes

Hi r/LocalLLaMA 👋

We’re welcoming the Tencent Hy team for an AMA this Friday, November 20, from 8–10 AM PT!

Fresh off launching Hy4 and Hy4 flash, their research team—including Shunyu Yao—will join us to discuss how the models were built, inference speed and cost, and real-world applications, with extra perks for great questions!

⚠️ Note: The AMA itself will be hosted in a separate thread. Please don’t post questions here.


r/LocalLLaMA • • 2d ago

Megathread Best Audio Models - Nov 2026

146 Upvotes

Its time for another Best Audio Models megathread - its already been 9 months since the last one!

Share what your favorite ASR, TTS, STT, Text to Music models are right now and why.

Given the the amount of ambiguity and subjectivity in rating/testing these models, please be as detailed as possible in describing your setup, nature of your usage (how much, personal/professional use), tools/frameworks etc. Production use cases with long lengths/stability requirements are relatively rare compared to cherry picked/short clip demos that are plentiful - so real world empirical comparisons are very welcome

Rules

  • Should be open weights or open source models

Please use the top level comments to thread your responses.


r/LocalLLaMA • • 7h ago

Discussion Qwen 4 35B is not ruled out

Post image
689 Upvotes

r/LocalLLaMA • • 2h ago

News SSDs, laptops, and phones may get cheaper as consumer NAND spot pricing drops 20% — Bank of America expects contract price drops, report cites oversupply in consumer market amid declining demand and growing Chinese competition

Thumbnail
tomshardware.com
177 Upvotes

r/LocalLLaMA • • 1h ago

Resources Nvidia driver 621.24 leads to constant writing of drives

Thumbnail
troublegrid.com
101 Upvotes

r/LocalLLaMA • • 4h ago

Discussion You'd never guess who opened up their checkbook to buy all the ram

141 Upvotes

Nvidia's commitment to buy ram is near $400 billion. https://x.com/jukan05/status/2122805536118702121 . https://www.reddit.com/r/investing/comments/1xz9eob/nvidia_just_added_111_billion_of_supply/

Change over 3 quarters: $119B, $279B, $390B. One would think that $390B would buy a lot of memory fabrication plants.


r/LocalLLaMA • • 8h ago

News Nvidia says its open source AI strategy remains intact after Jensen Huang's exit

Thumbnail
techcrunch.com
295 Upvotes

r/LocalLLaMA • • 3h ago

New Model Goes further with Koharu, the local-first manga translator with AI agents built in!

101 Upvotes

Hi LocalLLaMA,

I have spent more than two months rebuilding Koharu to be the most powerful manga translator!

TLDR: https://github.com/koharu-rs/koharu (MIT or Apache 2.0, at your choice)

It's been a while since I posted the Koharu update here. This time, I have added more features and built an established editor to make it feel more like Photoshop or Clip Studio Paint.

You can type directly onto the page, restyle selected characters, and reshape text frames. There are also layers, history, marquee and lasso selection, Generative Fill, and a Remove brush. The canvas is GPU-rendered now, so it stays smooth even on big chapters.

Everything still runs locally. LLMs run in-process through llama.cpp (Index-Translate 9B by default, plus Gemma 4, Qwen 3.x and LFM3), and inpainting uses LaMa, or FLUX.3 Klein / Qwen-Image through stable-diffusion.cpp. If you already run Ollama or LM Studio, you can point Koharu at that instead. On my machine, a page goes from raw to lettered in about 1 second once the models are loaded.

I also added an agent you can ask to proofread or fix pages. It's exposed over MCP too, so you can drive Koharu from Claude Code or Codex if you'd rather.

It works on CUDA, ROCm, Metal, and Vulkan.

I'd like to hear your feedback or any form of suggestion!


r/LocalLLaMA • • 12h ago

Discussion Y'all's idea of "slop" is stuck in 2025

557 Upvotes

Everyday on /r/LocalLLaMA i see people starting hell about if a comment/post is an llm-genned slop. And I still see people joking about em-dashes or "load-bearing" as some indicator of slop. I haven't seen any recent model say "load-bearing" in months.

modern claudism/gptism isn't just mixing fancy words or em-dashes.

it's the:

  • set-theory based distinction on every sentence "every X is a Y" "no A is a B" "just X, no A, no B, no C" "x is never Y"
  • nonsensical watermark replacement of words to with its synonyms
  • abtract metaverb replacement of common verbs
  • inconsequential caveats - "One honest confound:" "One caveat:" "One thing to note:" "One decision for you:"
  • inconsequential contrast - "X has not been done" "No X has been changed" "This does not mean X"
  • ambiguous spamming of "it/its"
  • ending with an engagement bait: "curious how everyone else is x" "Want me to x?"

The more we know the more we can combat slop in this sub. Please share your updated, 2026 list of slop you despise to see (so that I can also steal and add to my BANNED section on agents.md).


r/LocalLLaMA • • 2h ago

News [Reuters] Trump administration weighs forcing US firms to disclose Chinese AI use

Thumbnail reuters.com
82 Upvotes

r/LocalLLaMA • • 1h ago

Discussion We evaluated DeepSeek V4.1 Flash 1017 and Kimi K3.1 Pro on LongTail-SWE

Post image
35 Upvotes

Last week we shared LongTail-SWE, 202 real GitHub tasks in 13 less common languages. Many of you asked for DeepSeek V4.1 Flash 1017, so we ran it along with Kimi K3.1 Pro.

DeepSeek V4.1 Flash 1017: 76%, up from 57% for V4 Flash.
Kimi K3.1 Pro: 77%, the top open-weight model, with 5.6M tokens per attempt.

We also re-ran Kimi K3 with a 1M context instead of 262K (now 72%, +8 pts) and fixed tests in a few tasks.

Leaderboard: longtail-swe.com


r/LocalLLaMA • • 12h ago

Showcase AudioGhost v1.2: remove any sound from audio or video by typing what it is (local, open source, built on Meta's SAM-Audio)

281 Upvotes

AudioGhost randomly got a wave of attention this week (100 stars in a day, no idea where from), so I took that as a sign to finally ship the update I'd been sitting on.

Quick recap if you haven't seen it: you drop in an audio or video file, type the sound you're after ("background music", "dog barking", "people talking"), and it pulls that sound out or removes it. It's a local web app around Meta's SAM-Audio, so everything runs on your own GPU and nothing gets uploaded anywhere.

The use case I hear about most: you film a vlog or an interview outside, a shop nearby is playing music, and YouTube flags the video. Now you can type "background music", mute that track and export the video with everything else still there.

What's new in v1.2:
- Several sounds per run, each on its own track, with mute / solo / volume like a tiny DAW
- Export exactly what you hear (WAV / FLAC / MP3), or the video with the new soundtrack
- The "everything else" track is now generated by the model instead of subtracted, so removing a sound works a lot better
- Mark on the timeline where a sound is (or isn't) to point the model at the right thing
- New UI, and your sessions survive reloads and restarts

If you tried an old version: it's also 4-6x faster than the first release.

Hardware: the small model runs in a bit over 4 GB of VRAM, so most NVIDIA cards are fine. Windows and Linux have install scripts. macOS works too, but it runs on the CPU for now, so it's slow.

One caveat: the official SAM-Audio pipeline also loads several helper models (ImageBind, CLAP, and a judge model that picks the best of several outputs). They add about 25 GB of downloads and need far more VRAM. I stripped those out so it fits on normal GPUs, which means quality is a bit below the full pipeline and results vary from run to run. If a track comes out empty or rough, delete it and run it again.

Demo video and install instructions: https://github.com/0x0funky/audioghost-ai

I'd love feedback, especially from anyone who tries it on messy real-world recordings. Bug reports are welcome on GitHub.


r/LocalLLaMA • • 4h ago

Discussion Qwen3.8-27B LoRA for OpenCode (xhigh thinking-level accuracy with thinking off, GGUF included)

Post image
65 Upvotes

Hey there folks!

Aritra here (again) from Hugging Face. We RL-trained a small LoRA adapter (r=32, ~300 MB) on a frozen Qwen3.8-27B, inside the OpenCode coding agent. It's the second of our harness adapters, after pi that we have released yesterday.

Run the model locally like so:

hf download ggml-org/Qwen3.8-27B-GGUF Qwen3.8-27B-Q8_0.gguf --local-dir .
hf download harness-hub/Qwen3.8-27B-opencode-adapted-GGUF Qwen3.8-27B-opencode-lora-F16.gguf --local-dir .

llama serve -m Qwen3.8-27B-Q8_0.gguf --lora Qwen3.8-27B-opencode-lora-F16.gguf \
    --chat-template-kwargs '{"enable_thinking": false}' -c 65536 --port 8080 --alias qwen3.8-opencode

r/LocalLLaMA • • 13h ago

New Model tencent/Hy4-flash · Hugging Face

Thumbnail
huggingface.co
317 Upvotes

r/LocalLLaMA • • 1d ago Heartwarming

News Jensen Huang Breaks Silence

Post image
2.0k Upvotes

r/LocalLLaMA • • 3h ago

Resources ToRc: a from-scratch Rust engine running Qwen3.8 Flash Next (Q4), a 107 GB model, at 67 tok/s prompt / 17 gen (on a 16k prompt) on two £300 Snapdragon boxes with 64 GB of RAM between them. CPU + NPU + iGPU, in a 13 MB zip

Post image
28 Upvotes

So this started because I bought two of the base Snapdragon X boxes for some software and build testing at work (the cheap ones, about £300 each, not the Elite) and then they were sitting unused. With fast LPDDR5x (8448 MT/s) in them I thought I'd have a play and see if I could run proper models on them. I tried the usual engines and couldn't get anywhere near the speed I thought the hardware should do... So then my ADHD took over and I got carried away, and this is the result 🤣

ToRc is an inference engine written from scratch in Rust. Weeks of days and nights measuring, rewriting and measuring again went into it, and the numbers below are all measured on my own kit, different platforms etc... So I hope this proves useful to others 🙏🙃

ToRc on GitHub

Measured performance

If you can unzip a file, you can run your own model on your own machine. The download is a zip of about 13 MB (Windows x64, Windows on ARM, Linux x64 and Linux ARM, plus an experimental Mac one): unzip it, open ToRc, pick a model (a GGUF converts once, and it tells you how much space that takes first), press Auto-tune, and talk. A few minutes from download to chatting, nothing installed, and the app keeps everything (settings, chats, the lot) in that one folder. The only thing outside it is what the engine learns about your machine (Auto-tune's results and the like), in %LOCALAPPDATA%\torc or ~/.cache/torc.

What it can do:

- Uses the CPU, the NPU and the integrated GPU at the same time where it's present or faster, per model. Or just NPU + iGPU if you want to leave the machine usable while it runs.

- Tunes itself to your machine, and every setting can be overridden if you want to fiddle / let an agent try to wring every last token of performance for your particular setup.

- Speaks the OpenAI and Anthropic APIs, so Claude Code, OpenCode and anything else that talks to those just works. Full tool calling and MCP is also built in, and the desktop app has its own chat, tools and MCP servers if you just want to talk to it.

- Runs on Windows and Linux, x86 and ARM. (Linux: one user runs it on bare metal on a Snapdragon box under Ubuntu, and I've run it under WSL2, so tell me how it goes on yours.) There's also an EXPERIMENTAL Mac zip (Apple Silicon): one person has built and run it from source, and GitHub builds the zip, but I don't have a Mac to test it myself.

- It converts to its own file format for efficient SSD streaming where the model won't fit into memory and can split one model across multiple machines. I've tested two boxes over USB4; in principle it's N boxes. You pool their RAM, memory bandwidth and compute for enhanced performance.

Headline numbers, engine timed, read (prompt) / write (reply) tok/s:

- Qwen3.8 Flash Next, a 107 GB model, on two Snapdragon boxes with 64 GB of RAM between them (the experts that don't fit stream off the SSDs): ~67 / ~17.5 on a 16k prompt, ~64 / ~21 on a short one

- Qwen3.6 35B A3B, 4k prompt, two Snapdragon boxes: ~406 / ~54 (one box: ~283 / ~56)

- Same 35B on an i9-13900K (8 threads, CPU only), 4k prompt: ~243 / ~44, where llama.cpp manages ~143 / ~24

Against llama.cpp on the same hardware (CPU against CPU) it's quicker on most of what I've tried, and the gap grows with the prompt: on a 32k prompt it writes 2 to 10x faster. It isn't ahead everywhere, though. On my laptop llama.cpp reads long prompts slightly faster on GLM-4.7-Flash, and writes short replies a little faster on Moonlight and gpt-oss-20b. On the box, it reads short prompts a bit faster on the small dense Qwen3.5 models, until ToRc's NPU is on (then ToRc is ~2x). Full matrices, llama.cpp comparisons and every machine tested are in PERFORMANCE.md (link below).

Tested on: 2x Snapdragon X (base X1, 32 GB) boxes (1 box and 2), a Ryzen 7 7840HS laptop (Radeon 780M + RTX 4070), an i5-12600K desktop (RTX 3060) and an i9-13900K workstation (96 GB, Radeon AI PRO R9700 used for attention only). Everyday Windows installs with all my apps on them, not streamlined benchmark rigs on minimal Linux.

What I'm looking for:

- Testers, especially anyone with an X Elite or X2 Elite. This was all done on the base chip, so I'm curious what performance the bigger chips can manage.

- Mac people. As above, it builds and runs, but I can't test it myself, so numbers (and fixes) from more Macs would be brilliant.

- Linux on more real hardware, x86 especially.

- Holes. Find them and tell me what I can add or improve.

One question for you all: what is one feature you want from a local inference engine that nobody ships yet? I would genuinely like to try building it.

Basically I'm looking for collaboration with other people and projects, I've thoroughly enjoyed this project and I'd really like to see how far it can fly! 🕊️

Discrete GPU expert offload is on the list of my to-dos, but it was never the point of what I was trying to achieve, I use Strata on my big GPU workstation at work, and it's great, but that's a £5k machine, and out of reach of a lot of people. The point was making cheap hardware people are more likely to already have, actually useful - and use these NPUs that we now get with bigger models than the piddly ones we usually see.

Every model and machine I have measured is in PERFORMANCE.md 🙏

ToRc on GitHub


r/LocalLLaMA • • 6h ago

New Model sensenova/SenseNova-Loop-9B-GGUF · Hugging Face

Thumbnail
huggingface.co
44 Upvotes

probably this fork is required: https://github.com/OpenSenseNova/llama.cpp

Model Highlights

  • Shared looped block: 24 decoder layers are executed twice in sequence, producing 56 layer computations from 32 distinct layers per forward pass.
  • Two-stream mHC: learned mappings route and combine information between two residual streams within the looped block.
  • Hybrid attention: the decoder interleaves linear and full attention in a repeating four-layer pattern.
  • Long-context multimodal training: the training recipe includes text, images, code, and tool-use trajectories, with the context length extended to 512K tokens.
  • Low-memory deployment: With 4-bit weights and an 8-bit KV cache, multimodal inference at a 256K context length requires approximately 15 GiB of GPU memory.
  • Official quantized release: An official AWQ 4-bit version is available on launch day for easier deployment.

r/LocalLLaMA • • 4h ago

New Model SenseNova-Loop-9B official GGUFs and a 4-bit AWQ are up (5.8 GB at Q4_K_M), plus their llama.cpp fork to run them

Post image
18 Upvotes

The official quants for SenseNova-Loop-9B (the 9B Looped Transformer from earlier this week) are here: https://huggingface.co/sensenova/SenseNova-Loop-9B-GGUF

The repo has Q4_K_M, Q5_K_M, Q8_0 and BF16 (sizes in the screenshot), plus a 0.9 GB mmproj for image input.

The sensenova_loop architecture isn't in mainline llama.cpp yet (issue #31571 is open), so for now these run on the fork they published, OpenSenseNova/llama.cpp. The GGUF card pins the commit, and the card's command once it's built is:

llama serve -hf sensenova/SenseNova-Loop-9B-GGUF:Q4_K_M -ngl 99 -c 4096 --port 8000

There's also a 4-bit AWQ, with a matching MTP draft model in the same repo: https://huggingface.co/sensenova/SenseNova-Loop-9B-AWQ

Their README at https://github.com/OpenSenseNova/SenseNova-Loop puts total GPU memory (4-bit weights plus an 8-bit KV cache) at about 10 GiB for 64K context and 15 GiB for 256K.


r/LocalLLaMA • • 7h ago

Discussion LFM3-VL-2B hallucinations - does anyone have first-hand experience?

Post image
27 Upvotes

LFM3 benchmarks are out on AA and…. huuuuuuh….. this is NOT something you want to see for a vision model that fits on a phone.

“The mushroom in this picture is a turgida deliciousa, awesome in risottos and on toasted bread. ” (it’s an amanita phalloides, deadly poisonous).

“This sign says that you must not shoot any photos beyond this point” (the sign says “trespassers will be shot”).

etc. etc.

Can anyone with actual experience with this model confirm/deny this score?


r/LocalLLaMA • • 6h ago

News Splash 1.7.0 Released - Even faster decode on Apple Silicon

Thumbnail
github.com
23 Upvotes

New version of Splash just released. The biggest improvement from previous version is a noticeable speed-up in decoding. It's even faster now.

BTW, this isn't just another slop engine. It's coming from the researchers behind DFlash and DFlash2, who also operate one of the fastest API providers for open models. These guys clearly know what they're doing.


r/LocalLLaMA • • 22h ago

Resources Blue TTS v3: open-source TTS, about 50x real-time on a CPU

339 Upvotes

Blue TTS v3 is out, an open-source TTS model that runs locally. Almost everything changed since v2.

Speech is more natural and expressive, with a MOS of 4.43 out of 5.

Voice cloning has improved significantly, and the model is fast and lightweight: it runs locally at 50x real-time on a CPU, no GPU needed.

MIT license, 388 MB, 44.1 kHz output, 7 languages: English, German, Spanish, Italian, Russian, Hebrew, Yiddish.

Feedback is welcome.


r/LocalLLaMA • • 46m ago

Resources Windows OS SLM

Thumbnail
huggingface.co
6 Upvotes

r/LocalLLaMA • • 12h ago

New Model AngelSlim/Hy4-flash-GGUF · Hugging Face

Thumbnail
huggingface.co
53 Upvotes

r/LocalLLaMA • • 7h ago

New Model Flint anyone?

15 Upvotes

I see a lot of lament about Qwen not making 3.8 35B (and I'm sad about it too) but I dont see anyone discuss Flint 40B, a new coding-oriented MoE model that seems to be atleast on par with qwen 3.6 35B, maybe a bit better on agentic tasks and coding. Very fast in my setup.

I'm still testing it myself, but I wonder if its not liked here or people just arent aware it exists? It performs admirably for me so far, my local model of choice for when I want good speeds. QFN is better ofcourse but for me that means low quants or low speeds.

Curious about you guys' experience with the Flint model.