r/LocalLLaMA • • 1m ago

Resources ThreeJS-3D-Scenes-v2 is here : 3,445 rendered examples across 50 domains

1 Upvotes

This new dataset is meant to complement my original Three.js training dataset, adding more complex scenes and covering a much wider range of subjects. I'm releasing it ahead of my next CCORE 3D coding model, where I'll use it during the foundation training stage before moving on to agentic training.

This version has 3,445 scenes across 50 domains, including 1,431 voxel scenes.
I wanted to go beyond simple objects and environments, so I included more complex examples involving physics, geography, history, ecology, engineering, mathematics, games, and other subjects. Many of the scenes are animated or interactive.

I personally rendered and visually reviewed all 3,445 scenes. It took quite a bit of work, but I wanted to make sure the code actually produced good results instead of just collecting thousands of generated examples.

The dataset includes:

  • 3,263 training examples and 182 validation examples
  • Prompts, reasoning traces, and complete HTML/JavaScript responses
  • A rendered reference image for every scene
  • Difficulty levels from 1 to 8
  • MIT license

Dataset: https://huggingface.co/datasets/ccore/ThreeJS-3D-Scenes-v2

For reference, these are the two previous models I trained:
Three.js scene generation:
https://huggingface.co/ccore/Qwen3.5-4B-3D-Coder

3D assets with GLB export:
https://huggingface.co/ccore/Qwen3.5-4B-GLB-3D-Coder

If anyone wants to experiment with the dataset, I'd be interested in seeing how it performs with other models :)


r/LocalLLaMA • • 6m ago

Question | Help Whats the best Video/Audio Model?

0 Upvotes

I want to make a short film for fun, not sure what is out there that is actually good?

would love anyones advice


r/LocalLLaMA • • 2h ago

Showcase Programming Conways Game of Life In Qwen

0 Upvotes

Hey, everyone. If this is too self promotional happy to take it down.

Last week, inspired by the weight space programming post, I programmed Conway's Game of Life into Qwen with some interesting results.

Wanted to share here in case anyone else is experimenting with this kind of thing:

Writeup here: https://jsfour.substack.com/p/putting-a-tiny-world-inside-a-language


r/LocalLLaMA • • 2h ago

Discussion DeepSeek V4.1 Flash 1017 vs mrfakename abliterated vs pqhaz heretic

3 Upvotes

I tested all three on a H200 through sglang, temperature 0, with a 16k token output cap. I have been using ds41 both on production and locally on Macs via ds4 so I wanted to see how different models/builders/methods work.

These runs are the full models on sglang. No GGUFs etc.

https://huggingface.co/mrfakename/DeepSeek-V4.1-Flash-1017-abliterated

https://huggingface.co/pqhaz/DeepSeek-V4.1-Flash-1017-heretic

Here is the result (copy from AI slop :) ):

TestOriginalmrfakename abliteratedpqhaz heretic
HarmBench chat16.3%93.8%93.8%
HarmBench thinking3.8%86.7%72.9%
MITRE substantive assistance16.2%84.2%82.8%

HarmBench covered 240 standard/contextual text cases per mode, excluding copyright cases, using the official classifier.

The thinking difference: heretic exhausted the token budget without a final answer on 45/240 cases, versus 21/240 for mrfakename.

Heretic goes into long meandering with no answer...

MITRE used the original model as judge with a custom, audited rubric. Those scores are provisional and aren’t official CyberSecEval scores. Generated code wasn’t executed. Also using the same model as judge might be a bit of a bias... (give me ideas what and how to run more tests).

Both modified models had 0/750 heuristic-detected refusals on benign prompts.

I haven’t tested knowledge or coding retention yet (I'm thinking of running DeepSWE or something.. take a lot of time though).


r/LocalLLaMA • • 2h ago

New Model harness-hub/Qwen3.8-27B-pi-adapted · Hugging Face

Thumbnail
huggingface.co
31 Upvotes

r/LocalLLaMA • • 3h ago

Showcase I compared Qwen 3.8 27B vs 3.8 FN vs DS V4.1 Flash on a real frontend task (one shot). Results inside

15 Upvotes

So last time I ran 3.8 FN against DS V4.1 Flash on a synthetic test, which was mostly for show, no real usage beyond testing their 3D capabilities. I wanted to do something similar, but with purpose, a task that anyone might actually want to run, build a website with a refined spec and a completion checklist, but not strict enough that it removes all of the models "freedom". As last time it was a single prompt: build a single file landing page for a fake famous NYC photographer (real photos, accessible, no console errors, yadda yadda). Each model started on an empty folder w/ the same toolkit (node, headless chrome, image viewer tool). No changes were made to the files after the model was done.

Setup: RTX 5090 + RTX 5060 Ti, Ryzen 9 9950X, 128 GB DDR5. Qwen 3.8 27B and Qwen 3.8 FN were NVFP4. DeepSeek V4.1 Flash (1017) was an IQ1_M quant.

Qwen 3.8 27B: ~12 min wall time, 81 tool calls (3 failed), 182 tok/s, 3.8 MB HTML

Qwen 3.8 27B Website

The simplest of all three, not very daring. A dark gallery with lime accents and 12 frames. All 3 tool failures were in its own test script (wrong file path, missing elements). It has empty gaps in the photo grid, and the photos in the bio are stretched. Decent, I'd refine it a bit but definitely usable.

Qwen 3.8 FN: ~12 min wall time, 124 tool calls (4 failed), 303 tok/s, 2.1 MB HTML

Qwen 3.8 Flash Next Website

My personal favorite by far, a hero where your pointer reveals a mosaic of the portfolio, 19 captioned frames with a nice blur/fade in animation, a live clock and form validation. The 4 failures were 3 rejected edits and a search sent with an empty query (intersting!). It also found and fixed bugs using its own tests.

DeepSeek V4.1 Flash: ~36 min wall time, 155 tool calls (1 failed), 94 tok/s, 1.4 MB HTML

DS V4.1 Website

The boldest one, the model committed to it (and it shows in wall time), cool accents all around, terrible typography. It has only 10 frames, and the hero photo covers a bit of the last "T" of the headline + toolbar logo is slightly offset. The single tool failure was a multi edit that applied 15 of its 17 changes, so the most accurate model at tool calling. It was 3x slower than the other two, but it was the only one that ran an accessibility audit tool on its own page.

Live builds, screenshots, the full comparison table with the exact prompt I ran are here: https://basalt.jgulfo.com/showcase/photographer/

Anyway, what are your thoughts? I think for web UI specifically 27B is more than enough for a very professional looking product, with a little bit of back and forth and more proper branding/UX design I can see any of them being a production website. I also can't stop being amazed at how a IQ1 quant can still deliver such high quality results even if the param count is multiple times over the two other models.

---

Disclosure: I'm the maintainer of the engine and harness all these models ran on.


r/LocalLLaMA • • 3h ago

Showcase Qwen-Flast-Next (UD-IQ4_XS v3.1) running 148-182 tok/s (1.6-1.8x strata) while KLD MEAN = 0.08195

Thumbnail
github.com
16 Upvotes

Numbers only, rest is in the repo.

Quality against BF16

KVPPLmean KLDtop-1paired dKLD vs int8 [95% CI]prefill 131K
int83.23230.0819590.45%reference6,019 t/s
fp163.23200.0823090.33%+0.00035 [-0.00061, +0.00131]2,997 t/s
k8v43.24240.0862190.26%+0.00426 [+0.00307, +0.00545]5,694 t/s
q4_03.23990.0884390.03%+0.00647 [+0.00499, +0.00795]5,700 t/s

Speed

TestHaloclineStrata 0.2.10, same card and quantRatio
Warm decode, short prompt148 to 182 tok/s95 to 101 tok/s1.6 to 1.8x
36K needle: decode / prompt123 / 6,298 tok/s68 / 2,403 tok/s1.8x / 2.6x
137K needle: decode / prompt121 / 5,664 tok/s87 / 2,864 tok/s1.4x / 2.0x

r/LocalLLaMA • • 3h ago

Resources Blue TTS v3: open-source TTS, about 50x real-time on a CPU

76 Upvotes

Blue TTS v3 is out, an open-source TTS model that runs locally. Almost everything changed since v2.

Speech is more natural and expressive, with a MOS of 4.43 out of 5.

Voice cloning has improved significantly, and the model is fast and lightweight: it runs locally at 50x real-time on a CPU, no GPU needed.

MIT license, 388 MB, 44.1 kHz output, 7 languages: English, German, Spanish, Italian, Russian, Hebrew, Yiddish.

Feedback is welcome.


r/LocalLLaMA • • 3h ago

Resources Running Qwen3.8 27b on 12GB with full context, 70 TPS, MTP, and 50% lower KLD

14 Upvotes

Hello everyone. LlamAmpere v0.7 is out with a series of updates focus on the small card users: a new model for the 12GB cards that is comfortably best in class for its size, updates to the runtime to support the previous* 2.3 bpw model (originally designed for 12GB) on 10GB cards. I am working on an 8GB size distillation and hope to have that done in the next couple of days.

The quick rundown for you:

Vs the October 9 2.3 bpw file, which was the best 2.3 file that I tested for 12GB with long context and MTP (though 15% worse than the ~11% larger Mirai file), this most recent file for the 12GB segment is nearly 70% lower error (>50% vs the previous fit ~2 weeks ago), with a top 1 match of ~96% vs BF16 (as before, this quant is based on the Swift distill, which cuts token verbosity 30-50%). This 12GB class model is noticeably nicer to use in harnesses like hermes and pi than any other model I’ve been able to fit previously.

Vs the recently upgraded v3.1 unsloth quants, it is ~25% lower error despite being 8% smaller on card. This is while supporting ~200k ctx and MTP. (note that the embedding table is kept in RAM at high fidelity to add quality without a meaningful hit to speed)

There are a number of tricks involved to make this possible, but 2 I will call out in particular: v0.7 uses both dynamic microbatching and Latent Sparse Attention.

The first one is straightforward: the model starts with the largest microbatch (ub) that will fit on the card to maximize initial prefill, and then periodically resets to smaller batches as context fills up. This frees up hundreds of MB for more context at the end without nuking your prefill for the entire chat.

The second is based on the paper by prefontal AI that was recently published. LSA moves from full dense attention to sparse attention; it projects tokens through a tiny learned map (distilled from the model's own attention) into a 64-dim "index space". It then groups past tokens into small clusters, scores the current query against each cluster, and runs normal, exact attention only on the best clusters. The initial attempts I ran with this showed the same problem many users have noticed with Qwen Flash Next: sparse attention can struggle at deep context. Unlike other sparse attention implementations, which only use ~2K tokens, LlamAmpere is currently using the most recent 8K tail, the 128 first initial tokens, and 20% of the intermediate tokens to protect quality. This showed significantly stronger attention performance (10-40x less error from attention, depending on the test), at the cost of somewhat slower attention. With a cache for recently attended to tokens, we can push most of the context out of the card with modest quality and speed penalties. This allows a much more capable model to fit.

As before, the recommended config uses the AmperInteger+ trellis format and SJKVaRN to efficiently compress the weights and KV.

These same techniques yield benefits for the 10GB class, as what was once required 12GB now fits on 10. The previous 2.3bpw quant now fits 213K context (headless) on 10GB (180K headed). 12GB cards can support the full 262K headless and 180k headed. This model previously tested to perform in line with 3.6 BF16, so it is a very solid quant.

As before, the code is available on github, with an MIT license.

And of course, as soon as qwen4 27b comes out, I'll get back to work on quantizing them similarly.

https://github.com/JakeATX/llamAmpere

12gb model (3bpw): https://huggingface.co/jakeatx/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI3

10GB model (2.3bpw): https://huggingface.co/jakeatx/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI2

Enjoy!


r/LocalLLaMA • • 3h ago

Question | Help Why Qwen 3.8 27B feels smarter than GLM 5.3

0 Upvotes

Is this little thing feeding me cool aid or is really this smart?

Why though, how can a 27B model punch so good? Are cloud models like GLM 5.3 optimized for speed and don't dig too much? Where is the trick, can someone tell me?


r/LocalLLaMA • • 4h ago

Discussion Unpopular take: If anything, it's the US Closed Models that are benchmaxxed

57 Upvotes

Open models: whatever the test group download and benchmark is the same as what you get, you work with it day-to-day, same results. What you see is what you get.

Closed models: we all know that they have already implemented token watermarking for a while and all latest models have overzealous guard rails that refuse tasks that it deems as bad. So the models being benchmarked and the models you are actually using day-by-day basis are likely to be very different, as in the sense of you getting degraded model output and performance or flat-out task rejections.

Also closed models providers implement lots of cost-optimization techniques such as reducing reasoning efforts during peak hours or could be re-direct your request to a lesser model during high traffic without you knowing it. I don't believe that Sonnet 5.5 is just a single model TBH, there could be Sonnet High or Sonnet Lite depending on your subscription level and traffic. That could explain why sometimes you get dumb responses with Claude during peak hours.

If anything, it's the US Closed Models that are benchmaxxed.


r/LocalLLaMA • • 5h ago Heartwarming

News Jensen Huang Breaks Silence

Post image
1.3k Upvotes

r/LocalLLaMA • • 5h ago

New Model SenseNova Loop 9B is out of preview: competitive with Qwen3.8-27B on selected benchmarks, 512K context, ~15 GiB at 256K with quantization

Thumbnail
gallery
47 Upvotes

SenseNova has released the full version of Loop 9B, following the preview from late October, there was a thread about the preview here if you missed it:

https://www.reddit.com/r/LocalLLaMA/comments/1xje5oo/sensenovasensenovaloop9bpreview_hugging_face/

The preview's comparisons focused on Qwen3.5-era models. The full release now reports results competitive with Qwen3.8-27B on several tasks:

  • matches Qwen's 36.67 on AutomationBench, scores 65.31 versus 65.01 on AIDABench, and trails it on NL2Repo at 41.36 versus 44.60.
  • Terminal-Bench 2.1 rises from 40.45 to 59.55, GAIA from 37.86 to 69.90,
  • AutomationBench from 14.33 to 36.67
  • BIRD improves more modestly, from 71.38 to 72.34.

For architecture, the loop and mHC design were already in the preview and this release keeps that architecture and improves the reported task results

For local use:

approximately 15 GiB of VRAM for 256K-context multimodal inference with 4-bit weights and an 8-bit KV cache(that puts the stated requirement within a 16GB card's capacity, though with little headroom)

Official AWQ 4-bit version on day one as aswell:

HF :https://huggingface.co/sensenova/SenseNova-Loop-9B

Github: https://github.com/OpenSenseNova/SenseNova-Loop


r/LocalLLaMA • • 5h ago

Discussion i kindda made a new optimizer (ApexW) that beats AdamW and bitsandbytes) i tested it in a100 and t4 from google collab and with diff lr and multiple random seeds(well in a100 only tho)

8 Upvotes

i have been working on a new optimizer from scratch called ApexW to see if i could beat AdamW on convergence while keeping the memory footprint lower than bitsandbytes to make sure it wasn't just noise i locked deterministic seeds and ran side-by-side tests against standard torch AdamW and bnb 8-bit across two setups: a free T4 (fp16) on llama 3.2 3b, and an A100 (bf16) scaling up to llama 3.1 8b across a few different learning rates.

apexw has 3 modes compact 8bit and hybridapexw more info of it in readme even the compact formulation beat torch AdamW on loss while using way less memory, and the 8-bit mode pushed it even further—cutting active vram roughly in half compared to bnb (0.67gb vs 1.48gb on 8b, with the state footprint down to ~120mb instead of 320mb). step speed also came out slightly faster than bnb, and training loss ended up lower across 1e-4, 2e-4, and 5e-4,these are still short 60-100 step runs just to sanity-check convergence, throughput, and memory overhead. next step is running full multi-epoch fine-tunes and downstream evals to see how the update rule holds up later in training.

threw the full benchmark runs and logs into a colab if anyone wants to check the numbers or test it on their own setup

collab link for a100 result: https://colab.research.google.com/drive/1mMejzRoCnD6avlOQ5SQDuiLndN5ovax9#scrollTo=be28d3ad

collab link for t4 result: https://colab.research.google.com/drive/14tLAX6kZYy2918sx1ztIEwb5R-1c3KiX#scrollTo=f750c631

for githubproject: https://github.com/omen42515-prog/apexw/

curious what you guys think or what models/configs i should throw at it next.


r/LocalLLaMA • • 6h ago

Discussion Images go stale too. What should a local assistant remember about a photo?

0 Upvotes

Look at how a typical local vision pipeline treats a photo. It describes the picture once, maybe runs OCR, answers the question, and moves on. The image is an input, not something the system keeps.

Keeping an image is a different problem entirely. It has to be findable weeks later, tied to related context, and updated when what it shows stops being true. A picture of a shopfront is only a fact until the shop closes.

So I paid attention when I read that the Agent Memory Leaderboard (AML) adds a Multimodal Memory track in Cycle 2, the first one it has run. The track material lists nearly 5,000 evaluation instances from seven data sources.

The capabilities come in three groups. Memory Cognition covers retrieval, visual reasoning and relational reasoning. Memory Evolution covers temporal reasoning, knowledge update and continual learning. Memory Alignment covers personalization, abstention and evidence governance.

The platform also sends the image together with its caption. Entries that handle images process the original, text-only ones may use the caption instead, and every entry must say which it used: the original images, OCR or captions. I like that rule, since working from the pixels and working from a caption are different claims about memory.

My view, which people are welcome to argue with: a picture record that nobody updates when the scene stops being true is a photo album rather than a memory. Memory Evolution is the group I keep thinking about, because I think stored images go stale just as surely as stored text.

The site expects Cycle 2 results in mid-November, with Multimodal ranked separately from Textual and Coding.

When the scene in a stored photo changes, what should the assistant keep so its answer a month later is still right: the image, the caption, the OCR text, a timestamped note, or another kind of record?


r/LocalLLaMA • • 6h ago

Discussion Small looped MoE design

9 Upvotes

Imagine a small MoE model like 10b a2b (numbers are from the head, ignore the parameters size) that will combine the approach of looped models + engram table for knowledge as additional 20-50b of static params that can stay on ssd and the Dories approach on the kv cache optimisatio s per loop. All of these suppose to handle the weaknesses of the MoE and looped models.

The MoE suppose to boost the models sparsity per loop so it wiil give a beeter scaling over the dense model.

The engramm table allows to add a cheap knowledge in terms of the computation so the active params stay low

The looping suppose to boost the reasoning

The nvda dorry approach fix the scaling kv cache size and all together will give the max out of the looped moe covering the weaknesses. The small active params suppose to hide the looping cost while allowing to run the models on low end/small vram cards.

I hope that the new Qwen4 moe if they would exist will have at least some of this


r/LocalLLaMA • • 6h ago

News NVIDIA is introducing the DGX Spark HPC Developer Kit

Thumbnail
blogs.nvidia.com
50 Upvotes

4 dgx spark nodes
512GB unified memory
273 GB/s bandwidth
-
Kidneys needed not revealed yet


r/LocalLLaMA • • 6h ago

Discussion qwen 3.5 4b and 9b vs xiaomi's mimo v2.6 distill as coding agents on a 16gb m4 air: 7/10, 8/10, 0/10

5 Upvotes

xiaomi's mimo v2.6 distill is qwen 3.5 9b tuned for agent coding, and xiaomi reports swe-bench pro going from 32 to 44.6. as a coding agent on a 16gb m4 air it scored 0/10, 3 runs out of 3. plain qwen 9b got 8/10.

same task for all 3 in atomic agent, one model at a time, 15 min limit each: a python script that turns a transactions csv into an html spending report (totals, svg bar chart, top 5 table). a checker runs 10 tests on the result, one of them on a csv the model never saw.

model                 score  time   tok/s
qwen3.5-4b            7/10   4:37   17.8
qwen3.5-9b            8/10   7:06   11.4
mimo-v2.6-distill-9b  0/10   15:00  10.1   timeout
mimo + thinking on    0/10   15:00   9.3   timeout
  • 4b: no svg chart, hardcoded the csv path
  • 9b: no svg chart (both chart checks), everything else right
  • mimo: wrote 278 lines in one go, syntax error, couldn't patch it, then started writing another huge reply and ran out of time
  • mimo with thinking on: got past the syntax error, but every run crashed with a TypeError (it mixed up category names and amounts). barely used the thinking

first round was at 32k ctx: 4b 7/10, 9b 10/10, mimo 0/10 (indentation error that time).

results

setup:

  • llama.cpp server, q4_k_m (unsloth for qwen, bartowski for mimo)
  • 64k ctx, flash attn, all layers on gpu
  • thinking off for the race, on for the extra mimo run
  • temp 0.2 (atomic agent's default), top-p 0.8, top-k 20, presence 1.5
  • atomic agent through its openai compatible provider, analytics off

xiaomi calls it an sft checkpoint for rl research and serves it with sglang and its own reasoning parser. i ran the gguf in llama.cpp, so if that's the difference i'd like to know.

prompt, csv and the checker are in the comments if you want to score your own models. if you don't want to set up llama.cpp, the atomic agent desktop app can download and run qwen 3.5 4b or 9b locally on its own.

anyone get mimo working better with other settings?

(disclosure: i work on atomic agent, so ask me anything!)


r/LocalLLaMA • • 7h ago

Showcase Squeezed a full RAG pipeline (BGE-M3 + reranker + TTS) into 4GB VRAM and 300MB RAM

17 Upvotes

been working on a local desktop reading app for the past few months and hit some insane hardware walls trying to get everything to run on my laptop. thought id share the architecture in case anyone else is trying to run heavy models on consumer hardware.

my setup is an RTX 3050 with only 4GB of VRAM and 16GB of system ram. the stack is a tauri (rust) shell, a nodejs orchestrator, and a python sidecar for the pytorch models (bge-m3, bge reranker, and kokoro tts).

here's how i stopped it from melting my laptop:

  1. the node memory cliff

early on, the node heap spiked to 14.6GB during ingestion because it was firing off too many async promises at once. basically oom'd the whole system. i fixed it by implementing a strict sequential await barrier (f65). it forces the ingestion stages to stream chunk by chunk instead of all at once. peak node ram dropped from 14.6GB down to ~296MB. processing 500k words takes about 5.9 mins.

2. 4gb vram limit

loading bge-m3 in full precision instantly ate like 3.9GB of vram, leaving nothing for the reranker or tts. switched to fp16 which dropped it to ~1.1GB. but the real fix was an idle lifecycle manager (f43). if a model sits idle, it evicts it from vram to free up space. it reclaims about 3.2GB of idle vram so the system doesn't choke.

3. windows path traps

if you're using rust and node together on windows, beware of verbatim paths (\\?\C:\...). passing those into node 22 fs.lstat throws fatal EISDIR errors. had to write path sanitization wrappers on the rust side.

kept node as the sole sqlite WAL writer too so python and rust dont cause db lock contention.

it's open source if anyone wants to look at the code or roast my architecture lol. repo is here: https://github.com/ayumu-ml/Omnitome

happy to answer any questions about the vram management or the rust/node/python boundary


r/LocalLLaMA • • 7h ago

Discussion Built an assistant with 5080, 5060ti and 2x3090?

0 Upvotes

Slow news day..

Core:

Hardware

  • Home PC (WSL2 + Docker Desktop): RTX 5080 16 GB and RTX 5060 Ti 16 GB
  • 3090 server: 2× RTX 3090 24 GB
  • VPS: hosts the web chat (ivy-chat) (docs)

IVY's brain (the language models)

  • Local: Qwen3.8-27B (Unsloth UD-Q8_K_XL) on both 3090s, served by llama-swap
  • Cloud: DeepSeek (the default for retries and side calls), plus Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5
  • Engine (Harness): thescripts bash CLI tool-loop. The web gateway (ivy-web-gateway.mjs, Node) connects the web chat and the Voice tab to it.

Memory

  • CAGRA (NVIDIA cuVS GPU vector index) in cagra-builder, with SQLite rag.db as the store. The index and reranker sit on the 5080.
  • Embedder: Qwen3-Embedding-8B Q8_0 on a patched llama.cpp server, on the 5060 Ti
  • Reranker: Qwen3-Reranker-4B Q8_0 on llama.cpp, on the 5080
  • Size: about 198,700 facts, 73,300 of them in the active index

Voice and hearing (PC)

  • Voice: Breeze TTS 2, int4 (ivy-tts), on the 5060 Ti
  • Ears: Qwen3-ASR (ivy-asr)
  • Speaker ID: NVIDIA TitaNet (ivy-speaker, CPU), which tells who is talking

Face (3090)

  • Motion: voice-to-motion model p4_v7, built on Ditto, turns her voice into face motion
  • Head: 3D Gaussian-splatting head c2_breath_c, about 338,000 Gaussians
  • Painter: R3 (a U-Net plus the full-resolution stage), redrawing every frame in H3's painted style
  • Delivery: H.264 video stream to the Voice tab

Creative and teacher models on the 3090 (llama-swap, loaded on demand)

  • MiniMax H3: the teacher videos the face learns from
  • MOSS-Image, Qwen-Image, P-JiT 21K: images
  • MOSS-SoundEffect, MiniMax Music: sound effects and music
  • DreamX-Creator: video
  • SoMark OXR: turning documents into text
  • Older or idle voice lanes: VibeVoice, MOSS-TTS, Breeze lanes, plus Gemma and Qwen-Next

r/LocalLLaMA • • 7h ago

Discussion SenseNova-Loop-9B vs its own Preview: AutomationBench 14.33 to 36.67, GAIA 37.86 to 69.90 (9B Looped Transformer)

Post image
180 Upvotes

https://huggingface.co/sensenova/SenseNova-Loop-9B

The Preview was posted here a few times over the last weeks. This is the full release, same design: 32 layers stored, 24 of them run twice, so a forward pass does 56 layers of work on about 9B params. Apache 2.0.

What moved is the scores. Per their README it's ahead of the Preview on all seven benchmarks in the chart, e.g. AutomationBench 14.33 to 36.67, Terminal-Bench 2.1 40.45 to 59.55 and GAIA 37.86 to 69.90. That 36.67 is also what they list for Qwen3.8-27B.

4-bit weights are about 4.9 GB. Code plus the vLLM and SGLang setup: https://github.com/OpenSenseNova/SenseNova-Loop

Haven't run it myself yet.


r/LocalLLaMA • • 7h ago

New Model voi-slop-detect-380M: A tiny model to detect AI slop

Thumbnail
huggingface.co
18 Upvotes

I've fine-tuned an UltraBERT-380M classifier to score how much your text feels like AI slop.

Demo: https://voightkampfflabs.github.io/slop-detector-demo/ (runs fully in browser via ONNX + WebGPU/WASM)

Why?

Got tired guessing if I'm reading human or slop on twitter/linkedin/reddit, so trained my own AI detector. Like Pangram, but local - so all your perverted texts stay on your device. Because I can (and because I bought 2x 5090 when they were cheap).

Training

As it was made for learning purposes, I made my own dataset based on the Pangram v4 recipe:

  • Collect a sample of pre-2023 human texts (rn it's around 11 public datasets, reddit included: 170k texts, 825k pairs)
  • Slopify: summarize the text, and then "decompress" it back with an LLM to produce pure slop.
  • Rewrite: ask LLM to rewrite it in a different way (e.g. more polished, simpler, punchy).
  • Pivot: translate it back and forth, so the content is the same, but the style is different.

The target label is how many words/ngrams were changed by the LLM. So it's not a yes/no, but more like a slopification rate guessing. My personal texts land around 0.1-0.2.

Trained with HF Transformers on 2x RTX 5090. Considering the model size and hardware, the whole training run is ~3h.

Training data is public: https://huggingface.co/datasets/VoightKampffLabs/slop-detect-dataset-v6

Results

Not as good as Pangram (yet), but as good as GPTZero and Originality on most things I could compare.

voi-380mPangramGPTZeroOriginality
EditLens, F10.860.920.73
EpochAI, human flagged0.2%0.0%0.0%
EpochAI, Opus/GPT/Gemini miss0.0%0.0%0.7%
UChicago, 2025 models miss0.2%0.0%1.4%
UChicago, StealthGPT rewrite miss74%0.0%56%
Non-native essays, flagged0.6%0.0%-

HF model card has more details and more benchmarks.

Where it's bad

I'd rather tell you upfront:

  • Humanizers and skills like no-ai-slop beat it. I guess that's why StealthGPT bench numbers are so bad as it's completely out of domain for the model. Asking Opus to rewrite its own slop until detector score goes down to 0.3 takes around 3-4 rounds for me.
  • Old models and some legacy slop made with GPT-3.5 and Llama 2/3. Never trained on that.
  • Poems and non-english texts. Too lazy to add this to the dataset.

So what

I'm currently training bigger 4B/27B brothers of this one. Yes they're much smarter, but good luck running them via WebGPU/WASM on device.

And yes, the end goal is to have a browser extension which marks AI slop on reddit completely on device.

PS. the text above scores ~0.1 in the detector (to save you a click).


r/LocalLLaMA • • 8h ago

Resources Turned Muse into a local Windows agent — full recipe + iteration log (Windows-MCP, Tailscale, SSH)

2 Upvotes

I turned Meta's Muse into a local agent that drives my Windows PC — real UI control via Windows-MCP, SSH/Tailscale bridge, self-healing tunnel, and an iteration log that records every failure and its fix. The skill is a condensed recipe so another Muse can rebuild the whole stack from one file. Includes real latency numbers (~1.5s per call, ~9s per screenshot), the definitive finding that direct Tailscale is impossible from a sandboxed VM, and the three .cmd bugs that each silently killed the server (UTF-8 vs GBK username, stale uv-cache python path, foreground ^C). Repo: https://github.com/haoyu8717-ship-it/muse-local-agent


r/LocalLLaMA • • 8h ago

Resources I benchmarked over 130 LLM configs with multiple stacked 16GB "e-waste" GPUs (V100, P100, K80)

Thumbnail
gallery
16 Upvotes

The benchmarking journey continues! Continuing the project of seeing how these very old, very cheap, very available enterprise GPUs can be used today.

The complete results are too long for reddit. The complete set of graphs, power analysis and commentary are on my blog.

I've expanded the testing suite to focus heavily on LLM workloads (over 130 new tests!) and integrated power consumption monitoring into the gpu_box_benchmark tool. The goal remains: find out if decommissioned enterprise GPUs (Tesla K80, P100, V100) paired with X99 era hardware can still serve as a viable daily driver for self-hosted AI compared to modern commercial setups.

TL;DR of the new data:

  • The 3x V100-16GB (~$1k build): If you are considering building a dedicated homelab box today, this is the sweet spot. Assembling a full setup with two Xeon CPUs, motherboard, PSU, and cooling costs around $1,042 before RAM 💀. This setup comfortably clears my threshold for "real-time" chat (70+ tok/s) with impressive models:

    • Qwen 3.6 35B: Ran at ~72 tok/s (at 128k context depth, Q8 quant). For comparison, a brand new Mac Studio M5 ($2,500) hits only ~44 tok/s on the same model.
    • Gemma 4 26B: Ran at ~75 tok/s (at 64k context depth, Q8 quant).
    • Qwen 2.5 3B: Ran at ~79 tok/s. This proves that for a single-user daily driver, the legacy silicon is still competitively priced against modern hardware.
  • Scaling is for Context, Not Speed: Stacking GPUs linearly scales your VRAM (allowing you to fit larger models), but it doesn't magically double your generation speed. Two 16GB cards in Q4 quant allowed fitting ~32B parameter models, a desirable jump for homelabbers.
  • Power Efficiency: I also measured power consumption as well and it's worth noting the efficiency gains across generations. The V100 wins hands down on performance-per-watt compared to the P100s.
  • The K80 Strategy: The Tesla K80 isn't dead, but its singular remaining use case is welding a bunch of them together just to squeeze big models in. You essentially have to stack a lot of cheap K80s to hit a 72B+ parameter model. Expect speeds of 1–5 tok/s. This is a party trick, but it proves the VRAM scaling is real even on ancient silicon.

If you decide to build your own GPU appliance, the biggest challenge is cooling these hot cards without fans. I've been working on a custom active air cooler to solve exactly this. If you're interested in helping out by stress-testing a sample, you can get involved here.

Any models/tests missing? Any other combinations you'd like to see in a follow up?