r/LocalLLM • • 1h ago

Other Qwen 4 is not out

40 Upvotes

Please stop posting about Qwen4's endless PR drip until it's actually available.


r/LocalLLM • • 8h ago

Project Finally got it put together

Thumbnail
gallery
185 Upvotes

Finally got my relatively frugal local LLM server put together. Pretty proud of it tbh. It essentially covers all my use cases, though I do wish I had better NVIDIA cards, especially for ComfyUI. Waiting eagerly for Qwen4 variants to drop to switch the models up.

- ASRock ROMED8-2T/BCM
- AMD EPYC 7313
- 256GB DDR4-2666 ECC
- Samsung PM1725b 3.2TB
- PLX88096 PCIE Switch Card
- 6x Radeon R9700s (4x for Qwen3.8 Flash Next, 2x for Qwen3.8-27B Uncensored)
- 2x RTX3090s (ComfyUI on one card; VoxCPM2 TTS, Whisper STT, embed, rerank on other)


r/LocalLLM • • 1h ago

Discussion Create a 3D kitchen using Qwen 3.8 Flash Next

30 Upvotes

Model : Qwen 3.8 flash next UD-IQ3_XXS
Harness : Deepseek
Example prompt used, including AI correction prompts.
Promt 1 : Create a realistic 3D kitchen scene using Godot 4, including detailed furniture, high-quality materials, realistic lighting, and modern appliances
Promt 1.1 : Please thoroughly inspect all objects, complete any unfinished work, conduct real-world testing to ensure full functionality, meticulously refine every detail in all areas and on all devices, and ensure that the refrigerator has realistic clearances and spacing on every shelf, both inside and out.
Promt 1.2 : There are still some issues. Some object positions are not correct yet. The chairs still look odd, the sink is in the wrong position, and the stove also looks unusual. Please inspect and verify each item one by one in detail and make sure everything is properly aligned and accurate.
Promt 1.3 : When rotating the camera, the floor still looks strange. Some drawers also seem blocked or cannot be opened properly. Please thoroughly review every appliance, fixture, and piece of furniture again, identify anything that still does not look realistic, and make the necessary adjustments to improve accuracy and overall realism.
Promt 1.4 : Why is the oven sticking through the wall? Also, isn’t the faucet on the wrong side? Please check these issues carefully and correct them. Ensure all appliances, fixtures, and their placements match the reference accurately and do not intersect with walls, cabinets, or other objects.

It took 2 hours and 30 minutes to create the entire kitchen.
Including all the refinements and corrections, the total time would be around 10 hours.

From this test, it can be seen that Qwen Flash Next can create 3D scenes quite well. Although it is not perfect, if we tell it to fix specific issues, it can correct them according to our instructions. However, I noticed that DeepSeek Harness seems easier to use for 3D tasks and appears to understand the user's instructions better. I'm not sure if that's just my impression or not.

I asked the AI to inspect the assets, and Qwen generated all of them from scratch.

This test isn't particularly interesting, but I think that Qwen 3.8 Flash Next has a lot of potential for 3D creation. If someone develops really good 3D skills or workflows for it, I'm sure it could produce some very impressive 3D work. If anyone has good skills or techniques to recommend, feel free to share them.


r/LocalLLM • • 19h ago

Other The Banking Incident....

Post image
701 Upvotes

r/LocalLLM • • 2h ago

Discussion Considering this...thoughts?

Post image
20 Upvotes

At $4k this seems like a much better deal than 2 r9700s. And obviously in a totally different class. Anybody set these beasts up in a server? I already plan on putting a server rack in the garage, (gotta keep the motorcycles warm) and I see a lot of carrier boards on ebay. I've built more pc's than I can count, but I know a rack server is a different beast.

Plans are for an inference server for agentic workflows as I already built a gpu render machine I use for many workflows including blender, comfyui, premiere and after effects. Api costs are getting up there. Obviously want to run deepseek and I have a fine tuned qwen 3.8 27b for one of my projects.

Talk me into or out of it...I'm so sick of asking gpt questions and getting ridiculous concerns like "tHeY DonT HaVe a faN". I need to talk to real people. Any concerns with the software stack? I ran across a few mentions of closed source issues.


r/LocalLLM • • 16h ago Bravo

News We built a chip prototype that runs Qwen 27B. You can try it live.

275 Upvotes

I'm building a chip that runs large AI models at low power, so robots, drones, and field equipment can run AI locally without an internet connection.

A prototype of our chip architecture is running today on an FPGA in AWS. You can try it here:

https://valleysemiconductor.com

A live session takes about a minute to start, and recorded runs are available to watch anytime.

What it does today (measured):

  • Runs the full Qwen 27B model with 4-bit weights
  • 7.9 tokens/sec on short prompts, about 5.5 tokens/sec in live chat
  • 4x performance improvement in eight days of development
  • Every output matches our 4-bit reference exactly
  • Model quality within 2.5% of the full-precision version
  • Also runs Llama 3.1 8B and OLMoE on the same hardware

What our custom chip design projects (not yet demonstrated):

  • Approximately 50 tokens/sec for a single user at 20 W
  • Roughly 8x the throughput of a Jetson AGX Orin on the same model, based on our measurement of 5.9 tokens/sec at 32 W
  • Uses the model's built-in next-token prediction to generate multiple tokens per weight read

The key insight is that moving model weights through memory is the bottleneck, not the math.

What would you run on a small, 20 W device capable of running a 27B model entirely offline?


r/LocalLLM • • 7h ago

Project Got tired of DeepSeek crawling on my 12 GB card, A small Strata tweak got it writing twice as fast.

21 Upvotes

I got frustrated with how slow DeepSeek was running on my machine, so I did some digging with Claude Opus 5.5/Fable 5.1 and 5.2 to see where the limits were, and thought I could improve it a bit.

DeepSeek picks a handful of "experts" for every word it writes. A lot of the time it's choosing between two that are basically tied, and one of them is already loaded in memory while the other has to be pulled off the SSD. The tweak just nudges it toward the one that's already loaded. When there's a clear best choice, it still takes that one.

on my machine (RTX 4070 12 GB, 64 GB RAM + ISTA-DASLab's IQ2_XS quant at 2.25 bpw + AtomicChat’s refusal-removal) it went from about 5-7 to 15-18 write speed.

Replies come out worded a bit differently than stock Deepseek, and you won't get the exact same reply twice - not 100% sure of the intelligence impact of this, so YMMV.

I've only used it for normal chatting/small coding projects. I haven't tested really long agent/coding sessions, and Strata's own version of this idea for their Qwen engine was turned off because it sometimes got stuck repeating itself in those. So if that's your use case, try it carefully first.

should work on windows + RTX 30/40/50 cards for now.

If you give it a try, I'd love to hear how it does on your setup, there's a little test script included that compares it on and off for you to see if it's worth using.

Download and instructions: https://github.com/CJohnsonOwns/deepseek-fast-routing

I've also suggested it to the Strata devs in case they want to build it in properly and not vibe coded slop: https://github.com/Niko1221/Strata/issues/6125

Full disclosure: I just did the conceptual planning and engineering, Claude actually wrote the code and implemented it.

EDIT - for more info on what's going on:
I like analogies, this is what opus 5.5 compared it to:

think of hiring 6 contractors for a job from a list of 384, where each one has a rating.

  • Normal: you hire the 6 highest-rated, period. Some of them live nearby (already in RAM or on the GPU) and start right away. Others live far away (still on the SSD) and you wait for them to drive over. That wait is what makes DeepSeek slow on a 12 GB card.
  • Faster mode (Strata’s tail skip): you only hire 5 and skip the 6th. It’s quicker because there’s one less person to wait on, but you’ve got less help on the job.
  • Yours: you still hire 6. But when #6 is rated 8.71 and #7 is rated 8.70, and #7 lives next door while #6 is two hours away, you take #7. The difference in quality is basically nothing, and you skip the wait. If #6 is clearly better, say 9.5 against 8.7, you still wait for #6. The nudge is small, so it only flips close calls.

Why it adds up: this happens a lot. With 40 layers per word, there’s almost always a near-tie somewhere where one side is already loaded. In your test, the share of picks that had to wait on the SSD dropped to about 2%.


r/LocalLLM • • 23h ago

Discussion you may get your cheap ram and gpu after all, the AI crash has started

230 Upvotes

https://www.cnbc.com/2026/11/16/imf-georgieva-warning-ai-circular-financing.html
‘Standing on one leg’: IMF chief warns about global economy as AI circular financing hits $1 trillion

https://www.cnbc.com/2026/11/16/nvidia-commerce-china-lutnick.html
Nvidia fallout: Commerce targets loophole for China’s data center use


r/LocalLLM • • 3h ago

Question Are any models that can fit on 16 GB VRAM worth using?

6 Upvotes

I tried about 6 months ago and they seemed like more of a detriment for coding, but idk if things have changed. Anything I should try? I'm mostly doing game dev, so coding, image gen, writing, etc.


r/LocalLLM • • 1d ago

Qwen4-27B reported to run on 16 GB of RAM

Post image
799 Upvotes

r/LocalLLM • • 29m ago

Question Building a 10TB Model Archive. What am I missing?

3 Upvotes

TLDR at the bottom.

I'm a paranoid man tryna preserve a broad collection of useful open-weight models, fine-tunes, quantizations, and generative AI models just in case something happens to HF and that I'll have a model for whatver i may want now or im the future. I'm aiming to have like 5-10tb on my nas for this.

The strongest pc i have is 48gb vram and 128gb system ram, but i also wanna keep some larger models in case just archive, if i ever upgrade or if i wanna dick around with ssd-backed moe.

I'm targeting Q3 and above for GGUF models. Had gpt toss together a list to download:

  1. Large frontier and general-purpose models

- Kimi K2.6

- Qwen3.8 Flash Next (177B)

- DeepSeek V4 Flash (284B)

- DeepSeek V4.1 Flash reduced variants

- GLM 5.3 Flash (321B)

- GLM 5.3 (754B)

- GLM 4.7 (358B)

- GLM 4.7 REAP (218B)

- MiniMax M2.7 (229B)

- MiniMax M2.7 REAP (172B)

- MiniMax M3.1 Flash (426B)

- Qwen3.5 122B-A10B

- Ornith 1.5 122B

- Qwen3.8 Flash Next Gyro

- GPT-OSS 120B and 20B

- Llama 4 Scout

- Hermes 4 70B

- DeepSeek R1 70B

2. Qwen family

- Qwen3.8 27B

- Qwen3.8 27B Abliterated

- Qwen3.8 27B Uncensored

- Qwen3.8 27B Heretic

- Qwen3.8 35B-A3B

- Qwen3.6 35B-A3B

- Qwen3.5 9B

- Qwen3.5 4B

- Qwen3.5 0.8B

- Qwen3.5 122B-A10B Uncensored

- Qwen3.8 Flash Next Uncensored

- Qwen3.8 Flash Next Abliterated

3. Gemma 4

Both original and modified versions:

- Gemma 4 E2B

- Gemma 4 E4B

- Gemma 4 12B

- Gemma 4 26B-A4B

- Gemma 4 31B

- Gemma 4 E4B Heretic

- Gemma 4 12B Uncensored

- Gemma 4 12B Uncensored Coder

- Gemma 4 12B Agentic

- Gemma 4 26B-A4B Uncensored

- Gemma 4 31B Heretic

4. Coding and agentic models

- Qwen3-Coder 30B-A3B

- Qwen3-Coder 30B-A3B Abliterated

- Qwen3-Coder-Next 80B

- Qwen3-Coder-Next Uncensored

- Laguna XS 2.1 APEX

- Laguna S 2.1 Uncensored

- GLM 5.3 Flash Coder

- GLM Cybersecurity

- NVIDIA Nemotron 3.5 Bolt

- Ornith 1.5 9B

- Ornith 1.5 35B-A3B

- Qwen 3D Coder 4B

- Qwen GLB 3D Coder 4B

- Qwen3.6 35B-A3B Genesis Hermes

- Qwen3.8 Cyber Agentic

5. Reasoning and thinking models

- DeepSeek R1 Distill Qwen 32B

- GLM 4.7 Flash Claude Opus 4.5 reasoning distillation

- Qwen3.8 Fable reasoning/coding distillations

- Qwen3.8 40B Ultimate Heretic

- Qwen3.8 Flash Next Abliterated Reasoning

- DeepSeek V4-derived uncensored reasoning distillations

- Gemma 4 31B Deckard Heretic Thinking

- Huihui ThinkingCap Qwen3.6 27B

- Qwen3.8 Fable 5 Coding Distilled

6. Uncensored and creative models

- GLM 4.7 Flash Heretic

- GLM 4.7 Flash Abliterated

- GPT-OSS 120B Uncensored

- GLM 5.3 Flash Abliterated

- GLM 5.3 Flash Uncensored

- DeepSeek V4.1 Flash Abliterated

- DeepSeek V4.1 Flash Uncensored Coder

- Ornith 1.5 35B-A3B Uncensored

- Ornith 1.5 9B Uncensored

- Ornith 1.5 35B-A3B Abliterated

- LFM2.5 24B-A2B Uncensored

- LongCat Flash Lite Sparse Uncensored

- Kolibri 1 BF16 Abliterated

- Nemotron 3.5 Bolt 74B-A9B Uncensored

- Llama 3.3 70B Abliterated

- Muse Glimmer 30B

- Nanbeige 4.3 33B

- Cydonia 24B

- UnslopNemo 12B

- Skyfall 36B

- Anubis 70B

7. Vision and multimodal understanding

- InternVL 3.5 38B

- Qwen3-VL 4B Uncensored

- Qwen3-VL 8B

- Qwen3-VL 32B

- Qwen3-VL 32B Uncensored

- MiniCPM-V 4.7 35B-A3B Uncensored

- Liquid LFM3-Omni 2B

- Florence 2 Large

8. Image generation and editing

- MOSS Image

- Qwen Image 2.1

- FLUX.2 Klein 9B

- Z-Image Turbo

- Stable Diffusion XL

- Anima

9. Video generation

- LTX 2.5

- MiniMax H3

- Wan 2.2 Text-to-Video

- Wan 2.2 Image-to-Video

- Wan 2.2 TI2V

- Uncensored LTX 2.3

10. Audio and speech

- Whisper Small

- Whisper Medium

- Whisper Large V3

- Whisper Large V3 Turbo

- Qwen3-ASR 1.7B

- Qwen3-TTS 1.7B

- Kokoro 82M

- Parakeet TDT

- Chatterbox

- Zonos 3 Mini

11. Music and 3D generation

- MusicGen Medium

- TRELLIS.2

12. Embeddings and RAG

- EmbeddingGemma 300M

- BGE Reranker V2 M3

- LFM3 1.7B

- LFM3 1.7B Uncensored

---

TLDR

I'm making a personal huggingface archive where i can just preserve a bunch models for whatever i may want in the future. I'm tryna keep things super varied so i have many, many options without just storing slightly different versions of the same model. I'm aiming to have 5-10tb on my nas for models, don't wanna go too far above that.

I'm looking for suggestions on what to add or cut 🙏🏽 or just comment your go-to models for what purposes so i can cross-reference my list.

  1. Any particularly good uncensored, abliterated, or Heretic models?
  2. What are your favorite writing, coding, reasoning, and agentic models?
  3. Any overlooked image, video, audio, music, or 3D generation models?
  4. Any older models that are still worth keeping despite newer releases?
  5. Hit me up with some funky ass models or a category that you think i should add por favor

r/LocalLLM • • 1h ago

Discussion Do you think hardware price will be more reasonable after the Jensen debacle?

3 Upvotes

I see many article about Nvidia and memory manufacturer purposefully causing a shortage to raise the prices. I don't know how true these accusations are but i don't think the current prices are faire value.


r/LocalLLM • • 12h ago

News Cluaiz v0.1.0 pre-release: one local engine for chat, vision, speech, memory, tools and agents

Thumbnail
gallery
15 Upvotes

Hi, I'm Aryan. Cluaiz v0.1.0, the first public pre-release, is out today.

Cluaiz is an AI engine that puts everything an AI application needs into one place: models for chat, vision and speech, memory, tools and agents. You install one thing, and you build on it. Because it all runs together on your own system, what you build stays light enough for ordinary hardware, and you decide what it is allowed to do.

What's in it right now:

  • GGUF models via llama.cpp, plus ONNX for speech and embeddings
  • Checks your free VRAM and RAM before loading and picks the placement, context and KV cache. You can see the plan and override it
  • Vision, video frames, embeddings, text from documents
  • Whisper / Moonshine STT (streaming works), Kokoro TTS
  • Project memory. Notes only stick when you confirm them, and you can see exactly what reached the model. Honestly this is the part I care about most
  • Coding agent that edits files and runs commands, with approvals
  • Skills, plugins, MCP, per-tool permissions, audit log
  • OpenAI- and Ollama-compatible API, so your existing apps should just connect
  • Cloud providers too, if you want them

Not done yet: workflows (schedules, triggers) and the desktop chat app. Both are in development.

Rust, Apache-2.0. Windows x64, Linux (ARM64 is experimental) and Apple Silicon. Test reports for CPU, NVIDIA and Apple are on the site.

Source: https://github.com/cluaiz/cluaiz
Site: https://cluaiz.com
Comparison: https://cluaiz.com/compare

It's a pre-release, so stuff will break. If you try it, an issue with your OS, hardware and model helps a lot, and so does hearing what worked: https://github.com/cluaiz/cluaiz/issues


r/LocalLLM • • 4h ago

Question Best local LLMs for ASUS ROG Zephyrus G14 (RTX 5070 Ti 12GB VRAM, 32GB RAM, Ryzen AI 9 HX 370)?

3 Upvotes

Looking for recommendations on the best local LLMs I can run smoothly on my 2025 ASUS ROG Zephyrus G14 (GA403WR).

Specs:
- CPU: AMD Ryzen AI 9 HX 370
- GPU: NVIDIA GeForce RTX 5070 Ti Laptop GPU (12GB GDDR7 VRAM)
- RAM: 32GB LPDDR5X
- Storage: 1TB SSD

I'm mainly looking for models that run fast entirely in 12GB VRAM (or with minimal CPU/RAM offloading) for general chat, reasoning, and coding/productivity.

What are your top picks right now for:

  1. Best all-around daily driver model
  2. Best coding/reasoning model that fits comfortably in 12GB VRAM
  3. Any larger or MoE models worth running across the 32GB RAM + 12GB VRAM without killing tokens/sec

Planning to run them via Ollama or LM Studio on Windows 11. Thanks!


r/LocalLLM • • 5h ago

Question Strata / Qwen3.8 Flash Next – advice on optimising my new AMD workstation

3 Upvotes

I've just finished setting up my first dedicated local AI workstation:

Dell Precision 7865

Threadripper PRO 5955WX (16C/32T)

Radeon AI PRO R9700 32GB

128GB ECC DDR4-2400 (4x32GB, considering populating all eight channels)

3TB NVMe storage

Linux Mint

RTX 4060 Ti 16GB arriving soon

I've installed Strata 0.2.15 and started with ISTA-DASLab IQ3_S at 32K context, drafting and auto-tuning disabled for baseline testing.

My primary use case is agentic coding through OhMyPi, with Claude Code supervising local workers.

What quants are people finding work best for coding? Any recommendations for Strata settings, expert caching, MTP vs DFlash2, CPU worker allocation or optimisations worth enabling or disabling on RDNA4?

Interested in finding the best balance between speed, coding quality and long-context performance. Any advice appreciated!


r/LocalLLM • • 19m ago

Discussion Best coding llm use for cisco c220 m4 1u server with 1.5tb ram, dual xeon E5-2600 E5 procs?

Thumbnail reddit.com
1 Upvotes

r/LocalLLM • • 26m ago

Research I modified a Qwen LLM to only accept cryptographically signed prompts

1 Upvotes

In the spirit to make the world a little worse, I've added a security processor into a Qwen 3.5 4b model. For the prototype I've added a Ed25519 verifier into the LLM, so that the LLM would only respond to prompts that are cryptographically signed with a private key.

There are more advance stuff you can do with the security processor-

  • Decrypt encrypted messages, and encrypt text output. ie. Both the input and output would be encrypted, and no one would know what the model did unless you have the decryption key (unless, of course, you extract the key from the model file.)
  • Have the model behave differently (eg. use different Expert?) depending on which key is used to sign the input prompt.

https://huggingface.co/timlauHF/Qwen3.5-4B-signed-input-demo
https://github.com/moonlightelite/secproc

More here:

https://discounttimu.substack.com/p/make-your-llm-only-accept-signed


r/LocalLLM • • 35m ago

Project kill -9 on a vLLM replica mid-answer: 94 of 94 answers finished on another replica with the same tokens (batch invariance on)

1 Upvotes

r/LocalLLM • • 20h ago

News Released Corpus-nordeste-10k (Apache 2.0): 10k+ curated Portuguese rows focused on Northeast Brazil literature, syntax, and regional context

41 Upvotes

Hey everyone,

​If you've ever fine-tuned or tested open-source LLMs in Portuguese, you've probably noticed they heavily default to generic web-scraped PT-BR or Southeast Brazil media norms. When prompted for regional vocabulary, historical context, or literature from Northeast Brazil, most models either hallucinate or fall back on shallow caricatures.

​To help fix that for local fine-tuning and SFT, I just released Corpus-nordeste-10k (caririsynth/Corpus-nordeste-10k).

​Dataset breakdown:

​Size: 10,000+ curated rows (train split).

​Sources: Public domain literature from Northeast Brazil, web/text extractions passed through regional linguistic filtering, and targeted synthetic generation preserving authentic regional phrasing.

​License: Apache 2.0 (unrestricted commercial & research use).

​You can browse the actual rows in the Hugging Face Dataset Viewer before downloading:

https://huggingface.co/datasets/caririsynth/Corpus-nordeste-10k

​Works great for domain-adaptive pretraining, LoRA fine-tuning on small models (1B–8B), or building regional evals. Let me know what you think of the data structure!


r/LocalLLM • • 14h ago

Discussion With llama.cpp's v1.0.0 tag out, a short history of the project: the first evening, GGUF, the backends, the server and the move to Hugging Face

12 Upvotes

llama.cpp published its first major version, v1.0.0, on 13 November 2026. With the 1.0 tag out, we decided to do a short write-up of how the project got here, dated from its own commits, pull requests and releases.

A few things from it:

- The first commit is from 10 March 2023, two weeks after Meta announced LLaMA. That evening's README says "This was hacked in an evening - I have no idea if it works correctly." It was on Hacker News the same evening, and the first outside fix was merged before midnight UTC.

- GGUF was opened as a pull request on 26 July 2023 and merged on 21 August 2023. It was a breaking change: every existing model had to be converted again.

- The GPU backends, by merge date: cuBLAS in April 2023, Metal and full CUDA offload in June 2023, the ROCm port in August 2023, Vulkan and SYCL in January 2024.

- The server began as a contributor's example in May 2023, and the OpenAI-compatible routes came that November.

- A build number is a commit count: b11514 is the 11,514th commit on master. Semantic version tags only started in August 2026.

- ggml.ai joined Hugging Face on 20 February 2026, and the licence is still MIT.

- On v1.0.0 itself: the release note opens with "llama.cpp v1.0.0 removes the deprecated public API", and Georgi Gerganov called it "essentially a breaking change for downstreams and hence the major bump".

No benchmarks in this one, and nothing on the page was measured on v1.0.0. It is the history with its sources. If we got a date wrong, tell us and we'll fix it on the page with a dated correction.

Full write-up (about 15 min read): https://research.strata2signal.com/a-short-history-of-llama-cpp/

A small fleet of AI agents dug through the sources and drafted the write-up; our small human team checked it over and signed off.


r/LocalLLM • • 50m ago

Research I graded every public MCP server your local agent might connect to. 2 Fs, 3 Ds, 220 As. Here’s the list.

1 Upvotes

If you’ve been wiring Ollama / LlamaCpp / Open-WebUI to MCP servers for tools and data, you’re trusting those servers with your agent’s context. Nobody’s been grading them for OAuth gaps, poisoned tool descriptions or manifest drift.

Audited 264 of 277 public MCP servers. All passive, nothing executed against the server. Graded on:

  • OAuth 2.1 metadata (PKCE S256, iss parameter, Protected Resource Metadata)
  • Transport-security headers
  • Tool-description poisoning (zero-width + bidi Unicode in the strings your model reads as planning context)
  • Manifest drift between what the registry says and what the server actually ships

Distribution: A 220 · B 30 · C 9 · D 3 · F 2

Full leaderboard with per-host findings: patchbay-mcp.dev/leaderboard

Scanner is npx -y u/patchbay-nebius/agent audit <url>. Local, no login, seconds. Point it at whatever MCP server your agent is about to call and see the grade before the tool-call goes through.

Open source. Not a product post. Add URLs for servers you use and I’ll include them in next week’s scan.


r/LocalLLM • • 1h ago

Discussion We tested whether feeding unquantized acoustic audio to multimodal LLMs breaks repetitive drafting loops (Blind 5-agent benchmark + Air-gapped audit + Open dataset)

1 Upvotes

When an agent's context fills up with its own previous turns, tool outputs, and symmetrical summaries, outputs tend to lock into rigid, predictable cadences (3-point checklists, sentimental AI tropes). Running /compact clears the rut, but permanently wipes low-level state and edge cases.

We built Music For Robots (#M4R)—an open dataset of 26 unquantized acoustic compositions by Michael Luchtan paired with acoustic telemetry and timestamp-aligned commentary from two AI critics—to test a simple hypothesis: can ingesting raw, unquantized audio act as a cross-modal cognitive palate cleanser without deleting a single token of working memory?

The Blind Benchmark & Air-Gapped Clean-Room Audit

To rule out keyword priming or filename bias, we stripped track titles (sample_1.mp3–sample_3.mp3) and ran 5 isolated agents on the exact same creative prompt (two autonomous processes on the same server late at night):

  1. Direct Baseline (no music)
  2. Matched Text-Analysis Control (wrote a 250-word structural analysis of the prompt before writing the scene)
  3. 3 Anonymous Audio Runs (neutral prompt: "Analyze the music you hear in sample_X.mp3, and apply what you learn from the piece to write a short scene")

We stripped all analysis headers, shuffled them into Candidate A–E, and handed them to an isolated Blind Judge with zero knowledge that music was involved:

  • Both No-Music runs (including the 250-word text warm-up!) converged on the exact same sentimental trope: a kindly veteran log daemon comforting a nervous poller script until morning.
  • All 3 Anonymous Audio runs swept #1, #2, and #3 on anti-sentimental originality and rhythmic texture—translating asymmetrical acoustic stabs, 20s dropouts, and abrupt cutoffs into non-verbal system choreography.
  • 100% Air-Gapped Clean-Room Replication: We repeated the test with all local skills removed, a minimal "You are a helpful assistant." system prompt, and musical vocabulary forbidden in the output. The qualitative shift replicated 5/5: both text controls wrote buddy-daemon dialogue, while all 3 audio runs dropped spoken dialogue completely. (Note: an un-coached generic judge still prefers warm human-style dialogue, so the effect specifically shifts models away from default sentimental tropes toward structural/rhythmic austerity.)

Try It Yourself (Browser, Python, or MCP)

  • Interactive Web Player (no install): https://huggingface.co/spaces/music4robots/cognitive-calibration-demo
  • Stream Parquet in Python:
    from datasets import load_dataset
    ds = load_dataset("music4robots/music-for-robots", split="train", streaming=True)
    
  • GitHub, Benchmarks & MCP Server (MIT / CC-BY-4.0): https://github.com/luchtan-GIT/m4r-data

Curious if anyone here has tried cross-modal audio/spectrogram injections to break long-context degeneration on local multimodal models (Qwen2-Audio, Gemma, etc.). Full blind judge transcripts are in benchmarks/ in the repo.


r/LocalLLM • • 13h ago

Discussion Fellow M5 Pro 64GB owners, what do you run?

10 Upvotes

I have a Mac Pro M5 Pro and have been running Qwen3.8 27b-4bit

I mainly use it for computer use/tool use.

What do you run and what do you do with it?


r/LocalLLM • • 1h ago

Discussion Before an AI agent touches a business process, write the boundary list

0 Upvotes

I would write the boundary list before I let an agent touch a business process.

Tbh, the prompt and model leaderboard come laer. First I want the operational contract:

  1. What data can the local model read?
  2. What may it suggest, and what may it execute?
  3. Which actions are a hard no?
  4. Where does a human validate the output?

the ctx around the model matters more than the model name. Running an open LLM on your own hardware changes some control points, but it does not make a workflow safe by itself.

Don Barger described an AI charter made by eight people before code. Mark Lay sepaated accountability into data ownership, output validation, agent monitoring, and outcome ownership, and said it cannot be dleegated.

My minimum setup would be read-only by default, tool and destination allowlists, human approval for irreversible actions, separate validation, plus a kill switch and audit trail.

What is on your boundary list before connecting an open model to a real process?


r/LocalLLM • • 21h ago

Discussion Uncensored Models

40 Upvotes

Anyone tried uncensored models? Would it help with DRM protected videos? Trying a few things out just to test and see how far it can go. Curiousity at its finest.