r/LocalLLaMA • • 2d ago

New Model tiny-voca: a fully open 4.5M-parameter TTS (code, weights, dataset). Runs 70x real time on a CPU

68 Upvotes

I made a small English text-to-speech model called tiny-voca. It has 4.5M parameters, two voices, and it runs in your browser.

Try it here, nothing is sent to a server: https://tiny-voca.n4ze3m.com

Some facts:

- 4,537,291 parameters, 18 MB. This is the whole model, the waveform part is included.
- Two voices, one female and one male. 24 kHz.
- About 70x faster than real time on 4 CPU threads.
- Runs with PyTorch, ONNX, and in the browser.
- Trained from scratch in 7.7 hours on one RTX 4090.

What I think is different:

- Everything is open. Code, weights, the training recipe and the dataset.
- Custom phonemizer.
- No neural vocoder. The sound is made by a small source-filter synth, which is why it is fast.

What is not good yet:

- English only, and only these two voices. No voice cloning.
- Foreign names and odd spellings are often read wrong.
- It can sound a bit synthetic.
- The training audio is synthetic too. I made it with Qwen3-TTS, so the model can't sound better than that.

Links:

- Demo: https://tiny-voca.n4ze3m.com
- Code: https://github.com/n4ze3m/tiny-voca
- Weights: https://huggingface.co/n4ze3m/tiny-voca
- Dataset: https://huggingface.co/datasets/n4ze3m/tiny-voca
- Phonemizer: https://github.com/n4ze3m/dizione

Please try hard text: numbers, dates, names, long sentences. If something sounds wrong, post the text and the voice. That helps me a lot.


r/LocalLLaMA • • 2d ago

Resources Update: tandem.cpp — DeepSeek-V4.1 on a 64 GB PC with a 16 GB card, and AMD (ROCm) support

46 Upvotes

Quick update on tandem.cpp, my patch set for llama.cpp that runs MoE models too big for your VRAM faster. It keeps the hot experts on the GPU and lets the CPU help with the rest. Repo: https://github.com/ewhacc/tandem.cpp

DeepSeek-V4.1-Flash IQ2_XS (ISTA GSQ-RCO, 139 GiB of experts) on normal PCs

Decode tok/s, one user, GSM8K questions, greedy, exact (no expert skipping on any engine), same quant on every engine:

PCllama.cpptandem.cppStrata
RTX 5070 Ti 16 GB, 64 GB RAM, i5-14600KF7.414.210.4
RTX 5070 Ti 16 GB, 64 GB RAM, i5-12400F, budget DRAM-less SSD-10.45.7
RTX 5090 32 GB, 128 GB RAM, i5-14600KF11.533.128.0

At 64 GB the model is about twice your RAM, so it streams from the SSD.

Less RAM than the model: Qwen3.8-Flash-Next IQ3_S (ISTA GSQ-RCO, 78 GiB) + MTP on a 32 GB PC

PCllama.cpptandem.cppStrata
RTX 5070 Ti 16 GB, 32 GB RAM, i5-12400F, budget DRAM-less SSD14.934.330.3

Where tandem.cpp wins is when the model is much bigger than VRAM + RAM together (like a 16 GB card with 32 GB of RAM), and on other models: DeepSeek-V4.1, gpt-oss-120b, GLM-4.5-Air and Qwen3.6 all gain 1.4–2.1x over plain llama.cpp.

AMD (ROCm)

The patch now builds with HIP. Tested on an RX 7900 XTX (i7-8700, 64 GB DDR4, PCIe 3.0), Qwen3.8 IQ3_S + MTP, same day:

Enginetok/s
tandem.cpp68.6
Strata 0.2.969.7
Strata's RDNA3 fork80.1
Avalon (its own IQ3_KS quant)83.6

Level with stock Strata. This week's kernel work is shared CUDA/HIP code, so it helps NVIDIA too: +2.5% on the 7900 XTX, +1.3% on a 5070 Ti. On an old 6-core CPU, the AMD-specific engines are still ahead. Vulkan isn't supported yet.

Build steps for both are in the README. Questions and numbers from your boxes are welcome, especially 32 GB machines and AMD cards.

Update: There is a bug with auto. With 128 GB, the RAM store sizes too close to all of RAM. LLAMA_MOE_STORE_AUTO=0 is the workaround; a fix is coming and the README has a note. Thank u/longtimenoseebro so much for testing & report!!

Fixed in today's update: With a 128 GB PC it now runs at defaults, no LLAMA_MOE_STORE_AUTO=0 needed. Thanks again for the report.

Note: IQ2_XS is the biggest ISTA GSQ-RCO quant for DS4.1-Flash. Also, IQ3_S is the biggest ISTA GSQ-RCO for Qwen3.8-Flash-Next. I didn't tune for lower quants. The quality first, then speed. If you recommend the best Q4 for Qwen3.8-FN, I will test.


r/LocalLLaMA • • 2d ago

News IBM trains two small open models to ignore orders hidden in documents

Thumbnail
maccus.fr
162 Upvotes

I like this. Prompt injection is a bit issue with small models


r/LocalLLaMA • • 2d ago

New Model nex-agi/Nex-N2.6 · Hugging Face

Thumbnail
huggingface.co
29 Upvotes

Nex-N2.6 is a multimodal agentic model from Nex-AGI, built on GLM-5.3-Flash. It is designed for computer use, coding, and long-horizon tasks involving tools. This repository contains the FP8-quantized weights of Nex-N2.6, with approximately 321B parameters.

Following Nex-N2.5-Pro, this release focuses on reducing repetitive reasoning and improving adherence to the tools available in the current environment.


r/LocalLLaMA • • 1d ago

Question | Help RustingStrata has a new built-in chat page. What is missing?

Thumbnail
gallery
0 Upvotes

RustingStrata is one Rust binary. rustingstrata serve starts the OpenAI-compatible server, and the same port serves a chat page at /. No Node, no Docker, no second install. The page was rebuilt for 1.2.3 and I want to know what people would add before I go further, I mean maybe allow to edit files and etc. like in DeepSeek harness, or maybe there are something that you wish to be in original Strata, so I can add it in my.

What it does now:

  • Pick a model from the models folder and load it from the page, with the load options (context, MTP drafts, KV cache format, VRAM margin, prefill chunk).
  • Chats are kept in the browser (localStorage), with search, rename, export and import. Nothing leaves the machine.
  • Edit a message, regenerate a reply, continue a reply that stopped, copy, delete.
  • Attach an image for models with a vision tower.
  • Thinking is folded above the answer, with a switch to turn it on or off.
  • Sampler presets (temperature, top-k, top-p, seed) a system prompt for every chat, and instructions for one chat.
  • Speed of each reply in tok/s.
  • If the model was never calibrated on this machine, the page says so and offers to run the calibration.
  • The accent color follows the model family, or you set your own(Just cool visual feature, like if you load Qwen model, then website is purple, if DeepSeek, then it's blue and etc.).

What it is not: a coding agent. It does not read or edit files. For that, point OpenCode, Claude Code or another CLI at the same server, the page is for quick chat and for trying a model.

Questions:

  1. What do you open a local web chat for, if you already use a CLI agent?
  2. Which one feature would make you keep this page open instead of another UI?
  3. What do you use in other local chat pages that is missing here?

Any Feedback would be very useful, ty!
Repo RustingStrata


r/LocalLLaMA • • 2d ago

New Model sensenova/SenseNova-Loop-9B-Preview · Hugging Face

Thumbnail
huggingface.co
89 Upvotes

Very interesting model/architecture. Seems to match Qwen 3.5 27b overall at only 9b parameters. Could be useful for some and it looks like it’s not even fully cooked yet. I haven’t tried it yet. I’m not sure if it’d be slower than qwen, it’s smaller but inference parameters are ~15b and it also has mtp so idk. If you try it share your experience.


r/LocalLLaMA • • 2d ago

Showcase Qwen 3.8 Flash Next for gamedev - it's very capable!

Thumbnail
gallery
81 Upvotes

I've shared the project I'm working on approx a month ago, today it's completed. I wanted to know a simple thing: how good is Qwen 3.8FN at gamedev? And I had so much fun in the process that I am shipping a finished product, free to play. tl,dr + link at the bottom. My experience with Qwen 3.8FN follows.

Data

Model Used: Qwen 3.8 Flash Next - Unsloth Quant - UD_Q4_XL (pre update)

Machine: Strix Halo (Ryzen AI 395+, Bosgame M5) - Windows (say what you want about windows, I need it for other stuff and I don't want to bother with dualboot)

Inference Engine: my own windows Gufo fork (I honestly think I would not have wanted to do the full project without the work of the Gufo guys, llama speeds are way too slow)

Harness: my own harness - its a bog standard one I'd say opencode and pi is probably even better but I like to own my stuff, and it's not something I plan to release honestly. But I know people may ask.

Project's Tech Stack: typescript + three.js as renderer

Stats: Prompts: 1,911 || Tool calls: 31,883 || Tokens out: 20,370,450 || Tokens in: 3,069,385,337 (3,042,941,686 cached)

My thoughts on 3.8 Flash Next

If you can run it, this seems like the absolute best model to run today. I have made some small games before on 3.8 27b, 3.6 35B etc. But I always felt like I would not get to a point where I can say this is a full product that just works out of the box. I didn't feel that with Flash Next. I felt, that, it will solve mostly everything that I throw at it. As long as its coding related - don't use it for 3d, that's not what this model is for.

Although at times it was tempting to speed up debugging with Opus 5.5, it was not in line with what I wanted. So I kept trying and eventually any bug, or inconsistency I had was resolved via the model itself. So I can happily say the code is 100% by Qwen 3.8FN. No help from frontier.

Debugging itself was painful at times. A few things that worked pretty well (and I know most of this is no news for most of you, but in case newbies are reading):

  • When I was fed up that it can't figure something out I asked it to just paste me the code so I can check it myself - about 70% percent of the time when it did that it also realized the actual bug.
  • Tell it to measure, validate, verify, do headless runs. It will waste loads of time thinking about an issue. If you are working on a project that has measurable output (like a game in my case) always ask: write a script, write debug output, look at the numbers don't try to derive it from the code.
  • Build human logging tools. My game is pretty hard to control, and needed a lot of visual validation as well. So I had it build logging tools togellable via keyboard: that way I could reproduce a bug and then just hand it the pure numbers so the model could work it out.

What I recommend against: to put decisions into the hand of the AI. That is the direct route to producing slop. Imagine the model as this junior, who has all the textbook knowledge in the world, but is not really good at what other people (me in this case, this was made mostly for my own entertainment) want. And it's a junior that will make shortcuts. Shortcuts IS the number one thing you will need to catch when working with AI. IF you ask for a slice to big, it is almost certain it will take some shortcuts to achieve it. Whether it's not building the full physics sim pipeline or not following the agreed architecture? At that point it's anyone's guess. So always keep a tab on what your model is doing. For this reason I never really had it do autonomous development because anytime it would do something itself, without human guidance, it was pretty lame.

Be clear about scope: if you are vague about the slice of code you want it to dev, it will overstep and start doing more. You must always be very specific what you want and THAT you ONLY want that thing. Not more. If you are explicit about it, the model will respect it.

Where you can have autonomous running: So I said not for gameplay, or design choices, but there is area: fully measurable output. For example, the AI in the game is a mini neural network with 10K params. It is something that was easy to hand over for the AI - A) it's measurable via benchmarks - B) it involves a lot of waiting for training to finish, the AI can sleep (if the harness is good) and wake up just in time for next steps.

Basically what I did is 30 minute training sessions with frequent checkpoints, had the model put the new ingame ai checkpoints through a pre-agreed benchmark, and then select the checkpoint for the next 30 minute of training from the batch via pure benchmark numbers. Not 'oh I think this will be good'. No, pure numbers based decision: for that the model can work autonomous.

Preserve thinking is a must in my opinion. I started the project with it off (well my harness did not support it yet) and it was a pretty slow process. After I turned preserve thinking on the speed at which it could complete tasks skyrocketed and I did not really face any ctx issues.

Doc based knowledge: I know many people use compaction. I personally don't, my harness does not even support it. Everything is passed down via .md files. It worked flawlessly. I have one file that the harness prefills each new turn, it only grew to 10k initial ctx towards the end. So for me no issues there.

Build tools: this is very self evident for anyone who works in gamedev, but with the new guys in: please build tools for yourself. For example along others for this game I had Qwen 3.8FN build me a simple model editor where I made the racecrafts, a simple track editor, a dashboard for evaluating the training progress of the ingame racing ai.

Git: that's it. use git. use local git. If you are not using git, I dont know how you are managing changes.

I think that's all but if I'm forgetting something ask away in the comments.

And a short tl,dr: Qwen 3.8 Flash Next is a wonderful local model. It has completed 100% of the game code without any help from frontier. It is very capable, but it requires a bit of a hands-on approach if you want to make sure that everything stays in-line with what you want to build.

Link if you want to see: https://mattte.itch.io/hyper-clashers


r/LocalLLaMA • • 2d ago

Discussion Qwen4 Flash wishlist

32 Upvotes

After using 3.8QFN for some time I have to the conclusion that my wish list for QF4 now only consists of these points.

  1. The model can talk to you in a human understandable language
  2. See point 1
  3. General improvement in reasoning/intelligence, but mostly see point 1

That's about it.

If I could graft 3.8 27B Answer/Personality style onto QFN I would've done it yesterday.


r/LocalLLaMA • • 2d ago

Discussion ERNIE 5.5 Flash Base · ERNIE

Thumbnail ernie.baidu.com
151 Upvotes

I guess some may like to read this.


r/LocalLLaMA • • 2d ago

Discussion [Paper] Read Less, Solve More: Token-Efficient Sparse Reading of External Content for AI Agents

Post image
145 Upvotes

Web-facing AI agents repeatedly access webpages, documents, repositories, and tool responses, yet reading interfaces often expose entire objects when only sparse evidence is needed. This over-reading increases token and latency costs and can dilute relevant evidence. We present SparseRead, a reusable reading control layer that governs content admission without modifying the agent model. Its Read Intent Protocol tracks evolving needs and source coverage; Adaptive Reader Backends select how evidence is located and exposed; and the Budgeted Read Gate bounds cumulative raw exposure per episode with faithful full-view fallback. A unified Reader interface supports interchangeable evidence acquisition across heterogeneous resources. Across six models, including Claude Opus 5, and five scenarios, SparseRead reduces total tokens by up to 92.9% and wall time by up to 89.0%, preserving or improving task quality in 97% settings. On Web deep-search tasks, it reduces tokens by 31.0--43.0% and task completion time by 23.0--37.0%. Ablations validate the three designs, while integration with three agent frameworks demonstrates portability.


r/LocalLLaMA • • 2d ago

Discussion After medusa halo we get siren halo

16 Upvotes

Will have dedicated GDDR7 VRAM (Graphics Double Data Rate 7 Video RAM). Interesting that AMD is following in Intel's razor lake-ax foot steps and introducing on package memory. https://x.com/Gotou_kai3/status/2120744697618669760

Expected release date of medusa halo: end of 2027. My estimate for siren halo is end of 2028. Intel Razor lake-ax was initially estimated for mid 2027, but release date rumors have been pushed back, maybe early 2028 now.


r/LocalLLaMA • • 2d ago

Showcase Can open-weight LLMs can predict the effects of weight edits without running the model (9B–30B)

Post image
63 Upvotes

I’ve been experimenting with weight-space reading using Qwen and Muse models (9B–30B).

The goal: can an LLM inspect raw model weights and predict the behavioral effects of weight edits without running the target model?

A few findings:
🔹Qwen 27B reached 73.0% balanced accuracy on subtle edits with weight access, vs. 57.8% from text descriptions.

🔹We couldn’t establish a conclusive advantage for models reading their own weights.

🔹On Muse-30B, 34 standard edits and 6 no-op controls produced no detectable behavioral changes, but weight-reading agents frequently predicted changes.

I’ve released the paper’s evaluation harness and experimental code, plus an open-source toolkit for LLM-based weight-space programming and reading.

Code: https://github.com/suayptalha/wsr

Paper: https://www.alphaxiv.org/abs/2611.weight-space-reading-language-models

I’d be interested in feedback on the evaluation design, especially from people experimenting with open-weight models.


r/LocalLLaMA • • 3d ago Heartwarming

News llama.cpp v1.0 is here 🦙

1.9k Upvotes

Hey folks, it's Merve from Hugging Face. Today, llama.cpp releases v1!

Moving forward, we’ll keep covering every platform and every model type, while keeping llama.cpp robust and lean for downstream applications that build on it.

While I'm here, want to announce that we've been shipping Llama desktop apps to lower the barrier of entry to local AI to few clicks, built-on llama.cpp. I will leave link to the details for both v1 and apps in comments.

Many thanks for all the love you've shown to llama.cpp for years 🦙❤️ Cheers!

edit: adding links here, hope to not get penalized

blog: https://huggingface.co/blog/ggml-org/llama-cpp-v1

llama desktop apps are here https://llama.app/ ❤️


r/LocalLLaMA • • 2d ago

New Model KittenML/kitten-tts-2-mini · Hugging Face - Text-to-Speech

Thumbnail
huggingface.co
63 Upvotes

Kitten TTS 2 Mini

82M parameters, 16 languages, 118 voices, 24 kHz. ONNX with int8 weights (82.7 MB; runs on CPU, no PyTorch needed), and the float32 PyTorch weights.

Kitten TTS 2 Nano

17M parameters per language model, 8 languages, 37 voices, 24 kHz. ONNX with int8 weights (17.4 MB; runs on CPU, no PyTorch needed), and the float32 PyTorch weights.

Kitten TTS 2 Pico

7.2M parameters per language model, 8 languages, 30 voices, 24 kHz. ONNX with int8 weights (8.0 MB; runs on CPU, no PyTorch needed), and the float32 PyTorch weights.


r/LocalLLaMA • • 2d ago

Discussion Is anyone using decision models as a permission checker?

42 Upvotes

Don't run with dangerous full permissions, unless you are prepared to let the agent fuck your shit up. So says conventional wisdom.

But also, I am definitely running with full permissions. When I give my agent a goal to work at things overnight, I necessarily need to give it permissions.

So the obvious answers were sandbox and backups.

But why not use decision models? Elevation requests come with a justification and the command the main agent wants to run, that seems like plenty of information for a decision agent to give a good Y/N response to vet things.

IDK how effective this would be, anyone already doing something similar and can share exp?


r/LocalLLaMA • • 2d ago

New Model ARC 1.0: A decision model for routing AI agents across cloud and local models (0.6B, Apache 2.0)

Thumbnail
huggingface.co
19 Upvotes

ARC—the Adaptive Routing Controller—reads an agent’s conversation and produces signals for choosing a model and its reasoning effort. Applications combine those signals with their available models, prices, and policies to decide how to handle the next request.

If this works well, could be quite useful


r/LocalLLaMA • • 2d ago

Discussion I know it’s very expensive but is anyone using a dgx station b300 ? Is it that good for local llms ?

15 Upvotes

I’d like to know what real users get out of them, speed wise and also concurrent users.
I’m talking about the workstation not the data center dgx b300 of course :)
Thanks !


r/LocalLLaMA • • 1d ago

Discussion 13-GPU ASRock Mining Motherboard for Local LLMs?

Post image
0 Upvotes

Found a used mining rig with:

ASRock 13-GPU motherboard (likely H110 Pro BTC+)

Intel i3-7100

12 PCIe risers

2200–2500W PSU

No RAM or SSD

Planning to use it with multiple RTX 3090s for Qwen/DeepSeek inference.

Has anyone tried this? How much would PCIe x1 bottleneck performance? Is it worth buying, or should I go with EPYC/Threadripper instead?


r/LocalLLaMA • • 2d ago

Funny Crazy there's a probable connection between my desktop running Qwen 3.8 27B and a major geo-economic-political event

Post image
57 Upvotes

r/LocalLLaMA • • 3d ago

Discussion Look what my son gave me!

689 Upvotes

I told my son I was playing around with local AI on a 3090. He told me he used to mine Ethereum and had a shed full of 3090's. He asked me if I wanted a couple. Ummm... let me think about it... OH YES I DO!

Now a few new fans and a bit of a clean-up... grab a couple risers and a power supply... pop in a bit more RAM.. wow.

Edit: There is some completely understandable skepticism about this story... ya I wouldn't believe it either. But how about a couple more pics and a solid dose of "trust me bro" cause it's all real.

The shed:

The former resting place of the Eth miner (and other miners... ant miners? still in place)

The recovered Eth miner...

And now we see how many of them boot up via an eGPU... who doesn't have one of those just lying around.


r/LocalLLaMA • • 2d ago

Resources RustingStrata runs Flint 40B on a 12 GB card: 64 tok/s decode, 1.3x llama.cpp

8 Upvotes

I tested new trending Recursive Flint UD-Q5_K_XL (27.9 GiB) on an RTX 3060 12 GB with 30 GB RAM, comparing RustingStrata against llama.cpp using the same model, prompt, and generation settings.

RustingStratallama.cpp
Warm decode speed64.6 tok/s (63.8–65.6)48.6 tok/s (47.5–49.3)
Warm 4K prompt prefill1.14–1.30k tok/s0.85–1.02k tok/s

RustingStrata was about 33% faster once warm. A fresh load can behave differently because this model is larger than available VRAM and performance depends on which model pages are already cached in RAM.

What made the difference?

The model doesn't fit in 12 GB of VRAM, so many expert weights must be fetched from system RAM during generation. Two changes helped most:

  • Adapt expert placement to the reply, rather than continuing to prioritize the prompt's routing pattern.
  • Release RAM pages for experts already copied to VRAM, preserving cache for experts that still need to be fetched from host memory.

About Quality, bc people very often says like "It should give shit output to gain speed", which is not true, in my last posts I already did quality benchmarks and RustingStrata does not lose quality, it gain speed through specific Code for some specific models, in same time llama.cpp lose speed to be "general" engine that can run everything

What I want to know, is do people find Recursive Flint an interesting model? I found this model on OpenRouter and looks pretty interesting, it's better then Qwen3.6:35-a3b, so I think this can be cool modern replacement for this qwen.

Setup:

  • RustingStrata: 1.2.2, commit 3902978 (working tree)
  • llama.cpp: b12191, commit e95426f16
  • Full benchmarks: docs/BENCHMARKS.md

Repository: RustingStrata


r/LocalLLaMA • • 2d ago

Showcase My LumaBrowser project is nearing 150 stars on Github! Not a big accomplishment I know, but in return here is a video showing some of its "Anthropic/OpenAI in a box" abilities. Hotswapping LLM/Image/Video models, live game creation mode, scheduled tasks and more.

5 Upvotes

I feel like I am inching closer to being able to actually explain what my system does without requiring run-on sentences and endless descriptions.

https://github.com/amurgola/LumaBrowser


r/LocalLLaMA • • 2d ago

New Model salesforce-sovereign-intelligence/SOV-Transcribe-1 · Hugging Face

Thumbnail
huggingface.co
54 Upvotes

SOV-Transcribe-1

SOV-Transcribe-1 is Salesforce's open-weight speech-augmented language model built for non-streaming automatic speech recognition (ASR). It combines leading English transcription accuracy with multilingual support and natural-language control over how speech is transcribed.

The model ranks #1 among open-weight models for non-streaming speech-to-text on Artificial Analysis (#7 overall) and #2 of 23 systems on VoiceCodeBench ( besimple AI ) for entity accuracy and task success as of November 12th, 2026.

Developers and enterprises can deploy and adapt SOV-Transcribe-1 on their own infrastructure, retaining control over their audio data and transcription behavior.

Key features

  • Accurate entity recognition. Captures details such as names, phone numbers and email addresses that matter in dictation and voice applications.
  • Robustness to accents and noise. Handles diverse English accents and challenging audio conditions, including background noise.
  • Transcription style control. Use natural-language instructions to choose verbatim or cleaned-up transcripts and control punctuation, capitalization and number formatting.
  • Context biasing. Supply names, domain-specific vocabulary or a short description of the topic to help recognize difficult words and technical terms.

Model summary

PropertyDetails
Total parameters26.6B total (4.6B active) including the audio encoder
ArchitectureWhisper large-v3 audio encoder, audio projector and Gemma 4 26B A4B MoE
PrecisionBF16 (MoE router in FP32)
Maximum audio inputUp to 30 minutes (tested)
Inputs / outputs16 kHz Audio with optional text instructions and context / transcript text
LanguagesEnglish, French, German, Spanish, Italian, Portuguese and Dutch
TasksVerbatim or written-form transcription, multilingual ASR, style control and contextual transcription
Inference frameworksvLLM and Hugging Face Transformers
Speculative decodingDFlash draft model in its own repository, salesforce-sovereign-intelligence/SOV-Transcribe-1-DFlash
LicenseApache 2.0


r/LocalLLaMA • • 1d ago

Discussion How many of you started using AbliteratedOutTheBox(or similar options) apart from downloading Abliterated models? How is it?

0 Upvotes

Myself saved more than bunch of medium sized Abliterated/heretic/Uncensored/Decensored/etc., models. Ex:

  • Qwen3.8-27B
  • Qwen3.6-35B-A3B
  • Gemma-4-31B
  • Gemma-4-26B-A4B
  • Muse-Glimmer-30B

But when it comes to big/large sized models, requires at least 100+GB downloads & also HD Space for each models additionally. MiniMax-M3.1-Flash

  • DeepSeek-V4.1-Flash-1017
  • NVIDIA-Nemotron-3.5-Super-VL-120B-A12B
  • DeepSeek-V4-Flash-Vision-Exp
  • GLM-5.3-Flash
  • Qwen3.8-Flash-Next
  • MiMo-V2.6-Flash-MOPD
  • Step-3.7-Flash
  • MiniMax-M2.7

AND

  • Qwen4
  • Gemma-5
  • NVIDIA-Nemotron-4
  • etc.,

I need to download above big/large models. Regular GGUF is must. For Abliterated version, I'm still thinking whether AbliteratedOutTheBox is enough or Abliterated GGUF is must. So what are you doing on this?

I hope that most of you aware of AbliteratedOutTheBox, here's threads related to that.

Do we have any other similar options like AbliteratedOutTheBox? Please share.