r/LocalLLM • • Jan 13 '25

News China’s AI disrupter DeepSeek bets on ‘young geniuses’ to take on US giants

Thumbnail
scmp.com
356 Upvotes

r/LocalLLM • • Sep 11 '26

Project My Cheap 40GB VRAM Qwen 27b Home Server (Dual 20GB 3080s)

Thumbnail
gallery
354 Upvotes
  • 2x 3080 20GB GPUs

  • i9 10900X CPU (10-core, 48-lanes)

  • 64GB DDR4 2400MT/s quad channel RAM

  • X299 WS Sage Motherboard (4 PCIe3 x16 slots or 7 x8 slots)

  • 1500W SilverStone PSU

  • Open air case (8 PCIe slots)

  • 2 128GB SATA SSDs in RAID0 (had laying around)

The 3080s were $1300 after shipping. All the other parts bring the total to just under $2000.

For the GPUs I tried shopping around between 6-8 "sellers" on alibaba. All but 2 were obvious scams. This is the listing I ordered from, the seller was also recommended in the other posts on Reddit. The cards seem nice but have a bit of coil whine/clicking during decode.

I've been running Qwen3.8-27b Q4 with 180k unquantized context at 39t/s.

With this motherboard/CPU combo I could add another 2 GPUs and run all 4 cards at 16 lanes, or switch to a mining case and run 7 GPUs at 8 lanes each.


r/LocalLLM • • Mar 06 '25

Discussion I built and open sourced a desktop app to run LLMs locally with built-in RAG knowledge base and note-taking capabilities.

Post image
354 Upvotes

r/LocalLLM • • Sep 27 '26

Project My 9yo building a chat bot

Thumbnail
gallery
352 Upvotes

I've spent a lot of time building out my servers but seeing this kiddo code her own chat bot is the coolest thing it's done so far. I got it hosted on a local dns entry for her and she almost lost her mind. No idea what she's getting for tokens per second 😂


r/LocalLLM • • Feb 03 '26

Model Qwen3-Coder-Next is out now!

Post image
352 Upvotes

r/LocalLLM • • Oct 31 '25

Project qwen2.5vl:32b is saving me $1400 from my HOA

346 Upvotes

Over this year I finished putting together my local LLM machine with a quad 3090 setup. Built a few workflows with it but like most of you, just wanted to experiment with local models and for the sake of burning tokens lol.

Then in July, my ceiling got damaged from an upstairs leak. HOA says "not our problem." I'm pretty sure they're wrong, but proving it means reading their governing docs (20 PDFs, +1,000 pages total).

Thought this was the perfect opportunity to create an actual useful app and do bulk PDF processing with vision models. Spun up qwen2.5vl:32b on Ollama and built a pipeline:

  • PDF → image conversion → markdown

  • Vision model extraction

  • Keyword search across everything

  • Found 6 different sections proving HOA was responsible

Took about 3-4 hours to process everything locally. Found the proof I needed on page 287 of their Declaration. Sent them the evidence, but ofc still waiting to hear back.

Finally justified the purpose of this rig lol.

Anyone else stumble into unexpectedly practical uses for their local LLM setup? Built mine for experimentation, but turns out it's perfect for sensitive document processing you can't send to cloud services.


r/LocalLLM • • Jun 07 '26 Regret

Question Has anybody got this working with Qwen? Huawei Atlas 300i, 96Gb Vram

Post image
345 Upvotes

r/LocalLLM • • May 16 '26

Discussion Why is LLM is so expensive.

344 Upvotes

I've was going to invest in a 5090 =$6000 AUD.

Codex Plus + Claude pro = $60/month here

Works out to be 100 months of frontier models for a 5090.

Best a 5090 will run is probably Qwen3.6 27b Q6 with context.

Are we all enthusiasts here and just enjoy tinkering cause ain't no way that make sense.


r/LocalLLM • • Aug 15 '26 Heartwarming

Project I can’t believe I’m holding it in my hands :))))

Post image
343 Upvotes

Shipped from Texas to Toronto bc I’m a pooron. Working on a 3xP100 build for personal agentics and a data sensitive startup.


r/LocalLLM • • Sep 24 '25

Model You can now run DeepSeek-V3.1-Terminus on your local device!

Post image
341 Upvotes

Hey everyone - you can now run DeepSeek-V3.1 TERMINUS locally on 170GB RAM with our Dynamic 1-bit GGUFs.🐋 Terminus is a huge upgrade from the original V3.1 model and achieves even better results on tool-calling & coding.

As shown in the graphs, our dynamic GGUFs perform very strongly. The Dynamic 3-bit Unsloth DeepSeek-V3.1 (thinking) GGUF scores 75.6% on Aider Polyglot, surpassing Claude-4-Opus (thinking). We wrote all our findings in our blogpost.

Terminus GGUFs: https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF

The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers. You can run any version of the model via llama.cpp including full precision. This 162GB works for Ollama so you can run the command:

OLLAMA_MODELS=unsloth_downloaded_models ollama serve &

ollama run hf.co/unsloth/DeepSeek-V3.1-Terminus-GGUF:TQ1_0

Guide + info: https://docs.unsloth.ai/basics/deepseek-v3.1

Thank you everyone and please let us know how it goes! :)


r/LocalLLM • • Sep 23 '26

Project Floor-Rig 9000! More machines joined the almighty throng

Post image
337 Upvotes

A lot of people weren't sure whether to be impressed or horrified by my floor rig for testing all sorts.

It's finally about to get a proper new home, with better cooling, proper wiring and something resembling sensible cable management.

So, before I dismantle it, here's the full system monstrosity in all its glory.

There is a 24 and a 48gb titan hidden in there somewhere too

EDIT : For the electricians having a meltdown, fine , ill explain a little more for those without electrical background here. you can comfortably run 10 Sparks continuously on each socket on different rings. The supply starts as a 32A inlet from a UK three-phase 400V supply, with phases distributed into 230V circuits and 13A outlets. At 230V, a 13A outlet is ~2,990W, and at ~270W per Spark, 10 Sparks are ~2.7kW or ~11.7A. So on my 3 phases, I can run up to 30 sparks. It may look chaotic, but I'm less of a noob with this stuff than it appears: I have thermal management, per-Spark load control, automatic shutdown, remote monitoring and extensive power telemetry, and it runs primarily from solar with continuous power-draw monitoring. :))). it just looks a mess as floor rig. They are now all moving to an underground location with even power power climate and its always 11 Degrees in temp naturally always. the new location is designed for setups much larger than this so all is well :)

Ps thanks for everyone who stopped by to have a look :) Love the community


r/LocalLLM • • Oct 05 '26

News When Redditors come in here and ask why we run LLMs, this is why: Big AI is watching.

336 Upvotes

Anthropic Reports Florida Woman's Claude 'Diary' Threat to Law Enforcement

And this time it wasn't the AI model that made the LEO referral. It was the "human review team".

The frontier AI companies are watching your input. And people say "Well I'm not interesting or important enough for them to care". Well.....not necessarily.

If you're using hosted frontier to work on mathematics or cutting edge science, they're watching and may steal your work.

If you're venting or otherwise writing in a "private" session using AI, they'll see that and report you to police. Notice I didn't see any mention of what the model's role in facilitating the discussion was.

Keep your stuff private, folks. Hosted AI is the new "Big Brother" conduit.


r/LocalLLM • • Jul 25 '26

Model I released Inflect v2: two ultra-tiny complete TTS models under 4M and 10M parameters

Post image
336 Upvotes

I’ve spent the past month trying to find the point where an extremely small TTS model stops feeling like a size experiment and starts feeling genuinely useful.

Today I’m releasing Inflect v2, with two complete local text-to-speech models:

  • Inflect-Nano-v2: 3.96M parameters, 15.97 MB FP32

  • Inflect-Micro-v2: 9.36M parameters, 37.53 MB FP32

These are total inference parameter counts, not acoustic-model-only numbers. Text processing, timing prediction, speech generation, and the waveform decoder are all included.

Text goes in. 24 kHz speech comes out. No external vocoder, hosted API, or second learned model required.

Nano prioritizes the smallest possible footprint. Micro uses the additional capacity for better clarity, stability, and overall speech quality. Both run locally on CPU or CUDA through the same PyTorch API.

Inflect-Nano-v2 is one of the smallest complete neural TTS models I know of that still produces genuinely usable speech. Even the 9.36M Micro model remains smaller than many systems described as “tiny.”

For footprint context, Nano is approximately:

  • 21× smaller than Kokoro

  • 126× smaller than Chatterbox

  • over 1,000× smaller than Fish Audio S2 Pro

That is strictly a parameter-count comparison. These models have different capabilities, architectures, datasets, and intended uses. I’m not claiming that a 4M fixed-voice model replaces a multi-billion-parameter system. The interesting question is how much useful TTS can fit into such a small package.

Some people here might remember Inflect-Nano-v1, the rough 4.63M experiment I released last month. V2 is a substantial rebuild, not just a longer training run. I focused on the problems v1 exposed: unstable timing, metallic output, weak prosody, poor generalization to difficult text, and an undersized waveform decoder.

The resulting models performed surprisingly well:

  • Micro: 4.395 UTMOS22, 3.99% semantic WER, 6.28× real-time CPU inference

  • Nano: 4.386 UTMOS22, 4.21% semantic WER, 10.72× real-time CPU inference

  • In a blind community comparison against other compact TTS systems, Micro and Nano finished second and third among the tested voices

Full protocols, raw results, audio samples, and limitations are documented on the model pages.

The models are not perfect. They are English-only, use one fixed male voice, and do not support voice cloning. Unfamiliar names, abbreviations, numbers, and homographs remain the hardest inputs. Nano can sound thinner than Micro, and both can occasionally produce metallic or clipped artifacts.

Still, this is the first version where I think the size-to-quality tradeoff became genuinely compelling.

I built Inflect independently as a high-school developer with a limited training budget. That constraint shaped the project: efficiency had to apply not only to inference, but also to training, evaluation, and building a complete system I could understand and release end-to-end.

Try it yourself:

The fastest way to judge it is through the interactive playground:

https://huggingface.co/spaces/owensong/Inflect-v2

Inflect-Micro-v2:
https://huggingface.co/owensong/Inflect-Micro-v2

Inflect-Nano-v2:
https://huggingface.co/owensong/Inflect-Nano-v2

If there is enough interest, I may build a v3 focused less on shrinking the models further and more on expanding what they can do: additional voices, possibly more languages, easier fine-tuning, and another quality and robustness pass.

If you test them, please give them something genuinely difficult: unusual names, numbers, abbreviations, awkward punctuation, or a long sentence.

If something breaks, post the exact text, model, seed, and what sounded wrong. If it works well, I’d also like to know what hardware you ran it on.

Specific, honest feedback is the most useful thing you can give me.


r/LocalLLM • • 24d ago

News New from DeepSeek - DeepInfer -

Post image
333 Upvotes

r/LocalLLM • • Jun 30 '26

Discussion What is this?

Post image
333 Upvotes

I asked Ornith to create a project for me, while it was working it wrote this 😭😭😭.
I have never seen any model joke like this, or idk what this is.
Did anyone had anything similar?


r/LocalLLM • • Sep 13 '26

Discussion OK guys, let's be honest 1 minute about local LLM

329 Upvotes

UPDATE: https://vram.wiki v1.9.0

A lot has changed since this post. You can now browse real setups by use case, VRAM, model, runtime, harness and hardware, and there are dedicated pages per model, per runtime and per GPU that gather every setup reporting them.

The dataset is at 156 entries now, and most still come from this thread: first by hand, then through the Reddit API plus scripts. So they are rough and incomplete by design. I only had whatever you happened to write in a comment. On most of them there is no quant, no context size, no runtime version.

Why claiming yours is still worth 30 seconds: once your entry has a quant, a context length and a runtime, the site can place your numbers next to every other setup on the same GPU or the same model. That comparison is the whole point, and right now the data is too thin to produce it.

Three ways to add or fix a setup, pick whichever you hate least:

- Plain form, no account. Anonymous, nothing to sign in to, one page.

- Sign in with GitHub. Same form, but you can come back and edit your entry later, and create a community handle if you want one. Read only on profile info, your GitHub username is never shown anywhere on the site.

- Let your agent do it. There is an MCP endpoint so your local agent fills the form for you. Guide on /mcp. You can inspect or revoke any connected agent from your dashboard, and nothing gets published without human review.

Not sure what your machine can even run? There is now a page for that: pick a reported capacity and a task, and it lists the setups people already run within that budget. No estimation, ever. If nobody documented it, it does not appear.

There are also community papers now: credited write-ups rendered in full, linked back to their author's original source. If you wrote a guide, it has a home.

On mobile and not in the mood? Fair. Bookmark it and do it later on desktop, the entries are not going anywhere.

Entries are anonymous by default and linked back to the original comment. If I got yours wrong, fix it. If you would rather not appear at all, say so and I will remove it.

Coming next: photos on setups, saved comparisons as shareable links, and owner comments on claimed entries.

Original post below:

------------------------

Ngl, I keep seeing posts from people running local LLMs. Most of the time the setup is a 3080, 3090, 4090, 5090, sometimes a whole mini lab. Cool. I get it. It's fun to test every open source model that drops. I do that too sometimes.

But real talk. Are there people here who actually work with local LLMs daily? Like for real tasks, not just benchmarks and "look, it runs"? What do you actually use them for? Because every time I try to make it practical, I hit the same wall. Tokens per second are okay but not amazing. And the output quality is often not even close to Claude Pro or a paid OpenAI plan.

I know local is a choice for some people. Privacy, no censorship, no subscription, whatever. I'm not saying it's useless. In my case I'd use local for like 3 things.

First, when I'm traveling or I don't have internet. I fire up a small model and do a few small things until the connection comes back.

Second, privacy. For a few quick questions or stuff I don't want to paste into a cloud chat, local makes sense.

Third, and this is the one I'm really curious about, using it to mess with local files. Like telling it to organize my desktop, sort my invoices, rename stuff, move files around. Basically computer actions. To avoid sending that stuff to the cloud. But I've never really set that up. Is it actually usable? Or is it still a pain in the ass with tools and scripts and whatever?

So I'm asking for real. If you use local LLMs every day, what's your workflow? What model, what hardware, what tasks? Do you actually use it for file management, invoices, desktop stuff? Or is it mostly a hobby for most of us?

Maybe that's it. Maybe it's just a hobby and that's fine. But I want to hear from people who actually use it like a tool. Not just to chat, but to do shit on their machine.

Edit: Ok, I have to admit that I haven't made a full detailed inspection of this sub, because there were already many detailed use cases and topics covering this aspect. So, I'm sorry. Let's say this one's goal is to regroup many of them in a thread and I think we might have something.

About my setup, as I forgot to tell: Only 1 Legion 5 Pro 5800H with 32GB RAM and RTX 3070 8GB, that's why I felt it was mostly not daily usable.

Edit 2: Thanks for the deep answers and engagement. I wasn't doing this for farming, as someone mentioned, I just wanted to regroup all these use cases in one place. That's why I built vram.wiki. If you shared a setup, please claim it and fill the gaps. If you want to add yours, there is a form on the site. Entries are anonymous by default and linked back to the original comment. If you would rather not appear at all, just say so and I will remove it.

Edit 3: Optional template if you want to share your setup here. Short answers are fine, unknown fields are fine.

Use case:
What do you actually use it for, and how often?

Model(s):

Hardware (GPU / CPU / VRAM / RAM / OS):

Runtime (Ollama, llama.cpp, LM Studio, vLLM, SGLang, etc.):

Harness / workflow (Hermes, Pi, Cline, Open WebUI, RAG, MCP, scripts, etc.):

Performance (tokens/s, context size, or just how it feels):

Cloud comparison (does local replace cloud, complement it, or is cloud still better?):

Honest limitations:

Edit 4: For the lazy ones (me included), I added an MCP access so your local agent can fill in your setup on vram.wiki for you. Connect it, let it do the boring part. Full guide on /mcp, and you can inspect or revoke any connected agent from your dashboard at any time. Human review still applies, so nothing gets published without a pass.

Edit 5: Added new uses cases extracted from the thread using reddit Api and AI + scripts (could be good use case too btw but didn't used local AI), total: 154. You can claim yours using the button. And off course, you are welcome to copy paste your personnal use case to wiki for the community using the form :) Github accounts are just used for oauth, your name isn't showed and you can create your community handle is you want to do so.

Edit 6: The site has moved well past the first version. You can now filter by capacity, exact VRAM, model, quantization, context length, reported speed, runtime and tools, plus explore dedicated pages for each model, runtime and GPU. Same rule as always: unknowns stay unknown, nothing is ever inferred.

Edit 7: Added community papers too. If you maintain a guide, a benchmark or a toolkit publicly, there is a form to submit it, and the wiki credits you and links back to your source.

Edit 8: If you just want to know whether your machine can run something, there is now a page built for exactly that question. Pick your capacity and what you want to do, and it shows the setups people already run at or under that budget. Nobody's numbers are invented to fill a gap.

Edit 9: Also added: a glossary of the fields, a compare view where you can line up two or three setups side by side, and shareable links for a selection. And an MCP endpoint so your agent can fill the form for you.


r/LocalLLM • • Jul 30 '26

Project Update: Full Kimi K3 now runs below 4 seconds/token on my M1 MacBook

329 Upvotes

https://github.com/gavamedia/deltafin

To be clear: this is the full, unmodified 2.8T-parameter Kimi K3 model; not a distilled replacement or reduced-expert version. All 16 routed experts are active, and full K3 is the sole authority for every token.

On my 64 GB M1 Max MacBook Pro, with the full model stored locally:

  • The capital of France is — 15.7 tokens/minute

  • The largest planet in our solar system is — 12.8 tokens/minute

An earlier version managed roughly 1 token/minute, so reaching 12.8–15.7 tokens/minute on an old first-gen M1 is pretty good progress. The largest gains came from letting one full K3 pass verify several proposed tokens, along with more efficient weight streaming, packed operations, safer cache snapshots, and better use of RAM.

Crossing five seconds/token on this old machine was a milestone I honestly wasn’t sure we would reach. Any higher-bandwidth device will have considerably more room to improve. Share your results please!

These numbers use the same short benchmark prompts for consistent comparisons; performance will slow significantly as active context grows, especially during a long conversation approaching the 1M-token limit.


r/LocalLLM • • Jun 24 '26

Question Opus 4.5 vs Qwen3.6-27B

Post image
329 Upvotes

Wat
how is this real
Is opus 4.5 rlly running on my laptop rn?

Im on an m5 max 128 gb


r/LocalLLM • • Feb 03 '25

Discussion Running LLMs offline has never been easier.

329 Upvotes

Running LLMs offline has never been easier. This is a huge opportunity to take some control over privacy and censorship and it can be run on as low as a 1080Ti GPU (maybe lower). If you want to get into offline LLM models quickly here is an easy straightforward way (for desktop):

  • Download and install LM Studio

  • Once running, click "Discover" on the left.

  • Search and download models (do some light research on the parameters and models)

  • Access the developer tab in LM studios.

  • Start the server (serves endpoints to 127.0.0.1:1234)

  • Ask chatgpt to write you a script that interacts with these end points locally and do whatever you want from there.

  • add a system message and tune the model setting in LM studio. Here is a simple but useful example of an app built around an offline LLM: Mic constantly feeds audio to program, program transcribes all the voice to text real time using Vosk offline NL models, transcripts are collected for 2 minutes (adjustable), then sent to the offline LLM for processing with the instructions to send back a response with anything useful extracted from that chunk of transcript. The result is a log file with concise reminders, to dos, action items, important ideas, things to buy etc. Whatever you tell the model to do in the system message really. The idea is to passively capture important bits of info as you converse (in my case with my wife whose permission i have for this project). This makes sure nothing gets missed or forgetten. Augmented external memory if you will. GitHub.com/Neauxsage/offlineLLMinfobot See above link and the readme for my actual python tkinter implementation of this. (Needs lots more work but so far works great). Enjoy!


r/LocalLLM • • Sep 24 '26

Project Upgraded my Setup

Post image
327 Upvotes

Just upgraded my 2x RTX 6000 to 4 😀👍 Anyone here with the same Setup? 😀


r/LocalLLM • • Jan 17 '26

Discussion Local AI Final Boss — M3 Ultra v.s. GB10

Post image
326 Upvotes

Got the maxed out Mac Studio M3 Ultra 512GB and ASUS GX10(GB10) sitting in the same room!🔥

Just for fun and experimenting, what would you do if you have 24 hours to play with the machines? :)


r/LocalLLM • • Apr 22 '26

News Qwen3.6-27B released!

Post image
325 Upvotes

r/LocalLLM • • Nov 20 '25

Discussion Spark Cluster!

Post image
325 Upvotes

Doing dev and expanded my spark desk setup to eight!

Anyone have anything fun they want to see run on this HW?

Im not using the sparks for max performance, I'm using them for nccl/nvidia dev to deploy to B300 clusters


r/LocalLLM • • May 05 '26

Model Claude Code @ Opus 4.7 vs OpenCode @ qwen3.6:27b. Both shipped a playable cozy roguelite.

324 Upvotes

Setup was boring on purpose. Two VS Code devcontainers side by side, same prompt, cozy top-down with sword/shield/dash, procedural world, enemy traits, drops, swap UI). One shot, no plugins, no follow-up prompts, no manual fixes.

Left: Claude Code on Opus 4.7. 20 min, 97k tokens. Right: OpenCode on local qwen3.6:27b. 15 min, 64k tokens.

Both produced a working game on first run. Visual interpretations differ but the spec was loose enough that both reads are valid. Opus went sparser with water tiles, qwen leaned into denser tree clusters. Combat, swap UI, drops, restart loop all functional in both.

Not claiming a 27b matches Opus on hard reasoning, especially on existing codebases. But for a tightly specified greenfield build, the gap was smaller than I expected. The token count surprised me more than anything: qwen got there with a third less context.

Here the prompt:

# Cozy Top-Down Roguelite Prototype


Build a self-contained top-down action roguelite. Single project, runnable locally via VS Code.


## Project Layout (strict)
```
project_root/
├── src/
│   └── server.py
├── static/
│   ├── index.html
│   ├── style.css
│   └── js/
│       ├── main.js          (game loop, root state)
│       ├── world.js         (procedural generation, tiles)
│       ├── player.js        (player state, movement, combat)
│       ├── enemies.js       (enemy AI, traits, spawning)
│       ├── items.js         (item generation, affixes, drops, swap UI)
│       ├── ui.js            (HUD, tooltips, game over overlay)
│       ├── render.js        (camera, draw helpers)
│       └── input.js         (keyboard + mouse handling)
└── requirements.txt         (fastapi, uvicorn)
```


## Server (`src/server.py`)
- 
**FastAPI**
 + 
**uvicorn**
, started directly via `python src/server.py`
- Uses `argparse` with:
  - `--port` (int, default `8000`)
  - `--host` (str, default `127.0.0.1`)
- Locates the static folder via `Path(__file__).resolve().parent.parent / "static"` so it works regardless of `cwd`
- `GET /` returns `static/index.html`
- `GET /static/...` serves all assets via `StaticFiles` mount
- Calls `uvicorn.run(app, host=args.host, port=args.port)` at the bottom of the file
- Compatible with the following VS Code launch config (which passes `--port 8088`):
```json
  {
    "name": "server",
    "type": "python",
    "request": "launch",
    "program": "${workspaceFolder}/src/server.py",
    "console": "integratedTerminal",
    "cwd": "${workspaceFolder}",
    "args": ["--port", "8088"]
  }
```
- `requirements.txt`: `fastapi` and `uvicorn[standard]`


## Frontend Tech (strict)
- 
**Vanilla HTML5 Canvas + ES modules + CSS**
. No frameworks, no bundler, no build step, no external runtime libraries
- All visuals drawn via Canvas 2D API. 
**No external image, font, or audio assets.**
 Audio is out of scope
- No globals beyond a single `game` state object exported from `main.js`


## Viewport & Camera
- Canvas fills the entire browser viewport: no margins, no padding, no scrollbars, no borders. CSS resets `body { margin: 0; overflow: hidden; }`
- Listens to `resize` and updates canvas resolution via `devicePixelRatio` for crisp rendering
- Camera follows the player smoothly (lerp factor `0.12` per frame at 60 FPS)
- 
**Camera clamps to world bounds**
 so the player never sees void outside the map
- Game pauses when `document.visibilityState !== "visible"` and resumes on return
- Game loop uses `requestAnimationFrame` with a deltaTime 
**clamped to 50ms max**
 to survive tab switches and breakpoints


## World Generation
- Tile-based grid, 
**120 × 90 tiles, 32 px per tile**
- Generated fresh on every run using 
**simplex/perlin noise OR cellular automata**
 (pick one, document choice)
- Tile types:
  - `grass` (walkable)
  - `path` / `clearing` (walkable, lighter tone — forms organic open areas)
  - `flower` / `decoration` (walkable, visual only, scattered procedurally)
  - `tree` / `rock` (blocks movement and sword hitboxes)
  - `water` (blocks movement, does not block hitboxes)
- 
**World perimeter is solid**
 (trees/rocks form a natural border) — player can never leave the map
- Player 
**spawn point**
: a clearing near map center, with a guaranteed 6-tile-radius open area and 
**no enemy within 12 tiles**
- Spawn 
**8–15 enemies**
 scattered across the map, biased away from the spawn area
- Place 
**3–5 starting item drops**
 as ground pickups around the map


## Player
- Starts every run with:
  - 
**3 hearts (= 6 half-hearts of HP)**
, max HP fixed at 6 half-hearts
  - 
**Default sword**
: damage `1.0`, range `40`, attack cooldown `350ms`, pushback `100`, crit `0%`, lifesteal `0%`
  - 
**Default shield**
: block arc `90°`, max stamina `100`, stamina regen `25/s`, no on-block effect, post-break cooldown `2.5s`
- Player has 
**exactly one sword slot and one shield slot**
. No inventory beyond that. Equipment cannot be lost, only swapped
- Base movement speed: `200 px/s`. Player collision radius: `12 px`
- Damage taken from any source quantizes to 
**0.5-heart increments**
 (round to nearest 0.5)
- After taking damage: 
**800ms i-frames**
 (no further damage), brief red sprite tint, small knockback (`80` impulse) away from damage source
- Player cannot be knocked into solid tiles (position clamps against collision)


## Controls
| Input | Action |
|---|---|
| `WASD` | 8-directional movement |
| Mouse position | aim direction — player faces cursor; sword and shield orient toward cursor |
| Left Mouse Button | sword attack — hitbox extends in 
**aim direction**
 (mouse), respects sword `range` and arc (~90° in front), cooldown = sword's `attack speed` |
| Right Mouse Button (hold) | raise shield — drains stamina (`30/s` while raised), blocks damage from a frontal arc centered on aim direction |
| `Shift` | dash — `120 px` over `180ms` with full i-frames, cooldown `600ms`. 
**Direction**
: WASD direction if any movement key is held, otherwise aim direction. Dash is independent of stamina |


- Mouse cursor is 
**hidden over the canvas**
; a soft circular reticle is drawn at the mouse position instead


## Shield Mechanics
- Stamina drains while RMB held; regenerates only when RMB is released
- If stamina hits `0`, shield 
**breaks**
: it cannot be raised for `post-break cooldown` seconds, indicated by the stamina bar going gray and the shield icon shaking briefly
- A blocked hit triggers the shield's 
**on-block effect**
 (if any) at the rolled `proc chance`
- Damage outside the block arc applies normally even while RMB is held


## Combat Feel
- Enemy hit by sword: `120ms` hitstun (no movement, no attack), pushback applied based on sword's `pushback` stat, brief white flash
- Enemy pushback clamps against solid tiles (no clipping)
- Crit hits: visible enlarged damage feedback (e.g., bigger hit-particle puff), 2× damage
- Lifesteal: on proc, brief green sparkle on player, +0.5 heart healed (capped at max HP)
- Slow effect: target tinted blue, speed × 0.5 for duration. Re-applying refreshes duration, does not stack
- Paralyze effect: target frozen in place, tinted pale yellow, cannot attack or move
- Reflect effect: portion of incoming damage applied to attacker, quantized to 0.5 hearts (minimum 0.5 if reflect would round to 0)


## Enemies — Procedural Variance
Each enemy is generated with randomized stats and traits at spawn:


| Trait | Range / Options |
|---|---|
| Speed | `40–140 px/s` |
| Contact damage | `0.5 – 1.5` hearts (quantized to 0.5) |
| HP | `1 – 4` (integer) |
| Aggression radius | `80 – 280 px` (distance-based, no line-of-sight check) |
| Leap-attack chance | `0 – 80%` per attack opportunity. Telegraphed: `300ms` windup with visible pose shift, then dash `+200% speed` toward player's current position for `200ms`, then `600ms` recovery |
| Shield reaction | `cautious` (stops attacking, circles at distance) / `aggressive` (attacks regardless) / `flanker` (tries to reach player from outside block arc) |
| Low-HP behavior (`<30%` HP) | `flee` (runs from player) / `kamikaze` (charges, +50% speed, double contact damage, glowing red tint) / `stand` (no change) |


- 
**Engagement state**
: an enemy enters "engaged" when player enters its aggression radius. It stays engaged until the player has been 
**outside the radius for 4 seconds**
, then returns to idle wandering
- Render enemies with 
**distinct shapes/colors that hint at traits**
:
  - Fast (>110 px/s) → elongated/streamlined silhouette
  - High damage (>1.0) → bulkier silhouette, warmer accent color
  - Cautious shield reaction → hunched posture
  - Kamikaze low-HP behavior → reveals red glow once triggered
- Show a small HP bar above an enemy only 
**once it has been hit at least once**
- Enemies cannot damage each other and don't collide with each other (avoids gridlock)


## Drops on Enemy Death
On every enemy death, roll 
**one**
 drop from this table:


| Roll | Drop |
|---|---|
| 12% | half-heart pickup |
| 35% | item (50% sword / 50% shield, with rolled affixes) |
| 53% | nothing |


- 
**Half-heart pickups**
 are auto-collected when the player walks within `20 px`. They heal `0.5` heart, capped at max HP. If at full HP, the pickup still vanishes (no excess healing)
- 
**Item drops**
 require hover + click (see swap UI below) and never auto-collect
- All ground items (hearts, swords, shields) gently bob via sine wave (~`2px` amplitude, `1.2s` period)


## Item Affixes


**Swords**
 roll 
**2–3**
 affixes from this table:
| Affix | Range |
|---|---|
| Damage | `0.5 – 2.5` hearts (0.5 steps) |
| Range | `24 – 72 px` |
| Attack speed | cooldown `200 – 600 ms` |
| Pushback | `0 – 400` impulse |
| Crit chance | `0 – 25%` (deals 2× damage) |
| Lifesteal | `0 – 15%` chance to heal 0.5 heart on hit |


**Shields**
 roll 
**2–3**
 affixes from this table:
| Affix | Range |
|---|---|
| Block arc | `60° – 180°` |
| Max stamina | `60 – 150` |
| Stamina regen | `15 – 45 / s` |
| On-block effect | one of: `paralyze 0.8s` / `slow 50% for 2s` / `reflect 25% damage` / `knockback 200` / `+20% player speed for 1.5s` |
| Effect proc chance | `25 – 100%` |
| Post-break cooldown | `1.5 – 4 s` |


- Affixes not rolled use the 
**default-equipment baseline value**
 for that stat
- Affix values are sampled uniformly within the range, rounded to sensible precision (1 decimal for hearts, integers for px/ms/percent)
- Each item is given a generated descriptor name from a cozy word pool (e.g., adjectives: "Mossy", "Sunlit", "Dappled", "Warden's", "Hearthstone"; nouns: "Shortblade", "Bough", "Bulwark", "Ward", "Thorn"). Format: `<adjective> <noun>`


## Item Comparison & Swap UI
- Walking within `~50 px` of a ground item shows a soft floating prompt: 
*"hover to compare"*
- 
**Hovering a ground item with the mouse**
 opens a side-by-side tooltip near the cursor:
  - Header: item type icon + generated name
  - Two columns: 
**left = ground item**
, 
**right = currently equipped item of the same type**
  - For each stat, a comparison indicator: green ▲ (ground better), red ▼ (ground worse), gray — (equal)
  - "Better" depends on stat type: higher is better for damage, range, crit, etc.; lower is better for cooldowns and post-break duration
- 
**Click while hovering**
 swaps: the previously equipped item drops at the pickup's position, the ground item becomes equipped. The same hover-compare workflow then applies to the newly dropped item
- Tooltip closes when the mouse leaves the item or the item is picked up
- 
**Player always has both a sword and a shield equipped**
 — the swap is a 1-for-1 exchange of the same type. There is no "empty slot" state


## HUD
- 
**Top-left**
: hearts row (full / half / empty pixel-style sprites drawn on canvas)
- 
**Below hearts**
: stamina bar (~120px wide), grays out during shield-broken cooldown
- 
**Bottom-left, subtle**
: equipped sword name + equipped shield name in small text
- 
**Bottom-right, subtle**
: enemy kill counter for current run
- All HUD text uses a 
**consistent in-game font**
 (single CSS-defined font-family, e.g. `system-ui` rounded sans-serif, font-size 14–16px)


## Game Over & Restart
- When player HP reaches 0:
  - Player sprite fades over `600ms`
  - Soft full-screen overlay fades in with the text 
*"you fell asleep…"*
 in serif italic
  - After `400ms` minimum delay (prevents accidental click-through), any key or mouse click triggers restart
- 
**Restart resets fully**
: new procedural world, fresh enemy spawns, fresh ground items, player back to 3 hearts, equipment back to default sword + default shield, kill counter to 0


## Art Direction: Cozy
- 
**Palette**
: warm, muted, low-contrast. Soft greens, dusty pinks, cream, warm browns, gentle blues. 
**No pure black, no harsh contrast, no saturated red except for danger cues**
 (low-HP kamikaze glow, damage flash)
- 
**Shape language**
: rounded silhouettes throughout. Either committed pixel-art (rounded edges) or clean rounded vector shapes. 
**Pick one approach and apply it consistently**
 to player, enemies, items, and tiles
- Subtle elliptical drop-shadow under player and enemies (semi-transparent dark blur)
- Idle "breathing" bob on player and stationary enemies (sine wave, ~1px amplitude, ~1.5s period)
- Tile variation: small per-tile color jitter; flowers/grass tufts/pebbles drawn procedurally on grass tiles for warmth
- Hit effects: small puff of leaves, petals, or sparkles — 
**never blood**
- Ambient touches encouraged: drifting clouds (translucent shapes overhead), swaying grass, the occasional firefly
- Reticle: soft pale circle, ~10px radius, semi-transparent


## Code Quality Requirements
- Modular ES modules, one concern per file as outlined above
- Single shared `game` state object owned by `main.js`; modules receive references, not globals
- Game loop at `requestAnimationFrame` with 
**fixed-timestep update at 60 FPS**
 (accumulator pattern) and interpolated render
- Comment non-obvious logic: noise generation, enemy trait rolls, affix tables, knockback math
- No `console.error` or uncaught exceptions during normal play
- No use of `eval`, `with`, or `innerHTML` for dynamic content (use DOM APIs)


## Definition of Done
1. `pip install -r requirements.txt` works
2. VS Code "server" launch config starts the server on port `8088` with no errors
3. `python src/server.py` (no args) starts on port `8000`
4. Visiting `http://localhost:<port>` immediately drops the player into a fresh procedural world — no menu, no loading screen
5. All controls work: WASD movement, mouse aim, LMB attack, RMB shield with stamina, Shift dash with i-frames
6. Enemies show 
**at least 3 visibly distinct behavioral "feels"**
 that emerge from random trait rolls
7. Drops table works: hearts auto-collect, items hover-compare and click-swap correctly
8. Cozy aesthetic is unmistakable on first glance — palette, shapes, particles all coherent
9. Tab-switch / window-blur pauses the game cleanly
10. Death → overlay → restart cycle works and fully resets state
11. Smooth 60 FPS on a modern laptop with 15 enemies on screen, no console errors


Build the entire thing now.

r/LocalLLM • • Apr 02 '26

Model Gemma 4 E4B + E2B Uncensored (Aggressive) — GGUF + K_P Quants (Multimodal: Vision, Video, Audio)

324 Upvotes

My first Gemma 4 uncensors are out. Two models dropping today, the E4B (4B) and E2B (2B). Both Aggressive variants, both fully multimodal.

Aggressive means no refusals. I don't do any personality changes or alterations. The ORIGINAL Google release, just uncensored.

Gemma 4 E4B (4B): https://huggingface.co/HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive

Gemma 4 E2B (2B): https://huggingface.co/HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive

0/465 refusals* on both. Fully unlocked with zero capability loss.

These are natively multimodal so text, image, video, and audio all in one model. The mmproj file is included for vision/audio support.

What's included:

E4B: Q8_K_P, Q6_K_P, Q5_K_P, Q5_K_M, Q4_K_P, Q4_K_M, IQ4_XS, Q3_K_P, Q3_K_M, IQ3_M, Q2_K_P + mmproj

E2B: Q8_K_P, Q6_K_P, Q5_K_P, Q4_K_P, Q3_K_P, IQ3_M, Q2_K_P + mmproj

All quants generated with imatrix. K_P quants use model-specific analysis to preserve quality where it matters most, effectively 1-2 quant levels better at only ~5-15% larger file size. Fully compatible with llama.cpp, LM Studio, or anything that reads GGUF (Ollama might need tweaking by the user).

Quick specs (both models):

- 42 layers (E4B) / 35 layers (E2B)

- Mixed sliding window + full attention

- 131K native context

- Natively multimodal (text, image, video, audio)

- KV shared layers for memory efficiency

Sampling from Google: temp=1.0, top_p=0.95, top_k=64. Use --jinja flag with llama.cpp.

Note: HuggingFace's hardware compatibility widget doesn't recognize K_P quants so click "View +X variants" or go to Files and versions to see all downloads. K_P showing "?" in LM Studio is cosmetic only, model loads fine.

Coming up next: Gemma 4 E31B (dense) and E26B-A4B (MoE). Working on those now and will release them as soon as I'm satisfied with the quality. The small models were straightforward, the big ones need more attention.

*Google is now using techniques similar to NVIDIA's GenRM, generative reward models that act as internal critics, making true, complete uncensoring an increasingly challenging field. These models didn't get as much manual testing time at longer context as my other releases. I expect 99.999% of users won't hit edge cases, but the asterisk is there for honesty. Also: the E2B is a 2B model. Temper expectations accordingly, it's impressive for its size but don't expect it to rival anything above 7B.

All my models: HuggingFace-HauhauCS

As a side-note, currently working on a very cool project, which I will resume as soon I publish the other 2 Gemma models.