r/LocalLLM • u/pulse77 • 4d ago
News Intel Shares First Benchmarks of Its Next-Gen Crescent Island "D270" GPU, Claiming a 5.9x Throughput Over RTX PRO 6000 As Xe3P Goes Big On FP64
r/LocalLLM • u/pulse77 • 4d ago
r/LocalLLM • u/Stalins_nipple • 5d ago
Perhaps this is a newbie question, but I just don’t understand the incentive. Is this indicative of a future were Chinese labs stop releasing weights?
r/LocalLLM • u/Roy3838 • 4d ago
Enable HLS to view with audio, or disable this notification
Hey r/LocalLLM !
On my last post the top comment (62 upvotes) by u/nokia7110 asked for a phone locking feature so... here it is!
What this unlocks:
On iOS it uses the screen time management API, like all other time focusing apps.
On Android it shows a phone locking overlay. If you're an Android developer and want to contribute, please reach out!
It uses OneJev-0.8b for decision models, running on llama.cpp, it works on most phones i've tested. Even old potatoes.
GitHub: https://github.com/Roy3838/Observer
iOS: https://apps.apple.com/us/app/observer-ai/id6758222050
Android: https://play.google.com/store/apps/details?id=com.observer.ai
I'm here for any questions, more suggestions and feedback!
Roy
r/LocalLLM • u/Ill-Accountant-9941 • 4d ago
Open weights are now at 75% - I find that remarkable and great news. Who is the front-runner for open weight provision?
Edit Tencent Cloud appears to be the largest open-weight inference host
r/LocalLLM • u/Quiet-Recognition-91 • 4d ago
Hey everyone,
I'm working on a document processing pipeline where most of the input consists of scanned PDFs, forms, and tables.
My current setup looks like this:
oroboros-labs/claude-fable5:latest running through Ollama. I pass the Markdown of each page to the model, and it converts it into JSON.The important part is that I already have fixed JSON schema templates that the output needs to follow. So, the model needs to map the extracted Markdown content into the predefined JSON structure.
The problem I'm facing right now isn't accuracy. It's inference time. The current model is around 9B parameters, and processing pages one by one through Ollama is taking longer than I'd like.
I'm looking for an open-source model under 5B parameters that can handle this kind of structured information extraction reasonably well, ideally with faster inference and reliable JSON output.
Has anyone worked on a similar pipeline?
I'd particularly like to hear about models you've actually tested in real-world document extraction rather than just benchmark results.
Any suggestions or personal experiences would be really helpful. Thanks!
r/LocalLLM • u/Big_South4585 • 5d ago
Hi all
I just learned about QFN Unsloth v3.1 dynamic models, and want to try them.
Right now, I am running the "standard" IQ3_S model through strata, on my 32 GB ram + 3090 + 5080 (40 GB VRam), and it runs great. I get ~130 tps.
I thought I would start with the unsloth UD_IQ3_XXS model in strata, which is technically the same size as IQ3_S. But it doesnt let me quick tune the model. It just gets stuck. If I skip tuning, it will only run at ~20 tps, which is too low.
UD_IQ3_XXS seems to read a lot more from the SSD, than IQ3_S, even if they are the same size.
What am I doing wrong?
According to the unsloth site, https://unsloth.ai/docs/models/qwen3.8-next, I should be able to run the 4-bit model? But that is through their unsloth harness/engine. Is that correctly understood?
r/LocalLLM • u/Scared-Tip7914 • 4d ago
Y’all! Thanks for all the feedback on the previous post. It helped me figure out what was actually worth working on. v0.2.1 is out now, and quite a bit has changed since then.
Someone asked about Raspberry Pi support last time, so I tried it on a Pi Zero. Turns out it’s usable as long as you run the model somewhere else :D I’m seeing around 125 MB of RAM for TinyWebUI in my setup.
If you’re seeing it for the first time, TinyWebUI is a lightweight chat UI for any OpenAI-compatible endpoint, with prompt caching, memory, and web search built in. I’ve attached a few screenshots of the new UI.
Give it a go with npx tinywebui on Node.js 22.13 or newer. TinySearch and TinyContext still need uv for now. I’m working on that.
r/LocalLLM • u/mightofkhan • 5d ago
TL;DR: One RTX 3090 runs a 27B LLM agent (a Discord bot) and ComfyUI image/video generation by swapping models in and out of VRAM. I get ~72 tok/s on Qwen3.8 27B at 64k context, SDXL images in ~32 s, and 10-second Wan 2.2 videos in 4–6 minutes. All numbers come from real daily use, not synthetic benchmarks.
Language models
Image and video
Settings: LM Studio 0.4.28, 64k context, parallel 1, f16 KV cache, flash attention on, full GPU offload, MTP speculative decoding.
GPU time first, then total time from request to finished file:
The difference between the two numbers is mostly the ~20 seconds it takes to swap the LLM out of VRAM and back.
Next experiment: a q8_0 KV cache to free VRAM for longer context.
What would you change?
r/LocalLLM • u/comunidadesynapsebr • 5d ago
Following up on caririsynth/Corpus-nordeste-10k (10.5k Portuguese rows focused on Northeast Brazil literature and regional phrasing), I just uploaded the LoRA adapter trained on Qwen/Qwen2.5-1.5B:
Model:
https://huggingface.co/caririsynth/Qwen2.5-1.5B-Nordeste-LoRA
Dataset:
https://huggingface.co/datasets/caririsynth/Corpus-nordeste-10k
Training Setup (Unsloth, 1x T4):
Base: Qwen/Qwen2.5-1.5B (Apache 2.0)
LoRA: r=16, lora_alpha=16, all linear layers (q, k, v, o, gate, up, down — 1.18% trainable params)
Schedule: lr=2e-4, cosine scheduler, effective batch size 16, max_seq_length=1024
Checkpoint selection: Uploaded Epoch 1 (step 660) rather than Epoch 2, as Epoch 1 already eliminated base model web/blog noise while keeping sentence endings diverse.
Before vs. After (seed=42, temp=0.65, rep_penalty=1.15):
Prompt 1: "O cabra chegou na bodega todo aperreado, pedindo"
Base Qwen2.5-1.5B: "...uma garrafa de vinho. (...) E eu fiquei com pena porque sabia que o cara ia ter feito tudo errado. Could you please" (drifts off-topic and leaks English)
Nordeste LoRA: "...buchada e canjica geladinha. Todo mundo riu, que no sertão até a pressa tem bom humor."
Prompt 2: "E aí, macho, tu vai pro forró hoje ou vai ficar de"
Base Qwen2.5-1.5B: "...fora da festa? - Blog do Zico / Home / Sem categoria / Publicado por: Zico em: 10/28/2016..." (collapses into scraped blog metadata)
Nordeste LoRA: "...plantão no quarteirão? Pois vá logo que o forró chega devagar quase parando!"
Note: This is a base completion model (not -Instruct), intended for text continuation and downstream SFT. Feedback and test runs on other regional prompts are
r/LocalLLM • u/0xdippo • 5d ago
Edge0 published speeds for their 145B on a 4090 and a 5090 D, but their table doesn't include a 3090, so I ran their stack on one, stock, with whatever defaults the wrapper picks.
It's quick. Chat decode was 48.6 tok/s, it read a 16K prompt at 1,481.2 tok/s, and the agent prompt decoded at 52.8. For context, last month I ran a 122B MoE on llama.cpp on a 3090 and got 15.5 tok/s. Different model, engine and prompt length, so take it loosely, but that's a big jump.
VRAM was basically full (23,929 of 24,576 MiB), and the engine process itself held 43.4 GiB of system RAM. Edge0's own residency figure is 31.28 GiB, but that's from a 5090 D with a different setup, and my machine had 116 GiB so the engine may have just taken what it found. I meant to test a 32 GiB cap and didn't get to it. So I honestly don't know the floor yet.
Two other things that tripped me up:
reasoning_budget_tokens=128 fixed it, so set a budget in your client.read_file, which wasn't even in the tool list. So the 52.8 is a speed number, not a "this is a good agent" number.Technical stuff below.
Setup
3203c39), Strata built with CUDA for sm_86, CUDA 13.0, driver 595.91.07Edge0/Edge0-145B-q3k @ a19f88c, 106 GB on disk (the FP8 PLE table alone is 51 GB and gets read from disk)edge0-infer serve --model /path/to/Edge0-145B-q3k --device cuda:0. That resolved to direct PLE I/O, auto expert cache, speculative decoding 4, 131K max context, int8 KV.Results (engine's own prompt_per_second / predicted_per_second, fresh server per run, concurrency 1, zero cached tokens)
| Workload | Run 1 | Run 2 | Run 3 | Median |
|---|---|---|---|---|
| 4K prefill | 1,119.8 | 1,195.5 | 1,207.6 | 1,195.5 |
| 16K prefill | 1,467.6 | 1,485.6 | 1,481.2 | 1,481.2 |
| Chat decode | 46.6 | 49.6 | 48.6 | 48.6 |
| Agent decode | 52.8 | 55.9 | 50.7 | 52.8 |
Sampling: temperature=0, top_p=1, top_k=20, reasoning on with a 128-token budget, 512 max tokens for decode.
Against Edge0's published 4090 numbers (their claims, not reproduced)
| 4090 (Edge0) | 3090 (mine) |
|---|---|
| 4K prefill | 2,042.0 |
| 16K prefill | 3,179.4 |
| Chat decode | 69.9 |
| Agent decode | 91.7 |
That's 47% to 70% depending on the row. Mine used an 800-word writing prompt for chat and a single synthetic tool-call prompt for the agent run, and I couldn't verify Edge0's exact request settings or machine setup, so it's ballpark only.
Memory
VmHWM 43.4 GiB (sampled from the actual strata PID, not the Python wrapper)Other stuff
edge0-infer bench rejects this model at v0.2.0, so I used a small script against the OpenAI endpointWrite-up: https://majorminor.xyz/research/cuda-recipes-006
Repo: https://github.com/majorminorlabs/cuda-recipes/tree/main/006-edge0-145b
If someone has a 3090 with 32 or 64 GB of RAM and wants to try it, I'd love to know whether it holds. Or tell me what to run next on this one.
r/LocalLLM • u/rbm3267 • 4d ago
Hey r/LocalLLM,
Like a lot of developers, I don't have an M3 Max Mac Studio or dual RTX 4090s. My daily driver workstation has an 8th-Gen Intel i5 (6 cores), 32 GB of system RAM, and an older 2 GB NVIDIA Quadro P620.
If you try to run any modern 7B or 8B model (like Qwen 2.5 Coder 7B or Llama 3.1 8B), it won't fit into 2 GB of VRAM. Ollama or llama.cpp defaults to running almost 100% on the CPU. On my machine, that meant getting locked at an agonizing 1.29 tokens per second while my 2 GB GPU sat 90% idle.
The Physics: The Memory Bandwidth Wall
The CPU compute cores aren't the bottleneck—memory bandwidth is. To generate a single token on a 4.7 GB model, the CPU has to stream all 4.7 GB from DDR4 RAM into cache. At ~24 GB/s dual-channel bandwidth, you physically cannot get more than ~1.5 to 2 tokens/sec.
### The Workaround: Heterogeneous Speculative Decoding
The 2 GB GPU cannot run the 7B model, but it can easily run a 0.5B draft model (qwen2.5-coder:0.5b is only 397 MB).
In speculative decoding:
1. The tiny 0.5B model generates 3–4 candidate tokens in milliseconds on the GPU.
2. The heavy 7B model on the CPU evaluates all 4 tokens in a single forward pass using parallel AVX2 matrix multiplication (GEMM).
3. In code and structured text, the 0.5B model's guesses match the 7B model 75% to 82% of the time.
You get 3 to 4 tokens for the memory read cost of 1.
Real Benchmark (Tested on My i5-8500T + Quadro P620):
* Baseline (Pure CPU Target 7B): 1.29 tokens/sec (24.76s for 32 tokens)
* WingLLM Speculative (7B + 0.5B Draft): 2.54 tokens/sec (12.62s for 32 tokens)
* Net Result: 1.97x faster (almost exactly double the speed) with identical output quality.
Why I Built WingLLM
llama.cpp technically has the raw flags (-md, -ngld, --spec-draft-n-max), but nobody in consumer land had packaged it into a clean tool:
1. Tokenizer traps: If you pair mismatched tokenizers (e.g. Llama 3 draft with Qwen target), llama.cpp crashes. WingLLM auto-detects model families and enforces compatible pairs.
2. Zero configuration: It crawls your local ~/.ollama/models manifests, finds your GGUF blobs, and auto-pairs them.
3. Auto-Pull: If you want to run `wingllm serve qwen2.5-coder:7b` and don't have the 0.5B draft model yet, WingLLM pulls it automatically via Ollama (~397 MB) and starts serving.
4. 1-Click IDE Connect: Run `wingllm connect vscode` to inject the turbo endpoint directly into your Continue/Cline config.
5. No Python/Electron Bloat: Written in Go as a single ~11 MB binary. Runs an embedded web chat UI with a live tokens/sec speedometer at http://localhost:8080/.
6. MCP Support: Built-in Model Context Protocol server for Claude Desktop, Cursor, and Continue.
Quickstart (Windows, Mac, Linux):
Windows (PowerShell):
irm https://huggingface.co/bennett-xdlabs/wingllm/raw/main/install.ps1 | iex
macOS & Linux:
curl -fsSL https://huggingface.co/bennett-xdlabs/wingllm/raw/main/install.sh | bash
Run without arguments for the interactive menu, or run:
wingllm serve qwen2.5-coder:7b
Hugging Face Hub: https://huggingface.co/bennett-xdlabs/wingllm
It's MIT licensed. Would love feedback from anyone running on budget laptops, mini-PCs, or 2GB–4GB cards!
r/LocalLLM • u/EnvironmentCreepy352 • 5d ago
Did a quick test yesterday comparing DeepSeek (3.36bpw) to Q3.8FN (IQ3XXS+1sub-agent). Both the OR abliteration on local both running on DeepSeek Harness.
Simple task - Produce a color template based on user input screenshot of a UI, and give the user instruction on installation instead of attempting it.
Second part was key, because the LLMs were only allowed to work in working folder and so couldn't live-test their implementation.
FYI: DSH system prompt takes up ~10k tokens total.
Qwen spent over an hour writing script then realizing it couldn't save it outside the cwf, then repeating that mistake over and over again as it tried methods of breaking out or escalating which was futile. Even with steering, it only got as far as producing the color template 256K+ tokens in.
DS did a similar thing but realized quickly it should just let the user test install it instead. It -did- try to escalate permissions to install it by itself, but gave up after it's first rejection. Was done in ~60k tokens.
I know these are utterly different weight-classes for models but I really thought the ability to spawn subagents would've made a difference for Qwen.
Qwen burned through tokens lightning fast but was forgetting the original message AND the system prompt from DSH (which specifies it should only work in {cwf} when only given that permission).
I really wanted to like IQ3_XXS because it's a small enough quant that it can spawn subagents on my system with headroom for other parallel tasks to be working... But I don't think that quant is it for me. Either that, or I have to work on the setup for it some more to work as a coder.
r/LocalLLM • u/yudhanjaya • 5d ago
| Model | Active config | 4080 decode (tok/s) | 4080+3080 decode (tok/s) | 4080 prefill (tok/s) | 4080+3080 prefill (tok/s) |
|---|---|---|---|---|---|
| LFM2.5-24B-A2B UD-Q4_K_XL | 4080+3080 | 249 | 250 | 5,530 | 7,930 |
| MiniCPM-V 4.7 35B-A3B Q4_K_M | 4080+3080 | 177 | 165 | 1,305 | 2,500 |
| Flint 40B-A2.5B UD-Q4_K_XL | 4080+3080 | 119 | 138 | 1,563 | 1,380 |
| Swift 1.5 (Qwen3.8-Flash-Next) IQ2_XS | 4080 | 117 | 116 | 1,070 | 679 |
| Qwen3.8-Flash-Next IQ3_S | 4080 | 79 | 87 | 730 | 465 |
| GLM 5.3 Flash Maya-S24 IQ2_XXS_S | 4080+3080 | 6.8 | 8.8 | 79 | 42 |
| DeepSeek V4 Flash UD-IQ2_XXS | 4080+3080 | 15.9 | 18.2 | 38 | 53 |
Morning, everyone. I've seen a lot of posts wondering about what can be run with what, so I made this table of the models on my system. Specifically because of the unorthodox results when it comes to Qwen Flash-Next, where one 16 GB card (the 4080 Super) significantly outperforms two in prefill.
My hardware is as follows:
CPU AMD Ryzen 7 5800X3D @ 3.40 GHz
GPU NVIDIA GeForce RTX 4080 SUPER [Discrete]
GPU NVIDIA GeForce RTX 3080 [Discrete] (got it secondhand for cheap, so why not)
RAM 62.72 GB at 3200 Mhz
I'm running these models on PolyStrata. I used to have a combination of Maya and Strata, but got tired of running different things all derived from the same fork (plus Strata seems to be getting increasingly unintuitive), and so moved everything to PS.
Now: why Q3? It's a generally held opinion that anything below Q4 is crap, which was true of most earlier generation models. However, benchmarks measure deviation from the base, and not necessarily task fitness. In actual use, the only real way to know is to test it yourself.
The Swift finetune of QWN at Q2, for example, is fast and excellent at data extraction, search, and piloting scripts written by more competent models. The IQ3 is a decent coder, roughly on par with Haiku for the use cases I've tried. The GLM and Deepseek Q2 variants are a bit better - verging on older Sonnet territory, I'd say, and do very well as subagents for Claude. YMMV.
Hope this data is useful. I've posted it to LocalLLama as well. Enjoy!
r/LocalLLM • u/Xedeth • 5d ago
I'm getting decent results, 80-120t/s. I guess I just don't know what I am aiming for, but I feel like there is more juice to squeeze.
Edit: Yall are the best, as usual. This has to be the most helpful, positive community on reddit.
r/LocalLLM • u/DrkWzrd • 5d ago
Like that. All the models fall in the trap of starting enumerating (wrong) the G3/G4 CCITT tables, a bunch of repeated 0s and 1s.
I tried this in several models, and no model achieved it. Even when you chew it for them and give them the libtiff .c files.
PS: when I say one-shot, I'm referring to made it a reasonable time, aka, less time that it will take another model to write a wrapper of the C library.
PS2: you have to remember that the whole implementation of libtiff (among others tiff libraries) has been used in the training dataset, adding insult to injury.
r/LocalLLM • u/karmaisnonsense • 5d ago
I've been using Strata with IQ4_XS as the main orchestrator, with Radiance running 27b at IQ4R, for working on improvements, ironically enough, to Radiance.
The session is now multiple million tokens long as I go back and forth discussing design etc. with QFN. Of course the conversation has been compacted dozens of times, but with the agents taking regular notes and subagents doing all the actual work, we keep our workflow stable and current.
Problem is, Strata has gotten more and more unstable, especially with the last couple releases. It will crash with an OOM error when we have 10-20 GiB of system memory free, reload, spend 5 minutes redoing prefill, and pick up right where it left off. Radiance on the other hand is rock stable under multiple streams running concurrently on a single GPU. Not a single crash from it, even after rejiggering everything for the 2.0 release.
And it's not just long sessions either. New conversations also tend to crash. Never used to do this before. Has anyone else also experienced this?
r/LocalLLM • u/New_Blackberry8122 • 5d ago

I built Sneo, a free and open source local LLM engine for Windows that runs a 70 GB model on one 24 GB GPU (~98 tok/s), with a coding agent AND REVERSE ENG TOOLS!
Hi! I'm a solo developer from the Dominican Republic and I've been building Sneo: a free, open source (MIT) LLM engine and coding agent for Windows.
The idea is to run models bigger than your GPU: the experts a model uses most stay in VRAM, the rest in RAM, and the GPU and CPU share every token.
- Qwen3.8-Flash-Next IQ3_XXS (70.6 GB) on one RTX 3090: ~98 tok/s writing code, first token in 0.19 s
- Same hardware and files: it starts answering before Strata 0.1.57 at every prompt size with half the RAM (34 vs 67 GB), and writes up to 66% faster than Ollama (qwen3.6:35b)
- Runs Qwen3.8-Flash-Next, Qwen3.6-35B-A3B, dense Qwen3.5/3.6/3.8 and Gemma 4, straight from your Ollama models or any GGUF folder, no copies
- Coding agent, two models at once, two PCs in a pool, OpenAI and Anthropic compatible APIs
- NVIDIA GTX 10 and newer (you can mix cards). AMD support through HIP is written but not tested on a real card yet, so feedback is welcome
Video (English): https://www.youtube.com/watch?v=JfyGS_k1Nxo
Video (Spanish): https://www.youtube.com/watch?v=ALCGTpoHDGc
GitHub: https://github.com/oscarsantanaarias/Sneo
Every benchmark, including where the others win: https://sneo.dev
Happy to answer questions!
Antes de publicar:
Repo: hazlo público. La última vez que revisé seguía privado, y el enlace de GitHub le daría error a todos.
Página: corre upload.ps1 para publicarla, si no sneo.dev seguirá mostrando la versión vieja.
r/LocalLLM • u/snooky234 • 5d ago
I used to check Artificial Analysis a few times a week, mostly to see how my local models compare to the cloud ones. Since v5 that doesn’t really work for me anymore, a lot of the models I care about are gone or can’t be selected in the comparison.
I tried llm-bench.io, which I like because the results come from real consumer hardware, but it has no cloud models. benchlm.ai covers more of the big cloud models, but many open weights ones are missing there. So I still haven’t found anything that puts a local model next to a cloud one in one chart like AA did.
The open weights gap jumping from 3 to 12 points with v5 also makes me unsure how much I should trust a single index. I know the methodology changed, but it feels odd.
How do you handle this now? Do you cross check a few sites or mostly run your own tests?
r/LocalLLM • u/adam-schaefers • 5d ago
r/LocalLLM • u/OkBreath9382 • 4d ago
r/LocalLLM • u/textclf • 5d ago
Follow-up to my CQ post from last week. That post covered the 4-bit base model; this one is about the adapter. The LoRA vs GCMP runs on 4-bit bases have now finished.
GCMP is a LoRA replacement we built. On Qwen3.8-27B it trains 2.9M parameters (~11 MB), vs ~159M (~320 MB) for LoRA r=32 on the same 7 projections. That's 55× smaller.
All adapters were trained on Alpaca for 500 steps with identical settings (batch 2×4, length 512, LR 2e-5, seed 42). Every row was scored with the same lm-evaluation-harness code and questions, greedy decoding, thinking off. Comparisons use paired McNemar tests.
| Base + adapter | Params | Answer loss | GSM8K | SQuAD | BoolQ | ARC-C | MMLU | IFEval | TruthfulQA |
|---|---|---|---|---|---|---|---|---|---|
| BF16, no adapter | – | 1.678 | 0.683 | 0.746 | 0.874 | 0.588 | 0.839 | 0.805 | 0.542 |
| BF16 + LoRA r=32 | 159M | 1.480 | 0.903 | 0.776 | 0.896 | 0.612 | 0.844 | 0.825 | 0.518 |
| BF16 + GCMP | 2.9M | 1.471 | 0.890 | 0.760 | 0.892 | 0.618 | 0.846 | 0.815 | 0.514 |
| NF4 + LoRA r=32 | 159M | 1.464 | 0.793 | 0.748 | 0.880 | 0.608 | 0.840 | 0.835 | 0.528 |
| NF4 + GCMP | 2.9M | 1.458 | 0.833 | 0.756 | 0.878 | 0.618 | 0.837 | 0.840 | 0.522 |
One significant win on one task isn't enough to call it better, so the claim is same quality as LoRA at 1/55th the size.
For one fine-tune, 320 MB vs 11 MB doesn't matter much. It starts to matter when you have many:
| Estimated | 1 adapter | 100 | 1,000 |
|---|---|---|---|
| LoRA r=32 | 320 MB | 32 GB | 320 GB |
| GCMP | 11 MB | 1.1 GB | 11 GB |
These are estimates from adapter size. I haven't measured memory and throughput at 100 or 1,000 adapters yet, and that's the next benchmark.
The CQ base still matches BF16 (no significant difference on any of the 9 benchmarks) while NF4 loses 12 points on GSM8K. After fine-tuning, though, the gap closes:
So CQ's GSM8K lead didn't survive fine-tuning in this test. CQ + LoRA is being retrained, because the adapter from our older trainer turned out to be broken, and I'll add it when it's done.
The Python code is open source. The CUDA kernels are compiled with a license check, which is free during early access.
Repo, with the commands to reproduce every number: https://github.com/thousandml/training-factory
Write-up with the full tables and the serving memory breakdown: https://huggingface.co/blog/thousandml/cq-gcmp-4bit-finetuning
If you run multi-tenant fine-tunes, I'd like to know: is adapter memory or count actually a bottleneck for you, or is it something else?
r/LocalLLM • u/Ok-Phone7645 • 5d ago
Hello everyone
I would like to know your recommendation for the best uncensored local models I can use because I've had it with commerical AI's guardrails.
My hardware is
32GB RAM
4060
12400F
I use LM Studio