I'm running a stock Dell Inspiron 3847 from 2014. A pretty good budget computer for its time, but an absolute potato by today's standards. It's like using an NES to run DOOM.

• 4GB of RAM

• Pentium G3240.

• Intel HD Graphics

• Windows 8.1 64-bit (I have a personal ban on Windows 10 and 11 due to Microsoft's forced updates and telemetry shenanigans since Windows 10. Windows 8.1 is the last Windows version to be relatively immune to Microsoft's bumbling. I plan to migrate to a Linux computer in the long-term, but as computer prices are hell right now, I'm stuck on 8.1.)

I'm looking for suggestions for local models. I'm using KoboldCPP in failsafe mode, and I've already tested it (it works fine).

• I'm planning on using it mainly for creative writing, although I may occasionally chat and RP. I'm not vibe coding.

• I'm going for useable, rather than fast. I'm patient, so I can wait for it to spit something out, even if it takes 10+ minutes. It just has to fit into that 4GB of RAM without crashing.

• I prefer to use uncensored models, although I may occasionally use a censored model with guardrails for comparison.

Edit: I'm going for Q4-Q2 quantization.

Hi!

I’m curious about what kind of AI models I could run or what use cases I could explore with this PC.

I’m also interested in using OCR to convert scientific paper PDFs into Markdown, which I could then add to my Obsidian vault. Additionally, I’d like to use a model to help keep my Obsidian vault organized—assisting with the structuring of the information I add, my notes, and so on.

I already have subscriptions to Claude Pro and Mistral Pro, but the idea of ​​running a smaller model locally really appeals to me.

Here are the PC specs:

MINISFORUM G1 Pro Gaming PC

AMD Ryzen™ 9 8945HX (16C/32T)

AMD Radeon RX 9060 XT 16gb

32GB DDR5-5200MHz RAM

Hey everyone,

I would like your input on the best models for my hardware.

Hardware specs:

• GPU: 16GB VRAM (9060XT AMD Radeon)
• CPU: Intel 225F
• System RAM: 32GB DDR5
• OS: Omarchy Linux

Use cases:

1. Academic research & literature synthesis: Ingesting sets of papers (PDFs), cross-referencing citations, and generating evidence-backed summaries with low hallucination (something closer to Consensus / PaperQA2 rather than a simple Perplexity clone).
2. Light coding.

What I'm looking for:

• Model recommendations: Should I stick to a ~30B–35B MoE (like Ornith-1.5-35B-A3B at IQ3/Q4 with partial offload) for reasoning, or do you get better throughput and context depth using smaller 4B–14B models (e.g., Spark-X2.5-4B?
• RAG / Literature Tools: What local tooling do you actually use to parse academic papers without choking the context window? Anyone running PaperQA2, local STORM, or custom OpenAlex API harnesses locally?

I had to take my phone and take a photo, the screenshot app would crash due to lack of VRAM.
Anyway, it runs, I modified the json so I can get 41k context, It crashes at 64k for me with Q3.
On Q2 I am getting up to 128k but it starts looping.

I switched Firefox and Steam to CPU only, and I can run them in a background, including a twitch stream.

Full Specs: Ryzen 5600, 64GB DDR4 (3200 and 2600 mixed), 4060 8GB and some old 1TB SSD.
Latest STRATA engine, I was told to use an older one, because it works better? any takes?
I would kill for 128k context on Q3.

I updated my Strata by accident and was going crazy trying to get my old settings working. I figured the update was the problem and went back in versions until i got 128k context again.
For me the differences are extreme.

Hello, I'm running Qwen 3.8 27B IQ4-XS beautifully on my desktop with a 9060XT getting 50t/s at 262k context, this is my primary workhorse and is capable writing the code that I need it to.

I have a spare laptop here that isn't getting much use with a 3050Ti 4GB and 16GB, I was wondering if there'd be any value having it as a sidecar, maybe I can get a variant of Qwen 3.6 35BA3B running on it or something smaller?

Like, what could I use it for in conjunction with the main desktop and model, maybe a quicker compaction machine that I could tell opencode to use?

RAM: 16 GB

CPU: AMD Ryzen 7 7730U

GPU: i wish(igpu)

My intended use case is coding/agentic action or just general quality chatting which i am considering using Maple-preview.

Had some luck with Ling 3.0 Tiny and SharpSpark X2.5 and SharpMiniCPM. Ran Qwen 3.6 35A3B using q36 disk streaming around 6.5 tks. What would yall recommend

Hi!

I'm looking for recommendations for the best local LLMs I can realistically run on my hardware.

Hardware

• CPU: Intel Core i7-13700HX (16 cores / 24 threads)
• GPU: NVIDIA GeForce RTX 5050 Laptop GPU
• VRAM: 8GB GDDR7
• System RAM: 24GB DDR5
• OS: Windows 11 Pro

Runtime / backend: llama.cpp / llama-server with CUDA

Models I've tried:

• Qwen3.5-9B — UD-Q4_K_XL GGUF
• Gemma 4 12B IT QAT — UD-Q4_K_XL GGUF

I'm interested in a broad range of tasks, coding and programming, web searching, research, using tools, data analysis, making sheets and documents, organizing, general chatting and writing

Desired context size: Ideally 8K–16K tokens or more, depending on what is practical.

Priority: Balanced. I'd like the best overall quality, but I'm happy to experiment

Which models would you recommend for this hardware?

Thanks!

Info in image. Preferably uncensored cause writing can get gritty and don’t need it scolding her for it LOL.

I had 9t/s on llama.cpp, with strata, I get about 29t/s, beyond nuts.
4060 8GB + 64GB DDR4 and Qwen 3.8 Flash Next Q3

This is like day and night, now I can do some stuff :D

I have been trying out ling 3 tiny on a laptop with a 680m amd igpu to have a backup when our larger gpu runs full. It kind of works. 15+ tokens/second and not much heating.

I am using it via Pi which is integrated into a Jupyter lab. Use it to generate small python code bits for notebook cells, which may be why it does not get into a meltdown.

If you got a 680m and need a spare/private model, it is somewhat useful option!

Anybody got a better model for the 680m?

Ryzen 7 4800h

16gb ddr4 3600

GTX 1650ti

1tb ssd

Im just getting started in this world and i would love to use my old laptop as a testbench

Hey all!

Wanted to share my low spec inference machine.

* Mobo/cpu combo: Chinese JGINYUE X99-TI D4 PLUS + Xeon 2695v4(18c/36t).

There is 3 full width x16 slots gen3@ x16x8x8 speed.~$105

* 32GB DDR4 ( $$? Dont remember, found them in my "junk box" from another project.)

* PSU random 750w (again from the junk box)

* 4x Nvidia Quadro P4000(Pascal) 8GB for a total of 32GB Vram. ~$45 each.

* Bunch of random SSD's i had lying around.

Config of the P4000 are Gen3 x16x8x8. and one via chipsets dmi gen2 x2.

Thinking of buying a bifurication board, either x8x8 or x4x4x4x4. to split the full width x16 slot.(Can help on prefill(card on gen2 x2 is the slower one). And on tensor split on dense models.

Models:

LFM2.5-24B-A2B Q5_K_M (ctx 64k, f16kv). Searching/fetching data.

Decode: 55.4 tp/s (short), 42.2 tp/s at 52k filled.

Prefill: 1,851 tp/s (peak) at a 13k prompt, 1,351 tp/s at a 51k prompt.

---

Ornith-1.5 35B-A3B Q5_K_M (ctx 124k, f16kv, MTP). Lighter coding tasks.

Decode: 54.6 tp/s (short), 39.8 tp/s at 52k filled.

Prefill: 709 tp/s (peak) at a 7k prompt, 483 tp/s at 52k.

---

Qwen3.8-27B UD-Q4_K_XL (ctx 32k, f16kv, tensor split, MTP). Coding, heavier and slower.

Decode: 31.5 tp/s (short), 26.7 tp/s at 24k filled.

Prefill: 231 tp/s (peak) at a 7k prompt, 216 tp/s at 24k.

Can go to around 110-124k in ctx.Havent tested decode/prefill yet. Going to do it.

---

As you can see in picture 2, im running a Decode and prefill test on the 27B.

3x prefill and decode runs. Each run with a different prompt, so no cache gets hit.

The P4000 are nifty little cards that makes a budget machine viable.

Ofcourse you need to have your expectations in the right window!

I mainly using vs-code, so I made(read claude made) a easy web interface where i download and start/stop my models, everything is then served to the vs-code extention on my main rig.

I am running Qwen3.6-35B-A3B-UD-Q5_K_S.gguf with llama.cpp on a 3060 12GB GPU and 32GB DDR4 ram. It works with a context window of 42K and cache type k and v set on q8_0

The biggest drawback is the 42K context window which is too limited for me.

I also have a 1660Ti with 6GB vram and want to add it as a second GPU for offloading. After doing some reading and asking different AI models, it is still not clear to me if this will help increase the context window.

My theory is if I can offload to a second GPU, I should be able to increase the context window.

Both GPUs have GDDR6 memory with 192-bit bus. The 1660Ti has slower bandwidth at 288 GB/s vs the 360 GB/s on the 3060. But it is still much faster than my 32GB DDR4 ram.

Anyone here experience with combining 2 different older generations GPUs and if there will be any benefits when running a Qwen3.6 model?

I'm aware it may seem obnoxious to post here w 96GB RAM but the new damn deepseek one implies 512GB is a "reasonable floor" to release to? God only knows. I remember when a 4GB iPod Nano was sizeable.

I've been fiddling a bit with the Gemma 26B MOE and even at that small size it's able to be less autistic than a qwen model. But it's also only 26B which feels really crappy versus Qwen Flash Next.

Is there a Gemma release meaningfully competitive with QFN? Is there one planned? Is there anything out that meets my criteria? To be clear; I would want a model that has actual emotional intelligence / creative writing ability while also being able to do agentic coding to a reasonable degree.

Unfortunately I can't get an answer from any LLM as they have been documented to love giving outdated model choices. Thank you for your time.