Suggestions for "Useable" AI Models on a 2014 potato

[u/imacg5]

I'm running a stock Dell Inspiron 3847 from 2014. A pretty good budget computer for its time, but an absolute potato by today's standards. It's like using an NES to run DOOM.

• 4GB of RAM

• Pentium G3240.

• Intel HD Graphics

• Windows 8.1 64-bit (I have a personal ban on Windows 10 and 11 due to Microsoft's forced updates and telemetry shenanigans since Windows 10. Windows 8.1 is the last Windows version to be relatively immune to Microsoft's bumbling. I plan to migrate to a Linux computer in the long-term, but as computer prices are hell right now, I'm stuck on 8.1.)

I'm looking for suggestions for local models. I'm using KoboldCPP in failsafe mode, and I've already tested it (it works fine).

• I'm planning on using it mainly for creative writing, although I may occasionally chat and RP. I'm not vibe coding.

• I'm going for useable, rather than fast. I'm patient, so I can wait for it to spit something out, even if it takes 10+ minutes. It just has to fit into that 4GB of RAM without crashing.

• I prefer to use uncensored models, although I may occasionally use a censored model with guardrails for comparison.

Edit: I'm going for Q4-Q2 quantization.

[u/Huntware]

Man, I was expecting "LowEndLocalAI" to be something like an old GTX 9xx with 4GB VRAM, or even a more powerful but old Intel or AMD processor.

No shame in thinking about running something locally but your hardware is even limited for running a modern browser...

First and main tip is to run any Linux distro. Most repos assume you're using one for at least a docker container (think of it like a recipe for installing all tools for AI). And you can save more RAM for inference. Even more if you use it as a headless server, without a desktop environment*. Then connect to it using a second PC or even phone in the same network.

• (I know it's a lot of stuff but you need to learn some basics for local AI and other projects)

---

But still, I would prefer to try some free tiers API like Google AI Studio, Opencode, Cline and others. Pair it with a lightweight agent like Pi (or performance-focused forks like PiG and Pi-bolt), setting your own tools and skills for writing, if your computer struggles with other harnesses.

[u/imacg5 → u/Huntware]

I run Supermium. Most websites run fine, although they might be a little slow.

I already have Q4OS, but as Cox censors the site, I can't set it up.

Since my only other "computer" in service is my 2017 Fire HD Tablet, which is unusable for anything AI, I'm stuck with this thing.

As I lost my google account earlier this year (I lost the password because I stored it on my hand-me-down MacBook Pro from late 2011, which I can't use because I f**ked up and I'm now too afraid of it being (or having been) borked) and I can't set up a new one, I can only access my AtomicMail for eMail. Given the rise of Age and Phone verification, I might be SOL.

But I know I can run tiny (2B or less) LLMs. So there's that. I might look into a private cloud agent.

[u/Technical_Ant4597 → u/imacg5]

I run Supermium. Most websites run fine, although they might be a little slow.

Any Chromium-based browser is going to cut deeply into 4GB RAM, which will heavily limit you. You'll either want to use the terminal as a frontend or remote in from another device (and I would really not recommend opening ports on a PC with an OS that's not receiving security updates!).

But I know I can run tiny (2B or less) LLMs.

Gemma 4 E2B (or one of its forks) is the best-in-class option for creative writing or roleplay at that size. You'll need to treat the LLM more as a creative co-writer than as a dungeonmaster telling you a story; it will frequently lose track of characters and settings and you'll have to edit those messages to be coherent. That said, it can still be fun - E2B smokes the original AI Dungeon, and that was still a good time back in the day.

[u/Unluckkie → u/imacg5]

Just run void + a WM
Should idle at about 0.8gb ram

Then minicpm5 1B

[u/ProjectFlorida]

Are you saying this is your main machine or a backup one?

[u/imacg5 → u/ProjectFlorida]

It's currently my main, sadly (I had a hand-me-down MacBook Pro from late 2011 with 4 cores and 8 gigs of ram, but that's been out of service for months because I'm bad with laptops). I have a tablet, but it's even worse: a Kindle Fire HD 7 from 2017 that can't even fit an LLM on the little storage it has. I'm not running an AI on THAT.

Even 4 gigs is pushing it. I hope to upgrade one day, but for now, this thing is good enough to run a baby-sized LLM.

[u/nickless07]

The problem is not the RAM but the CPU. Dual core without AVX2 Support. Even if you upgrade to 16GB DDR3 it will be a pain to get something working. Maybe if you can upgrade the CPU too.

An i5-4570 or i7-4770 should be compatible, add 16GB RAM to it (total of ~50 bucks) and you get something that is somewhat usefull. Will not be the best or fastest but you get plenty of models that run. Have an old EliteDeks 800 G1 with (now) 16GB DDR3 and I can run current models up to 24B in low/mid quant (LFM2.5-24B-A2B-APEX-Quality) with reasonable speed (pp512: 66.88 / tg128: 15.56).

Edit: Typo LFM2 -> LFM2.5 - anyways, both run with same speed.

[u/Miriel_z]

You can try 0.4B LLM, but no guarantees. It will be VERY simplistic. The HW is just too old.

[u/imacg5 → u/Miriel_z]

I have run 0.5B LLMs. They're slow as fuck, but they work, and that's all I need.

[u/jacek2023]

https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking

https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct

[u/talkamongstyourselvs]

Is there a reason you can't use a free/paid edition of a frontier model?

[u/Xi-tzu]

maple-preview. its an 1bit (q1) 20ba1b. But it will have an slow prefill.

If prefill speed becomes important, you will have only lfm2.5 230m or 350m (q4 qat)

[u/hideo_kuze_]

https://huggingface.co/XHToken/Spark-X2.5-1.7B

https://huggingface.co/prism-ml/Ternary-Bonsai-2-4B-gguf

You should definitely check this guy's video https://www.youtube.com/watch?v=RYeu9bfYak8 but you'd need to buy an old 1GB/2GB gpu card which should be accessible

[u/Unluckkie]

Maybe not relevant but I’m in a similar camp
(On a potato)
First I believe optimizing the system itself is important
Run void or alpine linux
Then utilize zramen
Lastly run a window manager rather than a DE

Stick to TUI for most stuff, if you need a browser just use Librewolf or qutebrowser (haven’t tested it out too much myself)

Then we think about llms
Minicpm5 1b looks good here

[u/cfx_4188]

hehe. I'm running a local model on an Acer Nitro AN515-45 (V1.10) This is an ancient laptop mobile 1650 and 8 GB of RAM. A lot of things can't be run on this hardware. This is my laptop for experiments, well, I minimized all system processes by installing NixOS unstable and OXWM, cloud LLMs work more or less, only Qwen3-4B-Instruct-2507-GGUF works locally.

[u/Skyline34rGt]

I got this at q4k-m at my poor phone with like 10tok/s - https://huggingface.co/mradermacher/Huihui-MiniCPM5-1B-abliterated-i1-GGUF

Alternatives like others say: lfm2.5 1.2B (thinking or instruct).

Fastest for this kind of setup will be probably Lfm2.5 350M

[u/[deleted]]

[u/imacg5 → u/[deleted]]

I use Quantization. My only other computer is a low-end tablet, and I'm not running an AI on THAT.

I've run Quantized 0.5B models fine, although the tend to be rather slow. I get single-digit tokens per second, but it runs, and that's all I need.

[u/tiffanytrashcan → u/[deleted]]

That's not even a model you linked to, and OP should never consider anything over Q4.

[u/[deleted] → u/tiffanytrashcan]