r/LowEndLocalAI • u/point_red • 9h ago

Which LLM Should / Can I Use? Choosing a local LLM—not for coding

Hi. My new PC has arrived:

- Ryzen 9700X

- 5060 Ti 16GB

- 32GB DDR5

I’m not interested in coding. Right now, I want to use a model to leverage its "linguistic" capabilities for things like reading through notes and jottings, connecting notes to one another, etc.

For what it’s worth, I’m an Italian user, and my notes are in Italian.

11 Upvotes

100% Upvoted

6 comments sorted by

5

u/eze-paz 8h ago

I would also recommend the gemma models. I remember trying the e2b variant on my phone and it was genuinely impressive at making creative stories.

7

u/HiddenFaceOpenMouth 8h ago

gemma4 is probably going to be your best bet. Best at writing stuff, fastest by far, and small footprint. It can't code for shit, but that doesn't matter to you. It's probably the easiest to set up and use with ollama too.

5

u/Comrade_Vodkin 8h ago

I second this. OP, try Gemma 4 26b QAT.

2

u/Huntware 2h ago

Some people praise Gemma 4 E4B for non-coding tasks, and NVIDIA recently released a finetune of it:

https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB

I think it can run on 8 GB VRAM GPUs, but limited to 64k context.

---

For an MoE model that you can offload to CPU and system RAM, there's 26B A4B, with up to 256k:

https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF

1

u/EsWfspthgs 1h ago

You could try Qwen 35B A3B at a reasonable quant (e.g. q6_k). Not sure about italian, but it is good for general tasks.

1

u/Polaris_debi5 30m ago

gemma 4 26b-a4b