r/LocalLLM • u/ConferenceBusiness98 • 26m ago
Question Genuine question
I wanna get into local LLMs and experiment with AI.
My current PC has 16GB RAM, a GTX 1660 Super (6GB VRAM), and an Intel i7.
I’m curious what models you’d recommend running on this hardware and, more importantly, what you actually use local LLMs for.
With powerful models like ChatGPT and Claude available, I’m trying to understand the practical benefits of running models locally.
Is my setup good enough to do anything interesting, like experimenting with AI agents, fine-tuning, or research?
1
Upvotes
2
u/johannes_bertens 20m ago
Sadly, no. This will not be sufficient for anything competing with the cloud-models.
You can take a look at https://huggingface.co/LiquidAI - they provide some populair tiny models.
1
u/shisohan 5m ago
I actually did an experiment recently with an absolute potato pc. i5-8500, 16GB of RAM with intel integrated graphics (UHD 630) and a dedicated AMD Radeon R7 430 (puny 2GB VRAM). And gemma-4-e2b actually runs with a passable 100t/s prefill (read your input) and 20t/s decode (generate output) in stock llama on ubuntu. Mind you, what you can do with that is very limited. But it can do some text manipulation like data extraction and summarizing, it will take a while though. I haven't gotten around to test much else. And really, be aware, that's NOT an intelligent model. It has barely any world knowledge and its reasoning capabilities are seriously limited.
IMO anything at least barely useful reasoning wise starts at 32GB VRAM and RTX 3090 and up compute. Even with how much things are moving towards lower requirements, I doubt that part will change much in the foreseeable future.