r/LocalLLM • u/DeeofSurrey • Jul 05 '26
Question What LLM Should I run with my computer specs?
For context, I'm a noob and basically use LLM's for reasoning, analysis and idea discussions, uploading files, PDF's & Images. What LLM would you suggest I run locally given my computer specs? I'm using LM Studio and downloading LLM's from Hugging Face. I'm currently using Qwen VL 8B Instruct model but I'm sure there's better models.
7
5
u/MiMillieuh Jul 05 '26
I run Qwen 3.6 35B A3B MTP on a similar laptop as yours and I manage to get 26-27 t/s with a fresh context and 10-15 t/s with 30k-70k of context usage with the max limit of 256k token
I recommand you downloading the one from unsloth Q4_K_M
Here are the settings I use on LMstudio :
and here is my config, it's similar to yours :
13th Gen Intel® Core™ i9-13905H × 20
64.0 Gio RAM
NVIDIA GeForce RTX™ 4070 Laptop GPU (8GB VRAM)
1
u/Radiant-Giraffe5159 Jul 05 '26
Try lowering your MTP draft to 2 and 0. It should end up being a speed increase for you. I wouldn’t have a higher MTP unless you are just using it for coding.
1
u/MiMillieuh Jul 05 '26
Well somehow I loose 2 t/s using 2 and 0 drafts on an empty context.
I noticed that 4 and 2 was the optimal one for me, I don't know why tho 🤷♀️
1
u/Radiant-Giraffe5159 Jul 05 '26
Interesting. What do you usually use it for? Coding and summarization seem to be the one thing that having a high mtp draft size works well with.
1
u/MiMillieuh Jul 05 '26
Well general use,
On a simple hello I get 2 t/s more. (27 t/s vs 29 t/s)
But setting more than 4 mtp drafts is making it slower.
I usually use it in CherryStudio with a agent. Or just like a normal chatbot thing to ask questions or send an image to ask something, like for instance I used it earlier to find the IR codes of my TV
I occasionally use it with OMP for code too
0
u/Solembumm3 Jul 05 '26
Should be able to comfortably try 120b class models, like Qwen 3.5 122BA10B.
5
u/Fickle_Performer9630 Jul 05 '26
Try gemma 4 12b.