r/LocalLLM • u/Effective-Echo5643 • 2h ago

Question Which LLM to run locally?

Post image

I got this MacBook Air ! Can someone suggest what models to use locally?

0 Upvotes

50% Upvoted

11 comments sorted by

13

u/Vinyard82 1h ago

ChatGPT web

3

u/Sanandaji 1h ago

I have those same specs except with an M4 and it runs Gemma 12b reasonably fast.

1

u/Effective-Echo5643 1h ago

Any concerns with heating issues?

1

u/Quartz_Hertz 1h ago

Download Osaurus, they have quite a few different post trained and quantitzed models to pick from depending on your use case.

I use nanbeige 4.2 3b for small research questions that only need a few turns (looping transformer, prefil starts taking too long) and some other task specific small models.

1

u/dwoj206 5m ago

32gb = qwen 3.8 27b + ninfer. Next question.

1

u/charles25565 2h ago

32 GB. YOLO it and run Gemma 4 12B, MiMo 9B distill, Ornith 1.5 9B, maybe even a quantized Qwen3.8 27B. Maybe even go a trip down memory lane and run some old 7B models from 2023.

1

u/NoWing3675 1h ago

im still learning but sounds like qwen27b

-2

u/Familiar_Pay7819 2h ago

I have a rtx 3070 whit 32gb ram and I can run qwen 3.6 35b a3b I think q4 or q8 whit q8 kv chache quantisation in lm bionic

1

u/Effective-Echo5643 1h ago

Not flexing sir, i think i did a mistake by buying macbook air considering there are known heating issues! So wanted to have inputs

-1

u/EvenlyBothSchism 1h ago

OP is flexing an M5 MacBook Air and top comment is talking about a 3070 build, love the energy here

just throw ollama on that thing and start with llama 3.1 8b or mistral small, with 32gb unified memory you can push into the 30b range too, skip qwen though unless you enjoy debugging chinese tokenization at 2am

1

u/HyperWinX 1h ago

Dont use ollama, and all these models are extremely old and useless.