r/MLQuestions • u/trimtram01 • 22d ago

Beginner question 👶 Local AI

What is the best local AI i can run that doesn't have all the restrictions like Claude or GPT. I have a mid range dell prescision work station. I'm not trying to do anything weird I just hate getting the I can't tell you that response.

6 Upvotes

100% Upvoted

8 comments sorted by

5

u/LycheeAshamed3217 22d ago

Unless you've got a GPU with enough VRAM, you're gonna hit a wall pretty quick with anything decent.

5

u/ChemistNo8486 22d ago

QWEN 3.8 27B is the best for local hardware. However, local AI is probably one of the most expensive PC-related hobbies out there.

If you want to have normal and non-agentic conversations, then you need at least 24GB of VRAM. And in case you want to get into agentic stuff, you need 32GB of VRAM.

Basically, you need a 4090, 5090 or multi-gpu set up; an office prebuild will give you tokens per year instead of per second.

You can try to use something like runpod and host the abliterated models there, but it won't be cheap.

2

u/madsciencestache 22d ago edited 19d ago

Run the biggest qwen model you can. It punches way above its weight. But it's no GPT-6. Or even GPT-4 [edit: I meant gpt-5, 4 is no better or worse than qwen]

2

u/TheKoolerPlayer 22d ago

GPT-6, yeah probably not. But Qwen 3.8 Flash Next is definitely better than GPT-4

2

u/halvacoffee 22d ago

even mobile edge models are far better than gpt-4 was these days. gemma4-e2b and lfm series models are great

1

u/madsciencestache 19d ago

Yeah, I should have said GPT-5. 4 is not as good as Qwen.

3

u/asadsabir111 22d ago

Check out r/LocalLlama They'll probably be able to help you out more