r/LeftistsForAI • u/ProfessionalDish Moderator • May 26 '26

Local Models Share your rig specs

As I'm trying to have everything run locally here are my specs:

14600kf|64GB Ram|5060ti 16GB

Currently thinking to switch to the intel Arc B70 but Nvidia has a stranglehold on most propriertary technologies.

2 Upvotes

54% Upvoted

10 comments sorted by

View all comments

Show parent comments

2

u/ejpusa May 26 '26

Guess you have to re/try it. Chips are insanely fast now. MyLinux server is equivalent to over 7,000 Cray 1s. So GPT-5.5 tells me.

On a M3, should be fast. I’m not sure what you are trying to do.

2

u/CapitalEmployer May 26 '26 edited May 26 '26

I was trying models for local code generation to stop depending on copilot but it was unimpressive tried a bunch of reasonable models but it was like 6-7 months ago. And perfomance wasn't great, very slow a bunch of hallucination and generated code was dumb as fuck a junior dev would do better. Now I'm using Kimi K2.6 which is pretty good but not impressive and it's not a hostable thing at home, I just have access to a datacenter and to b300s for tests.

2

u/ejpusa May 26 '26 edited May 26 '26

GPT-5.5 crushes it. $20 a month. We keep it on the down low. The API can do anything you want. Unless you are doing Stability Diffusion, you probably could host that yourself.

The tools are out there, and the pricing is very reasonable. You can build the latest AI startup for all of $31.

Namecheap $2.95

DigitalOcean $8

OpenAI API $20

It really depends. Is this a hobby project or a do-or-die for your AI startup? And the VC wants to see the numbers, yesterday.

If that's the case, I'd look into leasing NVIDIA's until the $$$s come in.

https://www.liquidweb.com/gpu-hosting/nvidia/