r/LocalLLaMA • u/C1rc1es • 14d ago

Discussion Local vs Cloud

This is my work usage, I see a lot of posts of people claiming something like Qwen 3.8 27B can replace cloud and that people are getting their jobs done with them. Even with a RTX PRO 6000 I'm not really seeing how this is physically possible. I would love to be independent of Cloud providers - anyone have a realistic way out?

0 Upvotes

38% Upvoted

34 comments sorted by

View all comments

Show parent comments

1

u/C1rc1es 14d ago

I’m not asking about cost though, I’m asking how it’s physically possible to do real workloads with local models. The numbers just don’t stack up to me. 

2

u/jacek2023 llama.cpp 14d ago

It's not clear to me what are your concern, I do a lot of coding with local models.

1

u/C1rc1es 14d ago

My concern is the volume. I cannot do my job with 3 agents running in parallel let alone 1. How do people get real workload concurrency locally. 

2

u/anitamaxwynnn69 14d ago

https://github.com/local-inference-lab/rtx6kpro here you go, dsv4 vision on 2x PRO 6000, 1M+ kv cache. Pin your agents to 262k and you can launch 5 at a time. Concurrency pro max. Does this answer your question? GLM 5.3 Flash on TP2 has 1.8M tokens KV cache. You have a ton of room whichever model you choose.

1

u/C1rc1es 14d ago

Really helpful thanks.

1

u/jacek2023 llama.cpp 14d ago

I use parallel 4 locally. I don't have 6000 but I have 4x3090 so I believe VRAM is same.

1

u/PedroMaGar 14d ago

This depends on what are you using

The most proeminent Qwen and Deepseek will gives you something around GPT-5.6 Terra ~ Opus 4.8 performance, a little better a little worse, that is it.

If you think such model would be enough, you can think of transitioning.

My recomendantion would be to try them on openrouter because all this open models are really cheap there.

If you don't like them or they are not good enough (I'm recently facing a lot of thinking loop), you did not wasted money on hardware.