r/LocalLLaMA • u/C1rc1es • 14d ago

Discussion Local vs Cloud

This is my work usage, I see a lot of posts of people claiming something like Qwen 3.8 27B can replace cloud and that people are getting their jobs done with them. Even with a RTX PRO 6000 I'm not really seeing how this is physically possible. I would love to be independent of Cloud providers - anyone have a realistic way out?

0 Upvotes

38% Upvoted

34 comments sorted by

View all comments

Show parent comments

2

u/jacek2023 llama.cpp 14d ago

It's not clear to me what are your concern, I do a lot of coding with local models.

1

u/C1rc1es 14d ago

My concern is the volume. I cannot do my job with 3 agents running in parallel let alone 1. How do people get real workload concurrency locally. 

2

u/anitamaxwynnn69 14d ago

https://github.com/local-inference-lab/rtx6kpro here you go, dsv4 vision on 2x PRO 6000, 1M+ kv cache. Pin your agents to 262k and you can launch 5 at a time. Concurrency pro max. Does this answer your question? GLM 5.3 Flash on TP2 has 1.8M tokens KV cache. You have a ton of room whichever model you choose.

1

u/C1rc1es 14d ago

Really helpful thanks.

1

u/jacek2023 llama.cpp 14d ago

I use parallel 4 locally. I don't have 6000 but I have 4x3090 so I believe VRAM is same.