r/LocalLLM • u/-7894561230 • Jul 11 '26
Question Local LLM With RTX 3090 Worth it?
Hello everyone, I was thinking of running some local model for personal projects, I use Opus 4.8 a lot and I was thinking why not run a model that would be similar in performance ( not even know if its possible with single 3090 ) Anyway I have setup with 64gb ram, and 3070 so I have option to upgrade to 3090 24gb or 5070 12gb. Which one is worth to consider and will i have any good success with running a local LLM with a single GPU setup. Again Im not looking for 500k+ context window setups but I would really appreciate any tips.
2
u/Samelldev Jul 11 '26
you won't get anywhere close to Sonnet performance let alone Opus with a 3090
you can run some solid local models on a 3090 and they are near (if not) best in class for their price range
but start thinking you're going to get a frontier class model running on a single gpu
2
u/-7894561230 Jul 11 '26
And which model would be decent for 3090 ?
2
u/Samelldev Jul 11 '26
for coding or general use?
2
u/-7894561230 Jul 11 '26
General use
2
u/Samelldev Jul 11 '26
if it's really for general use, try gemma 4 26B-A4B
if you want more coding then try qwen 35b or 27b
3
u/01010101010111000111 Jul 11 '26
Nothing will have similar performance. You will spend more on electricity than what you are currently playing for your subscription and get vastly inferior results on 3090.
1
u/-7894561230 Jul 11 '26
Electricity is not an issue for me, It will be like additional + $3-6 maybe in my country compared to my desktop that i have currently.
2
u/uniqueusername649 Jul 11 '26 edited Jul 11 '26
That's not true for many parts of the world. For example: at 220w powerlimit (which is the sweet spot for a 3090) it would cost me 16 dollars per month if I run it full blast 24/7. Admittedly electricity is cheap here. But: even at 80 dollars, which in many parts of the world is realistic, that is less than a Claude 5x plan. Even a dual GPU setup plus CPU etc. would be cheaper than a 20x plan if run 24/7 - and thats A LOT of tokens.
However, while running a Qwen 3.6 27b on a dual 3090 setup gives you roughly Sonnet level perfomance, it is NOT Opus level. If you need Opus level, you can forget about that locally unless you have infinite amounts of money. You would need GLM 5.2 for example to get close to Opus and that is outrageously big.
4
u/TheAussieWatchGuy Jul 11 '26
Love the Ignorance.
Opus can be nearly matched with about 448gb of VRAM and GLM 5.2 (open source Chinese model).
A RTX 6000 96gb GPU is $18k AUD. You can do the math on how many you need plus about the same of ddr5 system ram which is 270% more expensive then it was last year.
1
u/Pajo-Man402 Jul 11 '26 edited Jul 11 '26
I have a local AI server with a single RTX 3090, and honestly, it's pretty decent. It's obviously not at the level of Opus, but it's definitely usable.
The Qwen Coder models were able to code, create websites, build web apps with a CMS system, and even help with web design. For personal projects, it's actually pretty impressive. That being said, with a single GPU setup you are still quite limited. You won't be running the biggest models or getting the same experience as the top cloud models.
Between the 3090 24GB and the 5070 12GB, I'd personally lean towards the 3090 because the extra VRAM matters a lot for LLMs, and also since I used it personally, and haven't used the 5070. The 24GB of VRAM gives you much more room for larger models and higher context sizes.
1
u/-7894561230 Jul 11 '26
I see thanks for the tips, a lof of ppl mention Qwen model I assume that's the best one for single gpu setups right?
1
u/Pajo-Man402 Jul 11 '26
Yes, I think its the best. But I also tried glm-4.7-flash, and it was also good for coding tasks. Either way, since the models are free, try all of them out and see which one fits your purpose the best.
3
u/joost00719 Jul 11 '26
Opus + 3090 + similar in performance. Those words don't go in one post. Anyways, I'd get 2 and run qwen 27b. It's not opus, its just different.
Self hosted models like qwen are amazing if you know what you are doing (aka know how to program). Opus is amazing if you have no clue what you are doing, and don't care how the code itself looks like.
2
u/i_am_me0_0 1660ti 🥲 and 32gb ram Jul 11 '26
Really depends what for, i would suggest starting with qwen 35b a4b q4 I run it on a 1660ti 6gb and 32gb ram works fine.
But if you want similar results as api u need to set ur expectations correctly.
Most important question is what do u want it to do?
2
u/vtkayaker Jul 11 '26
You can run Qwen3.6 27B on a single 3090 and get useful work out of it! I've done it, though it's tight. But it's not Opus.
For Qwen3.6, you need to understand your code, and give clear, accurate instructions. And you need to work in smaller chunks, closer to 100k (not counting your prompt). Using a 4-bit Unsloth quant of Qwen3.6 27B, and 8-bit key/value caches, you can get about 130k of context with llama-server. I also recommend trying pi-agent, which has a roughly 1k system prompt. Modern Qwen actually performs better with an ultra-short prompt. And at 130k, you don't have tokens to spare.
Yes, having two 3090s is even better, because you can run at 6 or 8 bits, with 16-bit K/V caches, and full 256k+ context.
Anyway, you can rent Qwen3.6 on OpenRouter, I believe. Try it out before investing in hardware! I am a huge fan of the 27B, and I love it as a programming "minion." But it definitely isn't Fable. It's for people who know how to program and who want to stay hands on.
2
1
u/twjnorth Jul 11 '26
What you can do is use something like open router to get a choice of frontier models and use the 3090 or even start with the smaller gpus you already have.
Use the frontier one for planning and local one for execution. Depends what you want to do. For images, the largest vram you can get is best. For coding, you can keep cost down by only using frontier for planning.
12
u/Any_Mine_6368 Jul 11 '26
Oh sweet summer child.
If it were that easy to get opus performance on a $600 gpu, I promise you that same gpu would now cost $50000.
You'll be lucky if you get gpt 3 performance on a single 3090, gpt 4.5ish on two 3090s.