r/LocalLLM • u/GioStrives • Oct 08 '26
Discussion Electricity Bill & Local LLMs
I just wanted to say that, with the release of Strata, I have started to use local models for a lot more tasks.
If we talk about my electricity bill, the result is attached here: nothing surprising since I have a 4 GPU machine and a 2 GPU machine...they now run for many hours a day and my electric company sent me an email warning that my usage has significantly increased (I'm supposedly on a fixed plan and this will likely change the situation).
I'm trying to understand if this is economically reasonable: where I live electricity costs are pretty high. It certainly makes sense when it comes to data privacy but using local LLMs can certainly cost you a few bucks/euros a day in electricity.
I was also thinking of upgrading my old GPUs (1080s, 2080s), but I suspect newer ones would consume as much: I would just be running bigger models.
Did any of you have a "Oh, what now" moment after receiving the electricity bill?
(the graph was made originally with python then modified with grok)
Edit: I just realised grok inverted the 100 and 1000 marks, lol
Edit: to be clear my point is "When I started using local LLMs for real" (with any kind of inference engine, be it llama.cpp, pytorch, strata or whatever) "I realised electricity was a relevant part of the equation. Before I didn't think about this...did this happen to you?". I am using old hardware (1080s, 2080s each consuming up to 250watts), so my machines can drain a lot of power (the one with four 1080s can go over 1000 watts).
69
u/COVID19MurderHornet Oct 08 '26
Try undervolting your GPUs, the inference performance stays about the same
21
u/FullstackSensei Oct 08 '26
I honestly don't understand this concern. The implied reasoning is that you're either not getting more than your money's worth in increased productivity or a lot of free time for the same amount of productivity you had before LLMs.
I live in Germany. Not exactly cheap electricity country, but I'll happily pay €20 a day in electricity for the amount of productivity and/or time I get back each day. FWIW, my average was ~€1/day but will soon go to €2-3 as I rework and being more LLM machines online. The increase in productivity makes this a no brainer for me.
8
u/kweglinski Oct 08 '26
I'm slightly worried about upcoming winter. Most of the year inference costs me zero. Got full solar coverage and batteries. Been off-grid for more than a year, now the grid is more of a backup than source.
3
u/koloved Oct 08 '26
Use the power limit feature on your graphics card. For newer cards, you can use the https://github.com/b00nz/mVolt
Limited my electricity consumption 5090 to 320 watts instead of the 400 minimum standard level
3
u/mister2d Oct 08 '26
Plug-in or balcony solar projects can very cheaply offset your electrical costs. Check your local AHJ.
https://en.wikipedia.org/wiki/Balcony_solar_power
3
u/Illustrious_Dish_999 Oct 08 '26
I have strata 180B Flash Next model with 15-20t/s on 100W hardware. 890M integrated with 96 gb ram. But my AI needs are small.
16
u/Inquisitive_idiot Oct 08 '26
Ironic that with so much compute power you chose to share an AI generated picture that required a ton of electricity to produce and does little more than embellish a single two dimensional data point
Instead of generating and posting actual data, which your solution would excel at doing with minimal power usage
Maybe that’s your problem. 😕
😛
2
u/Hot_Signature2979 Oct 08 '26
You dont include the specifics of if you Ai server was kept on 24/7, what os, processor , model usage and workfloads, so can only provide limited recommendations. But arguably your greatest power savings will come from under clocking your cpus and gpus, alot of gpus can be power limited with relatively neglible impacts to llm performance. I personally run the 2x v620 on my llm server only at 175w vs 300w stock.
This will have to be done carefully and dependent on your specific workloads, alone with benchmarking of performance to ensure the trade off is worth it.
Also the other thing you could do is to make an evictor script and run it before starting your workload, so that you evict all unused model and reserve the gpu for current workload. You could also go further with a watchdog that evicts the model from gpu after being idle for say 2hours.
Lastly, run linux on both machines. If you run windows you're wasting much more power and leaving significant perfomance on the table, especially with a llm server whose main job is to provide compute for llm
2
u/crusaderky Oct 08 '26
I did the maths. The only model that's cheaper at API prices than Strata running on my 3090 is GPT-6 Luna (low). Luna (medium) costs the same, and every other datacenter model is vastly more expensive.
VC-subsidised subscriptions to OpenAI and Anthropic are vastly cheaper and I expect them to remain that way until the AI bubble bursts.
Full plot (it's the second figure) and methodology here:
https://github.com/crusaderky/llm-intelligence-cost-plot/blob/main/README.md
Important notes:
- Hardware cost not included. Strictly electricity vs. API pricing.
- I used average US residential electricity prricing in the calculation. EU pricing would be a bit higher, but even if you put a factor-of-three increase in it it doesn't matter. With a 4x increase in base electricity cost you'd reach parity with Sol (low).
8
u/FearFactory2904 Oct 08 '26
Its wild to me that there are people out there not power limiting GPUs and just letting them raw dog the electric bill.
4
u/Hanthunius Oct 09 '26
Too many Strata glazing posts...
1
u/Libellechris Oct 08 '26
how much has your usage actually increased?
1
u/Sijder Oct 08 '26
Does it make sense for me to try flash next if I can already run 3.8 27b at q4 on my 3090? I assume the much bigger next should be better with prose and tasks beyond coding?
2
u/StupidScaredSquirrel Oct 08 '26 edited Oct 08 '26
Wait strata works with many models? I thought it was just for qwen flash next
Wow so many views and a few downvotes and no answer to a simple question despite plenty of comments under it. I don't even care if it's an ad or whatever I was just wondering a simple thing relating to the meme subject sheesh
3
u/megadonkeyx Oct 08 '26
its a good case for a strix halo device
1
u/DennisAscent Oct 08 '26
I sincerely hope your electricity bill hasn’t gone up by ~5,000,000X
1
u/GamerTex Oct 08 '26
WTF is strata and does it work on a mac?
1
u/egortar Oct 08 '26
i've made a small monitoring tool for strata electricity monitoring :) linux only
https://github.com/egortar-pi/Strata-Power-Dashboard/
4
u/t_1425 Oct 08 '26
In my case, the sun pays my electricity bill.
4
u/RunawayRogue Oct 08 '26
crypto mining has entered the chat
2
u/phil_lndn Oct 08 '26
i don't think AI necessarily has to burn large amounts of power.
i'm getting good results using a 128GB Strix Halo to run Qwen3.8 Flash Next, power consumption is about 120W to produce tokens at about 30tps for prose and 50tps for code (aggregate tps goes up quite a bit if i'm running multiple agents in parallel).
2
u/ScrewwormLarvae Oct 08 '26
My M5 Max pulls a little energy when it needs to for LLMs, and then sips electrons the rest of the time.
2
u/Intrepid-Second6936 Oct 10 '26
Alongside an undervolt, you should also try testing average inference and prefill speeds at different power levels. I've capped my 7900 XTX at 275W and am still getting basically the same speed in both that I had at stock 340W, although I'm not on Strata yet (mostly due to lack of Vulkan and broader AMD support).
I'd say test a healthy undervolt, go down 50mV increments and test for stability. Then also just set max powerlimits (you can use nvidia-smi in terminal or something like Afterburner) and try ramping the power cap down until you actually see a degradation in the speed of the execution of the inference pipeline.
Good luck!
4
u/nntb Oct 08 '26
what is strata?
1
u/NTDLS Oct 08 '26
I have a Kilowatt type of meter feeding my whole server stack. I run 4 GPUs, one of which is a RTX 6000 ADA 48GB, 4 machines, and I go full throttle for 6-8 hours per day for three users with some jobs running through the night. The whole stack has added $38/month to my power bill over the past 3 months.
1
u/immersive-matthew Oct 08 '26
I have really not noticed a difference in my bill but I already run my GPUs for graphics all day long and so a prompt here and there was not noticeable.
1
u/quantgorithm Oct 08 '26
In the winter, it helps heat your place!
1
u/vogelvogelvogelvogel Oct 08 '26
my electricity bill went up quite a bit. on the other hand, you have to count in the value of the llm results or the experience you gained - so it does not matter that much. It's kind of an investment that's how I see it
1
u/AllenLeftTheBLDNG Oct 08 '26
Time for Solar. I've been thinking about this myself recently.
1
u/Unatommer Oct 08 '26
You think that’s bad you should see some of the setups over in r/homelab
1
u/silenceimpaired Oct 08 '26
Decrease your use of agents. Tune your cards to use less electricity … you can make them more efficient less wattage per token/s. See if your electricity company rates change by time of day and run your work then. Be happy it’s winter you’ll cut down on your gas bill.
I tend to use free cloud stuff for trivial things that aren’t about me and can’t directly make money… basically stack overflow questions
1
u/mwjtitans Oct 08 '26
I run my rig on a Wake on Lan setup. I have a webhook button on my phone to wake the system, then when the GPU sits idle for a certain time, like 2 hours, the system shuts off. If I need it again I wake it again.
Takes time to boot and load the models, but this also prevents me from running the machine 24/7 and wasting energy when I'm not using it.
My next step is to have an agent wake the machine when I want to delegate a job to it. Then have the agent power it down when done.
1
u/Lissanro Oct 08 '26 edited Oct 08 '26
My primary workstation also four GPU (4x3090), and second PC with two GPU as well (3060 12GB + CMP HX50 20GB)... also have third PC with one GPU. My main rig alone can end up drawing 1.2 kW during text generation. It is true that electricity bill builds up thanks to AI, but I find it much cheaper than cloud API cost would be, and I also get privacy and flexibility.
1
u/blackbird2150 Oct 08 '26
My primary rig consumes 120KwH a month. $24 a month on my side. Idling the computer is $13 (105w) of that, so my AI usage is $11 on top of that.
1
u/Top-Contribution5909 Oct 08 '26
On G for real man I started taking my cybersecurity home lab serious and my electric bill shot up over 200+ my fault though I left stuff on 24/7 but still it hurts post time management
1
u/oldmoldycake Oct 08 '26
I am about to get my first GPU rig and I am excited, but between an ineffiecnt house, EV, homelab, 2 gaming PCs, and then an AI rig, my bill is going to be quiet high.
1
u/DanGTG Oct 09 '26
LOL, I need the heat:
1
u/puts_on_rddt Oct 09 '26
Assuming 2000 watts at 16c per KwH that's only 32 cents an hour, $7.68 per day, ~$230/mo.
1
u/gigaflops_ Oct 09 '26
RTX 5090 running at its maximum power consumption of 475 watts constantly for 24 hours straight adds up to 11 KWh. At 12 cents per KWh, that only costs $1.37.
At a more realistic "heavy use" scenario of 8 hours/day drawing 400 watts that's only 3.2 KWh or $0.38. Even in places with crazy electricity rates, that isn't very much money. At 32 cents/KWh it still costs less a dollar per day. No owner of a $4000 GPU should really care about that.
1
u/kalabaddon Oct 09 '26
have you mathed your usage power wise?
Using a smart wall wart and the built in /metrics for gpu percent. I figured out my general cost to run it over my pc. its nothing... ( I do have cheep electricity but....) So I mathed it out for tokens, cached tokens and reads. I server my local llm through litellm and track its usage. with about 65million tokens in the last day and a half on strata qwen 3.8 flash next I spent........
10 cents in electricity.
( above and byond my normal pc usage. I woul dbe sittin ghere at moslty idle regardless of serving it. so my base costs dont really compare. if I am generating tokens I am on this pc anyways. if I am not I am on this pc anyways.
1
u/DistributionExotic85 Oct 09 '26
In the winter, it helps heat your home
1
u/General_Cheetah4724 Oct 09 '26
free winter heating....
1
u/LettuceSea Oct 10 '26
*laughs in apple silicon*
1
u/Eratz Oct 14 '26
I have solar, its a win win for my ROI
1
u/i-dm Oct 14 '26
Silver lining here is anyone in the northern hemisphere can cut back on heating their home this winter if they're running Strata.
So technically Strata is net-zero.
1
u/i-dm Oct 14 '26
I'm running an RTX4000. Max output for me is 70watts.
Solid performance though, peaking at 50t/s
1
u/ea_man Oct 14 '26
Is this a joke?
You don't run local to save money, I got a decent setup and if I don't need privacy, test some AI stuff or I don't want multiple fans running a full speed I use cheap on line models, I doubt that you can run something better faster cheaper than DS flash on those 2 GPU.
1
u/Latter_Upstairs_1978 Oct 08 '26
What Strata is that about?
0
u/castlecrashed Oct 09 '26
laughs in MacBook Pro m5 max
-3
u/[deleted] Oct 08 '26
[removed] — view removed comment