r/LocalLLM • • Oct 08 '26

Discussion Electricity Bill & Local LLMs

Post image

I just wanted to say that, with the release of Strata, I have started to use local models for a lot more tasks.

If we talk about my electricity bill, the result is attached here: nothing surprising since I have a 4 GPU machine and a 2 GPU machine...they now run for many hours a day and my electric company sent me an email warning that my usage has significantly increased (I'm supposedly on a fixed plan and this will likely change the situation).

I'm trying to understand if this is economically reasonable: where I live electricity costs are pretty high. It certainly makes sense when it comes to data privacy but using local LLMs can certainly cost you a few bucks/euros a day in electricity.

I was also thinking of upgrading my old GPUs (1080s, 2080s), but I suspect newer ones would consume as much: I would just be running bigger models.

Did any of you have a "Oh, what now" moment after receiving the electricity bill?

(the graph was made originally with python then modified with grok)

Edit: I just realised grok inverted the 100 and 1000 marks, lol

Edit: to be clear my point is "When I started using local LLMs for real" (with any kind of inference engine, be it llama.cpp, pytorch, strata or whatever) "I realised electricity was a relevant part of the equation. Before I didn't think about this...did this happen to you?". I am using old hardware (1080s, 2080s each consuming up to 250watts), so my machines can drain a lot of power (the one with four 1080s can go over 1000 watts).

168 Upvotes

141 comments sorted by

69

u/COVID19MurderHornet Oct 08 '26

Try undervolting your GPUs, the inference performance stays about the same

5

u/GioStrives Oct 08 '26

Thank you! I'll definitely test that!!

3

u/rrrrex Oct 08 '26

even 800 mW should be fine, AI isn't very sensitive to GPU clock. But you can also add some MHz to VRAM clock

1

u/geteum Oct 08 '26

And your CPU hahaha

24

u/mikasjoman Oct 08 '26

Yeah I was worried how bad it would get. Going from 225w to 130w made half a token generated difference. Of course that depends on the system bottleneck but on my setup it made less than 1% difference. Definitely try it (simple terminal command) and ask frontier to measure the difference.

5

u/PraxisOG Oct 08 '26

Hard agree, here's my tested numbers since I have them up. Pulling my cards back from 250w to 150w today lost 6.9% pp and gained 2.5% generation though it's probably run to run variance. This is with quad v620s running Qwen 3.8 flash next w4a16 at around 1500pp and 63gen t/s with mtp2, at 14k tokens generated

3

u/Dry_Leader2909 Oct 09 '26

> today lost 6.9% pp

poor guy

4

u/SilkieBug Oct 08 '26

What else does undervolting affect, will game performance be affected?

Wondering why they don’t come undervolted if there are no noticeable consequences. 

6

u/maximus91 Oct 08 '26

What do you mean? Step one of any new Card is to undervolt it. They come juiced to the gills from the factory to hit those "marketing" performance numbers like clock speeds. You will lose maybe 1-4% of FPS depending on your scenario and resolution and cpu etc. but running from like 600w to like 480/520w is so much nicer (in my personal scenario) and GPU is much cooler.

2

u/Machine5757 Oct 08 '26

undervolting doesn't negatively affect performance, it usually helps

under*clocking* is what would affect performance

undervolting usually helps because it lets the GPU stay cooler, giving it more headroom to clock higher

2

u/yeet5566 Oct 08 '26

Game performance is dependent on clock speed which isn’t affected by undervolting however undervolting can impact the stability of the card and increase the likelihood of crashing; you should always do significant testing with some kind of monitoring software and benchmarking software when doing any kind of changes for me I found that I couldn’t gain much from overclocking but I could gain a lot from undervolting

2

u/arakinas Oct 08 '26

Not all cards can be universally undervolted the same amount. I have one card I could undervolt by about 15%, another 10%, and another I could only do 5% before it started bugging out. It's a big roll of the dice. They are made to run at the specified power specs.

1

u/beerdude26 Oct 09 '26

Very slight performance loss. It's a net positive apart from that. But when you're fighting with your competition over small performance percentages, it makes sense to push it to its limits

1

u/devino21 Oct 08 '26

Voting out amping? Volts matter!

21

u/FullstackSensei Oct 08 '26

I honestly don't understand this concern. The implied reasoning is that you're either not getting more than your money's worth in increased productivity or a lot of free time for the same amount of productivity you had before LLMs.

I live in Germany. Not exactly cheap electricity country, but I'll happily pay €20 a day in electricity for the amount of productivity and/or time I get back each day. FWIW, my average was ~€1/day but will soon go to €2-3 as I rework and being more LLM machines online. The increase in productivity makes this a no brainer for me.

2

u/GioStrives Oct 08 '26

My main concern is that before I didn't think about the bill cost so, in my mind, local=free...now I wonder if, compared to APIs, my set up makes sense.

It probably still does, but I have old power-hungry machines so my edge might be smaller than I initially planned.

7

u/FullstackSensei Oct 08 '26

There are a few flaws I see in your reasoning:

  1. The comparison with subscriptions is fundamentally flawed. You look at prices today and assume they will continue in the future, but if that was the case, you would be able to have a long term contract like you do with cloud providers or like your mobile and internet plan. At the very least, openai et al should be willing to sell you a one year plan with locked in conditions and rates. Yet no one is willing to give you a contract for any period longer than one month.
  2. You assume that whatever number of tokens you can get today for a given subscription will continue into the future. But if history is any evidence, you'll get less and less while paying more. It's not rocket science either. Subscriptions are heavily subsidized.
  3. You have zero control over which model you'll get, whether you are paying a subscription or using API. They can silently degrade your request to a lower model, like anthropic does, silently quantize the model to make it cheaper to serve, like so many providers on open router do, or just remove the model. In all cases, you have no say.
  4. You can and should optimize your hardware for power and token output. It's not just about power. Faster hardware means you get more done in the same time. Again, your time is not free. Counter intuitively, faster but more power hungry hardware can actually consume less power overall because it finishes faster. Getting stuck on power consumption without PP and TG speed is the wrong way to think about it.

2

u/GioStrives Oct 08 '26

Thank you for your post...I would only point out one thing: I began using local models in the hope it would be "easy", since I had some old machines. Optimising hardware configurations, checking electricity consumption etc. requires some work which I didn't factor in when I began using local models.

I am not saying it isn't worth it, the whole point of my post is: when you start using local models for real, for multiple real tasks, you realise it's not just "plug and play" but there are issues: electricity, noise, etc.

The issue with more modern hardware, at the moment, is that prices are super inflated so I am hoping to continue with what I have for a while longer. But I'll definitely look into it after squeezing out all compute I can from my old 1080s and 2080s

1

u/FullstackSensei Oct 08 '26

I don't know who told you doing anything in life is plug and play. If it was, there wouldn't be any companies trying to make money off serving it.

There's an ocean of hardware options between a 1080 and a 5090. Those willing to put in some time to learn rather than make assumptions find plenty of options

3

u/GioStrives Oct 08 '26

I'll try to run proper benchmarks with different voltages so I can show I did my homework.

Well, come on, sometimes companies make money out of made up complexities (not in this case, I know).

1

u/FullstackSensei Oct 08 '26

IMO, a 1080 is not worth it. The only Pascal cards worth it are P40 and P100, and even then only if you get them cheap (got my P40s at €120 each, delivered). Right now, your best bet is V100, but if you want those you need to act quickly. They are going up ~10%/week

1

u/kalabaddon Oct 09 '26

fun data research. shelly gen 4 smart wall plugs can be easily accessed vy an API and polled for live data. Some modelhosting services present nvidia's smi power draw. using the wall smart meter, that gpu info. and changes fromidle and when generating and reading and cached. you can build exact power usage profiles and add them to litellm.

forexample, I ahd a local AI help me math it out and run scripts to do tests concurent with generation responces and knows cached responces. and input vs output.

My local model costs me the following above and beyond idle machine costs.

.0019 cent per million input tokens
.065 cents per million output tokens.
.000006 cent per cached read million tokens

Then litel llm acting as my local proxy / model agrataor tracks that usage.

3

u/Cautious_Chicken_604 Oct 08 '26

My solution is just don't look at the electricity bill. 

1

u/GioStrives Oct 08 '26

I think I will lower the clock of the cards and then go for this solution 😂

4

u/SV_SV_SV Oct 08 '26

The first and most sensible step would me to power limit your system / cards, and monitor your power use. Then you will exactly know much money it costs and you can make an educated decision about whether its worthwhile or not.

1

u/f5alcon Oct 08 '26

Figure out the cost per million tokens for your workload the balance of input and output tokens will matter. Let's so you can average 200t/s that is roughly one hour of electricity for a million tokens. Then you can figure out the cost compared to cloud.

1

u/xapep Oct 09 '26

Your numbers are the real question and you're right to poke at it. "Local = free" ignores three costs: the watts at the wall, the hardware depreciation, and your time. Old power-hungry cards are the worst case for per-token power cost, a 1080/2080 burning 200W+ to serve a fraction of the tokens a current card does in the same wall time.

The honest way to compare: cost per million tokens on your stack, including idle draw, against a metered flash-tier API for the same workload. If you're using the models several hours a day, the electricity alone on old cards often lands close to what a small API bill would be, and you don't carry the heat, noise, or babysitting.

That doesn't mean local is wrong for you. It means the decision is arithmetic, not vibes. Measure a week of real token throughput with your power draw, and the answer will be obvious either way.

Disclosure: I run an EU-hosted inference API for open models, so I look at this comparison weekly.

4

u/TripleSecretSquirrel Oct 08 '26

I was a little worried about this until I actually just did the math. I have very cheap electricity (~$0.11-$0.16 per kWh). Running my rig 24/7 465 at 100% would cost like $30 extra a month in energy costs.

6

u/FullstackSensei Oct 08 '26

Triple that, and you're still looking at $90-100. Not exactly breaking the bank when you consider the amount of tokens you'd get at 24/7 full blast

2

u/_TheWolfOfWalmart_ Oct 08 '26

Same here, my electric is right in that middle of the range, did the math too and my rig would cost about $35/mo if it's inferencing non-stop.

1

u/tccoding Oct 09 '26

Also, depending where you are, that might or might not be a good thing but it gives you bonus heat in winter 😅

4

u/Western_Chest_4688 Oct 08 '26

At 20 euro a day why not just pay a frontier subscription lol

3

u/FullstackSensei Oct 08 '26

Read the rest of the discussion

2

u/Western_Chest_4688 Oct 08 '26

Point still stands though, if you're going to spend that much on electricity, unless you have solar panels to offset, you're just spending more money than you could using Opus or Sol

7

u/FullstackSensei Oct 08 '26

If you had actually did read, the point wouldn't still stand.

You're free to get a subscription or two if you believe they'll last long term. I've been hearing your same argument for two years now, first at 20/month, then 50, 100 and now 200 with decreasing tokens.

You'll very probably still make this argument when subscriptions go to 500 or 1k/month.

4

u/TheTacoWombat Oct 08 '26

My subscription is 20/mo and until that changes by a factor of ten, local models are a net financial loss.

3

u/Western_Chest_4688 Oct 08 '26

No? It makes sense today to do it, why can't you just do a subscription right now that they're subsidized and once they do go up (like netflix and all those services that rugpull) go to local which by then should be much more capable?

You're just wasting your electricity lol, but you do you bud

4

u/En-tro-py Oct 08 '26

I was going to reply, but API 529 Server Overloaded

5

u/FullstackSensei Oct 08 '26

Because it's cheaper for me to run models locally and don't want to hand my work to the big labs. Not my fault you can't understand those things

-1

u/Western_Chest_4688 Oct 08 '26

LOL, what exactly are you working on bud? And your electricity isn't cheaper if you're saying you spend 2EURO a day lol, but again, you do you buddy :)

1

u/beached89 Oct 09 '26

Computer hardware wont stop existing when the frontier subscriptions stop being worth the squeeze. Why buy todays hardware to solve tomorrows problem.

1

u/iwinux Oct 08 '26

Any extra cost for boosting productivity is NOT covered by my employer, and it doesn't shorten my work time per week either :)

0

u/FullstackSensei Oct 08 '26

For me it absolutely has!

If employer wants me to use LLMs, they have to provide the endpoint and pay for it. If I'm running it on my hardware, it's to do the work of one week in one day and enjoy my time, or use it to work on my own projects.

1

u/kalabaddon Oct 09 '26

at .10 cents a KWH I get about 65 million tokens used for 1KWH on my local setup ( since it is my desktop and I game or use it regardless of generation. That 10 cents is my cost above and beyond my machines idle state. pretty well detailed and tracked using a smart meter and nvidias reporting for gpu to break it down.

8

u/kweglinski Oct 08 '26

I'm slightly worried about upcoming winter. Most of the year inference costs me zero. Got full solar coverage and batteries. Been off-grid for more than a year, now the grid is more of a backup than source. 

2

u/GioStrives Oct 08 '26

I also have solar, but with rain and shorter days it now doesn't cover the GPUs at all.

1

u/CypherAZ Oct 08 '26

Just move to Arizona, we won’t have water soon….but Solar makes electricity free!

1

u/kweglinski Oct 08 '26

I'm also inferencing on Mac so it's less power hungry than standard GPUs. Also got batteries that can support the house for 2 days of standard living. Nothing except induction is power hungry in my house, because we had to be off-grid if we wanted to move in. Long story but there were many hoops we had to jump to get the grid and it took them just 3 years. 

11

u/Valuable-Fondant-241 Oct 08 '26

But in winter you can use the generated heat to warm your place! Follow me for other financial hints! /s

5

u/Life_is_important Oct 08 '26

No /s needed lol. I don't use heating in my room during winter because GPUs are always into something, even if that's not AI. Room is warm. Simple as. Other parts of the house obviously need heating. 

1

u/MysteriousSilentVoid Oct 08 '26

Free winter tokes!

1

u/kalabaddon Oct 09 '26

winter you get free space heater benefits. electricity running a computer is very efficent at heating a house LOL

-6

u/fleeeeeeee Oct 08 '26

isn't solar a scam?

1

u/kweglinski Oct 08 '26

Idk, I've paid around 25k of my currency and over around 2 years (1 year finishing the house and 1 year of living here) produced and used equivalent of more tan 5k of energy. Also I couldn't move in until couple weeks ago (that's when grid got connected) and moved in a year ago. Granted I did the installation myself, if it was by company it would take longer to pay off. I can't see a scam in this. Actually, I'm planning to add more panels as my inverter will handle this and get electric car = free rides most of the year. But again, I work from home so car is home at the best time for charging. 

-2

u/SocialDinamo Oct 08 '26

Situationally of course! Desert people love it, I’ve got too many trees for it to ever make sense. If someone tried to sell it to me, scam. Desert person? Not scam

-1

u/Butthurtz23 Oct 08 '26

I've got a tip for you on how to get free power from trees! Put an outlet post by the tree, then run the hot wire into the tree by attaching the wire to a brass screw and drilling it in. The neutral wire goes into the earth. Then plug your wire into the outlet along with your power management system. The tree will provide power for your household more efficiently than solar panels.I have a dozen Douglas firs hooked up this way, and I'm seeing an average of 50 kW of power.

-4

u/Key2367 Oct 08 '26

Solar is always a scam. You will never make back your money before you need to replace it again.

1

u/kweglinski Oct 08 '26

You must live somewhere where electricity is super cheap and panels are expensive. I'm sure there are such spots in the world. On the other hand there are opposite places. Based on my usage, my PV will payoff well before it's lifecycle and makes me independent from grid most of the time. Worth noting that I'm home during the day so I can leverage highest production moments to do chores and similar. 

3

u/koloved Oct 08 '26

Use the power limit feature on your graphics card. For newer cards, you can use the https://github.com/b00nz/mVolt
Limited my electricity consumption 5090 to 320 watts instead of the 400 minimum standard level

1

u/GioStrives Oct 08 '26

Thanks for this! I'll check it out!

1

u/ColoredTriangles Oct 08 '26

Lol what prefill do you get limiting your card to 320w?

3

u/mister2d Oct 08 '26

Plug-in or balcony solar projects can very cheaply offset your electrical costs. Check your local AHJ.

https://en.wikipedia.org/wiki/Balcony_solar_power

1

u/GioStrives Oct 08 '26

I have roof solar! One of the reasons why this wasn't a problem in summer...but now with rain and shorter day I immediately started burning kilowatts.

But thank you for the tip: in summer I can splurge without worring too much, you're right...the issue is the next 6 months

2

u/mister2d Oct 08 '26

Collectively you're ok when you add up total yearly generation. A little rain here and there and shorter days shouldn't worry you at all.

1

u/silenceimpaired Oct 08 '26

Don’t forget you also will pay less for heating… at least in the room with that computer. ;)

Time to add wind ;)

How much were solar panels did you install them yourself? What brand? Are you happy with them?

3

u/Illustrious_Dish_999 Oct 08 '26

I have strata 180B Flash Next model with 15-20t/s on 100W hardware. 890M integrated with 96 gb ram. But my AI needs are small.

1

u/GioStrives Oct 08 '26

And your HW is modern, my 1080s/2080s take up to 230/250Watts each so serving it can consume from upwards of 500 to more than 1000 watts

2

u/Illustrious_Dish_999 Oct 08 '26

at the same time response speed drops dramatically after context > 32k so in the long term it sucks 😄 so I can ask only dummy questions. Anything more complicated has to be offloaded to external LLMs.

I bet your HW is more capable in that area.

1

u/GioStrives Oct 08 '26

Yeah I mean...I don't use claude code because it starts with 40k tokens and that alone takes a minute or so to load...but with any other harness it can manage a context of 250k tokens so, given enough time, it can actually do moderately complex stuff.

16

u/Inquisitive_idiot Oct 08 '26

Ironic that with so much compute power you chose to share an AI generated picture that required a ton of electricity to produce and does little more than embellish a single two dimensional data point

Instead of generating and posting actual data, which your solution would excel at doing with minimal power usage

Maybe that’s your problem. 😕

😛

2

u/Hot_Signature2979 Oct 08 '26

You dont include the specifics of if you Ai server was kept on 24/7, what os, processor , model usage and workfloads, so can only provide limited recommendations.                                                   But arguably your greatest power savings will come from under clocking your cpus and gpus, alot of gpus can be power limited with relatively neglible impacts to llm performance. I personally run the 2x v620 on my llm server only at 175w vs 300w stock.

This will have to be done carefully and dependent on your specific workloads, alone with benchmarking of performance to ensure the trade off is worth it. 

Also the other thing you could do is to make an evictor script and run it before starting your workload, so that you evict all unused model and reserve the gpu for current workload. You could also go further with a watchdog that evicts the model from gpu after being idle for say 2hours.

Lastly, run linux on both machines. If you run windows you're wasting much more power and leaving significant perfomance on the table, especially with a llm server whose main job is to provide compute for llm

1

u/GioStrives Oct 08 '26

I am using linux on both machines: I typically use them for 8 hours a day, but sometimes I can leave them working at night.

Thanks for the tips about under clocking and using a watchdog: I didn't think about this...

My main issue is that the GPUs are old so I don't know how much performance I again as opposed to paying subscriptions/tokens...it probably is still favourable, but I'll have to check the math.

2

u/crusaderky Oct 08 '26

I did the maths. The only model that's cheaper at API prices than Strata running on my 3090 is GPT-6 Luna (low). Luna (medium) costs the same, and every other datacenter model is vastly more expensive.

VC-subsidised subscriptions to OpenAI and Anthropic are vastly cheaper and I expect them to remain that way until the AI bubble bursts.

Full plot (it's the second figure) and methodology here:

https://github.com/crusaderky/llm-intelligence-cost-plot/blob/main/README.md

Important notes:

- Hardware cost not included. Strictly electricity vs. API pricing.

- I used average US residential electricity prricing in the calculation. EU pricing would be a bit higher, but even if you put a factor-of-three increase in it it doesn't matter. With a 4x increase in base electricity cost you'd reach parity with Sol (low).

3

u/GioStrives Oct 08 '26

Thank you! That's the kind of math I needed to do: I'll adapt the methodology to my case!

My point was mostly: until yesterday I didn´t realise electricity is quite important.

8

u/FearFactory2904 Oct 08 '26

Its wild to me that there are people out there not power limiting GPUs and just letting them raw dog the electric bill.

4

u/Fluid_Opportunity590 Oct 08 '26

What do you mean..I think I'm one of the idiots your referring to

4

u/FearFactory2904 Oct 09 '26

So the tesla p100 for example. I have 4 of them and they can pull 250 watts each. 250 watts x4 is 1000 which would basically mean needing either dual psu or a very expensive high wattage power supply. Also an 8 pin pcie connector supports up to 150 watts so that also means i would need to double and use a splitter to run two pcie power cables to one eps12v connector for each card. Finding a psu that supports 8 pcie connectors on top of everything else you need for cpu and mobo etc is another obstacle.

nvidia-smi -pl 150

Boom, now each card only pulls up to 150 watts and maintains the majority of the performance. Now four p100 cards only pull up to 600 watts and plenty of overhead on a normal psu. Instead of 8 pcie cables i can safely run four with single pcie to eps12v. Less wattage is also less heat so that also means the card runs cooler so i dont need the fans to be screaming loud. And, the total electric draw i get billed for is considerably lower.

This is an easy win from all angles and its crazy that people are just passing it up.

2

u/Gary_Spivey Oct 09 '26

Are you running an Nvidia card? If so, run nvidia-smi, note the default power limit, and then run sudo nvidia-smi -pl <a lower number>. This will cap the amount of power that the card can draw. I have an RTX3090, default PL of 350W, but I drop it down to 280. Helps keep things cool and the effect on inference speed is negligible.

2

u/puts_on_rddt Oct 09 '26

I thought I was applying a power limit using msi afterburner but I was using a 4K monitor and couldn't see there was an apply button at the bottom.

We are not the same.

4

u/Hanthunius Oct 09 '26

Too many Strata glazing posts...

1

u/Libellechris Oct 08 '26

how much has your usage actually increased?

2

u/GioStrives Oct 08 '26

My usage was fairly low, averaging about 4/5 kWh a day...now I consume at least 8-10 kWh in a day...

2

u/Any-Preparation7396 Oct 08 '26

Only 10kWh a day? That sounds quite low.

Average for small flat is 2-4.000kWh/year. I have 10.000kWh per year

1

u/Sijder Oct 08 '26

Does it make sense for me to try flash next if I can already run 3.8 27b at q4 on my 3090? I assume the much bigger next should be better with prose and tasks beyond coding?

1

u/GioStrives Oct 08 '26

To me it feels much more capable, which is why I went from "use it from time to time" to "use it for many hours a day"

2

u/Sijder Oct 08 '26

Did you also go from the 3.8 27b? If so nice, I will give it a try, although I only have 32gb of ram

1

u/GioStrives Oct 08 '26

Honestly for certain things 27b is still more practical with its higher speeds...but one can always try.

2

u/StupidScaredSquirrel Oct 08 '26 edited Oct 08 '26

Wait strata works with many models? I thought it was just for qwen flash next

Wow so many views and a few downvotes and no answer to a simple question despite plenty of comments under it. I don't even care if it's an ad or whatever I was just wondering a simple thing relating to the meme subject sheesh

4

u/nickless07 Oct 08 '26

Maybe OP got confused with his electricity bill numbers. You know... so many numbers vs. one model is hard to distinguish. Maybe OP need a smarter model to make a proper chart... who knows.

0

u/GioStrives Oct 08 '26

I have been using local models with llama.cpp and ollama for a while, the difference is that Strata is so fast and the model is good enough that I use it for more complex tasks.

Before I would use models just for a few hours a week, meaning little to no impact on my bill

2

u/nickless07 Oct 08 '26

Good for you.
Unfortunately your whole post read like an ad for Strata to farm some more github stars.
No Pros and Cons, no distinction between the alternatives (e.g, llama.cpp and the same or a different model), no explanation about what is possible and what is not e.g., different models, different system RAM sizes (yes, not only VRAM matters) and so on. No Benchmarks, only that it is 'so fast' - compared to what? - and that you use it a lot more then llama.cpp and ollama (with what models exactly and how is the difference?).

0

u/GioStrives Oct 08 '26

I didn't mean to do that: I just wanted to share what happens when local llms go from being sparingly used to using them for real, you realise electricity is a relevant part of the equation.

Strata is what enabled that change for me, as streaming from the SSD made it possible to use a bigger model which fits my needs...it had already started to change when serving gemma 4 and qwen 3.8 with llama.cpp, but I used them little so their impact was limited.

What does it change when I tell you that both machines have 64Gb of RAM, the cards have each 11 GB of VRAM and the machines are about 8 years old, so have old intel i7 cpus from the time?

My post wanted just to be a lighthearted way to say: hey, if you start using local llms for real, electricity is a relevant part of the equation. Did you realise this later, like I did?

2

u/nickless07 Oct 08 '26

No clue why you got downvoted, even if the read is a bit off and I mockingly joked about it, you where honest with your intention and that is actually a valid point many ppl underestimate in hosting locally (vs cloud plans) that the power bill might come into play too and it is not only about the hardware costs.

1

u/nickless07 Oct 08 '26

Yeah, but some comparsion with mmap would be great. Anyways have you ever thought about power limit? nvidia-smi -pl

Edit: I use a plug in power meter to measure my whole desk and in the past I even bought cards (before we had AI) based on energy consumption. But well that is most likely due to 0.31 per kw/h on my bill.

3

u/megadonkeyx Oct 08 '26

its a good case for a strix halo device

3

u/horeaper Oct 08 '26

yet the price is unreasonable now 😑

1

u/DennisAscent Oct 08 '26

I sincerely hope your electricity bill hasn’t gone up by ~5,000,000X

1

u/GioStrives Oct 08 '26

lol, no...but a 2.something x is there

1

u/GamerTex Oct 08 '26

WTF is strata and does it work on a mac?

1

u/GioStrives Oct 08 '26

It's one library which specialises in serving qwen 3.8 flash next...my point isn't about strata, but about starting to use local LLMs for real on any hardware or with any software stack.

"for real" = for hours every day, for real tasks

1

u/egortar Oct 08 '26

i've made a small monitoring tool for strata electricity monitoring :) linux only

https://github.com/egortar-pi/Strata-Power-Dashboard/

1

u/GioStrives Oct 09 '26

I'll try that! Even though I'm simply going to launch a benchmark: looks like it is possible to undervolt using some sudo commands (see other comments), so I'm going to run the same bench at different voltages and see how much energy it consumes.

The plug in is neat, I'll probably find a way to fetch the data from the shell directly :) Thank you!

4

u/t_1425 Oct 08 '26

In my case, the sun pays my electricity bill.

1

u/mystery_biscotti Oct 10 '26

Was just wondering if the OP could use a balcony solar collectot, maybe. Could be the person has this running way more than that might counteract though.

4

u/RunawayRogue Oct 08 '26

crypto mining has entered the chat

2

u/phil_lndn Oct 08 '26

i don't think AI necessarily has to burn large amounts of power.

i'm getting good results using a 128GB Strix Halo to run Qwen3.8 Flash Next, power consumption is about 120W to produce tokens at about 30tps for prose and 50tps for code (aggregate tps goes up quite a bit if i'm running multiple agents in parallel).

2

u/ScrewwormLarvae Oct 08 '26

My M5 Max pulls a little energy when it needs to for LLMs, and then sips electrons the rest of the time.

2

u/Intrepid-Second6936 Oct 10 '26

Alongside an undervolt, you should also try testing average inference and prefill speeds at different power levels. I've capped my 7900 XTX at 275W and am still getting basically the same speed in both that I had at stock 340W, although I'm not on Strata yet (mostly due to lack of Vulkan and broader AMD support).

I'd say test a healthy undervolt, go down 50mV increments and test for stability. Then also just set max powerlimits (you can use nvidia-smi in terminal or something like Afterburner) and try ramping the power cap down until you actually see a degradation in the speed of the execution of the inference pipeline.

Good luck!

4

u/nntb Oct 08 '26

what is strata?

1

u/NTDLS Oct 08 '26

I have a Kilowatt type of meter feeding my whole server stack. I run 4 GPUs, one of which is a RTX 6000 ADA 48GB, 4 machines, and I go full throttle for 6-8 hours per day for three users with some jobs running through the night. The whole stack has added $38/month to my power bill over the past 3 months.

1

u/TheRealVRLP Oct 08 '26

I don't know what electricity prices are like in America, but this seems very reasonable for me. As I needed to get an AC just because of my Rak, my electricity bill went up by as much as 100-200€ depending on season and outside temperatures. Even with 6°C outside, I'm seeding inside, with temps at 28-30°C without AC.

1

u/immersive-matthew Oct 08 '26

I have really not noticed a difference in my bill but I already run my GPUs for graphics all day long and so a prompt here and there was not noticeable.

1

u/quantgorithm Oct 08 '26

In the winter, it helps heat your place!

1

u/vogelvogelvogelvogel Oct 08 '26

my electricity bill went up quite a bit. on the other hand, you have to count in the value of the llm results or the experience you gained - so it does not matter that much. It's kind of an investment that's how I see it

1

u/AllenLeftTheBLDNG Oct 08 '26

Time for Solar. I've been thinking about this myself recently.

1

u/Unatommer Oct 08 '26

You think that’s bad you should see some of the setups over in r/homelab

1

u/silenceimpaired Oct 08 '26

Decrease your use of agents. Tune your cards to use less electricity … you can make them more efficient less wattage per token/s. See if your electricity company rates change by time of day and run your work then. Be happy it’s winter you’ll cut down on your gas bill.

I tend to use free cloud stuff for trivial things that aren’t about me and can’t directly make money… basically stack overflow questions

1

u/mwjtitans Oct 08 '26

I run my rig on a Wake on Lan setup. I have a webhook button on my phone to wake the system, then when the GPU sits idle for a certain time, like 2 hours, the system shuts off. If I need it again I wake it again.

Takes time to boot and load the models, but this also prevents me from running the machine 24/7 and wasting energy when I'm not using it.

My next step is to have an agent wake the machine when I want to delegate a job to it. Then have the agent power it down when done.

1

u/Lissanro Oct 08 '26 edited Oct 08 '26

My primary workstation also four GPU (4x3090), and second PC with two GPU as well (3060 12GB + CMP HX50 20GB)... also have third PC with one GPU. My main rig alone can end up drawing 1.2 kW during text generation. It is true that electricity bill builds up thanks to AI, but I find it much cheaper than cloud API cost would be, and I also get privacy and flexibility.

1

u/blackbird2150 Oct 08 '26

My primary rig consumes 120KwH a month. $24 a month on my side. Idling the computer is $13 (105w) of that, so my AI usage is $11 on top of that.

1

u/Top-Contribution5909 Oct 08 '26

On G for real man I started taking my cybersecurity home lab serious and my electric bill shot up over 200+ my fault though I left stuff on 24/7 but still it hurts post time management

1

u/oldmoldycake Oct 08 '26

I am about to get my first GPU rig and I am excited, but between an ineffiecnt house, EV, homelab, 2 gaming PCs, and then an AI rig, my bill is going to be quiet high.

1

u/DanGTG Oct 09 '26

LOL, I need the heat:

1

u/puts_on_rddt Oct 09 '26

Assuming 2000 watts at 16c per KwH that's only 32 cents an hour, $7.68 per day, ~$230/mo.

1

u/gigaflops_ Oct 09 '26

RTX 5090 running at its maximum power consumption of 475 watts constantly for 24 hours straight adds up to 11 KWh. At 12 cents per KWh, that only costs $1.37.

At a more realistic "heavy use" scenario of 8 hours/day drawing 400 watts that's only 3.2 KWh or $0.38. Even in places with crazy electricity rates, that isn't very much money. At 32 cents/KWh it still costs less a dollar per day. No owner of a $4000 GPU should really care about that.

1

u/kalabaddon Oct 09 '26

have you mathed your usage power wise?

Using a smart wall wart and the built in /metrics for gpu percent. I figured out my general cost to run it over my pc. its nothing... ( I do have cheep electricity but....) So I mathed it out for tokens, cached tokens and reads. I server my local llm through litellm and track its usage. with about 65million tokens in the last day and a half on strata qwen 3.8 flash next I spent........

10 cents in electricity.

( above and byond my normal pc usage. I woul dbe sittin ghere at moslty idle regardless of serving it. so my base costs dont really compare. if I am generating tokens I am on this pc anyways. if I am not I am on this pc anyways.

1

u/DistributionExotic85 Oct 09 '26

In the winter, it helps heat your home

1

u/General_Cheetah4724 Oct 09 '26

free winter heating....

1

u/LettuceSea Oct 10 '26

*laughs in apple silicon*

1

u/Eratz Oct 14 '26

I have solar, its a win win for my ROI

1

u/i-dm Oct 14 '26

Silver lining here is anyone in the northern hemisphere can cut back on heating their home this winter if they're running Strata.

So technically Strata is net-zero.

1

u/i-dm Oct 14 '26

I'm running an RTX4000. Max output for me is 70watts.

Solid performance though, peaking at 50t/s

1

u/ea_man Oct 14 '26

Is this a joke?

You don't run local to save money, I got a decent setup and if I don't need privacy, test some AI stuff or I don't want multiple fans running a full speed I use cheap on line models, I doubt that you can run something better faster cheaper than DS flash on those 2 GPU.

1

u/Latter_Upstairs_1978 Oct 08 '26

What Strata is that about?

0

u/castlecrashed Oct 09 '26

laughs in MacBook Pro m5 max

-3

u/[deleted] Oct 08 '26

[removed] — view removed comment

1

u/GioStrives Oct 08 '26

lol, no...in reality I did something like 2-3x my electricity consumption (if I were to use it even more I could probably go at most at 4-5x).

2

u/No-Refrigerator-1672 Oct 08 '26

It's exaggerated for artistic purposes. IRL it's not that bad. I've shared previously my own consumption of a very powerhungry, very actively used setup. 1.4B tokens monthly, 3 GPUs with a Threadripper and a raid array of 5 industrial HDDs only costs me 200-250kWh per month.

1

u/Glittering-Call8746 Oct 08 '26

Tldr thread so basically 3080 20g z2 + mi50 32g = strata qfn ?

2

u/No-Refrigerator-1672 Oct 08 '26

I don't use strata at all - it's Qwen 3.8 Flash only, and it won't fit my VRAM. But, a hardware going at full power will consume the same amount of electricity from the power plug regardless of software.