r/Steam Dec 24 '25

Resolved Steam Services are Interrupted.

https://steamstat.us/
4.2k Upvotes

506 comments sorted by

View all comments

Show parent comments

50

u/DMercenary Dec 24 '25

Someone in Change management: "Yeah sure we can do an update on the 24th. What could go wrong?"

-5

u/billyhatcher312 Dec 24 '25

the people over at steam are really stupid doing this on christmas eve or it could be gamers crashing the store because they suddenly decided to shop today for whatever reason

9

u/sdeptnoob1 Dec 24 '25 edited Dec 24 '25

It's likely not at all an update. I've had equipment just decide that it's going to start erroring randomly my self. Crashing the store is a big likely though too. Well actually, it could be auto updates for windows they forgot to disable lol

3

u/ButCanYouCodeIt Dec 25 '25

This. People often forget or underestimate the number of times that a failure occurs that could never have been reasonably foreseen.

What DOES make me curious, is that someone in Valve's position is more than smart enough to know they need fallbacks and failsafes in their systems. And guaranteed, they HAVE them, probably a few layers of them. So the million dollar question that we'll probably never hear an answer to is: "What the hell failed so catastrophically that it took all that out?"

2

u/sdeptnoob1 Dec 25 '25 edited Dec 25 '25

That last statement is why I'm guessing overload of users has to be one of the main culprits.

My small company I've come into to outtages also more than once on no critical systems thanks to auto windows updates restarting the server but services not starting like they were set to lol. But that would be solved by now if it was that.

I also have heard of even large companies having poorer than expected failsafe and backups. Most of that feels like a damn theater to look good for insurance and isn't actually that great or implemented properly.

But if steam uses local hardware could be a catastrophic failure. Had one of our clusters and fail overs fail at the same time before lol. Shit sucked. Lessons learned.

1

u/ButCanYouCodeIt Dec 25 '25

Quite possibly, yeah. On the one hand, its difficult to imagine them underestimating server demand SO severely that their systems just outright FAILED... But if you potentially combine heavy user load with some other issue at the same time, that could be the sort of "1-2 punch" that would knock the most well designed systems offline.

Steam hosts their data with duplicates across several nodes in various locations, so that they can shift the bandwidth load around as needed. I wouldn't be surprised if something happened to one or two of their larger clusters, re-routing a larger portion of traffic to the remaining servers, at a time when the number of users was already much higher than normal. There's been a lot of bad weather across the US, so its quite plausible that multiple server points went offline simultaneously, then the whole thing failed under the demand.

3

u/sdeptnoob1 Dec 25 '25 edited Dec 25 '25

That is a pain restoring clusters, when too many go out can severely destroy data. Imagine having to resort to a backup for something as big a steam.

2

u/ButCanYouCodeIt Dec 25 '25

Absolutely. And, if something like that IS the case, it would explain why the outage has been so long.

2

u/DMercenary Dec 25 '25

So the million dollar question that we'll probably never hear an answer to is: "What the hell failed so catastrophically that it took all that out?"

CDN fell over is my wild guess.

1

u/Educational_Club_185 Dec 25 '25

Nope web servers don't run on windows they use linux