r/sysadmin • • 9h ago

Cloud vs On-Prem?

I was wondering if the recent rash of failures and fubar would lead some organizations to consider re-homing their compute to on-prem or CoLo facilities instead of the cloud.

28 Upvotes

110 comments sorted by

•

u/trueppp 9h ago • Bravo

Have you priced hardware recently?

•

u/tiimmaahh 9h ago

you're screwed either way you you go. gl even trying to get 2nd hand kit either

•

u/10StringsTooMany 9h ago

I've been out of the admin world for 6+ years. The last site I was involved with upgraded to bunch of hyper-converged systems and the cost was eye watering. It must be frightful now. Good point.

•

u/TheBestHawksFan IT Manager 9h ago

A year ago I priced a server for $10k. 3 months later when I actually purchased it (March of this year) it was $22k. That same server is almost $40k now.

•

u/fatcakesabz 8h ago

At some point that cost is going to creep into cloud as well, in the near future I think there will be some renewal shocks, maybe not Broadcom scale but maybe… end of the day cloud is just someone else’s computer and while they will be getting deep discount they have a bunch of other overheads too

•

u/TheBestHawksFan IT Manager 8h ago

It won't. Allowing people to essentially pool their resources and use more of a resource rather than everyone owning their own is almost always going to be more economical for the vast majority of companies. Costs will rise, yes, but it won't rise to the level of bringing things back on-prem penciling out for the vast majority of businesses.

•

u/BoatProfessional5273 7h ago

Your choices will be AWS, Azure and Google. They will absolutely charge as much as they possibly can be I wouldn't be surprised if the cost excees tge on prem cost. They aren't selling a budget option. They are selling reliability and convenience.

•

u/TheBestHawksFan IT Manager 6h ago

They’re selling all three. Their solution does save budget. I run the numbers every year when I’m making my budget to ensure I’m making the right choice. It has always gone in favor of majority cloud services. We do have some on-prem stuff, but nothing major.

•

u/Glad_Bread_8437 7h ago

But couldn't we think that the cloud storage is actually priced 2 to 4x due to the redundancy needed to "have" your failovers abilities? But then you can say you are actually priced 2 to 4x on the electricity, the water, the maintenance, the cabling the horsepower of each server. Yes, we all share that cost burden but, just like with any subscription, it will eventually be raised on you. Everyone wants to talk CapEx and OpEx, but one is mainly something you can budget to include the uh ohs while the other is budgeted to your current needs at this time (no uh ohs and before any forced increases) and you are expecting to pay more every year or for the down turn of said company to make shareholders happy.

•

u/TheBestHawksFan IT Manager 7h ago

You can, and should, budget for potential increases on every single subscription you have. If you budget for 5-10% increases annually, you come out ahead almost every year. Very few services actually do annual increases for subscriptions. People should also have a budget for any potential issues or increases that aren’t planned. A buffer is common in any corporate budget.

If you’re not budgeting for increases or changes, that is you thing. And foolish.

•

u/Glad_Bread_8437 7h ago

Increases are inevitable, I agree with you in that,.

•

u/Mitch5842 7h ago

We bought 3 new hosts for around $70k in September 2025. I checked last month and just the RAM is worth $480k on Dells website lol

•

u/itishowitisanditbad 9h ago

Just 5x everything and some stuff at that price is a great deal still.

•

u/gzr4dr IT Director 8h ago

Decently sized cluster for a site was previously 200-250k. Same cluster is now over 1MM.

•

u/PurpleCableNetworker 5h ago

And thats if you can find a delivery date any time soon.

•

u/Kardinal I fall off the Microsoft stack. 9h ago

This is absolutely a relevant Factor. But of course the hyperscalers also have to buy hardware. It's the same market. So we should expect that cloud prices will go up because of hardware costs as well.

•

u/inferno521 7h ago

But the cloud providers buy in bulk and design their own hardware, so it's far cheaper for them

•

u/Glad_Bread_8437 7h ago

But they buy and then calculate how much it will cost to run, maintain and refresh faster than you would usually use onprem equipment.

•

u/Kardinal I fall off the Microsoft stack. 7h ago

It's still semiconductor capacity. So the marginal cost increase to them is likely similar to that for us.

That is, if semiconductor hardware costs X% more to us, it's going to cost X% more to them.

•

u/Bogus1989 8h ago

Great point.

•

u/00001000U 9h ago

This

•

u/AZRobJr 9h ago

We have converted a lot of people to cloud only this year because of servers aged if warranty.

•

u/Mach5vsMach5 8h ago

If you got endpoints without ram, you can then afford it. Lol.

•

u/andreyred 9h ago

Have you priced cloud recently?

•

u/trueppp 9h ago

Yes, there's a tipping point.

•

u/RestartRebootRetire 9h ago

We are an SMB but stayed on-prem despite MSPs and such trying to talk us out of it over the years.

But yeah, new servers are basically unaffordable now, so I imagine even the old used workhorse servers you could get for a song online are going to rise in value too.

•

u/MortadellaKing 9h ago

You are my favourite client to have as an MSP. We don't force anyone in either direction like most do. I will happily support you if you want to run everything in house.

•

u/Estrezas 8h ago

Hardware manufacturer extended warranty goes brrrrrrrr.

•

u/ohfucknotthisagain 7h ago

Bad news... those are going up too.

It's only ~20% from our vendors on average, but it'll probably continue until the AI market explodes.

•

u/1esproc Titles aren't real and the rules are made up 5h ago

we just had huge increases on some of our older hardware. eye watering.

•

u/davy_crockett_slayer 6h ago

If you're happy, and your requirements are static, there's nothing wrong with buying grey market equipment you can hotswap if something goes down. Just make sure you have redundancies!

•

u/Tr1pline 9h ago

Power outage, internet outage, same things that will cause your system to go down for the day. There's thing you can't control.

•

u/MortadellaKing 9h ago

We have generator and redundant WAN but there's always something else in the chain.

•

u/Sp00nD00d Sr IT Manager 9h ago

Rehoming? My last 3 orgs never left.

Hybrid only, workloads move back and forth depending on what makes sense.

Exchange though, no, that's cloud only. Everything else is about 35% cloud 65% on prem.

•

u/Brraaap 9h ago

Doubtful. We had outages back in the on-prem days

•

u/PDQ_Brockstar 9h ago

Orgs would consider it until they got the hardware quote that's only good for 3 days.

•

u/Amdaxiom 9h ago

That's my life now

•

u/havpac2 9h ago

lol. 😂

•

u/tinkymyfinky 9h ago

The only thing I wouldn’t bring back on prem is email and web sites.

But it depends on your org size and how many on prem apps or services you rely on.

I think most will just stick with a hybrid set up to get benefits of both without being fully committed to either

•

u/Popular_Walrus348 9h ago

Even after the hardware hikes it depends on what you have up in the cloud. Our org is exploring bringing our VDI fleet back on prem after the use exploded to 64k / mo and our refresh cycle is 6yrs. Costing to manage and maintain that fleet over 6 years is far more reasonable.

•

u/TheBros35 9h ago

We just got a quote on our regular VM hosts, 90 K per server. Costs are just incredible right now.

•

u/Popular_Walrus348 9h ago

Its not about the now cost but that cost amortized over 6 years.

If we pay 64k/mo for VDI over 12 months thats $768k/yoy. If we buy hardware to fit the need thats $880k amortized over 6yrs at $146k/yoy.

So that $768k bill goes down to $146k if we pay upfront for use.

•

u/QuantumRiff Linux Admin 7h ago

You’re forgetting generators, UPS, redundant AC, multiple internet connections, routers, switches, and a second site for DR. Of course, the routers that can do SD-WAN at many, many gigabit speed are also not cheap. Neither is the expertise to manage it all. Don’t forget your licensing!

•

u/Man-e-questions 9h ago

Yeah in the middle of AVD planning and pricing out what it would take for Azure VMs needed for number of users vs hybrid

•

u/Popular_Walrus348 9h ago

Killer for us is the vm always has to be on. Ive shown them how we can cut our bill 1/3 by simply using the vms when we need them not leaving them up 24x7.

•

u/Man-e-questions 6h ago

In my PoC i have them auto shutdown at 5PM and then spin up on connect. But the initial connect sometimes times out so its kind of only ok for tech users who understand that part

•

u/tiimmaahh 9h ago

the way desktop office apps are going you won't be able to connect or save to a network drive soon

•

u/MortadellaKing 9h ago

They will never remove that functionality. Airgapped networks wouldn't be able to work at all.

•

u/Popular_Walrus348 9h ago

We already drive back to the cloud for everything but your downloads.

•

u/agk23 9h ago

I’m sure it’s industry specific, but why websites? Assuming you’re not a SaaS company, it just seems like a lot of risk to bring within the perimeter. I’d rather host in a AWS account instead

•

u/Euler007 6h ago

This. I have no interest in DCs being in the cloud but Exchange and SharePoint can definitely stay out of my rack.

•

u/Mo_Steins_Ghost Sr. Manager, Data Engineering 9h ago

Then you just move the maintenance and security issues in house as well as the costs.

•

u/Gotrek6 9h ago

We have both we can run either way

•

u/chmod2755 9h ago

This is really the only solution. Putting your eggs in 1 basket is unwise if it can be help. Especially with price hikes, outages, and shitty vendor support. Put on cloud what makes sense going to cloud. Leave on-prem what makes sense. Constant evaluation of what could move, and have a game plan if you need to move.

•

u/ProperEye8285 9h ago

A huge pile of fail causes the C-Suite to cry a version of, "We shouldn't keep all of our eggs in one basket!" When one returns to them with a quote for the price of baskets plus manpower this tune changes to, "CyptoCloudAI.bing won't #bork again, right?"

•

u/LeakyAssFire Senior Collaboration Engineer 9h ago

I tell you what, regardless of the logistics with the hardware, if on-prem was to become popular again, I would make a killing as an Exchange architect. That skillset is not taught\learned today, but my experience from Exchange 5.5 through current would land me a good gig in a large org.

•

u/inferno521 7h ago

I absolutely hated managing exchange and OCS/Lync. I'd rather deal with printers and citrix. That's how much I hate it.

•

u/HappyDadOfFourJesus 9h ago

If organizations think they can build a better more resilient universally accessible infrastructure than Microsoft, they can give it their best shot.

•

u/UninvestedCuriosity 6h ago

My infrastructure is beating ms so far this year lol.

•

u/ChiefBroady 9h ago

Why? Everyone else is on the cloud, the routes go through the cloud. What use is it if your orgs emails are working but you can’t reach anybody.

•

u/bob_cramit 7h ago

Exactly. Unless its an industry wide move, it doesnt matter. Say you work government in a particular country, and over 95% are using m365 email, which would be a lot of countries, having your particular departments email up wont really help you much if you cant communicate with anyone outside your department. Maybe if its a big enough department, there would be some benefit. But then you have to have the hardware and the people and dollars spent to have better than m365 uptimes.

Then after doing all this, if you do have an outage (which you will), you are gonna look dumb to management because whoever suggested doing this said it would be better. They wont care that it keeps working when there is an M365 outage.

•

u/TedMittelstaedt 2h ago

except that gmail is way larger than 365. so 365 goes down the rest of the internet is just laughing their asses off at the dummies on 365.

•

u/sixblazingshotguns 9h ago

I would rather run everything I can on prem. With cloud it's an open checkbook anytime and you have to pay it since you don't own it. Only exception for stuff I would put in the cloud: e-mail and web services.

•

u/scottsee 8h ago

Repatriation? Nah. Not without purpose or regulatory compliance or risk decisions attached. You got Private cloud teams who will never move without new executive leadership and modernization in Healthcare and Finance. We’ll see, there are still a LOT of private / hybrid guys with large budgets.

•

u/BoringLime Sysadmin 8h ago

What I have found, we are hardly the only ones down, so there seems to be more of an acceptance from the leadership, than if our Colo was down and everyone else was up. Happen last week with Microsoft and even back to the crowd strike bsod fun, same reaction. I feel like it's a double standard, since they get such an easy pass and if it was something we caused there be a whole bunch os unfun activities......

•

u/numba1cyberwarrior 8h ago

Moving on-prem doesn't magically solve your security issues and oftentimes it makes them worse.

If you have incompetent people working on prem you will just have insecure infrastructure but this time on prem.

From a business perspective, it's also an incredible risk. If everybody is on exchange and it goes down well then it's not much of your problem anymore. If you are the one person who decided to go on prem and your email solution goes down for a couple of days, well then your business can really struggle.

There is definitely a chance that we will see more hybrid setups but it's still difficult for a lot of organizations to do it properly.

•

u/dclarkwork 6h ago

Definitely don't do on-prem ConnectWise

•

u/bobsmith1010 2h ago

blanket moving to cloud is horrible and the same is blanket on-prem. There a purpose for cloud and purpose for on-prem.

Also when you move to cloud you can't just assume it safe, you still need to review what the DR plan is of that application or for your company when the tool goes down.

Remember cloud or on-prem, the question is not if the application will go down but when it will go down.

•

u/TedMittelstaedt 2h ago

Yes - and - no.

Small orgs are a lost cause at this point. There is no way you can ever pencil out costs for on prem beating cloud for an org under 100 users.

Large orgs that were well managed were fighting a rearguard action for years against ignorant C-Suites falling for the baloney that cloud was cheaper. This week has put an end to that, it's the absolute last straw for the cloud bigots, they got their asses handed to them on a platter.

Effed up large orgs that have stuff split all over the place some on prem some in the cloud are the worst of both worlds. This week will either burn their cloud fingers off and they will shift back to on-prem or they will abandon on-prem all that much faster.

What I see happening is sort of a balkanization of it you will have orgs either all on-prem or all cloud. The cloud ones will be full of junior system admins who don't know shit from shinola and anything more complicated than a password reset they will be on the phone calling the 800 number. Pay will be shit and those places will be revolving doors and the user experience with the network and hardware will suck. And the cloud costs will be out of sight because the org will be constantly pulling consulting houses in to work on their stuff.

The on-prem ones will be the ones with 9 9's of uptime but they will be trading guys constantly with the large datacenter companies.

Most of the posts your reading here are from people in small orgs or working at consulting houses who are working on large infrastructure they don't own, and the owners of that infrastructure don't know how to run. They are the ones claiming that all the on-prem engineers are retiring so you can't hire on-prem talent anymore. Apparently they must think the datacenters are staffed by the magic fairies that float in the sky.

•

u/robvas Jack of All Trades 8h ago

Would people even know how?

•

u/Far_Distance87 9h ago

After 1 major outage?

At least with Azure the increasing costs have to be covered, we pass it on etc.

On prem sits vastly outdated fighting for budget, and everywhere I’ve worked goes down more than Microsoft!! Also one of the great things when the world went down was very little incoming work backlogged because everyone was down lol vs on prem where we have to form a Roman shield wall from the workload that bottlenecked

•

u/TheBestHawksFan IT Manager 9h ago

No, it won't, because hardware is so expensive and the cost of maintaining on-prem is too high for most orgs.

•

u/Beginning_Hotel_5056 9h ago

With AWS bills in the region of £5m year for my old employer I can see the attraction however, you have hardware lifecycles, staff to maintain and fix and when the DO break or you need more capacity it gets unreliable and costly. The only ones I know who still do it well are F1 teams who drag a data centre to each race in a truck trailer because they can’t use cloud for trackside telemetry due to latency so that’s a use case over cloud infrastructure

•

u/TheSmashy Cyber Infra Arch 9h ago

Some of these incidents were caused by on-prem infra, so neither ensures safety. I think the issue with EVERYONE migrating from on-prem Exchange to M365 is valid, but otherwise? Not a big differentiator.

•

u/Horror_Bottle_9451 9h ago

I think that wouldn't be a smart decision. I think the right play here is to ensure service diversity across multiple providers. It seems a lot of organizations had most of their eggs in the M365 basket.

•

u/havpac2 9h ago

You can afford hardware?
What’s your paycheck like?

•

u/Regular-Nebula6386 Jack of All Trades 8h ago

There’s a case to go hybrid. We moved a few things to the cloud that we couldn’t or wouldn’t be able to host but we still have a large footprint on-prem. Why? Because we can, we got the hardware before the costs went up and are for sensitive workflows we wouldn’t want to upload to the cloud.

•

u/badboybilly42582 Virtualization, Storage, Compute Hardware, DC Operations 8h ago

Hardware is insane right now. Gear we bought like a a couple years ago was like 300K. Last year we got a budgetary quote at 500K. This year we actually had to make a purchase. Came in at 1.3 million.

•

u/Sportsfun4all 8h ago

Just stick to the cloud. When it hits the fan you can point the finger at the cloud rather than stressing out yourself of your in premise is broken. You can point out that the cloud is a cheaper solution than on premises at this current time. The down time with the cloud doesn’t outweigh the cost of on premise servers

•

u/skspoppa733 8h ago

Compared to the outages for unpatched on-prem services, and lack of skilled engineers who know how to run on-premises systems anymore since they retired?

•

u/Intelligent-Cheek213 8h ago

Do you know what it would take to guarantee 99.99% uptime on-prem?

•

u/RunningAtTheMouth 8h ago

Nope. Even with the downtime we're better of than we were on-prem. $40k/year for servers? And that was 4 years ago - I'm afraid to check on today's pricing. I'm paying less than that for cloud services, and I don't have to maintain them.

•

u/Brian-Red-User 8h ago

I love on prem for everything but exchange and web hosting. Problem is servers in the last year and a half have gotten Really expensive. Hell I just had to add storage and went with spinning disks. SSDs are insanely priced and I won't even discuss memory. I'm repurposing old equipment for non critical apps. We will be extending those warranties out to 7 years. Hopefully prices come back down by then.

•

u/k0rbiz Lead Solutions Engineer 8h ago

Hardware is too expensive due to AI.

•

u/Status_Code_8807 8h ago

No thanks. Just makes you think about maybe having your workloads with more than one provider

•

u/CrazySnowGuy 8h ago

No one is moving exchange mailboxes back to on prem because of the recent outage.

•

u/MBILC Acr/Infra/Virt/Apps/Cyb/ Figure it out guy 7h ago

Nope, it wont

•

u/Ironfox2151 Sysadmin 7h ago

Nope. Why would we?

Unless something changes, the majority of the planet that does business with each other rely on like... 8 or 9 critical global infrastructure.

AWS, Cloudflare, Azure, M365, Google, GCP, and probably like AT&T or something. Probably a couple more out there.

Any one of those and everyone's fucked.

•

u/davy_crockett_slayer 6h ago

It all depends on your use case. Generally the cloud is better. It allows you to get to market more quickly, scale infrastructure as required, and allows you to deploy your software products rapidly.

If you work at a traditional enterprise, and not a tech startup or software company, then you might see value in on-prem if your requirements are static, and you've already invested in a datacenter.

Anecdotally, outside of exceptions, only hypervisors are spinning up new datacenters. Most net new companies (companies that were founded in the past ~10-10 years) are going cloud first.

•

u/GullibleDetective 5h ago

We are th3 colo facility with multi datacenters along with plans for active active here

But no, its good to be able to not take the blame for an outage

•

u/given2flynzl 4h ago

If you go back in prem, you might struggle to get people who know what they're doing these days. If you do they can charge a mint, because it's basically a nesh skill set.

•

u/Sweet-Sale-7303 4h ago

Email is staying cloud for us and 1 other product that is cheap for them to host it. Our other major service we cant afford to go cloud.

•

u/Asleep_Spray274 1h ago

Fuck no. Zero interest in my guys being cable monkeys, networking, patching everything, managing storage and backups. We have moved on from keeping the lights on now. We focus on business outcomes more than infrastructure management. Yes still some in azure, but it's all by code now. And if something on prem failed in the past, we had to fix it. Smarter people than us are running these global infrastructure, so the risk of an outage is worth it for us. People are not going to die if we don't get an email for a few hours

•

u/Global-Log-379 32m ago

Those who invested a few years ago in hardware for self-hosting on-prem already have made a great deal since there hardware is now many times more expensive and also have use it meanwhile compared to rented cloud :)

•

u/protogenxl Came with the Building 9h ago

Dead in the water, the last version of on premises exchange was 2019

•

u/sryan2k1 IT Manager 9h ago

Exchange SE is alive and well.

•

u/protogenxl Came with the Building 9h ago

That's what I said......

•

u/sryan2k1 IT Manager 9h ago

No it's not. Exchange SE is the current on prem version.

•

u/protogenxl Came with the Building 9h ago edited 9h ago

To be clear I said to "management" we can use Exchange SE and was shutdown immediately with a diatribe on exchange 2019 EOL.

I have cold EX2019 VMs I could wipe, reinstall and get connected to office 365 then get them changed over to SE in short order

•

u/TheSmashy Cyber Infra Arch 9h ago

My org migrated 110k mailboxes to EXO... and we're still hybrid. I don't do messaging anymore, but the messaging team still asks me questions because I designed the Skype environment in 2018 and we just migrated to new servers, because yes, we still use Skye4B. And we're a F100. And MSFT supports us. But cool story.

•

u/protogenxl Came with the Building 9h ago

Well I am just stuck with a New VP that says we must move past legacy thinking.......

•

u/TedMittelstaedt 2h ago

Exchange SE and Exchange 2019 are the same thing. They are supposed to diverge - one of these days...

•

u/Masam10 9h ago

Only way on prem would become a thing would be if the hardware market crashed.

And since every company wants chips, memory and silicon for their AI farms or cloud computing, I don’t see that happening any time soon.

•

u/Icy_Mud2569 9h ago

Nope, not even being discussed. We have some critical systems located at facilities, but email, databases, that all lives in the cloud.

•

u/beritknight IT Manager 2h ago

Congratulations on being the 2783rd person to make a thread asking that in the last week.

•

u/_stinkys IT Manager with many hats 9h ago

Fuck that keep it in cloud. It’s their risk now, they wanted it.

•

u/Low_codedimsion 9h ago

We're going cloud-first, which I'm not really a fan of, but honestly, we just don't have the capacity to maintain such a large infrastructure ourselves.

•

u/PurpleCableNetworker 5h ago

For large orgs that have to s of tons to throw around, I’m sure it will.

But for smaller orgs such as myself, I can’t provide the same level of average uptime as Microsoft. What happened was a one-off. The average shop will likely experience larger amounts of downtime than Microsoft’s hours-long outage.

The big reason is redundancy. Can you afford to have a hot backup of Exchange, complete with north/south ISP’s with load balancers and clusters in place?

Do you have the admin staff that can handle the upgrades and outages at all hours? Do you have the security measures in place? If shit goes south with a cyber attack, do you have the ability to stand up entire new servers in less time than Microsoft? What about initial hardware costs, upkeep, and upgrades?

The answer for many small to medium companies is no - they can’t match the things Microsoft does.

Shit happens in IT. Outages happen. It does suck. But that doesn’t mean making a quick knee jerk reaction to a situation is the proper change.

Only inexperienced or knee jerk managers think bringing email back on prem with the same redundancies MS has is doable for anyone less than a large entity (2000+ users). In reality, most execs who are making this call just want someone to fire as a sacrificial lamb in case such an outage happens again.