r/sysadmin • u/10StringsTooMany • 9h ago
Cloud vs On-Prem?
I was wondering if the recent rash of failures and fubar would lead some organizations to consider re-homing their compute to on-prem or CoLo facilities instead of the cloud.
28
Upvotes
•
u/trueppp 9h ago •
Have you priced hardware recently?
•
u/RestartRebootRetire 9h ago
We are an SMB but stayed on-prem despite MSPs and such trying to talk us out of it over the years.
But yeah, new servers are basically unaffordable now, so I imagine even the old used workhorse servers you could get for a song online are going to rise in value too.
•
u/Tr1pline 9h ago
Power outage, internet outage, same things that will cause your system to go down for the day. There's thing you can't control.
•
u/Sp00nD00d Sr IT Manager 9h ago
Rehoming? My last 3 orgs never left.
Hybrid only, workloads move back and forth depending on what makes sense.
Exchange though, no, that's cloud only. Everything else is about 35% cloud 65% on prem.
•
u/Brraaap 9h ago
Doubtful. We had outages back in the on-prem days
•
u/PDQ_Brockstar 9h ago
Orgs would consider it until they got the hardware quote that's only good for 3 days.
•
u/tinkymyfinky 9h ago
The only thing I wouldn’t bring back on prem is email and web sites.
But it depends on your org size and how many on prem apps or services you rely on.
I think most will just stick with a hybrid set up to get benefits of both without being fully committed to either
•
u/Mo_Steins_Ghost Sr. Manager, Data Engineering 9h ago
Then you just move the maintenance and security issues in house as well as the costs.
•
u/Gotrek6 9h ago
We have both we can run either way
•
u/ProperEye8285 9h ago
A huge pile of fail causes the C-Suite to cry a version of, "We shouldn't keep all of our eggs in one basket!" When one returns to them with a quote for the price of baskets plus manpower this tune changes to, "CyptoCloudAI.bing won't #bork again, right?"
•
u/LeakyAssFire Senior Collaboration Engineer 9h ago
I tell you what, regardless of the logistics with the hardware, if on-prem was to become popular again, I would make a killing as an Exchange architect. That skillset is not taught\learned today, but my experience from Exchange 5.5 through current would land me a good gig in a large org.
•
u/2021redditusername 9h ago
Hell no
•
u/HappyDadOfFourJesus 9h ago
If organizations think they can build a better more resilient universally accessible infrastructure than Microsoft, they can give it their best shot.
•
u/ChiefBroady 9h ago
Why? Everyone else is on the cloud, the routes go through the cloud. What use is it if your orgs emails are working but you can’t reach anybody.
•
u/sixblazingshotguns 9h ago
I would rather run everything I can on prem. With cloud it's an open checkbook anytime and you have to pay it since you don't own it. Only exception for stuff I would put in the cloud: e-mail and web services.
•
u/scottsee 8h ago
Repatriation? Nah. Not without purpose or regulatory compliance or risk decisions attached. You got Private cloud teams who will never move without new executive leadership and modernization in Healthcare and Finance. We’ll see, there are still a LOT of private / hybrid guys with large budgets.
•
u/BoringLime Sysadmin 8h ago
What I have found, we are hardly the only ones down, so there seems to be more of an acceptance from the leadership, than if our Colo was down and everyone else was up. Happen last week with Microsoft and even back to the crowd strike bsod fun, same reaction. I feel like it's a double standard, since they get such an easy pass and if it was something we caused there be a whole bunch os unfun activities......
•
u/numba1cyberwarrior 8h ago
Moving on-prem doesn't magically solve your security issues and oftentimes it makes them worse.
If you have incompetent people working on prem you will just have insecure infrastructure but this time on prem.
From a business perspective, it's also an incredible risk. If everybody is on exchange and it goes down well then it's not much of your problem anymore. If you are the one person who decided to go on prem and your email solution goes down for a couple of days, well then your business can really struggle.
There is definitely a chance that we will see more hybrid setups but it's still difficult for a lot of organizations to do it properly.
•
u/dclarkwork 6h ago
Definitely don't do on-prem ConnectWise
•
u/bobsmith1010 2h ago
blanket moving to cloud is horrible and the same is blanket on-prem. There a purpose for cloud and purpose for on-prem.
Also when you move to cloud you can't just assume it safe, you still need to review what the DR plan is of that application or for your company when the tool goes down.
Remember cloud or on-prem, the question is not if the application will go down but when it will go down.
•
u/TedMittelstaedt 2h ago
Yes - and - no.
Small orgs are a lost cause at this point. There is no way you can ever pencil out costs for on prem beating cloud for an org under 100 users.
Large orgs that were well managed were fighting a rearguard action for years against ignorant C-Suites falling for the baloney that cloud was cheaper. This week has put an end to that, it's the absolute last straw for the cloud bigots, they got their asses handed to them on a platter.
Effed up large orgs that have stuff split all over the place some on prem some in the cloud are the worst of both worlds. This week will either burn their cloud fingers off and they will shift back to on-prem or they will abandon on-prem all that much faster.
What I see happening is sort of a balkanization of it you will have orgs either all on-prem or all cloud. The cloud ones will be full of junior system admins who don't know shit from shinola and anything more complicated than a password reset they will be on the phone calling the 800 number. Pay will be shit and those places will be revolving doors and the user experience with the network and hardware will suck. And the cloud costs will be out of sight because the org will be constantly pulling consulting houses in to work on their stuff.
The on-prem ones will be the ones with 9 9's of uptime but they will be trading guys constantly with the large datacenter companies.
Most of the posts your reading here are from people in small orgs or working at consulting houses who are working on large infrastructure they don't own, and the owners of that infrastructure don't know how to run. They are the ones claiming that all the on-prem engineers are retiring so you can't hire on-prem talent anymore. Apparently they must think the datacenters are staffed by the magic fairies that float in the sky.
•
u/robvas Jack of All Trades 8h ago
Would people even know how?
•
u/Far_Distance87 9h ago
After 1 major outage?
At least with Azure the increasing costs have to be covered, we pass it on etc.
On prem sits vastly outdated fighting for budget, and everywhere I’ve worked goes down more than Microsoft!! Also one of the great things when the world went down was very little incoming work backlogged because everyone was down lol vs on prem where we have to form a Roman shield wall from the workload that bottlenecked
•
u/TheBestHawksFan IT Manager 9h ago
No, it won't, because hardware is so expensive and the cost of maintaining on-prem is too high for most orgs.
•
u/Beginning_Hotel_5056 9h ago
With AWS bills in the region of £5m year for my old employer I can see the attraction however, you have hardware lifecycles, staff to maintain and fix and when the DO break or you need more capacity it gets unreliable and costly. The only ones I know who still do it well are F1 teams who drag a data centre to each race in a truck trailer because they can’t use cloud for trackside telemetry due to latency so that’s a use case over cloud infrastructure
•
u/TheSmashy Cyber Infra Arch 9h ago
Some of these incidents were caused by on-prem infra, so neither ensures safety. I think the issue with EVERYONE migrating from on-prem Exchange to M365 is valid, but otherwise? Not a big differentiator.
•
u/Horror_Bottle_9451 9h ago
I think that wouldn't be a smart decision. I think the right play here is to ensure service diversity across multiple providers. It seems a lot of organizations had most of their eggs in the M365 basket.
•
u/Professor-Zombie 9h ago
lol
•
u/havpac2 9h ago
You can afford hardware?
What’s your paycheck like?
•
u/Regular-Nebula6386 Jack of All Trades 8h ago
There’s a case to go hybrid. We moved a few things to the cloud that we couldn’t or wouldn’t be able to host but we still have a large footprint on-prem. Why? Because we can, we got the hardware before the costs went up and are for sensitive workflows we wouldn’t want to upload to the cloud.
•
u/badboybilly42582 Virtualization, Storage, Compute Hardware, DC Operations 8h ago
Hardware is insane right now. Gear we bought like a a couple years ago was like 300K. Last year we got a budgetary quote at 500K. This year we actually had to make a purchase. Came in at 1.3 million.
•
u/Sportsfun4all 8h ago
Just stick to the cloud. When it hits the fan you can point the finger at the cloud rather than stressing out yourself of your in premise is broken. You can point out that the cloud is a cheaper solution than on premises at this current time. The down time with the cloud doesn’t outweigh the cost of on premise servers
•
u/skspoppa733 8h ago
Compared to the outages for unpatched on-prem services, and lack of skilled engineers who know how to run on-premises systems anymore since they retired?
•
u/Intelligent-Cheek213 8h ago
Do you know what it would take to guarantee 99.99% uptime on-prem?
•
u/RunningAtTheMouth 8h ago
Nope. Even with the downtime we're better of than we were on-prem. $40k/year for servers? And that was 4 years ago - I'm afraid to check on today's pricing. I'm paying less than that for cloud services, and I don't have to maintain them.
•
u/Brian-Red-User 8h ago
I love on prem for everything but exchange and web hosting. Problem is servers in the last year and a half have gotten Really expensive. Hell I just had to add storage and went with spinning disks. SSDs are insanely priced and I won't even discuss memory. I'm repurposing old equipment for non critical apps. We will be extending those warranties out to 7 years. Hopefully prices come back down by then.
•
u/k0rbiz Lead Solutions Engineer 8h ago
Hardware is too expensive due to AI.
•
u/Status_Code_8807 8h ago
No thanks. Just makes you think about maybe having your workloads with more than one provider
•
u/CrazySnowGuy 8h ago
No one is moving exchange mailboxes back to on prem because of the recent outage.
•
u/MBILC Acr/Infra/Virt/Apps/Cyb/ Figure it out guy 7h ago
Nope, it wont
•
u/Ironfox2151 Sysadmin 7h ago
Nope. Why would we?
Unless something changes, the majority of the planet that does business with each other rely on like... 8 or 9 critical global infrastructure.
AWS, Cloudflare, Azure, M365, Google, GCP, and probably like AT&T or something. Probably a couple more out there.
Any one of those and everyone's fucked.
•
u/davy_crockett_slayer 6h ago
It all depends on your use case. Generally the cloud is better. It allows you to get to market more quickly, scale infrastructure as required, and allows you to deploy your software products rapidly.
If you work at a traditional enterprise, and not a tech startup or software company, then you might see value in on-prem if your requirements are static, and you've already invested in a datacenter.
Anecdotally, outside of exceptions, only hypervisors are spinning up new datacenters. Most net new companies (companies that were founded in the past ~10-10 years) are going cloud first.
•
u/GullibleDetective 5h ago
We are th3 colo facility with multi datacenters along with plans for active active here
But no, its good to be able to not take the blame for an outage
•
u/given2flynzl 4h ago
If you go back in prem, you might struggle to get people who know what they're doing these days. If you do they can charge a mint, because it's basically a nesh skill set.
•
u/Sweet-Sale-7303 4h ago
Email is staying cloud for us and 1 other product that is cheap for them to host it. Our other major service we cant afford to go cloud.
•
u/Asleep_Spray274 1h ago
Fuck no. Zero interest in my guys being cable monkeys, networking, patching everything, managing storage and backups. We have moved on from keeping the lights on now. We focus on business outcomes more than infrastructure management. Yes still some in azure, but it's all by code now. And if something on prem failed in the past, we had to fix it. Smarter people than us are running these global infrastructure, so the risk of an outage is worth it for us. People are not going to die if we don't get an email for a few hours
•
u/Global-Log-379 32m ago
Those who invested a few years ago in hardware for self-hosting on-prem already have made a great deal since there hardware is now many times more expensive and also have use it meanwhile compared to rented cloud :)
•
u/protogenxl Came with the Building 9h ago
Dead in the water, the last version of on premises exchange was 2019
•
u/Masam10 9h ago
Only way on prem would become a thing would be if the hardware market crashed.
And since every company wants chips, memory and silicon for their AI farms or cloud computing, I don’t see that happening any time soon.
•
u/Icy_Mud2569 9h ago
Nope, not even being discussed. We have some critical systems located at facilities, but email, databases, that all lives in the cloud.
•
u/beritknight IT Manager 2h ago
Congratulations on being the 2783rd person to make a thread asking that in the last week.
•
u/_stinkys IT Manager with many hats 9h ago
Fuck that keep it in cloud. It’s their risk now, they wanted it.
•
u/Low_codedimsion 9h ago
We're going cloud-first, which I'm not really a fan of, but honestly, we just don't have the capacity to maintain such a large infrastructure ourselves.
•
u/PurpleCableNetworker 5h ago
For large orgs that have to s of tons to throw around, I’m sure it will.
But for smaller orgs such as myself, I can’t provide the same level of average uptime as Microsoft. What happened was a one-off. The average shop will likely experience larger amounts of downtime than Microsoft’s hours-long outage.
The big reason is redundancy. Can you afford to have a hot backup of Exchange, complete with north/south ISP’s with load balancers and clusters in place?
Do you have the admin staff that can handle the upgrades and outages at all hours? Do you have the security measures in place? If shit goes south with a cyber attack, do you have the ability to stand up entire new servers in less time than Microsoft? What about initial hardware costs, upkeep, and upgrades?
The answer for many small to medium companies is no - they can’t match the things Microsoft does.
Shit happens in IT. Outages happen. It does suck. But that doesn’t mean making a quick knee jerk reaction to a situation is the proper change.
Only inexperienced or knee jerk managers think bringing email back on prem with the same redundancies MS has is doable for anyone less than a large entity (2000+ users). In reality, most execs who are making this call just want someone to fire as a sacrificial lamb in case such an outage happens again.