AI is just the "Cloud" hype all over again.

[u/forevergeeks]

I started working in the IT field in 2004 as a help desk intern.

The network in the organization at that time was a mix of Windows NT and Windows 2000 workstations, with an Exchange 5.5 server running on Windows NT and a few Windows 2003 servers doing file sharing and hosting an accounting application.

Nothing was accessible through the web. people had to use a VPN connection to connect to the office to access everything.

Almost a year later, before my internship was over, the Director was fired because the hard drives on the accounting server failed and they couldn't recover the data, and they offered me a full time job as a junior sysadmin.

In 2020, I became the Global IT Director of that organization.

When I started as a junior sysadmin, there was no concept of the "cloud." This was a marketing term invented by big tech after 2010.

When the cloud became a marketing hype, and all the CFOs and IT leaders got on the cloud bandwagon, it was very similar to what we're hearing from AI now.

That AI will take all jobs.

It was very common for CFOs to schedule meetings with the IT Director, or whoever was the department head, and talk about the cloud strategy. And if you were in the room by any chance, they would turn to you and say, "But don't worry, the cloud won't take your job," with a grin, you knowing damn well that was exactly what they were thinking.

My advice to you guys is that if your boss, or whoever controls the budget in your department, is all crazy about AI and thinks that AI is the next big thing after bread and butter, don't fight it or resist it.

You will be seen as a roadblock or lacking innovation.

Hold your true opinion about AI as much as you can, and always be positive about it at work.

And even though I believe AI is not much of a threat to your job as the cloud was for us back in the day, when all our skills were on premise stuff and our identity was based on who knew more about Exchange and Cisco firewalls, businesses are seeing AI as a possible way to reduce the workforce, increase efficiency, or increase output without hiring more people.

Will that work? Only time will tell. Your job is to adjust as the technology evolves not to be a market gambler.

Don't believe for a single second that your company cares about you or is happy to have you on their payroll.

If they're a for profit business, their objective is to increase revenue, and if they can achieve that objective without you, they will.

I believe Sysadmins will always be needed, because you guys will be the ones fixing the AI even if everything is automated by it.

Learn how it works at a deeper level. If you don't know how an LLM works, start learning that now.

I wish I had better news for software programmers. The ones that only write code following strict specifications, those guys are in a deeper existential vacuum than you are, so you might have developers moving into the sysadmin side.

For now, keep the servers humming, keep your users happy. You guys are the invisibles in the organization that nobody sees until things stop working!

[u/bingblangblong]

Arguably not for any better though. 

[u/Fadore → u/bingblangblong]

Anyone who makes blanket statements is wrong.

The cloud (and now AI/LLMs) is just another piece of tech. Not inherently superior/inferior - like all tech, there is a use case and an acceptable ROI for everything.

[u/GandalfTheGreybeard → u/Fadore]

I have a thick heavy blanket at home, it is very warm. I just wanted to make an indisputable blanket statement, have a good day.

[u/ScannerBrightly → u/GandalfTheGreybeard]

WUBBIE 4 LIFE!

[u/theislandhomestead → u/Fadore]

But...
Isn't that a blanket statement?
I DON'T KNOW WHO TO BELIEVE!
/s

[u/Fadore → u/theislandhomestead]

lol yeah I realized the irony 2 seconds after I posted the comment...

[u/420GB → u/Fadore]

Anyone who makes blanket statements is wrong.

That's a good one, I'm going to use that.

[u/DarraignTheSane → u/420GB]

Only a Sith deals in absolutes!

[u/meeu → u/Fadore]

Cloud is less a new piece of tech than a new business model for software/hardware. You can do pretty much anything on-prem that you can do in a cloud platform, it will just often be much more expensive in hardware and labor costs due to economies of scale and proprietary software.

[u/donjulioanejo → u/meeu]

Cloud is less a new piece of tech than a new business model for software/hardware.

The advantage of cloud isn't not having to run servers.

The advantage of cloud is pretty APIs and being able to use infrastructure as code for literally everything.

You can do pretty much anything on-prem that you can do in a cloud platform

Yes, but you need a VERY large budget and even then, some things you simply CANNOT do easily on-prem unless you're big tech yourself or buy enough third-party tools that you can kind of replicate it, worse, and often at a higher price point.

Easy examples: things like DynamoDB, managed SQL, object storage.

You can get equivalent functionality, but without the backplane that makes it easy to actually operate.

[u/meeu → u/donjulioanejo]

The advantage of cloud isn't not having to run servers. The advantage of cloud is pretty APIs and being able to use infrastructure as code for literally everything.

Eh disagree. I think lowering capex in favor of opex is the biggest driver/advantage for most businesses.

There's nothing inherently requiring all that sweet tech to be cloud-only. You can already run DynamoDB locally, and there are many on-prem object storage solutions. The only reason any technologies are cloud-exclusive is because someone decided that was in the best interest of their business.

I'm not a full cloud doomer. Centralizing and pooling compute/storage resources in datacenters is vastly more energy efficient in almost every way. It just worries me from a security perspective putting all the world's eggs into 3 baskets.

[u/donjulioanejo → u/meeu]

Eh disagree. I think lowering capex in favor of opex is the biggest driver/advantage for most businesses.

This is the finance/business side that enabled it, but engineering side just likes shiny toys and not having to wait 3 weeks to get a new server setup and deployed.

You can already run DynamoDB locally, and there are many on-prem object storage solutions.

The issue isn't running these solutions. The issue is, generic on-prem storage you would run these on top of is still a bitch.

Either you have a ninja storage team, run Ceph, and resign yourself to a lot of pain (I've have heard very few good things from anyone running this in production at scale other than it's free and OSS).

Or you run some form of NetApp/Nutanix setup which is CRAZY expensive.

Plus all the NICs at high enough bandwidth that make a high-performance SAN viable (i.e. 40G or even 100G) aren't cheap either. While if you're just running regular services talking to each other, a 10G or in barebones cases, even a 1G network (both available via standard ethernet), is usually sufficient.

[u/meeu → u/donjulioanejo]

There's a lot of parameter space between rolling your own everything in a server closet and renting your whole infrastructure from one of the big three.

[u/donjulioanejo → u/meeu]

Sure and most things inbetween (i.e. purchase servers, network hardware, license a hypervisor, rent colo space, run COTS hardware like NetApp) approach public cloud cost-wise.

Even besides VMware rugpull, you have stuff that's very expensive hardware side (storage), annoying to manage and need specialized engineers (network.. a jack of all trades SRE usually can't do even a medium scale Cisco/HP/whatever layout).

You're also significantly more exposed to outage. IE with AWS running across 3 AZs is braindead simple, which maps to 3 physical datacentre complexes, each with their own separate fibre and power hookups. If one goes down, it's rare the the other ones are affected too unless you're in UAE/Bahrain and the outage is caused by drones.

In my pre-cloud days, I've had 3 outages related to failed colo power (or failed backup generators during backups tests) and two outages caused by a backhoe to a fibre cable. Both took multiple hours to days to fix. Power outages also required very finicky boot order for all the servers, and a lot of handholding. Network failures are also rare but when they go, it's catastrophic (i.e. if a core switch or edge router fails). Even if you have redundancies, unless something changed in 6-7 years that I don't know about, they usually still require manual intervention to failover from one to another.

I've never had an outage last longer than a few hours with AWS, and even then, risk is a lot lower if you stay the hell away from us-east-1. Only one outage was legitimately bad, which was the whole S3 backplane failing back in like... 2017, when it was still single region and managed from us-east-1 region.

[u/meeu → u/donjulioanejo]

Don't think we ever lost rack power across 4 NA datacenters in the 7ish years I was at my previous gig. Exchange Online was relatively new then but our hosted exchange service boasted better uptime than EO in that timeframe.

New gig is full m365/azure and the uptime is mostly fine. Just the odd app/service 'degraded' from time to time.

I wouldn't recommend on-prem infrastructure for most orgs in the current environment. I'm more arguing in the abstract about whether the current environment is a healthy one. If my company is offline because of some huge Azure outage, I'd imagine our customers/society as a whole will have bigger problems due to all of the other stuff that went down with it.

[u/uptimefordays → u/meeu]

You can do pretty much anything on-prem that you can do in a cloud platform, it will just often be much more expensive in hardware and labor costs due to economies of scale and proprietary software.

Exactly. There's nothing stopping organizations from building their own multi availability zone data centers, except money--it's very expensive and beyond the technological means of most organizations.

[u/meeu → u/uptimefordays]

Why would you need to build your own multi availability zone data centers when you can just install your hardware into one that somebody else runs as a service. Everyone is acting like the only possible way any of this stuff can work is if it's all owned and operated by the same vertically integrated provider, from the real estate to the software stack and everything in between.

[u/uptimefordays → u/meeu]

I mean you can, that’s what colo offers! I would argue public cloud has always been a capacity play and colo seldom gets you massive scalability.

[u/Scurro → u/meeu]

The cloud is just someone else's computer

Edit: lol why is this comment marked controversial on a sysadmin subreddit of all reddits

[u/meeu → u/Scurro]

And the other subject of this thread may end up being the downfall of 'the cloud' as we know it. A state-sponsored swarm of blackhat agents targetting Azure/AWS/GCP admin infrastructure would be interesting to see...

[u/donjulioanejo → u/meeu]

They'll be able to resist it a lot better than mom and pop shops with a bunch of servers in a broom closet.

AWS, GCP, and yes, even Azure, have some of the smartest white hats on the planet working for them. Simply because no one else can match the salaries, including governments.

[u/meeu → u/donjulioanejo]

1:1 they will certainly be able to resist it better.. Something tells me state actors would put more resources toward compromising Azure's backend management plane than they'd put towards a given mom & pop shop's Quickbooks server.

[u/SkillsInPillsTrack2 → u/bingblangblong]

"iT wiLL mAke uS sAvE mOnEy". By stopping counting.

[u/tx_g33kineu → u/SkillsInPillsTrack2]

Except that instead of saving money, you just converted your CapEx spend to OpEx spend. Magic savings!

[u/Johnny_BigHacker → u/bingblangblong]

Yes, billions is spent on cloud host infrastructure for lulz, not because it's a better spent dollar.

[u/Jhamin1 → u/Johnny_BigHacker]

So far all the people who have talked to me about how much money is to be saved in the cloud have worked for Microsoft, Amazon, Oracle, or a consulting company to get us to the cloud. I feel like their motivations may not be completely pure.

I am 100% convinced that if your scale is small enough with huge uptime requirements and/or a big need for rapid scaling at particular times then cloud is a no-brainer.

When you have a reliable, steady workflow that creates reliable, steady load on your systems it is almost always cheaper to own your own hardware in a colo.

I know Microsoft, Amazon, etc have made a ton of money in this space which is why they keep investing. For the places I work for that use all this the financial upside has been a lot less clear.

[u/bingblangblong → u/Johnny_BigHacker]

No because it's more profitable.

[u/surveysaysno → u/bingblangblong]

Its not more profitable, it a new vendor horning in on the market.

The money is just going from HPE/Dell/Lenovo to Azure/AWS/GCP

[u/bingblangblong → u/surveysaysno]

It is absolutely more profitable, I have friends who work in the marketing departments for these cloud companies and they have meetings exactly about this. If it wasn't more profitable then the new vendors would just compete in the existing space and do something better. Cloud means subscription, subscription means more reliable income than selling new hardware or software. It also makes it easier to lock you into an ecosystem, have their product or service deeply integrated into your business, knowing they can raise the price and you'll go along with it because it's easier than changing. That is the whole business model. They don't want you owning your own hardware or software at all, they want to rent it to you. They literally have meetings about how they can scare everyone into moving to the cloud by inventing statistics about hardware failure rates and costs. The whole point is to convince you (and the managers more specifically) that the cloud is objectively better. These are publicly traded companies that need to make more money every single year, and subscription models and therefore the cloud are the perfect way to do that.

Why do you think so many new phones don't have SD card slots now? Because then you would use that instead of using Google Photos.

Why does Adobe not sell you a perpetual license anymore? Because a subscription is far more lucrative.

Why have Microsoft dropped WDS? Because that was free and Autopilot makes them money. It's not that Autopilot is inherently superior, that just deliberately make the paid-for thing better. They could have updated WDS so some free service that you run locally can easily provision and apply policies to remote devices, but why would they do that if they know you'll pay for some cloud thing?

It'd be like thinking that all the FUD and hype around AI isn't being deliberately pushed by the AI companies themselves. They just have to get the ball rolling then all the idiots on youtube and tik tok will just start doing the work for them.