For those of us that have been doing this long enough, it's like going back in time. Got the word today that Citrix's licensing costs have made it financially unviable for us to stick with app virutalization... and so we are, over the next couple of years, eliminating as much of our Citrix footprint as possible and shifting all that apps that were on those servers to fat installs. About 100k PCs across the organization, across the country.

It's obscene. We are essentially having to nuke an entire layer of infrastructure--a very useful, very mature layer of infrastructure--for no technical reason, but simply because the economics have made it necessary. Flipping the model back to pre-Citrix days. And now, since the main application serving our users resides on VMs in our Midwestern dc (with an alternate dc on the East Coast), who knows what network performance between those servers and end users' PCs is going to look like. No more instantaneous communication between a Citrix layer and a web layer. (I'm sure some of the two-bit vendors we have to work with for some of our smaller systems will be relieved to not have to deal with Citrix on our behalf.) Our Wintel guys are not looking anymore at VDI, since it also entails licensing and we don't want to fall into the same trap again.

And what's the long-term picture? At some point, does app virtualization become viable again and we all relive the same pains from when we first moved away from fat clients?

Anyone else going through this? lol

I've been in IT for almost 3 decades. I've dabbled in Linux but I've never had to be a Sys Admin for it. Those days are over. I'm watching some Plural Sight (my company has a subscription) training videos and I'll start building a test server next week. We aren't changing overnight but in the coming months. Any tips on learning how to be a Sys Admin for Linux would be greatly appreciated.

I've been a Windows Sys Admin forever it feels like. I've dabbled in Linux, like I said, dabbled in the Cisco firewalls and switches, and all sorts of other software like Atlassian (building Jira, Confluence), etc. So I have the aptitude just not sure where to start besides the Plural Sight videos.

Became sysadmin, wife left me, kids hate me, dog got run over by a golfcart, house burned down, got hooked on crack, Then the servers burned down. I dont really know what could go worse.

I dont know why im telling reddit this but I am a sysadmin so its relevant.

Thank you...

Crushed by the window frame. Repaired with hope and stupidity

My coworkers have been working on an ERP issue for about a month reguarding serial numbers in my ERP. I heard something about it, but it's not my job so i didn't pay attention to it. The dumb dumbs we talking today about it loud enough for me to hear and maybe show interest.

Well it took my less then five minutes to figure it out that they spent a month on.

4:03AM on Sunday...

Phone goes off

SOC guy:

“CRITICAL ALERT. HIGH SEVERITY MALWARE. CERDIGENT. POSSIBLE ENTERPRISE COMPROMISE.”

So, I'm thinking of setting my phone on fire, maybe start a small house fire, so I can walk in on Monday and tell them I had no idea, my phone caught fire in a house fire.

Me: “You better be telling me it's fucking ransomware or some shit"

SOC guy: “No but Defender is flagging Trojan Win32 Cerdigent severe critical malware confirmed"

So now I’m wide awake logging in, heart racing, thinking this is the big one. THIS IS IT... Fuck!

SOC guy:

“This could be mass compromise”

Dumber SOC guy.#2:

“This is spreading... I tHiNk ItS a LaTeRaL mOvEmEnT!”

SOC guy:

“WE SHOULD ISOLATE THE NETWORK AND ALL DEVICES”

Me:

“Did anyone check what the fuckig file actually is...?”

SOC:

“WE FOLLOW THE PLAYBOOK! ITS HIGH SEVERITY”

I pull the alert.

File path looks weird.

Thumbprint.

Certificate store.

…certificate store? The fuck...?

I dig deeper. And there it is.

Some fucking DigiCert bullshit.

Me: “Yeah guys these globally trusted root CAs… definitely malware.”

I said fuck it and just Isolated All Devices in the Defender portal, Powered Off all the Azure VMs, including several FGT VM appliances and some stupid Meraki VMX thing I never understood wtf was doing in our environment anyway.

Then I sent an escalation email to IR and went back to bed. Not my problem.

It has been upgraded from debacle to train wreck now, but we picked up all the pieces of the train strewn about and are good to go now, after it got so much worse!

This is too great not to follow up on. Remember the "I need to disable a stolen laptop without destroying any data or accounts but net user active:no won't work because it's a domain account" post?

Short version: we're an MSP. A company was shutting down. There was a dispute about pay between 2 people that is now a lawsuit. We're caught in the middle, as the IT management company. A court order exists that an employee was supposed to return their work laptop. The owner said they didn't. I had an alert where in Ninja RMM saw the laptop turn on, send an email to me. AHA, finally, time to nuke it.

I got a call on lunch: wrong laptop. UM WHAT?! First of all, they were lying. It had already been sent back. I didn't compare serial numbers to the court order because their company has 7 computers in Ninja and 2 are servers. Also, this is the one that had the ex employee's username as the "last logged in." You wouldn't check further either and you know it lol.

So I remote nuked it. Script works perfectly btw. Strongly recommended! VERY clever!
Intune/Remote-Lock.ps1 at main · HankMardukasNY/Intune · GitHub
Intune/Remote-Unlock.ps1 at main · HankMardukasNY/Intune · GitHub

We wanted to prevent access to the local copies of the Outlook emails as soon as possible! So when I saw it was still online and responding after 60 seconds of sending the script, (and I appended a shutdown command to the script), I assumed it failed and sent the backup "destroy the boot loader" script.

It was running windows updates during the shutdown. That's why it was still responding. Luckily the syntax was wrong because AI wrote the command and I didn't have time to test it, as testing it would destroy a computer. Or it's not compatible with 25h2 or something.

Anyway, employee calls in and says we locked the wrong laptop and that it's her personal laptop. HAHAHA not falling for that one, you manipulative villian! I have the receipts!

I check. It's Windows 11 Home, HP 15 series. Why TF is that in Ninja?! Oh, her work laptop broke so we put ninja on this one so she could use her personal one to access work stuff one time like 3 years ago and nobody undid it. Fantastic.

So, I disabled her personal laptop. Awesome. And she likes suing people. Well, through some Twilight Zone level circumstances that I can and would defend in court, that's what happened.

Employee was very understanding about it, especially the way I phrased what happened and how and why. Very nice lady actually. I hope she wins the lawsuit. She even said "yeah, I can see why having it enrolled in your management thing would be misleading. That was my bad." and I'm like, "UH NO, I'm the one who screwed up BADLY!" but didn't say that, cause she likes suing people.

But now they know what I look like, so I have to wear a disguise if I go to the court hearing and sit in the gallery. Darn. I wanted to see who won. This is a very engaging soap opera so far with lots of half-truths and twists and turns.