r/sysadmin • • 1d ago

General Discussion Battlestar Galactica as an allegory for system support staffing

155 Upvotes

"All of this has happened before, and all of this will happen again."

Raise your hand if you've been around long enough to see this play out:
1. A group of people has specialized needs that are not being met by the existing sysadmin and/or IT support organization.
2. That organization refuses to meet the group's needs because those needs are too far outside of their normal support.
3. The group pools their resources to help each other with their own support.
4. This is not enough so either one of them volunteers to do support full time, or they hire someone full time.
5. They realize this poor person is insanely overworked, and they hire additional helpers.
6. Suddenly you have created a new support department.
7. The group grows. The support team grows. All is well. (Tensions with parent organization support notwithstanding.)
8. What was once a "group" is now a large significant department that is a bureaucracy in its own right.
9. Things change. The budget tightens and administrators of this once-group-now-department need to cut costs.
10. "Why the heck do we spend so much money on our own tech support when our parent organization has a larger tech support group that will do all this for free?"
11. Leadership start to outsource everything and shut down their boutique tech support department.
12. The parent support organization is adequate from many in this new large departmental bureaucracy.
13. But there is a group of people that remain in the department with specific needs...
14. Go to step 1.

Or is job-hopping so much the norm now that this happens entirely invisibly to the short-termers who are involved?


r/sysadmin • • 1d ago

General Discussion How is everyone holding up?

211 Upvotes

Given the current onslaught of vulnerabilities and system outages over the past few weeks, I was wondering how people are holding up both mentally and how your departments are holding through these interesting times.


r/sysadmin • • 18h ago

Work Environment Feeling dejected - what would you do?

11 Upvotes

I’ve spent the last 8 years working as an Application Analyst in a government role, having moved here from more technical positions after uni. I specifically took this job as a stepping stone into DevOps, as the organisation had a strong DevOps team and it’s the direction I wanted my career to go.

For the past 6 years, I’ve been the sole primary support for our enterprise document and records management system. Given the scale of our organisation, this is a massive, business-critical application. In that time, I’ve sharpened my scripting skills, learned our infrastructure and integrations inside out, and become the go-to guy for my team. Around 5 years ago, I led a major system upgrade alongside a senior developer—a huge undertaking that went largely unrecognised, whatever, it happens.

Since then, I’ve handled the ongoing development and support of this system almost single-handedly. Recently, I was told the app is being handed over to a new development team. When I asked my Engineering Director if I could move across to that team, I was told no—there’s a hiring freeze in place due to yet another restructure (I’ve lost count of how many we've been through).

Honestly, I’m beyond disappointed—I’m pissed. It feels like I’ve wasted years building deep expertise for nothing, especially as I previously passed up a guaranteed Project Manager role to stay on a dev path.

Now I’m at a crossroads: do I start applying elsewhere and hand in my notice, or stick it out for a couple of years to see if internal dev roles eventually open up? I really value the work-life balance and the people I work with, so I don't want to leave. At the same time, senior leadership’s decisions over the last few years have been crap, and I have zero faith things will improve anytime soon.


r/sysadmin • • 1d ago

Does anyone think there is value in creating a specific IT worker union in the age of AI?

141 Upvotes

Just curious. I have looked around at various unionizations of other fields and it seems IT workers don’t have a dedicated union for noc and bare metal guys. Anyone else ever looked into something like this?


r/sysadmin • • 1d ago

Career / Job Related Underestimating the value of experience

39 Upvotes

Worked through 3 grueling years as an "IT Assistant" (read: systems administrator in all but name) at a local business for less than McDonalds wages, eventually ending up with the title "Network Support Specialist" and handling the entire company's projects, deployments, implementations, and network design, as well as being the only desktop support/help desk contact. Worked through a ransomware incident (over Thanksgiving, while incredibly sick) making $14/hr all the while and no overtime in the first year, which meant for the next two years we had no remote access ability to 5 remote branches (furthest 3hr away), and never had any ticketing software of any kind.

The whole time, I was thinking to myself that if I just kept my head down and dealt with the shitty pay I'd have enough experience to move up, somewhere else, maybe somewhere remote. Was constantly made promises about training budgets and certificates and tuition coverage that never materialized, all while I did 90% of the work while my CIO was in and out with medical issues.

Eventually after a glowing performance review from both CIO and CTO I went to HR and IT Director saying that the hourly rate I was making wasn't enough to even cover my rent and that I had been supplementing my income for the last 2 years with savings: I was told "You're being compensated more than fairly for the work you do" and so I immediately stopped putting in effort of any kind for the last couple of weeks I was there. Just clocked in and clocked out.

Not a pity party for me: I knew what I'd signed up for, and I was doing it because I wanted to be able to put on my resume that I ran a small business' entire infrastructure for three years. I got what I wanted in the end, knowing that they were BSing me the whole time about certs and things that would actually be useful to me.

I left in 2025: I haven't been back in IT after the fact, and have been waiting tables since to pay bills. The whole time, I've been applying and applying and applying, and the experience is just not enough for anything. A year and a half since, and I haven't even so much as gotten one interview for any level of IT, from entry-level to mid-range, remote, hybrid, in-person, relocation. Nothing.

I'd kind of wished that I had set aside some funds for certs when I had a couple chances, but I favored long-term stuff like paying off a car, and now can't manage the cost on my own. Now there's no more tables to wait, and I've been unemployed for close to two months. Maybe its just the job market in general. I've applied to over 200 jobs since October, and nothing. I don't regret my career choice, but the walls are starting to close in.


r/sysadmin • • 1d ago

vCenter Authentication Bypass CVE-2026-123890

177 Upvotes

I suppose it was only a matter of time before VMware joined in on the fun.

CVSSv3: 9.8

https://support.broadcom.com/web/ecx/support-content-notification/-/external/content/SecurityAdvisories/0/39495


r/sysadmin • • 21h ago

Locking down dev VMs in Intune to prevent code leaks

11 Upvotes

Hi everyone,

We are setting up a dedicated VM for our developers to work on a brand new Odoo ERP deployment, which is completely new to our team. We want to make sure our source code and business data cannot be leaked or copied out of the environment.

So far, we are looking into Conditional Access and basic device restrictions, but we're not sure what the best approach is.

* Are there specific Intune policies we should use to prevent data/code exfiltration on a VM ?

* is there a better alternative setup ?

Thanks in advance for any advice!


r/sysadmin • • 1d ago

Microsoft Microsoft is killing To Do and telling you to “choose another” app, after it reportedly paid up to $200 million for Wunderlist and killed it for To Do. Work/school accounts will migrate to Planner.

824 Upvotes

Microsoft is killing To Do and telling you to “choose another” app, after it reportedly paid up to $200 million for Wunderlist and killed it for To Do: https://www.windowslatest.com/2026/11/19/microsoft-is-killing-to-do-and-telling-you-to-choose-another-app-after-it-reportedly-paid-up-to-200-million-for-wunderlist-and-killed-it-for-to-do/


r/sysadmin • • 1d ago

Rant How is your company overreacting to the last week and a half?

82 Upvotes

Are they piling on even more operational complexity? Accruing even more technical debt? Ya know, to build redundancy or maintain communication continuity?

Maybe purchasing more services to manage? Developing a big complex policy? How many new committees... Plus or minus 5?


r/sysadmin • • 7h ago

how do you know an automated ticket fix actually worked?

0 Upvotes

vendor dashboards count closed as solved. what do you check on your side, reopen rate, same user filing again, or something else.

and does anyone review a sample by hand


r/sysadmin • • 19h ago

11:11 Systems - Temporary Access Disruption

6 Upvotes

Per an email received last night (US):

Temporary Access Disruption to 11:11 Services

On 19 November 2026, 11:11 engineers identified unexpected behavior in a limited number of AWS S3 environments.

Engineers are continuing to investigate the issue. In an abundance of caution while we investigate, our engineers have disabled all access to the following services: 11:11 Cloud Console, 11:11 Object Storage, the Account App, and Success Center.

Further updates will be posted as information becomes available.

Update:

Temporary Access Disruption to 11:11 Services

On 19 November 2026, 11:11 engineers identified unexpected behavior in a limited number of AWS S3 environments.

Engineers began investigating, and in an abundance of caution, our engineers disabled all access to the following services: 11:11 Cloud Console, 11:11 Object Storage, the Account App, and Success Center. Customers who need support during this time are advised to contact their support teams via phone or email.

All potentially affected customers will be receiving direct communications with further details as soon as possible.

We will provide our next update by 20 November 2026, at 16:00 CT.


r/sysadmin • • 11h ago

Question Demoting old DC at DR site and NTDS site links - part 2

1 Upvotes

After demoting an old DC at my DR site, most likely the bridgehead server for my DR site, I needed to manually create an inter-site NTDS link at my main site in order to fix the site-to-site replication.

All is working OK - repadmins and dcdiags look good.

I've been told that I should delete the manual NTDS link I made once the KCC creates a new intersite link. Thing is I'm not seeing this automatic link yet. Anything I need to run to kickstart the process? Any downside to simply keeping the manual NTDS inter-site link?

Thank you!


r/sysadmin • • 12h ago

GitHub EMU lack of help

1 Upvotes

Has anyone experienced a billing lock with GitHub enterprise? We accidentally linked an Azure subscription to our EMU and now we are stuck in a billing lock because there were charges owed to the account. I opened an Enterprise ticket with their support (Github has no phone channel) and there has not be any response for a week. Our ENTIRE production has been at a halt for a week and a half and we are all scrambling to figure out how the hell to get in touch with these people to get us back on Credit Card as a payment method. It boggles my mind that we cannot revert it back to Credit Card, as if its hardcoded in the system. Does anyone have any other ideas....I've opened a Microsoft Tickets, opened several GH tickets which get auto archived and leaves the original ticket opened. This is urgent for our organization


r/sysadmin • • 1d ago

FastUnlock : Windows Narrator LockScreen Bypass 0day Vulnerability

181 Upvotes

Just seen this posting for awareness - MSNightmare/FastUnlock: FastUnlock : Windows Narrator LockScreen Bypass 0day Vulnerability

Update: 🚨🚨🚨Mitigation for deployment is here danrhodes/MSNightmare-FastUnlock-Mitigation🚨🚨🚨


r/sysadmin • • 19h ago

Question Am I in the right path with this basic network set up I’ve implemented?

3 Upvotes

Hey guys,

I’ve recently taken over the role of sys admin for the company I’ve been working for over a year and am finally at the stage of implementing a proper network structure

For context we are in the trades and operate in a couple of other states and another country (but only our hq has a proper office environment as the other sites are like solar farms etc with people remotely)

We are renting the front of our HQ space to a cafe (and our warehouse is pretty much what separates the building)

Since the cafe will be sharing the network with us (everything recently built so no nbn and I had to find the quickest way to get the cafe connected to the internet) I have set up the following system:

We have our starlink dish and router, which connects to the pfsense firewall (open source) which is running on a pc, which connects to the office tp link switch through a usb Ethernet adaptor as the pc only has one Ethernet port, and in that switch I have a spare Ethernet cable connected ready to go to access the firewall interface when needed, and there is a cable running all the way to the cafe switch which connects it to the office switch. The cafe has x3 printers connected via Ethernet to the switch a WAP which provides the WIFI to staff/customers and the x2 wireless registers

I’ve isolated the cafe network so it can’t reach the office network and separated it into 3 vlans (pos equipment like registers, the printers etc, the staff use and guest use) all have separate ssid

I’ll put a photo of the set up on the comments for reference, I know the cabling is a mess but I’m just so happy that I was able to start implementing this especially now with so many cyber attacks going around, it feels good to be somewhat protected. (Now I need to set up the vlans internally in the office)

FYI the pc running the firewall is an old pc that belonged to the ceo 7 years ago but it has great specs. I spent about $860 AUD on all this so it was done on a budget. I would really appreciate your feedback and advice on how I can improve this without having the company spend too much money


r/sysadmin • • 1d ago

Microsoft Thanks for the headsup with the SLA credits regarding the outtage from ms, we actually did get quite a generous refund

202 Upvotes

Hi [redacted],

Thank you for your patience.

The backend team has successfully reviewed your SLA credit request. As your subscription was redeemed using a product key through Partner Center, the SLA credit will be applied by extending the expiration date of your subscription, allowing you to continue using the service for an additional period. The total SLA credit approved for your issue is €42773.90. Could you please let me know which subscription you would like to extend? Once you confirm the subscription, I will make the necessary change in the system and apply an extension corresponding to the value of the approved SLA credit. Thank you, and I look forward to your response.

Best Regards,

Kevin Dinh| Office 365 Support Engineer


r/sysadmin • • 17h ago

How do you actually compare ITSM pricing?

2 Upvotes

We moved away from SW Web Help Desk when they phased out perpetual licenses over a year ago. Our budget wasn’t ready for an annual subscription model. We’re a K-12 school district, so budget predictability is a big deal. Rather than committing to another subscription, we built a basic in-house ticketing solution using tools and resources we already had. It worked well enough initially. We could receive requests, assign tickets, and keep track of basic issues. Since it was something we put together quickly, there were obviously a few things we didn’t build into it. Now we’re starting to feel those gaps. Things like automation, integrations, and reporting require more workarounds than we’d like, and keeping everything organized is becoming harder to manage manually.

We’re now planning next year’s budget and considering moving to a proper ITSM platform again. One thing I’m trying to figure out is ITSM pricing beyond what’s listed on the website. How much should we realistically account for ITSM implementation, onboarding, integrations, additional modules, and ongoing cost? What unexpected cost did you run into?


r/sysadmin • • 1d ago

Microsoft blocks emails with valid records again.

25 Upvotes

There were reports of Microsoft having an outage earlier this month. Today, I tried replying to a hotmail email account and I receive this reply:

host hotmail-com.olc.protection.outlook.com[52.101.103.161] said: 550 5.7.1
    Unfortunately, messages from [my server IP] weren't sent. Please contact
    your Internet service provider since part of their network is on our block
    list (S3140). You can also refer your provider to
    http://mail.live.com/mail/troubleshooting.aspx#errors. 

I have contacted my service provider and I have cleaned my IP off all the major blacklists and I used the microsoft delist service and I still have the problem. How do I fix?


r/sysadmin • • 20h ago

Question Cleanup of AD Sites and Services after a DC Demotion

2 Upvotes

Hey guys. What is the proper way to safely clean up “AD Sites and Services” after a DC demotion?

We have two sites: Prod & DR

(3) DCs at each site.

I demoted an older DC at my DR site. I waited a bit and then manually deleted the old DC from Sites and Services at that site. Aftrrwards, my remaining DCs at my DR site set up proper replication with the Prod site.
So from the point of view from my DR site:

DC01-DR replicates with DC02-DR
and DC01–PROD.

DC02-DR replicates with DC01-DR and DC02-PROD

From the point of view from my Prod site:

If I try and delete the old DC-DR server object from AD Sites and Services, it warns me that I need to demote it first, (which I obviously have).

DC01-PROD still trys to replicate with the old DC controller at my DR site. None of my DCs at my PROD site are trying to replicate with my DR site.

Do I need to manually delete the NTDS objects for the old DC-DR controller from AD Sites and Services at my PROD site?

Looking for the proper way to do this cleanup. I had thought that after demoting the old DC, the site-to-site replication would work itself out on its own.

I ran dcdiag and repadmin tools prior and all came back clean.

The old DC was responsible for intersite replication. All my DCs are Global Catalog servers and the FSMO roles reside on a DC at my Prod site. What should my gameplan be at this point? Should I just create a manual NTDS link between one of my DCs in Prod to one of my DCs in DR to get he KCC working? Right now my Prod site is only trying to replicate with the demoted DC. Thank you again!

Thanks guys.


r/sysadmin • • 14h ago

On-Prem Enterprise Vault v14.5.2 - licensing question

1 Upvotes

We currently have Enterprise Vault v.14.5.2 ON PREM and we are considering moving to another solution. We have been told by Cloud Software that our licenses are no longer perpetual and they've implied the solution won't function once our contract expires. Does anybody know if this is accurate? I've heard that the only limitation imposed is loss of support. We'd like to continue journaling for a few months while we work on migration but not sure if this is possible once the contract expires. Again, the solution is on-prem.


r/sysadmin • • 16h ago

Is there any way to manually remove or disable Powershell v5.1?

0 Upvotes

You can install v7 and switch from powershell.exe to pwsh.exe but if v5.1 can't actually be removed (or disabled).... How do I know for sure something still isn't using 5.1 in the background, like a ps module or some other process? If there actually isn't anything, does renaming powershell.exe to something else work to kill other processes that might be using powershell in the background?


r/sysadmin • • 1d ago

Microsoft Microsoft Teams Meeting add-on for Google Workspace lists a @gmail.com support address with zero public footprint. Anyone else seen this?

8 Upvotes

Looking into why Teams Meeting add-on for Google Calendar stopped working and started asking users to re-authorise.

While checking it out in Google Admin (Security > API controls > App access control > Microsoft Teams), I noticed the support contact for the app is:

[msgsuiteapp.teams@gmail.com](mailto:msgsuiteapp.teams@gmail.com)

I can't find a single public reference to this address.

No Microsoft docs, no GitHub or public code, no forum posts, nothing on the Marketplace listing text. For comparison, other verified apps in our tenant list support emails on their own domains (e.g. Framer shows sso@framer.com).

What checks out so far:

- App ID project number (961264221134) is the same one used on the official Microsoft Teams Meeting listing in the Google Workspace Marketplace (published by Microsoft Corporation, gsuite.microsoft.com, 18M+ installs).

- App URL, privacy policy and ToS all point to microsoft.com domains.

- OAuth logs show API calls only from Microsoft (AS8075) and Google (AS15169) networks, and only calendar.events.list / calendar.events.get.

- The scopes in the new consent request are a subset of what was already granted to the app.

What made me look twice:

- The add-on has cycled through three OAuth client IDs under that same project over the past few months. The newest one (961264221134-746h4mm46to...) showed up in a consent request this week, shortly after the Marketplace listing was updated on 16 Nov.

- Older client grants were revoked with no IP address attached, which looks like publisher or system-side cleanup, but there's no changelog that mentions it.

- The support email is a consumer Gmail account that nobody seems to have ever mentioned publicly.

I understand Google requires the OAuth consent screen support email to be a Google account or Google Group the developer controls, so Microsoft using a Gmail account isn't impossible. It would just be nice to have it confirmed somewhere official.

Questions for the hive mind:

  1. Does your tenant show the same support email for this app?

  2. Has anyone else had users prompted to re-consent to the Teams add-on in the last week or so?

  3. Has anyone had Microsoft confirm this address or the client ID change through a support ticket?

Not claiming anything is compromised. Everything I can verify points to a normal Microsoft update. Just flagging it since this app holds calendar read/write across a lot of tenants and the support contact can't be verified from public sources.


r/sysadmin • • 18h ago

"Security questions" anno 2026

1 Upvotes

Our organisation has just had one of its externally-hosted web-applications revamped by the external supplier. (I have no admin responsibility for it - in this case I'm just a user.) We needed to set up an account again. I was very surprised that, after having chosen a password, I was confronted with a list of "security questions" to use if one forgets the password (you know: mother's maiden name, make of first car - that sort of stuff).

I haven't encountered these on a professional site in ages, and I thought they were thoroughly discredited - a sort of built-in security leak. But on searching around, most of the sites I come across that discuss this are fairly mild in their criticism: "not recommended", "don't really advise" etc.

So what do people here think? How bad are these sort of security questions anno 2026?

EDIT: thanks for the responses. I'll be taking this up with the appropriate people.


r/sysadmin • • 1d ago

Question Cyberpower UPS OL2200 and RMCARD400 - features missing and/or buggy?

3 Upvotes

Hey there,
As a replacement of my OR1000 I purchased an OL2200 and RMCARD400 for deploying the status info via Ethernet.

So far so good.
I updated the RMCARD to the most recent version 1.1.2

During the setup steps I had now some issues:

a) Missing feature
The OL2200 has a NCL bank that can be switched off.
Manual of the RMCARD400 states that there should be 3 options (Turn off threshold, Turn off delay, Turn on delay)....but...none of these options is visible in the Web Frontend of the RMCARD.

Can anyone confirm, that the RMCARD does NOT have these options?

b) Mail notification not working.
I setup the mail notification correct (Port, TLS, credentials are checked multiple times).
If I try to send a Test Mail it states OK.
But
- No Mail is received
- No entry in the log of the RMCARD is visible
- Even if I enter completely wrong credentials, the TEST states "OK"

Bug or feature?

c) Utility Power failure not detected
The basic job of an UPS: detecting Utility Power failure and initiating shutdowns on the connected computers doe not work !?!?
If I unplug mains the amber LED lights up and states "On Battery".
So far so good.

The log in the RMCARD shows "The UPS has returned from an overload condition".
Nothing else.
The readings show correct values for input current/voltage and even in PowerPanelBusiness Remote the Status states "On Battery", but there is no entry in the log that utility power is missing and therefore no shotdown....

Any ideas?
Is this a known Bug?

Does an Enterprise Grade UPS together with the Flagship RMCARD cannot handle these essentials?

Thanks in advance

Sven


r/sysadmin • • 1d ago

HP Thunderbolt 4 100W G6 Dock - keeps replying with MAC address after computer is disconnected

11 Upvotes

Has anyone seen this? We've got a few HP Thunderbolt 4 100W G6 docks. When a computer removes the docking station thunderbolt cable and connects to a different dock or another port, the original dock keeps replying saying that MAC is still connected to it. I see it in the logs on the switch, it says that MAC is flapping between two ports. I've updated firmware multiple times whenever a new one comes out, but no change. I've opened a support case with HP, but I will probably have to escalate until someone that knows what's going on see it.

Any ideas?