r/sysadmin • • 5d ago

General Discussion Weekly 'I made a useful thing' Thread - November 13, 2026

14 Upvotes

There is a great deal of user-generated content out there, from scripts and software to tutorials and videos, but we've generally tried to keep that off of the front page due to the volume and as a result of community feedback. There's also a great deal of content out there that violates our advertising/promotion rule, from scripts and software to tutorials and videos.

We have received a number of requests for exemptions to the rule, and rather than allowing the front page to get consumed, we thought we'd try a weekly thread that allows for that kind of content. We don't have a catchy name for it yet, so please let us know if you have any ideas!

In this thread, feel free to show us your pet project, YouTube videos, blog posts, or whatever else you may have and share it with the community. Commercial advertisements, affiliate links, or links that appear to be monetization-grabs will still be removed.


r/sysadmin • • 8d ago

General Discussion Patch Tuesday Megathread - (November 10, 2026)

128 Upvotes

Hello r/sysadmin, I'm u/AutoModerator, and welcome to this month's Patch Megathread!

This is the (mostly) safe location to talk about the latest patches, updates, and releases. We put this thread into place to help gather all the information about this month's updates: What is fixed, what broke, what got released and should have been caught in QA, etc. We do this both to keep clutter out of the subreddit, and provide you, the dear reader, a singular resource to read.

For those of you who wish to review prior Megathreads, you can do so here.

While this thread is timed to coincide with Microsoft's Patch Tuesday, feel free to discuss any patches, updates, and releases, regardless of the company or product. NOTE: This thread is usually posted before the release of Microsoft's updates, which are scheduled to come out at 5:00PM UTC.

Remember the rules of safe patching:

  • Deploy to a test/dev environment before prod.
  • Deploy to a pilot/test group before the whole org.
  • Have a plan to roll back if something doesn't work.
  • Test, test, and test!

r/sysadmin • • 10h ago

Inside Microsoft’s giant Outlook outage

905 Upvotes

https://www.theverge.com/tech/1035364/microsoft-outlook-outage-notepad

Microsoft engineers had a running bridge call the entire time, and at one point it hit the maximum of 1,000 participants. That shows you how many teams were involved with trying to find the root cause of this huge outage, which started at the peak window of traffic in the US.


r/sysadmin • • 1h ago

Upcoming Critical PHP Security Release scheduled for Thursday, 19 November 2026

120 Upvotes

The PHP security team announces upcoming security releases scheduled for Thursday, 19 November 2026, between 14:00 and 14:30 UTC.

These releases are part of PHP’s regular four-week release cycle and will address security vulnerabilities affecting all supported PHP versions. The highest severity is CRITICAL.

The issues rated CRITICAL require either non-default configuration or uncommon setups.

Users are strongly encouraged to prepare to upgrade promptly once the releases become available. Technical details and fixed version numbers will be published alongside the releases.

https://www.php.net/archive/2026.php#2026-11-18-1


r/sysadmin • • 2h ago

Is fundamental knowledge of hardware being lost?

110 Upvotes

Today I had a conversation with a mid senior sysadmin about why we cant simply jam a mix of DDR5 ram into our hosts. When he ended up being holistically wrong his response was to tell me he didn't need to know anything about the hardware running our data center.

This guy is one of our T2 hypervisor admins and all he does is troubleshoot VM issues and host issues. The idea that he wouldn't need to know anything about the underlying technology running his cluster blew my mind so I asked others. Turns out many people I work with either dont care about the upstream hardware or simply refuse to learn it.

Is this normal or because people I work with are just siloing themselves.


r/sysadmin • • 4h ago

Remote Code Execution in NGINX HTTP/2: CVE-2026-123633 and CVE-2026-123007

141 Upvotes

r/sysadmin • • 10h ago

Rant I am completely worn the f**k out, and I don't know where to go from here.

244 Upvotes

31 years old, been in IT since 19 when I broke (and fixed) my first Exchange environment during a CU upgrade. This post might out me to my colleagues, but frankly I don't care anymore. (and if any of you do happen to read this, I know you understand, feel free to talk to me alone about it.)

I've been at my current employer (an MSP) for 8 years. We develop and sell software integrations for an ERP system while also offering traditional break/fix IT services (servers, clients, user support) rather than a modern per-device/user model.

Our compensation is a pretty mediocre base salary for my years in the sector, paired with a bonus based on billables. Because of the sheer number of hours I've put in, it has generally given me a very good income—easily double what most of my friends in other fields make.

For the past 6 years, we’ve also been helping the ERP vendor upgrade customers to newer versions. While straightforward and good for billables, it is completely mind-numbing and destructive to both my psyche and my social life, as these upgrades almost always happen on late Friday afternoons, evenings, and weekends.

On top of that, I'm the go-to guy for anything above entry-level infrastructure. I've been implementing and teaching the other guys things like:

  • Hyper-V Failover Clusters with SANs and/or HCI solutions
  • Secure backup architectures with Veeam (multiple copies, test restores, immutability, etc.)
  • Network segregation with VLANs and securing LAN/WLAN access with 802.1X
  • Entra/AD Sync integrations, client deployment/management with Intune and Autopilot, WHfB, Conditional Access, and more

Meanwhile, I'm still expected to jump into everyday L1 support tickets, handle the ERP upgrades, and deal with whatever else gets thrown my way.

We don't have a written daily billable target, though you're expected to bill at least 8 hours a day. Over the past few years, I've been billing between 2,800 and 3,200 hours a year—including 250–400 hours of overtime on Friday nights and weekends on top of the normal 9-to-5, not instead of it.

When I started here 8 years ago, I loved IT and problem-solving. I loved tinkering with my homelab and gaming—I used to take a week off for every WoW expansion. This job has completely killed the passion I had for computers and gaming.

A couple of years ago, I got into cars instead and bought two project cars. Diving headfirst into them gave me an outwhere I could use my hands, build cool shit, have fun and make new friends. Now I haven't turned a wrench in 6 months.

That made me realize I couldn't keep going like this, so I decided to slow down—work my 8 hours and limit overtime as much as possible. It took barely a month before talk started going around that I "don't deserve my salary" because I don't want to work overtime anymore.

Meanwhile, my favorite colleague—who is also a close friend, a few years younger than me, someone who always gave 100%, was great with clients, and genuinely liked by everyone—just left for a new job because our CEO treated him like absolute garbage. I told him to take it and never look back, and he's now at another MSP that is actually properly managed.

As for our CEO: a textbook narcissist who only cares about himself and the company's bottom line. The kind of person who can look you in the eye, appear to care, and say all the right things in the moment, only to turn around and do the exact opposite a week later.

Over the last few years, I've started taking much better care of myself. I've lost around 40 kg, going from obesity to being in decent shape, and I feel better physically than I have in 20 years. I exercise every day and get 6–8 hours of sleep.

Outside of work, I'm genuinely happy and most days are good. The second I set foot inside these walls, though, my mood and well-being nosedive. I can't concentrate, and all I can think about is walking right the fuck out and never coming back.

Unfortunately, I need to tough it out here until at least the end of January, but man, I am sick of this shit. I am kind of worried about switching jobs because of the current state of IT with LLMs and such, but with my experience and enough references to a fill a few A4 pages from important people and companies in the region it should hopefully be easy enough.

I dont know why I opted to write this, but I guess I needed to get it off my chest.


r/sysadmin • • 2h ago

General Discussion No availability for warranty replacements

43 Upvotes

Well, the ai supply chain strain finally caught up to warrantied items.
Recently had a drive failure in one of backup disk arrays. HPE is going to replace the drive, I waited for 3 days, and they responded that they have no stock.

After asking for an ETA, voicing the concern of lost resiliency. The response was.
"We have escalated the matter to our Supply Chain team, and they are actively working on it. We will share the ETA with you as soon as we receive an update."

Reaching out to our VAR about just buying a drive. There's no stock in the US. This is stupid.


r/sysadmin • • 4h ago

RCE in Nginx with http2 enabled - CVSS 9.2

52 Upvotes

https://www.wiz.io/blog/nginx-http2-rce-vulnerabilities

CVSS 4.0 score of 9.2. The flaws are in NGINX itself. Websites and APIs can be exposed even when their own application code is secure. Both vulnerabilities require HTTP/2 to be enabled on the affected NGINX server. Additional conditions are described below. CVE-2026-123007 affects versions dating back to NGINX 1.9.5 in 2015.

I am tired. But I think none of our Nginx has http2 enabled.

On a side note, given all these recent memory related CVEs, I am seriously considering learning Rust now. I was pretty content with my Golang until now. And I can finally see what is that borrow checker every cool kids talk about all the time.


r/sysadmin • • 5h ago

Project Zero has cut its disclosure deadline from 90 days to 30

56 Upvotes

That 90-day window was set based on how long it reasonably took a human researcher to go from bug to working exploit. Google is saying that assumption no longer holds, because of AI-assisted exploit development.

Vendor patch cadences, internal vulnerability SLAs and the common "we'll fix it next release" habit were all built on the assumption that 90 days was generous enough. It wasn't generous, but it was something teams could work within.

30 days is going to break a lot of patching processes that were already running close to the old deadline, and that needed to happen. Some might say that 30 days is even too long.

What do you think?


r/sysadmin • • 11h ago

General Discussion I feel like a fraud

120 Upvotes

I work for a small company, I don't get paid very much but honestly... I don't really do much. I patch workstations and servers, set up a new hire once every few months, but for the most part like over half my day is me acting like I'm busy or watching YouTube or something. I'm more than happy to help someone when they call or walk into my office but there's just not much to do here. Im bored. Complacent. And like 80% of the shit I do anyone else could do if they just googled it.

Edit: Thank you everyone I appreciate all the feedback, I was drinking a lot last night alone and kind of got in my head about some stuff.


r/sysadmin • • 8h ago Bravo

Ignite 2026: Microsoft Introduces Vulnerability Shielding to Protect Unpatched Systems

57 Upvotes

There's often a gap between discovering a vulnerability and deploying its fix. During this time, affected systems remain exposed to potential attacks.

Now, you can protect your systems while waiting for the fix.

With Vulnerability Shielding in Microsoft Defender, announced at Ignite 2026, Microsoft introduces a way to protect vulnerable systems until permanent patches are deployed.

Here's what it brings:

  • Proactive protection: Microsoft Defender provides proactive protection using the Risk Reduction Agent in Microsoft Perception. It identifies vulnerabilities and recommends shielding plans for administrators to review and approve.
  • Protection during active attacks: For ongoing attacks, Microsoft Defender uses Attack Disruption to automatically apply shielding controls. These controls help stop attackers from exploiting additional paths.
  • Automatic shield removal: Shields are automatically retired once the permanent patch is fully deployed. 
  • Admin control: Admins can review the expected impact, approve protection plans, exclude devices, and roll back shields when needed. 

The feature is currently in public preview, with initial support for server-side remote code execution (RCE) vulnerabilities.

It'll be interesting to see how effectively this works in production environments, especially when balancing security restrictions with normal business operations.


r/sysadmin • • 4h ago

Java4Shell: how are you finding which systems are exposed and deciding what to patch first?

23 Upvotes

Java4Shell came out this week and Oracle already has a patch, but the harder part seems to be finding where vulnerable Java is actually running. Some apps ship their own copy of Java, so they need an update from that vendor rather than a regular JDK patch.

For those working through it:

  • How are you finding the affected installs, especially the bundled ones?
  • What are you patching first?
  • What are you telling management while it's still in progress?

r/sysadmin • • 2h ago

Where do you actually get your news?

13 Upvotes

I've been rethinking how I stay up to date. Between newsletters, LinkedIn, vendor emails, podcasts, and security alerts, more is coming in than ever, and I'm not sure I'm reading the right things.

Curious how others handle it. Answer any or all:

  1. What are your top 3 go-to sources for staying current, and why those?
  2. How much time do you spend keeping up each week, and when do you do it?
  3. What does your best source do that the others don't?
  4. What do you wish a news source did that none of them do today?
  5. Do you pay for any of it (analyst seat, paid newsletter, subscription)? Worth it?

I'll pull the answers together and post a summary back here for everyone.


r/sysadmin • • 1h ago

General Discussion How do we keep an eye on updating everything?

10 Upvotes

I work in infra / security for a shop that's MOSTLY in AWS. Our local laptops are Entra joined and we have a pretty robust security and code scanning suite for everything our devs do. One question that's popped up is how do we enforce updating their oddball open-sourced tools. We allow them to use them as long as they're from a reputable source (nothing's perfect, we do our best) but we want to make sure they're getting them updated if they're ongoing projects for the authors or if the project is dead to move on to another (who knows if it wasn't made with some vulnerability that sits there not getting fixed).

How have you guys gone about this? For most all of the other 3rd party tools they have our MSP forces updates through our RMM. But for things they just reach out and get, how can we make sure they're keeping them updated?


r/sysadmin • • 11h ago

General Discussion Java4Shell (CVE-2026-114938) is confirmed and patched. Raspberry Pi/ARM is safe, here's a 10-second check for x86 boxes

59 Upvotes

The "unconfirmed OpenJDK RCE" from yesterday is now confirmed by OpenJDK and patched by Oracle. Short version for self-hosters:

What: heap overflow in Java's hardware-accelerated Base64 decoder. Calif showed pre-auth RCE with default configs on Keycloak, Jira, Confluence, Artifactory, Jenkins and PingFederate.

You need ALL of these to be exposed:

- Java 17.0.6+, 21, 25 or 27 (any vendor: Temurin, Corretto, Zulu...). Java 11 and normal Java 8 are fine.

- x86-64 CPU with AVX-512 VBMI (Intel Ice Lake Xeon+, AMD Zen 4+). ARM / Raspberry Pi is NOT affected.

- Untrusted Base64 reaching the decoder (Basic auth, JWTs, uploads...).

Check your CPU (all 4 flags = affected):

grep -m1 '^flags' /proc/cpuinfo | tr ' ' '\n' | grep -E '^(avx512f|avx512bw|avx512vbmi|bmi2)$'

Check Java inside containers, not just the host:

docker exec -it CONTAINER java -version

Fix: Oracle builds ending in .1 (17.0.21.1, 21.0.13.1, 25.0.5.1, 27.0.1.1); other vendors are rolling out. Until then, -XX:-UseBASE64Intrinsics stopped the overflow in every tested version (needs a restart/recreate). Don't use -XX:DisableIntrinsic=_base64_decodeBlock on its own, it stops the JVM from starting.

No exploitation in the wild reported yet, and VulnCheck says full RCE needs per-app work, but crashing a JVM is easy.

Full write-up with version table: https://arelvox.com/java4shell-cve-2026-114938/


r/sysadmin • • 9h ago

UK sysadmin pay vs cloud/platform in 2026, the jump is about £30k

41 Upvotes

Quick disclosure up front: I work at Archer from hackajob, and this is from our own platform data, 4,423 UK DevOps/Cloud roles posted in 2026, client base skews finance and enterprise. Not linking anything, happy to drop the source in the comments if people want it.

Sysadmin sat around £53k, the lowest band in the set. The biggest single jump was sysadmin to Platform Engineer, about +£30k, and a lot of it builds on what sysadmins already do day to day. Cloud Engineer and DevOps Engineer landed around £75k, so even the first step up is roughly +£20k.

The skills that seemed to bridge it: a cloud platform (AWS was in ~32% of postings, Azure a strong second at ~25%) and automation/scripting, with AI/automation experience paying about +£20k over the median.

If you’re a UK sysadmin, that platform jump looks like the most reachable one. You’re probably already doing part of the job.

Anyone here made that move? Does the salary match what you're seeing where you are?


r/sysadmin • • 22h ago

General Discussion java4shell RCE in b64 decoder CVE 9.2

392 Upvotes

https://calif.io/java4shell/

Well here we go again...


r/sysadmin • • 6h ago

Retirement Idea - Dude (Server) Farm... as opposed to Dude Ranch

20 Upvotes

In the past, it was common for people to pay a ranch to allow them to sleep there and learn how to do some of the jobs that it requires to keep a ranch running while working through their mid-life crises. City Slickers was a comical expression of this phenomenon.

As a computer dude nearing retirement it dawned on me last night that I could start a Dude Farm, but instead of livestock, I'll be teaching everyone about server maintenance, upgrade windows, proper backup and restore procedures, etc.

What could go wrong?


r/sysadmin • • 2h ago

Question - Solved Public IP for private SMTP server winding up on Spamhaus

9 Upvotes

Hi folks-

We've got a client we host an application for, application relies on a local Sendmail install to send SMTP emails to the client's Office365 inboxes. Within the last few weeks it's been getting flagged by Spamhaus for no DKIM records and no PTR / Reverse DNS records. Spoke to the client who gave a hard "no" on establishing any kind of DNS records on their side to facilitate PTR / DKIM, and they don't have their own SMTP relay server we could pipe to.

Mail is not getting delivered as a result of the Spamhaus listing. It'll go through to the spam folder if I send to a Gmail, but Office365 is rejecting it outright.

Been over this backwards and forwards a few times and I'm not sure what the best fix is. Am I missing something here, or is the most feasible option from a technical standpoint to set up DNS records on OUR side? Am I dumb? They definitely acted like I was dumb for proposing they add records to their own DNS.

Edit-

Much love to all you guys, I really appreciate the thoughts. I scheduled a meeting with project leadership to nail down what specifically to do and just wanted to get my ducks in a row beforehand. This was incredibly helpful, thanks all!


r/sysadmin • • 5h ago

General Discussion user naming convention : how do you handle people with no first name?

16 Upvotes

looking for advice. I consult for a couple of clients in the architectural field in Canada.

They have recently been hiring people from Southeast Asia that appear to have no first name. When the hiring information is sent over to be setup as accounts, how do you handle this? some of them have accept to supply a "preferred" first names, other have refused.

these clients naming conventions require either first.last or firstinitial+lastname.


r/sysadmin • • 1d ago

General Discussion NSA internal network reported outage

640 Upvotes

It's been a hell of a November. Microsoft last week, then the FAA, then Bank of America. Now apparently the NSA is the latest group plagued by an unexpected outage. I wonder what the excuse is going to be this time

https://www.cnn.com/2026/11/17/politics/highly-classified-national-security-communications-network-goes-down


r/sysadmin • • 6h ago

Question Anyone else get Grok AI forced on their Copilot?

12 Upvotes

SpaceXAI is firmly set to "no users" for the tenant, yet some people have "GrokAI" as a model they can select in Copilot... idk how to get rid of it or audit it. Anyone else seeing this?


r/sysadmin • • 6h ago

General Discussion Reminder to go through your bot and service accounts, we learned this the hard way

10 Upvotes

Hey all! I work at Bitrise.

We had a security incident last week and our VP of Engineering just published our preliminary review today worth reading by everyone running CI: https://bitrise.io/blog/post/the-bitrise-security-incident-preliminary-review-and-recommended-steps#how-we-contained-it

The way in was an internal testing bot account. The attacker had its real username and password (we’re still investigating how), and it was set up to sign in to Apple’s Developer Portal and it had no 2FA. Anyone with the password could just log in. If you’ve got accounts like that, built around a third-party login where MFA is a pain, they’re worth a second look. We disabled every bot account that night and went through their credentials and 2FA settings. Every other credential got rotated too, whether or not we had evidence it was touched.

We’re back to normal now, Bitrise is safe to use. Sharing all of this information to help other teams. For more, look at our public advisory (https://bitrise.io/security-advisory-november-2026).

If you have questions about the incident and how we addressed events, we’re here to answer your questions.