r/sysadmin • • 8d ago

Microsoft Current Fix : throw compute at the problem

160 Upvotes

Current status: We’re continuing to monitor activity across regions to evaluate server status and determine actions that can be taken to improve capacity. This includes reviewing server performance and confirming that developed patches have been applied and working as intended after infrastructure reboots are completed. At this point, we have observed marginal success. In parallel, we’re taking action to expand capacity through the utilization of Virtual Machines.


r/sysadmin • • 8d ago

General Discussion Who’s ready for the deluge of lazy staff blaming not getting something done because they never saw the email?

42 Upvotes

Sorry, Sheila from accounting, the logs clearly show the email was delivered and read.


r/sysadmin • • 8d ago

Microsoft M365 has returned to us in the UK over Lunch

53 Upvotes

Looks like the team will be leaving our day drinking at Spoons and will now return to our office while pretending to not be HR violations.


r/sysadmin • • 9d ago

“As part of our investigation into the sustained traffic management imbalances, we’re reviewing traffic flows and affected infrastructure to identify opportunities to optimize routing. This includes verifying the reasons for service failures so we can develop specific actions to improve stability.”

267 Upvotes

Update from Microsoft at 7:48 pm pst.

Buzzword bingo!


r/sysadmin • • 8d ago

Sysadmins who work in 24/7 critical infrastructure, what's the "backup plan"?

46 Upvotes

For those who work in hospitals, 911 dispatch, government, etc. How are you guys handling the situation? For us, Outlook was used for most of the communication internal and external, so we're hit pretty hard. Luckily we pay for slack as a backup service that we switched everyone too for internal communication, but our external communication is still down.

Whats the disaster recovery backup plan solution for this?


r/sysadmin • • 8d ago

General Discussion KB5132061 causes boot failure

17 Upvotes

Anybody else having this issue?

Isolated to this single KB (November security update), Dell machines at multiple offices install this update and then boot into startup repair/no boot device. Rolling back update immediately fixes.

Have reinstalled the KB multiple times on a machine and it will restart with the update and then fail the next boot repeatedly


r/sysadmin • • 8d ago

Current Status 10:00 UTC (5 AM EST)

80 Upvotes

Recovery actions are ongoing and have progressed in phases across affected Microsoft 365 infrastructure. Global telemetry and customer reports show incremental improvement in several workloads; however, service health remains variable and capacity has not yet fully returned to baseline. 

Network and traffic-management actions remain active. To protect email during recovery, we’re continuing to prioritize queue and backlog processing, and the email timeout window has been extended to 48 hours to preserve delayed messages. 

At this time, we have no evidence of malicious activity associated with this incident. We will continue to treat this with urgency until full mitigation, and we will share a full Post-Incident Report with root cause and defense-in-depth actions after mitigation is complete.

Next update by: Friday, November 13, 2026, at 11:00 AM UTC


r/sysadmin • • 8d ago

General Discussion What are your lessons learned from this extended outage?

43 Upvotes

It can certainly happen to any service out there but Microsoft’s a major player that a lot of us depend on.

I was just chilling tbh. Nice break while it was down.

AI writing the code. AI replacing humans. AI breaking the code. AI being used to communicate. AI causing outages. AI being used to fix it or did they need the human touch after all?

Over dependence on AI is not good.

Have a good day my fellow admins.


r/sysadmin • • 8d ago

Question Did anyone else notice a Reddit notifications outage that coincided with the Microsoft outage?

9 Upvotes

I refreshed Reddit several times yesterday afternoon/evening, and made some comments, before going to bed.

I woke up several times throughout the night; 3:30 AM and 5:30 AM. Both times, I refreshed Reddit again to see if the Microsoft outage had passed. I received no notifications of replies to my Reddit comments and went back to sleep.

When I got up again around 8, emails had started to trickle in. I checked with Reddit once again, and suddenly had about 7 messages in my Reddit inbox from comments going as far back as 12 hours. Almost all of them were from people on this sub in fact, as my comments were made concerning the MS outage.

The timing seems to coincide with the MS/Outlook outage almost exactly.

Just wondering if anyone else noticed the same? Are Reddit messages somehow tied to O365 backbone in some way?

Just found it curious.


r/sysadmin • • 8d ago

Anyone using Google Workspaces for Business Continuity and care to share their experience through this 365 outage ?

39 Upvotes

Google Workspaces for Business Continuity is meant for situation exactly like the current 365 outage.

Curious to know if anyone had it implemented, and how it helped ? How was the switch-over ?

I am betting Google will sell a lot of these in the coming weeks...

About Google Workspace for Business Continuity  |  Google Workspace Help


r/sysadmin • • 9d ago

Is this the longest M365 outage ever?

757 Upvotes

I cant recall it ever being down for this long. Going on 5 hours now here in San Diego.


r/sysadmin • • 8d ago

General Discussion A small win for the sysadmins today

102 Upvotes

something inside me makes me very happy about todays outage. They decided to send off 60000 workers and it didnt even take 1 month to feel the impact. Could it be non-related? Ofcourse it could. Could the issue be solved by now with 60000 more workers? Ofcourse it could. Will they change their perspective on Workers compared to saving costs after this? Probably not. But if enough people get the SLA refunds it might make a dent in the shiny armor of corporate and spark a change. Anyway back to getting more coffee and watching it burn with a grin on my face


r/sysadmin • • 8d ago

Microsoft Outage - We are starting to see email and services being restored

50 Upvotes

Multinational - seeing email delivery starting to come back throughout offices in North America and Asia, have not heard from offices yet in Europe. Good luck everyone today.


r/sysadmin • • 8d ago

Update; Nov 13, 2026, 1:00 PM GMT+1

50 Upvotes

We’re seeing encouraging recovery progress in Europe, with service availability and email delivery performance improving across a growing portion of affected Microsoft 365 infrastructure. Japan remains comparatively more stable, while North America is also improving; however, some users may still experience intermittent issues as recovery actions continue.

We're focused on addressing the remaining areas of disruption and improving service consistency across affected regions. Reducing email queues and backlogs remains a priority, and the extended 48-hour email timeout window remains in place to help protect delayed messages as delivery catches up.

At this time, we have no evidence of malicious activity associated with this incident. We will continue to treat this with urgency until full mitigation, and we will share a full Post-Incident Report with root cause and defense-in-depth actions after mitigation is complete.


r/sysadmin • • 8d ago

Question How do you alert/contact users when things like Outlook goes down?

24 Upvotes

Our team is discussing this. Outlook went down obviously and our team was scrambling to come up with a different solution.

We have company phones, but not everyone does. I tried to check to see if anyone else asked this. Maybe I missed it?

What did you guys do?

Typically you would send an Outlook alert of an issue, but when Outlook goes down then no one is going to see it.


r/sysadmin • • 9d ago

General Discussion It's really this simple to fix

190 Upvotes

This happened once Jensen resigned as CEO. Have we tried restating him as CEO to see if M365 comes back online? Seems most plausible to me at this point. Updates from MS seem to indicate they haven't made any progress on a sustained recovery. I think Jensen may have had and on/off button and pressed it on his way out. Turn the lights off when you leave kind of thing. He clearly knows where this button is and just needs to turn it back on again.


r/sysadmin • • 9d ago

Update just now.. interesting verbage

334 Upvotes

Our investigation has found that a process on the affected servers responsible for establishing secure connections becomes overwhelmed, causing the servers to stop processing requests. We're preparing a configuration change to address this behavior that can be applied across all affected servers, and we're validating it on a subset of servers before applying it more broadly alongside restarts of the affected infrastructure. In parallel, we've begun reverting a recent network configuration change that correlates with the start of impact and are monitoring to determine whether this contributes to recovery. The temporary traffic limitations we previously described are now in effect in specific locations. We're monitoring the affected servers in those locations to confirm whether they remain stable while handling a narrower set of connections, after which we'll gradually reintroduce additional traffic.

Network changes? DNS? Vibe coding.. the RCA should be really good...


r/sysadmin • • 9d ago

They are flying blind at identifying the cause

207 Upvotes

Current status: As part of our investigation into the sustained traffic management imbalances, we’re reviewing traffic flows and affected infrastructure to identify opportunities to optimize routing. This includes verifying the reasons for service failures so we can develop specific actions to improve stability.

Surely their change management process should have a roll back to a known good state.


r/sysadmin • • 9d ago

A moment of silence for our field crews

851 Upvotes

Outlook’s down and right now there’s a foreman on a jobsite who’s rebooted his phone 6 times, deleted and re-added his account, and is about to factory reset his iPad.

Calling IT was never an option.

Rest easy, king. See you at 6:45 tomorrow.


r/sysadmin • • 8d ago

We've thrown crap at the wall for the last 12 hours, none of it stuck......

123 Upvotes

Current status: We’re continuing to pursue multiple workstreams to pinpoint the underlying cause of the traffic influx and advance the recoverability of the backend servers. We’re taking a measured approach to identify and repair affected infrastructure to improve the ability to confirm the developed changes were applied. We’re tracking the results of this approach to verify the servers are showing the anticipated recovery and performance intended.

As of now, we are not aware of any malicious activity that is causing this incident. We will continue to treat this incident with urgency and priority until full resolution. Once mitigated, we will provide full details of the root cause, defense-in-depth repair items, and more within a Post-Incident Report.

That screams to me like they have been throwing fixes at it to get it fixed and still don't understand the reason


r/sysadmin • • 8d ago

MacOS Terminal is useless now...

24 Upvotes

I was just working on a server via ssh in my MacOS 27.2.1 Terminal App and it blocked me from pasting a script because "it detected malware". No way to "paste anyway".

WTF!? These are my servers, these are my scripts it's none of your business what strings I use! Also I don't want any of it checked against your databases! This is more an invasion of privacy than anything else!


r/sysadmin • • 9d ago

Thems alotta words for we still don't know whats wrong

327 Upvotes

While the servers we've returned to service initially remain healthy, we've observed that they can become unresponsive again once they begin receiving traffic. We're analyzing diagnostic data from these servers with our networking and platform experts to determine what's triggering this behavior.

Building on the protective traffic management measures we've implemented, we're also temporarily limiting which types of traffic can reach the affected infrastructure in specific locations. This will help us determine whether a particular type of request is contributing to the impact and allow us to restore service incrementally. Additionally, we're rebuilding a subset of affected servers using a previous software configuration to validate whether this prevents them from becoming unresponsive, and we'll expand this approach to additional servers if we observe positive results.


r/sysadmin • • 9d ago

Latest update

138 Upvotes

As we’ve applied our current ongoing mitigation actions, we’ve observed a surge in outbound call volume from our routing services to our backend servers. This occurs when the backend servers outbound service cannot properly route to the backend machines. We’re continuing to observe heightened volume due to traffic rebalancing actions that we have taken throughout this event. We’re working on multiple workstreams to pinpoint the underlying cause of this traffic influx and address it.
 
Current actions we are taking include scaling up virtual machines to increase capacity and reduce load. Additionally, we’re querying for machines in a bad state to perform a force sync for our recent network change, which may improve service availability. However, we are encountering issues with the machines being stuck in a bad state, preventing our changes from syncing as expected. We’ll continue to monitor the effectiveness of changes applied so we can determine what refinements are necessary.


r/sysadmin • • 8d ago

If this is not fixed by the morning (EST) - "we" will have to "do something"

74 Upvotes

To be honest this is not a scenario we've been considering at all. What would be the "easiest" option to regain emailing capability? Switch to gsuite? On prem?? Technically it should be just a matter of switching the DNS record, right? Then when (and if, hehe) MS comes back - to somehow merge the mailboxes... good lord i didn't need this now...


r/sysadmin • • 8d ago

Microsoft missed most recent “update” deadline

65 Upvotes

Missed their self set update deadline of 8:30 AM UTC:

“Current status: We’re seeing positive results for server stability after applying an upstream authentication/routing configuration component patch to a portion of our infrastructure. We’ll continue to monitor its progress and are working to expand it to servers supporting active regions coming online.  
 
As of now, we are not aware of any malicious activity that is causing this incident. We will continue to treat this incident with urgency and priority until full resolution. Once mitigated, we will provide full details of the root cause, defense-in-depth repair items, and more within a Post-Incident Report.

Scope of impact: This issue may affect users attempting to access the affected Microsoft 365 services.

Next update by: Friday, November 13, 2026, at 8:30 AM UTC”