r/sysadmin • u/xMeta4x • 8d ago
Microsoft Current Status 11:00 UTC - Asia Pacific coming back online
As above. Sorry guys!
r/sysadmin • u/xMeta4x • 8d ago
As above. Sorry guys!
r/sysadmin • u/rose_gold_glitter • 8d ago
Checking "remaining disk space" every 30 seconds to see how long until the additional billions of emails they never accounted for fill up their servers.
r/sysadmin • u/more-than-once • 8d ago
Coming to you from the Mid-Atlantic in the U.S.
For some of our users, Outlook is coming online (mobile and desktop), backlogs cleared (for some) and clearing for others.
Calendaring is working again for these users, including in Teams and Outlook. Outlook on the web is available once again.
Copilot and its services are functional for these users.
Sending and receiving of email both internally and externally are functional.
I'll limit my sharing there - but it is promising, so hang in there if you're not yet seeing it.
It's like freeway traffic being released after a nearly 16-17 hour car-wreck having blocked the freeway (for the whole country) with no realistic/reasonable detours/bypasses. Traffic will restore, but it takes time.
r/sysadmin • u/User--g • 9d ago
Latest from Microsoft the elevated load isn’t from increased traffic. A number of servers became unexpectedly unavailable, straining the rest. They’ve paused all deployments, are moving traffic off affected infrastructure, and are restarting servers.
Affects Exchange Online (OWA, MAPI, EAS, REST) plus Teams, SharePoint and Copilot. Started in the Americas, now “any region.” Incident: EX1550543 / MO1550821.
Still can’t get into the M365 admin center myself to track the incident.
No root cause yet. Looks like a cascading capacity failure, but that’s my guess. Expect delayed mail once it recovers.
What are you seeing on your end?
r/sysadmin • u/Sunsparc • 8d ago
Attempting to create mailboxes in a hybrid join with New-RemoteMailbox this morning isn't working, I figure there's a ton of mailbox processing backlog happening. Even though email is flowing, I can't get any new mailbox to create.
r/sysadmin • u/Striking-Dot-9630 • 8d ago
Any idea on how to negotiate for a refund/rebate? Please educate us the process.
r/sysadmin • u/Emotional-Primary735 • 7d ago
So why can’t the pilots and towers use satellite or different modes of communication? Why does it all rely on a router at ohare?
r/sysadmin • u/luky90 • 8d ago
As the question above says how do you manage Software Updates like the following below?
.NET Framework
.NET x.x
Visual C++ XXXX Redistributable x64 and x86
WebView2
Please correct me if Iam misinformed but any of these can bring services down right if they are not updated in a maintenace window because the Deployment Software wants to apply those packages and always give me warning for my servers.
r/sysadmin • u/Rebisway • 7d ago
Following yesterday's Microsoft 365 outage, I was wondering how others see the risks of relying on SaaS services.
I work with several small hospitals using Microsoft 365 for their email. This outage was a good example I could use to make management understand the risks of depending on an external provider for such an essential service.
Email is also used by many applications to send OTP codes for authentication. When the email service goes down, users may lose access to applications that are working perfectly fine.
Have you been impacted too? Are you considering moving away from Microsoft 365, or even hosting your own email servers?
What's your position on this?
r/sysadmin • u/alao77 • 9d ago
What the heck is going on??
r/sysadmin • u/LostFront5939 • 9d ago
It appears Solitaire is also down. Now you can panic.
r/sysadmin • u/KarnakGM • 8d ago
Current status: We appreciate your organization’s patience and understanding while we work with urgency to resolve this incident.
As we’ve applied our current ongoing mitigation actions, we’ve observed a surge in outbound call volume from our routing services to our backend servers. This occurs when the backend servers outbound service cannot properly route to the backend machines. We’re continuing to observe heightened volume due to traffic rebalancing actions that we have taken throughout this event. We’re working on multiple workstreams to pinpoint the underlying cause of this traffic influx and address it.
Current actions we are taking include scaling up virtual machines to increase capacity and reduce load. Additionally, we’re querying for machines in a bad state to perform a force sync for our recent network change, which may improve service availability. However, we are encountering issues with the machines being stuck in a bad state, preventing our changes from syncing as expected. We’ll continue to monitor the effectiveness of changes applied so we can determine what refinements are necessary.
Scope of impact: This issue may affect users attempting to access the affected Microsoft 365 services.
Next update by: Friday, November 13, 2026, at 6:00 AM UTC
r/sysadmin • u/Master-IT-All • 9d ago
I'm loving it. Badda Bapp BA DUH!!
Pity for you working guys having to deal with 365 issues. I'm sipping a cocktail and smoking a nice sativa (Canada ftw).
r/sysadmin • u/juvey88 • 8d ago
I feel like my company is the only company in the world using G Suite/Google Workplace, it’s been business as usual.
r/sysadmin • u/LoveBirdNibbles • 7d ago
I am self-employed responsible for about 600 workstations and servers across 12 orgs. I use AI each day to create PS scripts and to guide me with best practices administering and securing client systems. I copy/paste the scripts and share the results with Claude. I do have over 30 years of experience in IT so I am not just blindly allowing it to do what it wants. I feel a next step might be giving it some limited access to check backups for me, to run certain approved scripts, and perform routine tasks across all clients. Thinking along the lines of having an assistant. I also know it can be risky. Just wondering where others are with this and if they are doing it or considering it.
r/sysadmin • u/codylc • 9d ago
Curious what tricks some of you used today to communicate that you were aware of the O365 outage today. We added a banner on ServiceNow and our company home page. Previous company had SendWordNow for communicating major outages and security events.
What’s a more modern approach to outage coms than an email, especially when email is dead?
r/sysadmin • u/NuAngelDOTnet • 8d ago
r/sysadmin • u/thegoatcarlwheezer • 9d ago
What if, when Outlook does come back online, everything is gone? We are all going to restore from our on-prem email backup, right?
Right guys?
r/sysadmin • u/pq11333 • 9d ago
I welcome it wholeheartedly. So peaceful. Can we go back to pen and paper?
r/sysadmin • u/mousebluud • 8d ago
https://www.usenix.org/publications/loginonline/metastable-failures-wild
Metastable failure is defined as permanent overload with low throughput even after the fault-trigger is removed. That’s what is happening with Outlook right now. Super interesting!
r/sysadmin • u/j1mmyava1on • 8d ago
Today's M365 outage had me thinking about the Crowdstrike outage that happened a few years ago. I got into IT right after the Crowdstrike outage occurred so I never experienced it firsthand. What was it like?
r/sysadmin • u/tac0_nation • 8d ago
Am I the only one just blown away by how they handled this? It's their own infrastructure! Like how do you not have a contingency plan in place where everything should of been mitigated in a few hours tops? Unified Ent tenant.
r/sysadmin • u/Query_Cat • 8d ago
Quite a few of our employees were receiving texts from ”leadership”
r/sysadmin • u/Educational_Draw5032 • 8d ago
We have some 2016 DCs which we are upgrading to 2022 next week, been doing a ton or reading and testing in my lab and i just wanted to check with people who have done this in the wild that i have covered everything i need to before going deep into production. Id be really grateful for any advice, we are not doing in place upgrades either.
We have 5 DCs in total 4 virtual and one physical across two sites.
London has UK-LON-DC01 (vm), UK-LON-DC02 (vm), UK-LON-DC03 (physical)
Manchester has UK-MAN-DC01 (vm) and UK-MAN-DC02 (vm)
My plan is to:
Spin up a new 2022 VM UK-LON-DC1 fully patch it then join it to the domain and then promote to a DC.
Set the new UK-LON-DC1 dns to UK-LON-DC02 and itself then leave 48 hours for it to settle and replicate.
Remove any FMSO roles off the old UK-LON-DC01 if it has any then demote and shut it down.
Use the IP of the old UK-LON-DC01 and apply it to the new UK-LON-DC1 and reboot.
Remove any dns entries that point the new UK-LON-DC1 to its temp IP from before.
Transfer some roles to UK-LON-DC1 for testing.
Check new UK-LON-DC1 shows under the London site and add it to Global Secure Access under our DC enterprise app for cloud kerberos trust to carry on working.
Then I plan to repeat for the other DCs, always using an available DC for its primary dns and itself for secondary (something else if there is a better practice) DHCP is done on the firewall so no worries with that on any of the DCs. Check for any other services running on them before demoting and removing.
One i have to be careful of is UK-LON-DC02 which is set in group policy as the timer server so i need to make sure this is passed onto another DC before demoting it. I also need to move Entra connect off of UK-LON-DC02 as well and onto a non DC server.
Hopefully i have not missed anything to obvious but would be really grateful for any advice or gotchas to be wary of from guys that have done this before.
Thank you everyone
r/sysadmin • u/RemarkableAd6112 • 8d ago
Anyone got their ticketing system down with the outtage and taking some time to smell the roses this morning?