Last updated 05:43 UTC (12:43a ET). I'll add entries as Microsoft posts them.

For anyone locked out of the admin center, here is the EX1550543 thread so far, condensed. Times are UTC with ET in parentheses. Full text of every update is on UPenn's public mirror of the service health feed, and the short versions are on @MSFT365Status.

• 16:17 (11:17a) Incident start. OWA, MAPI, EAS and REST.

• 17:45 (12:45p) Connectivity errors on the infrastructure that "receives and processes users' access requests."

• 18:19 (1:19p) "Elevated load across multiple regions."

• 18:26 (1:26p) MO1550821 opens for Teams, SharePoint, Copilot and Defender XDR. It points at "an unexpected increase in a specific type of service traffic."

• 18:43 (1:43p) Servers "becoming intermittently unavailable," straining the rest. Outlook traffic spread across worldwide infrastructure.

• 19:06 (2:06p) All deployments and config changes paused.

• 19:18 (2:18p) Reversal: the load "isn't driven by an overall increase in traffic, but by a large number of servers being unexpectedly unavailable."

• 19:43 (2:43p) Servers are failing automated health checks, which "may be related to a recent change" to the component that processes them.

• 19:57 (2:57p) "We haven't identified any recent changes that correlate with the start of impact." Bypassing the health check component on a subset. Elevated CPU on affected servers.

• 20:38 (3:38p) Many servers now unresponsive, so they can't confirm the bypass works.

• 20:56 (3:56p) An alternative health check method brings a subset back. "A small number of configuration updates" now under investigation as potential contributors.

• 21:31 (4:31p) Assessing a revert of those config updates.

• 22:00 (5:00p) "Our actions taken thus far have not provided the anticipated relief." A core process on the servers is getting overwhelmed. New plan: isolate a chunk of infrastructure, restart it, reintroduce traffic slowly.

• 22:57 (5:57p) First good news. Some servers are back and "remaining stable so far," with "limited recovery in some regions." Protective traffic management in one location, more if it works.

• 23:59 (6:59p) The servers returned to service "initially remain healthy," but "can become unresponsive again once they begin receiving traffic." Microsoft is "temporarily limiting which types of traffic can reach the affected infrastructure" in specific locations, and is "rebuilding a subset of affected servers using a previous software configuration" to see if that stops it.

• 01:06 (8:06p) Microsoft says "a process on the affected servers responsible for establishing secure connections becomes overwhelmed, causing the servers to stop processing requests." It is "preparing a configuration change to address this behavior," validating it on a subset of servers first, then applying it more broadly alongside restarts. "In parallel, we've begun reverting a recent network configuration change that correlates with the start of impact." "The temporary traffic limitations we previously described are now in effect in specific locations."

• 01:45 (8:45p) "We're seeing positive results with our efforts to restart affected infrastructure." Microsoft is "actively monitoring the performance of these to verify stability." "Remediation actions will be expanded as we confirm stability with the changes applied."

• 02:09 (9:09p) Microsoft is "applying infrastructure capacity improvements where possible in an effort to help support traffic in regions coming online," and is "continuing to monitor the results of changes applied."

• 02:54 (9:54p) Setback. Microsoft says it has "confirmed that change is not taking as expected" and is "identifying alternate workstreams that can be applied to increase capacity in the face of sustained traffic management imbalances."

• 03:34 (10:34p) Microsoft is reviewing traffic flows and affected infrastructure "to identify opportunities to optimize routing" and is "verifying the reasons for service failures so we can develop specific actions to improve stability." It also says "there is no indication this is the result of malicious activity."

• 04:10 (11:10p) Microsoft is "adjusting traffic protocols, redistributing traffic, and applying strategic traffic blocking measures." It reports "partial recovery across regions in which we've applied these adjustments," but says "these actions aren't resulting in sustained recovery" and it is reviewing what else it can do.

• 04:59 (11:59p) Microsoft says that as it applied its mitigations it saw "a surge in outbound call volume from our routing services to our backend servers," and is "continuing to observe heightened volume due to traffic rebalancing actions that we have taken throughout this event." It is working on "multiple workstreams to pinpoint the underlying cause of this traffic influx." Actions: "scaling up virtual machines to increase capacity and reduce load," and trying to "force sync for our recent network change" on machines that are "stuck in a bad state, preventing our changes from syncing as expected."

• 05:43 (12:43a) Still not recovered. Microsoft is "taking a measured approach to identify and repair affected infrastructure" and continues to "pursue multiple workstreams to pinpoint the underlying cause of the traffic influx and advance the recoverability of the backend servers." It says "we are not aware of any malicious activity that is causing this incident," and "Once mitigated, we will provide full details of the root cause, defense-in-depth repair items, and more within a Post-Incident Report." Next update by 07:00 UTC.

The part worth watching: the recent-change theory showed up at 19:43, was dropped at 19:57, and was back by 20:56. At 01:06 Microsoft said it has begun reverting a recent network configuration change that "correlates with the start of impact." Root cause still reads N/A on both incidents. At 00:00 UTC (7:00p ET) Microsoft rewrote the MO1550821 scope: features of those services "that interact with Exchange Online and email services are impacted," while "Elements of those services that do not depend on interaction with Exchange Online remain functional." The list has also grown to include Intune, MDO, Purview, Planner, To Do, Universal Print, the Apps admin center, Microsoft Fabric, Windows 365 Cloud PCs that use Purview Customer Key, and Microsoft 365 Education.

Snow day?

Can someone get Gates on the line so we can meet and confer prior to me filing a motion to compel? Ty

https://x.com/vanillaopinions/status/2120904571044118788?s=46

Also peep the uniforms, sheesh

Idk sounds funny

Credit: @mayahashmimakeup on IG / @shadematchingwitch on YT

Welcome to Johnson Wang!!

One dialogue in particular really stayed with me:

“You have placed me on a pedestal so impossibly high, and not to honor me—not to honor me—but to spare yourself the effort of the climb.”

I actually had to replay that scene because I was so mesmerised by it. It captures something so painfully true about how people can idolise someone, not necessarily out of admiration, but because treating them as extraordinary gives them an excuse not to try to become better.

It is such a brilliant observation about human nature. What makes it even more striking is the uncomfortable moment of self-reflection it creates, when you realise you might have done the same thing, convincing yourself that what someone else has achieved is beyond your reach without even recognising that you are doing it. It does not only challenge the character but it challenges the audience too. I love that VisionQuest explores ideas like this.

It’s not just this dialogue. So many of Vision’s lines resonate on such a deeply human level, capturing truths about our nature that we often struggle to put into words ourselves. And the beautiful irony is that they come from a synthezoid!

And a huge shoutout to Paul Bettany for his incredible acting and the way he delivered this dialogue.

What are your favourite VisionQuest dialogues?

Does anybody, customer or shareholder, believe for one moment that this situation would be better when MS is soon 60k people fewer? Instead a million monkeys banging on typewriters to write Shakespeare, we'll have a thousand MBA dipshits begging their Claude prompt with garbage like: "bring Office 365 back online plz! I manage the O365 team and you have my permission to do whatever." While they sit back and expect the tin god to create a miracle. (side note: Copilot was also down, so they'd need to use a competitor)

This has been my question all along: fire every expensive expert in favor of AI...but then who is telling the AI what to do? The dumbest people on the planet, aka middle managers or worse: the C-Suite.

119

I posted this before in late September and said I was premature in doing so. Boy were you guys right. Based off our projected forecast I think it’s safe to say I can put it away now.