On November 16, Charles Lamanna, who oversees Copilot and other services at Microsoft, announced on X the cause of the major Microsoft 365 outage that occurred four days prior. A latent bug in a patch program, which began distribution in July, surfaced when increased load combined with reduced processing capacity. At its most critical point, over 80% of the gateway servers that accept Microsoft 365 connections became unresponsive.
According to Lamanna, there were three main causes:
First, a bug in a patch program specifically for the gateway caused internal server processing to stall under certain load conditions.
Second, an increase in load on the gateway occurred approximately 15 hours before the outage due to a Teams communication path switch. Microsoft 365 has two communication path systems, and switching Teams traffic from one path to another, also used by Exchange Online, altered traffic volume and patterns.
Third, a separate issue led to a roughly 14% reduction in the gateway server's processing capacity.
The combination of these three factors triggered a bug that had not surfaced until then.
Traffic to unresponsive servers was rerouted to normally operating servers. However, the rerouted destinations also contained the same bug, leading to a cascading failure each time the load shifted. Mechanisms designed to contain the impact to regions or other units also failed due to a separate issue, causing the outage to spread globally. Services like Teams and OneDrive recovered earlier because they could bypass the problematic gateway, but Exchange Online could not take the same approach, resulting in a longer recovery time.
Furthermore, during recovery efforts, when servers were brought back online, waiting traffic surged, causing them to become unresponsive again. Microsoft isolated servers in groups, applied a fix to revert the problematic sections to their previous healthy state, and restarted them sequentially. The company has already stated that cyberattacks or AI agents were not the cause and plans to publish the final report later this week.