Microsoft released a preliminary post-incident report Sunday detailing the technical failures behind a global Microsoft 365 outage that disrupted email, workplace communications and cloud services for nearly 24 hours last week. The company attributed the November 12-13 disruption to a previously undetected software defect that was triggered by increased network traffic and reduced server capacity, causing failures to cascade across its infrastructure. At the height of the incident, more than 80% of servers supporting a critical network gateway were unresponsive, leaving users worldwide unable to access essential services. The findings, published under incident identifiers MO1550821 and EX1550543, provide the company’s most detailed explanation of the disruption since services were restored.
The outage primarily affected Exchange Online, Microsoft’s cloud-based email platform, preventing most affected users from accessing their mailboxes through Outlook and other email clients. Many users also experienced delayed email delivery, while services including Microsoft Teams, Copilot, SharePoint, OneDrive, Defender and Purview experienced disruptions involving features dependent on Exchange Online. Microsoft said the severity of the impact varied by region, connection method and time, with some customers experiencing intermittent access throughout the incident. Although several applications were affected, some services remained partially operational, including Teams chats and meetings, which functioned for most users during much of the outage.
According to the report, the first contributing factor was a software defect introduced through an update that began deployment in July and remained undetected for more than three months. The update, developed specifically for servers supporting Microsoft’s networking gateways, contained a defect that caused competing internal processes to overwhelm available processor and memory resources under certain conditions. Microsoft acknowledged that its testing procedures failed to identify the problem because laboratory environments relied primarily on less powerful hardware, while the defect became more pronounced on newer systems with greater processing capacity. The company emphasized that neither the underlying defect nor its associated software update affected general Windows installations, limiting the issue to specialized infrastructure supporting Microsoft 365 services.
The second factor emerged when Microsoft Teams traffic was automatically transferred from an alternative networking gateway to the primary gateway also supporting Exchange Online approximately 15 hours before the outage began. The transfer, part of Microsoft’s normal service-resilience procedures, increased demand on infrastructure already operating with reduced capacity. Microsoft disclosed that nearly 14% of the gateway’s server capacity had been unavailable because of an unrelated issue identified the previous day, although the reduction alone was not sufficient to cause an outage. The combination of additional traffic, altered network patterns and reduced capacity ultimately triggered the dormant software defect during a period of peak demand in the United States.
Once the defect was triggered, Microsoft’s automated traffic management systems attempted to redirect affected users to healthy servers, a process ordinarily designed to prevent widespread interruptions. Instead, the redirected traffic overwhelmed additional servers, producing a cascading failure as increasingly large volumes of requests were distributed across a shrinking pool of available infrastructure. Microsoft said the problem spread across its server fleet in less than two hours, while separate failures in safeguards intended to isolate regional infrastructure allowed the disruption to expand globally. Recovery efforts were complicated by servers becoming unresponsive again immediately after being restored, forcing engineers to develop and deploy a targeted software update before gradually returning infrastructure to service.
Microsoft identified the corrective software update early Friday and began systematically deploying it across affected regions, prioritizing infrastructure supporting customers entering peak business hours. Engineers isolated groups of servers, applied the fix and restarted the affected systems before gradually restoring network traffic to prevent further overloads. The company reported that global webmail availability returned to expected levels Friday afternoon, with the active incident ending at 4:10 p.m. UTC, or 11:10 a.m. Eastern time, on November 13. Microsoft continued monitoring the infrastructure and processing delayed email messages before announcing that service had been fully restored across affected customers and regions.
The disruption extended beyond individual users, affecting businesses, government agencies and financial institutions that depend on Microsoft’s cloud-based services for daily operations. According to Reuters, the outage disrupted government operations in several countries, delayed corporate announcements and forced some financial firms to rely on alternative communication methods, including fax machines, to complete transactions. Downdetector recorded more than 7.4 million problem reports worldwide during the incident, illustrating the scale of the disruption, although those reports do not represent a verified count of individual users affected. Microsoft said the incident was technical in nature and was not caused by a cyberattack or artificial intelligence systems.
Microsoft said it has completed several immediate corrective measures, including restoring the affected connection-handling software, adding infrastructure capacity and introducing additional monitoring to detect potentially dangerous traffic patterns. The company also extended its email delivery window to prevent delayed messages from being automatically returned as undeliverable while engineers worked through the backlog. Additional safeguards have been implemented to reduce the risk of automated traffic redirection triggering similar cascading failures in the future. A final post-incident report, expected later this week, will outline further corrective measures and may revise the preliminary findings, with a more comprehensive list of repair actions scheduled for November 20.



Comments