Microsoft reveals extent of CrowdStrike glitch
Microsoft reveals: 8.5 million Windows devices affected by CrowdStrike glitch. The incident had far-reaching effects, including in Germany. Airports, banks and hospitals struggled with IT problems. Microsoft has revealed the true extent of the recent CrowdStrike incident disclosed. David Weston, Vice President of Enterprise and OS Security at Microsoft, said that around 8.5 million Windows devices were affected by the faulty update from cybersecurity company CrowdStrike.
This number corresponds to less than one percent of all Windows machines worldwide. The incident, which began on July 19, caused massive disruptions in various sectors around the globe, including Germany. Banks, airlines and media companies were among those affected who had to deal with the consequences of the problematic update. The faulty update of the CrowdStrike Falcon sensor caused continuous reboots on the affected Windows PCs, accompanied by error codes 0x50 or 0x7E – better known as the “Blue Screen of Death” (BSOD).
Far-reaching effects
Although the number of affected devices may seem small in relation to the total number of Windows systems, the impact was significant. In Germany and other countries, several critical areas were affected:
- Airports: Berlin Airport warned of delays due to “technical problems”. International airlines such as Lufthansa, Delta and United Airlines were also affected by disruptions.
- Banking systems: Several German banks reported disruptions to their online services. Customers had temporary problems accessing their accounts or carrying out transactions.
- Healthcare: Some hospitals in Germany had to postpone planned operations because their IT systems were compromised.
- Media: The British news channel Sky News was temporarily unable to broadcast live news.
These examples demonstrate the central role that CrowdStrike products play in many companies and organizations. CrowdStrike says it serves over 298 of the Fortune 500 companies, 43 U.S. states, 6 of the 10 largest healthcare providers, and 8 of the 10 leading financial services providers in the United States.
Cross-industry collaboration
The challenge of repairing 8.5 million affected PCs is immense. Microsoft expects that full recovery could take several weeks. To speed up the process, Microsoft is working closely with CrowdStrike and other leading cloud providers, including Amazon AWS and Google Cloud. David Weston emphasized the importance of collaboration in such crisis situations in his blog post:
This incident demonstrates the interconnectedness of our broad ecosystem – global cloud providers, software platforms, security vendors and other software providers, and customers. It also reminds us how important it is for all of us in the tech ecosystem to prioritize operations with secure deployment and disaster recovery practices using the mechanisms in place. David Weston on the Microsoft blog
Lessons from the incident
The incident highlights the need for cybersecurity companies like CrowdStrike to exercise greater caution and care when deploying updates to a large number of systems. However, the rapid response and collaboration of various technology companies also shows the strength of the industry in crisis situations. For IT professionals and system administrators in Germany and around the world, this incident offers important insights:
- Importance of backup systems: The incident highlights the importance of having functioning backup systems and emergency plans, especially for critical infrastructure such as hospitals and airports.
- Be careful with automatic updates: Although automatic updates usually increase security, it may be worth delaying the update of critical systems to avoid such problems.
- Diversify security solutions: Over-reliance on a single security provider can lead to far-reaching problems in the event of such an incident.
Technical details and solutions
According to initial analyses, the cause of the blue screen appears to be a so-called null pointer error. The code attempted to access an invalid memory address without first checking its validity. For systems already affected, CrowdStrike recommended the following interim solution:
- Starting the system in safe mode
- Navigate to the folder c:windowssystem32driverscrowdstrike
- Deleting the file “C-00000291*.sys”
- Restart the system
However, this solution poses significant challenges for IT administrators, especially for cloud-based servers or remote workstations. In addition, users in managed environments often require the Bitlocker recovery key, which further complicates the situation.
Outlook and consequences
While the technology industry continues to work to resolve the aftermath of this incident, it remains to be seen what long-term consequences companies and cybersecurity providers will face. This incident is expected to lead to increased discussions about best practices for software updates and securing critical infrastructure. What do you think about this incident? Did you also have to deal with the aftermath in your company? Share your experiences and thoughts with us in the comments. We are curious to hear your perspectives – whether from small businesses or large corporations!
What does CrowdStrike do? CrowdStrike is a leading cybersecurity company specializing in endpoint protection. It offers security solutions for Windows and Linux systems that are used in many large companies and organizations around the world. According to its own information, CrowdStrike serves over 298 of the Fortune 500 companies, 43 US states, 6 of the 10 largest healthcare providers and 8 of the 10 leading financial services providers in the US. This underlines the company’s central role in the global IT security landscape. What was the reason for the outage? The massive outage was caused by a faulty update of the CrowdStrike Falcon sensor.
This update led to a so-called “Blue Screen of Death” (BSOD) on affected Windows systems, resulting in continuous reboots. According to initial analyses, the cause appears to be a null pointer error. The code attempted to access an invalid memory address without first checking its validity. This basic programming error should have been discovered during careful testing. How many devices were affected? According to Microsoft, approximately 8.5 million Windows devices were affected by the faulty update.
This number represents less than one percent of all Windows machines worldwide. Despite the relatively small proportion of the total number of Windows systems, the impact was significant, as many critical infrastructures and large companies were affected. Full recovery of all affected systems could take several weeks. Which industries were affected? The outage had a far-reaching impact on various industries worldwide.
Airports and airlines were particularly affected, struggling with delays and flight cancellations. Banking systems also reported disruptions to their online services, causing problems with account access and transactions. In the healthcare sector, some hospitals had to postpone planned operations. Even media companies such as the British news channel Sky News were affected and were temporarily unable to broadcast live news. How was the problem resolved? CrowdStrike has since withdrawn the faulty update and is working on a global solution. For systems already affected, a workaround has been recommended that involves booting into safe mode and manually deleting a specific file.
Microsoft is working closely with CrowdStrike and other leading cloud providers such as Amazon AWS and Google Cloud to speed up the recovery process. Nevertheless, it is expected that the full fix could take several weeks. Have there been similar incidents before? It is reported that there have been similar problems with CrowdStrike updates in the past. In April of this year, an update is said to have caused crashes on servers running Debian and Rocky Linux.
In these cases, too, deficiencies in CrowdStrike’s testing processes apparently became apparent. There was criticism that certain operating system versions were not part of the test matrix, which led to compatibility issues. These incidents raise questions about quality assurance at CrowdStrike. What does CrowdStrike say about this? George Kurtz, CEO of CrowdStrike, commented on the incident in a television interview. He expressed his deep regret over the incident and assured that the company is working with each individual customer to bring the systems back online. Kurtz indicated that CrowdStrike will launch an internal investigation into the incident. However, his statements were met with criticism by some IT managers because they did not adequately address the severity of the incident and the underlying errors.
