Microsoft Threat Protection leads in real-world detection in MITRE ATT&CK evaluation

The latest round of MITRE ATT&CK evaluations proved yet again that Microsoft customers can trust they are fully protected even in the face of such an advanced attack as APT29. When looking at protection results out of the box, without configuration changes, Microsoft Threat Protection (MTP):

• Provided nearly 100 percent coverage across the attack chain stages.
• Delivered leading out-of-box visibility into attacker activities, dramatically reducing manual work for SOCs vs. vendor solutions that relied on specific configuration changes.
• Had the fewest gaps in visibility, diminishing attacker ability to operate undetected.

Beyond just detection and visibility, automation, prioritization, and prevention are key to stopping this level of advanced attack. During testing, Microsoft:

• Delivered automated real-time alerts without the need for configuration changes or custom detections; Microsoft is one of only three vendors who did not make configuration changes or rely on delayed detections.
• Flagged more than 80 distinct alerts, and used built-in automation to correlate these alerts into only two incidents that mirrored the two MITRE ATT&CK simulations, improving SOC analyst efficiency and reducing attacker dwell time and ability to persist.
• Identified seven distinct steps during the attack in which our protection features, which were disabled during testing, would have automatically intervened to stop the attack.

Microsoft Threat Experts provided further in-depth context and recommendations for further investigation through our comprehensive in-portal forensics. The evaluation also proved how Microsoft Threat Protection goes beyond just simple visibility into attacks, but also records all stages of the attack in which MTP would have stepped in to block the attack and automatically remediate any affected assets.

While the test focused on endpoint detection and response, MITRE’s simulated APT29 attack spans multiple attack domains, creating opportunities to empower defenders beyond just endpoint protection. Microsoft expanded defenders’ visibility beyond the endpoint with Microsoft Threat Protection (MTP). MTP has been recognized by both Gartner and Forrester as having extended detection and response capabilities. MTP takes protection to the next level by combining endpoint protection from Microsoft Defender ATP (EDR) with protection for email and productivity tools (Office 365 ATP), identity (Azure ATP), and cloud applications (Microsoft Cloud App Security [MCAS]). Below, we will share a deep-dive analysis and explanation of how MTP successfully demonstrated novel optic and detection advantages throughout the MITRE evaluation that only our solution can provide.

Incident-based approach enables real-time threat prioritization and remediation

Analyzing the MITRE evaluation results from the lens of breadth and coverage, as the diagrams below show, MTP provided exceptional coverage for all but one of the 19 tested attack stages. This means that in real life, the SOC would have received alerts and given full visibility into each of the stages of the two simulated attack scenarios across initial access, deployment of tools, discovery, persistence, credential access, lateral movement, and exfiltration. In Microsoft Threat Protection, alerts carry with them rich context—including a detailed process tree showing the recorded activities (telemetry) that led to the detection, the assets involved, all supporting evidence, as well as a description of what the alert means and recommendations for SOC action. Note that true alerts are attributed in the MITRE evaluation with the “Alert” modifier, and not all items marked as “Tactic” or “Technique” are actual alerts.

Figure 1: MTP detection coverage across the attack kill-chain stages, with block opportunities.

Note: Step 10, persistence execution, is registered as a miss due to a software bug, discovered during the test, that restricted visibility on Step 10—“Persistence Execution.” These evaluations are a valuable opportunity to continually improve our product, and this bug was fixed shortly after testing completed.

The MITRE APT29 evaluation focused solely on detection of an advanced attack; it did not measure whether or not participants were able to also prevent an attack. However, we believe that real-world protection is more than just knowing that an attack occurred—prevention of the attack is a critical element. While protections were intentionally turned off to allow the complete simulation to run, using the audit-only prevention configuration, MTP also captured and documented where the attack would have been completely prevented, including—as shown in the diagram above – the very start of the breach, if protections had been left on.

Microsoft Threat Protection also demonstrated how it promotes SOC efficiency and reduces attacker dwell time and sprawl. SOC alert fatigue is a serious problem; raising a large volume of alerts to investigate does not help SOC analysts understand where to devote their limited time and resources. Detection and response products must prioritize the most important attacker actions with the right context in near real time.

In contrast to alert-only approaches, MTP’s incident-based approach automatically identifies complex links between attacker activities in different domains including endpoint, identity, and cloud applications at an altitude that only Microsoft can provide because we have optics into each of these areas. In this scenario, MTP connected seemingly unrelated alerts using supporting telemetry across domains into just two end-to-end incidents, dramatically simplifying prioritization, triage, and investigation. In real life, this also simplifies automated response and enables SOC teams to scale capacity and capabilities. MITRE addresses a similar problem with the “correlated” modifier on telemetry and alerts but does not reference incidents (just yet).

Figure 2: MTP portal showing 2^nd day attack incident including correlated alerts and affected assets.

Figure 3: 2^nd day incident with all correlated alerts for SOC efficiency, and the attack incident graph.

Microsoft is the leader in out-of-the-box performance

Simply looking at the number of simulation steps covered—or, alternatively, at the number of steps with no coverage, where less is more—the MITRE evaluation showed MTP provided the best protection with zero delays or configuration changes.

Microsoft believes protection must be durable without requiring a lot of SOC configuration changes (especially during an ongoing attack), and it should not create friction by delivering false positives.

The chart below shows Microsoft as the vendor with the least number of steps categorized as “None” (also referred to as “misses”) out of the box. The chart also shows the number of detections marked with “Configuration Change” modifier, which was done quite considerably, as well as delayed detections (“Delayed” modifier), which indicate in-flight modifications and latency in detections.

Microsoft is one of only three vendors that made no modifications or had any delays during the test.

Similarly, when looking at visibility and coverage for the 57 MITRE ATT&CK techniques replicated during this APT29 simulation, Microsoft’s coverage shows top performance at 95 percent of the techniques covered, as shown in the chart below.

A product’s coverage of techniques is an important consideration for customers when evaluating security solutions, often with specific attacker(s) in mind, which in turn determines the attacker techniques they are most concerned with and, consequently, the coverage they most care about.

Figure 5: Coverage across all attack techniques in the evaluation.

MTP provided unique detection and visibility across identity, cloud, and endpoints

The powerful capabilities of Microsoft Threat Protection originate from unique signals not just from endpoints but also from identity and cloud apps. This combination of capabilities provides coverage where other solutions may lack visibility. Below are three examples of sophisticated attacks simulated during the evaluation that span across domains (i.e., identity, cloud, endpoint) and showcase the unique visibility and unmatched detections provided by MTP:

• Detecting the most dangerous lateral movement attack: Golden Ticket—Unlike other vendors, MTP’s unique approach for detecting Golden Ticket attacks does not solely rely on endpoint-based command-line sequences, PowerShell strings like “Invoke-Mimikatz”, or DLL-loading heuristics that can all be evaded by advanced attackers. MTP leverages direct optics into the Domain Controller via Azure ATP, the identity component of MTP. Azure ATP detects Golden Ticket attacks using a combination of machine learning and protocol heuristics by looking at anomalies such as encryption downgrade, forged authorization data, nonexistent account, ticket anomaly, and time anomaly. MTP is the only product that provided the SOC context of the encryption downgrade, together with the source and target machines, resources accessed, and the identities involved.
• Exfiltration over alternative protocol: Catching and stopping attackers as they move from endpoint to cloud—MTP leverages exclusive signal from Microsoft Cloud App Security (MCAS), the cloud access security broker (CASB) component of MTP, which provides visibility and alerts for a large variety of cloud services, including OneDrive. Using the MCAS Conditional Access App Control mechanism, MTP was able to monitor cloud traffic for data exfiltration and raise an automatic alert when a ZIP archive with stolen files was exfiltrated to a remote OneDrive account controlled by the attacker. It is important to note the OneDrive account used by MITRE Redteam was unknown to the Microsoft team prior to being automatically detected during the evaluation.
• Uncovering Remote System Discovery attacks that abuse LDAP—Preceding lateral movement, attackers commonly abuse the Lightweight Directory Access Protocol (LDAP) protocol to query user groups and user information. Microsoft introduced a powerful new sensor for unique visibility of LDAP queries, aiding security analyst investigation and allowing detection of suspicious patterns of LDAP activity. Through this sensor, Microsoft Defender ATP, the endpoint component of MTP, avoids reliance on PowerShell strings and snippets. Rather, Microsoft Defender ATP uses the structure and fields of each LDAP query originating from the endpoint to the Domain Controller (DC) to spot broad requests or suspicious queries for accounts and groups. Where possible, MTP also combines and correlates LDAP attacks detected on the endpoint by Microsoft Defender ATP with LDAP events seen on the DC by Azure ATP.

Figure 6: Golden Ticket alert based on optics on Domain Controller activity.

Figure 7: Suspicious LDAP activity detected using deep native OS sensor.

Microsoft Threat Experts: Threat context and hunting skills when and where needed

In this edition of MITRE ATT&CK evaluation, for the first time, Microsoft products were configured to take advantage of the managed threat hunting service Microsoft Threat Experts. Microsoft Threat Experts provides proactive hunting for the most important threats in the network, including human adversary intrusions, hands-on-keyboard attacks, or advanced attacks like cyberespionage. During the evaluation, the service operated with the same strategy normally used in real customer incidents: the goal is to send targeted attack notifications that provide real value to analysts with contextual analysis of the activities. Microsoft Threat Experts enriches security signals and raises the risk level appropriately so that the SOC can focus on what’s important, and breaches don’t go unnoticed.

Microsoft Threat Experts notifications stand out among other participating vendors as these notifications are fully integrated into the experience, incorporated into relevant incidents and connected to relevant events, alerts, and other evidence. Microsoft Threat Experts is enabling SOC teams to effortlessly and seamlessly receive and merge additional data and recommendations in the context of the incident investigation.

Figure 8: Microsoft Threat Experts alert integrates into the portal and provides hyperlinked rich context.

Transparency in testing is key to threat detection, prevention

Microsoft Threat Protection delivers real-world detection, response, and, ultimately, protection from advanced attacks, as demonstrated in the latest MITRE evaluation. Core to MITRE’s testing approach is emulating real-world attacks to understand whether solutions are able to adequately detect and respond to them. We saw that Microsoft Threat Protection provided clear detection across all categories and delivered additional context that shows the full scope of impact across an entire environment. MTP empowers customers not only to detect attacks, offering human experts as needed, and easily return to a secured state with automated remediation. As is true in the real world, our human Threat Experts were available on demand to provide even more context and help with.

We thank MITRE for the opportunity to contribute to the test with unique threat intelligence that only three participants stepped forward to share. Our unique intelligence and breadth of signal and visibility across the entire environment is what enables us to continuously score top marks. We look forward to participating in the next evaluation, and we welcome your feedback and partnership throughout our journey.

Thanks,

Moti and the entire Microsoft Threat Protection team

Related Links:

READ MORE HERE