Threat Hunting: Rethinking ‘Needle in a Haystack’ Security Defenses

In cyber, needles (that is, threats) can disappear quickly, for a variety of reasons, and long often after hackers have completed what they came to do.

Business executives are finally getting the message from IT and security leaders that they need to be more proactive when it comes to cybersecurity. They can’t afford to let their cybersecurity teams wait for alerts that may come too late to stop a minor intrusion from becoming a major breach. Threat hunting is the approach business leaders need to detect these incidents early enough to stop them.

Where leadership holds both authority and responsibility for these functions, they may not know enough about threat hunting to provide much-needed direction. It’s often up to frontline defenders to figure out how to get that initiative on stable footing.

At any organization, the sheer number of “events” to sort through can make early detection daunting. It’s easy to reach for the “needle in a haystack” metaphor, but this is a flawed perception of the problem. The old saying assumes that you know there is a needle, that you know what a needle looks like, and that it is in fact a needle you’re looking for. This doesn’t address the fact that, in the cyber world, needles (that is, threats) can disappear quickly for a wide variety of reasons — and often long after the malicious party has completed what he or she came to do.

Although there are many factors for cybersecurity teams to juggle, getting started isn’t hard.

Read the Hacker Playbook 
Cybersecurity professionals who support detection and response have an advantage over their adversaries that might not be obvious. Independent groups like MITRE have conducted research on the techniques and tactics used by threat actors, which they have released under the ATT&CK framework. By studying and understanding this knowledge base, analysts and other professionals can focus their efforts to remain ahead of threats.

Where other models oversimplify categories of techniques, attempt to apply a one-size-fits-all approach to complex behaviors, or assign too much significance to the early pre-compromise stages of an attack, ATT&CK is a comprehensive and threat-agnostic resource that emphasizes the importance of a data-driven approach. By using a resource like ATT&CK and adopting a quantitative method of measuring coverage, teams responsible for monitoring and response can more effectively hunt.

Take Action
The ATT&CK framework can seem overwhelming at first, given that it enumerates hundreds of individual techniques and tactics across Windows, Linux, and macOS systems. New threat-hunting teams without clear direction from their leaders may feel they need to tackle everything at once. That leads to doing none of them well and contributes to poor retention and satisfaction rates that leave major gaps in the cybersecurity teams.

Fortunately, full coverage isn’t necessary to significantly improve a cybersecurity program. Starting small and building momentum gives threat-hunting teams a chance to earn some early success and learn more about how to conduct threat hunts.

There is no prescribed approach to getting started, but a data-driven approach helps provide some guidance. In my experience, the most effective place to start is an assessment of available sources of evidence such as running processes and network metadata for availability, timeliness, and quality. By understanding your data, security teams can understand which threat-hunting actions are possible in their environment. They can also learn where they need to make visibility improvements to be able to do more.

Each new hunt will become easier for the defenders as they get a better understanding of the processes. They will also improve their understanding of their operating environment. From there, they can expand the scope of adversary behaviors they’re looking for to find more malicious activity and prepare a defense for a wider variety of attacks.

Important to the continuing support of this program is active, quantifiable measurement. Being able to show IT cybersecurity and organizational leaders that threat hunts are having a measurable impact on the team’s ability to stop breaches helps them justify continuing to provide or even increase budgets and other resources.

Qualitative Assesments
The ATT&CK matrix can help by giving cybersecurity teams a concrete pin on which to hang their results. Using qualitative scales for assessment — such as “low,” “medium,” or “high” — leaves organizations guessing about whether adversaries are active in their environment. But those who adopt a quantitative scale can point to entire categories or individual techniques where attacks weren’t active or where they were prevented.

This continuing stream of information about the success of the threat-hunting program as it expands will win friends with the relevant decision makers. It is also important for cybersecurity teams to have a champion in the organization to enable continued success.

Cybersecurity teams are sometimes seen as the “bad guys” of the IT department because security professionals often make their jobs harder to do. Having a champion who can demonstrate the unseen benefits of a cybersecurity program will reduce the amount of “political capital” execustives need to spend to maintain an effective threat-hunting program. Organizations that are struggling to make progress with threat hunting and detection may be trying to take on too much too soon, failing to quantify their results, or expending more political capital than they earn.

Threat hunting may seem a like a daunting task, and the bigger the enterprise to defend, the more daunting it seems. Starting small against the most common hacker techniques and building steadily will make every search a little easier every time. (You can click here for further tips on setting up a threat-hunting program.) Some of the most common techniques and data sources for threat hunting are covered in this recent talk at BSides Charm 2018. Knowing the hacker playbook and using it against them makes it easier to stop threats before they make the company another breach headline.

Related Content:

Learn from the industry’s most knowledgeable CISOs and IT security experts in a setting that is conducive to interaction and conversation. Register before July 27 and save $700! Click for more info. 

Devon Kerr is a principal researcher at Endgame, focusing on detection and response technologies. Formerly a Mandiant incident response and remediation lead, Devon has over 6 years of experience in security professional services where he has worked with clients in a nearly … View Full Bio

More Insights

Read More HERE