Threat hunting has been around for a while, but only recently it has become a sort of buzzword in cybersecurity circles. Besides security professionals, more and more organizations recognize that proactive hunting needs to play a role in their overall security practices.

Threat hunting (TH) is the human-driven, proactive search through systems, networks, endpoints, and datasets to detect vulnerabilities or malicious activity that existing tools could not catch. In other words, it is a search for threats relying on security solutions or services rather than automated solutions.

It may seem like a step back from technology to manual labor. And in a way it is. But it’s a necessary step in a world where there is no such thing as a secure computer and the stakes are getting ever higher, as more and more organizations and services move to the cloud or connect to the Internet.

And it works. A 2020 survey on threat hunting conducted by the SANS Institute, a prominent US security company, showed that thanks to threat hunting 89% of organizations saw improved detection creation and fewer false positives, 68% – increased breakout time, 70% – increased exfiltration detection, and 75% of respondents needed fewer resources for remediation activities.

What is required for cyber threat hunting?

To develop a proactive threat-hunting strategy, good preparation, and planning are crucial.

First, there are threat hunters – people who will be the major security force in your organization. Besides them, you will need actionable data and, somewhat surprisingly, a toolset. We’ll talk about these below.

Team

You may have an in-house team of hunters, or you may be the one hunting. In either case, a good level of expertise is important. Having said that, you or your security experts don’t have to belong to the 1% elite in the security world. Though TH is a new term, security analysts have been hunting for threats for years. You can start with basic techniques and build on them.

But it is critical to be adaptive, grow your knowledge of the threat landscape, constantly learn about your IT environment, be creative and intuitive – those are the core qualities for any cyber threat hunter.

Data

Due to the proactive nature of cyber threat hunting, you will be heavily using collected data to identify and categorize potential threats. And you will need a lot of it.

This data will be coming from logs from network transactions, operating system events, applications, etc. Emails, employee information, and access privileges are also useful for detecting internal anomalies. Keep as much data as you can store.

Another good way to collect data for threat hunting is by studying published materials about recent attacks, new threats, attacker behaviors, etc. Good sources for such data are Threat Intelligence Providers (TIPS) and industry threat data banks. Using their data, a security analyst can create new hypotheses and identify future threats.

Tools

Once you have collected what you could about your environment, you or your security analyst can start combing through the data. You will need an effective way of making sense of it, and it should not have to be a manual process. To boost the efficiency of your data analysis, use automation, AI, machine learning, and user and entity behavior analytics (UEBA) solutions. These tools can automate regular tasks like generating activity summaries or searching for suspicious entities in data.

While purpose-built tools like Threat Hunting Platforms (THPs) can help you hunt at scale and simplify advanced hunting procedures. For example, Sqrrl’s Threat Hunting Platform has been specifically created to make fusing different datasets significantly simpler.

Besides THPs, for security data with actionable indicators, there are such solutions as Security Information and Event Management (SIEM), statistical analysis tools (SAS), and the Financial Services Information Sharing and Analysis Centers (FSIAC).

A few common hunting techniques

Okay, you have your hunting team, data, and tools sorted out. Now you can study your environment and start making hypotheses.

Let’s talk about the most common techniques in your arsenal.

Searching

Searching involves combing through data (logs, full packet data, flow records, alerts, system events, digital images, and memory dumps) for artifacts or patterns – things that could indicate malicious activity. This requires finely defined search criteria, not too broad and not too narrow. Otherwise, the results may be overwhelming. This technique asks for significant time investment and is far more difficult when there aren’t signatures available.

Clustering

This is where machine learning and AI come in handy. It involves isolating clusters of similar data points that share particular characteristics from a larger data set. Because it can accurately find aggregate behaviors, such as an uncommon number of instances of an occurrence, it shines best in outlier detection. Most effective with a large group of data points, since for a human it would be a daunting task to analyze big chunks of data.

Grouping

This technique involves searching for multiple unique artifacts that appear together using predetermined search criteria. Although similar to Clustering, Grouping supposes searching for a particular set of artifacts that have already been deemed as suspicious. Groups within these artifacts may represent an attacker’s tool or a TTP.

Stack counting (Stacking)

Stack counting involves creating statistics for values of a particular type and trying to find outliers among those results. It is most effective when dealing with filtered inputs such as endpoints of a particular function, and is less effective when dealing with large datasets. Organize, filter, and sort the data as much as possible before trying to find any anomalies.

Conclusion

For a newcomer, cyber threat hunting may seem too complex. But it is important to just start with what little resource you have. Bad actors won’t wait. Using the common threat hunting techniques described briefly above is a good start.

You are not alone. The 2023 Threat Hunting Report from Cybersecurity Insiders showed 65% of surveyed organizations plan to start threat hunting programs over the next three years and 83% agree that threat hunting should be a top security initiative.

This study reiterated the benefits organizations are already getting from threat hunting. These include improved detection of advanced threats – 66%, reduction of time spent on chasing false leads – 50%, and reduced investigation time – 59% of respondents. Over time, you or your security analysts will build up a system that will speed up the hunting process. Iterate the techniques, try new ones, track overall results, improve on them, and you’ll develop a comprehensive, focused approach for uncovering adversary TTPs within your environment.