August 2023 (1)

False positive IOCs

An integral part of our work as cyber security professionals has to do with threat intelligence from IOC (Indicator of Compromise) feeds. IOC feeds are streams of actionable data providing information about potential cybersecurity threats. These feeds contain indicators that can be used to identify the presence of malicious activities or security breaches within an organization's network or systems. IOCs include malware hashes, C2 domains, phishing URLs and more.

Hunters ingests IOC feeds from different threat intelligence platforms (AlienVault OTX, URLHaus, MalwareBazaar, and FeodoTracker) providing intel on IPs, domains, URLs, and hashes. Hunters also supports the ingestion of customers’ IOC resources (like Anomaly and STIX). This data is then correlated with the customer organization data onboarded to the Hunters platform, to detect the presence of IOCs. Often we find that IOC feeds provide many false positive results that spam the platform and waste precious time and resources on triage and mitigation.

Why is this happening?

IOC feeds have a hard time identifying which indicators are strictly malicious and which could be malicious if used by malicious actors. A good example of this is cmd.exe which is most commonly used innocently but can also be used by threat actors. This tagging is then imported into SIEM and SOC platforms which automatically mark the indicator as a threat when it could be completely benign.

The passage of time and changes in the nature of indicators can also affect these findings. For instance, IPs that were once rightfully tagged as malicious and appear in an IOC feed can later be deemed innocent, but their status will still remain suspicious as they are part of a list of malicious indicators.

The result is a long list of leads and alerts based on false-positive IOC findings that clutters the SOC teams’ queue and makes it harder for organizations to find the real threats inside their data.

How did we solve this?

In an effort to reduce false positives on the Hunters platform, we’ve researched a large sample group of leads. As part of this research, we were able to identify patterns of creation of false positive IOCs. Each type of IOC was investigated separately (IPs, Hashes, URLs, and domains), to maintain a high level of accuracy of the research findings.

For each IOC type, we established how false positives are created and explored ways to diminish the error rate. Once we concluded the research, we implemented a new policy for handling and reducing the amount of false positive IOC findings in the system.

Using these new policies, we were able to enhance our detection life cycle, by "shifting left" blacklisting benign IOCs from the investigation (post-detection) phase to the detection phase.
As a result of this research and subsequent improvements to the system, Hunters now eliminates false positives before they become leads in the system, thus decluttering the platform and saving many hours of redundant triaging efforts.

Let’s dive into the different IOC types we’ve handled.

Hashes

We came up with a few steps that proved to greatly lower the amount of false positive hashes. The first step is to incorporate NSRL data. The NSRL (National Software Reference Library) is a project managed by the National Institute of Standards and Technology (NIST) in the United States. It plays a crucial role in cybersecurity by providing a reference for "known good" software and files. Incorporating NSRL safe hashes data allowed us to compare these known hashes against those in our system to identify "known good" files, which reduced the number of false positives in the system.

The second step is to refer back to the original IOC feed in which the hash was initially flagged as malicious and examine whether the hash was deleted from the list. This eliminates any hashes that were mistakenly flagged as malicious or that were flagged but has since proved to be benign.

The third step we take to evaluate a hash is to use VirusTotal data, alongside a proprietary logic developed at Hunters. First, we score each hash on three main verticals:

• Validity - is the hash a valid file?
• Relevancy - how relevant is the report about this hash to the current time?
• Reliability - how reliable is the report about this hash? Is it stable through time?

Once we score each hash based on these verticals, we will put our complex IOC assessment logic into play. As part of this logic, we consider the time difference between the first submission and last analysis of the hash on VirusTotal, we employ a different approach towards hashes with different VirusTotal scores, and we consider whether hashes were signed by Microsoft or tagged as Known-Distributer. In addition, we assess the hash based on the strength of the engine reporting the hash. Some engines are more trustworthy than others and hashes reported by them are more likely to be malicious. For instance, if a hash is reported by Kaspersky, CrowdStrike Falcon, Microsoft, Palo Alto, and more we will be more inclined to include it in the IOC list, while hashes reported by ML-based, statistical, and generally less accurate engines will most likely end up blacklisted from the IOC list.

By essentially automating the manual steps performed by analysts when assessing the validity of an IOC, we were able to reduce false positive hashes by ~86% only after establishing that the IOCs are most likely benign.

IPs and domains

Much like the Hashes policy, the IPs and domains policy is comprised of integrating external data from reliable vendors to enrich the IOCs, which allows Hunters to take an informative decision on whether they are false positives or not, as well as a Hunters research-based logic used to weed out any irrelevant IOCs.

For IPs and domains, Hunters incorporates data from PulseDive, a threat intelligence platform that leverages open-source threat intelligence (OSINT) feeds and user submissions from all over the world to deliver high-fidelity, actionable intelligence. This allows us to blacklist IOCs based on several factors, such as IP activity, direct-to-IP URLs, domain registration time, and more. In addition, Hunters integrates data from Cisco Umbrella (Umbrella 1M) which provides a list of most queried domains based on passive DNS usage across Cisco Umbrella global network. Therefore domains found in the list are likely to be benign. These domains are then blacklisted from Hunters’ IOC list and do not generate findings on the platform.

Another blocker of unreliable IOCs is a TTL consideration; to reduce noise even further, Hunters doesn’t generate leads for IOCs whose last sample time is older than a specific date, depending on the IOC type:

• IPs - 30 days
• Domains - 120 days
• URLs - 60 days

Conclusion

An initial examination of the results shows a ~98% elimination of false positive IOC leads, across all IOC types, per week. These results demonstrate the importance of our concentrated efforts to tackle the challenge of false positive IOCs, significantly enhancing the efficiency and effectiveness of our offering.

By systematically addressing each IOC type – hashes, IPs, and domains – through a combination of data enrichment, advanced logic, and automation, we've achieved a substantial reduction in false positive leads. This transformative shift empowers SOC teams to focus their expertise on genuine threats, streamlining workflows, conserving resources, and ultimately fortifying organizations against evolving cyber risks.