Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Analytics

11/7/2018
04:10 PM
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%

Finding Gold in the Threat Intelligence Rush

Researchers sift through millions of threat intel observations to determine where to best find valuable threat data.

Threat intelligence feeds, sold for hundreds of thousands of dollars per year, are marketed on a specific premise: If an entity is seen acting maliciously in one place, it can be expected in others.

But that's not necessarily true, according to two researchers from SensePost SecureData. Founder and chief strategy officer Charl van der Walt and security analyst Sid Pillarisetty have spent six months analyzing the ability of threat intelligence to predict malicious activity. Their conclusion: There are both good and bad places and means to unearth reliable threat data on the Internet.

Van der Walt and Pillarisetty are part of a managed services team that conducts threat detection on behalf of UK customers. One of the issues they (and many security pros) deal with is detecting potentially harmful activity by IP addresses on customers' perimeters, van der Walt says. This includes people doing vulnerability scans, port scans, activity related to suspicious IP addresses, and anything that isn't obviously malicious but could warrant an investigation.

"The big question: How much effort does that sort of information warrant on behalf of enterprises?" he explains. "What should you be doing about it?"

Back in June, the duo began preliminary research on a relatively small dataset of threat indicators. They have since expanded their investigation to include more than 1 million online threat indicators and 1.3 billion correlations, or where suspicious events overlap.

At Black Hat Europe, in London this December, van der Walt and Pillarisetty will take the stage to share their findings in "Don't Eat Spaghetti with a Spoon: An Analysis of the Practical Value of Threat Intelligence." They hope to "move the needle along" in terms of understanding threat intelligence and equip other researchers with the data structures, tooling, methodology, and language to enable future research in the space, van der Walt says.

Different Companies Face Different Threats
In detecting malicious activity, the researchers have amassed indicators of compromise and IP addresses for several different customers. "What you end up having is threat intelligence, which we collect from one customer and is potentially applicable to another customer," van der Walt says.

This notion drives the business model of commercial threat feeds, which are sold to enterprises on the basis that they can drive intelligence-led security. Companies are told they can use feeds to pre-emptively block IP addresses that have appeared malicious for other customers.

These feeds are expensive in two ways, van der Walt explains. Businesses pay a lot of money to get them, for starters. When they do, the data demands attention and effort for security teams to respond. But in collecting and analyzing threats across companies, the researchers found that IP addresses that appear suspicious at one organization may not prove malicious at another.

For example, IP addresses that interact with honeypots prove malicious across businesses, they found. The duo set up a network of honeypots to correlate their observations of IP addresses and see how activity varied with the honeypot and with other networks. They learned the threat intelligence they collected via honeypots had a significantly higher fidelity than the threat data they directly gathered from customers' perimeters, van der Walt says.

Businesses would see a higher ROI by ingesting IP addresses from a honeypot and blocking those than by ingesting suspicious IP addresses from other feeds, Pillarisetty explains.

"What our initial research suggests – and we're trying to prove with a bigger dataset – is the proportion of suspicious IP addresses we observe at more than one customer is actually extremely low," van der Walt says. This implies companies relying on threat intelligence feeds spend a lot of time chasing shadows. "There's actually very little value in there," he adds.

At Black Hat Europe, the researchers also want to discuss whether certain processes need to be followed before the data they collect is actionable, Pillarisetty continues. They plan to investigate whether the IP addresses they get need to be processed further based on other factors in an environment.

"Only then can we say this is more malicious than other activity on your network," he says. It fits into the broader conversation of proposing better ways to gather threat intelligence.

Van der Walt says their research questions the underlying notion driving the threat intelligence business model. As consumers of threat feeds, he says, it changes how they view their value. Looking ahead, he anticipates they'll be able to verify some of the popular notions around the longevity of threat intelligence and the amount of time businesses have to respond to it.

In their initial study, van der Walt cites as an example, they observed multiple occurrences of the same IP address appearing in a two-day window. After that, the probability of seeing the same addresses "dropped off dramatically." In addition to analyzing the time frame of malicious IPs, he hopes they'll be able to determine other patterns. i.e., whether an IP seen at two companies will likely be seen at a third, or whether certain behavior indicates a reappearance of an IP address elsewhere.

Related Content:

 

 

 

Black Hat Europe returns to London Dec 3-6 2018  with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions and service providers in the Business Hall. Click for information on the conference and to register.

Kelly Sheridan is the Staff Editor at Dark Reading, where she focuses on cybersecurity news and analysis. She is a business technology journalist who previously reported for InformationWeek, where she covered Microsoft, and Insurance & Technology, where she covered financial ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Where Businesses Waste Endpoint Security Budgets
Kelly Sheridan, Staff Editor, Dark Reading,  7/15/2019
How Attackers Infiltrate the Supply Chain & What to Do About It
Shay Nahari, Head of Red-Team Services at CyberArk,  7/16/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment: This comment is waiting for review by our moderators.
Current Issue
Building and Managing an IT Security Operations Program
As cyber threats grow, many organizations are building security operations centers (SOCs) to improve their defenses. In this Tech Digest you will learn tips on how to get the most out of a SOC in your organization - and what to do if you can't afford to build one.
Flash Poll
The State of IT Operations and Cybersecurity Operations
The State of IT Operations and Cybersecurity Operations
Your enterprise's cyber risk may depend upon the relationship between the IT team and the security team. Heres some insight on what's working and what isn't in the data center.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-13971
PUBLISHED: 2019-07-19
OTCMS 3.81 allows XSS via the mode parameter in an apiRun.php?mudi=autoRun request.
CVE-2019-13972
PUBLISHED: 2019-07-19
LayerBB 1.1.3 allows XSS via the application/commands/new.php pm_title variable, a related issue to CVE-2019-17997.
CVE-2019-13973
PUBLISHED: 2019-07-19
LayerBB 1.1.3 allows admin/general.php arbitrary file upload because the custom_logo filename suffix is not restricted, and .php may be used.
CVE-2019-13974
PUBLISHED: 2019-07-19
LayerBB 1.1.3 allows conversations.php/cmd/new CSRF.
CVE-2019-13977
PUBLISHED: 2019-07-19
index.php in Ovidentia 8.4.3 has XSS via tg=groups, tg=maildoms&idx=create&userid=0&bgrp=y, tg=delegat, tg=site&idx=create, tg=site&item=4, tg=admdir&idx=mdb&id=1, tg=notes&idx=Create, tg=admfaqs&idx=Add, or tg=admoc&idx=addoc&item=.