Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

10/6/2015
10:30 AM
Jay Jacobs
Jay Jacobs
Commentary
Connect Directly
Twitter
LinkedIn
RSS
E-Mail vvv
50%
50%

Dont Be Fooled: In Cybersecurity Big Data Is Not The Goal

In other words, the skills to be a security expert do not translate to being able to understand and extract meaning from security data.

With any hyped-up topic, there is a disconnect between what people think can be accomplished and what can actually be accomplished.  The latest example is “Big Data.” There’s something different about big data because when executed properly, modern data analysis can seem like nothing short of magic to outsiders.  It’s this perception of magic that is so attractive to businesses looking to pull the next rabbit out of a hat. 

What’s also unique about “big data” is the way the term is perceived and used in the security industry, which is much different than the approach of  data scientists. This blog is an attempt to bridge the divide and talk about “big data” behind the scenes because when it comes down to it, big data is not a goal, it is logistics, and many of security’s big data problems are being solved by small data solutions.

Big data is just hiding the small data

In many instances, “Big Data Analytics” is an embellishment, a little white lie with good intentions.  In reality, the primary task done directly with big data is figuring out how to turn it into small data. In many cases, big data will be reduced down by counting, comparing or some other aggregation.  Another technique is to pull a small subset or sample from the large data set to get a more manageable data set. Either way, many of the big data jobs are done to produce small data. 

For example, if you are working with log data from thousands of systems, you may want to produce a small data file where each system is reduced to a single line with counts of sources and login statuses (or whatever is being measured).  Don’t be fooled by the use of “small” here, the output may still be in the megabyte or even gigabyte range, which may be just small enough to load into memory on a laptop for analysis. 

If that deflated the mystique of big data for you, don’t worry, there is a small set of big data implementations that will perform analytics at scale, for example, if you need to build a unique and specific model on each application across all of your servers (This is a technique we use to develop security ratings.) While each application may represent small data, doing analysis across all applications represents a challenge.  And finally there is a very small sliver of analytics doing complex computations at scale, but these are rare and chances are your problems are just not that special. 

Big data enables [over]confidence

While much of the big data analysis is being done on small data, that doesn’t mean it’s the same old small data analysis.  Big data is ushering in its own sets of challenges because most of the classic statistics were developed using pencil and paper with just a few dozen samples, perhaps going into the hundreds.  The good news is that most of the techniques are actually improved with more data. Analyses can find ever more subtleties as more data is used. Smaller differences can be discovered and more and more nuanced patterns can be detected.  This is where some of that magic comes from: there can be big gain from small advantages.  However this is not without side effects.  One of the classic measures of statistical significance, the p-value, is often irrelevant on large samples.  Because analysis can uncover smaller and smaller differences in big data, in reverse, smaller differences now become significant.  The effect is that big data can fool those unaware of this effect. 

Big data, small samples

Even though big data still has that new-hype smell about it, underlying that hype are techniques that can be traced back centuries -- and all point to a single constant: good data analysis.  It’s a hard lesson to learn, but good data analysis is a different skill than domain expertise.  In other words, the skills to be a security expert (for example) do not translate to being able to understand and extract meaning from security data.  Neither is good data analysis an intuitive skill; it is not picked up by proximity to data day after day.  It is only learned by intentional study of statistics and related fields.

As an example, in the age of big data where a million data points is labelled as small, a sample of a few hundred and even a few thousand seems meager and perhaps pathetic to the uninitiated.  I’ve seen people dismiss research results because they thought the samples were too small (and they were over a thousand).  We must keep in mind that data analysis is not intuitive and even though we have a lot of data, it may not take a lot of data to provide insight or support a business decision.  So shake off this notion that big data is a goal and get down to the business of learning from data.

Jay Jacobs has over 15 years of experience within IT and information security with a focus on cryptography, risk, and data analysis. Most recently, he has joined BitSight Technologies, the Standard in Security Ratings, as their Senior Data Scientist. Previously, he was a Data ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Aviation Faces Increasing Cybersecurity Scrutiny
Kelly Jackson Higgins, Executive Editor at Dark Reading,  8/22/2019
Microsoft Tops Phishers' Favorite Brands as Facebook Spikes
Kelly Sheridan, Staff Editor, Dark Reading,  8/22/2019
Capital One Breach: What Security Teams Can Do Now
Dr. Richard Gold, Head of Security Engineering at Digital Shadows,  8/23/2019
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
7 Threats & Disruptive Forces Changing the Face of Cybersecurity
This Dark Reading Tech Digest gives an in-depth look at the biggest emerging threats and disruptive forces that are changing the face of cybersecurity today.
Flash Poll
The State of IT Operations and Cybersecurity Operations
The State of IT Operations and Cybersecurity Operations
Your enterprise's cyber risk may depend upon the relationship between the IT team and the security team. Heres some insight on what's working and what isn't in the data center.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2016-6154
PUBLISHED: 2019-08-23
The authentication applet in Watchguard Fireware 11.11 Operating System has reflected XSS (this can also cause an open redirect).
CVE-2019-5594
PUBLISHED: 2019-08-23
An Improper Neutralization of Input During Web Page Generation ("Cross-site Scripting") in Fortinet FortiNAC 8.3.0 to 8.3.6 and 8.5.0 admin webUI may allow an unauthenticated attacker to perform a reflected XSS attack via the search field in the webUI.
CVE-2019-6695
PUBLISHED: 2019-08-23
Lack of root file system integrity checking in Fortinet FortiManager VM application images of all versions below 6.2.1 may allow an attacker to implant third-party programs by recreating the image through specific methods.
CVE-2019-12400
PUBLISHED: 2019-08-23
In version 2.0.3 Apache Santuario XML Security for Java, a caching mechanism was introduced to speed up creating new XML documents using a static pool of DocumentBuilders. However, if some untrusted code can register a malicious implementation with the thread context class loader first, then this im...
CVE-2019-15092
PUBLISHED: 2019-08-23
The webtoffee "WordPress Users & WooCommerce Customers Import Export" plugin 1.3.0 for WordPress allows CSV injection in the user_url, display_name, first_name, and last_name columns in an exported CSV file created by the WF_CustomerImpExpCsv_Exporter class.