Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

2/11/2020
10:00 AM
Joshua Goldfarb
Joshua Goldfarb
Commentary
Connect Directly
Twitter
RSS
E-Mail vvv
50%
50%

Keeping a Strong Security Metrics Framework Strong

Don't just report metrics -- analyze, understand, monitor, and adjust them. These 10 tips will show you how.

It takes a significant effort by security teams to build robust metrics that serve the organization well and add value. But keeping that framework strong over time is also an area that requires strategic investment. Unfortunately, it's an area that is often overlooked.

Here are 10 tips to help you maintain the value of your security metrics framework.

Tip 1: Check in with your audience. Metrics are developed to provide important information to the security organization's audiences, not for the sake of the metrics themselves. As such, it's critical to ensure that what you report addresses your audiences' information needs, questions, and concerns. Check in with stakeholders regularly to solicit, accept, and incorporate their feedback. Your audiences aren't a drag on your metrics, they're the reason for them.

Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them. If you see that one or more metrics are trending in an uncomfortable manner, dig deeper to understand why that's the case and what the ramifications are for the business. When you monitor metrics on a continuous basis, you will ensure that the risk those metrics measure does not rise to unacceptable levels. If risk levels do rise too high, you can course correct to effectively manage that risk.

Tip 3: Ensure data accuracy. A framework is only as good as the data underlying it. You may have the most relevant and timely metrics, but if the data used to calculate them is inaccurate, inconsistent, and/or flawed, the metrics will be as well. Reliable data serves as an input to reliable metrics while unreliable data, by default, produces unreliable metrics.

Tip 4: Experiment with different models and aggregations. Maybe the way you modeled your framework and aggregated your metrics worked well for you last year. But perhaps things have changed since then and that approach will no longer work. If you've built your metrics modularly, you'll be able to leverage them across a variety of different models and aggregations. Find the one that works for your present-day business environment.

Tip 5: Keep after controls. A mature metrics framework includes proper mapping back to controls. Keep after this mapping. Over time, controls may change in substance, importance, and/or priority. Further, mappings may evolve to be incorrect. Ensuring accurate mapping between controls and metrics allows the security team to continually assess and measure the efficacy of controls to the overall security posture of the business.

Tip 6: Keep after risk. Risk is not static or distinct. It is continuous, dynamic, and fluid. Keeping an eye on the changing risk landscape allows an organization to focus on mitigating the organization's most important and relevant risks, while reducing time and resources spent on less important and relevant issues. This allows finite security resources to be applied to the maximal risk mitigation.

Tip 7: Mind your ranges. When a metric is designed and measured, it creates a data point. Usually that data point is a number or a percentage, which, in and of itself, tells very little of the overall picture and offers no context. To add important context to the risk equation, you need to set an acceptable range and acceptable deviations from that range. Over time, those ranges may require adjustment to reflect changes in the evolution of the business environment and the threat landscape which will affect the tolerance level for the various data points that you measure. Minding your ranges will ensure that your tolerances are in line with acceptable risk levels.

Tip 8: Leverage intelligence. In addition to aiding and informing preventative and detective capabilities, intelligence can also inform metrics. Good intelligence can help you stay informed of existing threats and become aware of new threats. This in turn helps you to continually assess whether or not your metrics have addressed the right set of threats to your organization.

Tip 9: Stay connected. Peer organizations, industry groups, and experts can help an organization see where it lies relative to other organizations of similar size, industry, and geography. These connections can provide essential information that will keep your metrics framework strong.

Tip 10: Be efficient. No metrics framework is sustainable if the process of putting together and reporting the metrics is a headache in and of itself. In order for metrics to be practical and to provide value on a continual basis, they need to be scalable. Consolidate data required for metrics into as few systems as possible. Leverage automated reporting and dashboards to simplify the process of generating metrics when required, ideally automatically and in near real-time. This ensures that metrics will always be fresh. It also reduces your investment in to creating, designing, developing, and generating new metrics, which will, in turn, encourage innovation, creativity, and forward-thinking.

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's top story: "C-Level & Studying for the CISSP."

Josh (Twitter: @ananalytical) is an experienced information security leader who works with enterprises to mature and improve their enterprise security programs.  Previously, Josh served as VP, CTO - Emerging Technologies at FireEye and as Chief Security Officer for ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
tdsan
50%
50%
tdsan,
User Rank: Ninja
3/31/2020 | 9:42:09 AM
Great post and valid points
Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them

I do think this is the most important aspect of the post (I do like the areas where you indirectly referred to staying vigiliant and ready). This is a sliding scale that should be adjusted all the time and carefully looking at the metrics to help with this process is essential (basically using a SIEM or brain to capture information and format the data in a way where it makes sense).

But I have to be the bearer of bad news, companies are doing that (again, not all but a large majority of organizations, those who have been hacked - https://www.csis.org/programs/technology-policy-program/significant-cyber-incidents. There is something called Actionable intelligence where the metrics are used to determine if there is a problem and determines what can be done but we need to start looking beyond the numbers and metrics and look into "Prescriptive" security methods where the data is prioritized, listed and categorized where the SIEM solution or some other automated tool can run programs to modify parameters in a networked device. We need to be able to reach that level of potential before the threat becomes a reality.

Industry 4.0 Innovation Map Reveals Emerging Technologies & Startups

 

We need to be able to tie in all of these sources of data to create a "Sentinel" to identify, analyze, prioritize, react, and learn.

T
BDeeds
50%
50%
BDeeds,
User Rank: Author
2/12/2020 | 4:32:43 PM
Using Metrics to Assess Controls
Using metrics to help assess the effectiveness of controls is a great tip for auditors. As auditors, we can sometimes face difficult conversations when control owners are having trouble processing shortcomings of a control(s). By matching the control to a metric, we maybe able to better help them understand our view. 
COVID-19: Latest Security News & Commentary
Dark Reading Staff 8/3/2020
Pen Testers Who Got Arrested Doing Their Jobs Tell All
Kelly Jackson Higgins, Executive Editor at Dark Reading,  8/5/2020
Researcher Finds New Office Macro Attacks for MacOS
Curtis Franklin Jr., Senior Editor at Dark Reading,  8/7/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
Special Report: Computing's New Normal, a Dark Reading Perspective
This special report examines how IT security organizations have adapted to the "new normal" of computing and what the long-term effects will be. Read it and get a unique set of perspectives on issues ranging from new threats & vulnerabilities as a result of remote working to how enterprise security strategy will be affected long term.
Flash Poll
The Changing Face of Threat Intelligence
The Changing Face of Threat Intelligence
This special report takes a look at how enterprises are using threat intelligence, as well as emerging best practices for integrating threat intel into security operations and incident response. Download it today!
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-12777
PUBLISHED: 2020-08-10
A function in Combodo iTop contains a vulnerability of Broken Access Control, which allows unauthorized attacker to inject command and disclose system information.
CVE-2020-12778
PUBLISHED: 2020-08-10
Combodo iTop does not validate inputted parameters, attackers can inject malicious commands and launch XSS attack.
CVE-2020-12779
PUBLISHED: 2020-08-10
Combodo iTop contains a stored Cross-site Scripting vulnerability, which can be attacked by uploading file with malicious script.
CVE-2020-12780
PUBLISHED: 2020-08-10
A security misconfiguration exists in Combodo iTop, which can expose sensitive information.
CVE-2020-12781
PUBLISHED: 2020-08-10
Combodo iTop contains a cross-site request forgery (CSRF) vulnerability, attackers can execute specific commands via malicious site request forgery.