Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Threat Intelligence

2/11/2020
10:00 AM
Joshua Goldfarb
Joshua Goldfarb
Commentary
Connect Directly
Twitter
RSS
E-Mail vvv
50%
50%

Keeping a Strong Security Metrics Framework Strong

Don't just report metrics -- analyze, understand, monitor, and adjust them. These 10 tips will show you how.

It takes a significant effort by security teams to build robust metrics that serve the organization well and add value. But keeping that framework strong over time is also an area that requires strategic investment. Unfortunately, it's an area that is often overlooked.

Here are 10 tips to help you maintain the value of your security metrics framework.

Tip 1: Check in with your audience. Metrics are developed to provide important information to the security organization's audiences, not for the sake of the metrics themselves. As such, it's critical to ensure that what you report addresses your audiences' information needs, questions, and concerns. Check in with stakeholders regularly to solicit, accept, and incorporate their feedback. Your audiences aren't a drag on your metrics, they're the reason for them.

Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them. If you see that one or more metrics are trending in an uncomfortable manner, dig deeper to understand why that's the case and what the ramifications are for the business. When you monitor metrics on a continuous basis, you will ensure that the risk those metrics measure does not rise to unacceptable levels. If risk levels do rise too high, you can course correct to effectively manage that risk.

Tip 3: Ensure data accuracy. A framework is only as good as the data underlying it. You may have the most relevant and timely metrics, but if the data used to calculate them is inaccurate, inconsistent, and/or flawed, the metrics will be as well. Reliable data serves as an input to reliable metrics while unreliable data, by default, produces unreliable metrics.

Tip 4: Experiment with different models and aggregations. Maybe the way you modeled your framework and aggregated your metrics worked well for you last year. But perhaps things have changed since then and that approach will no longer work. If you've built your metrics modularly, you'll be able to leverage them across a variety of different models and aggregations. Find the one that works for your present-day business environment.

Tip 5: Keep after controls. A mature metrics framework includes proper mapping back to controls. Keep after this mapping. Over time, controls may change in substance, importance, and/or priority. Further, mappings may evolve to be incorrect. Ensuring accurate mapping between controls and metrics allows the security team to continually assess and measure the efficacy of controls to the overall security posture of the business.

Tip 6: Keep after risk. Risk is not static or distinct. It is continuous, dynamic, and fluid. Keeping an eye on the changing risk landscape allows an organization to focus on mitigating the organization's most important and relevant risks, while reducing time and resources spent on less important and relevant issues. This allows finite security resources to be applied to the maximal risk mitigation.

Tip 7: Mind your ranges. When a metric is designed and measured, it creates a data point. Usually that data point is a number or a percentage, which, in and of itself, tells very little of the overall picture and offers no context. To add important context to the risk equation, you need to set an acceptable range and acceptable deviations from that range. Over time, those ranges may require adjustment to reflect changes in the evolution of the business environment and the threat landscape which will affect the tolerance level for the various data points that you measure. Minding your ranges will ensure that your tolerances are in line with acceptable risk levels.

Tip 8: Leverage intelligence. In addition to aiding and informing preventative and detective capabilities, intelligence can also inform metrics. Good intelligence can help you stay informed of existing threats and become aware of new threats. This in turn helps you to continually assess whether or not your metrics have addressed the right set of threats to your organization.

Tip 9: Stay connected. Peer organizations, industry groups, and experts can help an organization see where it lies relative to other organizations of similar size, industry, and geography. These connections can provide essential information that will keep your metrics framework strong.

Tip 10: Be efficient. No metrics framework is sustainable if the process of putting together and reporting the metrics is a headache in and of itself. In order for metrics to be practical and to provide value on a continual basis, they need to be scalable. Consolidate data required for metrics into as few systems as possible. Leverage automated reporting and dashboards to simplify the process of generating metrics when required, ideally automatically and in near real-time. This ensures that metrics will always be fresh. It also reduces your investment in to creating, designing, developing, and generating new metrics, which will, in turn, encourage innovation, creativity, and forward-thinking.

Related Content:

Check out The Edge, Dark Reading's new section for features, threat data, and in-depth perspectives. Today's top story: "C-Level & Studying for the CISSP."

Josh (Twitter: @ananalytical) is an experienced information security leader who works with enterprises to mature and improve their enterprise security programs.  Previously, Josh served as VP, CTO - Emerging Technologies at FireEye and as Chief Security Officer for ... View Full Bio
 

Recommended Reading:

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
tdsan
50%
50%
tdsan,
User Rank: Ninja
3/31/2020 | 9:42:09 AM
Great post and valid points
Tip 2: Stay alert and attuned. Don't just report metrics — analyze, understand, monitor, and adjust them

I do think this is the most important aspect of the post (I do like the areas where you indirectly referred to staying vigiliant and ready). This is a sliding scale that should be adjusted all the time and carefully looking at the metrics to help with this process is essential (basically using a SIEM or brain to capture information and format the data in a way where it makes sense).

But I have to be the bearer of bad news, companies are doing that (again, not all but a large majority of organizations, those who have been hacked - https://www.csis.org/programs/technology-policy-program/significant-cyber-incidents. There is something called Actionable intelligence where the metrics are used to determine if there is a problem and determines what can be done but we need to start looking beyond the numbers and metrics and look into "Prescriptive" security methods where the data is prioritized, listed and categorized where the SIEM solution or some other automated tool can run programs to modify parameters in a networked device. We need to be able to reach that level of potential before the threat becomes a reality.

Industry 4.0 Innovation Map Reveals Emerging Technologies & Startups

 

We need to be able to tie in all of these sources of data to create a "Sentinel" to identify, analyze, prioritize, react, and learn.

T
BDeeds
50%
50%
BDeeds,
User Rank: Author
2/12/2020 | 4:32:43 PM
Using Metrics to Assess Controls
Using metrics to help assess the effectiveness of controls is a great tip for auditors. As auditors, we can sometimes face difficult conversations when control owners are having trouble processing shortcomings of a control(s). By matching the control to a metric, we maybe able to better help them understand our view. 
COVID-19: Latest Security News & Commentary
Dark Reading Staff 6/3/2020
Stay-at-Home Orders Coincide With Massive DNS Surge
Robert Lemos, Contributing Writer,  5/27/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Current Issue
How Cybersecurity Incident Response Programs Work (and Why Some Don't)
This Tech Digest takes a look at the vital role cybersecurity incident response (IR) plays in managing cyber-risk within organizations. Download the Tech Digest today to find out how well-planned IR programs can detect intrusions, contain breaches, and help an organization restore normal operations.
Flash Poll
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-2196
PUBLISHED: 2020-06-03
Jenkins Selenium Plugin 3.141.59 and earlier has no CSRF protection for its HTTP endpoints, allowing attackers to perform all administrative actions provided by the plugin.
CVE-2020-2197
PUBLISHED: 2020-06-03
Jenkins Project Inheritance Plugin 19.08.02 and earlier does not require users to have Job/ExtendedRead permission to access Inheritance Project job configurations in XML format.
CVE-2020-2198
PUBLISHED: 2020-06-03
Jenkins Project Inheritance Plugin 19.08.02 and earlier does not redact encrypted secrets in the 'getConfigAsXML' API URL when transmitting job config.xml data to users without Job/Configure.
CVE-2020-2199
PUBLISHED: 2020-06-03
Jenkins Subversion Partial Release Manager Plugin 1.0.1 and earlier does not escape the error message for the repository URL field form validation, resulting in a reflected cross-site scripting vulnerability.
CVE-2020-2200
PUBLISHED: 2020-06-03
Jenkins Play Framework Plugin 1.0.2 and earlier lets users specify the path to the `play` command on the Jenkins master for a form validation endpoint, resulting in an OS command injection vulnerability exploitable by users able to store such a file on the Jenkins master.