Abstract
Metrics are tools to facilitate decision making and improve performance and accountability. Measures are quantifiable, observable, and objective data supporting metrics. Operators can use metrics to apply corrective actions and improve performance.
Regulatory, financial, and organizational factors drive the requirement to measure IT security performance. Potential security metrics cover a broad range of measurable features, from security audit logs of individual systems to the number of systems within an organization that were tested over the course of a year. Effective security metrics should be used to identify weaknesses, determine trends to better utilize security resources, and judge the success or failure of implemented security solutions.
Cyber security metrics and measures can help organizations (i) verify that their security controls are in compliance with a policy, process, or procedure; (ii) identify their security strengths and weaknesses; and (iii) identify security trends, both within and outside the organization’s control. Studying trends allows an organization to monitor its security performance over time and to identify changes that necessitate adjustments in the organization’s security posture. At a higher level, these benefits can be combined to help an organization achieve its mission by (i) evaluating its compliance with legislation and regulations, (ii) improving the performance of its implemented security controls, and (iii) answering high-level business questions regarding security, which facilitate strategic decision making by the organization’s highest levels of management.
CONTRASTING METRICS AND MEASURES
The term metric is often used to refer to the measurement of performance, but it is clearer to define metrics and measures separately.
A measure is a concrete, objective attribute, such as the percentage of systems within an organization that are fully patched, the length of time between the release of a patch and its installation on a system, or the level of access to a system that a vulnerability in the system could provide. A metric is an abstract, somewhat subjective attribute, such as how well an organization’s systems are secured against external threats or how effective the organization’s incident response team is. An analyst can approximate the value of a metric by collecting and analyzing groups of measures, as is explained later.
