The Most Important Security Metrics to Maintain Compliance

Every week, dozens of data breaches are reported with some reaching into the tens, or even hundreds of millions of individuals impacted.

Customers and regulators alike are increasingly concerned about the information security programs of organizations and how they plan to prevent security incidents and safeguard sensitive data. 

As a result, many laws and regulations have been established to promote cybersecurity risk management and to protect confidential information that may be stored or transferred across organizations.

These industry-specific and general data protection laws tend to be extensive and require constant monitoring to ensure regulatory compliance across your organization and your vendors' organizations. 

That's why it's important to establish a set of security metrics that measure the effectiveness of, participation in the use of security controls. A well-chosen set of metrics will help guide future security decision-making and improve the security posture of your organization. 

Without a quantitative approach to threat intelligence, organizations become more susceptible to attacks which can impact revenue and reputation. Read our post on the average cost of a data breach to learn more. 

What are Security Metrics?

Security metrics or cybersecurity metrics are a measurable value that demonstrates how well a company is achieving its cybersecurity risk reduction goals. Organizations use security metrics at multiple levels to evaluate how well they are meeting their security standards and information security management requirements.

High-level security metrics may focus on the overall performance of the organization and are typically owned by the Chief Information Security Officer (CISO) or CTO and shared with senior management, while low-level security metrics may focus on penetration testing, vulnerability scan, security training, and risk assessment results. Low-level metrics are typically owned by security teams made up of security professionals who report into the CISO.

While the main goal of security metrics is to assess how well your organization is reducing security risk, there are also different metrics that can provide insight into the performance of the program itself. These metrics are often provided by security tools designed to provide real-time, actionable feedback.

Regardless of what metrics you pick, remember that they should be quantifiable and influence day-to-day behavior as well as the long-term strategy. They should also comply with your information security, data security, mobile security, and network security frameworks.

And ideally, you should have a set of metrics that are easily understandable for non-technical stakeholders, such as shareholders, Board members, and regulators. Hard numbers and industry benchmarks are a great way to avoid confusion while highlighting key areas for improvement. 

Why are Security Metrics Important?

As Peter Drucker said, what gets measured, gets managed. If you can't measure the results of your security efforts, you won't know how you're tracking. 

Cybersecurity isn't a one-time affair. Cyber criminals and the cyber attacks they employ are constantly evolving, and the processes and technology needed to combat them must continuously evolve to stay relevant. 

You need to have metrics in place to assess the effectiveness of the security controls you have invested in. 

Security metrics are important for two main reasons:

1. The analysis of key performance indicators (KPIs), key risk indicators (KRIs), and security postures provides a snapshot of how your security team is functioning over time. This helps you better understand what is working, which information technology assets are most at risk, and which areas are worsening. This information aids in decision-making around budget, people and technology investment.
2. Good metrics provide quantitative information that you can use to show management, board members, customers, and even shareholders that you take confidentiality, integrity, and availability seriously.  

How to Choose the Right Security Metrics

These seven principles will help you choose the right security metrics for your organization:

1. Purpose: Metrics should support business goals and regulatory requirements, connecting metrics to the business can help with stakeholder buy-in as well as ensuring resources are efficiently used
2. Controllability: For metrics to be worthwhile, they must demonstrate a goal is being met. So metrics must measure controllable processes and outcomes. 
3. Context: Don't just take the results of pen-testing or a security tool and call it a metric, it have meaning. Answer questions like "why are we collecting it?", "what story does it tell?", and "how do does this compare to our industry?"
4. Best practices: Know what you're trying to achieve and what a best-in-class metric looks like.
5. Quantitative: Metrics need to be quantitative so they can be compared across time and organizations. With that said, the qualitative context around objective key results (OKRs) is very important. 
6. Data quality: Metrics are only as good as the data that is used to create them, ensure it has a high level of accuracy, precision, and reliability. 
7. Ease of collection and analysis: The best metric is useless if you can't easily collect and analyze the data needed. It shouldn't take you a long time to prepare and report your metrics. Ideally, you should have an always up-to-date dashboard that anyone in your organization can look at.

Following these principles will help you identify data and services needed to build the metrics you need. It's important to remember your business environment will influence what data is collected to form your security metrics. These metrics will also depend on the technology you deploy, security controls you use, and the number of third-party vendors you outsource to. 

What Metrics are Useful for Measuring our Security Posture?

Your security posture (or cybersecurity posture) is the collective security status of the software, hardware, services, networks, information, vendors and service providers your organization uses. 

The most important metric to track your security posture is a security rating or cybersecurity rating. 

A security rating is a data-driven, objective, and dynamic measurement of your organization's security posture. It is created by a trusted, independent security ratings platform, much like a credit rating is created by an outside party. 

Just like credit ratings, security ratings aim to provide a quantitative metric of cybersecurity risk.  

The higher the organization's security rating, the better its overall security posture. 

Security ratings help security and risk leaders: 

• Understand the impact of their investments in cybersecurity controls or technology
• Align investments and actions to those that will mitigate the most critical risks
• Efficiently and dynamically allocate your limit resources on critical areas
• Facilitate data-driven, risk-based conversations about cybersecurity with key nontechnical stakeholders such as Board members, Vice Presidents, regulators, investors, and key business partners.  
• Benchmark internal security performance against industry peers

According to Gartner, cybersecurity ratings will become as important as credit ratings when assessing the risk of existing and new business relationships…these services will become a precondition for business relationships and part of the standard of due care for providers and procurers of services. Additionally, the services will have expanded their scope to assess other areas, such as cyber insurance, due diligence for M&A and even as a raw metric for internal security programs.

Read more about security ratings here. 

Beyond your security rating, you should consider tracking:

• Dwell time: The mean time an attacker has undetected access to sensitive data without being removed, the higher the dwell time, the longer the attacker has to remove sensitive data and the more damaging the attack.
• Vulnerabilities: The number of known vulnerabilities, like those listed on CVE, on internal systems and vendor-controlled systems. Vulnerability management can help you rank vulnerabilities based on severity, improving prioritization. 
• Cybersecurity awareness training: How many employees have completed your cybersecurity awareness program?

Read our post on the 14 cybersecurity KPIs to track. 

What Metrics are Useful to Track Regulatory Compliance?

Security metrics can help ensure you are meeting any applicable regulatory requirements, such as PCI DSS, HIPAA, GDPR, CCPA, CPS 234, LGPD, PIPEDA, FIPA, The SHIELD Act, GLBA, and FISMA.

The metrics you choose to track need to effectively measure your organization's ability to maintain regulatory and general data protection requirements. This is not only useful for improving your cybersecurity program, but can also help you avoid fines, lawsuits, and other penalties.

PCI DSS

The Payment Card Industry Data Security Standard (PCI DSS) is an information security standard for organizations that handle branded credit cards from major credit card schemes. 

PCI DSS aims to increase controls around cardholder data to reduce credit card fraud. PCI compliance requires that you meet twelve requirements:

1. Installing and maintaining a firewall configuration to protect cardholder data. The purpose of a firewall is to scan all network traffic, block untrusted networks from accessing the system.
2. Changing vendor-supplied defaults for system passwords and other security parameters. These passwords are easily discovered through public information and can be used by malicious individuals to gain unauthorized access to systems. Read our password security checklist for more information.
3. Protecting stored cardholder data. Encryption, hashing, masking, and truncation are methods used to protect cardholder data.
4. Encrypting transmission of cardholder data over open, public networks. Strong encryption, including using only trusted keys and certifications reduces the risk of being targeted by malicious individuals through cyber attacks.
5. Protecting all systems against malware and performing regular updates of anti-virus software. Different types of malware can enter a network in numerous ways, including Internet use, phishing emails, mobile devices or storage devices. Up-to-date anti-virus software or supplemental anti-malware software will reduce the risk of exploitation via malware.
6. Developing and maintaining secure systems and applications. Vulnerabilities in systems and applications allow unscrupulous individuals to gain privileged access. Security patches should be immediately installed to fix vulnerabilities and prevent exploitation and compromise of cardholder data.
7. Restricting access to cardholder data to only authorized personnel. Systems and processes must be used to restrict access to cardholder data on a “need to know” basis. Read more about the principle of least privilege here.
8. Identifying and authenticating access to system components. Each person with access to system components should be assigned a unique identification (ID) that allows accountability of access to critical data systems.
9. Restricting physical access to cardholder data. Physical access to cardholder data or systems that hold this data must be secure to prevent unauthorized access or removal of data. Read more about data breaches here.
10. Tracking and monitoring all access to cardholder data and network resources. Logging mechanisms should be in place to track user activities that are critical to prevent, detect or minimize the impact of data compromises.
11. Testing security systems and processes regularly. New vulnerabilities are continuously discovered. Systems, processes, and software need to be tested frequently to uncover vulnerabilities that could be used by malicious individuals.
12. Maintaining an information security policy for all personnel. A strong security policy includes making personnel understand the sensitivity of data and their responsibility to protect it.

Examples of metrics that are helpful for addressing PCI DSS compliance include:

• The percentage of your IT infrastructure that is regularly and consistently evaluated for vulnerabilities and exploits.
• The percentage of known vulnerabilities that have been patched or mitigated.
• The number of personnel who have access to sensitive cardholder information.

HIPAA

The Health Insurance Portability and Accountability Act of 1996 (HIPAA) was signed into federal law by President Bill Clinton on 21 August 1996. 

Of particular importance to security professionals are the HIPAA Privacy Rule and HIPAA Security Rule.

The Privacy Rule establishes a set of national standards for the protection of patients' rights and protected health information (PHI). The major goal of the Privacy Rule was to strike a balance between the confidentiality, integrity, and availability (CIA triad) of healthcare data while still being able to protect the public's health and well being. 

The HIPAA Security Rule establishes a national set of security standards for protecting certain health information that is held or transferred in electronic form. The Security Rule operationalizes the protections contained in the Privacy Rule by addressing the technical and non-technical safeguards.

Examples of metrics to track to ensure HIPAA compliance include:

• The average time it takes for your incident response plan to address known data breaches. 
• The number of cybersecurity incidents reported by employees, stakeholders, and third-party vendors.
• The number of exceptions to your data loss prevention strategy.

GDPR

The European Union's (EU) General Data Protection Regulation (GDPR) is an extraterritorial law designed to protect the personally identifiable information (PII) of EU citizens. 

As an extraterritorial law, it applies to any organization handling EU citizen data, regardless of whether they are in the EU or not. It aims to provide users with greater access to their sensitive data.

Furthermore, any third-party vendors you use must also be compliant with GDPR. This is why vendor risk management (VRM) has become so important. 

Fines for non-compliance are significant, either:

• Up to €10 million, or 2% annual global turnover (whichever is higher)
• Up to €20 million, or 4% annual global turnover (whichever is higher)

Examples of metrics to track for GDPR compliance include:

• The number of data leaks and data breaches detected. Note, you must report personal data breaches no longer than 72 hours after becoming aware of them. 
• The average security rating of your third-party vendors. UpGuard Vendor Risk can help you automatically do this.

CCPA

The California Consumer Privacy Act (CCPA) or AB 375 is a new law that became effective on January 1, 2020, designed to enhance consumer privacy rights and protection for residents in the state of California by imposing rules on how businesses handle their personal information.

The CCPA is akin to GDPR and is the most extensive consumer privacy legislation passed in the United States to date.

Examples of metrics to track for CCPA compliance include:

• The percentage of all internal and vendor systems utilizing data encryption 
• The percentage of vendors who have completed a vendor risk assessment questionnaire mapped to CCPA

Read more about CCPA compliance here. 

CPS 234

CPS 234 is an APRA Prudential Standard that aims to ensure that an APRA-regulated entity takes measures to be resilient against information security incidents (including cyber-attacks) by maintaining an information security capability commensurate with information security vulnerabilities and threats.

It applies to all APRA-regulated entities and failure to comply with CPS 234 can result in the loss of RSE license. 

Examples of metrics to track CPS 234 compliance include:

• The percentage of third and related parties who have had the design of their information security controls assessed against CPS 234
• The number of unapproved changes deployed to production

Read more about how to comply with CPS 234.

LGPD 

The Brazilian General Data Protection Law (Lei Geral de Proteção de Dados Pessoais or LGPD) is a new law that was passed by the National Congress of Brazil on August 14, 2018 and comes into effect on August 15, 2020. 

The LGPD creates a legal framework for the use of personal data of individuals in Brazil, regardless of where the data processor is located just as CCPA and GDPR have for Californians and Europeans respectively.

Examples of metrics to track for LGPD compliance are:

• The percentage of vendors and service providers who have been assessed via third-party security ratings tools and security questionnaires
• The number of security incidents that exposed sensitive data

Read more about LGPD compliance.

PIPEDA

The Personal Information Protection and Electronic Documents Act (PIPEDA) is the federal privacy legislation for private-sector organizations in Canada. 

PIPEDA became law in April 13, 2000 to promote trust and data privacy in ecommerce and has since expanded to include industries like banking, broadcasting and the health sector.  

Examples of metrics to track for PIPEDA compliance are:

• The number of notifiable data breaches involving personal information 
• The number of high-risk vendors processing personal information

Read more about PIPEDA compliance.

FIPA

The Florida Information Protection Act of 2014 (FIPA) came into effect July 1, 2014, expanding Florida's existing data breach notification statute requirements for covered entities that acquire, use, store or maintain Floridian's personal information.  

FIPA is similar to CCPA, GDPR, and LGPD albeit less extensive. Examples of metrics to track FIPA compliance include:

• The percentage of your IT infrastructure that is fully patched and up to date
• The number of intrusion attempts

Read more about FIPA compliance.

The SHIELD Act

The New York Stop Hacks and Improve Electronic Data Security Act (SHIELD Act) or Senate Bill 5575, was enacted on July 25, 2019 as an amendment to the New York State Information Security Breach and Notification Act. The law goes into effect on March 21, 2020.

The goal of the SHIELD Act is to update New York's data breach notification laws to keep pace with current technology by broadening the scope of information covered under the notification law, as well as broadening the scope of what a data breach is. 

Examples of metrics to ensure compliance with the SHIELD Act include:

• The number of cyber attacks stopped
• The mean time for vendors to respond to security incidents

Read more about SHIELD Act compliance.

GLBA 

The Gramm-Leach-Bliley Act (GLBA) is a federal law in the United States that requires financial institutions to explain how they share and protect their customers' nonpublic personal information (NPI).

There are three major components of the GLBA, designed to work together to govern the collection, disclosure, and protection of customers' nonpublic personal information (NPI), namely:

• The Financial Privacy Rule: Restricts the sharing of nonpublic personal information (NPI) about an individual and requires financial institutions to provide each consumer with a privacy notice at the start of the customer relationship and annually thereafter. 
• The Safeguards Rule: Requires financial institutions to develop an information security plan that describes how the company is prepared for and plans to continue to protect customers' and former customers' nonpublic personal information (NPI).
• Pretexting Protection: Pretexting or social engineering occurs when someone tries to gain access to nonpublic personal information without authority to do so. This may entail requesting private information by impersonating the account holder by phone, by mail or by phishing or spear phishing. GLBA encourages organizations to implement safeguards against pretexting. 

Examples of metrics to ensure compliance with GLBA include:

• Percentage of staff who have completed social engineering awareness training
• Percentage of customers who have been given a privacy notice this year

Read more about how to comply with GLBA here.

FISMA

The Federal Information Security Management Act of 2002 (FISMA) is a United States federal law that defines a comprehensive framework to protect government information, operations, and assets against natural and manmade threats.

The framework is further defined by the National Institute of Standards and Technology (NIST) who has published standards and guidelines such as FIPS 199 Standards for Security Categorization of Federal Information and Information Systems, FIPS 200 Minimum Security Requirements for Federal Information and Information Systems and the NIST 800 series. 

FISMA requires federal agencies to develop, document, and implement an agency-wide information system that supports their operations and assets, including those provided or managed by third-parties.

Examples of metrics that can help with FISMA compliance include:

• Percentage of risks remediated from last cybersecurity risk assessment based on 
• Percentage of assets that are continuously monitored

Read more about FISMA compliance requirements.