MIT Internet Policy Research Initiative

Summary Report: Cyber Risk Measurement Expert Group Meeting

Tuesday, March 1st & Wednesday, March 2nd, 2022

Jointly hosted by
The Federal Reserve Board of Governors, Federal Reserve Bank of Richmond, and the Massachusetts Institute of Technology

Overview

Select firms across the Financial Services sector, as well as Third-Party Data Providers, met in early March for two half-day sessions with The Federal Reserve Board of Governors, Federal Reserve Bank of Richmond, and the Massachusetts Institute of Technology. The purpose of this initial Expert Group Meeting was to discuss cybersecurity risk measurement practices, trends, challenges, and future state aspirations in an informal forum before delving into particular topics of interest in the coming months.

It is well understood by attendees that there is no comparative advantage in security — we all work together, or risk collective failure. There is a clear opportunity to identify a common set of questions and goals to better manage cyber risk and pinpoint the underlying data that would support answering those questions or achieving those goals. In this vein, meeting participants acknowledged the benefits of consistent definitions and risk measurement methodologies, as well as the desire to leverage better mechanisms to share meaningful cyber risk data, with an emphasis on loss data resulting from a cybersecurity incident.


Detailed below is a brief summary of the Expert Group Meeting outcomes, including common goals identified, notable insights, and proposed next steps to make progress towards better cyber risk management through measurement.

Identified Common Goals

Over the two-day discussion, meeting participants identified five key goals driven by topics that  resonated with participants:

 

1.Identify and implement enhanced methods for data collection of monetary losses from cybersecurity incidents, contextualized with controls data.

There is a dearth of easily accessible data on the effectiveness of controls. Controls, in the  context of security, can be defined as the processes and capabilities that identify, detect,  prevent or respond to cybersecurity threats to mitigate associated data confidentiality, integrity

and availability risks. Controls effectiveness data provides information such as the mean time to  detect malicious activity or the frequency of a control failure.

During the discussion, it was emphasized that access to monetary loss data in the context of  control failures would help inform the creation of robust cyber risk models akin to market and  credit risk models used today. This could help financial institutions identify and monitor outsized  exposure. Additionally, these models could derive support through:
● Better measurements to quantify control efficacy
● Contextualization of associated losses when a control fails
● Collection of data and creation of metrics related to near misses
● Automation of controls data collection, including for cyber incidents and events

2. Compare control efficacy across peers by creating a mechanism to share and benchmark controls data among consenting institutions.

While threat and vulnerability information is more readily available across firms today, there  remains an ongoing need to share and compare controls data given the lack of cross-firm  visibility in this area. This would help institutions make better-informed risk-based decisions,  and drive adjustments to the greatest areas of comparative weakness. 

3. To achieve #1 and #2 noted above, normalize control taxonomies and metrics, resulting in more efficient and achievable data collection, data sharing, and cyber risk analysis across the financial services industry.

There are several ongoing efforts within the financial services industry to normalize data types,  but broader agreement and adoption is still desired. This should be driven by specific use  cases, especially as they pertain to the creation of meaningful cyber risk models. 

Relatedly, there is a need to contextualize cyber-related data and quantification within the  broader landscape of operational risk management and operational resilience. For example, it  would be beneficial to determine how to appropriately aggregate and disaggregate operational  risk loss data.

4. Incorporate systemic risk data and data on third-party interconnections and interdependencies (including financial services technology) into cyber risk quantification efforts.

While firms have visibility into their own controls and losses (sometimes), most organizations  lack both a mechanism and the data to properly understand, measure, and manage the  systemic risk of interconnected institutions. This is a key piece missing from risk appetite 

conversations. Academic institutions, regulatory agencies and third-party vendors are well positioned for significant productive research in this domain. For example, research could help  clarify how a singular, significant breach of a large financial institution would impact other  stakeholders in the industry and the economy.

Additional Insights

Several notable items raised by speakers also stood out:

● A mix of leading and lagging indicators can be leveraged to manage cyber risk. Categories of leading indicators include threat trends and proactive KRIs, whereas lagging indicators align to control improvements / KPIs.
● Pairing data can yield new insights. Examples shared include threat vectors aligned with associated controls, linked controls that often fail together, and associating cyber risks with underlying business risks and processes.
● Board-level cyber risk metrics should be prioritized and contextualized to ensure meaningful discussions and decision-making.

Next Steps

Based on our discussion, we propose the following next steps:
● Connect with stakeholders on commonly-shared, high-priority cyber risk measurement issue areas that touch upon the identified goals, and work together in small groups to develop concrete proposals that can advance these goals in the near term. We are looking to focus on areas of shared interest and mutual benefit that cannot be realized alone. Proposed tasks for working groups include:
○ Creation of a common data taxonomy and predefined data structures to measure loss. Articulate methodologies for collecting and characterizing this data that incorporates loss-measurement methodologies
○ Agreement on concrete privacy and security requirements for data sharing platforms, including consideration of leakage risk from auxiliary information attacks.

○ Formation of data access and model transparency requirements for controls data collected over the course of future efforts.
● Create a cyber risk measurement “academic research wish-list” on the part of financial services, including strategies for making data available for research.

● Follow up with a larger cybersecurity risk measurement conference targeted for the fall.