Findings Methodology

Introduction

CyberGRX is excited to launch a new and improved methodology to determine Findings on a company's cybersecurity assessment! This enhanced Findings methodology is generally available with the go live of Risk Navigator in January 2024. 

The purpose of this article is to share where users can view the new and old findings, describe the new Findings methodology as well as highlight the improvements over the previous one. 

Availability

The new findings are available to view at the following locations:

• Risk Navigator Controls Table
  • Controls Table on the Third Party Profile -> Risk Profile tab
• Risk Navigator Spreadsheet
  • "Download (XLSX)" button on the Third Party Profile -> Risk Profile tab
• Risk Navigator API V2
  • GET /v2/portfolio/third-parties/{company_id}/risk-navigator/{framework_id}

The previous findings will continue to be available with limited support into 2024 to ensure users have time to understand the new process and transition. Previous findings are available to view at the following locations:

• Assessment Results Spreadsheet
  • "Download Results (XLSX)" button on the Third Party Profile -> Assessment or Company Information tabs
• Assessment Report PDF
  • "CyberGRX Report" button on the Third Party Profile -> Assessment or Company Information tabs
• Get Residual Risk API V1 
  • GET /v1/third-parties/{third_party_id}/residual-risk/{report_id}

Enhanced Findings Methodology

Overview

The enhanced GRX Findings methodology is designed to be Explainable, Industry Leading, Comprehensive, Actionable, Transparent, Consistent and Adaptable.

Explainable

The new Findings methodology is GRX Control Score Based with Control Classification. The straightforward logic used to determine if a GRX control will be flagged as a finding and the associated severity is presented in the following table:

Score*	Finding Severity		Possible Scenarios
	Essential Controls MITRE + Critical	Comprehensive Controls All Other
Less than 50	HIGH	MEDIUM	Tier 1 Control Answered No Tier 2 Control Answered No Predictive Control
50 - 69.99	MEDIUM	LOW	Tier 1 Control Answered Yes, with low Metrics Answers Predictive Control
70 - 84.99	LOW	Not a finding	Tier 1 Control Answered Yes, with decent Metrics Answers Predictive Control
Greater than or equal to 85	Not a finding	Not a finding	Tier 1 Control Answered Yes, with solid Metrics Answers Tier 2 Control Answered Yes Predictive Control
NA / No score / Not asked	Not a finding	Not a finding	Tier 1 Control Answered NA Tier 2 Control Answered NA Tier 1 Control Not Answered, earlier version attested Tier 2 Control Not Answered, earlier version attested Controls not yet in Predictive

* score thresholds may change over time to ensure they are optimal based on the data

Industry Leading

As indicated in the above table, GRX Controls are classified either as Essential Controls or Comprehensive Controls. This classification is intended to help prioritize Findings and provide guidance for customers and third parties alike, especially in cases where the number of Findings is significant.

Essential Controls are given a higher severity to help prioritize Findings for review and remediation. These Essential Controls are comprised of the GRX Controls that are MITRE Relevant and / or a Critical Control, leveraging external industry standards and internal GRX standards.

MITRE ATT&CK is perhaps the largest, most in-depth, organized, and strongly supported knowledge base of adversarial behavior. To read more on MITRE ATT&CK explore their webpage or our knowledge base article here. View which controls are currently mapped to MITRE by selecting the "MITRE Full Technique List v12" framework in Risk Navigator. As new versions of MITRE are released, CyberGRX re-evaluates and updates the GRX Control mapping.

The GRX Critical Controls are 60 Controls determined to be basic security hygiene as noted in security-focused documentation (i.e. VDBR, Owasp/CISA/CSA) and are also controls that are reasonable to provide evidence proving the implementation. These are the ones that undergo the Validation process when requested. While the majority of these Critical Controls overlap with the Controls that are MITRE relevant, the new methodology ensures that all Critical Controls with a poor score will be flagged with a High severity, ensuring they are prioritized for remediation. View which controls are classified as Critical Controls by selecting the "CyberGRX 60 Critical Controls" framework in Risk Navigator. At this time the 60 Controls marked as Critical are fairly stable to ensure consistency for members and are reviewed yearly.

The Comprehensive Controls are made up of All Other Controls in our GRX Assessment. These Comprehensive Controls are given a lesser severity, depending on their score, if they are determined to be a Finding. These controls are important and recommended for comprehensive security hygiene and should be considered for all companies, especially High Inherent Risk vendors. 

Comprehensive

The Findings methodology evaluates all controls in the GRX Assessment. While controls are weighted differently based on their classification, customers can be assured that the complete assessment is reviewed to determine findings. The score driven approach ensures that if a Third Party completed and authorized a Tier 1 with metric questions, the answers from the metric questions are reflected in the control score to drive Finding Severity. 

Customers can then evaluate the results and determine which of those findings should be turned into an issue for further action. As the GRX content grows over time due to new technology or changes in the threat landscape, new controls will fall into these classifications so customers can continue their preferred business process. 

Actionable

Predictive assessment Findings are available for evaluation on most companies in the GRX Exchange. The Predictive assessment provides predictions of how a cyber risk program is implemented and managed to enable timely decisions especially in scenarios where there is limited time to evaluate a third party, the third party may resist requests for an assessment or be significantly delayed in its completion. Use the High and Medium Findings, especially those with Significant or Moderate Impact, to prioritize conversations with a Third Party. Depending on the Inherent Risk associated with that Third Party, request an assessment for the Attested view of the Findings.

Attested assessment Findings are available after a company has completed, attested and authorized the GRX Assessment. The Attested assessment provides a robust examination of how a cyber risk program is implemented and managed. For Tier 2 assessments, only Controls answered "No" will be flagged as a finding. Use the mitigation information provided to determine the preferred course of action. For Tier 1 assessments, review the metric question responses in addition to the mitigation information provided to determine the preferred course of action.

Workflows for tracking and actioning assessment Findings can be configured in the ProcessUnity Workflow platform, with our pre-built GRX integration. 

Transparent

A third party on the Exchange looking at their own profile will see the same or more Findings and equivalent or higher Finding Severity as a Customer viewing the same Third Party profile. This is true for both Predictive Assessments as well as Attested and Authorized Assessments. 

This will ensure that when a Customer reaches out to a Third Party about a Finding, the Third Party will always have the most information about their own Findings. The table below outlines if a Third Party has a certain level of data on the Exchange and their Customer is authorized to see some data, what is the differences in that view. 

Third Party Exchange Data	Customer Authorized Data	Findings Number and Finding Severity View
Predictive	Predictive	The Third Party and Customer see the same number of findings and equivalent severity
Any Attested	Predictive	The Third Party can always view their own Predictive Findings in addition to their Attested Findings
Tier 2	Tier 2	The Third Party and Customer see the same number of findings and equivalent severity
Tier 2 V	Tier 2	The Third Party and Customer see the same number of findings and equivalent severity
Tier 2 V	Tier 2 V	The Third Party and Customer see the same number of findings and equivalent severity
Tier 1 V	Tier 2	The Third Party may see a higher number of findings with higher severity based on their Metric answers
Tier 1 V	Tier 2 V	The Third Party may see a higher number of findings with higher severity based on their Metric answers
Tier 1 V	Tier 1 V	The Third Party and Customer see the same number of findings and equivalent severity

Consistent

A finding methodology that is score based, with control classification, allows for consistency in determining Findings for both Predictive Assessments as well as Attested Assessments. This allows the pre-assessment review process leveraging Predictive Assessments to be closely aligned with the post-assessment review process for Attested Assessments.

Additionally, Findings at the control level means that customers will have a consistent experience between reviewing Tier 1 assessments and Tier 2 assessments.

Adaptable

This Findings methodology leaves space for additional improvements. We believe it allows for Customers to understand and adapt these Assessment Findings into your business process in a way that makes the most sense for your company. You can decide which level of Findings to review, paired with the related "Maximum Impact" on a control. We welcome feedback and questions. 

Summary

• Easier methodology to explain
• MITRE and Critical Controls are treated more severely
• Full representation and examination of all controls
• Third Parties have highest level of visibility into Findings
• Attested Tier 1, Attested Tier 2, and Predictive Assessments are treated the same way
• Aligns API traceability for improvement
• Room for additional improvements

Previous Findings Methodology

Overview

The previous Findings methodology was designed to be MITRE Control Based with Impact Weighting. This methodology is Industry Leading. The methodology was also Comprehensive when it was introduced, based on the Assessment Content at that time. Over time as the cybersecurity landscape evolved and CyberGRX innovated, the previous methodology did not serve the additional goals of being Explainable, Comprehensive, Transparent, Consistent and Flexible. 

MITRE Control Based

Gaps, along with any remediation recommendations, indicate potential vulnerabilities discovered during CyberGRX's analysis of applicable threat scenarios (kill chains), tactics, outcomes, and targeted industries collected from years of observed and theoretical cybersecurity events. 

The discovered gaps are then compared against the company's assessment answers to determine how well they have been compensated. For Tier 2 assessments, controls that were answered No are adjudicated as High risk gaps due to lack of mitigation and assigned a remediation recommendation. The remaining gaps are marked as Low risk but assigned no recommendation to acknowledge that even a Yes answer to a control provides only coarse indication of its actual deployed robustness.

For Tier 1 assessments where strength, timeliness, and coverage are available, individual control metrics are adjudicated separately as High, Medium, or Low risk gaps based on preset scoring thresholds. 

Impact Weighting

Gaps are also influenced by asset impact (e.g. Data, Applications, Networks, Facilities, etc.) to better identify threats that target areas where a customer actively engages their third parties to provide specific services. High and Medium risk gaps are assigned individually tailored remediation remediations with Low risk gaps given no recommendation due to their favorable assessment performance.

Leveraging the asset impact associated with a control has great value and CyberGRX is exploring possibilities for highlighting Impact in the enhanced Findings methodology while keeping to the design goals.

Limitations

Scoring has a dependence on CyberGRX controls being mapped to a MITRE technique or a MITRE technique being mitigable.  For example, Device Driver Discovery cannot be easily mitigated with preventive controls since it based on the abuse of system features.  Additionally, controls specific to your facilities, such as, Sensitive Printed Material Access Controls, would require custom techniques not addressed in MITRE.

As a result, 110 controls are mapped to MITRE techniques as a primary control. If the mapped control is closely related, it is given a primary role while if the mapped control only exists in a supporting capacity or has secondary applicability, it is given a supporting role.  The total number of controls mapped as primary or supporting to MITRE is 148** leaving 72** controls unreported.

** Number accurate at the time of writing. 

Summary

• The methodology was not transparent to users of the platform
• The methodology at tier 1 effectiveness level was complicated to explain
  • For example, three findings per metric level and the threshold for High vs Medium
• Limiting the candidate finding set by design
  • Not all controls were being analyzed, in other words, controls not mapped to MITRE resulted in customer complaints since they were not reported as a findings
• Limiting the candidate finding set by design
  • Even if all ~110 controls mapped to MITRE were found to be a finding for a Tier 2, only 63 findings were shared to help with prioritization
• An assessed company could have different findings by customer based on asset interaction
• An assessed company would not have visibility into the different findings by customer based on asset interaction
• Tier 2 YES answers could be flagged as a LOW finding and customers did not have a clear action to take 
• Predictive findings did not fit as well into the methodology

Change Management Expectations

We understand that transitioning to the new Findings methodology and integrating it into your business process will take time. This is why the previous Findings will continue to be available for a period of time to help with the changeover. Below are examples and an analysis of what to expect for changes in the number of findings. 

For a Tier 1 assessment, the number of findings in the enhancement methodology will likely decrease from the previous methodology. This is due to reporting findings at the Control level for improved consistency and not at the Metric level, which led to redundancy and sometimes an overwhelming number of findings to manage.

For a Tier 2 assessment, the number of findings in the enhancement methodology will likely increase from the previous methodology. This is due to now evaluating all the Controls in the Assessment and not only the MITRE relevant controls as well as not capping the findings output at 63. 

Examining only the High findings for existing Tier 2 assessments at Tier 2 report level allows an assessee to be informed to set expectations.  We recommend not treating this as a before and after analysis.  This is to gauge future findings output of an existing assessment. 

Note: Controls answered No for tier 2 assessments will end up being High if MITRE/Critical and Medium otherwise.  Yes answers are given a rating of 85 resulting in no Low findings.  This is by design.

Interpretation of figure:
Previous Findings scoring for a Tier 2 had a product design upper bound of 63 gaps to assist with prioritization. 
New Findings scoring will look at all 172 MITRE mapped controls for a Tier 2.
As a result, High finding output is expected to increase by 2.75 times the current finding total.  This is due to the original artificial limit of 63 findings.  The linearity of the above figure shows that the number of findings would have risen to roughly this level if not limited before.