Evidence Types and Methods of Evaluation

CyberGRX has developed a strict set of standards for what constitutes acceptable evidence. The following sections of this document describe how those standards are applied to the validation process.

Evidence Types

Evidence Evaluation

Independence

Evidence Types

In general, evidence is categorized into three types:

• Verbal Evidence: Verbal evidence is any evidence which is self-attested. Examples include discussions, written notes or explanations, previously completed self-assessments, etc.

• Written Evidence: Written evidence is any type of evidence in a narrative documentation format. Examples include policies, procedures, emails, etc.

• Demonstrated Evidence: Demonstrated evidence is any type of primary, technical evidence. Examples include actual live demonstrations, screenshots, exports, etc.

The following bullets summarize when each type of evidence may be used to validate a Third Party’s assessment answers:

• Verbal evidence is never acceptable for validation. However, verbal evidence may provide context that is beneficial in the overall validation process.

• Written evidence is acceptable in limited situations where narrative documentation is the focus of the evaluated security control. For example, a copy of an Access Control Policy likely provides acceptable evidence for a control requiring the development of an Access Control Policy.

• Demonstrated evidence is the predominant type of evidence that is requested and used for validation. For example, a screenshot showing the configuration settings of an AWS S3 instance may provide acceptable evidence for a control requiring encryption of data at rest.

Evidence Evaluation

CyberGRX Analysts are tasked with measuring the accuracy of a Third Party’s assessment responses based on the evaluation of evidence artifacts. The following general practices are employed to ensure that the evaluation is fair, correct, consistent, and efficient:

• In order to validate a Third Party’s answers, evidence must directly illustrate the implementation or effectiveness of the evaluated control.

• Analysts may not infer or make assumptions about information that is not clearly illustrated in the provided evidence.

• All evidence must be attributable to the Third Party being assessed.

• Evidence must be attributable to the type of asset that is the focus of the Third Party's chosen assessment answer.

• CyberGRX will accept certain secondary assessment artifacts as evidence (e.g. SOC, ISO, and PCI-DSS reports and certifications) as long as the provided artifacts clearly meet the following criteria:

  • The assessment was conducted by an organization that is wholly independent from the ThirdParty.
  • The assessment must have been conducted within the previous 12months (or have an official bridge letter extending the "expiration" to within 12 months).
  • The scope of the assessment and its relation to the Third Party is explicitly defined.
  • The requirements and objectives of all tested controls can be directly correlated to the Third Party's CyberGRX assessment controls and Third-party answers.
  • The results of the independent assessment tests are clearly documented.
  • The results of the independent assessment resulted in no exceptions from the stated control.

Independence

Analysts must maintain appropriate independence from Third Parties involved in the validation process. In order to maintain this independence, Analysts: 

• Will not provide explicit direction to Third Parties regarding specific evidence. For example, an Analyst will not coach a Third Party on how to navigate to Active Directory and locate a particular GPO which satisfies a control being validated.

• Will not provide explicit remediation recommendations to Third Parties or Customers. Analysts are often called upon to provide expert opinion on various cybersecurity topics. If asked, an Analyst can provide high - level remediation recommendations (e.g. "A properly implemented SIEM tool would improve your ability to collect and analyze log data.")but may not make explicit recommendations(e.g."You should purchase Splunk. That will lower your risk.").

• Will not provide direction to Third Parties on what answer options to choose. Analysts are often asked to sit in on calls or meetings with Third Parties who need clarification during the GRX assessment. Analysts can provide clarity on what a particular control, question, or answer option means, but can not tell the Third Party which answer to choose.