NIST AI RMF ASSURANCE DATA MODEL

Competency and expertise

Part of: MS: MEASURE — Assessment, Metrics & Testing

Description

AI system measurement activities are performed by individuals and teams with appropriate domain knowledge, technical expertise, and diversity. The quality of AI measurement is bounded by the competence and diversity of the measurement team.

Suggested Actions

1

Assemble multi-disciplinary measurement teams including data scientists (technical evaluation), domain experts (contextual validity), ethicists (fairness and impact assessment), and affected community representatives (lived experience perspective)

2

Ensure evaluators possess required technical competencies: statistical testing methodology, fairness metric computation, adversarial testing tools, and domain-specific evaluation frameworks

3

Include diverse perspectives in measurement teams to identify impacts that homogeneous teams miss — research consistently shows diverse teams identify more risks and failure modes

4

Provide ongoing training on measurement tools, bias detection methodologies, responsible AI evaluation frameworks, and emerging threats

5

Ensure independence between measurement teams and development teams for high-risk systems — the people who built the system should not be the sole judges of whether it works

6

Document team composition and competency assessments for each evaluation — auditors and regulators will ask who conducted the assessment and whether they were qualified

Related Controls

AI Acceptable Use Policy