Auditing third-party AI systems
Description
Periodic audits or assessments of third-party AI systems to verify contractual compliance, security controls, and responsible AI claims.
Implementation Guidance
Audit Program
Establish a risk-based audit schedule: annual audits for critical AI suppliers, biennial for significant suppliers, and triggered audits following incidents or major changes. Define audit scope covering: technical performance, security controls, responsible AI practices, and contractual compliance.
Audit Methods
Use appropriate audit approaches: review supplier SOC 2 or ISO 27001/42001 reports, conduct independent testing of AI system outputs (bias testing, accuracy verification), review supplier incident reports and response records, and assess data handling practices against contractual requirements.
Findings and Remediation
Document audit findings with severity ratings. Require remediation plans for significant findings with defined timelines. Track remediation to completion. Consider supplier relationship continuity for critical or recurring findings.
Evidence Requirements
- Audit schedule and scope documentation
- Completed audit reports per supplier
- Third-party certifications and assessment reports
- Remediation tracking and closure records