Management of third-party AI systems
Description
Ongoing management of third-party AI dependencies including performance monitoring, compliance verification, and relationship governance.
Implementation Guidance
Ongoing Monitoring
Continuously monitor third-party AI performance: track SLA compliance, measure output quality and fairness metrics on organizational data, monitor for unannounced model changes (output distribution shifts), and maintain incident tracking for supplier-related issues.
Relationship Governance
Establish governance structures: regular performance review meetings (quarterly for critical suppliers), escalation procedures for SLA breaches, change notification requirements (supplier must notify before model updates), and executive sponsor assignment for critical AI suppliers.
Dependency Management
Maintain awareness of AI supply chain risks: track concentration risk (over-reliance on single supplier), identify alternative suppliers, plan for supplier failure or discontinuation scenarios, and assess cascading impacts of supplier changes.
Evidence Requirements
- Supplier performance monitoring reports
- SLA compliance tracking records
- Governance meeting minutes
- Dependency and concentration risk assessments