AI Incident Response Plan

Incident Taxonomy

This incident response plan covers incidents that are unique to or significantly affected by AI systems. Traditional cybersecurity incidents (malware, unauthorized access, DDoS) follow the existing IR plan; this plan supplements it for AI-specific scenarios.

Category 1: Prompt Injection Exploitation

• Description: An attacker successfully injects instructions that override the AI system's intended behavior
• Indicators: Unexpected AI outputs, system prompt exposure, unauthorized actions by AI agents, anomalous output patterns detected by monitoring
• Examples: Customer-facing chatbot providing unauthorized information, AI agent executing unintended tool calls, system prompt extracted and published

Category 2: AI Data Leakage

• Description: Sensitive data is exposed through AI system outputs, whether from training data memorization, RAG retrieval errors, or prompt-response logging
• Indicators: AI outputs containing PII, credentials, or confidential data not present in the user's input; unauthorized data appearing in AI-generated content
• Examples: AI chatbot revealing another customer's data, code assistant outputting API keys from training data, RAG system surfacing confidential documents to unauthorized users

Category 3: Model Compromise

• Description: The AI model itself is tampered with — poisoned training data, modified weights, or substituted model
• Indicators: Sudden behavioral changes, degraded accuracy on validation sets, new biases or harmful outputs that were not present previously
• Examples: Supply chain attack on model weights, adversarial fine-tuning through data poisoning, model swapping in the deployment pipeline

Category 4: AI-Enabled Attack Amplification

• Description: An attacker uses the organization's AI systems to amplify a traditional attack — generating phishing content, automating reconnaissance, or exploiting tool integrations
• Indicators: AI systems generating high volumes of outbound communications, unusual tool usage patterns, AI-generated content used in social engineering
• Examples: Attacker uses internal AI to generate targeted phishing emails, AI agent exploited to exfiltrate data through approved API integrations

Category 5: Bias and Fairness Incidents

• Description: AI system produces discriminatory, harmful, or unfair outputs affecting individuals or groups
• Indicators: Customer complaints about discriminatory treatment, media reports, internal detection through fairness monitoring
• Examples: Hiring AI systematically disadvantaging a protected group, content moderation AI disproportionately flagging content from specific demographics

Severity Matrix

Classification Decision Tree

1. Is there confirmed data exposure involving regulated data (PII, PHI, PCI)? → If yes, minimum SEV-2; if mass exposure, SEV-1
2. Is the AI system actively being exploited? → If yes, minimum SEV-2; if exploitation affects customers, SEV-1
3. Has the AI model been compromised (weights, training data, configuration)? → If yes, minimum SEV-2
4. Is the AI system producing harmful or discriminatory outputs to end users? → If yes, minimum SEV-2
5. Is this a confirmed vulnerability with no evidence of exploitation? → SEV-3
6. Is this a policy violation or potential vulnerability with no confirmed impact? → SEV-4

Severity Reclassification

Severity may be upgraded at any point during the response as new information becomes available. Severity downgrades require approval from the Incident Commander and documented justification.

Phase 1: Detection and Triage (0-30 minutes)

1. Alert Received: Incident detected through monitoring, user report, or external notification
2. Initial Assessment: On-call security analyst reviews alert context, confirms it is AI-related, and performs initial severity classification
3. Incident Commander Assigned: Based on severity:

- SEV-1/SEV-2: Senior security engineer or CISO

- SEV-3/SEV-4: On-call security analyst

1. Communication Channel Established: Dedicated incident channel created in [TOOL] with naming convention: ai-incident-[DATE]-[SEQ]
2. Initial Notification: Stakeholders notified per the severity escalation matrix

Phase 2: Containment (30 minutes - 4 hours)

Immediate Containment Actions by Category

Phase 3: Investigation (4-48 hours)

1. Evidence Collection: Preserve all logs, prompts, responses, model versions, and system configurations
2. Root Cause Analysis: Determine how the incident occurred, what vulnerability was exploited, and what the blast radius is
3. Impact Assessment: Identify all affected users, data, and systems
4. Timeline Construction: Build a detailed timeline from initial compromise to detection

Phase 4: Eradication and Recovery (1-7 days)

1. Vulnerability Remediation: Implement fixes for the root cause
2. System Restoration: Restore AI systems from verified clean state
3. Verification Testing: Run the AI red team playbook against the remediated system
4. Monitoring Enhancement: Deploy additional monitoring for the specific attack pattern

Internal Communication

Notification Matrix

Status Update Cadence

Status Update Template

AI INCIDENT STATUS UPDATE
Incident ID: [ID]
Severity: [SEV-X]
Status: [Investigating / Containing / Eradicating / Recovering / Closed]
Update Time: [TIMESTAMP]

Current Situation: [1-2 sentence summary]
Actions Taken Since Last Update: [Bullet list]
Next Steps: [Bullet list]
ETA for Next Update: [TIMESTAMP]
Incident Commander: [NAME]

External Communication

Regulatory Notification

Customer Communication

Customer notification is required when AI incidents result in exposure of customer data, provision of materially incorrect AI-generated advice, or discriminatory outcomes affecting customers. All customer communications must be reviewed by Legal and PR before distribution.

Post-Incident Review Meeting

Timeline: Conducted within 5 business days of incident closure

Attendees:

• Incident Commander
• All responders involved in the incident
• AI System Owner
• Security Lead
• Engineering Lead
• [ROLE TITLE] (AI Governance Committee representative)

Agenda:

1. Incident Timeline Review (15 min) — Walk through the complete timeline from detection to closure
2. Root Cause Analysis (30 min) — Present and discuss the technical root cause
3. Response Effectiveness (20 min) — Evaluate what went well and what could be improved
4. Detection Gap Analysis (15 min) — Assess why the incident was not detected sooner
5. Remediation Validation (10 min) — Confirm all remediations are in place and verified
6. Action Items (15 min) — Assign and schedule follow-up actions

Post-Incident Report

The Incident Commander produces a written report within 10 business days of the review meeting:

Metrics Tracked

Lessons Learned Process

Capture

Lessons learned are captured from three sources:

1. Post-Incident Review Meeting: Facilitator documents lessons identified during discussion
2. Responder Retrospectives: Each responder submits individual observations within 3 business days of incident closure
3. Metrics Analysis: Quantitative analysis of response metrics compared to targets

Classification

Each lesson is classified by category:

Distribution

Integration into Organizational Processes

Lessons learned must result in concrete updates to at least one of the following:

1. This IR Plan: Update procedures, categories, or severity criteria based on new incident types
2. AI Red Team Playbook: Add new attack scenarios discovered during incidents
3. Prompt Injection Defense Checklist: Update technical controls based on observed attack techniques
4. AI Deployment Validation Checklist: Add checks that would have prevented the incident
5. Training Materials: Update AI security training with real-world case studies (sanitized)
6. Monitoring and Detection: Deploy new detection rules and alerting based on observed indicators

Tracking

All lessons learned are logged in the AI Incident Knowledge Base maintained by [DEPARTMENT]. Each entry includes: incident reference, lesson description, category, action taken, date implemented, and effectiveness assessment (evaluated at the next quarterly review).