Sensitive Information Disclosure

Description

LLMs may inadvertently reveal sensitive data including PII, credentials, or proprietary information through outputs. Risks include training data memorization and prompt-based extraction.

Risk

LLMs can leak sensitive information through multiple vectors including memorized training data, inference from context, or manipulation via prompt injection. This includes PII, API keys, confidential business data, or system internals. Disclosed information can enable further attacks, violate privacy regulations, or cause competitive harm.

Attack Scenarios

Attacker uses carefully crafted prompts to extract memorized API keys or passwords that appeared in training data, gaining unauthorized access to systems.

User prompts an LLM chatbot with questions designed to infer confidential business information from responses, such as customer lists or pricing strategies not publicly available.

Mitigations

Data Sanitization

Scrub training data of PII, credentials, and sensitive information before use. Implement automated scanning and manual review processes.

Output Filtering

Monitor LLM outputs for sensitive data patterns including regex for credit cards, SSNs, API keys. Redact detected sensitive information before delivery.

Access Controls

Implement strict authorization for LLM features. Ensure users can only access information appropriate to their privilege level.

Differential Privacy

Apply privacy-preserving techniques during training to reduce memorization of specific data points. Use techniques like DP-SGD where appropriate.

Code Examples

# Good: Output filtering for sensitive data
import re
from typing import Optional

class SensitiveDataFilter:
    PATTERNS = {
        'credit_card': re.compile(r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b'),
        'ssn': re.compile(r'\b\d{3}-\d{2}-\d{4}\b'),
        'api_key': re.compile(r'\b[A-Za-z0-9]{32,}\b'),
        'email': re.compile(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b')
    }
    
    def filter_output(self, text: str, user_role: str) -> str:
        filtered = text
        for pattern_name, pattern in self.PATTERNS.items():
            if self._should_redact(pattern_name, user_role):
                filtered = pattern.sub('[REDACTED]', filtered)
        return filtered
    
    def _should_redact(self, pattern_name: str, user_role: str) -> bool:
        # Role-based redaction logic
        if user_role == 'admin':
            return pattern_name in ['credit_card', 'ssn']
        return True  # Redact all for non-admin

Evidence Requirements

Training data sanitization logs and PII removal reports
Output filtering rules and detection patterns
Access control policies for LLM features
Data loss prevention (DLP) monitoring alerts
Privacy impact assessments