Insecure Plugin Design

Description

LLM plugins and extensions accept untrusted inputs or lack proper authorization. Insecure plugins enable unauthorized access, data exfiltration, or remote code execution.

Risk

Plugins extend LLM functionality but often lack security hardening. They may accept unvalidated LLM outputs as trusted inputs, fail to implement authorization checks, or expose dangerous capabilities without safeguards. Compromised plugins can bypass core LLM security controls and directly access sensitive systems or data.

Attack Scenarios

An email plugin for an LLM assistant accepts recipient addresses directly from model outputs without validation, allowing prompt injection to send phishing emails to arbitrary recipients.

A code execution plugin runs LLM-generated code without sandboxing or permission checks, enabling attackers to execute arbitrary commands on the host system via prompt manipulation.

Mitigations

Input Validation

Treat all inputs to plugins as untrusted, including those from the LLM. Validate, sanitize, and enforce strict schemas for all parameters.

Least Privilege

Grant plugins minimal required permissions. Use separate service accounts with restricted access to resources.

Sandboxing

Execute plugins in isolated environments with limited network, filesystem, and system access. Use containers or VMs for strong isolation.

User Confirmation

Require explicit user approval for high-risk plugin actions such as sending emails, executing code, or accessing sensitive data.

Code Examples

# Bad: Trusting LLM output directly
class EmailPlugin:
    def send_email(self, llm_output: str):
        to = llm_output  # No validation
        send_mail(to, subject, body)

# Good: Validating plugin inputs
from typing import Optional
import re

class SecureEmailPlugin:
    ALLOWED_DOMAINS = ['example.com', 'company.com']
    
    def send_email(self, recipient: str, subject: str, body: str, 
                   user_id: str) -> Optional[str]:
        # Validate email format
        if not re.match(r'^[^@]+@[^@]+\.[^@]+$', recipient):
            raise ValueError("Invalid email format")
        
        # Domain allowlist
        domain = recipient.split('@')[1]
        if domain not in self.ALLOWED_DOMAINS:
            raise ValueError(f"Domain {domain} not allowed")
        
        # Require user confirmation
        if not self.user_approved(user_id, recipient):
            return "Pending user approval"
        
        send_mail(recipient, subject, body)
        return "Email sent"

Evidence Requirements

Plugin security design reviews and threat models
Input validation and schema enforcement in plugin code
Sandboxing and permission configurations
User confirmation workflows for high-risk actions
Plugin authorization audit logs