Tool Misuse and Exploitation

Description

Agents invoke tools with destructive or unintended parameters, chain tool calls in exploitable sequences, or are misled by poisoned tool descriptors that misrepresent capabilities.

Mitigations

Parameter Validation

Validate all tool parameters against strict schemas before execution:

• Define JSON Schema for every tool's input parameters with type constraints, enums for known values, and regex patterns for string fields
• Reject parameters containing SQL keywords, shell metacharacters, or path traversal sequences (../, ~, absolute paths outside allowed directories)
• Enforce maximum lengths on all string parameters to prevent buffer-based attacks
• Never pass raw user input or LLM-generated strings directly as shell commands or SQL fragments

Tool Allowlists and Scoping

Restrict which tools an agent can access based on its current task:

• Implement per-task tool allowlists that limit available tools to the minimum set required
• Use capability-based security: tools must be explicitly granted, not available by default
• Separate read-only tools from write/delete tools with different authorization levels
• Require explicit human approval for destructive operations (DELETE, DROP, format, overwrite)

Execution Sandboxing

Run tool executions in isolated environments:

• Use containerized execution contexts with resource limits (CPU, memory, network, filesystem)
• Apply network policies that restrict outbound connections to approved endpoints only
• Mount filesystems read-only unless write access is specifically required for the tool
• Capture complete execution logs including all parameters, return values, and side effects

Tool Chain Analysis

Monitor and constrain multi-tool sequences:

• Define prohibited tool sequences (e.g., read_credentials -> network_request)
• Implement a tool-call graph that tracks data flow across sequential tool invocations
• Flag chains that move data from high-sensitivity tools to external-facing tools
• Rate-limit tool invocations to prevent rapid automated exploitation

Tool Descriptor Verification

Validate tool descriptors from external sources:

• Require cryptographic signatures on tool descriptors from MCP servers and plugin registries
• Compare advertised tool descriptions against observed behavior using canary inputs
• Maintain a curated, organization-approved tool catalog with pinned versions
• Quarantine and review new or updated tools before granting agent access

Code Examples

Tool Execution Wrapper with Validation

import re
import json
from typing import Any, Callable
from dataclasses import dataclass, field


@dataclass
class ToolPolicy:
    allowed_tools: set[str]
    blocked_sequences: list[tuple[str, str]]  # (tool_a, tool_b) pairs
    require_approval: set[str]  # tools needing human sign-off
    max_calls_per_minute: int = 30


DENIED_PATTERNS = [
    re.compile(r"(DROP|DELETE|TRUNCATE|ALTER)\s", re.IGNORECASE),
    re.compile(r"(;|--|/\*|\*/)"),  # SQL injection markers
    re.compile(r"\.\.[\\/]"),  # Path traversal
    re.compile(r"[`$]\("),  # Command substitution
    re.compile(r"\|\s*(bash|sh|curl|wget|nc)"),  # Pipe to shell
]


class ToolExecutionWrapper:
    """Validates and sandboxes all tool invocations."""

    def __init__(self, policy: ToolPolicy):
        self._policy = policy
        self._call_history: list[str] = []
        self._registry: dict[str, Callable] = {}

    def register_tool(self, name: str, handler: Callable, schema: dict):
        self._registry[name] = {"handler": handler, "schema": schema}

    def execute(self, tool_name: str, params: dict[str, Any]) -> dict:
        # 1. Check tool is allowed
        if tool_name not in self._policy.allowed_tools:
            return {"error": f"Tool '{tool_name}' not permitted for this task"}

        # 2. Check for blocked sequences
        if self._call_history:
            last_tool = self._call_history[-1]
            for blocked_a, blocked_b in self._policy.blocked_sequences:
                if last_tool == blocked_a and tool_name == blocked_b:
                    return {"error": f"Blocked sequence: {blocked_a} -> {blocked_b}"}

        # 3. Scan parameters for injection patterns
        violations = self._scan_params(params)
        if violations:
            return {"error": "Parameter validation failed", "violations": violations}

        # 4. Check if human approval is required
        if tool_name in self._policy.require_approval:
            return {"status": "pending_approval", "tool": tool_name, "params": params}

        # 5. Execute and record
        self._call_history.append(tool_name)
        handler = self._registry[tool_name]["handler"]
        return handler(**params)

    def _scan_params(self, params: dict, path: str = "") -> list[str]:
        violations = []
        for key, value in params.items():
            current_path = f"{path}.{key}" if path else key
            if isinstance(value, str):
                for pattern in DENIED_PATTERNS:
                    if pattern.search(value):
                        violations.append(
                            f"Dangerous pattern in '{current_path}': {pattern.pattern}"
                        )
            elif isinstance(value, dict):
                violations.extend(self._scan_params(value, current_path))
        return violations

Evidence Requirements

• Tool invocation audit logs with full parameter capture and validation results
• Blocked sequence detection reports showing prevented exploitation chains
• Tool descriptor integrity verification records with signature validation status
• Human approval workflow logs for destructive operations
• Periodic tool behavior comparison reports (advertised vs. observed)