What are the most effective defenses against prompt injection?

Effective defenses against prompt injection involve a layered approach that includes input validation, context management, and runtime enforcement to prevent manipulation of the model's instructions or actions. These controls address the OWASP LLM01 Prompt Injection risk by establishing trust boundaries and validating actions.

• Input Validation and Transformation: Implement input transformations on untrusted content to sanitize and normalize data before it reaches the model. This includes separating user input from system instructions and establishing trust boundaries for retrieved or tool-generated content.
• Context Management and Trust Tagging: Tag each segment of context with provenance and a trust level, and condition the model to respect these tags. Use a hierarchical context structure with a sealed top layer for system prompts and policies that is never compacted, and a sticky middle layer for session-critical facts. This helps prevent context corruption and exhaustion attacks.
• Runtime Verification and Enforcement: Employ an LLM gateway or AI proxy in front of every model invocation to enforce authentication, apply content policies, and perform PII detection and redaction. Implement tool-call validation gates, including schema validation, allowlisted tools/actions, and parameter constraints, as schema validation is a cheap and effective check.
• Intent Re-verification and Human Oversight: Before any consequential action, re-derive whether the action aligns with the originally attested intent, rather than the agent's potentially corrupted current reasoning. For high-impact actions, incorporate human-in-the-loop confirmation.
• Output Filtering and Content Classification: Implement output filtering and content classification on outgoing data to prevent context from being exfiltrated through tool calls or external responses. This also helps in preventing sensitive information disclosure (OWASP LLM02).
• Adversarial Testing: Conduct adversarial testing to identify vulnerabilities, as static defense filters are often insufficient against sophisticated prompt injection techniques like Logic-Layer Prompt Control Injection (LPCI).