What is OWASP LLM06 excessive agency and how do I mitigate it for a tool-using agent?

OWASP LLM06 Excessive Agency occurs when an agent possesses more functionality, permissions, or autonomy than necessary, allowing a manipulated model to execute harmful actions. Mitigating this risk for tool-using agents involves implementing strict controls over the agent's capabilities and actions.

To mitigate Excessive Agency, engineers should implement the following controls:

• Least-privilege tools: Ensure tools are designed with the minimum necessary permissions.
• Allow-listed and narrowly-scoped tool schemas: Restrict the tools an agent can use and define their capabilities precisely. Schema validation on every tool call is a highly effective and inexpensive check.
• Human approval for high-impact/irreversible actions: Implement human oversight for actions that carry significant risk or cannot be easily undone. This can involve synchronous approval for high-risk actions or multi-party approval for catastrophic-risk actions.
• Spend/refund thresholds: Set limits on financial transactions or resource consumption to prevent unbounded damage. Budget-based autonomy can halt an agent that exceeds configured limits.
• Scoped credentials: Provide credentials with limited scope and duration to tools.
• Rate limits: Implement rate limiting on tool usage to prevent resource exhaustion or denial-of-wallet attacks. Per-task and per-agent budgets, circuit breakers, and timeout enforcement are also relevant.
• Intent re-verification: Before any consequential action, the system should re-verify that the action aligns with the originally attested intent, rather than the agent's potentially corrupted current reasoning.
• Action rollback: Design agent tools with reversibility in mind, such as soft-delete defaults or two-phase commits for high-stakes actions, to allow for undoing mistakes.
• Autonomy policy: Define explicit, machine-readable autonomy policies with clear precedence rules to prevent ambiguity. This policy should be expressed at the effect level (outcomes), not just the action level (specific API calls), to prevent autonomy shopping.
• Tiered authorization: Implement a system where low-risk actions have a fast path, medium-risk actions undergo anomaly checks, and high-risk actions require synchronous approval.