How do I prevent model denial of service (OWASP LLM04) on an AI agent?

To prevent model denial of service (OWASP LLM04), implement resource limits and access controls to manage consumption and prevent unbounded querying.

Practical controls include:

• Rate limits and quotas should be applied to API endpoints to prevent resource exhaustion.
• Token and spend caps should be implemented to control the volume and cost of LLM API calls, especially given that a single user task in a multi-agent system can consume tens of thousands of tokens.
• Abuse detection mechanisms should be in place to identify and respond to suspicious activity.
• Access controls on model endpoints and weights are crucial to prevent unauthorized access and potential model extraction or theft.
• Resource budgets should be set per task and per agent, along with circuit breakers and timeout enforcement, to manage consumption at the agent framework and deployment layers.
• Compute autoscaling should be configured to respond to request queue depth, in-flight task count, or token-per-second throughput, rather than just CPU, to handle increased load efficiently and prevent a system from becoming a denial-of-service target.
• Provider rate limits should be considered a binding constraint, requiring negotiation of committed throughput with providers and architectural planning around quota.
• Cross-tenant interference can lead to latency or availability degradation, so resource isolation at the namespace and node-pool level should be implemented.