How do I rate-limit and cap spend on AI agents?

To rate-limit and cap spend on AI agents, implement per-task and per-agent budgets, circuit breakers, and timeout enforcement, alongside a robust cost accounting system that tracks and attributes token usage.

Here are concrete controls for rate-limiting and spend capping:

• Rate-limiting and Resource Exhaustion (OWASP LLM Top 10: L3, L4): Implement sliding window rate limits for requests to prevent resource exhaustion. Set per-task and per-agent budgets, use circuit breakers to interrupt operations when limits are reached, and enforce timeouts for LLM calls to prevent indefinite waiting and excessive spend.
• Cost and Token-Usage Accounting (NIST AI RMF: Govern): Establish an internal economy for agents by normalizing provider-specific usage into a canonical schema, multiplying tokens by per-million pricing tables, and attributing spend to the correct model and cache category. This allows for cost attribution, exportable metrics, and protection against runaway spending.
• Threshold Gates (NIST AI RMF: Govern): Accumulate per-session totals for token usage and cost, and implement a threshold gate that warns or blocks the user when spending crosses a predefined limit. This provides runaway-spend protection.
• Max Tokens Cap (OWASP LLM Top 10: L3): Enforce a maximum token cap for LLM responses to limit the length of generated output and control associated costs.
• Inventory and Monitoring (NIST AI RMF: Map, Measure): Maintain an inventory of all running agents to enable the application of controls like rate limits proactively. Monitor agent behavior and resource consumption to detect deviations from established baselines.