What does the NIST AI RMF require for monitoring AI systems?

The NIST AI RMF requires organizations to monitor AI systems post-deployment, track risks over time, and have incident response plans in place.

• Post-deployment monitoring and incident response: Organizations must implement post-deployment monitoring and an AI/agent incident-response plan that covers detection, escalation, containment, communication, and learning. This directly addresses the NIST AI RMF's MANAGE function.
• Risk tracking: Approaches for tracking identified and emergent risks, including monitoring, logging, and drift detection, must be established. This aligns with the MEASURE function of the NIST AI RMF.
• Mechanisms to sustain value and retire safely: Procedures should exist to deactivate, roll back, or retire AI systems that exceed risk tolerances, such as kill-switches or rollback capabilities for agents. This falls under the MANAGE function.
• Security and resilience evaluation: AI system security and resilience, including adversarial robustness and prompt-injection resistance, must be evaluated and documented. This relates to the MEASURE function and addresses OWASP LLM01 (prompt injection) and LLM04 (model DoS).
• Transparency and accountability: Mechanisms to log decisions and trace AI behavior are required. For example, logging every AI decision with structured data can implement this control. This is part of the MEASURE function.