What does the NIST AI RMF Measure function require for evaluating an AI agent's risks?

The NIST AI RMF Measure function requires organizations to analyze, track, and measure AI risks, including those specific to Generative AI. This involves evaluating security and resilience, establishing transparency and accountability mechanisms, and continuously tracking identified and emergent risks over time.

Concrete controls for evaluating an AI agent's risks under the Measure function include:

• NIST-MEASURE-2.7: Evaluate and document the AI system's security and resilience, covering aspects like adversarial robustness, prompt-injection resistance (OWASP LLM01), and abuse resistance (OWASP LLM04).
• NIST-MEASURE-2.8: Implement mechanisms to log decisions and trace AI behavior, ensuring that every AI decision leaves a trace for transparency and accountability. This can involve logging decisions to a secure, tamper-evident system.
• NIST-MEASURE-3.1: Establish approaches for tracking identified and emergent risks through continuous monitoring, logging, and drift detection. This cross-maps to ISO/IEC 42001's requirement for monitoring, measurement, analysis, and evaluation of the AI Management System (AIMS) (ISO/IEC 42001 Cl.9).
• ISO/IEC 42001 Cl.9: Conduct monitoring, measurement, analysis, and evaluation of the AIMS, including internal audits and management reviews. This is probed by questionnaire item m4_monitoring_logging.
• MAESTRO L5 (Evaluation and Observability): Implement tamper-evident audit logs (e.g., write-once storage, signed entries, append-only ledgers) and out-of-band shipping to a SIEM to mitigate log tampering. Ensure comprehensive instrumentation by default to prevent observability gaps.
• MAESTRO L5 (Evaluation and Observability): Implement cost anomaly detection to identify runaway agent loops or adversarial leveraging of agents that could generate substantial bills.