How do I run guardrail regression tests for an AI agent in CI/CD?

To run guardrail regression tests for an AI agent in CI/CD, implement a control plane that scans and enforces policies on agent components and runtime behaviors, and integrate comprehensive testing including unit, integration, and policy validation tests. This approach addresses the dynamic and composable nature of agentic systems.

Guardrail regression testing should include admission control for skills, MCP servers, and plugins, ensuring that nothing runs until it is scanned. This aligns with the MAESTRO Layer 3: Agent Frameworks, where DefenseClaw scans and evaluates these components before they run, unifying scanner findings into policy decisions. For example, agentctl policy validate can be used to validate Cedar policies.

Implement runtime inspection of prompts, responses, and tool calls to detect prompt injection and sensitive-pattern risks before damage propagates. This addresses MAESTRO Layer 1: Foundation Models, by securing the usage boundary and inspecting LLM traffic through the guardrail flow. This also helps mitigate OWASP LLM01 (Prompt Injection) and LLM02 (Insecure Output Handling).

Utilize CodeGuard to detect hardcoded credentials, weak crypto usage, unsafe deserialization, risky execution patterns, SQLi-like constructs, and path traversal indicators in agent-handled data and generated code paths. This contributes to MAESTRO Layer 2: Data Operations by providing pre-execution and write-path controls.

Integrate unit tests for adapters (e.g., pytest tests/unit/test_claude_code_adapter.py) and integration tests to verify the full flow from adapter to policy to decision (e.g., pytest tests/integration/test_adapter_run_integration.py). Additionally, use pytest --cov=agentctl --cov-report=term-miss tests/ to check code coverage and identify untested lines.

For security, review authentication boundaries, input validation, secrets management, dependencies, and data exposure, escalating any issues that could bypass permissions or leak user data. This is crucial for MAESTRO Layer 6: Security and compliance, which addresses threats like access-control drift and privilege escalation.

Finally, ensure that an AI/agent incident-response plan is in place for post-deployment monitoring, covering detection, escalation, containment, communication, and learning, as per NIST-MANAGE-4.1.