What are the security risks of multi-agent systems and how do I manage them?

Multi-agent systems introduce security risks such as cross-layer attack paths, data exfiltration, and compromised agent identities, which can be managed through robust controls across all architectural layers.

Managing these risks involves implementing specific controls across the MAESTRO framework's seven layers:

• Foundation Models (L1): Threats like prompt injection and adversarial inputs can be mitigated by constraining tasks, grounding tool outputs, using deterministic checks, and comparing grader decisions against human review samples.
• Data Operations (L2): To counter memory poisoning and sensitive data retention, use read-only stores for trusted reference material, separate working stores for unverified lessons, implement provenance on memory writes, establish redaction workflows, and apply review gates before promoting dream outputs.
• Agent Frameworks (L3): Workflow hijacking and tool misuse can be addressed with least-privilege tools per specialist, strict delegation schemas, coordinator-owned final writes, bounded subagent outputs, and rubrics that penalize unsupported claims.
• Deployment and Infrastructure (L4): Container compromise and insecure custom tools require sandbox hardening, network restrictions, scoped credentials, explicit tool approvals for sensitive actions, file path conventions, and resource budgets. Resource isolation at namespace and node-pool levels can mitigate cross-tenant interference.
• Evaluation and Observability (L5): Grader manipulation and incomplete audit trails can be managed with immutable event logs, separate operational dashboards for grader results, human sampling of passed outcomes, alerting on repeated max-iteration failures, and independent security evaluations for high-risk workflows.
• Security and Compliance (L6): To prevent access-control drift and privilege escalation, implement data classification, memory retention rules, role-based and capability-based access controls, compliance review of rubrics, and explicit human approval for irreversible or regulated actions.
• Agent Ecosystem (L7): Agent impersonation and unsafe external API interactions can be mitigated through authenticated agent rosters, version-pinned agents, signed tool definitions, traceable inter-agent messages, and deny-by-default access to external systems. Additionally, treat fallback paths as production paths for security review to address supply chain risks.