How do I detect and prevent RAG knowledge-base and retrieval poisoning?

To detect and prevent RAG knowledge-base and retrieval poisoning, it is crucial to implement robust data governance, integrity checks, and adversarial testing throughout the data lifecycle. This addresses the OWASP LLM04 Data and Model Poisoning risk, where RAG-corpus data is manipulated to introduce vulnerabilities.

• Source Vetting and Content Classification: Implement rigorous vetting of data sources and classify content upon ingestion into the RAG corpus. This helps ensure the integrity and trustworthiness of the information being used.
• Change Tracking and Audit: Maintain detailed change tracking with audit logs for the RAG corpus to monitor for unauthorized or malicious modifications.
• Isolation of Content: Isolate user-contributed content from curated content to prevent malicious user inputs from directly poisoning the core knowledge base.
• Adversarial Testing: Conduct periodic adversarial retrieval testing to proactively identify and mitigate potential poisoning attempts. The LAAF framework highlights that static filters are insufficient against patient, automated adversaries, emphasizing the need for runtime logic validation.
• Provenance Tracking: Track the provenance of all datasets and models to ensure their origin and integrity, which is a control for OWASP LLM03 Supply Chain risk.
• Least Privilege and Data Classification Propagation: Apply the principle of least privilege, ensuring agents only retrieve necessary data, and propagate data classifications throughout the data lifecycle. This means security properties of derived data are traceable to their source, preventing accidental loss of classification.
• Memory Contamination Mitigation: Implement strict per-tenant memory scoping and separate physical or logical vector indexes for confidential data to prevent data leakage across users or tenants.