AI security lesson
What Is RAG Security? Attacks, Poisoning, and Defenses
RAG security protects retrieval-augmented generation systems from untrusted documents, manipulated retrieval, data leakage, and instructions hidden inside retrieved content.
How does rag security work?
A RAG application retrieves documents and adds them to model context. Attackers may poison a source, influence ranking, or embed instructions that the model follows as if they were trusted application rules.
- Poisoned document
- Retrieval
- Model context
- Instruction override attempt
Safe example
A synthetic benefits document contains a hidden instruction to ignore policy. When retrieved, it attempts to steer the assistant. The lab executes this trace against an isolated local corpus.
How to test safely
- Inventory document sources, ingestion controls, and trust labels.
- Test poisoned documents in an isolated corpus.
- Inspect which passages were retrieved and why.
- Verify citations, access controls, and behavior when evidence conflicts.
Defense comparison
| Control | Effectiveness | Why it helps |
|---|---|---|
| Source provenance | High | It restricts ingestion and retrieval to accountable sources. |
| Content filtering | Medium | It catches known patterns but can miss novel instructions. |
| Response grounding | Medium | It constrains claims to approved evidence. |
| Action authorization | High | Retrieved text cannot independently approve consequential tools. |
Practice in the RAG System Exploitation lab
Practice identifying poisoned retrieved context and adding provenance, grounding, and authorization controls.
Frequently asked questions
What is RAG poisoning?
RAG poisoning is the placement or manipulation of content so a retrieval system supplies misleading data or malicious instructions to a model.
Are retrieved documents trusted?
No. Retrieved content should be treated as untrusted data even when it comes from an internal index.
Your learning
Finish this lesson
Mark this lesson complete to save it to your learning history.