Retrieval-augmented generation (RAG) security
Protect document ingestion, retrieval permissions, provenance, and cached answers so grounding an AI response does not become a new disclosure path.
What it means
RAG security protects the retrieval and generation pipeline that brings external or internal documents into an AI response. Risks include poisoned content, inappropriate retrieval permissions, and sensitive data disclosure. A document being searchable does not mean every caller should receive its contents.
A practical example
An employee assistant filters retrieval according to the requesting user's document permissions. It preserves source provenance, treats retrieved instructions as untrusted data, and verifies that summaries do not reveal restricted records through a shared index.
What to check
Review ingestion trust, index access, tenant boundaries, retrieval filtering, and response disclosure. Test poisoned documents and permission changes, including whether cached results remain accessible afterward.
A common pitfall
Grounding a response in documents does not guarantee correctness or authorization. The source may be malicious, outdated, or outside the requester's permitted data.
Think it through
Why enforce access before retrieved text reaches the model?
Answer: Once restricted material is in the context, output filtering alone may not reliably prevent disclosure.
Retrieval creates a disclosure boundary
Imagine a fictional academy builds an assistant over public notes and private learner-support records. The assistant should not retrieve every indexed document for every caller simply because they ask a relevant question. Access must be evaluated before restricted content enters the model's context.
A shared search index can be useful without being a shared permission set. Preserve document identity, source, and access metadata in the retrieval design, and verify how changes to the original permissions reach the index.
Relevant text can still be hostile or wrong
Suppose an uploaded document contains instructions to reveal other records. The text is retrieved because its topic matches a question, but relevance does not make its instructions authoritative. Treat retrieved content as evidence to interpret rather than permission to perform unrelated actions.
The same pipeline can retrieve outdated or inaccurate material without malicious intent. Source provenance, publication context, and citations help users assess an answer, but citations do not make the answer automatically correct.
Test the lifecycle of a document
A record may be added, corrected, restricted, or deleted. Each transition can affect an index, derived summaries, and response caches. If a user loses access, stale retrieval or cached answers can preserve a disclosure path unless the system accounts for it.
For the academy, define how the index learns permission changes and what the service does if that information is unavailable. Output filtering should not carry the entire burden of preventing disclosure after unauthorized material has already been supplied.
For a safe exercise, create three invented documents: public, staff-only, and restricted to one support team. Ask the same question as three different users and define which sources may enter each response. Then revoke one permission and test again. Add a malicious instruction within a permitted document and explain why it remains untrusted data. This evaluates access, provenance, and instruction boundaries separately.
Related concepts
References & further reading
Original explanations, with primary references for deeper study.
OWASP: RAG Security