RAG access control: keeping retrieval within each user's permissions

Why the prompt is the wrong place for this

A common first attempt is to retrieve from the whole index and add an instruction: "only use documents this user may see". That fails for a simple reason. Once a passage is in the prompt, it has already been disclosed to a component that can repeat it. The model has no reliable knowledge of your access rules, and a cleverly worded question or a planted instruction in a document can change what it does. LLM guardrails and prompt injection defence explains why such instructions cannot be treated as a security boundary.

So the rule is that the model should only ever receive text the current user could already open in the source system. If that holds, a manipulated model has nothing extra to reveal. OWASP lists this class of problem under LLM08:2025 Vector and Embedding Weaknesses, including data from one group surfacing in another group's queries when they share a vector store, and it recommends permission-aware stores and logical partitioning.

Decide what model of access you are copying

An index is a copy of your content, but it does not automatically copy the rules that protected it. Before designing anything, write down how access works in each source:

• Tenant separation. In a multi-customer product, each customer sees only their own data. The rule is simple and the cost of failure is severe.
• Role or group access. Finance can read one space and engineering another. Most internal knowledge bases work roughly like this.
• Per-document sharing. Individual files are shared with named people, as in many drive and wiki products, often with inheritance from folders.
• Attribute rules. Access depends on values such as region, clearance or contract, and can be difficult to reduce to a list of names.

Sources often differ, so a single assistant may need all of these at once. If a source's rules cannot be represented or queried, leave it out of the first version rather than approximating them. How to scope an AI project covers raising this question early, while it is still cheap to answer.

Three ways to enforce permissions at query time

Separate indexes or partitions

Each tenant gets its own collection, namespace or table partition. Mistakes are harder to make, because a query can only touch one partition, and deleting a tenant is straightforward. The weakness is scale and flexibility: it suits a tenant boundary, but not documents shared across many overlapping groups. Vendors' advice also varies. Qdrant's multitenancy guide, for example, recommends one collection partitioned by payload for most cases, and multiple collections only for a limited number of tenants needing strict isolation. Check the guidance for your own store.

Early binding: permissions stored in the index

Each chunk carries metadata listing the groups or users allowed to read it, and the query filters on the caller's identities. This is fast and works well with ranking, because only permitted chunks compete for the top places. Its weakness is staleness: the index reflects permissions as they were at the last sync.

The filter is only as trustworthy as the code that builds it. Microsoft's security filter pattern for Azure AI Search makes this explicit: the principal is just a string in a filter expression, with no authentication or authorisation behind it. The same document points to built-in access control list support as the first option where it can be used.

Late binding: checking with the source

The system retrieves candidates, then asks the source system whether this user may read each one. This gives the freshest answer, but it adds latency and load, and it can leave too few results if most candidates are rejected. In practice it pairs well with early binding: filter in the index for speed, then recheck the final passages for sensitive sources.

Filter during the search, not after it

Running a top-20 search and then dropping forbidden results looks safe, and it does prevent the leak. But if 19 of the 20 are forbidden, the user gets a thin answer even though permitted documents exist further down. This failure is easy to miss, because it looks like poor retrieval rather than an access problem.

Some approximate indexes behave this way even when the filter is written into the query. The pgvector README states that with approximate indexes, filtering is applied after the index is scanned. Its example is a condition matching 10% of rows, which with HNSW and the default hnsw.ef_search of 40 returns only 4 rows on average. It suggests iterative index scans, partial indexes or partitioning, depending on the filter. Test recall for users with narrow access, not only for administrators, whose results can look perfect.

Here is the shape of the query path. The identity comes from the verified session and never from the request body or the model's output:

# Illustrative only. Helpers and field names are placeholders.
def retrieve_for_user(session, question):
    principals = resolve_principals(session.verified_user_id)
    if not principals:
        return []

    chunks = vector_search(
        embed(question),
        limit=20,
        filter={"tenant_id": session.tenant_id,
                "allowed_principals": {"any_of": principals}},
    )
    # Fail closed: a chunk with no access data is never returned.
    return [c for c in chunks if c.metadata.get("allowed_principals")]

This sample is illustrative: the helpers and filter syntax are placeholders and will differ by store. It fails closed, returning nothing when the identity or the access data is missing. A retrieval tool given to an agent needs the same treatment, with the identity bound in code rather than offered as an argument the model can fill in. Agentic RAG explained shows how such tools fit together.

Using row-level security as a second layer

If your vectors live in PostgreSQL, row-level security can enforce the rule below the application, so a forgotten filter does not become a leak:

-- Illustrative only. Table, column and setting names are placeholders.
ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON chunks
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

This SQL is illustrative, and needs testing against your schema. Read the PostgreSQL documentation closely: superusers and roles with BYPASSRLS always bypass row security, and table owners normally do too unless you use FORCE ROW LEVEL SECURITY. An application that connects as the table owner may therefore have no protection at all. Connect as a restricted role. With connection pooling, also make sure the tenant setting cannot carry over from one request to the next.

Keeping permissions in sync

A person leaves a team and a file is unshared, yet the index still says otherwise. Plan for this directly:

• Treat permission changes as their own events. Updating access metadata should not require re-embedding the content, and it may need to run far more often than a content sync.
• Agree a maximum lag. Ask the content owner how long a revoked permission may remain effective, and use late binding where the answer is "not at all".
• Store groups, and resolve members at query time. This means a change in team membership takes effect without touching any chunks.
• Handle inheritance. Flatten folder or space permissions on to each chunk, and recompute when a parent changes.
• Delete properly. Remove every chunk, summary and cached result derived from a deleted document, and confirm that the deletion took effect.
• Alert on sync failures. A sync that fails silently turns into wrong permissions that nobody notices.

Close the side channels

Filtering retrieval handles the main path, but restricted content can still escape by other routes:

• Caches. An answer cached for a director must not be reused for an intern. Scope cache keys by permissions, as described in LLM caching strategies.
• Conversation history. Shared threads can expose answers built from documents the new reader cannot open.
• Derived content. A summary spanning several documents needs permissions no broader than the most restrictive source, or it should stay out of shared indexes.
• Long-term memory. A fact remembered from one user's session can resurface in another's unless memory is scoped.
• Logs and traces. Prompts usually contain retrieved text, so tracing tools can give engineers access they would not otherwise have. See LLM observability and tracing.
• Existence signals. Citations, titles, result counts and suggested questions can reveal that a confidential project exists, even if no content is shown. Filter them too.
• Index access. Embeddings derive from the source text, so give the vector store the same protection as the documents themselves.

Test it as a security feature

Ordinary quality testing will not find these bugs, because it usually runs under a single account. Add tests such as these:

1. Seed marker documents. Plant unique strings in restricted test documents, then run queries as users who lack access and assert that the markers never appear in retrieved chunks, answers, citations or logs.
2. Test retrieval directly. Check the retrieved set rather than only the final answer, since a model may happen not to mention a passage it received.
3. Test revocation. Remove access, wait for your agreed lag, and confirm the content has gone.
4. Test empty states. A missing identity, group lookup failure or chunk with no metadata should return nothing.
5. Test tampering. Try changing identifiers in API requests, and asking the assistant to act as another user.

Run these on every change to ingestion or retrieval, and log which principal saw which documents so that you can investigate an incident. Add the recall checks for narrow-access users to the wider evaluation described in how to evaluate a RAG system.

A sensible starting point

Start with a tenant boundary enforced by the data layer, plus group metadata on each chunk, filtered during the search and failing closed. Add a recheck against the source for your most sensitive content. If you would like help designing this, the RAG development page describes what I can help with.

Frequently asked questions

Can a system prompt enforce permissions in RAG?

No. A prompt can shape behaviour, but it is not a security control, and any text placed in it has already reached the model. Enforce permissions before retrieval results are assembled, in code or in the data layer.

Should each customer have a separate vector index?

Not necessarily. Separate indexes give strong isolation but can be costly with many tenants, while a shared index relies on a correct filter. Check your store's guidance and weigh it against the sensitivity of the data.

What is the difference between early and late binding?

Early binding stores permissions with each chunk and filters during the search, which is fast but can lag behind the source. Late binding asks the source system at query time, which is fresher but slower. Many systems combine them.

How quickly must revoked access take effect?

That is a business and compliance decision, so agree it with the content owner. Design the sync to meet that limit, monitor for failures and use a live check against the source for content where no delay is acceptable.