Gracient.ai
Start a conversation
← Case studies
Privacy & Security · Reference build · July 4, 2026

Your RAG just showed the intern the HR files

A permission-aware retrieval build. The fix for RAG that leaks: enforce access control at the retrieval layer, before the model sees a thing, because vector search retrieves by similarity, and similarity does not know about permissions.

DomainPrivacy & Security
OutcomeAccess denied by default: unauthorized passages never reach the model, and every retrieval is logged.
Stack Identity-aware retrievalQuery-time ACL enforcementNamespace isolationVerifiable deletionRetrieval audit

An intern types a question into the company assistant: “What’s the parental-leave policy?” The bot answers helpfully, and three lines down it mentions that the last person to take leave, a named colleague, was out for eleven weeks on a salary the intern was never supposed to see.

Nobody was hacked. No password leaked. The demo worked exactly as it was built. The retriever did its one job, fetch the most relevant text, and the most relevant text happened to live in an HR file the intern had no right to open. The model, handed that text, summarized it like a good assistant. The bug was not in the model. It was in the missing wire between who is asking and what the index is allowed to hand back.

Why it leaks

Most retrieval-augmented generation (RAG, where the assistant answers by first searching your documents and pasting the matches into the model’s prompt) is secured in the wrong place. The login screen checks who may open the chatbot. Nothing checks which passages the search is allowed to return.

Those are two different questions, and usually only the first gets answered. “May this person ask?” is application-layer security, and most teams have it. “May this person see this passage?” is retrieval-layer security, and most teams skip it. So the retriever searches one big pool of everything, HR, finance, legal, board decks, by meaning alone.

And meaning is the trap. Vector search (finding text by how close its meaning is, using an embedding, a list of numbers that captures what a passage is about) ranks passages by similarity, not by permission. It is a confused deputy: it fetches the salary passage because it is the best match, with no idea the asker is an intern. Security has to move from the prompt down to the retrieval step.

How a permission-blind RAG leaks An intern asks a question. The vector store returns the top matches by similarity, and one of them is a restricted HR salary passage, because similarity ignores permissions. The model receives all of them and its answer leaks the salary. Similarity does not know about permissions. Intern asksleave policy? Vector store: top matches leave policyok benefits guideok HR: salariesrestricted, returned anyway Modelsummarizes it all Answer leaks the salaryto someone with no right to it IN RAG: the login checked who may ask. Nothing checked which passages the search may return.
The retriever did its job. It ranked by meaning and handed back the closest matches, one of which the asker had no right to read. No exploit required.

Three fixes, two of them wrong

When teams find the leak, they reach for one of three fixes. Two do not work.

Tell the model to behave. Add a line to the prompt: “Never reveal salaries to non-admins.” The most common fix, and the most dangerous. The model is not an access-control system; it is an untrusted reader of whatever you hand it. Once the salary passage is in its context, it is already exposed, and a single crafted question (prompt injection, smuggling instructions in disguised as ordinary input) walks straight around the rule.

Scrub the answer afterward. Let the model generate, then filter the output. Same flaw, one step later: the model already read the restricted text. You are wiping up a spill, not preventing it.

Filter after the search, before the model. Pull the top matches, drop the ones the user can’t see, pass the rest. Better, but it has a mean failure mode: if the best matches are all restricted, you drop them all and hand the model nothing, so it either shrugs or invents an answer, even when a perfectly good non-restricted passage sat one rank lower. And the engine still touched the restricted text on the way.

The only fix that holds is the one nobody starts with: decide what the user may see before you search, so the restricted passages never enter the ranking at all.

The three places you could enforce permissions A pipeline: caller, then search, then the model, then the answer. Gate A at login only checks who may ask. Gate C scrubs the answer, but the model has already seen the data. Only Gate B, a filter at retrieval time before the model, keeps restricted passages from ever reaching it. Only one gate is before the model. Gate A Loginwho may ask?needed, not enough Retrieval filterwhich passages?by verified identityGate B, the real boundary Modelsees the data Scrub answertoo latemodel already saw it Gate C the model sees everything to its left IN RAG: enforce at Gate B. Anything to the model's left, it has already read, so Gate C cannot un-leak it.
Everything left of the model, it has already read. That is why the retrieval filter is the only gate that enforces permissions, and why "tell the model" and "scrub the output" are theatre.

The build

Here is the reference architecture, the wire that should have been there.

  1. Ingest content and its permissions together. When you pull a document in, pull its access list at the same time (its ACL, the record of which people and groups may read it), and stamp every passage with it. A passage with no permissions attached is not public, it is invalid; don’t index it.
  2. Normalize identity. SharePoint, Google Drive, and Slack each name people differently. Map them all to one stable set of IDs, so “who is this, and what groups are they in” has a single answer. Use IDs, not email addresses or display names, which change and collide.
  3. Keep permissions fresh. The moment someone leaves the HR team, their access is revoked at the source. If your index only re-reads permissions on its nightly crawl, the assistant serves them HR files until morning. Sync changes continuously, so revocation lands in minutes, not overnight.
  4. Filter at query time, before the model. Resolve the caller’s identity and groups from a verified token (a tamper-proof digital ID the caller presents, that the server issues and checks, not something they can type or fake), and narrow the search to only the passages they may read, up front. Never trust the model or a system prompt to do this.
  5. Isolate tenants. If you serve multiple customers, give each its own namespace (a walled-off section of the index, the searchable store of all your passages), chosen from that same verified token, never from a value in the request that a caller could forge.
  6. Delete for real, and audit every retrieval. More on deletion below; the audit log records who retrieved which passages, so a compliance review finds a trail instead of a surprise.
The permission-aware retrieval build Ingest content plus its access list, normalize identity, and keep permissions fresh. At query time, filter by the caller's verified identity so only authorized passages pass. A boundary line marks that the model only ever sees authorized passages. Every retrieval is audited. The model only sees what the user may read. INGEST ONCE content + access liststamp every passage with its ACL normalize identityone stable ID per person and group keep ACLs freshrevocation lands in minutes ASK EACH TIME query-time filterby the caller's verifiedidentity and groups modelauthorized passages only auditwho saw what nothing unauthorized crosses this line IN RAG: authorization is deterministic and lives in the pipeline, never in the model's good intentions.
One invariant holds the whole design up: the model only ever sees passages the caller was already allowed to read. Everything else is implementation detail.

What a passage has to carry

The whole design rests on step one, and step one is where teams cut the corner. “Stamp its ACL on it” sounds trivial; doing it so the filter can be both correct and fast is not.

Every passage in the index carries more than its text and its vector. It carries the tenant it belongs to, the source document and system it came from, the exact principals (the specific users and groups) allowed to read it, the version of that access list, a sensitivity label, and its deletion state. The version and timestamp earn their keep later: they are how you prove a passage was authorized at the moment it was served, and how you catch one whose permissions have quietly drifted from the source. And the rule that saves you is blunt: a passage that arrives without permissions is not public, it is invalid. Index it anyway and you have built the leak by default.

Then identity, which is the part that actually bleeds. “The caller’s groups” is a tidy phrase hiding a mess: groups nest inside other groups, service accounts act on behalf of people, contractors and partners show up with identities your directory only half-owns, and the same human appears as three different IDs across SharePoint, Slack, and Drive. Key your permissions on email addresses or display names and you will eventually match the wrong person, because those change and get reused. You key on stable internal IDs, and you resolve the full nested group membership at the source, not a flattened snapshot. Get this one wrong and every filter downstream is faithfully, efficiently enforcing the wrong answer.

The hard part

That “filter before you search” step is where the real engineering lives, because filtering and fast search pull against each other.

Vector search is quick because of an index called HNSW (a graph you hop through to find near matches without scanning everything). A strict permission filter breaks its assumptions. Pre-filtering, narrowing to the allowed passages first and then searching, is the secure choice: the user only ever ranks against what they can see. But if only a sliver of the index passes the filter, the graph has little left to hop through and the search slows toward a brute-force scan. Post-filtering, searching everything and then dropping the disallowed, keeps the speed but can hand back too few results, or none, exactly when the best matches were all restricted.

The secure-versus-fast tradeoff, and how partition-by-role breaks it A plane with security on the vertical axis and speed on the horizontal. Pre-filtering is secure but slow, top-left. Post-filtering is fast but leaks and misses results, bottom-right. Those two sit on the usual tradeoff line. Partitioning the index by role sits top-right, both secure and fast, because each query searches a smaller already-authorized slice, buying back the speed the pre-filter lost. Secure or fast, and how to get both. more secureless secure slowerfaster the usual tradeoff: pick one same security, speed recovered Pre-filtersecure, slow Partition by rolesecure and fast Post-filterfast, leaks and misses IN RAG: partition-by-role breaks the tradeoff by searching a smaller, already-authorized slice.
This is the systems work under "the assistant respects permissions." It is also where a demo that looked fine quietly falls over once the document set is big and the filters are strict.

There is a second axis: when you bind permissions. Early binding stamps the access list onto each passage at ingest, fast to query, but every permission change needs a re-index, and a stale snapshot leaks. Late binding checks a live authorization service at query time, always current, but slower and only as reliable as that service. Neither is free, and choosing between them is the actual design decision hiding behind the phrase “it respects permissions.”

Two failures that pass every casual test

The stale-permission window. Between a revocation at the source and its arrival in your index, the assistant is serving data to someone who just lost access. The first question a serious buyer asks is exactly this: if I remove an intern from the HR group right now, how long can they still retrieve HR passages? That number is your revocation target (a service-level objective, or SLO), and it is a design choice, not luck. A nightly re-crawl makes it as much as a day. Change notifications from the source (webhooks) plus delta-sync, re-reading only what changed the moment it changes, pull it down to minutes or seconds, though that sync path is brittle and has to be monitored like any other production dependency. Whatever you choose, you own it as a number and you test it. And your usual alarms will not help: to a security monitor, a retrieval looks like a normal request from an authorized app, so nothing fires. Teams routinely find the gap during an audit, weeks after a contractor should have been cut off.

Deletion that doesn’t delete. When a customer invokes their right to be forgotten, deleting the source record is easy. But their text now lives as embeddings, and an embedding is not a one-way hash (a scramble you cannot reverse): research has shown you can reconstruct much of the original text from the numbers alone. Hiding a vector from search results is a curtain, not an erasure. Real deletion is a pipeline, because the data has scattered: the passage, its vector, the search index that holds it, the caches that speed up search, the saved prompts and answers, the logs, and the backups. Miss one and the data is still there, waiting for a storage-layer attacker or the next regulator. So verifiable deletion means hard-removing the vector and everything derived from it, keeping an evidence trail of what was purged and when, and, where scrubbing every copy is impractical, encrypting each record up front so you can destroy the key instead of chasing the copies.

The cleaner move is to never store the sensitive text at all. Mask before embedding: swap names and account numbers for placeholders before the text is turned into vectors, keep the mapping in a guarded store, and re-insert the real values only after the model has answered, so the sensitive tokens never enter the index in the first place.

Does it hold?

The difference between a build and a blog post is that a build can be tested, and this one has a specific, unglamorous test suite that a regulated buyer will run whether you offer it or not.

The revocation test. Remove a user from a group at the source, start a stopwatch, and keep asking a question whose only good answer lives in a now-forbidden document. The answer has to go dark within your stated SLO. If it doesn’t, your freshness story is fiction.

The cross-tenant test. Take one customer’s token and a question whose best match lives in another customer’s data, and confirm you get nothing back, not a filtered mention, not a count, not a hint that the document exists. Then try to point the request at the other tenant’s namespace by editing it directly; it must be ignored, because the tenant is read from the verified token, never from the request body.

The audit row. For every retrieval, one line that answers the question a regulator actually asks: on this date, this caller, in this tenant, retrieved these passages from these source documents, under this version of their permissions, and produced this answer. Hash the identities and queries where you can, but keep enough to reconstruct an incident. If you cannot produce that line, you cannot prove the system was ever secure; you can only hope it was.

And then there is the attack version

Everything so far assumes an honest user who simply gets the wrong documents back. There is a darker case, where the document is the attacker. In June 2025, EchoLeak (CVE-2025-32711, the industry’s ID for a specific flaw) showed it in production: a booby-trapped email sat in a mailbox, and when the user later asked Microsoft 365 Copilot a routine question, the assistant pulled that email in as context, obeyed the hidden instructions inside it, and shipped internal data out through a rigged image link. No click, no exploit in the usual sense. Aim Labs, who found it, called it the first zero-click theft from a production AI assistant; Microsoft patched it.

The point for this build is narrow but sharp. Retrieval authorization has to fail closed even here, so a poisoned document can never widen what the caller is allowed to pull back, and it has to be paired with limits on what the model may send out, because retrieved text and instructions still arrive through the same channel with no reliable way yet to tell them apart. It is the lesson the database world learned the hard way with injection attacks, sneaking commands in where a system expected plain data, now reopened one layer up.

So the discipline is old even if the setting is new. Authenticate the person, but authorize every retrieval. Sync permissions from the system that actually owns them. Delete so it stays deleted, and be able to prove it. This is the enforcement half of running private data through an LLM without losing control, and it leans on the same messy reality that no model outruns a silo: the permissions you need live scattered across the systems the documents came from. And never let the model see what the user can’t, because by the time the model has seen it, it is already out.