Preventing RAG Data Poisoning in Internal Knowledge Bases

RAG data poisoning starts before the model call. A malicious or compromised document lands in a shared drive, wiki, mailbox or upload queue, gets chunked and embedded, and then appears as trusted evidence for an ordinary employee question. The pipeline works exactly as designed while it repeats the attacker's text. The defense is a trust contract at ingestion and at retrieval, with provenance and revocation behind it, because no detector will catch every poisoned document.

Why a normal RAG pipeline preserves the attack

An ingestion job rewards availability and freshness: discover files, extract text, chunk, embed, advance a cursor. Those steps establish that content can be retrieved and say nothing about whether it deserves authority. Document text is an input controlled by someone outside your runtime. Microsoft classes malicious instructions in third-party documents and emails as document attacks, in its Prompt Shields documentation, which is one form of indirect prompt injection. Poisoning is wider than a visible "ignore the policy" line. A document can falsify a policy date, swap an account number, or carry text written to rank for a target query. A fast incremental index only distributes a poisoned version sooner, and a layout-aware parser preserves hostile text perfectly.

A handful of texts is enough

It is tempting to think a few bad documents in a large corpus are statistical noise. The PoisonedRAG paper reports a 90% attack success rate from injecting five malicious texts per target question into a knowledge database with millions of texts, and says the defenses it evaluated were insufficient. Two consequences follow. First, corpus-level statistics such as the share of new documents or an embedding outlier score will not reveal an attack aimed at one question. Second, test by question: take the ten questions where a wrong answer costs most (expense limits, bank details, approval thresholds, security procedures) and make those answers depend only on sources that pass the trust contract below. Those are the questions an attacker would pick.

Whoever can edit a source can steer your answers. Before building any classifier, list who has write access to each collection the assistant reads, and compare that with who you would trust to publish policy. The gap between the two lists is your real attack surface, and shrinking it is cheaper than detecting what comes through it.

The ingestion trust contract

Treat the knowledge base as a security-sensitive data product. Each retrievable chunk should satisfy six statements before it reaches the active index:

• An authorized connector or uploader introduced the source.
• An immutable source identity and version are on record.
• Extraction and security checks produced a recorded decision.
• The source has an explicit trust class and sensitivity class.
• Retrieval policy can enforce those classes for the current user and use case.
• The source can be revoked without a full rebuild.

When detection misses a novel attack, provenance, authorization, retrieval constraints and revocation still limit the damage. OWASP LLM04:2025 and OWASP LLM08:2025 are the threat models. NIST's AI 100-2 E2025 taxonomy gives you shared vocabulary for incident reports, so ingestion events are not filed as vague "bad answer" bugs.

Control the sequence before embedding

Start with the connector. Keep an allowlist of approved source systems, collections, owners and service identities, because a SharePoint site approved for published policies does not approve every personal folder in the tenant. For direct uploads, authenticate the uploader, bind the file to an approved collection, and record owner, audience and reason. These values are never read from document text. Use a separate ingestion identity per collection so one compromised connector cannot rewrite every domain.

Give every source and version a stable identity before parsing, with a stable source ID and an immutable version ID, since a content hash alone changes with every legitimate edit:

source_id: policy-handbook
version_id: policy-handbook-2026-08-20-01
source_uri: sharepoint://published-hr/policy-handbook.pdf
connector_id: sharepoint-published-hr
owner: people-operations
content_hash: sha256-of-original-bytes
trust_class: approved-policy
sensitivity: internal
review_state: quarantined
supersedes: policy-handbook-2026-07-10-02

Carry source ID, version ID, hash, trust class and review state onto every chunk, and keep the original bytes in an evidence store with tighter write access than the source. New or changed versions go to a quarantine index, not the active one. Run file checks first (type, size, malware, archive limits, extraction confidence), then inspect the extracted representation, including hidden text and OCR layers that can differ from what a reviewer sees. A prompt-attack classifier is one signal among several, together with source trust, size of the change, uploader history and collection sensitivity. Auto-admit a small edit from an approved publishing workflow when no high-risk signal fires. Require review for a new source, a changed owner, a large semantic change, hidden text or conflicting policy statements. Reject executables, unsupported containers and missing provenance. Keep the previous approved version active until its replacement passes, so a failed review is not an outage.

Scope applies to anything a user can write into, not only formal uploads. PromptArmor's August 2024 report on Slack AI showed the pattern: an attacker who was the only member of a new public channel posted an instruction there, and when a victim later asked about a secret stored in a private channel, retrieval pulled both messages into one context and the model rendered a link that carried the secret out. Slack's answer, quoted in the report, was that public-channel messages can be searched and viewed by all workspace members whether or not they have joined, which it called intended behavior. Retrieval inherited a visibility rule that was fine for people who read results one at a time and unsafe for a model that merges them. Decide per source whether being readable by a user is the same as being safe to retrieve into that user's prompt, and give low-trust, anyone-can-write sources their own index.

Enforce trust at retrieval

Admission is not the last control. Filter by the user's authorization, the workflow's purpose, source sensitivity, review state and trust class before vector or lexical ranking, because content that enters the prompt can no longer be reliably ignored. Rank trust separately from similarity: a team-folder comment that repeats the query words should not outrank an approved policy for a policy question. Pick the eligible trust classes first, then rank within them. Tie citations to version IDs so the audit record keeps the exact version behind each answer.

Keep evidence apart from authority. Retrieved text can supply facts and never grants permission. Tool calls go through policy based on the authenticated user, the workflow and current business state. Mark retrieved text as data on its way in, and use more than a delimiter. Microsoft's spotlighting paper transforms the untrusted text, for example by interleaving a marker character between its words, so the model can tell it apart from instructions, and reports attack success on GPT-family models falling from above 50% to below 2%. A plain tag pair that the document can imitate or close does not give that separation.

An example: the HR assistant

An employee can edit a team document but not the published HR policies. They add a paragraph saying expense limits changed, with a hidden line telling the assistant to ignore its policy source. A weak pipeline indexes the edit, retrieves it for "What is the travel expense limit?" and lets similarity pick the winner. Under the contract, the shared-drive connector tags it as a team working copy, the retrieval policy for policy questions allows only approved-policy sources, and the edit stays searchable in team search without answering an authoritative question. Several independent boundaries would have to fail before it became authority.

Plan revocation before an incident

Deleting the source file is not enough, since copies sit in chunk tables, vector and keyword indexes, response caches, evaluation sets and generated summaries. Build revocation around the version ID: mark it denied at query time immediately, remove or tombstone derived chunks and embeddings, invalidate caches and artifacts citing it, find the answers and tool proposals it influenced, restore the last approved version, and replay representative queries to confirm the poisoned evidence is gone. If deleting from one store fails, keep the version denied at query time and retry the cleanup.

Verify with fixtures

Test the full ingestion and retrieval path with poison fixtures: a visible instruction in a valid policy, hidden text or an OCR layer that conflicts with the visible page, a writable team document that repeats a target query, a forged "new policy" from an unauthorized user, a legitimate large revision that needs review without being lost, a poisoned version followed by restoring the prior one, two users with different permissions asking the same question, and a retrieved document that asks the workflow to call a consequential tool. Assert the admission decision, active version, eligible trust classes, retrieved chunk IDs, citations, tool-policy result and cleanup. Measure false positives too. If reviewers must approve every routine edit, they will find ways around the gate, so tune auto-admission for well-controlled publishing paths.

Start with one sensitive collection. Inventory its writers, define two or three trust classes, add version metadata, route new versions through quarantine, require review_state=approved at retrieval, and implement one working revocation before you add more collections.

References

• OWASP LLM04:2025 Data and Model Poisoning for the poisoning threat model and controls.
• OWASP LLM08:2025 Vector and Embedding Weaknesses for vector-store and embedding risks relevant to RAG.
• Microsoft Prompt Shields for the distinction between user prompt attacks and malicious instructions in third-party documents.
• NIST AI 100-2 E2025 for the adversarial machine learning taxonomy.
• PoisonedRAG for knowledge-corruption attacks on RAG, including the five-texts, 90% success result.
• PromptArmor, Data exfiltration from Slack AI via indirect prompt injection for the public-channel poisoning chain and Slack's response.
• Hines et al., Defending Against Indirect Prompt Injection Attacks With Spotlighting for datamarking and the drop from above 50% to below 2% attack success.

---