RAG Citation Accuracy: Verifying Answers Against Retrieved Evidence

A RAG answer can carry a real source link that does not support the sentence next to it. The retriever found a relevant document and the model wrote fluent text, yet the reader cannot verify the claim. This guide covers a verification layer that sits behind your existing retriever and generator: exact source spans, citations on atomic claims, separate support and coverage checks, and a counterfactual probe for the failure most checks miss, a correct citation the model never actually used.

Why a citation marker is not evidence

Most RAG pipelines treat citations as formatting. They pass retrieved chunks to a model, ask for bracketed source numbers, and replace those numbers with links. That proves the model emitted a valid marker. It does not prove the referenced text entails the claim.

Several failures can hide behind a polished answer:

• A sentence contains two factual claims, but its citation supports only one.
• The source discusses the same subject without supporting the stated number, date, exception, or conclusion.
• The cited chunk is valid, but stale page offsets send the reader to a different page.
• The answer combines details from two chunks and cites only the first.
• A factual sentence has no citation because the model treated it as common knowledge.
• The citation points to a retrieved result that the final answer did not actually use.

Retrieval quality and answer support are separate checks. Microsoft's RAG evaluator guidance separates retrieval measures from groundedness and response completeness. A relevant chunk can still be attached to an unsupported answer. A grounded answer can also omit an important part of the requested information.

The ALCE citation-evaluation paper makes a similar separation for cited answers. It evaluates citation correctness and citation completeness as different dimensions. Correctness asks whether a cited source supports its claim. Completeness asks whether every claim that needs evidence has a citation.

A correct citation can still be fake

Support checks answer one question: does the cited text entail the claim? There is a second question they cannot answer. Did the model use that text, or did it answer from memory and then attach a passage that happens to agree? Wallat et al. call this post-rationalization and separate citation correctness from citation faithfulness. Their paper reports that current attributed answers often lack faithfulness, in up to 57 percent of citations in their experiments. The authors do not claim this rate holds for every model or corpus, so measure your own.

An entailment check passes a post-rationalized citation, because the passage really does support the sentence. Output inspection cannot tell the two cases apart. A counterfactual probe can: take an answer you already verified, edit the cited passage so it states a different value, and rerun generation. If the answer keeps the old value, the model was answering from memory and the citation is decoration. Run this on a sample of high-risk answers in an offline test, not on live traffic. It costs one extra generation per probe, and it is the only check in this guide that tests whether the evidence drove the answer.

Define the contract before changing prompts

Set the application's acceptance rules before changing the system prompt.

A practical contract has five rules:

1. Every externally verifiable factual claim must map to one or more retrieved source spans.
2. Every citation must resolve to a source that was available to the generator for that answer.
3. The cited span must support the whole attached claim, including qualifiers and values.
4. Source identity and location must come from ingestion metadata, not model-generated text.
5. An unresolved or unsupported claim must be removed, rewritten from available evidence, or sent to review.

The contract covers externally verifiable facts, not opinions, interface instructions, or clearly labeled recommendations. For example, "Select a review threshold based on your risk tolerance" is engineering advice. "The policy permits refunds for 30 days" is a factual claim and needs exact policy evidence.

Decide what "support" means for your domain. A support assistant may accept a direct statement from a current help article. A contract assistant may require the same clause, document version, jurisdiction, and effective date. A policy assistant may reject an older document even when its wording matches. The verifier needs those rules as data, not hidden assumptions inside a model prompt.

Preserve evidence coordinates during ingestion

Citation verification starts before retrieval. Each chunk needs enough immutable metadata to reconstruct the evidence a user should see.

Store at least these fields:

• document_id: stable identity for the source document
• document_version: immutable source version or content hash
• chunk_id: stable identity within that version
• page_number: reader-facing page when the format has pages
• source_start and source_end: offsets in canonical extracted text
• source_url: canonical reader-facing location
• section_path: heading hierarchy or equivalent structural path
• text: the exact indexed span

Keep the canonical extracted text for each immutable document version. Do not calculate reader links later from a newly parsed copy. If a parser update changes whitespace, reading order, or overlap behavior, old offsets no longer describe the new text.

The failure has occurred in practice. A reported FinSight-AI failure describes overlapping PDF chunks retaining stale character offsets and producing wrong citation pages. This is an author report, not a universal product defect. Its failure shape still belongs in the test suite because a correct answer can have broken attribution when chunk coordinates drift.

Validate coordinates as part of ingestion. Resolve every chunk back to the stored source version and assert that the referenced span matches the indexed text after your documented normalization. Quarantine chunks that fail. Rebuilding an index should create a new version rather than mutating the coordinates beneath prior answer records.

Generate structured claims, not citation-shaped prose

The generator should return a claim graph or another structured representation before the application renders prose. Each factual claim carries source references selected from the retrieved set.

{
  "claims": [
    {
      "claim_id": "c1",
      "text": "Expense reports must be submitted within 14 days.",
      "evidence": [
        {
          "chunk_id": "policy-v7-p12-c2",
          "quote": "Submit all expense reports within 14 calendar days",
          "source_start": 418,
          "source_end": 473
        }
      ]
    }
  ]
}

The model may propose the evidence mapping, but the application owns validation. Check that every chunk_id came from the retrieved context for this request. Resolve offsets against stored text. Verify that the quoted evidence is an exact or explicitly normalized slice. Never let the model supply a URL, page number, or document identity that the retrieval layer did not provide.

OpenAI file search returns file-citation annotations that an application can resolve into source labels. LlamaIndex's CitationQueryEngine example demonstrates generating granular citation sources from retrieved chunks and controlling citation chunk size. Both features produce useful references. The application must still test whether the referenced span supports the attached claim.

Keep claims atomic. "The plan renews annually and can be cancelled within 30 days" contains two assertions. Split them unless one evidence span clearly supports both. Atomic claims make failures repairable: the system can remove or regenerate one unsupported statement without discarding the entire answer.

Verify support and coverage separately

Run two passes after generation and before rendering.

The support pass evaluates each claim and its cited evidence. Start with deterministic checks:

• The source exists in the retrieved set.
• The document version still exists and is allowed for the user.
• The source span resolves and matches stored text.
• Required qualifiers such as date, region, product, or policy version agree.
• Numeric values and named entities in the claim appear in, or are directly entailed by, the evidence.

Then use a constrained entailment check for claims that pass structural validation. Give the evaluator one atomic claim and only its proposed evidence. Ask for a small result schema such as supported, unsupported, or insufficient, plus the unsupported fragment. Do not give it the full retrieved context, because a claim supported somewhere else should not rescue a wrong citation.

Ragas defines faithfulness by identifying answer claims and checking whether each can be inferred from the retrieved context. That pattern is useful, but citation verification narrows the context further: test each claim against the evidence attached to that citation, not against every chunk the retriever happened to return.

The coverage pass works in the other direction. Extract factual claims from the rendered answer and verify that each required claim has at least one accepted citation. This catches uncited facts even when every existing citation is correct.

Track both rates:

• Citation support rate: accepted citations divided by all rendered citations
• Claim coverage rate: supported cited claims divided by all claims requiring evidence

Do not merge them into one score at the release gate. A system can achieve perfect support by citing only one safe sentence while leaving the rest uncited. It can achieve complete coverage by attaching weak sources everywhere. Set separate thresholds and retain the raw failure reasons.

Choose a failure policy by consequence

A verifier needs a deterministic disposition. Otherwise the application logs a warning and publishes the same bad answer.

Use these decision rules:

• If a citation cannot resolve, block the claim. This is a data-integrity failure.
• If evidence contradicts the claim, block the whole answer and record the conflict.
• If evidence is insufficient, remove the claim or regenerate it using only accepted spans.
• If a required qualifier is missing, ask a clarifying question or state the narrower supported fact.
• If no source supports the user's request, abstain instead of producing an uncited answer.
• If the workflow can cause a business action, never let a generated citation authorize that action. Apply normal application policy and approval controls separately.

For low-risk internal search, partial answers may be acceptable when unsupported claims are removed and the result says what could not be verified. For legal, financial, medical, or operational decisions, route failed answers to a reviewer with the claim, proposed evidence, source version, and failure reason visible together.

Avoid an automatic retry loop that simply asks the same model to "fix citations." Retry once with the rejected claims and accepted evidence made explicit. If verification still fails, stop. Repeated generation can change wording without improving support and can hide a systematic retrieval or parsing defect.

Test the complete attribution path

Build fixtures from real failure shapes, not only ideal question and answer pairs. Each fixture should freeze the source versions, retrieved chunks, expected claims, acceptable evidence spans, and expected disposition.

Include at least these cases:

• One citation fully supports one atomic claim.
• One sentence contains two claims but evidence supports only one.
• A correct fact points to the wrong page after chunk overlap.
• Two source versions disagree and only one is current.
• The answer cites a relevant section that lacks the stated number.
• A claim needs two independent chunks for full support.
• Every citation is valid, but one factual sentence is uncited.
• Retrieval returns no supporting evidence and the system must abstain.
• A user lacks permission to open an otherwise supporting source.
• The citation target is deleted after generation but before display.

Run the fixtures after parser, chunking, embedding, retriever, prompt, model, or citation-renderer changes. Store verifier outcomes with the answer trace so an engineer can distinguish retrieval misses, offset corruption, unsupported generation, stale sources, and rendering defects.

Test the reader experience too. Clicking a citation should open the correct document version and place the supporting passage in view. If the original format cannot support a stable deep link, show a verified excerpt with document identity and page information rather than pretending a generic document link is precise.

When claim-level verification is too much

Claim-level checking adds an entailment call per claim and a second pass for coverage. For a low-risk help widget that is often the wrong first investment. For scale, Liu et al. audited four commercial generative search engines and found that on average only 51.5 percent of generated sentences were fully supported by their citations, and only 74.5 percent of citations supported the sentence they were attached to (Evaluating Verifiability in Generative Search Engines). Cited answers fail often even in shipped products. A cheaper first contract is a state derived by code, not by a model: grounded when at least one evidence chunk was selected into the prompt, insufficient evidence when none was, and retrieval unavailable when the vector search errored. Expose only a citation-safe projection of each source (label, id, title, URL, effective dates) and never raw chunk text or scores.

That layer gives the caller a reliable "answered with evidence" signal and stops a retrieval outage from looking like a grounded answer. It does not prove any individual sentence is supported. Start there if your risk is low, and add the claim-level checks above where a wrong sentence has a cost: refunds, legal terms, medical or financial guidance.

Common mistakes that keep bad citations alive

Grading groundedness against all retrieved context marks a claim supported even when its visible citation points elsewhere. Grade the actual attachment. Page numbers belong to ingestion metadata, so models should reference source handles and application code should render locations. Keep support, coverage, resolution, version, and permission failures as separate counters, because each has a different owner. A citation can inform a user but it cannot authorize a tool call: business actions still need server-side authorization and approval.

Put the verifier in the release gate

Start with a shadow pass on current traffic. Record claims, source mappings, support decisions, and coverage without changing answers. Review failures by category and fix coordinate corruption before tuning model prompts. Then enforce blocking for unresolved citations and unsupported high-risk claims. Add partial-answer behavior only after the failure reason is visible to the reader.

Take 20 recent answers, split them into atomic claims, and resolve every citation to a stored span. Turn each failure into a regression fixture before your next parser or model release. Then pick five high-risk answers and run the counterfactual probe.

References

1. Microsoft RAG evaluators supports the separation of retrieval, groundedness, and response-completeness checks.
2. Ragas faithfulness metric supports claim extraction and evidence-based faithfulness evaluation.
3. LlamaIndex CitationQueryEngine supports granular citation generation from retrieved chunks.
4. OpenAI file search supports the use of application-resolved file-citation annotations.
5. ALCE citation-evaluation paper supports separate citation-correctness and citation-completeness evaluation.
6. Wallat et al., Correctness is not Faithfulness in RAG Attributions supports the correctness-versus-faithfulness distinction and the 57 percent upper figure.
7. FinSight-AI issue 1108 provides the practitioner-reported wrong-page failure caused by chunk and offset misalignment.
8. Liu et al., Evaluating Verifiability in Generative Search Engines supports the 51.5 percent fully supported and 74.5 percent citation-precision averages across four commercial engines.

---