RAG Document Parsing: Handling Complex PDFs and Tables

RAG document parsing can fail long before retrieval or generation. A parser returns fluent text while it interleaves two columns, separates a table value from its headers, or skips an image-only page, and the job still finishes and writes embeddings. Later the answer cites a number from the wrong row. A bigger chunk window does not fix this. Put a parsing gate before chunking: route each page to a suitable extraction method, keep structure and source locations, reject suspicious output, and test with fixtures built from your own documents.

Why clean extracted text can still be wrong

PDF is a page-description format and makes no promise that text is stored in reading order. A page may hold individually positioned characters, images, OCR layers, forms and tables, and extracting every visible token does not rebuild the relationships between them. The damage appears as interleaved columns, a table whose header, row label and value land in different chunks, a scan with no usable text layer, or headings and captions flattened into plain paragraphs. These are parsing failures, and checking answers after generation makes them expensive to find, because the bad answer sits several stages from the damaged source.

The cost is measurable. The OHRBench study, OCR Hinders RAG, separates OCR noise into semantic noise and formatting noise, shows a trend between the degree of noise and RAG performance, and concludes that none of the OCR solutions it evaluated is competent for building high-quality RAG knowledge bases. Treat extraction quality as a variable you measure, the same as embedding choice.

Consider not parsing the page at all

For visually dense documents such as slide decks, chart-heavy reports and forms, text extraction may be the wrong first step. The ColPali paper, Efficient Document Retrieval with Vision Language Models, embeds images of document pages directly and introduces the ViDoRe benchmark for page-level retrieval. Its authors report that this largely outperforms modern text-based document retrieval pipelines and is simpler and faster. The abstract gives no figures, and results on your documents will depend on their layout and language. The trade-offs are practical: page-image retrieval gives you no exact-text matching for identifiers, a vision model has to read the retrieved page at answer time, and the index is larger. A sensible pilot is a hybrid, with the text pipeline for born-digital contracts and invoices and page-image retrieval for the deck-and-chart class. Run your fixture queries against both and let the numbers pick.

Define a parsed-document contract

Give every parser the same output schema instead of accepting arbitrary Markdown. Each element carries: document ID and source version, page and sequence number, type (title, heading, paragraph, list item, table, caption, image description), normalized text, a source locator (coordinates or span) where the parser provides one, the parent heading path, extraction method and parser version, warnings or confidence, and a content hash. The Microsoft Document Intelligence layout model returns structure across PDF, image and Office formats, and Docling exposes layout, reading order, tables, formulas, images and OCR in a structured representation. Both can fill this schema, which keeps the rest of the pipeline independent of any one library. Keep the original file immutable and link every element to it, so a citation can name document, version, page and span, and a parser upgrade cannot silently rewrite history.

Route by page

A .pdf suffix says little about difficulty. One file can have clean digital text, another can be a scan, and a third can mix both, so inspect pages and route from what you observe: whether a text layer exists, characters per page, image coverage, available font and coordinate data, signs of columns, detected tables, replacement characters, and large gaps between rendered and extracted content. Use native extraction when it preserves order, layout analysis for columns, forms and tables, and OCR for pages without a dependable text layer. The Unstructured partitioning documentation frames PDF strategies (fast, hi_res, ocr_only) the same way, as choices with different costs and failure modes and not as quality tiers where the most expensive always wins.

inspect(document)
for page in document.pages:
    if page.native_text_is_complete and page.layout_is_simple:
        elements = native_extract(page)
    elif page.needs_layout_reconstruction:
        elements = layout_extract(page)
    else:
        elements = ocr_extract(page)
    validate_page(page, elements)
    persist(elements, parser_name, parser_version)

Store the chosen route and the reason next to the output. If a parser update changes results, the record tells you which fixtures and indexed documents to replay.

Keep tables as relationships

A table must keep the header path attached to each value. For a simple table, write each row with explicit column names. For grouped headers, carry both levels, and for tables that continue across pages, repeat the header. Store a structured table object for inspection and a retrieval text that repeats row and column context around each value. A twenty-page financial table is too big to embed whole and fixed character splits destroy it, so split by logical row groups and repeat the title, headers, units and section heading in each unit. Test merged cells, blank cells, repeated headers, wrapped values, footnotes and continuations. A parser that supports tables has not necessarily extracted yours correctly.

Chunk from elements and gate before embedding

Chunk from the normalized element tree, never from raw text. Group under heading paths, keep a caption with its figure, keep a list with its introduction, and apply size limits after semantic grouping, splitting at paragraph or list boundaries with the heading context repeated. Record the element IDs in each chunk so any chunk can be traced back, and a parser upgrade can be diffed element by element before you replace embeddings.

A successful parser call means only that the program returned. Run cheap checks on every page and quarantine output outside the expected range: empty output from a non-empty page, implausibly low text coverage, replacement-character bursts, duplicated lines from overlapping text layers, reading-order jumps, table rows with inconsistent cell counts, headings with no content, low OCR confidence, mismatched page counts, and output that changes when replayed with the same parser version. Set thresholds per document class from labeled samples, since a slide deck, a contract and an invoice have different densities. Pin parser versions so a later incident can be tied to the code that handled the file.

Hosted PDF features can change the parser underneath you. Anthropic's PDF support page says that on Amazon Bedrock's Converse API a PDF is handled in one of two modes. Without citations enabled, the request gets plain text extraction, which cannot see charts or visual layout and costs about 1,000 tokens for a 3-page file. With citations enabled, each page is processed as text and image, at about 7,000 tokens for the same 3 pages. The page also warns that dense PDFs can fill the context window before the page limit is reached, because each page is processed as an image. A flag you set for an unrelated reason can therefore change both what the model can read and what the call costs. Record the mode and the parser profile with every ingested document, and put a chart-bearing fixture in the regression suite so a silent drop to text-only fails a test.

Fixtures and failure handling

Keep a fixture suite small enough for CI: clean digital text, multiple columns, scans, mixed native and scanned pages, tables with merged cells and units, repeated headers and footnotes, forms, rotated pages, diagrams with captions, and malformed or password-protected files. Assert structure and avoid byte-for-byte snapshots, which break on harmless parser changes. Check that a heading precedes its section, a known table value stays tied to its row and column headers, and a scanned page produces text or a visible quarantine decision. Then test retrieval with questions that depend on the structures most likely to break, such as a value that needs both a row label and a unit, or evidence that turns to nonsense if columns interleave.

Run parsing like a versioned build. Write elements and chunks to staging, validate them, and publish a document version only after every required page passes or gets an approved quarantine disposition, so a job that dies halfway leaves the previous version active. Retry transient failures such as an OCR service error within a bound, and send an unsupported encryption scheme or unreadable scan to review, since repeating the call will not repair the file. Keep explicit states (received, inspected, parsing, quarantined, validated, published, superseded) with reason codes, so you can answer which documents are missing from retrieval and why.

Start with ten to twenty real files from your corpus. Define the element schema and route metadata, then write one structural assertion for each failure you have seen. Do this before tuning embeddings or prompts, since downstream stages cannot recover relationships the parser already lost.

References

• OCR Hinders RAG (OHRBench) for the link between OCR noise and RAG performance.
• ColPali: Efficient Document Retrieval with Vision Language Models for page-image retrieval and the ViDoRe benchmark.
• Microsoft Document Intelligence layout model for layout-aware extraction of text, tables and structure.
• Unstructured partitioning documentation for native, high-resolution, OCR and table-inference strategies.
• Docling documentation for the structured document model covering layout, reading order, tables, formulas, images and OCR.
• Anthropic PDF support for the Bedrock Converse text-only and citations modes, token costs per page and the dense-PDF context warning.

---