RAG for PDFs and Tables: Parsing Awkward Documents

Why PDFs cause so much trouble

A PDF is designed to display a page, not to describe its logical content. Internally, a page is largely a set of drawing instructions that put text at particular positions. Headings, paragraphs and tables are often things a reader infers from the layout, and the file may not record them explicitly. A parser has to reconstruct that structure, and it can go wrong without raising an error.

The usual symptoms show up in retrieval later on. Two columns are merged line by line into nonsense. A page footer appears in the middle of a sentence. A table becomes a stream of numbers with no headers. Each failure produces text that embeds poorly and misleads the model, which may still sound confident. This is why the step deserves attention before you adjust the chunking strategy or the prompt.

Step 1: sort your documents by type

Running one parser over everything is the commonest shortcut, and a collection is rarely uniform. Take a sample and look at what you really have:

Digital PDFs with a text layer. These were exported from software, and text can be extracted directly. Simple layouts may need little more.
Scanned PDFs. These are images of pages, so they need optical character recognition (OCR) or a vision model.
Layout-heavy documents. Reports, brochures and academic papers can have columns, sidebars and captions that need layout analysis.
Table-heavy documents. Financial statements, specifications and price lists need specific table handling.
Mixed documents. One file may contain all of the above, so it is worth routing at page level where the volume justifies it.

A cheap check can do part of this routing. If a page yields almost no characters from its text layer but contains a large image, it is probably scanned. Be careful, though: some files carry a poor hidden text layer from an earlier OCR run, so the presence of text does not prove that it is good.

Step 2: choose parsers by the failure you need to avoid

Parsing tools broadly fall into a few categories, and many pipelines combine more than one:

Text-layer extractors are fast and inexpensive. They can return words and positions but may offer limited insight into structure.
Layout-aware parsers try to detect headings, columns, lists and tables, often with machine learning models. They usually cost more compute.
OCR engines recover text from images. Accuracy depends on scan quality, language and layout.
Vision-capable LLMs can transcribe a page image into Markdown and may cope with complicated layouts. They add cost and latency, and they can produce plausible text that is not on the page, so they call for verification.

Compare candidates on your own hardest pages. Pick perhaps a few dozen that represent the difficult cases, run each parser and read the output beside the original. Tools change quickly, so a test on your documents will tell you more than a general ranking.

Step 3: keep the structure that retrieval needs

Aim to output clean, structured text, such as Markdown, together with metadata. Several details matter:

Reading order. Columns, sidebars and footnotes should not be interleaved with the main text.
Heading paths. Store the path to each section, such as "Section 4 > Warranty > Exclusions", and consider adding it to each chunk. A passage then keeps its context once it is separated from the page.
Repeated furniture. Remove running headers, footers and page numbers. Text that recurs at the same position on many pages is a useful clue, but make sure that you do not remove a legal notice someone may need.
Broken words and lines. Rejoin words hyphenated at line ends and lines wrapped within a paragraph.
Page references. Keep the source file and page number on each chunk, so that answers can cite a location a reader can check.

Step 4: decide how to represent tables

A table's meaning lies in relationships. "42" means nothing until you know its row and its column. A good representation preserves that link. There are several common options, each with its own trade-offs:

A whole table as Markdown or HTML. This is simple and suits small tables. HTML can represent merged cells, whereas Markdown cannot do so directly. Large tables can exceed a sensible chunk size.
Row-level text. Each row becomes a self-contained statement that repeats its headers. This is useful for precise lookups, though it can inflate the index.
A summary pointing to the full table. You embed a short description of what the table contains, then send the full table to the model when it is retrieved. This helps with long tables, but the summary must mention what users will search for.
A structured store. You load the values into a database and query them. This is the right direction for totals, filters and comparisons.

Here is the row-level approach. This sample is illustrative: it assumes the headers and rows have already been extracted correctly, which is often the hard part.

def table_to_chunks(title, headers, rows, source, page):
    chunks = []
    for row in rows:
        pairs = [f"{h}: {v}" for h, v in zip(headers, row)]
        chunks.append({
            "text": f"{title}. " + "; ".join(pairs),
            "metadata": {"source": source, "page": page, "type": "table_row"},
        })
    return chunks

A row such as "Basic, 5 users, 10 GB" then becomes text that names its plan, users and storage explicitly, so it still makes sense when retrieved alone.

Table problems to look for

Tables that continue across pages. Detect when a table carries on, merge the parts and avoid treating the repeated header as data.
Merged and nested headers. Flatten them so that each value receives its full label, such as "2024 Revenue" and not simply "Revenue".
Empty cells. A blank may mean zero, unknown or "same as above". Do not guess silently; where a merged cell clearly applies downwards, fill it in deliberately.
Units and footnotes. "Figures in thousands" may appear only in a caption. Carry this information with the table, or answers may be wrong by a large factor.
Wrongly aligned columns. Borderless tables can shift values into the next column. This is especially harmful because the text still looks reasonable.

Step 5: handle scans, images and charts

For scanned pages, image quality has a large effect. Correcting rotation and skew can help before OCR. Keep any confidence scores your OCR tool provides, and send low-confidence pages for review or mark them, particularly where digits matter, because a misread number looks as valid as a correct one.

Charts and diagrams are a separate question. Decide whether their contents are likely to be asked about. If they are, a vision model can produce a text description for indexing, but treat values read from a chart as approximate. Link back to the original figure, so a reader can confirm the result. Text inside documents can also carry instructions intended for the model, so parsed content should remain untrusted input, however it was extracted.

Step 6: check extraction before you tune retrieval

Add quality checks to the ingestion pipeline, so that problems surface before a user finds them:

Flag pages with unusually little text, a high share of odd characters or very short lines.
Compare the number of tables you detected with a sample count by hand, to estimate what is being missed.
Spot-check numbers from extracted tables against their source pages.
Record which parser and version processed each document, so that affected files can be reprocessed later.

Next, separate parsing failures from retrieval failures in your testing. Include questions whose answers sit in tables, footnotes and scanned pages, then trace each miss back. Was the answer present in the extracted text? Was it in a chunk? Was that chunk retrieved? How to evaluate a RAG system covers how to build this test set, and hybrid search and reranking helps when users search for exact codes and figures found in tables.

When retrieval is the wrong answer for tables

A question such as "What was the total across all regions?" needs every relevant row and a correct calculation. Retrieving a few chunks and asking a model to add them up is fragile. Extract such tables into a database and answer through a query instead, using the precautions set out in text-to-SQL agents: risks and safeguards. A system can route lookups to the document index and calculations to the database.

If your documents are proving difficult to work with, the RAG development page describes what I can help with.

Frequently asked questions

What is the best PDF parser for RAG?

No single parser suits every collection. Compare a few on a sample of your hardest pages, including columns, tables and scans, and judge the output by whether a person could still understand it and answer questions from it.

Should tables be split across chunks?

Avoid splitting them in a way that loses headers. Small tables can stay whole. A large table can be divided by row or group, provided each piece repeats the table title and relevant column headers.

Do scanned PDFs work with RAG?

Yes, once their text has been recovered through optical character recognition or a vision model. Recognition errors can harm retrieval and answers, so check quality on a sample and keep a link to the original page.

Can RAG add up the values in a table?

Not reliably on its own. Retrieval may miss relevant rows, and a language model can make arithmetic errors. For totals and comparisons, extract the table into a structured store and calculate the result with a query.

RAG for PDFs and tables: parsing awkward documents