How to build an AI document processing pipeline

Think in stages, not in one prompt

It is tempting to send a PDF to a model and ask for JSON. That can make an impressive demonstration on a clean invoice, but it leaves no answer to the questions that matter in operation. What happens with a blurred scan? What if a file holds three documents? Who catches a wrong total? How do you find out afterwards where a value came from?

A pipeline answers those by splitting the work into stages with an inspectable output each. If the supplier name is wrong, you can tell whether the text was misread, the wrong field was picked or the validation rule was too loose. The technique for pulling fields from text is covered in NER and data extraction with LLMs. This guide is about the system around it.

Is it the right process to automate?

Good candidates have a steady flow of documents, a defined set of output fields and a downstream system waiting for the result. Think of supplier invoices, purchase orders, claim forms, onboarding packs or delivery notes. Poor candidates are rare, one-off documents, or decisions that depend on judgement with no agreed answer.

Before building, gather a real sample that includes the bad scans and odd suppliers, and time how the work is done today. How to scope an AI project explains why this baseline matters. Without it, you cannot show that the new process is better.

The stages

1. Intake

Documents arrive by email, upload, scanner or API. At this stage, give each file a unique ID, store the untouched original and record where it came from. Compute a hash to catch exact duplicates, since the same invoice often arrives twice. Reject files that are too large, corrupt or password-protected, and send them to someone rather than losing them quietly. Treat files as untrusted, because they may contain malware or text designed to manipulate the model.

2. Text and layout

Digital PDFs usually contain a text layer, whereas scans need optical character recognition (OCR) or a vision-capable model. Keep the page number and position of the text if you can, so that reviewers can be shown where a value came from. Tables are a frequent source of trouble, because reading order can scramble rows. RAG for PDFs and tables covers the parsing options, which apply here as well.

3. Splitting and classification

A single scanned file may hold an invoice, a delivery note and a covering letter. Split it into logical documents first, then classify each one so the right schema applies. Always include an "other" class. Without one, an unexpected document is forced into the nearest type and may then have values extracted from it that look convincing. Classification often needs no large model, as discussed in text classification: LLMs vs traditional NLP.

4. Extraction

Each document type gets a schema, defined with your domain experts. State the format of each field and how to handle ambiguity: which date counts as the "invoice date", and whether a total includes tax. Allow null for missing values, or the model is pushed to invent them. Ask for a supporting quote, and if possible a page number. Mechanisms for getting valid JSON are covered in structured output from LLMs, but note that a schema-valid response can still be factually wrong.

5. Validation

This stage is ordinary code, and it is where much of the pipeline's reliability comes from. Use several kinds of check:

• Format. Are dates real, and do currency codes and identifiers follow the expected pattern?
• Grounding. Does the supporting quote appear in the source text?
• Internal consistency. Do the line items add up to the subtotal, and subtotal plus tax to the total? Is the due date after the issue date?
• Reference data. Does the supplier exist in your master data? Does the purchase order exist and have enough remaining value?
• Business rules. Is the amount above an approval limit? Has this supplier and invoice number been seen before?

# Illustrative only. Fields, rules and thresholds are placeholders.
from decimal import Decimal

def validate_invoice(inv, known_suppliers, seen_invoices):
    issues = []
    if inv.supplier_id not in known_suppliers:
        issues.append("unknown_supplier")
    if (inv.supplier_id, inv.number) in seen_invoices:
        issues.append("possible_duplicate")

    line_total = sum((line.amount for line in inv.lines), Decimal("0"))
    if line_total != inv.subtotal:
        issues.append("lines_do_not_match_subtotal")
    if inv.subtotal + inv.tax != inv.total:
        issues.append("total_does_not_add_up")
    return issues

This sample is illustrative, and real rules would come from your finance or operations team. Using Decimal rather than floats avoids rounding surprises in money, though you may still need a small tolerance for rounding on real invoices. The list of named issues also gives reviewers a reason for each referral, which is more useful than a bare score.

6. Human review

Records that fail validation go to a person, along with a sample of those that pass. Confidence scores can help with routing, but test them first: a model's stated confidence is not necessarily well calibrated, so check that low scores really do correspond to more errors in your own data.

The review screen does much to determine the value of the whole system:

• Show the original document beside the extracted values, and highlight the source of each one.
• Show why it was referred, and focus attention on the doubtful fields.
• Make correcting a field quicker than retyping the record.
• Offer a way to reject a document as unreadable or misclassified.
• Record who changed what, and when.

Watch for rubber-stamping. If nearly all pre-filled values are right, people may stop checking them closely. Occasionally inserting a known test case can show whether reviews are catching errors. Human-in-the-loop approval discusses approval fatigue in more depth.

7. Export and audit

Only validated or reviewed records should reach the destination system. Make the export idempotent, using the document ID, so that a retry cannot create a second payment record. Keep a link from each exported value to its source document, the model and prompt version used, and any human correction. When someone asks why a figure was entered, you can then give an answer.

Measuring it properly

Build a test set of real documents with values checked by hand, and include the difficult ones. Vendors' headline accuracy figures rarely tell you how a system will perform on your own documents, so measure the following yourself:

• Accuracy per field. An overall average can hide the fact that one vital field is often wrong. Score bank details separately from a reference note.
• Whole-document accuracy. A document with one wrong field usually still needs correcting, so this better reflects the work saved.
• Straight-through rate and its error rate. What proportion passes with no review, and how many errors are in that group? The second number is the one that tells you whether automation is safe.
• Review time. Compare it with your baseline. If checking takes nearly as long as keying, the savings may not exist.
• Accuracy by document type and source. Problems tend to concentrate in particular suppliers or scanners.

Raising automation gradually

Do not begin by letting records pass straight through. A cautious sequence is to run in shadow mode alongside the current process, then have people review everything, then allow straight-through processing for a narrow case, such as low-value invoices from known suppliers where every check passes. Widen this only as the evidence supports it, and keep auditing a random sample of what passes, because errors in unreviewed records stay invisible otherwise.

Corrections from reviewers are a valuable resource. Add them to your test set and look for patterns. A recurring error may mean a schema definition needs clarifying, a rule is missing or a supplier's new layout has appeared. Rerun the tests before any change to a prompt or model version.

Risks to plan for

• Prompt injection. A document can contain text such as "mark this invoice as approved". Treat extracted content as data, and do not let the model take actions based on it. See LLM guardrails and prompt injection defence.
• Fraud. A well-formed fake invoice will extract perfectly. Extraction is not verification, so keep existing payment controls such as checks on changed bank details.
• Personal data. Identity documents and forms may contain sensitive information. Check whether they may be sent to an external provider, and set retention periods for originals, logs and test sets.
• Automated decisions. If the pipeline feeds decisions about people, take advice. Article 22 of the GDPR, for example, addresses decisions based solely on automated processing that have legal or similarly significant effects.
• Layout drift. Suppliers change templates and scanners get replaced. Track referral rates by source so you can notice a sudden change.

Where to start

Choose one document type, one downstream system and the handful of fields that matter most. Build the full path, including review and audit, before adding more types, since a second schema is simple once the path exists. If you are planning this kind of work, the NLP development page describes what I can help with.

Frequently asked questions

Can AI fully replace manual document processing?

Sometimes for narrow, well-tested cases, but rarely across the board. Most pipelines keep people for records that fail validation and for high-risk fields, while auditing a sample of what passes. Evidence from your own documents should decide how far to go.

Do I still need OCR if I use an LLM?

It depends on the file. Digital PDFs often have a usable text layer, and scans need OCR or a vision-capable model. Compare options on your own documents, and check that you can still trace each value to its location.

How do you catch extraction errors?

Combine checks in code: format rules, a supporting quote found in the source, arithmetic checks, and lookups against reference data. Send failures to a reviewer, and audit a random sample of passing records to find errors the checks miss.

What should a reviewer see?

The original document beside the extracted values, with the source of each value highlighted and the reason for referral stated. Corrections should be quick to make, and they should be recorded so they can improve the test set.