RAG development

What it is

Retrieval-augmented generation, or RAG, is a pattern for giving a language model information it was not trained on. When a question arrives, the system searches a knowledge source, selects the most relevant passages and passes them to the model along with the question. The model then writes an answer based on that context, ideally with references back to the sources.

A RAG system has two main parts: an ingestion pipeline that prepares and indexes content, and a query pipeline that retrieves and generates. The guide what RAG is and when to use it explains the pattern in more depth.

When it suits a project

RAG suits projects where answers must reflect specific, changing or private content, such as internal documentation, policies, product manuals or support articles. Because the content lives in an index rather than in the model, it can be updated without retraining anything.

It suits a project less well if the questions need aggregate calculations across structured records, where a database query is the natural tool, or if the whole knowledge base is small enough to include directly in the prompt. Some systems combine approaches and route each question to the right one.

How it is typically approached

• Understand the content and questions. Review document formats and gather realistic questions, because both shape every later decision.
• Build ingestion. Parse the source files, clean them, split them into chunks and store useful metadata such as source, section and date. See choosing chunking strategies for RAG.
• Index for retrieval. Create embeddings for vector search, and often add keyword search as well, since exact terms such as product codes can be missed by semantic similarity alone. See hybrid search and reranking for RAG.
• Refine the results. Apply metadata filters during the search and, where it helps, add a reranking step that reorders candidates by relevance to the query.
• Generate with care. Instruct the model to rely on the supplied context, cite its sources and say so when the context does not contain an answer.
• Evaluate each stage. Measure whether the right passages were retrieved separately from whether the final answer was faithful and useful, so that problems can be traced to their source. See how to evaluate a RAG system.
• Respect access rules. If not every user may see every document, permissions need to be enforced at retrieval time, before content reaches the model.

What I can help with

I can help with designing and building a new RAG system, connecting it to a user interface, or reviewing where an existing pipeline is losing quality. When retrieval is one step among several, it can also become part of a larger workflow; see multi-agent systems.

Frequently asked questions

What is the difference between RAG and fine-tuning?

RAG supplies information to the model at question time, whereas fine-tuning adjusts the model itself using training examples. RAG is generally the better match for facts that change, and fine-tuning is more often considered for consistent style or task behaviour.

Can RAG use a database we already run?

Often, if it supports vector or full-text search. Data volume, filtering needs and operational effort decide whether that is enough, and a test on a sample of your content can show this before adding a separate system.

How are wrong answers handled?

They cannot be ruled out, so the design plans for them: citations let users check a claim, a fallback applies when no good source is found, and evaluation shows which question types fail and at which stage.

What types of content can be used?

Text-based content is the most straightforward. PDFs, tables, scans and images can also be handled, but they usually need extra parsing work, so a sample of your real documents should be reviewed early.