AI

RAG

Retrieval-Augmented Generation: a technique that enriches an LLM's answers by injecting relevant documents retrieved from a knowledge base before generation.

Bécher de verre rempli d'un liquide bordeaux lumineux, une petite pelle métallique suspendue au-dessus y versant une dose puisée dans une pile de cartes de verre sombres posée à côté, sur fond noir sous un faisceau blanc
RAG — Image générée par IA (Qwen-Image 2.1, ComfyUI local)

RAG (Retrieval-Augmented Generation) is a technique that improves an LLM's answers by supplying it with relevant documents pulled from a knowledge base before generation. RAG grounds answers in verified factual data, reducing hallucinations and making the model usable on private or recent data.

Technical definition

The classic RAG pipeline

RAG works in three phases:

1. Indexing — Source documents (PDFs, web pages, databases) are split into chunks (fragments of 200 to 1,000 tokens), turned into numerical vectors (embeddings) by an embedding model, and stored in a vector database (Pinecone, Weaviate, Chroma, pgvector).

2. Retrieval — When a question comes in, it is vectorized by the same embedding model and compared with the stored chunks using cosine similarity. The k closest passages are retrieved (top-k retrieval).

3. Augmented generation — The retrieved passages are injected into the LLM's prompt along with the user's question. The model generates its answer based on those sources, with the option of citing its references.

Agentic RAG

A major evolution of classic RAG. The AI agent takes control of the retrieval phase: it dynamically decides how many queries to run, rephrases search terms, evaluates how relevant the results are, and supplements with external sources (web search) if the internal base falls short. Retrieval becomes an iterative action within the ReAct cycle, not a fixed step.

This interactive diagram needs JavaScript.

Index the documents, ask a question and compare the LLM's answer with and without a vector database.

Current challenges

Chunking quality

How documents are split into chunks is the number-one quality factor in a RAG system. Poor chunking (too small: context is lost; too large: noise) degrades the relevance of results. Recent approaches use semantic chunking (splitting by logical sections) rather than fixed-size chunking.

Cost and scalability

A production RAG system involves a hosted vector database, an embedding model, an LLM for generation and potentially a reranker. Costs grow with the volume of indexed documents and the number of queries. Tuning top-k and using lightweight embedding models are common levers.

Standards and Specifications

EU AI Act

RAG is a technical approach to mitigating hallucination risks, relevant for AI Act compliance

Frequently Asked Questions

How does RAG work?

RAG works in three steps: 1) Indexing — documents are split into chunks and turned into numerical vectors (embeddings) stored in a vector database. 2) Retrieval — when the user asks a question, it is vectorized too and compared with the chunks to find the most relevant passages. 3) Generation — the retrieved passages are injected into the LLM's prompt, and the model generates its answer based on those sources.

What is the difference between RAG and agentic RAG?

Classic RAG follows a linear pipeline: one search, one answer. Agentic RAG adds intelligence to retrieval: the agent decides how to phrase its queries, evaluates whether the results are sufficient, rephrases if needed, supplements with external sources and iterates until it gets a satisfactory answer. Retrieval becomes a dynamic action within the agent's ReAct cycle.

Does RAG replace fine-tuning?

No, the two approaches are complementary. RAG injects knowledge at inference time (no need to retrain the model), which is ideal for data that changes often. Fine-tuning modifies the model's weights so that it adopts a specific style or behavior. In practice, most enterprise use cases rely on RAG because it is cheaper, faster to deploy and easier to maintain.

Resources and Documentation