Tools & agents
Retrieval-Augmented GenerationRAG
Also known as: retrieval augmentation
A technique that lets a model look up relevant information from your own documents and use it when answering, instead of relying only on what it learned in training.
Retrieval-augmented generation, or RAG, is a way to make an AI answer using your information, your company handbook, your notes, a specific set of documents, rather than only its general training. It works like an open-book exam: before answering, the system looks up the most relevant pages and reads them, so the reply is grounded in real, current material instead of the model's memory alone.
RAG combines two steps: retrieve, then generate. When you ask a question, the system first searches a collection of documents for the passages most relevant to your question, then hands those passages to the language model along with your question, so it can answer using them.
Why it is useful
- Current and private knowledge: it can use information the model never saw in training, including your own files.
- Fewer made-up answers: grounding the reply in retrieved text reduces hallucinations.
- Traceability: because answers come from specific passages, you can cite sources.
How the search works
Documents are split into chunks and turned into embeddings, numeric fingerprints of meaning, stored in a vector database. Your question is embedded too, and the system finds the chunks whose meaning is closest.
RAG augments a generative model with a non-parametric memory: at inference a retriever selects relevant passages from an external corpus and conditions generation on them, rather than relying solely on knowledge baked into the weights (Lewis et al. 2020). The canonical pipeline chunks documents, encodes them into dense embedding vectors, indexes them in a vector store, and at query time embeds the query, runs approximate nearest-neighbour search (often hybridised with lexical/BM25 retrieval and a re-ranking stage), and injects the top-k passages into the context window.
RAG addresses three limitations of parametric models at once: knowledge staleness, lack of access to private data, and hallucination, while enabling attribution. Its quality is bounded by retrieval quality, chunking strategy and context placement; failure modes include retrieving irrelevant context and the model ignoring or overriding supplied evidence. It is frequently combined with agentic tool use, where retrieval is one of several actions an agent can invoke.