Retrieval-augmented generation, usually called RAG, combines a language model with an external knowledge source. Instead of relying only on information stored during training, the system retrieves relevant evidence and includes it in the model prompt.
The basic RAG pipeline
Documents are divided into useful chunks and converted into vector representations. At question time, the query is also embedded. A search component finds similar chunks, and the language model uses them as context when generating the answer.
Why retrieval helps
Retrieval can provide current, private, or domain-specific information without retraining the model. It can also make answers easier to verify when the system displays the evidence used to produce them.
Retrieval quality is critical
A fluent model cannot repair missing or incorrect evidence reliably. Chunk size, metadata, filtering, embedding quality, and the number of retrieved passages all affect the result. Hybrid search can combine semantic similarity with keyword matching when exact terminology matters.
Generation needs boundaries
The prompt should tell the model to use the provided evidence, distinguish facts from uncertainty, and avoid inventing details. Citations are helpful only when they actually support the generated claim.
Evaluate the complete system
RAG evaluation should separate retrieval from generation. Measure whether the correct evidence was found, whether the answer is supported by that evidence, and whether it addresses the user’s question. A good demo is easy to build; a reliable RAG system requires careful data preparation, testing, and monitoring.