Definition:
RAG, short for Retrieval-Augmented Generation, is an artificial intelligence architecture that retrieves information from external sources and adds it to the context of a generative model before producing an answer. It therefore combines a retrieval system with a language model.
The retrieved information may come from documents, knowledge bases, catalogues, websites, or internal systems. RAG can consult content that is not stored in the model parameters, but it does not guarantee correct, current, or supported answers by itself. The result depends on the complete quality of the process.
Origin and scope of RAG
The name became established through the work published in 2020 by Patrick Lewis and other researchers, which combined parametric and non-parametric memory for knowledge-intensive tasks. RAG has since expanded into an architectural pattern that can use different models, retrievers, and sources.
RAG is not a language model, a database, or a particular product. Nor does it require an internet search. A system may retrieve public or private, structured or unstructured information, provided that it can select it and turn it into usable context for generation.
The architecture is useful when knowledge changes frequently, belongs to an organisation, or needs provenance. Updating sources may be faster than retraining a model, although preparing, indexing, and monitoring those sources still requires operational work.
How RAG works
Implementations vary, but the usual flow separates knowledge preparation from the response to each query. The basic process includes these stages:
- Prepare sources: Select documents, clean content, preserve metadata, and define access permissions.
- Split content: Create chunks whose size and overlap suit the information and expected questions.
- Represent and index: Generate an embedding or another representation and store it in a searchable index.
- Retrieve: Find candidate chunks through similarity, keywords, filters, relationships, or a combination of methods.
- Augment context: Add the query, instructions, and selected chunks within the context window.
- Generate an answer: Ask the model to respond from the available context and, where appropriate, show references or acknowledge missing evidence.
A reranker may reorder candidates after retrieval, while rules may remove duplicates and unauthorised content. Semantic search is common, but RAG is not limited to it: hybrid retrieval can combine different signals to improve selection.
System components
Embeddings represent content in a numerical space so that semantic proximity can be compared. A vector database can store those representations and find neighbours, but neither component is RAG on its own. Lexical indexes, relational databases, graphs, and search services may also be used.
The retriever decides which candidate information reaches the model. Its performance depends on the indexed content, chunking, query, filters, and ranking criteria. If it retrieves irrelevant passages or misses an essential source, the generator starts with poor context.
The generative model interprets the query and writes the output. It may follow instructions, synthesise several passages, and format an answer, but it may also ignore evidence, merge sources, or add unsupported claims. RAG reduces some opportunities for fabrication; it does not remove the capacity for error.
Fine-tuning changes a model’s behaviour or parameters through additional training. RAG supplies information during inference. The two approaches address different problems and may be combined when both behavioural adaptation and external knowledge are needed.
Applications of RAG
RAG is used when an application must answer from an identifiable body of information. Common use cases include:
- Customer support: Consult manuals, policies, incidents, and product records to inform a response.
- Internal knowledge: Search procedures, technical documentation, reports, and repositories under access control.
- Specialised fields: Locate legal, scientific, educational, or healthcare material before producing a synthesis that requires expert review.
- Assisted search: Answer questions about catalogues or collections while retaining links to retrieved documents.
- Document creation: Prepare drafts, summaries, or comparisons from specific sources without replacing editorial checks.
The same architecture may support assistants, search tools, support systems, or agent workflows. Its value does not come from displaying a direct answer alone, but from connecting the task with relevant information while retaining controls over its use.
System evaluation
Evaluation should separate retrieval from generation. A fluent answer may rely on unsuitable documents, while correct retrieval may end in an incomplete synthesis. A representative question set should be used to examine these dimensions:
- Coverage: Whether the system retrieves the passages needed to answer.
- Precision: Whether selected candidates are relevant and avoid unnecessary noise.
- Grounding: Whether each important claim is supported by the supplied context.
- Answer quality: Whether the output resolves the query, acknowledges limits, and uses references correctly.
- Operations: Latency, cost, availability, updates, and permission compliance.
Automatic metrics help compare configurations, but they need human-reviewed samples and tests of difficult cases. Monitoring should detect source changes, new queries, access failures, and degradation. Measuring only satisfaction or appearance may conceal factual errors.
Limitations of RAG
RAG cannot repair a false, outdated, or contradictory source. Nor can it use a document that was not ingested, cannot be retrieved, or was divided in a way that removed its meaning. Content governance and update cycles shape final reliability.
Displaying a citation does not prove that it supports the claim. The system should connect answer and passage, avoid invented references, and allow the original source to be opened. In sensitive contexts, outputs require additional verification and should not be presented as automatic professional decisions.
Sources may contain malicious instructions, personal data, or material that a user should not access. The design should apply permissions before retrieval, record access, protect sensitive information, and treat retrieved content as untrusted data, not as commands for the model.
Indexing, storage, retrieval, and generation also create costs, latency, and maintenance. The decision to use RAG should begin with the task, available evidence, and risk level. For stable queries or deterministic processes, conventional search or a structured response may be a simpler solution.