Home > Notes > AgenticAI > RAG

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is an advanced AI technique that enhances the capabilities of large language models (LLMs) by integrating external knowledge retrieval with text generation. Introduced in a 2020 paper by Facebook AI Research (now Meta AI), RAG addresses key limitations of standalone LLMs, such as outdated knowledge, factual inaccuracies (hallucinations), and lack of domain-specific expertise.

Instead of relying solely on the model's pre-trained parameters, RAG dynamically retrieves relevant information from a knowledge base at query time and uses it to "augment" the generation process. This results in more accurate, contextually grounded, and up-to-date responses.

As of 2025, RAG has become a cornerstone of AI applications, powering systems like enterprise chatbots, search engines, and research assistants. It's particularly valuable in scenarios where knowledge evolves rapidly (e.g., news, legal, or scientific domains) or where proprietary data must be incorporated without retraining the LLM.

RAG is not a single model but a framework that combines retrieval systems (often vector databases) with generative models. It can be implemented in various ways, from simple setups using open-source tools to enterprise-scale deployments with cloud services. The core idea is to "retrieve" relevant documents or data chunks based on semantic similarity and then "generate" a response conditioned on that retrieved context.

Components

RAG's architecture typically consists of several interconnected components, forming a pipeline that handles querying, retrieval, and generation. These can be customized based on the use case.

How It Works

RAG operates in a multi-stage pipeline, which can be synchronous (for quick queries) or asynchronous (for complex tasks). Here's a step-by-step breakdown:

  1. Query Input and Embedding:
  2. The user submits a query (e.g., "What are the latest advancements in quantum computing as of 2025?").

  3. The query is pre-processed (e.g., tokenized) and passed through the embedding model to generate a query vector.

  4. Retrieval:

  5. The query vector is sent to the retriever, which searches the vector database for similar vectors (top-k results, e.g., k=5).
  6. Retrieved items are ranked by similarity score and optionally filtered (e.g., by recency). If hybrid, keyword matches are incorporated.

  7. Output: A list of relevant document chunks with metadata (e.g., source, score).

  8. Context Augmentation:

  9. The retrieved chunks are concatenated into a context string, often with formatting (e.g., "Document 1: [text]\nDocument 2: [text]").

  10. This augmented context is combined with the original query in a prompt template.

  11. Generation:

  12. The prompt is fed to the LLM, which generates a response grounded in the context.

  13. Post-processing may include extracting citations, rephrasing, or verifying facts.

  14. Output and Iteration:

  15. The final response is returned to the user. In advanced RAG (e.g., iterative RAG), the system may refine the query and re-retrieve if needed.

This process ensures responses are factual and relevant, with typical latencies under a second for retrieval on optimized systems.

Python Code Demonstration: Simple RAG Implementation

Here's a practical Python example using Hugging Face's sentence-transformers for embeddings, FAISS for the vector database, and OpenAI's GPT-3.5-turbo for generation. This demonstrates building a small knowledge base, retrieving from it, and generating a response. Install dependencies: pip install sentence-transformers faiss-cpu openai.

import numpy as np
import faiss
from sentence_transformers import SentenceTransformer
import openai

# Set your OpenAI API key
openai.api_key = 'your-openai-api-key'  # Replace with your actual key

# Step 1: Prepare Knowledge Base
documents = [
    "Quantum computing uses qubits for faster calculations than classical bits.",
    "In 2025, IBM released a 1000-qubit processor, advancing error-corrected quantum systems.",
    "Google's Sycamore achieved quantum supremacy in 2019, but advancements continue with hybrid quantum-classical models.",
    "Challenges in quantum computing include decoherence and error rates."
]

# Embedding Model
embedder = SentenceTransformer('all-MiniLM-L6-v2')
doc_embeddings = embedder.encode(documents)

# Build FAISS Index (Vector Database)
dimension = doc_embeddings.shape[1]
index = faiss.IndexFlatL2(dimension)  # L2 distance for similarity
index.add(np.array(doc_embeddings))  # Add embeddings to index

# Step 2: Query Processing
query = "What are the latest advancements in quantum computing as of 2025?"
query_embedding = embedder.encode([query])

# Retrieve Top-K (k=2)
D, I = index.search(query_embedding, k=2)  # D: distances, I: indices
retrieved_docs = [documents[i] for i in I[0]]
print("Retrieved Documents:", retrieved_docs)

# Step 3: Augment and Generate
context = "\n".join(retrieved_docs)
prompt = f"Based on the following context:\n{context}\n\nAnswer the query: {query}"

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": prompt}]
)
generated_text = response.choices[0].message.content
print("Generated Response:", generated_text)

Explanation of Code: - Knowledge Base Setup: Documents are embedded and indexed in FAISS for fast retrieval. - Retrieval: Query is embedded and searched against the index to fetch top-2 similar docs. - Generation: Retrieved docs form the context in the prompt, fed to OpenAI's LLM. - Output: Run this to see retrieved docs and a grounded response. In production, use a real vector DB like Pinecone for scalability.

This is a basic example; for advanced features, integrate LangChain: from langchain.vectorstores import FAISS; from langchain.chains import RetrievalQA.

Advantages

Disadvantages

Use Cases

Conclusion

RAG transforms LLMs from static knowledge regurgitators to dynamic, knowledge-augmented systems, making them more reliable for real-world applications. Its modular design allows customization, and with tools like LangChain, implementation is accessible. For production, consider security (e.g., encrypted vectors) and evaluation metrics (e.g., RAGAS for faithfulness). If you'd like expansions on variants (e.g., Naive RAG vs. Advanced RAG) or more code (e.g., with multimodal data), let me know!