AI Courses for Kids in Presov

Chapter 181 — Build a Retrieval-Augmented Generation (RAG) Pipeline Step by Step

1. Introduction: What is an End-to-End RAG Pipeline?

In previous modules, we examined the individual building blocks of Retrieval-Augmented Generation (RAG)—including Document Loaders, Text Splitters, Vector Embeddings, Vector Databases, and Large Language Models (LLMs). While studying these modules individually provides a strong conceptual foundation, building a production-ready AI application requires assembling all of these distinct components into a unified, continuous RAG Pipeline.

  ┌─────────────────────────────────────────────────────────────┐
  │                 End-to-End RAG Pipeline Flow                │
  ├─────────────────────────────────────────────────────────────┤
  │  Document Loader ──► Text Splitter ──► Vector Embeddings    │
  │                                              │              │
  │                                              ▼              │
  │  LLM Answer Generation ◄── Retriever ◄── Vector DB Storage  │
  └─────────────────────────────────────────────────────────────┘
    

Core Purpose of a RAG Pipeline:

• Combining Retrieval & Generation: The primary objective of a RAG pipeline is to seamlessly link document retrieval with answer generation within a single linear execution model.
• Overcoming LLM Limitations: By injecting relevant document context into the model at runtime, the pipeline overcomes major LLM bottlenecks—such as hallucinations (generating false facts) and context length constraints.
• Grounding & Verifiability: Every answer generated by the pipeline is strictly grounded in facts extracted from your uploaded enterprise documents.

2. The Complete 6-Stage RAG Architectural Workflow

A complete end-to-end RAG pipeline operates across six sequential workflow stages:

  ┌─────────────────────────────────────────────────────────────┐
  │                6-Stage RAG Pipeline Architecture            │
  ├─────────────────────────────────────────────────────────────┤
  │                                                             │
  │   1. Data Ingestion    ──► PyPDFLoader / TextLoader         │
  │   2. Text Chunking     ──► RecursiveCharacterTextSplitter   │
  │   3. Vector Indexing   ──► HuggingFace Embeddings + Vector DB│
  │   4. Retriever Setup   ──► Similarity Search (top-k matches)│
  │   5. LLM & Prompt      ──► Qwen2 0.5B / OpenAI + Templates │
  │   6. Chain Execution   ──► RunnablePassthrough & LCEL Pipe  │
  │                                                             │
  └─────────────────────────────────────────────────────────────┘
    

1. Document Ingestion: Ingests raw source data (PDFs, text files, CSVs, or web pages) using Document Loaders.
2. Text Chunking: Divides long document text into smaller, manageable chunks with overlap to preserve boundary context.
3. Vector Embeddings & Storage: Converts text chunks into high-dimensional numerical vectors and indexes them into a vector database (such as FAISS or ChromaDB) for high-speed retrieval.
4. Retriever Configuration: Sets up a retriever object to query the vector database and extract the top-k most contextually relevant document chunks based on semantic similarity.
5. LLM & Prompt Formatting: Loads an LLM (either an open-source model like Qwen2 0.5B or a cloud API like OpenAI) and prepares structured chat prompt templates with system/user roles and placeholder tags.
6. RAG Chain Construction & Execution: Connects the retriever, prompt formatters, and LLM using LangChain Expression Language (LCEL) and Runnable pipelines to execute user queries end-to-end.

3. Setting Up the Open-Source LLM Engine (Qwen2 0.5B & PyTorch)

While commercial cloud APIs (like OpenAI) offer quick setup, building private or cost-effective RAG pipelines often relies on open-source LLMs running locally or on self-hosted GPU/CPU hardware.

  ┌─────────────────────────────────────────────────────────────┐
  │              Qwen2 0.5B Model Setup Configuration           │
  ├─────────────────────────────────────────────────────────────┤
  │  • Model ID        : Qwen/Qwen2-0.5B-Instruct               │
  │  • Parameters      : 0.5 Billion trainable weights         │
  │  • Execution Precision: Float16 (GPU) / Float32 (CPU)       │
  │  • Device Mapping  : CUDA (GPU) if present, else CPU       │
  │  • Max New Tokens  : 128 tokens                            │
  │  • Temperature     : 0.1 (low creativity, high accuracy)  │
  └─────────────────────────────────────────────────────────────┘
    

A. Model Selection & Parameters:

• Qwen2 0.5B Instruct: An open-source language model containing 0.5 billion parameters (trainable model weights set during training) optimized for instruction-following and question answering.
• PyTorch & HuggingFace Transformers: Leverages torch for hardware acceleration alongside HuggingFace's AutoTokenizer and AutoModelForCausalLM.

B. Hardware Acceleration & Precision Settings:

• Device Mapping: Automatically detects whether CUDA GPU acceleration is present; if GPU hardware is available, it maps execution to GPU with float16 precision for significantly faster response generation; otherwise, it defaults to CPU execution.
• Temperature Control (temperature=0.1): Setting a low temperature value (0.1) restricts model randomness and creativity, ensuring precise, non-hallucinated answers strictly based on retrieved documents.
• Max Tokens (max_new_tokens=128): Restricts the maximum output token length to ensure fast execution during local CPU testing.

4. Constructing the RAG Chain with LangChain Runnables

To connect the retriever, prompt formatting functions, and LLM into a unified chain, modern LangChain pipelines use Runnables (RunnablePassthrough, RunnableLambda) and the pipe operator (|).

  ┌─────────────────────────────────────────────────────────────┐
  │                 RAG Execution Chain Construction            │
  ├─────────────────────────────────────────────────────────────┤
  │                                                             │
  │   User Question Input                                       │
  │        │                                                    │
  │        ├──────────────┐                                     │
  │        ▼              ▼                                     │
  │  Retriever(k=2)   RunnablePassthrough()                     │
  │        │              │                                     │
  │        ▼              │                                     │
  │  Joined Context       │                                     │
  │        │              │                                     │
  │        └──────┬───────┘                                     │
  │               ▼                                             │
  │     Formatted Prompt Template                               │
  │               │                                             │
  │               ▼                                             │
  │     Qwen2 LLM Generation                                    │
  │               │                                             │
  │               ▼                                             │
  │     Parsed Final Answer Output                              │
  │                                                             │
  └─────────────────────────────────────────────────────────────┘
    

Core Chain Components:

1. Retriever Object (vector_db.as_retriever(search_kwargs={"k": 2})): Configures similarity search to return the top 2 matching document chunks from the vector database.
2. RunnablePassthrough: Passes the raw user query forward unchanged so it can be combined with the retrieved document context.
3. RunnableLambda: Wraps custom Python formatting functions (such as format_for_qwen and generate_with_qwen) so they behave as native LangChain execution blocks.
4. Pipe Operator (|): Chains data sequentially: Context Extractor | Formatter | LLM | Output Cleaner.

5. Complete Hands-On Python Implementation in VS Code

Below is the complete, end-to-end Python script demonstrating how to build, compile, and execute a full RAG pipeline using LangChain, HuggingFace Transformers, PyTorch, and a local vector store:

# =====================================================================
# Full End-to-End RAG Pipeline Implementation (Topic 181)
# =====================================================================

import os
import torch
from dotenv import load_dotenv

# 1. Import LangChain & HuggingFace Modules
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_community.llms import HuggingFacePipeline
from langchain_core.runnables import RunnablePassthrough, RunnableLambda
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

# ---------------------------------------------------------------------
# STAGE 1 & 2: Document Ingestion & Chunking
# ---------------------------------------------------------------------
sample_knowledge_base = """
Retrieval-Augmented Generation (RAG) is a popular technique used in LangChain.
RAG combines document retrieval with language model answer generation into a linear pipeline.
RAG solves core LLM limitations such as hallucinations and context window limits.
Chains allow developers to combine multiple components together into automated AI applications.
"""

doc_file = "rag_knowledge.txt"
with open(doc_file, "w", encoding="utf-8") as f:
    f.write(sample_knowledge_base.strip())

loader = TextLoader(doc_file, encoding="utf-8")
documents = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=120, chunk_overlap=20)
doc_chunks = text_splitter.split_documents(documents)

# ---------------------------------------------------------------------
# STAGE 3 & 4: Vector Embeddings & Vector DB Retriever
# ---------------------------------------------------------------------
embedding_model = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
vector_db = FAISS.from_documents(doc_chunks, embedding_model)

# Configure retriever to fetch top k=2 similar matches
retriever = vector_db.as_retriever(search_kwargs={"k": 2})

# ---------------------------------------------------------------------
# STAGE 5: Open-Source LLM Setup (Qwen2 0.5B Instruct)
# ---------------------------------------------------------------------
model_id = "Qwen/Qwen2-0.5B-Instruct"

# Configure hardware device (CUDA GPU if available, else CPU)
device = "cuda" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

print(f"Loading Model {model_id} on Device: {device.upper()}...")

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch_dtype,
    device_map=device
)

# Create HuggingFace Text-Generation Pipeline
hf_pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=128,
    temperature=0.1,      # Low temperature to suppress hallucinations
    do_sample=True
)

llm = HuggingFacePipeline(pipeline=hf_pipe)

# ---------------------------------------------------------------------
# STAGE 6: Prompt Formatting & RAG Chain Construction
# ---------------------------------------------------------------------
def format_prompt(inputs):
    """Formats system instructions, retrieved context, and user question."""
    context_str = "\\n".join([doc.page_content for doc in inputs["context"]])
    question_str = inputs["question"]

    formatted = (
        f"System: Answer the question based strictly on the provided context below.\\n"
        f"Context:\\n{context_str}\\n\\n"
        f"User: {question_str}\\n"
        f"Answer:"
    )
    return formatted

def generate_answer(formatted_text):
    """Invokes LLM on formatted prompt and cleans whitespace output."""
    raw_response = llm.invoke(formatted_text)
    # Extract text content following the 'Answer:' tag if present
    if "Answer:" in raw_response:
        answer = raw_response.split("Answer:")[-1].strip()
    else:
        answer = raw_response.strip()
    return answer

# Construct RAG Chain using LCEL Pipe (|) Operators
rag_chain = (
    {
        "context": retriever,
        "question": RunnablePassthrough()
    }
    | RunnableLambda(format_prompt)
    | RunnableLambda(generate_answer)
)

# ---------------------------------------------------------------------
# STAGE 7: Pipeline Execution & Verification
# ---------------------------------------------------------------------
user_query = "What does RAG stand for?"
print(f"\\nUser Question: {user_query}")
print("Executing End-to-End RAG Pipeline...")

response = rag_chain.invoke(user_query)

print("\\n=== GENERATED RAG RESPONSE ===")
print(response)

# Retrieve and display context passages used by retriever
retrieved_docs = retriever.invoke(user_query)
print("\\n=== RETRIEVED CONTEXT PASSAGES ===")
for idx, doc in enumerate(retrieved_docs):
    print(f"[{idx+1}] {doc.page_content}")

# Clean up temporary file
if os.path.exists(doc_file):
    os.remove(doc_file)
    

Execution Results & Output Analysis:

When executed, the script initializes the Qwen2 model on the active device (CPU or GPU), queries the vector store, and returns the grounded answer:

• Question Input: "What does RAG stand for?"
• Generated Answer Output: "RAG stands for Retrieval-Augmented Generation."
• Retrieved Context Displayed:
  1. "Retrieval-Augmented Generation (RAG) is a popular technique used in LangChain."
  2. "RAG combines document retrieval with language model answer generation into a linear pipeline."
• Verification: The output confirms that the retriever extracted the exact matching passages, and the LLM generated a precise answer grounded in those passages.

6. Summary Comparison Matrix

Pipeline Stage	Module / Library Used	Execution Role	Primary Objective
1. Data Ingestion	TextLoader / PyPDFLoader	Reads raw document files.	Import external enterprise data.
2. Text Splitting	RecursiveCharacterTextSplitter	Splits documents into chunks.	Increases search speed & performance.
3. Vector Store	FAISS / ChromaDB	Stores high-dimensional vectors.	Enables fast similarity indexing.
4. Retriever	vector_db.as_retriever(k=2)	Extracts top-k matching chunks.	Fetches relevant document context.
5. LLM Engine	Qwen2 0.5B via pipeline	Generates text answers.	Applies reasoning over context.
6. RAG Chain	RunnablePassthrough & LCEL	Links retriever, prompt, and LLM.	Provides end-to-end question answering.