Online Robotics Classes for Kids in Belgrade

Chapter 179 — Vector Databases and Similarity Search Explained for RAG

1. Introduction & Why Relational Databases Fall Short for AI

In Retrieval-Augmented Generation (RAG) and semantic search systems, text is converted into high-dimensional numerical vectors (embeddings). While traditional relational databases (such as MySQL or PostgreSQL) excel at exact keyword lookups and structured SQL queries, they are not optimized for comparing multi-dimensional vector spaces.

  ┌─────────────────────────────────────────────────────────────┐
  │         Relational DBs vs. Vector Databases (Vector DBs)    │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Relational Database (SQL)     │ Vector Database (Vector DB) │
  ├───────────────────────────────┼─────────────────────────────┤
  │ • Exact keyword matching.     │ • Contextual & Semantic     │
  │ • Structured tabular rows &   │   similarity matching.      │
  │   columns.                    │ • High-dimensional vector   │
  │ • Suboptimal for vector       │   indexing & retrieval.     │
  │   similarity searches.        │ • Fast Approximate Nearest  │
  │                               │   Neighbors (ANN).          │
  └───────────────────────────────┴─────────────────────────────┘
    

Core Advantages of Vector Databases:

• Specialized Retrieval Engine: Vector databases are built specifically to handle high-dimensional vector retrieval and similarity comparisons at ultra-fast speeds.
• Fast Approximate Nearest Neighbor (ANN): Instead of scanning every vector in the database linearly, vector DBs use Approximate Nearest Neighbor (ANN) algorithms (inspired by k-Nearest Neighbors / k-NN) to rapidly identify closest matching vectors.
• Scalability: Essential for building scalable RAG systems that process queries from large numbers of concurrent users across massive document repositories.

2. Vector Database Landscape & Ecosystem

The modern vector database ecosystem offers both lightweight local libraries and fully managed cloud platforms:

  ┌─────────────────────────────────────────────────────────────┐
  │                 Vector Database Options & Tools             │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Lightweight Local Options     │ Managed Cloud Platforms     │
  ├───────────────────────────────┼─────────────────────────────┤
  │ • FAISS (Facebook AI          │ • Pinecone                  │
  │   Similarity Search).         │ • Weaviate.                 │
  │ • ChromaDB (Open-source       │ • Milvus.                   │
  │   LangChain integration).     │ • Fully managed cloud       │
  │                               │   infrastructure with zero  │
  │                               │   ops overhead.             │
  └───────────────────────────────┴─────────────────────────────┘
    

• FAISS (Facebook AI Similarity Search): Developed by Meta, FAISS is an extremely fast, lightweight vector indexing library ideal for local setups and quick prototyping.
• ChromaDB: An open-source vector database designed for seamless integration with the LangChain framework, making it a preferred choice for production software pipelines due to its scalability.
• Managed Cloud Platforms (Pinecone, Weaviate, Milvus): Fully managed cloud vector database solutions that handle infrastructure management and scaling automatically.

3. Storing Embeddings, Metadata, & Semantic Retrieval

When a document chunk is processed and stored in a vector database, the record contains two main components:

  ┌─────────────────────────────────────────────────────────────┐
  │                 Vector DB Storage & Query Flow              │
  ├─────────────────────────────────────────────────────────────┤
  │  Record Components:                                         │
  │    1. Numerical Vector Embedding (e.g., 384 dimensions)     │
  │    2. Associated Metadata (Source, Title, Page Content)     │
  │                                                             │
  │  Retrieval Mechanism:                                       │
  │    User Query ──► Query Embedding Vector ──► Vector DB      │
  │                                                   │         │
  │  Top-k Matches ◄── Cosine Similarity Metric ◄─────┘         │
  └─────────────────────────────────────────────────────────────┘
    

1. Embedding Vector: The high-dimensional numerical representation of the text chunk.
2. Metadata: Additional contextual details, such as document title, source URL, page numbers, and raw text content.

Semantic Search Execution:

When a user submits a query, the query string is converted into a vector embedding. The vector database performs a similarity search (often using Cosine Similarity) to return the top-k most contextually relevant document chunks. Unlike traditional keyword matching, semantic search retrieves documents based on shared real-world concepts and meaning.

4. Document Chunking Trade-Offs & Database Optimization

Before storing documents in a vector database, long text files must be split into smaller chunks. Selecting the appropriate chunk size requires balancing recall accuracy and retrieval speed:

  ┌─────────────────────────────────────────────────────────────┐
  │               Document Chunk Size Trade-Offs                │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Large Chunk Size              │ Small Chunk Size            │
  ├───────────────────────────────┼─────────────────────────────┤
  │ • Higher recall accuracy.     │ • Faster retrieval speed.   │
  │ • Slower retrieval speed.     │ • Lower recall accuracy.    │
  └───────────────────────────────┴─────────────────────────────┘
    

• Context Preservation via Overlap: Splitting text purely by character length does not convey semantic awareness. Adding a chunk overlap (e.g., 20 characters) between consecutive chunks preserves context across boundaries without losing critical information.
• Re-Indexing Optimization: As vector databases grow over time, executing re-indexing operations re-optimizes index structures to maintain high retrieval speeds and prevent performance degradation.

5. Complete Hands-On VS Code Implementation

Below is the complete, runnable Python script demonstrating how to load a document, split it into overlapping chunks using RecursiveCharacterTextSplitter, generate embeddings with HuggingFaceEmbeddings, and index them into a FAISS vector database using LangChain:

# =====================================================================
# Document Chunking & FAISS Vector DB Indexing Script (Topic 179)
# =====================================================================

import os
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS

# 1. Define Sample Long Text Document Content
sample_text = """
Retrieval-Augmented Generation (RAG) connects Large Language Models to external knowledge bases.
By converting text into high-dimensional vector embeddings, RAG enables semantic similarity search.
Vector databases like FAISS and ChromaDB store these embeddings alongside metadata for rapid retrieval.
Approximate Nearest Neighbor (ANN) algorithms allow vector databases to search millions of documents in milliseconds.
Chunking long documents with overlap ensures that contextual information is preserved across text boundaries.
"""

# Save sample text to a temporary text file for loading
file_path = "sample_document.txt"
with open(file_path, "w", encoding="utf-8") as f:
    f.write(sample_text.strip())

# 2. Load Document using LangChain TextLoader
loader = TextLoader(file_path, encoding="utf-8")
documents = loader.load()

# 3. Configure RecursiveCharacterTextSplitter
# Setting chunk_size=100 characters and chunk_overlap=20 characters
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=100,
    chunk_overlap=20,
    length_function=len,
    separators=["\\n", " "]
)

# Split document into chunks
doc_splits = text_splitter.split_documents(documents)
print(f"=== DOCUMENT SPLITTING COMPLETE ===")
print(f"Total Chunks Created: {len(doc_splits)}\\n")

for idx, chunk in enumerate(doc_splits):
    print(f"--- Chunk {idx} ---")
    print(f"Content: {chunk.page_content}")

# 4. Instantiate Hugging Face Sentence Transformer Embeddings
embedding_model = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2"
)

# 5. Index Chunks into FAISS Vector Database
print("\\nCreating Embeddings and Indexing into FAISS Vector DB...")
vector_db = FAISS.from_documents(doc_splits, embedding_model)

# 6. Verify Vector DB Creation
print("\\n=== FAISS VECTOR DATABASE INDEXING SUCCESSFUL ===")
print(f"Total Indexed Vectors in Database: {len(doc_splits)}")

# Clean up temporary text file
if os.path.exists(file_path):
    os.remove(file_path)
    

6. Summary Comparison Table

Component / Parameter	Value / Setting	Functional Description in RAG Architecture
Search Engine Type	Vector DB (ANN / k-NN)	Enables fast contextual & semantic retrieval.
Prototyping DB	FAISS	Lightweight, fast local vector indexing library.
Production DB	ChromaDB	Scalable open-source vector store for LangChain.
Cloud Vector DBs	Pinecone, Weaviate, Milvus	Fully managed cloud vector infrastructure.
Text Splitter	RecursiveCharacterTextSplitter	Splits document text into manageable chunks.
Chunk Configuration	chunk_size=100, chunk_overlap=20	Balances recall accuracy, speed, and context.
Similarity Metric	Cosine Similarity	Measures angular distance between embedding vectors.