AI Courses for Kids in Timisoara

Chapter 174 — LangChain Memory and Context Explained: Help AI Keep Track of a Conversation

1. Introduction: What is Memory in LangChain & Why is Context Essential?

In standalone Large Language Model (LLM) APIs, interactions are strictly stateless—the model evaluates each prompt as an isolated event and retains no memory of prior turns in a dialogue. Building interactive chatbots, personal AI assistants, and multi-turn conversational workflows requires preserving state across successive interactions.

In LangChain, Memory refers to the framework mechanism that stores, manages, and retrieves past conversation history.

  ┌─────────────────────────────────────────────────────────────┐
  │                 Stateless vs. Memory-Based AI               │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Stateless Raw LLM Call        │ LangChain Memory-Based Chain│
  ├───────────────────────────────┼─────────────────────────────┤
  │ • Treats every prompt in      │ • Preserves user chat       │
  │   isolation.                  │   history across turns.     │
  │ • Forgets user details        │ • Retrieves previous        │
  │   provided in earlier turns.  │   context for answers.      │
  │ • Cannot maintain multi-turn  │ • Enables personalized AI   │
  │   conversations.              │   assistants.               │
  └───────────────────────────────┴─────────────────────────────┘
    

Core Functions of LangChain Memory:

• Preserving Conversational History: Captures and stores user inputs and model outputs in Python data structures or external storage during runtime.
• Contextual Response Generation: Injects past dialogue context into the prompt template so the LLM can answer subsequent questions accurately based on previous turns.
• Personalization: Retains user preferences and personal context over extended interactions.

2. Core Types of LangChain Memory

LangChain provides several distinct memory strategies tailored to different session lengths, token budgets, and persistence requirements:

  ┌─────────────────────────────────────────────────────────────┐
  │                  4 Core Memory Types in LangChain           │
  ├─────────────────────────────────────────────────────────────┤
  │  1. ConversationBufferMemory                                │
  │     • Stores the COMPLETE chat history in a buffer.         │
  │     • Default option for short sessions.                    │
  │                                                             │
  │  2. ConversationSummaryMemory                               │
  │     • Generates and stores a SUMMARY of past chats.         │
  │     • Ideal for long sessions with low token overhead.      │
  │                                                             │
  │  3. ConversationBufferWindowMemory                          │
  │     • Retains a sliding WINDOW of recent chats.             │
  │     • Deletes older interactions to reduce memory size.     │
  │                                                             │
  │  4. Persistent Database Memory                              │
  │     • Stores chat logs in external databases.               │
  │     • Uses SQLite, MongoDB, or Vector Databases.            │
  └─────────────────────────────────────────────────────────────┘
    

Type 1: ConversationBufferMemory

• Mechanism: Stores the exact, complete transcript of all previous chat messages in an in-memory buffer.
• Default Usage: Serves as the standard default option in LangChain for handling simple conversational pipelines.
• Best Use Case: Short chat sessions where the total turn count is low.
• Limitations: In longer chat sessions, preserving the entire transcript increases prompt size significantly. This can exceed model token length limits and increase API processing costs.

Type 2: ConversationSummaryMemory

• Mechanism: Rather than storing raw transcripts, it uses an LLM to continuously summarize previous dialogue turns into a concise summary block.
• Best Use Case: Long, extended chat sessions where maintaining context is necessary without exceeding token budgets.
• Advantage: Dramatically reduces memory overhead and token consumption while preserving key facts and context.

Type 3: ConversationBufferWindowMemory

• Mechanism: Maintains a sliding window containing only the most recent k interactions.
• Pruning Logic: Automatically drops older conversation turns as new messages arrive, keeping the active memory size small and bounded.

Type 4: Persistent Database Memory

• Mechanism: Persists chat history across application restarts by saving logs into external databases such as SQLite, MongoDB, or Vector Databases.
• Best Use Case: Large-scale, production-grade AI assistants that require long-term user personalization and persistent memory across logins.

3. Hands-On Implementation 1: Building a Buffer Memory Chain

To implement memory in Python, we import memory components from langchain_classic.memory (or langchain.memory), configure an LLM instance (e.g., gpt-4o-mini with temperature = 0.3), and construct a prompt template using ChatPromptTemplate and MessagesPlaceholder.

  ┌─────────────────────────────────────────────────────────────┐
  │              Memory-Based Chain Construction Pipeline       │
  ├─────────────────────────────────────────────────────────────┤
  │  1. Memory Object ──► ConversationBufferMemory              │
  │                       (return_messages=True)                │
  │                                   │                         │
  │                                   ▼                         │
  │  2. Prompt Template ──► ChatPromptTemplate +                │
  │                         MessagesPlaceholder("history")      │
  │                                   │                         │
  │                                   ▼                         │
  │  3. Input Pass-Through ──► RunnablePassThrough              │
  │                            (passes input unchanged)         │
  │                                   │                         │
  │                                   ▼                         │
  │  4. Sequential Chain ──► History Loader | Input | Prompt | LLM│
  └─────────────────────────────────────────────────────────────┘
    

Complete Python Script (ConversationBufferMemory):

# =====================================================================
# ConversationBufferMemory Implementation in LangChain (Topic 174)
# =====================================================================

import os
from dotenv import load_dotenv

# Import memory modules from langchain_classic (or langchain.memory)
from langchain_classic.memory import ConversationBufferMemory
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables import RunnablePassThrough
from langchain_openai import ChatOpenAI

# 1. Load API Key
load_dotenv()

# 2. Instantiate LLM Model (gpt-4o-mini with 30% temperature)
llm = ChatOpenAI(
    model_name="gpt-4o-mini",
    temperature=0.3
)

# 3. Instantiate ConversationBufferMemory
# Setting return_messages=True converts history into ChatMessage objects
memory = ConversationBufferMemory(return_messages=True)

# 4. Define Chat Prompt Template with MessagesPlaceholder for history
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful chatbot. Remember the user's preferences and behave like a human."),
    MessagesPlaceholder(variable_name="history"),  # Placeholder for chat history
    ("human", "{input}")                           # Placeholder for current user input
])

# 5. Helper Function to Load History Variables
def load_history(_):
    return memory.load_memory_variables({})["history"]

# 6. Build Memory-Based Chain using RunnablePassThrough
# RunnablePassThrough passes raw input forward without modification
chain = (
    {
        "history": load_history,
        "input": RunnablePassThrough()
    }
    | prompt
    | llm
)

# 7. First Interaction (Message 1)
msg1 = "My name is Yasir and I teach AI to university students."
res1 = chain.invoke(msg1)

# Save context of turn 1 to memory
memory.save_context({"input": msg1}, {"output": res1.content})
print("=== RESPONSE 1 ===")
print(res1.content)

# 8. Second Interaction (Message 2 - Testing Context Retrieval)
msg2 = "What do you know about me?"
res2 = chain.invoke(msg2)

# Save context of turn 2 to memory
memory.save_context({"input": msg2}, {"output": res2.content})
print("\n=== RESPONSE 2 ===")
print(res2.content)
    

Execution Result & Context Verification:

• Input 1: "My name is Yasir and I teach AI to university students."
• Output 1: "Nice to meet you Yasir! Teaching AI sounds fascinating..."
• Input 2: "What do you know about me?"
• Output 2: "I know that your name is Yasir and that you teach AI to university students."
• Verification: The model retrieves past context from ConversationBufferMemory to answer the second turn correctly.

4. Hands-On Implementation 2: Building a Summary Memory Chain

For long conversations, ConversationSummaryMemory stores a running summary of past interactions generated by the LLM.

# =====================================================================
# ConversationSummaryMemory Implementation in LangChain (Topic 174)
# =====================================================================

from langchain_classic.memory import ConversationSummaryMemory
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables import RunnablePassThrough
from langchain_openai import ChatOpenAI
from dotenv import load_dotenv

load_dotenv()

# 1. Initialize LLM
llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0.3)

# 2. Instantiate ConversationSummaryMemory with LLM instance
summary_memory = ConversationSummaryMemory(llm=llm, return_messages=True)

# 3. Define Prompt Template
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant. Maintain a summary of the conversation so far."),
    MessagesPlaceholder(variable_name="history"),
    ("human", "{input}")
])

# 4. Helper Function to Load Summary History
def load_summary(_):
    return summary_memory.load_memory_variables({})["history"]

# 5. Build Chain
summary_chain = (
    {
        "history": load_summary,
        "input": RunnablePassThrough()
    }
    | prompt
    | llm
)

# 6. Execute Multi-Turn Conversation
turn1 = "My name is Yasir. I teach Data Science with Python."
res1 = summary_chain.invoke(turn1)
summary_memory.save_context({"input": turn1}, {"output": res1.content})

turn2 = "I also teach Robotics."
res2 = summary_chain.invoke(turn2)
summary_memory.save_context({"input": turn2}, {"output": res2.content})

# 7. Ask Model to Recall Summarized Facts
turn3 = "What should you remember about me?"
res3 = summary_chain.invoke(turn3)
print("=== SUMMARY MEMORY RECALL RESPONSE ===")
print(res3.content)
    

Execution Result:

• Recalled Output: "I should remember that your name is Yasir and you teach Data Science with Python as well as Robotics."
• Verification: Instead of retaining raw message strings, the memory module compiles the statements into a unified summary block.

5. Summary Comparison Matrix

Memory Type	Storage Mechanism	Token Consumption	Ideal Use Case
ConversationBufferMemory	Stores full chat transcript.	High in long sessions.	Short chat interactions.
ConversationSummaryMemory	Stores LLM-generated summary.	Low & constant.	Extended, multi-turn chat sessions.
ConversationBufferWindowMemory	Retains sliding window of k turns.	Fixed & bounded.	Recent-context-only workflows.
Persistent Database Memory	Saves logs to DB (SQLite, MongoDB).	Independent of RAM.	Enterprise AI assistants.