Online Robotics Classes for Kids in Bratislava

Chapter 182 — Build and Deploy an AI Chatbot with LangChain and Flask

1. Introduction & Architectural Overview

While previous modules focused on building LangChain chains, memory systems, and tools in standalone scripts, deploying artificial intelligence applications for real-world production requires wrapping them inside a scalable web server architecture.

Flask is a lightweight, comfortable, and highly efficient Python web framework ideal for serving Large Language Models (LLMs) to front-end clients via RESTful APIs.

  ┌─────────────────────────────────────────────────────────────┐
  │            LangChain + Flask System Architecture            │
  ├─────────────────────────────────────────────────────────────┤
  │  Client Request (Web UI / Postman / AJAX POST / JSON)       │
  │                             │                               │
  │                             ▼                               │
  │  Flask Server Route (`/chat` REST Endpoint)                │
  │                             │                               │
  │                             ▼                               │
  │  LangChain Pipeline (LLM + Tokenizer + Chat Memory)         │
  │                             │                               │
  │                             ▼                               │
  │  JSON Response (`jsonify({"status": "success", ...})`)      │
  └─────────────────────────────────────────────────────────────┘
    

Core Advantages of Using Flask with LangChain:

• Lightweight & Scalable: Flask integrates seamlessly with Python libraries without heavy framework overhead, making chatbot interfaces scalable.
• RESTful API Serving: Serves the underlying LangChain pipeline as an HTTP endpoint that can accept JSON payloads and return structured responses.
• Decoupled Frontend & Backend: Allows developers to build independent frontends (Web UIs, mobile apps, or testing tools like Postman) that communicate with the Flask backend asynchronously via AJAX.
• End-to-End Interaction Flow: The client sends a user query via an HTTP POST request to a Flask endpoint. Flask passes the query to the LangChain pipeline (combining the LLM model and memory). The LLM generates a response, which Flask returns as a JSON object to be displayed on the user's screen.

2. Key Flask & LangChain Helper Functions

Before defining web routes, two essential helper functions are created to format incoming prompts and clean raw LLM responses.

  ┌─────────────────────────────────────────────────────────────┐
  │                 Helper Processing Functions                 │
  ├─────────────────────────────────────────────────────────────┤
  │                                                             │
  │   User Text + History  ──► `create_qwen_prompt()`           │
  │                            • Sets System Role ("Helpful AI")│
  │                            • Injects Chat History           │
  │                            • Applies Chat Template          │
  │                                       │                     │
  │                                       ▼                     │
  │   Raw LLM Output       ──► `clean_qwen_response()`          │
  │                            • Strips Prompt Repetition       │
  │                            • Removes Extra Tags & Whitespace│
  └─────────────────────────────────────────────────────────────┘
    

A. Prompt Construction Function (create_qwen_prompt):

• Role Assignment: Configures a system message assigning the role: "You are a helpful AI assistant".
• History Injection: Accepts conversation history as an optional parameter (defaulting to empty if None) to preserve chat context across turns.
• Chat Template Application: Applies the model tokenizer's chat template (apply_chat_template) to structure the system instructions, past conversation history, and user input into proper prompt tokens.

B. Response Cleaning Function (clean_qwen_response):

• Stripping Prompt Repetition: Checks if the LLM repeated the input prompt in its raw output string and removes it.
• Removing Unwanted Tags: Strips leftover tokens, special formatting tags, and leading/trailing whitespace before returning the clean response.

3. Setting Up the Flask Web Server & RESTful API Endpoints

Setting up the Flask web application involves instantiating the app object, enabling Cross-Origin Resource Sharing (CORS), configuring error logging, and defining API routes.

# =====================================================================
# Flask Setup and RESTful Route Definition (Topic 182)
# =====================================================================

from flask import Flask, request, jsonify
from flask_cors import CORS
import logging

# 1. Instantiate Flask Application Object
app = Flask(__name__)

# 2. Enable Cross-Origin Resource Sharing (CORS) for Global AJAX
CORS(app)

# 3. Configure Flask Logging to Capture Server Errors
logging.basicConfig(level=logging.ERROR)

# 4. Define RESTful API Route for Chatbot Interaction
@app.route('/chat', methods=['POST'])
def chat():
    """POST endpoint receiving user queries and returning AI responses."""
    try:
        # Extract JSON payload from incoming HTTP request
        data = request.json
        user_message = data.get("message", "")

        # Format input prompt using helper function
        formatted_prompt = create_qwen_prompt(user_text=user_message, history=None)

        # Invoke LangChain LLM model to generate raw response
        raw_response = llm.invoke(formatted_prompt)

        # Clean response using helper function
        ai_response = clean_qwen_response(response=raw_response, prompt=formatted_prompt)

        # Return success status and cleaned response formatted as JSON
        return jsonify({
            "status": "success",
            "response": ai_response
        }), 200

    except Exception as e:
        # Return error status and exception details if an error occurs
        return jsonify({
            "status": "error",
            "message": str(e)
        }), 500
    

Key Implementation Details:

• methods=['POST']: The HTTP POST method is used because chat queries involve complex JSON payloads containing text strings and conversation history.
• CORS(app): Global CORS configuration allows web browsers to make asynchronous AJAX requests from external origins or frontends.
• jsonify(): Converts Python dictionaries into JSON responses accompanied by standard HTTP status codes (such as 200 OK for success).

4. Multithreaded Server Execution & Background Worker Logic

In Python interactive environments or complex applications, launching a web server directly on the main thread blocks subsequent code execution. To keep the server running continuously in the background, Flask is executed inside a background daemon thread.

  ┌─────────────────────────────────────────────────────────────┐
  │               Multithreaded Server Execution                │
  ├─────────────────────────────────────────────────────────────┤
  │  Main Program Thread ──► Launches Worker Thread             │
  │                          (Target: `run_flask`, Port: 5000)  │
  │                          `daemon = True`, `thread.start()` │
  │                                    │                        │
  │                                    ▼                        │
  │  3-Second Delay (`time.sleep(3)`) ──► Waits for Server Ready │
  └─────────────────────────────────────────────────────────────┘
    

A. Multi-Threading Concepts:

• CPU Cores & Threads: Modern multi-core processors can execute multiple processes or threads simultaneously.
• threading.Thread: Spawns a background thread targeting the Flask server function (run_flask) running on local IP port 5000.
• daemon = True: Sets the thread as a background daemon process so it shuts down automatically when the main program terminates.
• Startup Timer (time.sleep(3)): Introduces a 3-second delay to allow the Flask web server to complete initialization before accepting incoming API queries.

5. Testing & Verifying the Chatbot REST API

Once the multithreaded Flask server is active, the API endpoint can be tested programmatically using Python's requests library or external tools like Postman.

# =====================================================================
# Testing the Chatbot API Endpoint Programmatically (Topic 182)
# =====================================================================

import requests

# Define target Flask API endpoint URL
api_url = "http://127.0.0.1:5000/chat"

# Prepare JSON payload containing user prompt
payload = {
    "message": "Introduce yourself briefly"
}

try:
    # Send HTTP POST request to Flask endpoint
    response = requests.post(api_url, json=payload)

    # Check if HTTP status code is 200 (Success)
    if response.status_code == 200:
        response_data = response.json()
        print("=== CHATBOT RESPONSE RECEIVED (STATUS 200) ===")
        print(f"User Prompt: {payload['message']}")
        print(f"AI Response: {response_data['response']}")
    else:
        print(f"API Error! Status Code: {response.status_code}")
        print(response.text)

except Exception as e:
    print(f"Connection Failed: {str(e)}")
    

Verification Results:

When the POST request is executed against the endpoint, the server processes the query and returns an HTTP 200 OK response containing the chatbot's introduction:

"I am an artificial intelligence designed to assist with various tasks, including answering questions, providing information, and assisting with communication. I am available 24/7..."

This confirms that the LangChain pipeline has been successfully deployed as a functional web application endpoint.

6. Summary Comparison Matrix

Component / Function	Library / Tool	Primary Technical Function	Output / Response
Web Framework	Flask	Provides lightweight web server architecture.	RESTful web application.
CORS Middleware	flask_cors.CORS	Enables cross-origin resource sharing for AJAX.	Global frontend access.
API Endpoint Route	@app.route('/chat')	Maps HTTP POST requests to handler logic.	Accepts JSON request payloads.
Prompt Helper	create_qwen_prompt	Sets system role, context, and chat templates.	Formatted model prompt string.
Response Cleaner	clean_qwen_response	Removes prompt echo, tags, and whitespace.	Clean AI answer text.
Response Formatter	jsonify()	Formats dictionaries into JSON API responses.	JSON + HTTP Status 200.
Server Threading	threading.Thread	Runs Flask as a background daemon process.	Non-blocking execution.
API Testing Tool	Postman / requests	Sends test HTTP POST payloads to verify server.	Evaluates server health.