Python Coding Classes for Children in Daugavpils

Chapter 157 — Build a Basic Recurrent Neural Network Step by Step

1. Introduction & Environment Setup in Google Colab

Recurrent Neural Networks (RNNs) are specialized deep learning models built specifically to process sequential data step by step. Unlike feedforward architectures, RNNs maintain an internal hidden state that captures past information by incorporating feedback from previous time steps. This makes RNNs ideal for tasks such as text classification and time-series prediction.

To build and train an RNN efficiently, we use Google Colab, a free cloud-based platform that provides access to GPU hardware accelerators.

  ┌─────────────────────────────────────────────────────────────┐
  │                 Google Colab Environment Setup              │
  ├─────────────────────────────────────────────────────────────┤
  │  1. Hardware Acceleration Setup:                            │
  │     • Runtime ──► Change runtime type ──► Select T4 GPU     │
  │     • Shifts execution from local CPU to cloud GPU          │
  │                                                             │
  │  2. Connectivity Requirements:                              │
  │     • Requires an active internet connection to communicate │
  │       continuously with Google's cloud servers              │
  └─────────────────────────────────────────────────────────────┘
    

Importing Required Python Libraries:

We begin by importing TensorFlow, Keras model components, and data manipulation libraries:

• tensorflow (tf): Deep learning library providing GPU-accelerated computing.
• tf.keras.models.Sequential: Linear stack model structure used for time-series and sequential data.
• tf.keras.layers.SimpleRNN: Basic recurrent layer that extracts features across sequential time steps.
• tf.keras.layers.Dense: Fully connected layer used for output classification.
• pandas & numpy: Essential tools for CSV data ingestion and array operations.

2. Data Ingestion, Preprocessing & 3D Tensor Reshaping

A. File Upload & Session Storage:

In Google Colab, local data files must be uploaded to the temporary session storage (e.g., module_157_data.csv). Note that session storage files are temporary and cleared once the browser session ends; for permanent storage, files can be linked via Google Drive.

  ┌─────────────────────────────────────────────────────────────┐
  │               Dataset Structure & Reshaping                 │
  ├─────────────────────────────────────────────────────────────┤
  │  Raw CSV Dataset Dimensions:                                │
  │    • 2,000 Total Samples                                    │
  │    • 10 Sequential Features (step_0 through step_9)         │
  │    • 1 Binary Target Column (0 and 1 labels)                │
  │                                                             │
  │  2D to 3D Matrix Reshaping Requirement:                     │
  │    • Raw 2D Input Shape  : (2000, 10)                       │
  │    • Reshaped 3D Shape   : (2000, 10, 1)                    │
  │    • Keras RNN Target    : (Samples, Timesteps, Features)   │
  └─────────────────────────────────────────────────────────────┘
    

B. Feature Separation & 3D Reshaping:

1. Feature/Target Separation: The target label column is dropped from the input feature set X and stored separately in variable y.
2. 3D Reshaping Requirement: Keras RNN layers require input data formatted as a 3D tensor with shape (batch_size, timesteps, features). We convert our 2D array of shape (2000, 10) into a 3D tensor of shape (2000, 10, 1) by appending a feature dimension using .reshape().

3. Building the RNN Architecture with Keras Sequential API

We construct our deep learning network by initializing a Sequential model and stacking a SimpleRNN layer followed by a Dense classification layer:

  ┌─────────────────────────────────────────────────────────────┐
  │                   2-Layer RNN Architecture                  │
  ├─────────────────────────────────────────────────────────────┤
  │                                                             │
  │   Input Tensor Shape (10 Timesteps × 1 Feature)             │
  │        │                                                    │
  │        ▼                                                    │
  │   SimpleRNN Layer       ──► 32 Units / Neurons              │
  │                             Extracts sequential features    │
  │                             Output Shape: (None, 32)        │
  │        │                                                    │
  │        ▼                                                    │
  │   Dense Layer           ──► 1 Neuron, Sigmoid Activation    │
  │                             Binary Classifier (Yes/No)      │
  │                             Output Shape: (None, 1)         │
  └─────────────────────────────────────────────────────────────┘
    

• SimpleRNN(32, input_shape=(10, 1)): Contains 32 recurrent units receiving 10 timesteps of 1 feature each. At each timestep, it combines the current input with the previous hidden state to extract temporal features.
• Dense(1, activation='sigmoid'): A fully connected layer containing 1 neuron with Sigmoid activation, ideal for binary classification (predicting class probability between 0 and 1).

4. Model Summary & Parameter Breakdown (model.summary())

Executing model.summary() displays the complete layer structure and parameter count breakdown:

  ┌─────────────────────────────────────────────────────────────────────────┐
  │                        Model Summary Breakdown                          │
  ├──────────────────────────────┬───────────────────┬──────────────────────┤
  │ Layer (Type)                 │ Output Shape      │ Param #              │
  ├──────────────────────────────┼───────────────────┼──────────────────────┤
  │ simple_rnn (SimpleRNN)       │ (None, 32)        │ 1,088                │
  │ dense (Dense)                │ (None, 1)         │ 33                   │
  ├──────────────────────────────┴───────────────────┴──────────────────────┤
  │ Total params: 1,121 (4.38 KB)                                           │
  │ Trainable params: 1,121                                                 │
  │ Non-trainable params: 0                                                 │
  └─────────────────────────────────────────────────────────────────────────┘
    

Parameter Mathematical Formulas:

1. SimpleRNN Layer Parameters: Params = (Input Dim × Units) + (Units × Units) + Biases = (1 × 32) + (32 × 32) + 32 = 32 + 1024 + 32 = 1,088 parameters.
2. Dense Layer Parameters: Params = (Input Units × Output Units) + Biases = (32 × 1) + 1 = 33 parameters.
3. Total Trainable Parameters: 1088 + 33 = 1,121 trainable parameters (~4.38 KB).

5. Model Compilation & Hyperparameters

Before training, we configure the model hyperparameters using model.compile():

  ┌─────────────────────────────────────────────────────────────┐
  │                Hyperparameter Configurations                │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Hyperparameter                │ Configured Setting          │
  ├───────────────────────────────┼─────────────────────────────┤
  │ Optimizer                     │ Adam (adam)                 │
  │ Loss Function                 │ Binary Cross-Entropy        │
  │                               │ (binary_crossentropy)       │
  │ Performance Metric            │ Accuracy (accuracy)         │
  └───────────────────────────────┴─────────────────────────────┘
    

• Optimizer: Adam (Adaptive Moment Estimation), which dynamically adjusts learning rates using feature moments.
• Loss Function: Binary Cross-Entropy, the standard loss metric for binary classification tasks.
• Metric: Accuracy, used to track classification accuracy during training.

6. Training & Evaluation Dynamics (fit & evaluate)

We train the network using model.fit() with the following parameters:

• Epochs: 20 Epochs.
• Batch Size: 32 samples per mini-batch.
• Validation Split: 0.2 (20%), reserving 80% for training and 20% for validation.

  ┌─────────────────────────────────────────────────────────────┐
  │              Training Metrics Progression Log               │
  ├─────────────────────────────────────────────────────────────┤
  │  • Epoch 1  : Accuracy = 52.94% | Loss = 0.68               │
  │  • Epoch 2  : Accuracy = 77.00% | Loss = 0.48               │
  │  • Epoch 3  : Accuracy = 84.00% | Loss = 0.38               │
  │  • Epoch 20 : Accuracy = ~95.00%| Loss = ~0.16              │
  │                                                             │
  │  Final Evaluation (model.evaluate):                         │
  │    • Final Test Accuracy = 94.90% (~95%)                    │
  └─────────────────────────────────────────────────────────────┘
    

Training Execution:

Leveraging Colab's GPU runtime enables 20 epochs to complete in fractions of a second. Training accuracy starts at 52.94% in Epoch 1, reaches 77% in Epoch 2, 84% in Epoch 3, and stabilizes near 95% by Epoch 20. Simultaneously, loss drops from 0.68 down to ~0.16.

Calling model.evaluate() delivers a final test accuracy of 94.90% (~95%), demonstrating the effectiveness of basic RNNs for sequential classification.

7. Complete Executable Python Code

Below is the complete, end-to-end Python script executed in Google Colab:

# =====================================================================
# Complete Basic RNN Script using TensorFlow & Keras (Topic 157)
# =====================================================================

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import SimpleRNN, Dense
import pandas as pd
import numpy as np

# 1. Load Dataset from CSV File
df_loaded = pd.read_csv('module_157_data.csv')

# 2. Separate Features (X) and Target Labels (y)
X = df_loaded.drop(columns=['target'], axis=1).values
y = df_loaded['target'].values

# 3. Reshape 2D Data (2000, 10) into 3D Tensor (2000, 10, 1) for Keras RNN
X = X.reshape((X.shape[0], X.shape[1], 1))
print(f"Reshaped Input Tensor Shape: {X.shape}")  # (2000, 10, 1)

# 4. Construct RNN Architecture using Sequential API
model = Sequential([
    SimpleRNN(32, input_shape=(10, 1)),
    Dense(1, activation='sigmoid')
])

# 5. Display Model Summary & Parameter Breakdown (1,121 Parameters)
model.summary()

# 6. Compile Model Hyperparameters
model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)

# 7. Train RNN Model for 20 Epochs
history = model.fit(
    X, y,
    epochs=20,
    batch_size=32,
    validation_split=0.2
)

# 8. Evaluate Final Model Performance
loss, accuracy = model.evaluate(X, y, verbose=0)
print(f"\n--- Final Model Evaluation ---")
print(f"Final Test Accuracy: {accuracy * 100:.2f}%")  # Output: 94.90%
    

8. Summary Pipeline Table

Stage	Function / Method	Configuration / Operation	Output Result / Metric
GPU Setup	Colab Runtime Change	Enables T4 GPU hardware acceleration.	Fast execution speed.
Ingestion	pd.read_csv()	Loads dataset containing 10 feature steps & target.	2,000 samples loaded.
Preprocessing	X.reshape()	Converts 2D (2000, 10) matrix to 3D (2000, 10, 1) tensor.	Compatible Keras 3D shape.
Architecture	Sequential()	Stack SimpleRNN(32) + Dense(1, activation='sigmoid').	1,121 total parameters.
Compilation	model.compile()	Sets adam optimizer, binary_crossentropy loss, accuracy.	Configured training pipeline.
Training	model.fit()	20 Epochs, batch_size=32, 20% validation split.	Accuracy rises from 52.9% to ~95%.
Evaluation	model.evaluate()	Evaluates final accuracy with verbose=0.	94.90% Final Accuracy.