Python Programming for Students in Bologna

Chapter 152 — Build a Neural Network with TensorFlow and Keras

1. Introduction & Google Colab Cloud Environment

When training deep learning models, datasets and mathematical operations require massive computational throughput. While standard local computers often possess only CPUs, deep learning models perform best when accelerated by Graphics Processing Units (GPUs).

  ┌─────────────────────────────────────────────────────────────┐
  │                 Google Colab Infrastructure                 │
  ├─────────────────────────────────────────────────────────────┤
  │  • Free GPU Cloud Environment                               │
  │    - Provides free access to T4 GPUs for lightweight tests  │
  │    - Eliminates local GPU hardware prerequisites            │
  │                                                             │
  │  • Runtime Configuration:                                   │
  │    - Runtime ──► Change runtime type                        │
  │    - Hardware Accelerator ──► Select T4 GPU                 │
  │                                                             │
  │  • File Compatibility:                                      │
  │    - Executes Jupyter Notebook (.ipynb) and Python (.py)    │
  └─────────────────────────────────────────────────────────────┘
    

Google Colab provides a cloud-based environment where users can access GPU hardware accelerators (such as the T4 GPU) for free.

Setting Up the Environment:

1. Accessing Colab: Navigate to colab.research.google.com and log in with a Gmail account.
2. Hardware Accelerator Selection: Select Runtime → Change runtime type, select Python, and choose T4 GPU.
3. GPU Parallelization: Selecting a GPU restarts the hardware engine and enables parallel tensor computation, accelerating neural network training.

2. Data Ingestion & Preprocessing with Scikit-Learn

Before constructing the neural network, raw input data must be ingested, split, and standardized. For this hands-on implementation, we utilize the Wine Dataset (load_wine).

  ┌─────────────────────────────────────────────────────────────┐
  │            Data Ingestion & Preprocessing Pipeline          │
  ├─────────────────────────────────────────────────────────────┤
  │  Step 1: Ingest Wine Dataset (X = 13 features, y = 3 classes)│
  │                                                             │
  │  Step 2: Train/Test Data Split                              │
  │     • 80% Training Set (142 samples × 13 features)          │
  │     • 20% Testing Set                                       │
  │                                                             │
  │  Step 3: Feature Standardization (StandardScaler)           │
  │     • Normalize features so Mean = 0 and Variance = 1       │
  └─────────────────────────────────────────────────────────────┘
    

Preprocessing Workflow:

1. Dataset Overview: The Wine dataset contains 13 continuous feature dimensions and 3 target classes for multi-class classification.
2. Train/Test Partitioning: Using train_test_split, the data is divided using an 80/20 ratio (yielding 142 training samples with 13 feature columns).
3. Feature Scaling: Using StandardScaler, input features are standardized (X_scaled) to prevent scaling bias across input dimensions.

3. Constructing the Neural Network with tf.keras.Sequential

To build the artificial neural network architecture, we use TensorFlow's high-level API: tf.keras.Sequential.

  ┌─────────────────────────────────────────────────────────────┐
  │              3-Layer Neural Network Architecture            │
  ├─────────────────────────────────────────────────────────────┤
  │                                                             │
  │   Input Layer           Hidden Layers           Output Layer│
  │  (13 Features)         (Dense 16 & 8)           (Dense 3)   │
  │   ┌─────────┐        ┌─────────┐  ┌────────┐    ┌─────────┐ │
  │   │  x1..13 ├───────►│ 16 ReLU ├─►│ 8 ReLU ├───►│ 3 Softmax││
  │   └─────────┘        └─────────┘  └────────┘    └─────────┘ │
  │                      (Layer 1)    (Layer 2)      (Output)   │
  │                                                             │
  └─────────────────────────────────────────────────────────────┘
    

Layer Definitions:

• Hidden Layer 1: Dense(16, activation='relu', input_shape=(13,)) — A fully connected layer with 16 neurons, ReLU activation, receiving 13 input features.
• Hidden Layer 2: Dense(8, activation='relu') — A second fully connected layer containing 8 neurons with ReLU activation.
• Output Layer: Dense(3, activation='softmax') — The final layer containing 3 neurons (matching the 3 target classes) paired with the Softmax activation function. Softmax converts raw output logits into probability distributions, making it ideal for multi-class classification.

4. Model Parameters & Weight Breakdown

Executing model.summary() provides a complete breakdown of the trainable parameters (weights and biases) across all layers:

  ┌─────────────────────────────────────────────────────────────┐
  │            Parameter Calculation per Layer                  │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Layer Type & Neurons          │ Mathematical Weight Formula │
  ├───────────────────────────────┼─────────────────────────────┤
  │ Layer 1 (Dense 16 Neurons)    │ (13 inputs × 16) + 16 bias  │
  │                               │ = 224 Parameters            │
  ├───────────────────────────────┼─────────────────────────────┤
  │ Layer 2 (Dense 8 Neurons)     │ (16 inputs × 8) + 8 bias    │
  │                               │ = 136 Parameters            │
  ├───────────────────────────────┼─────────────────────────────┤
  │ Layer 3 (Dense 3 Output)      │ (8 inputs × 3) + 3 bias     │
  │                               │ = 27 Parameters             │
  ├───────────────────────────────┼─────────────────────────────┤
  │ Total Trainable Parameters    │ 224 + 136 + 27 = 387        │
  │ Total Model Memory Size       │ ~1.5 KB                     │
  └───────────────────────────────┴─────────────────────────────┘
    

• Layer 1: (13 × 16) + 16 = 224 parameters.
• Layer 2: (16 × 8) + 8 = 136 parameters.
• Output Layer: (8 × 3) + 3 = 27 parameters.
• Total Trainable Parameters: 387 total trainable parameters (requiring approx. 1.5 KB of memory).

5. Compiling the Model (Optimizer, Loss, Metrics)

Before training, the model must be compiled by specifying its optimization algorithm, loss function, and evaluation metrics:

  ┌─────────────────────────────────────────────────────────────┐
  │                Model Compilation Configuration              │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Component                     │ Configured Selection        │
  ├───────────────────────────────┼─────────────────────────────┤
  │ Optimizer                     │ Adam Optimizer ('adam')     │
  │ Loss Function                 │ Sparse Categorical          │
  │                               │ Cross-Entropy               │
  │ Performance Metric            │ Accuracy ('accuracy')       │
  └───────────────────────────────┴─────────────────────────────┘
    

1. Optimizer: Adam Optimizer ('adam'), an industry-standard choice for efficient gradient updates.
2. Loss Function: Sparse Categorical Cross-Entropy ('sparse_categorical_crossentropy'), the standard loss function for integer-encoded multi-class targets.
3. Performance Metric: Accuracy (['accuracy']), tracks classification accuracy during training.

6. Model Training & Evaluation (fit & evaluate)

  ┌─────────────────────────────────────────────────────────────┐
  │                 Model Training & Evaluation                 │
  ├─────────────────────────────────────────────────────────────┤
  │  1. Training Execution (model.fit):                         │
  │     • Trained for 20 Epochs with verbose=1                  │
  │     • GPU parallelization executes epochs rapidly           │
  │                                                             │
  │  2. Evaluation Execution (model.evaluate):                  │
  │     • Tested on unseen scaled test data                     │
  │     • Test Loss     = ~0.064                                │
  │     • Test Accuracy = 1.0 (100% Perfect Accuracy)           │
  └─────────────────────────────────────────────────────────────┘
    

A. Model Training (fit):

The network is trained using model.fit(X_train_scaled, y_train, epochs=20, verbose=1). Setting verbose=1 displays internal training logs per epoch, helping developers debug loss and accuracy trends in real time. On a GPU, all 20 epochs execute rapidly.

B. Model Evaluation (evaluate):

Evaluating the model on the unseen test dataset using model.evaluate(X_test_scaled, y_test, verbose=1) yields a test loss of ~0.064 and a test accuracy of 1.0 (100%). Because the Wine dataset is clean and linearly structured, deep learning achieves perfect classification performance.

7. End-to-End Executable Python Code

Below is the complete, runnable Python script implementing Topic 152 in Google Colab or Jupyter Notebook:

# =====================================================================
# Complete Neural Network Script using TensorFlow & Keras (Topic 152)
# =====================================================================

import tensorflow as tf
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# 1. Ingest Wine Dataset
wine = load_wine()
X = wine.data
y = wine.target

# 2. Train/Test Data Split (80/20)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# 3. Standardize Features
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

print(f"Training Feature Shape: {X_train_scaled.shape}")  # (142, 13)

# 4. Build Neural Network Architecture using Keras Sequential API
model = tf.keras.Sequential([
    tf.keras.layers.Dense(16, activation='relu', input_shape=(X_train_scaled.shape[1],)),
    tf.keras.layers.Dense(8, activation='relu'),
    tf.keras.layers.Dense(3, activation='softmax')
])

# Display Model Parameters Summary (387 parameters)
model.summary()

# 5. Compile Model (Optimizer, Loss, Metric)
model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

# 6. Train Neural Network (20 Epochs)
history = model.fit(
    X_train_scaled, y_train,
    epochs=20,
    verbose=1
)

# 7. Evaluate Model on Test Dataset
test_loss, test_acc = model.evaluate(X_test_scaled, y_test, verbose=1)

print("\n--- Model Final Evaluation ---")
print(f"Test Loss     : {test_loss:.4f}")
print(f"Test Accuracy : {test_acc * 100:.2f}%")
    

8. Summary Table: Keras Building Pipeline

Pipeline Stage	TensorFlow / Keras Function	Configuration & Purpose	Result / Metric
Cloud GPU Setup	Google Colab (T4 GPU)	Enables free GPU hardware acceleration.	Fast parallel training.
Data Ingestion	load_wine()	Loads 13 input features and 3 target classes.	Matrix shape (142, 13).
Scaling	StandardScaler()	Centers features around Mean = 0, Std = 1.	Standardized input data.
Architecture	tf.keras.Sequential	Builds 16-neuron ReLU → 8-neuron ReLU → 3-neuron Softmax.	Total 387 parameters.
Compilation	model.compile()	Sets adam optimizer, sparse_categorical_crossentropy loss, and accuracy metric.	Configured loss pipeline.
Training	model.fit()	Trains model for 20 epochs with verbose=1.	Rapid loss reduction.
Evaluation	model.evaluate()	Evaluates model loss and accuracy on test data.	100% Test Accuracy.