Chapter 166 — YOLO Object Detection Explained: How AI Finds Objects in Images
1. Introduction & Core Object Detection Concepts
In computer vision, visual recognition tasks are divided into two distinct sub-problems: Object Classification and Object Localization.
- Object Classification: Determines what object is present in an image, which is framed as a classification problem.
- Object Localization: Determines where the object is located and its spatial dimensions, which is framed as a continuous regression problem.
┌─────────────────────────────────────────────────────────────┐
│ Object Recognition Framework │
├───────────────────────────────┬─────────────────────────────┤
│ Object Classification │ Object Localization │
├───────────────────────────────┼─────────────────────────────┤
│ • Identifies WHAT object is │ • Pinpoints WHERE the object│
│ present in the image. │ is located. │
│ • Categorizes class labels. │ • Predicts bounding box │
│ │ coordinates and size. │
└───────────────────────────────┴─────────────────────────────┘
Core YOLO Architecture:
The YOLO (You Only Look Once) framework solves both classification and localization simultaneously in a single forward pass. Its architecture consists of two primary modules:
- CNN Backbone: A deep feature extraction backbone such as DarkNet, MobileNet, or ResNet.
- Detection Head: A fully connected regression head that outputs bounding box predictions.
Bounding Box Representation:
A bounding box encloses a detected object, providing both its spatial location and its size. By default, YOLO models come pre-trained on the COCO dataset, which contains 80 distinct object classes. When target objects fall outside these default 80 classes, developers must build a custom dataset, annotate images with bounding boxes, and fine-tune YOLO.
2. Environment & Library Setup in Google Colab
Because YOLO-style models are deep neural networks with millions of trainable parameters, training requires significant computational power.
┌─────────────────────────────────────────────────────────────┐
│ Google Colab Environment Setup │
├─────────────────────────────────────────────────────────────┤
│ 1. Enable GPU Acceleration: │
│ • Runtime ──► Change runtime type ──► Select GPU. │
│ • Connects session to cloud GPU hardware instance. │
│ │
│ 2. Core Python Libraries: │
│ • tensorflow : Deep learning GPU execution. │
│ • numpy : Matrix and array operations. │
│ • cv2 : OpenCV image transformations. │
│ • matplotlib : Data visualization and rendering. │
└─────────────────────────────────────────────────────────────┘
import tensorflow as tf
from tensorflow.keras import layers, models
import numpy as np
import cv2
import matplotlib.pyplot as plt
3. Dataset Creation & Label Annotation Structure
Label Representation (5 Output Parameters):
To train a YOLO-like model, every object label is annotated using a 5-element vector:
Label Vector = [x_center, y_center, w, h, class]
- x_center: Horizontal coordinate of the bounding box center.
- y_center: Vertical coordinate of the bounding box center.
- w: Spatial width of the bounding box.
- h: Spatial height of the bounding box.
- class: Target object class ID (e.g., binary class 1).
┌─────────────────────────────────────────────────────────────┐
│ Dataset Specification Details │
├─────────────────────────────────────────────────────────────┤
│ • Dataset Size : 200 total image samples. │
│ • Image Dimensions : 128×128 pixels × 3 channels. │
│ • Pixel Format : uint8 (8 bits per channel). │
│ • Train / Test Split : 80% Train (160) / 20% Test (40). │
└─────────────────────────────────────────────────────────────┘
4. YOLO-like CNN Network Architecture
The network is constructed using Keras Sequential API, stacking three pairs of Convolutional and Max Pooling layers followed by a classification and regression head:
┌─────────────────────────────────────────────────────────────┐
│ YOLO CNN Layer Flow │
├─────────────────────────────────────────────────────────────┤
│ Input Image (128 × 128 × 3 Color Channels) │
│ │ │
│ ▼ │
│ Conv2D Layer 1 ──► 16 Filters (3×3), ReLU │
│ MaxPooling2D Layer 1 ──► Pool Size (2×2) │
│ │ │
│ ▼ │
│ Conv2D Layer 2 ──► 32 Filters (3×3), ReLU │
│ MaxPooling2D Layer 2 ──► Pool Size (2×2) │
│ │ │
│ ▼ │
│ Conv2D Layer 3 ──► 64 Filters (3×3), ReLU │
│ MaxPooling2D Layer 3 ──► Pool Size (2×2) │
│ │ │
│ ▼ │
│ Flatten Layer ──► Reshapes 3D tensor to 1D │
│ Dense Hidden Layer ──► 128 Neurons, ReLU │
│ Dense Output Layer ──► 5 Output Nodes, Sigmoid │
└─────────────────────────────────────────────────────────────┘
# Construct YOLO-like CNN Architecture
model = models.Sequential([
# First Convolutional Block
layers.Conv2D(16, (3, 3), activation='relu', input_shape=(128, 128, 3)),
layers.MaxPooling2D((2, 2)),
# Second Convolutional Block
layers.Conv2D(32, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
# Third Convolutional Block
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
# Detection Head
layers.Flatten(),
layers.Dense(128, activation='relu'),
layers.Dense(5, activation='sigmoid') # Outputs: [x, y, w, h, class]
])
5. Model Summary & Parameter Analysis (model.summary())
Executing model.summary() displays the complete layer transformation pipeline:
┌─────────────────────────────────────────────────────────────────────────┐
│ Model Summary Breakdown │
├──────────────────────────────┬───────────────────┬──────────────────────┤
│ Layer (Type) │ Output Shape │ Param # │
├──────────────────────────────┼───────────────────┼──────────────────────┤
│ conv2d (Conv2D) │ (None, 126, 126,16)│ 448 │
│ max_pooling2d (MaxPooling2D) │ (None, 63, 63, 16)│ 0 │
│ conv2d_1 (Conv2D) │ (None, 61, 61, 32)│ 4,640 │
│ max_pooling2d_1 (MaxPooling) │ (None, 30, 30, 32)│ 0 │
│ conv2d_2 (Conv2D) │ (None, 28, 28, 64)│ 18,496 │
│ max_pooling2d_2 (MaxPooling) │ (None, 14, 14, 64)│ 0 │
│ flatten (Flatten) │ (None, 12544) │ 0 │
│ dense (Dense) │ (None, 128) │ 1,605,760 │
│ dense_1 (Dense Output) │ (None, 5) │ 645 │
├──────────────────────────────┴───────────────────┴──────────────────────┤
│ Total params: 1,629,889 (6.22 MB memory) │
│ Trainable params: 1,629,889 │
└─────────────────────────────────────────────────────────────────────────┘
- Total Parameters: The model contains 1,629,889 trainable parameters requiring 6.22 MB of memory.
- Dense Layer Dominance: The first Dense layer accounts for ~1.6 million parameters due to the flattened input vector size of 12,544 units.
6. Model Compilation, Training, & Loss Optimization
The model is compiled using the Adam optimizer and Mean Squared Error (MSE) loss:
model.compile(
optimizer='adam',
loss='mse'
)
# Train for 10 Epochs with batch size 8
history = model.fit(
X_train, Y_train,
epochs=10,
batch_size=8,
validation_split=0.2
)
┌─────────────────────────────────────────────────────────────┐
│ Training Progression Log │
├─────────────────────────────────────────────────────────────┤
│ • Epoch 1/10 : Loss = 0.0300 │
│ • Epoch 10/10 : Loss = 0.00157 │
│ • Test Evaluation (model.evaluate): │
│ Final Test Loss = 0.0012 │
└─────────────────────────────────────────────────────────────┘
Training over 10 epochs reduces loss from 0.03 down to 0.00157. Evaluating the trained model on test data yields a low test loss of 0.0012.
7. Evaluation Metric: Intersection over Union (IoU)
To evaluate how accurately predicted bounding boxes overlap with ground truth bounding boxes, we use Intersection over Union (IoU).
┌─────────────────────────────────────────────────────────────┐
│ Intersection over Union (IoU) │
├─────────────────────────────────────────────────────────────┤
│ │
│ Formula: │
│ IoU = Area of Overlap (Intersection) │
│ ─────────────────────────────── │
│ Area of Union (Total Area) │
│ │
│ Evaluation Results across Test Samples: │
│ • Example 1 IoU : 0.607 │
│ • Example 2 IoU : 0.700 │
│ • Example 3 IoU : 0.570 │
│ • Example 4 IoU : 0.870 (Highest Performance) │
│ • Example 5 IoU : 0.720 │
└─────────────────────────────────────────────────────────────┘
An IoU score closer to 1.0 indicates near-perfect spatial alignment between predicted and actual bounding boxes. In test evaluations, the model achieved IoU scores reaching up to 0.87.
8. Complete Executable Python Script
Below is the complete, runnable Python code implementing Topic 166:
# =====================================================================
# Complete YOLO Object Detection Script (Topic 166)
# =====================================================================
import tensorflow as tf
from tensorflow.keras import layers, models
import numpy as np
import cv2
import matplotlib.pyplot as plt
# 1. Dataset Generation Function
def create_dataset(num_samples=200):
images = []
labels = []
img_size = 128
for _ in range(num_samples):
# Create black canvas (128x128x3 uint8)
img = np.zeros((img_size, img_size, 3), dtype=np.uint8)
# Generate random bounding box coordinates
x_min = np.random.randint(10, 60)
y_min = np.random.randint(10, 60)
w = np.random.randint(20, 50)
h = np.random.randint(20, 50)
# Draw white rectangle on canvas
cv2.rectangle(img, (x_min, y_min), (x_min + w, y_min + h), (255, 255, 255), -1)
# Calculate normalized coordinates
x_center = (x_min + w / 2.0) / img_size
y_center = (y_min + h / 2.0) / img_size
norm_w = w / float(img_size)
norm_h = h / float(img_size)
cls_id = 1.0 # Class 1
images.append(img / 255.0) # Normalize pixel values
labels.append([x_center, y_center, norm_w, norm_h, cls_id])
return np.array(images, dtype=np.float32), np.array(labels, dtype=np.float32)
# 2. Generate Dataset and Split 80/20
X, Y = create_dataset(200)
split_idx = int(0.8 * len(X))
X_train, X_test = X[:split_idx], X[split_idx:]
Y_train, Y_test = Y[:split_idx], Y[split_idx:]
# 3. Construct YOLO-like Architecture
model = models.Sequential([
layers.Conv2D(16, (3, 3), activation='relu', input_shape=(128, 128, 3)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(32, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(128, activation='relu'),
layers.Dense(5, activation='sigmoid')
])
# 4. Compile and Display Model Summary
model.compile(optimizer='adam', loss='mse')
model.summary()
# 5. Train Model for 10 Epochs
history = model.fit(X_train, Y_train, epochs=10, batch_size=8, validation_split=0.2)
# 6. Evaluate Model Test Loss
test_loss = model.evaluate(X_test, Y_test, verbose=0)
print(f"\nFinal Test Loss: {test_loss:.6f}")
# 7. Intersection over Union (IoU) Calculation Function
def compute_iou(boxA, boxB):
# Convert normalized [x_c, y_c, w, h] to corner coordinates
xA1 = boxA[0] - boxA[2]/2.0
yA1 = boxA[1] - boxA[3]/2.0
xA2 = boxA[0] + boxA[2]/2.0
yA2 = boxA[1] + boxA[3]/2.0
xB1 = boxB[0] - boxB[2]/2.0
yB1 = boxB[1] - boxB[3]/2.0
xB2 = boxB[0] + boxB[2]/2.0
yB2 = boxB[1] + boxB[3]/2.0
# Intersection coordinates
xI1 = max(xA1, xB1)
yI1 = max(yA1, yB1)
xI2 = min(xA2, xB2)
yI2 = min(yA2, yB2)
inter_area = max(0, xI2 - xI1) * max(0, yI2 - yI1)
boxA_area = (xA2 - xA1) * (yA2 - yA1)
boxB_area = (xB2 - xB1) * (yB2 - yB1)
union_area = boxA_area + boxB_area - inter_area
return inter_area / union_area if union_area > 0 else 0.0
# 8. Test Predictions & IoU Evaluation
predictions = model.predict(X_test[:5])
for i in range(5):
iou = compute_iou(Y_test[i][:4], predictions[i][:4])
print(f"Sample {i+1} - True Box: {Y_test[i][:4]}, Pred Box: {predictions[i][:4].round(3)}, IoU: {iou:.3f}")
9. Summary Table
| Component / Parameter |
Description & Operation |
Setting / Value |
| Output Label Format |
Bounding box coordinates & class. |
[x_center, y_center, width, height, class]. |
| Default Dataset |
Pre-trained dataset class coverage. |
COCO Dataset (80 Classes). |
| Hardware Platform |
GPU hardware acceleration. |
Google Colab T4 GPU. |
| Input Tensor Shape |
Image spatial resolution & channels. |
(128, 128, 3). |
| Total Parameters |
Model parameter footprint. |
1,629,889 Parameters (~6.22 MB memory). |
| Optimizer & Loss |
Optimization and loss metric. |
Adam Optimizer, MSE Loss. |
| Evaluation Metric |
Measures box overlap accuracy. |
Intersection over Union (IoU). |
