Computational Thinking Classes in Bydgoszcz

Chapter 165 — Object Detection and Image Segmentation: Key Concepts Explained

1. Introduction & Visual Understanding Paradigms

In computer vision, advanced mathematical concepts underpin the most popular deep learning models used for visual understanding. Analyzing a digital image requires understanding both what objects are present and where they are located in spatial coordinates.

Computer vision solves visual understanding across two primary paradigms:

• Object Detection: Identifies objects in an image and predicts their spatial locations using bounding boxes. The most popular deep learning model for object detection is YOLO (You Only Look Once), which is frequently fine-tuned on custom datasets for fast inference.
• Image Segmentation: Classifies every individual pixel in an image by assigning it to a specific class label. The benchmark model for image segmentation is U-Net, which produces precise pixel masks.

Both paradigms rely heavily on foundational mathematical building blocks, including convolution operations, feature maps, differentiable loss functions, and gradient descent optimization.

2. Object Detection Mechanics & The YOLO Model

A. The Convolution Operation

Convolution serves as the primary tool for extracting local spatial features such as edges and textures. Mathematically, a kernel filter matrix \(K\) slides over an input image \(I\) of size \(m \times n\) using a sliding window operation to generate output feature maps.

B. Grid Regression & Output Variables in YOLO

YOLO simplifies object detection by framing it as a single regression problem.

• \(S \times S\) Grid Division: The input image is divided into an \(S \times S\) spatial grid.
• Bounding Box Coordinates: For each grid cell, the network predicts four continuous bounding box values: starting coordinates \((x, y)\) along with the object's width (\(w\)) and height (\(h\)).
• Objectness Score: Represents the probability and confidence that a detected region actually contains an object.
• Class Probabilities: Represents the probability that the detected object belongs to a specific category class.

The final prediction score for any detected object is computed by multiplying the objectness probability by the conditional class probability:

Final Output Score = P(Object) × P(Class | Object)
    

C. The Multi-Part Loss Function in YOLO

To optimize bounding box positions, confidence scores, and class predictions simultaneously, YOLO uses an end-to-end differentiable multi-part loss function:

L = λ_coord · L_coord + L_obj + L_noobj + L_class
    

1. Localization Loss (L_coord): Uses Mean Squared Error (MSE) in coordinate space to penalize inaccuracies in bounding box coordinates (x, y, w, h).
2. Confidence Loss (L_obj and L_noobj): Penalizes false detections in background regions while encouraging high confidence on true objects.
3. Classification Loss (L_class): Measures class prediction errors across object categories.

Combining these losses into a single differentiable function allows the network to be trained end-to-end using gradient descent, simultaneously learning what an object is and where it is located.

3. Image Segmentation Mechanics & The U-Net Model

A. U-Net Architecture & Building Blocks

The goal of image segmentation is pixel-wise classification. The U-Net architecture achieves this using an Encoder-Decoder structure:

• Encoder (Contracting Path): Uses standard convolution layers (f = σ(W · x + b)) for feature extraction alongside Max Pooling to reduce spatial dimensions.
• Bottleneck: Connects the deep feature representations between encoder and decoder.
• Decoder (Expanding Path): Employs Transposed Convolutions (x_up = f(Wᵀ · x + b)) to upsample feature maps back to original spatial dimensions for pixel-level prediction.
• Skip Connections: Pass high-resolution spatial details directly from encoder layers to decoder layers, ensuring precise pixel localization.

B. Loss Functions for Image Segmentation

Segmentation models are trained using specialized mathematical loss functions:

1. Binary Cross-Entropy (BCE) Loss:

   L_BCE = −(1/N) Σ [ yᵢ log(pᵢ) + (1 − yᵢ) log(1 − pᵢ) ]

   Measures logarithmic prediction errors across individual pixels.
2. Dice Loss & Intersection over Union (IoU):

   L_Dice = 1 − (2 · |P ∩ G|) / (|P| + |G|)

   Measures the spatial overlap between the predicted mask (P) and the ground truth mask (G).

4. Summary Comparison: YOLO vs. U-Net

Feature / Aspect	YOLO (Object Detection)	U-Net (Image Segmentation)
Output Format	Bounding box coordinates (x, y, w, h) and class scores.	Pixel-wise segmentation masks.
Primary Goal	Identify and localize objects in bounding regions.	Delimit precise object boundaries pixel-by-pixel.
Core Mathematics	S × S Grid regression & multi-part loss.	Transposed convolutions, skip connections & overlap loss.
Loss Evaluation	Multi-part loss (MSE, Confidence, Classification).	Binary Cross-Entropy & Dice Loss / IoU.

L = λ_coord · L_coord + L_obj + L_noobj + L_class