STEM Robotics Workshops for Kids in Pecs

Chapter 144 — Feedforward Neural Networks Explained: How Information Moves Through a Network

1. Introduction: What is a Feedforward Neural Network (FNN)?

In Artificial Neural Networks (ANNs), information processing follows specific architectural rules. A Feedforward Neural Network (FNN) is a fundamental neural network architecture where data flows strictly in one direction—moving from the Input Layer, passing through any Hidden Layers, and terminating at the Output Layer.

  ┌─────────────────────────────────────────────────────────────┐
  │         Unidirectional Information Flow in FNNs             │
  ├─────────────────────────────────────────────────────────────┤
  │                                                             │
  │   Input Layer             Hidden Layers          Output Layer
  │   ┌─────────┐             ┌─────────┐            ┌─────────┐│
  │   │ Feature ├────────────►│ Neuron  ├───────────►│ Predict ││
  │   ├─────────┤             ├─────────┤            ├─────────┤│
  │   │ Feature ├────────────►│ Neuron  ├───────────►│ Predict ││
  │   └─────────┘             └─────────┘            └─────────┘│
  │                                                             │
  │   Strict Forward Flow: NO Loops | NO Feedback Connections   │
  └─────────────────────────────────────────────────────────────┘
    

Core Architectural Rule:

• Strict Unidirectional Flow: Signals move strictly forward from input to output.
• No Loops or Feedback: There are no cyclic loops and no feedback connections inside a Feedforward Neural Network.
• Multilayer Perceptron (MLP): This architecture is also formally known as a Multilayer Perceptron (MLP). The MLP represents the simplest and historically first form of artificial neural network invented.

2. Layer-by-Layer Mathematical Calculations

As input data enters the network, each layer transforms the incoming signal using a two-step mathematical formula before passing the result to the next layer.

  ┌─────────────────────────────────────────────────────────────┐
  │             Mathematical Calculation per Neuron             │
  ├─────────────────────────────────────────────────────────────┤
  │  Step 1: Weighted Sum with Bias Calculation                 │
  │     • Formula: g = (w * x) + b                              │
  │     • x = Input vector                                      │
  │     • w = Connection weight vector                          │
  │     • b = Bias term                                         │
  │                                                             │
  │  Step 2: Activation Function Pass                           │
  │     • Formula: Output = Activation(g)                       │
  │     • Passes g into ReLU, Sigmoid, or Tanh                  │
  └─────────────────────────────────────────────────────────────┘
    

1. Weighted Sum with Bias (g): Each layer multiplies incoming inputs by their corresponding connection weights and adds a bias term: g = (w · x) + b. Where x is the vector of input feature values, w is the vector of connection weights, and b is the bias.
2. Activation Function Application: The weighted sum g is passed through an activation function (such as ReLU, Sigmoid, or Tanh) to compute the final neuron output: Neuron Output = Activation(g).
3. Propagating to Output: This process repeats sequentially across all hidden layers until reaching the Output Layer, which delivers the network's final prediction to the external world.

3. Key Characteristics and Best Use Cases

Feedforward Neural Networks possess distinct operational traits that determine their ideal deployment scenarios:

  ┌─────────────────────────────────────────────────────────────┐
  │                 Key Traits & Ideal Applications             │
  ├───────────────────────────────┬─────────────────────────────┤
  │ Key Operational Traits        │ Best-Suited Applications    │
  ├───────────────────────────────┼─────────────────────────────┤
  │ • One-way signal propagation  │ • Tabular structured data   │
  │ • Uses activation functions   │ • Simple image recognition  │
  │   (ReLU, Sigmoid, Tanh)       │ • Basic text feature tasks  │
  │ • Learns weights & biases     │ • Regression & classification│
  │   during supervised training  │                             │
  └───────────────────────────────┴─────────────────────────────┘
    

• Supervised Parameter Learning: During training on supervised datasets, the network automatically learns and adjusts its weights and biases.
• Best-Suited Tasks: FNNs perform exceptionally well on tabular data, simple image classification tasks, and basic text feature processing.
• Limitations: FNNs lack internal memory and cannot handle time-series data or sequential memory tasks. (Sequential data requires specialized architectures like Recurrent Neural Networks, covered in later modules).

4. Real-World Practical Example: Predicting Car Prices

To understand how an FNN processes real-world data, consider a regression problem predicting the estimated market price of a car.

  ┌─────────────────────────────────────────────────────────────┐
  │             Car Price Prediction Architecture               │
  ├─────────────────────────────────────────────────────────────┤
  │  Input Features (x):                                        │
  │    • x₁ = Car Mileage                                       │
  │    • x₂ = Car Age                                           │
  │    • x₃ = Engine Size                                       │
  │                                                             │
  │  Hidden Layer Processing:                                   │
  │    • Calculates g = (w · x) + b                             │
  │    • Applies Activation Function                            │
  │                                                             │
  │  Output Prediction (ŷ):                                     │
  │    • Final Estimated Car Price ($)                          │
  └─────────────────────────────────────────────────────────────┘
    

Numerical Walkthrough:

Suppose a single neuron in the network receives a simplified input value:

• Input Feature (x): 2
• Weight (w): 1
• Bias (b): 0

Step 1: Compute Weighted Sum (g): g = (x · w) + b = (2 · 1) + 0 = 2

Step 2: Apply ReLU Activation Function: Passing g = 2 into the ReLU (Rectified Linear Unit) activation function: Output = ReLU(2) = 2 (Because ReLU acts as an identity function for positive values, an input of 2 returns an output of 2).

Thus, the final forward pass output emitted by this neuron is 2.

5. The Foundation of Deep Learning

Feedforward Neural Networks serve as the core foundational building block for all modern deep learning architectures.

  ┌─────────────────────────────────────────────────────────────┐
  │            From Simple ANNs to Deep Learning                │
  ├─────────────────────────────────────────────────────────────┤
  │  Simple Artificial Neural Network (ANN / MLP):              │
  │    • Input Layer ──► 1 or 2 Hidden Layers ──► Output Layer  │
  │                                                             │
  │  Deep Learning (Deep Neural Network):                       │
  │    • Input Layer ──► DOZENS of Hidden Layers ──► Output     │
  │                                                             │
  │  Key Insight: Increasing hidden layers transforms a basic   │
  │               FNN into a Deep Learning model!               │
  └─────────────────────────────────────────────────────────────┘
    

When we scale up an Artificial Neural Network by significantly increasing the number of hidden layers, it transitions into Deep Learning.

6. Summary Comparison Table

Dimension / Property	Operational Meaning
Data Flow Direction	Strictly unidirectional from input layer to output layer.
Network Topology	Contains no loops and no feedback connections.
Alternative Term	Multilayer Perceptron (MLP) — the simplest form of ANN.
Neuron Formula	g = (w · x) + b, followed by Activation(g).
Primary Activation Functions	ReLU, Sigmoid, and Tanh.
Best Data Domain	Tabular data, simple images, and basic text features.
Primary Limitation	Ineffective for time-series data and memory tasks.