Feedforward Neural Networks

Feedforward Neural Networks

A feedforward neural network is the simplest type of artificial neural network. In this architecture, data flows in one direction, from the input layer to the output layer, without looping back.

Feedforward networks consist of:

• Input Layer: Receives the raw data.
• Hidden Layers: Perform computations and extract features.
• Output Layer: Produces the final result.

Each layer is fully connected to the next.

Forward propagation is the process of passing data through the network:

1. Inputs are multiplied by weights and added to biases.
2. Activation functions are applied to introduce non-linearity.
3. The output is computed and passed to the next layer.

Let’s create a simple feedforward network using Python. The network has:

• 2 input neurons
• 1 hidden layer with 3 neurons
• 1 output neuron

import numpy as np

# Define weights and biases
weights_input_hidden = np.array([[0.2, 0.8, -0.5], [0.5, -0.91, 0.26]])
bias_hidden = np.array([2, 3, 0.5])
weights_hidden_output = np.array([[0.1], [-0.14], [0.5]])
bias_output = np.array([0.1])

# Input data
inputs = np.array([1, 2])

# Forward pass
hidden_layer = np.dot(inputs, weights_input_hidden) + bias_hidden
output = np.dot(hidden_layer, weights_hidden_output) + bias_output

print("Output:", output)

Training involves adjusting weights and biases to minimize the error:

1. Compute the error (difference between predicted and actual values).
2. Use optimization algorithms like gradient descent to update weights.
3. Repeat the process for multiple iterations (epochs).

Feedforward networks are:

• Simple and easy to implement.
• Effective for tasks like regression and basic classification.
• Computationally efficient for small datasets.

Feedforward networks have some drawbacks:

• Lack of memory for sequential data.
• Prone to overfitting with small training data.
• Require careful tuning of hyperparameters like learning rate.

Feedforward networks are used in:

• Medical Diagnosis: Predicting diseases based on symptoms.
• Fraud Detection: Identifying suspicious activities in financial transactions.
• Stock Price Prediction: Estimating future stock values.

In this lesson, we:

• Explored the structure and working of feedforward neural networks.
• Built a simple feedforward network using Python.
• Discussed the advantages and limitations of this architecture.

Next, we’ll dive into the backpropagation algorithm, which enables neural networks to learn effectively by optimizing weights and biases.