Implementing Artificial Neural Networks (Part 1)

CSI4106 Introduction to Artificial Intelligence

Important

Deadline: Assignment 2 must be submitted no later than October 26 November 2, 2026, at 11 PM. Please refer to the assignment description available on Brightspace.

Prepare

• Russell and Norvig (2020), pages 750–788

Watch 3Blue1Brown videos on neural networks

The first two videos support this lecture; the backpropagation videos prepare for Lecture 13.

• But what is a Neural Network? (19 minutes)
• Gradient descent, how neural networks learn (21 minutes)
• What is backpropagation really doing? (14 minutes)
• Backpropagation calculus (10 minutes)

Narrative of PyTorch’s inception

• PyTorch (2024) Official PyTorch Documentary - Powering the AI Revolution (36 minutes)

Participate

• slides (PDF, Jupyter Notebook)

Practice

• Circular Separability : A self-study notebook comparing engineered features and learned neural-network representations, complementing the TensorFlow Playground exploration below.
• 2D Regression with MSE Residuals: Supports one-dimensional regression, configurable hidden neurons, and sliders for every weight and bias, which can be helpful to further develop your intuition for the Universal Approximation Theorem.
• Tinker With a Neural Network in Your Browser

References

Russell, Stuart, and Peter Norvig. 2020. Artificial Intelligence: A Modern Approach. 4th ed. Pearson. http://aima.cs.berkeley.edu/.