Training neural networks: challenges and practical choices
CSI4106 Introduction to Artificial Intelligence
Important
Deadline: Assignment 3 must be submitted no later than November 16, 2026, at 11 PM. Please refer to the assignment description available on Brightspace.
Participate
Practice
- Loss landscapes and gradient descent : Explore starting points, flat regions, local minima, and gradient clipping.
- Regularization: shrinkage and sparsity : Compare true coefficients, ordinary least squares, Ridge, and Lasso.
- Catastrophic forgetting on MNIST : Train on two disjoint digit groups, measure retention, and compare a replay strategy.
- Regularization with correlated features : A more advanced example with many correlated predictors.