Lecture 21: Minimizing a Function Step by Step

Description

In this lecture, Professor Strang discusses optimization, the fundamental algorithm that goes into deep learning. Later in the lecture he reviews the structure of convolutional neural networks (CNN) used in analyzing visual imagery.

Summary

Three terms of a Taylor series of \(F\)(\(x\)) : many variables \(x\)
Downhill direction decided by first partial derivatives of \(F\) at \(x\)
Newton’s method uses higher derivatives (Hessian at higher cost).

Related sections in textbook: VI.1, VI.4

Instructor: Prof. Gilbert Strang