Lecture 32: ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule

Description

Professor Strang begins the lecture talking about ImageNet, a large visual database used in visual object recognition software research. ImageNet is an example of a convolutional neural network (CNN). The rest of the lecture focuses on convolution.

Summary

Convolution matrices have \(\leq\) \(n\) parameters (not \(n\)²).
Fewer weights to compute in deep learning
Component \(k\) from convolution \(c*d\): Add all \(c(j)d(k-j)\)
Convolution Rule: \(F(c*d) = Fc\) times \(Fd\) (component by component)
\(F\) = Fourier matrix with \(j\), \(k\) entry \(= \exp (2 \pi i j k /n)\)

Related section in textbook: IV.2

Instructor: Prof. Gilbert Strang