Lecture 5 Part 3: Differentiation on Computational Graphs

Description: A very general way to think about the chain rule is to view computations as flowing through “graphs” consisting of nodes (intermediate values) connected by edges (functions acting on those values). When we propagate derivatives through the graph from inputs to outputs, we get the structure of forward-mode automatic differentiation; going from outputs to inputs yields reverse mode, which we will return to in lecture 8.

Instructors: Alan Edelman, Steven G. Johnson