Deep Learning and Generative Models: Diffusion and Autoregressive Models
Learn how deep generative models represent, learn, and sample from probability distributions. Part five in a five-part series, this online course in the Deep Learning and Generative Models program covers variational autoencoders, generative adversarial networks, diffusion models, flow matching, and autoregressive models, culminating in an original capstone experiment. You will develop skills for comparing generative approaches and evaluating their outputs. These skills provide a foundation for applications such as generating images and text or creating synthetic data, where assessing the quality, diversity, and limitations of generated samples is essential.
Course Information
Certificate Track
Learn for Free
About this Course
Generative models learn the patterns in observed data so they can create new samples. Begin with maximum likelihood, Gaussian models, and KL divergence as tools for modeling data distributions. Then learn how to tell whether a model generalizes beyond its training data or simply memorizes it.
Compare variational autoencoders (VAEs) and generative adversarial networks (GANs) with score-based and diffusion models that generate data by reversing noise. Explore flow matching, which learns a continuous path from noise to data, and autoregressive models, which generate one element at a time. Examine how generation order and teacher forcing shape autoregressive training.
In the PyTorch lab, train a VAE and vary its latent dimension and regularization strength one factor at a time. Compare diffusion sampling at 25 and 100 steps under the same conditions, weighing sample quality and diversity against computational cost.
The course concludes with a capstone experiment. Choose a research question, define the method and comparison, decide in advance what result would be meaningful, and report only the conclusions supported by the evidence.
This is the fifth of five courses in the Deep Learning and Generative Models Program:
What you'll learn
- Relate maximum likelihood and KL divergence to fitting probabilistic models
- Assess whether a generative model generalizes beyond its training examples
- Compare the objectives and tradeoffs of variational autoencoders and generative adversarial networks
- Evaluate tradeoffs among diffusion sampling steps, sample quality, and computational cost
- Factorize joint distributions for autoregressive generation and explain teacher forcing
- Design, conduct, and communicate a controlled capstone experiment
How you'll learn
Real-World LearningLearn from MIT faculty and experts who ground their teaching in real-world cases rather than mathematical models, making the material approachable for all.
Practical ApplicationApply your new knowledge with hands-on, practical exercises drawn from healthcare, sports, finance, sustainability, and more.
- Learn On Demand
Access all course content online with complete flexibility to study at your own pace.
AI-Enabled SupportDeepen your understanding of the course material and get help on assignments from AskTIM, the AI assistant built by MIT researchers.
Stackable CredentialsEarn an MIT Open Learning certificate at each milestone—module, course, and program—demonstrating your AI expertise. Available in paid courses only.
Prerequisites
This program is designed for:
- Aspiring data scientists and machine learning engineers who want a structured introduction to deep learning
- Quantitative researchers who want to apply deep learning to technical or scientific problems
- Working professionals who need to evaluate, adapt, or deploy modern machine learning models
- Advanced undergraduate and graduate students in computer science, data science, statistics, mathematics, engineering, and related quantitative fields
- Continuing education learners seeking hands-on experience with deep learning and generative models
You should be comfortable programming in Python and working with linear algebra, multivariable calculus, including partial derivatives and the chain rule, and undergraduate probability and statistics. The Probability and Statistical Data Analysis series or equivalent coursework provides suitable preparation. No prior deep learning experience is required.
The labs run in Google Colab and are designed for its free GPU tier, so no local installation is required.
Meet your instructors
Giannis Daras
Assistant Professor, MIT Sloan School of Management
Dr. Giannis Daras is an assistant professor in the Operations Research and Statistics group at the MIT Sloan School of Management. He earned his PhD in computer science at the University of Texas at Austin, advised by Alexandros Dimakis, and completed postdoctoral research at MIT with Antonio Torralba and Constantinos Daskalakis. He received his undergraduate degree in electrical and computer engineering from the National Technical University of Athens. His research addresses theoretical and practical questions in generative AI, particularly how to train and sample deep generative models when data are corrupted or imperfect. His interests include diffusion models, inverse problems, data-centric AI, scientific applications of AI, and agents. His work has applications in medical imaging, computational biology, robotics, economics, and neuroscience. He received the Best Contribution Award at the Biomedical and Astronomical Signal Processing conference and fellowship support from UT Austin, Onassis, Bodossakis, Leventis, and Gerondellis. At MIT, he co-designed and co-taught the graduate course Diffusion Models: From Theory to Practice with Constantinos Daskalakis, connecting mathematical foundations with applications of generative modeling. MIT Sloan bio, faculty website
