Deep Learning and Generative Models

Learning objectives

The course provides an in-depth study of fundamental and advanced concepts in deep learning and generative modelling, with particular emphasis on representation learning, different learning paradigms, and modern neural architectures.

Upon successful completion of the course, students will be able to understand and apply the main methodologies of deep learning, representation learning, self-supervised learning, reinforcement learning and generative modelling, as well as select appropriate architectures and training strategies according to the application problem.

Knowledge and understanding

Students will:

• understand the fundamental principles of deep neural networks and backpropagation;
• understand the main convolutional, recurrent and Transformer-based architectures;
• understand the main learning paradigms, including supervised, unsupervised, self-supervised, transfer, continual, federated and reinforcement learning;
• understand the principles underlying foundation models and multimodal architectures;
• understand the theoretical foundations and differences among the main families of generative models, including VAEs, GANs, diffusion models and modern flow-based approaches;
• understand the principles of the main explainability and feature-attribution methods for deep neural networks.

Applying knowledge and understanding

Students will be able to:

• design and implement deep learning systems using PyTorch;
• effectively use tools for data preparation, management and analysis;
• select appropriate architectures and learning strategies according to the problem and data modality;
• apply transfer learning, self-supervised learning and representation learning techniques;
• implement and evaluate deep generative models;
• design appropriate experimental protocols, identifying suitable benchmarks and baselines;
• apply explainability methods to analyze and visualize model behaviour.

Making judgements

Students will be able to:

• identify the most appropriate learning paradigm and model for a given problem;
• critically evaluate the strengths and limitations of different architectures and methodologies;
• recognize underfitting, overfitting and poor generalization;
• critically interpret experimental results;
• compare different deep learning and generative modelling approaches on the basis of quantitative and qualitative evidence.

Communication skills

Students will be able to:

• clearly describe the motivation, methodology and results of a deep learning study;
• present and critically discuss scientific papers;
• communicate experimental findings using appropriate technical terminology;
• discuss strengths, limitations and possible future developments of a method.

Lifelong Learning skills

Students will:

• develop autonomy in studying deep learning methodologies not directly covered during the course;
• acquire the ability to read and understand recent scientific literature;
• be able to independently explore emerging methodologies in deep learning and generative modelling;
• develop the ability to adapt the acquired methodologies and tools to new application problems.

The course accounts for 9 ECTS credits (CFU), corresponding to a total of 71 hours of teaching activities, including 56 hours of lectures (e.g., scientific paper discussions, specialized seminars, etc.) and 15 hours of laboratory/practical sessions.

Lectures will combine theoretical foundations with the analysis of deep learning and generative modelling architectures, methodologies, and applications.

During practical sessions, mainly based on PyTorch and Jupyter Notebooks, students will be guided through the design, implementation, training, and evaluation of deep learning and generative models, applying the methodologies presented during the course to practical problems.

Selected curricular topics may be delivered through seminars by international researchers and experts, with particular emphasis on recent methodologies and current research directions.

• Essential

  • Knowledge of the fundamental concepts of artificial neural networks.
  • Proficiency in Python programming.
  • Knowledge of the fundamental principles of machine learning, including training, validation, and testing.

  Important

  • Basic knowledge of linear algebra, particularly vectors, matrices, and matrix operations.
  • Basic knowledge of calculus, including derivatives and gradients.
  • Basic knowledge of probability and statistics.

  Useful

  • Familiarity with Python libraries for scientific computing and data analysis.
  • Basic knowledge of common machine learning problems, such as classification and regression.
  • Familiarity with interactive development environments, such as Jupyter Notebooks.

Attendance is strongly recommended. Active participation in lectures, practical sessions, seminars, and other teaching activities is considered an integral part of the learning experience and contributes to the final assessment.

The course is organized into two main parts: Learning Paradigms and Deep Learning, and Generative Models. The methodologies presented will be accompanied by practical examples and implementations using PyTorch and Jupyter Notebook, which will be used throughout the different parts of the course. Selected advanced topics may be further explored through seminars delivered by international researchers and experts.

PART I – LEARNING PARADIGMS AND DEEP LEARNING

• Foundations of Deep Learning: deep neural networks and backpropagation, activation functions and loss functions, optimization algorithms, regularization and normalization, initialization and training strategies, generalization in deep neural networks.
  • [Lectures: 4 hours]
• Convolutional Neural Networks: convolution, padding, stride, dilation and pooling, 2D and 3D convolutions, main architectures and variants (AlexNet, VGG, ResNet, DenseNet, Inception), transfer learning, fine-tuning and data augmentation.
  • [Lectures: 4 hours - Exercises and Laboratory: 3 hours]
• Recurrent Neural Networks and Sequence Modelling: recurrent neural networks, LSTM and variants, sequence modelling, limitations of recurrent architectures.
  • [Lectures: 2 hours - Exercises and Laboratory: 2 hours]
• Attention and Transformers: attention mechanisms, self-attention and multi-head attention, positional encoding, Transformer architecture, encoder, decoder and encoder-decoder architectures, Vision Transformer.
  • [Lectures: 4 hours - Exercises and Laboratory: 2 hours]
• Representation Learning and Self-Supervised Learning: representation and feature learning, autoencoders and variants, contrastive learning (SimCLR, MoCo and variants), non-contrastive self-supervised approaches, masked modelling and Masked Autoencoders (MAE), Joint-Embedding Predictive Architectures (JEPA), I-JEPA, V-JEPA and variants, predictive representation learning.
  • [Lectures: 6 hours]
• Learning Paradigms: supervised and unsupervised learning, self-supervised learning, transfer learning, few-shot and zero-shot learning, in-context learning, continual learning, federated learning.
  • [Lectures: 6 hours]
• Reinforcement Learning and Deep Reinforcement Learning: agent, environment, state, action and reward, Markov Decision Process, value functions and Q-learning, Deep Q-Network, policy-based methods, actor-critic approaches.
  • [Lectures: 4 hours - Exercises and Laboratory: 2 hours]
• Foundation Models: foundation model paradigm, Large Language Models, Vision Foundation Models, Vision-Language Models, multimodal foundation models, foundation models for segmentation and time series, fine-tuning and parameter-efficient adaptation.
  • [Lectures: 6 hours - Exercises and Laboratory: 3 hours]
• Explainable Deep Learning: explainability and interpretability, post-hoc explanation methods, saliency maps and feature attribution, Class Activation Maps (CAM), Grad-CAM and variants, Integrated Gradients, SHAP, explainability for convolutional and Transformer architectures.
  • [Lectures: 4 hours]

PART II – GENERATIVE MODELS

• Foundations of Deep Generative Modelling: generative and discriminative modelling, explicit and implicit generative models, latent variable models, latent representations for generation.
  • [Lectures: 3 hours]
• Variational Autoencoders and Variants: Variational Autoencoders, variational inference, Evidence Lower Bound (ELBO), reparameterization trick, Conditional VAE, β-VAE, VQ-VAE and variants, latent spaces and generation.
  • [Lectures: 3 hours]
• Generative Adversarial Networks and Variants: adversarial learning, GAN formulation and training, DCGAN, Conditional GAN, Wasserstein GAN, CycleGAN, StyleGAN and variants, training instability and mode collapse.
  • [Lectures: 3 hours - Exercises and Laboratory: 2 hours]
• Diffusion Models and Variants: principles of diffusion-based generative modelling, forward and reverse processes, Denoising Diffusion Probabilistic Models (DDPM), DDIM, score-based generative modelling, conditional diffusion, classifier guidance and classifier-free guidance, Latent Diffusion Models, Diffusion Transformers (DiT).
  • [Lectures: 3 hours - Exercises and Laboratory: 2 hours]
• Modern Generative Modelling: autoregressive generative models, Transformer-based generative models, flow-based generative modelling, Flow Matching, Rectified Flow, multimodal generative models, text-to-image, image-to-image and video generation, advanced applications and emerging research directions.
  • [Lectures: 4 hours]

1. S. J. D. Prince, Understanding Deep Learning, MIT Press, 2023.
2. S. Theodoridis, Machine Learning: From the Classics to Deep Networks, Transformers, and Diffusion Models, 3rd Edition, Academic Press/Elsevier, 2025.
3. P.-Y. Chen, S. Liu, Introduction to Foundation Models, Springer, 2025.
4. I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016.
5. Teaching materials, Jupyter Notebooks and selected recent scientific papers provided by the instructor.

Given the rapidly evolving nature of deep learning and generative modelling, textbooks will be complemented by recent scientific papers and additional material covering advanced and emerging topics.

The final assessment consists of the following components:

• Deep Learning / Generative Models Project (max 13 points): development, in groups of up to two students, of a project implemented in PyTorch and accompanied by a scientific-paper-style report. Methodology, implementation, experimental protocol, results and critical discussion will be assessed.
• Scientific Paper Presentation (max 5 points): presentation and critical discussion of a scientific paper selected from a list provided by the instructor or previously approved.
• Individual Oral Theory Assessment (max 10 points): questions on the topics covered during the course, including those addressed in curricular seminars. The assessment is individual and may result in different grades for students belonging to the same group.
• Class Attendance and Active Participation (max 2 points): regular attendance and active participation in lectures, seminars and other teaching activities will be assessed.

Working students and students officially exempted from attendance requirements: they will not be penalized for non-attendance. The 2 points associated with attendance and participation will be assigned on the basis of the other examination activities.

• Describe backpropagation and the role of the main optimization and regularization strategies.
• What are the main characteristics of Convolutional Neural Networks and Transformer-based architectures?
• Describe the self-attention mechanism and its use in Transformers.
• What are the main differences among contrastive learning, masked modelling and Joint-Embedding Predictive Architectures (JEPA)?
• Describe and compare different learning paradigms, such as self-supervised, continual, federated and reinforcement learning.
• What is a foundation model and which strategies can be used to adapt it to a specific downstream task?
• Describe how CAM, Grad-CAM and Integrated Gradients work and their main differences.
• Describe the principles of a Variational Autoencoder and the role of the ELBO.
• Describe the training process of a Generative Adversarial Network and its main limitations.
• Describe the forward and reverse processes of a diffusion model and the principle of classifier-free guidance.
• Compare VAEs, GANs and diffusion models, highlighting their main characteristics, advantages and limitations.
• Describe the principles of Flow Matching and its differences with respect to diffusion models.