Anno accademico 2023/2024 - Docente: CONCETTO SPAMPINATO

Risultati di apprendimento attesi

The course covers the fundamental concepts of machine and deep learning methods and how to use them for extracting, modelling and visualizing the learned knowledge.

Topics include: linear and logistic regression, K-NN. decision trees, support vector machines, neural networks with backpropagation, convolutional neural networks, recurrent neural networks, methods for representation learning, and how to use them under different learning regimes (supervised, unsupervised and reinforcement learning) and in variety of real-world applications ranging from computer vision, machine translation and medical image analysis.

The learning objectives are:

  • to understand and use the main methodologies and techniques for learning from data
  • to understand the main methodologies to design and implement machine learning methods for real-world applications
  • to understand how to extract and learn knowledge in scenarios when supervision cannot be provided
  • to understand and foresee the reliability of machine learning methods in operational scenarios.

Knowledge and understanding

  • To understand the main concepts of learning from data
  • To understand concepts and tools for building intelligent systems using supervision and no supervision
  • To understand the most important machine learning and artificial intelligence methodologies and techniques used by industries to make sense of data in order to support the decision process
  • To understand what are the most appropriate techniques to be used in different real-world applications

Applying knowledge and understanding

  • To be able to effectively understand and use the main tools for creating, loading and manipulating datasets.
  • To design and implement from scratch a machine learning system following application-derived constraints in terms of modelling and data
  • To understand proper benchmarks and baselines and analysing achieved results and their generalization in real-world applications
  • To be able to apply methodologies and techniques to analyse data.

    Making judgements 

    • To be able to identify the most suitable model to address a complex data analysis problem
    • To be able to identify the motivations of underfitting and overfitting by a specific model
    • To Iteratively refine a model by designing specific models to learn desired features 


      • Learning how to discuss critically the pros and cons of deep learning techniques 

      Lifelong learning skills

      • To be able to design a sound and complete methodological approach given a real-world data analysis problem 
      • To gain independence in handling machine learning techniques beyond the ones presented during the course
      • To design and develop robust, calibrated, effective and efficient deep learning-based pipelines adapted to specific problems

      Modalità di svolgimento dell'insegnamento

      The main teaching methods are as follows:

      • Lectures, to provide theoretical and methodological knowledge of the subject;
      • Hands-on exercises, to provide “problem solving” skills and to apply design methodology;
      • Laboratories, to learn and test the usage of related tools.
      • Paper reading and presentations to enhance understanding of the core concepts
      • Seminars by renowned experts (from both universities and industries) in the field to understand the current state of the art.

      Should teaching be carried out in mixed mode or remotely, it may be necessary to introduce changes with respect to previous statements, in line with the programme planned and outlined in the syllabus.

      Prerequisiti richiesti

      Python programming language, statistical learning basic concepts

      Frequenza lezioni

      Strongly recommended. Attending and actively participating in the classroom activities will contribute positively towards the overall assessment of the final exam.

      Contenuti del corso

      The course consists of two parts: the first one will be addressing the general and modern techniques based on machine and deep learning paradigms to create intelligent systems from data, while the second one on how to extract, represent and visualize knowledge from data and trained models.

      Part I: Methods and Architectures

      Machine Learning Basics

      • Linear and Logistic Regression
      • Non parametric methods: KNN, Decision trees
      • Clustering: mixture of Gaussians
      • Feature selection: PCA
      • Support Vector Machines
      • Ensemble methods

      Neural Networks and Backpropagation

      • Derivatives and Gradient Descent
      • Neural Network Representation, Gradient descent for Neural Networks
      • Forward and Back Propagation
      • The revolution of depth: deep learning
      • Optimization algorithms: Mini-batch gradient descent, Exponentially weighted average, Gradient descent with momentum, RMSprop, Adam optimization algorithm, Learning rate decay
      • Training aspects of deep learning: Regularization, Dropout, Normalizing inputs, Vanishing / Exploding gradients, Weight Initialization for Deep Networks

      Convolutional Neural Networks

      • Foundations: padding, strided convolution, dilation, 2D and 3D convolution, pooling
      • State of the art models: AlexNet, ResNets, DenseNets, Inception
      • Transfer Learning and Data Augmentation

      Recurrent Neural Networks

      • LSTM and variants
      • Attention mechanisms

      Part II: Knowledge Discovery from Data and Models

      Unsupervised Learning with Deep Networks

      • Representation and Feature Learning
      • Autoencoders and Variational Autoencoders
      • Generative Adversarial Networks

        Explainable AI

        • Principles of explainability vs interpretability 
        • Post-hoc explanatory methods (e.g., IG and CAM)
        • Model agnostic (e.g., SHAP)

        Deep Learning Frameworks:

        • Overview of the most used DL frameworks
        • PyTorch and Jupyter Notebooks


        • Computer vision
        • Medical Image Analysis
        • Machine translation

        Testi di riferimento

        1. Pattern Recognition and Machine Learning, C. Bishop, 2006

        2. Deep Learning. I. Goodfellow, Y. Bengio and A. Courville, MIT Press, 2016

        3. Programming PyTorch for Deep Learning, I. Pointer, O'Reilly Media

        4. Teaching materials and reading paper list provided by the instructor

        I. Goodfellow, Y. Bengio and A. CourvilleDeep LearningMIT PRESS2016978-0262035613
        I. PointerProgramming PyTorch for Deep Learning O'Reilly Media20191492045357
        C. BishopPattern Recognition and Machine LearningSpringer20060387310738

        Programmazione del corso

         ArgomentiRiferimenti testi
        1Neural networks: derivatives, gradient descent, back-propagation3
        2Deep Learning: basic concepts, optimization algorithms, training procedures1,3
        3Convolutional Neural Networks1,3
        4Explainable AI3
        5Recurrent Neural Networks1,3
        6Unsupervised Learning with Deep Networks: Representation and Feature Learning1,3
        7Autoencoders and Variational Autoencoders1,3
        8Generative Adversarial Networks1,3
        9Deep Learning Frameworks: PyTorch and Jupyter Notebooks2,3

        Verifica dell'apprendimento

        Modalità di verifica dell'apprendimento

        The final exam consists of the development of a project in Pytorch, addressing one of the topics discussed during classes, together with a final report (structured as a scientific paper) discussing motivation, models, datasets and results used in the project.

        The exam is evaluated according to the ability to create a deep learning model from scratch for extracting and learning knowledge from data on a given real-world problem, to understand how to properly measure its performance and to motivate the devised solutions.

        The grading policy for the course is:

        • 70%: Final project

        • 30%: Theory test

        The course also foresees intermediate assignments only for students attending the course. These assignments include: a) between 2 and 3 homeworks regarding python scripts to solve simple basic learning problems on datasets discussed during with the instructor and b) a theory test to verify the correct understanding of the presented techniques. The grading policy is this case is the following one

        • 55%: Homeworks

        • 35%: Theory test

        • 10%: Class attendance

        Learning assessment may also be carried out on line, should the conditions require it.

        Esempi di domande e/o esercizi frequenti

        Examples of questions and exercises are available on the Studium platform and on the course website.