9798849 - DEEP LEARNING
Modulo ADVANCED TECHNIQUES AND APPLICATIONS

The course provides an in-depth study of the fundamental and advanced concepts of machine and deep learning methods, with a focus on their use for extracting, modelling and visualizing knowledge from data. Topics include linear and logistic regression, support vector machines, neural networks with backpropagation, convolutional neural networks, recurrent neural networks, methods for representation learning, and application of these techniques under different learning regimes (supervised, unsupervised and reinforcement learning). Real-world applications covered range from computer vision and natural language processing to medical image analysis.

The learning objectives

Upon successful completion of the course, students will be able to:

• Understand and apply the main methodologies and techniques for learning from data.
• Design and implement machine learning methods for real-world applications.
• Analize and extract knowledge in scenarios with limited of no supervision.
• Evaluate the reliability and robustness of machine learning methods in operational scenarios.

Knowledge and understanding

Students will:

• Understand the key concepts of learning from data.
• Learn concepts and tools for building intelligent systems using supervision and no supervision.
• Acquire knowledge to the main machine learning and artificial intelligence methodologies used in industries to support decision-making.
• Understand what the most appropriate techniques are to be used in different real-world applications.

Applying knowledge and understanding

Students will:

• Be able to effectively understand and use the main tools for creating, loading and manipulating datasets.
• Design and implement from scratch machine learning systems following application-derived constraints in terms of modelling and data.
• Understand proper benchmarks and baselines and analyzing achieved results and their generalization in real-world applications.
• Apply methodologies and techniques to analyze data effectively.

Making judgements 

Students will be able to:

• Identify the most suitable model to address a complex data analysis problem.
• Recognize and explain factors leading to underfitting or overfitting.
• Iteratively refine models by designing architectures that captures desired features.

Communication

Students will learn to:

• Critically discuss the strengths and limitations of deep learning methodologies.

Lifelong learning skills

Students will:

• Develop the ability to design a sound and complete methodological approach given a real-world data analysis problem.
• Gain autonomy in applying machine learning techniques beyond the ones presented during the course.
• Learn to design and implement robust, calibrated, effective and efficient deep learning pipelines adapted to specific contexts. 

• Proficiency in Python programming
• Basic knowledge of statistical learning

The course will be addressing the general and modern techniques based on machine and deep learning paradigms to create intelligent systems from data and how it is possible to extract, represent and visualize knowledge from data and trained models. 

Convolutional Neural Networks

• Foundations: padding, strided convolution, dilation, 2D and 3D convolution, pooling
• State of the art models: AlexNet, ResNets, DenseNets, Inception
• Transfer Learning and Data Augmentation

Recurrent Neural Networks

• LSTM and variants
• Attention mechanisms
• Transformers

Unsupervised Learning with Deep Networks

• Representation and Feature Learning
• Autoencoders and Variational Autoencoder

Generative AI

• Generative adversarial networks
• Diffusion models

Explainable AI

• Principles of explainability vs interpretability 
• Post-hoc explanatory methods (e.g., IG and CAM)
• Model agnostic (e.g., SHAP)

Deep Learning Frameworks:

• Overview of the most used DL frameworks
• PyTorch and Jupyter Notebooks

Foundation models:

• Vision-language FM
• Segmentation
• Generation

1. Pattern Recognition and Machine Learning, C. Bishop, 2006
2. Deep Learning. I. Goodfellow, Y. Bengio and A. Courville, MIT Press, 2016
3. Programming PyTorch for Deep Learning, I. Pointer, O'Reilly Media
4. Teaching materials and reading paper list provided by the instructor

The final examination consists of:

• Individual written exam: open-ended questions on the theoretical content of the course.
• Group projects (max 2 students): development and discussion of a Deep Learning Project in PyTorch, accompanied by a written report, in the scientific paper-style form, describing the motivation, methods and results.

The grading policy for the course is:

• Maximum 15 points for the theory test
• Maximum 18 points for the final project

1. What are the main activation functions in neural networks, and how do they affect model learning?

2. What is invariance in convolutional neural networks, and why is it theoretically important?

3. In which tasks do LSTM networks have theoretical advantages compared to basic RNNs?

4. What are common pre-training tasks for transformers, and what role do they play in model learning?

5. How can explainable AI techniques theoretically support trust and understanding of AI systems?