This module covers the fundamental concepts of deep learning methods and how to use them for extracting, modelling and visualizing the learned knowledge.
Topics include: neural networks with backpropagation, convolutional neural networks, recurrent neural networks, methods for representation learning, and how to use them under different learning regimes (supervised, unsupervised and reinforcement learning) and in variety of real-world applications ranging from computer vision, machine translation and medical image analysis.
This module also covers techniques of explainable AI (XAI) for understanding and visualizing how deep models make decisions and their generalization capabilities.
The learning objectives are:
a) to understand and use the main methodologies and techniques for learning from data
b) to understand the main methodologies to design and implement neural networks for real-world applications
c) to understand how to extract and learn knowledge in scenarios when supervision cannot be provided
d) to understand and foresee the reliability of machine learning methods in operational scenarios.
Knowledge and understanding
To understand the main concepts of learning from data
To understand concepts and tools for building intelligent systems using supervision and no supervision
To understand the most important machine learning and artificial intelligence methodologies and techniques used by industries to make sense of data in order to support the decision process
To understand what are the most appropriate techniques to be used in different real-world applications
To understand and interpret how machine learning models work in order to uncover inner mechanisms of black-box methods
Applying knowledge and understanding
To be able to effectively understand and use the main tools for creating, loading and manipulating datasets.
To design and implement from scratch a machine learning system following application-derived constraints in terms of modelling and data
To understand proper benchmarks and baselines and analysing achieved results and their generalization in real-world applications
To be able to apply methodologies and techniques to analyse data.
The module will focus on the implementations of various machine learning techniques and their applications in various domains. The primary tools used in the class are the Python programming language and several associated libraries.
The main teaching methods are as follows:
Lectures, to provide theoretical and methodological knowledge of the subject;
Hands-on exercises, to provide “problem solving” skills and to apply design methodology;
Laboratories, to learn and test the usage of related tools.
Paper reading and presentations to enhance understanding of the core concepts
Seminars by renowned experts (from both universities and industries) in the field to understand the current state of the art.
Should teaching be carried out in mixed mode or remotely, it may be necessary to introduce changes with respect to previous statements, in line with the programme planned and outlined in the syllabus.
Lectures, hands-on exercises, paper reading, student presentations and seminars
The KD module consists of two parts: the first one will be addressing the general and modern techniques based on deep learning paradigm to create KD systems from data, while the second one on how to extract, represent and visualize knowledge from data and trained models.
Part I: Methods and Architectures
Neural Networks and Backpropagation
Derivatives and Gradient Descent
Neural Network Representation, Gradient descent for Neural Networks
Forward and Back Propagation
The revolution of depth: deep learning
Optimization algorithms: Mini-batch gradient descent, Exponentially weighted average, Gradient descent with momentum, RMSprop, Adam optimization algorithm, Learning rate decay
Training aspects of deep learning: Regularization, Dropout, Normalizing inputs, Vanishing / Exploding gradients, Weight Initialization for Deep Networks
Convolutional Neural Networks
Foundations: padding, strided convolution, dilation, 2D and 3D convolution, separable convolution, pooling
State of the art models: AlexNet, ResNets, DenseNets, Inception
Transfer Learning and Data Augmentation
Recurrent Neural Networks
LSTM and variants
Attention mechanisms
Part II: Knowledge Discovery from Data and Models
Unsupervised Learning with Deep Networks
Representation and Feature Learning
Autoencoders and Variational Autoencoders
Generative Adversarial Networks
Graph Neural Networks
Reinforcement Learning
Introduction to Reinforcement Learning
Policy Gradients
Actor-Critic Algorithms
Value Function Methods
Deep RL with Q-functions
Knowledge Discovery in Deep Models: Explainable AI (XAI)
Visualizing Convolutional Neural Decisions
Guided Backpropagation
Deep Generator Networks
CNN Visualization: Activation based and gradient based methods
Deep Learning Frameworks:
Overview of the most used DL frameworks
PyTorch and Jupyter Notebooks
Applications:
Computer vision
Medical Image Analysis
Machine translation
Introduction to the Course
Introduction to Machine Learning
Brief Python review and an overview of Numpy and Pandas
Review of data Characteristics of Data and Preparation and Preprocessing
Supervised Learning
Classification and Prediction using K-Nearest-Neighbor
Classifying with Probability Theory; Naïve Bayes
Building Decision Trees
Regression models
Evaluating predictive models
Ensemble Models: Bagging and Boosting
Unsupervised Learning
Clustering using K-Means
Hierarchical Clustering
Association Rule discovery
Principal Component Analysis and Dimensionality Reduction
Singular Value Decomposition
Brief note on Advance Topics
Matrix Factorization
Support Vector Machines
Search and Optimization Techniques
Markov models; time series analysis, sequential pattern mining
Real application domains
Text Mining and document analysis/filtering
Content analysis, TFxIDF transformation, text categorization, document clustering
Recommender systems
Neighborhood methods (user- and item-based)
Matrix factorization
Marketing and finance data analysis
Deep Learning. I. Goodfellow, Y. Bengio and A. Courville, MIT Press, 2016
Programming PyTorch for Deep Learning, I. Pointer, O'Reilly Media
Teaching materials and reading paper list provided by the instructor
Introduction to Machine Learning, Fourth Edition, By Ethem Alpaydin, MitPress ISBN: 9780262043793. 2020
Python Data Science Essentials - Third Edition by Alberto Boschetti, Luca Massaron, Packt Publishing, ISBN: 9781789537864, 2020
Teaching materials and reading paper list provided by the instructor