Difference between revisions of "Advances Deep Learning"

Revision as of 08:35, 6 February 2019

Course Aim and Objectives

Nowadays deep learning spans multiple fields in science and engineering, from autonomous driving to human machine interaction. Deep networks have outperformed traditional hand-crafted algorithms, and achieved human performance in solving many complex tasks, such as natural language processing and image recognition. A plethora of papers presenting the success of deep learning in different scenarios is steadily being published, and most often papers frame on a few, very successful, architectures. These architectures are nowadays becoming de facto standards in deep learning such as: convolutional neural networks (CNN), long-short time memories (LSTM), generative adversarial networks (GAN), graph networks, to name a few examples.

Our goal is to provide the students with the skills to understand, become familiar, and use for their research the most successful architectural patterns in deep neural networks. This is intended as an advanced course, thus proficiency in neural networks and basic notions of non linear optimization, i.e., backpropagation, and image/signal processing are assumed as pre-requirement to the participants. For instance, teachers will provide the basics to understand recurrent neural networks and convolutional networks, but feed forward architectures and backpropagation are assumed to be well known notions.

The course presents a general and unified view over most successful architectural patterns in deep neural networks, and the machine learning problems they are naturally suited for. Each lecture will be organized around one of the most relevant deep learning architectures or learning problems (described below), and will be structured as follows:

• a formulation of the addressed problem and brief recall of the required notions and mathematical background when needed
• a unified presentation of the considered model and possible theoretical results
• an overview of the most relevant/successful applications of the considered model
• a brief hands-on demo session to show how to use these models in Python
• an overview of research directions and papers presenting the most important achievements in the literature

These lectures will be planned in a coordinated manner to:

• better exploit the different background of the instructors (Matteucci: Machine Learning and Graphical Models, Boracchi: Image Analysis and Processing, Giusti: Image Classification and Applications)
• better present how these models interact to address the most challenging applications

Detailed Schedule of Lectures

The course is organized in 4 lectures (5 hours each) where the teachers illustrate the and, tentatively, 2 seminars (planned as full day)

• Lecture 1 (07/02/2019 from 09:00 to 14:00 in Sala Seminari - Ed. 20): Recurrent deep architectures and NLP: models for sequence to sequence learning and hierarchical sequence to sequence modeling, long-short term memories, gated recurrent units, memory networks, attention mechanism in natural language and video sequences processing.

• Lecture 2 (14/02/2019 from 09:00 to 14:00 in Sala Seminari - Ed. 20): Convolutional Neural Networks for advanced visual recognition tasks: Fully convolutional networks for segmentation, object detection, region-proposal networks, semantic annotation, adversarial examples, image captioning, understanding what CNNs learn.

• Lecture 3 (21/02/2019 from 09:00 to 14:00 in Sala Seminari - Ed. 20): Unsupervised deep models: Generative Models: Pixel RNN/CNN, Variational Autoencoders, Generative Adversarial Networks. Image Restoration through Deep-Learning models, adversarial examples attack.

• Lecture 4 (28/02/2019 from 09:00 to 14:00 in Sala Seminari - Ed. 20): Graphical Models: Graph Convolutional Networks, Graph to vector, vector to graph, graph to graph modeling, applications in shape analysis and non Euclidean spaces.

• Lecture 5 (14/03/2019 from 09:00 to 14:00 in Sala Seminari - Ed. 20): Seminars on advanced visual recognition tasks and unsupervised deep models
• Lecture 6 (28/03/2019 from 09:00 to 14:00 in Aula Seminari Alessandra Alario Ed. 21): Seminars on Graphical Models and recurrent deep architectures