Advances Deep Learning

18/03/2019: Scheduling of PhDpresentations is here
04/03/2019: Added slides from NVIDIA and links to graph libraries
23/02/2019: Several fixes to the PhD evaluation schedule
16/02/2019: Added detailed rules and schedule for PhD students evaluation
16/02/2019: Change in schedule, 14/03 lecture is canceled, an additional lecture is added (day to be decided)
14/02/2019: Added material from the first lecture and fixed course rooms
06/02/2019: Rooms for lectures have been changed due to increased, with respect to planned, audience!!!!
06/02/2019: This website is online, please check for material, updates, and announcements here!

Course Aim and Objectives

Nowadays deep learning spans multiple fields in science and engineering, from autonomous driving to human machine interaction. Deep networks have outperformed traditional hand-crafted algorithms, and achieved human performance in solving many complex tasks, such as natural language processing and image recognition. A plethora of papers presenting the success of deep learning in different scenarios is steadily being published, and most often papers frame on a few, very successful, architectures. These architectures are nowadays becoming de facto standards in deep learning such as: convolutional neural networks (CNN), long-short time memories (LSTM), generative adversarial networks (GAN), graph networks, to name a few examples.

Our goal is to provide the students with the skills to understand, become familiar, and use for their research the most successful architectural patterns in deep neural networks. This is intended as an advanced course, thus proficiency in neural networks and basic notions of non linear optimization, i.e., backpropagation, and image/signal processing are assumed as pre-requirement to the participants. For instance, teachers will provide the basics to understand recurrent neural networks and convolutional networks, but feed forward architectures and backpropagation are assumed to be well known notions.

The course presents a general and unified view over most successful architectural patterns in deep neural networks, and the machine learning problems they are naturally suited for. Each lecture will be organized around one of the most relevant deep learning architectures or learning problems (described below), and will be structured as follows:

• a formulation of the addressed problem and brief recall of the required notions and mathematical background when needed
• a unified presentation of the considered model and possible theoretical results
• an overview of the most relevant/successful applications of the considered model
• a brief hands-on demo session to show how to use these models in Python
• an overview of research directions and papers presenting the most important achievements in the literature

These lectures will be planned in a coordinated manner to:

• better exploit the different background of the instructors (Matteucci: Machine Learning and Graphical Models, Boracchi: Image Analysis and Processing, Giusti: Image Classification and Applications)
• better present how these models interact to address the most challenging applications

Detailed Schedule of Lectures

The course is organized in 4 lectures (5 hours each) where the teachers illustrate the models and the theory behind them, and 2 seminars (planned as half day so far) where PhD students, as part of the evaluation, present some papers to the class.

Note: Seminars are part of the lectures an thus they count for PhD attendance.

• Recurrent deep architectures and NLP (07/02/2019 from 09:30 to 13:30 in N.1.2): models for sequence to sequence learning and hierarchical sequence to sequence modeling, long-short term memories, gated recurrent units, memory networks, attention mechanism in natural language and video sequences processing.

• Convolutional Neural Networks for advanced visual recognition tasks (14/02/2019 from 09:30 to 13:30 in N.1.2): Convolutional neural networks, fully convolutional networks for segmentation, object detection, region-proposal networks, semantic annotation, adversarial examples, image captioning, understanding what CNNs learn.

• Unsupervised deep models (21/02/2019 from 09:30 to 13:30 in N.1.2): Generative Models (Pixel RNN/CNN), Variational Autoencoders, Generative Adversarial Networks. Image Restoration through Deep-Learning models, adversarial examples attack.

• Graph Models (28/02/2019 from 09:30 to 13:30 in Sala Coferenze - Ed. 20): Graph convolutional networks, graph to vector, vector to graph, graph to graph modeling, applications in shape analysis and non Euclidean spaces.

• Student Seminars 1 (28/03/2019 from 09:00 to 18:00 in Sala Conferenze - Ed. 20): PhD students seminars
• Student Seminars 2 (17/04/2019 from 09:00 to 18:00 in Sala Conferenze - Ed. 20): PhD students seminars

Course Evaluation

Final assessment differs for PhD students, who will be evaluated on a Pass or Fail scale, and MS students, who will be evaluated on a 30 grades scale for their 5 CFU exams. In particular:

• PhD students will be evaluated on class participation (75% of lectures attendance is mandatory) and on a seminar on the papers proposed by the teachers after the lectures.
• MS students will be evaluated on a small group project to be agreed with the teachers according to this schedule:
  • At the end of each the seminar day, according to the topic, a group of students presents the project to the audience (topic + dataset + model expected to be used + evaluation criteria) and receives comments.
  • By July each group presents a paper (up to 6 pages) reporting the outcome of the project (topic + dataset + modl actually used + comparison with respect to state of art + discussion of results)
  • End of July, or end of August, each group discuss the paper with the teachers

The project option is available also for PhD students not presenting a paper on the defined dates upon request.

PhD Students Evaluation Procedure

Course evaluation for PhD students is via seminar presentation of a paper. These are the general guidelines.

• Each person presents a paper
• Each person is also assigned two extra papers that will be presented by other students, and prepares a couple of interesting questions / comments / remarks to raise during the discussion of these papers.
• PhD students can gather in a group of two, and work together on an common organic presentation of two related papers; during the presentation, each student will deliver the part related to one of the two papers.
• Each presentation is 15 minutes for single person, 25 minutes for 2 persons.
• Presentations are graded in terms of: clarity (presentation must be accessible to competent students who did *not* previously read the paper!), structure, ability to hightlight interesting aspects of the paper while ignoring uninteresting details, ability to highlight relations to related problems. Overtime will be heavily penalized.

Schedule for course evaluation

• Sunday 03/03: students will enroll via a google form (already sent by teachers). Groups (of most 2 PhD students) have to be formed at this time. Each person suggests one paper via the form, papers should be related to the course topics. You are warmily encouraged to browse the proceedings of recent editions of conferences like ICCV, CVPR, ECCV, ICLR, NIPS, ICML or recent papers citing papers presented during classes.
• Sunday 10/03: teachers communicate schedule and final papers (teachers can change students proposals) + additional papers to students
• Thursday 28/03: full day of presentations in Sala Conferenze.
• Wednesday 17/04: full day of presentations in Sala Conferenze

We recall that presentations are part of the course and as such attendance is mandatory.

Course Material

The course material includes the teachers slides, references to the literature, and the papers to base seminars upon.

Teachers Slides

Recurrent deep architectures and NLP

• Course Intro: Course aims and objectives, course organization, schedule, evaluation procedures and the lik.
• Recurrent Neural Networks: Introduction to sequence modeling, recurrent neural networks, vaniscing gradient, long short-term memories
• Sequence 2 Sequence: Models for Sequence to Sequence modeling, attention mechanisms, Neural Turing Machines
• Word Embedding: Word embedding, word2vec, GloVe

Graph Models

• Learning on Graphs: node embedding, node2vec, neighbors aggregation, Graph Convolutional Networks

Convolutional Neural Networks for advanced visual recognition tasks

• Advanced CNN
• Advanced CNN continued

Unsupervised deep models

• TBC

Super resolution @ NVIDIA

• Super Resolution at NVIDIA: slides of the talk about Super Resolution by Marco Foco

References to Literature and Useful Links

• Domain adaptation survey paper
• Deep Graph Library: A pytorch library to perform deep learning on graphs