Difference between revisions of "Artificial Neural Networks and Deep Learning"
(24 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
The following are last minute news you should be aware of ;-) | The following are last minute news you should be aware of ;-) | ||
− | 08/08/2020: Published [[Media:AN2DL_Grades_20200715_challenges.pdf|the results of the 15/07/2020 written call]] with ALL challenges included | + | 14/09/2020: So far, accoding to the University rules Team 1 is defined by an EVEN Codice Persona, while Team 2 by an ODD Codice persona |
+ | 13/09/2020: The course is about to start ... stay tuned! | ||
+ | |||
+ | <!-- [2019/2020] 08/08/2020: Published [[Media:AN2DL_Grades_20200715_challenges.pdf|the results of the 15/07/2020 written call]] with ALL challenges included | ||
12/07/2020: Published [[Media:AN2DL_Grades_20200619_challenges.pdf|the results of the 19/06/2020 written call]] with ALL challenges included | 12/07/2020: Published [[Media:AN2DL_Grades_20200619_challenges.pdf|the results of the 19/06/2020 written call]] with ALL challenges included | ||
15/03/2020: Published [[Media:AN2DL_Grades_20200210_challenges.pdf|the results of the 10/02/2020 written call]] with ALL challenges included | 15/03/2020: Published [[Media:AN2DL_Grades_20200210_challenges.pdf|the results of the 10/02/2020 written call]] with ALL challenges included | ||
Line 22: | Line 25: | ||
04/10/2019: Today's lecture moved to 4.0.1!!! | 04/10/2019: Today's lecture moved to 4.0.1!!! | ||
26/09/2019: Uploaded slides about deep learning | 26/09/2019: Uploaded slides about deep learning | ||
− | 24/09/2019: Thursday 26/09/2019 lecture will be in room [https://www7.ceda.polimi.it/spazi/spazi/controller/Aula.do?evn_init=event&idaula=20&jaf_currentWFID=main B21] | + | 24/09/2019: Thursday 26/09/2019 lecture will be in room [https://www7.ceda.polimi.it/spazi/spazi/controller/Aula.do?evn_init=event&idaula=20&jaf_currentWFID=main B21] |
19/09/2019: No lecture on the 20/09/2019 ... check the detailed schedule. | 19/09/2019: No lecture on the 20/09/2019 ... check the detailed schedule. | ||
19/09/2019: The course starts today! | 19/09/2019: The course starts today! | ||
+ | --> | ||
==Course Aim & Organization== | ==Course Aim & Organization== | ||
Line 38: | Line 42: | ||
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant. | The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant. | ||
− | * [https://www.deib.polimi.it/ita/personale/dettagli/267262 Matteo Matteucci]: the course teacher | + | * [https://www.deib.polimi.it/ita/personale/dettagli/267262 Matteo Matteucci]: the course teacher and this is his [https://politecnicomilano.webex.com/join/matteo.matteucci webex room] |
− | * [https://www.deib.polimi.it/ita/personale/dettagli/549640 Giacomo Boracchi]: the course teacher | + | * [https://www.deib.polimi.it/ita/personale/dettagli/549640 Giacomo Boracchi]: the course co-teacher and this is his [https://politecnicomilano.webex.com/join/giacomo.boracchi webex room] |
− | * [https://www.deib.polimi.it/ita/personale/dettagli/846174 Francesco Lattari]: the course teaching assistant | + | * [https://www.deib.polimi.it/ita/personale/dettagli/846174 Francesco Lattari]: the course teaching assistant and this is his [https://politecnicomilano.webex.com/join/francesco.lattari webex room] |
===Course Program and Syllabus=== | ===Course Program and Syllabus=== | ||
Line 52: | Line 56: | ||
We have compiled a detailed syllabus of the course student can use to doublecheck their preparation against before the exam. | We have compiled a detailed syllabus of the course student can use to doublecheck their preparation against before the exam. | ||
− | * [[Media:AN2DL_Syllabus.pdf|Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam | + | * [[Media:AN2DL_Syllabus.pdf|[2019/2020] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam |
===Detailed course schedule=== | ===Detailed course schedule=== | ||
Line 59: | Line 63: | ||
Note: Lecture timetable interpretation | Note: Lecture timetable interpretation | ||
− | * On | + | * On Wednesday, in 2.0.2 (EX N.0.2), starts at 15:15, ends at 17:15 |
− | * On | + | * On Wednesday, in 2.1.2 (EX N.1.2), starts at 17:15, ends at 19:15 |
+ | * On Thursday, in teacher webex room, starts at 16:15, ends at 19:15 | ||
+ | {| border="1" align="center" style="text-align:center;" | ||
+ | |- | ||
+ | |Date || Day || Time || Room || Teacher || Type || Topic | ||
+ | |- | ||
+ | |16/09/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Course Introduction [Team 1] | ||
+ | |- | ||
+ | |16/09/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Course Introduction [Team 2] | ||
+ | |- | ||
+ | |17/09/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Perceptron and Hebbian Learning | ||
+ | |- | ||
+ | |23/09/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Python and Hebbian Learning Example [Team 1] | ||
+ | |- | ||
+ | |23/09/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Python and Hebbian Learning Example [Team 2] | ||
+ | |- | ||
+ | |24/09/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Feed forward neural networks + Backprop | ||
+ | |- | ||
+ | |30/09/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Backpropagation Example (with Hebbian parallel) [Team 1] | ||
+ | |- | ||
+ | |30/09/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Backpropagation Example (with Hebbian parallel) [Team 2] | ||
+ | |- | ||
+ | |01/10/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Error Functions and Weight decay | ||
+ | |- | ||
+ | |07/10/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN - Pytorch and FFNN [Team 1] | ||
+ | |- | ||
+ | |07/10/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN - Pytorch and FFNN [Team 2] | ||
+ | |- | ||
+ | |08/10/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Facing overffitting .. and other training tricks | ||
+ | |- | ||
+ | |14/10/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN - FFNN and Overfitting [Team 1] | ||
+ | |- | ||
+ | |14/10/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN - FFNN and Overfitting [Team 2] | ||
+ | |- | ||
+ | |15/10/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || -- No Lecture Today -- | ||
+ | |- | ||
+ | |21/10/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || Introduction to Image classification [Team 1] | ||
+ | |- | ||
+ | |21/10/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || Introduction to Image classification [Team 2] | ||
+ | |- | ||
+ | |22/10/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || Convolutional Training Netwroks | ||
+ | |- | ||
+ | |28/10/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || Famous CNN Architectures [Team 1] | ||
+ | |- | ||
+ | |28/10/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || Famous CNN Architectures [Team 2] | ||
+ | |- | ||
+ | |29/10/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || Training with data scarcity | ||
+ | |- | ||
+ | |04/11/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN - Convolutional Neural Networks [Team 1] | ||
+ | |- | ||
+ | |04/11/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN - Convolutional Neural Networks [Team 1] | ||
+ | |- | ||
+ | |05/11/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || Fully Convolutional CNN, CNN for image segmentation | ||
+ | |- | ||
+ | |11/11/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || --- || -- No Lecture (Prove in Itinere) -- | ||
+ | |- | ||
+ | |12/11/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || --- || -- No Lecture (Prove in Itinere) -- | ||
+ | |- | ||
+ | |11/11/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || CNN for localization and detection | ||
+ | |- | ||
+ | |18/11/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || GANs [Team 1] | ||
+ | |- | ||
+ | |18/11/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || GANs [Team 2] | ||
+ | |- | ||
+ | |19/11/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Recurrent neural networks + LSTM | ||
+ | |- | ||
+ | |25/11/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Word Embedding [Team 1] | ||
+ | |- | ||
+ | |25/11/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Word Embedding [Team 2] | ||
+ | |- | ||
+ | |26/11/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Sequence to sequence learning | ||
+ | |- | ||
+ | |02/12/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN [Team 1] | ||
+ | |- | ||
+ | |02/12/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || KERAS NN [Team 2] | ||
+ | |- | ||
+ | |03/12/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Attention Mechanism and Transformer | ||
+ | |- | ||
+ | |09/12/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || KERAS NN [Team 1] | ||
+ | |- | ||
+ | |09/12/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || KERAS NN [Team 2] | ||
+ | |- | ||
+ | |10/12/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || Questions and Answers | ||
+ | |- | ||
+ | |16/12/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || --- || -- Spare Lecture -- | ||
+ | |- | ||
+ | |16/12/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || --- || -- Spare Lecture -- | ||
+ | |- | ||
+ | |17/12/2020 || Thursday || 16:15 - 19:15 || Virtual Room || Lecture || --- || -- Spare Lecture -- | ||
+ | |- | ||
+ | |23/12/2020 || Wednesday || 15:15 - 17:15 || 2.0.2 || Lecture || --- || -- Spare Lecture -- | ||
+ | |- | ||
+ | |23/12/2020 || Wednesday || 15:15 - 17:15 || 2.1.2 || Lecture || --- || -- Spare Lecture -- | ||
+ | |- | ||
+ | |} | ||
+ | |||
+ | |||
+ | <!-- [2019/2020] | ||
{| border="1" align="center" style="text-align:center;" | {| border="1" align="center" style="text-align:center;" | ||
|- | |- | ||
Line 125: | Line 226: | ||
|- | |- | ||
|} | |} | ||
+ | --> | ||
===Course Evaluation=== | ===Course Evaluation=== | ||
Line 130: | Line 232: | ||
Course evaluation is composed of two parts: | Course evaluation is composed of two parts: | ||
− | * A written examination covering the whole program graded up to | + | * A written examination covering the whole program graded up to 26/32 ... may be less |
− | * A home project in the form of a Kaggle style competition practicing the topics of the course graded up to | + | * A home project in the form of a "Kaggle style" competition practicing the topics of the course graded up to 6/32 ... may be more |
The final score will sum the grade of the written exam and the grade of the home project. | The final score will sum the grade of the written exam and the grade of the home project. | ||
Line 164: | Line 266: | ||
===Kaggle Homeworks=== | ===Kaggle Homeworks=== | ||
− | As part of the evaluation (up to | + | As part of the evaluation (up to 6 marks in the final grade) we are issuing 3 homeworks in the form of "Kaggle style" competitions. They are meant to practice the course topics on simple image recognition tasks. |
+ | |||
+ | ====[2020/2021]==== | ||
+ | |||
+ | Not yet published | ||
+ | |||
+ | ====[2019/2020]==== | ||
*[https://www.kaggle.com/t/5e52f27cd4fb485185e6b627c1fb0335 Image Classification Homework]: the first homework is about image classification with convolutional neural networks. The deadline to submit the results is November the 30th. | *[https://www.kaggle.com/t/5e52f27cd4fb485185e6b627c1fb0335 Image Classification Homework]: the first homework is about image classification with convolutional neural networks. The deadline to submit the results is November the 30th. | ||
*[https://www.kaggle.com/t/ef3b29a4a5d74b2fbb3e3ec50ca4e206 Image Segmentation Homework]: the second homework is about image segmentation with convolutional neural networks and the like. The deadline to submit the results is December the 17th. | *[https://www.kaggle.com/t/ef3b29a4a5d74b2fbb3e3ec50ca4e206 Image Segmentation Homework]: the second homework is about image segmentation with convolutional neural networks and the like. The deadline to submit the results is December the 17th. | ||
*[https://www.kaggle.com/t/6d6fd208f06f4d1abd71d47bd36586ce Visual Question Answering Homework]: the third homework is about visual question answering with convolutional and recurrent neural networks ... plus word2vec. The deadline to submit the results is January the 15th. | *[https://www.kaggle.com/t/6d6fd208f06f4d1abd71d47bd36586ce Visual Question Answering Homework]: the third homework is about visual question answering with convolutional and recurrent neural networks ... plus word2vec. The deadline to submit the results is January the 15th. |
Revision as of 23:21, 13 September 2020
The following are last minute news you should be aware of ;-)
14/09/2020: So far, accoding to the University rules Team 1 is defined by an EVEN Codice Persona, while Team 2 by an ODD Codice persona 13/09/2020: The course is about to start ... stay tuned!
Contents
Course Aim & Organization
Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.
Nowadays, deep neural network can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).
The course major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.
Teachers
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.
- Matteo Matteucci: the course teacher and this is his webex room
- Giacomo Boracchi: the course co-teacher and this is his webex room
- Francesco Lattari: the course teaching assistant and this is his webex room
Course Program and Syllabus
This goal is pursued in the course by:
- Presenting major theoretical results underpinning NN (e.g., universal approx, vanishing/exploding gradient, etc.)
- Describing the most important algorithms for NN training (e.g., backpropagation, adaptive gradient algorithms, etc.)
- Illustrating the best practices on how to successfully train and use these models (e.g., dropout, data augmentation, etc.)
- Providing an overview of the most successful Deep Learning architectures (e.g., CNNs, sparse and dense autoencoder, LSTMs for sequence to sequence learning, etc.)
- Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.
We have compiled a detailed syllabus of the course student can use to doublecheck their preparation against before the exam.
- [2019/2020] Course Syllabus: a detailed list of topics covered by the course and which students are expected to know when approaching the exam
Detailed course schedule
A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all days we have lectures!!
Note: Lecture timetable interpretation * On Wednesday, in 2.0.2 (EX N.0.2), starts at 15:15, ends at 17:15 * On Wednesday, in 2.1.2 (EX N.1.2), starts at 17:15, ends at 19:15 * On Thursday, in teacher webex room, starts at 16:15, ends at 19:15
Date | Day | Time | Room | Teacher | Type | Topic |
16/09/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Matteo Matteucci | Course Introduction [Team 1] |
16/09/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Matteo Matteucci | Course Introduction [Team 2] |
17/09/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Perceptron and Hebbian Learning |
23/09/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Matteo Matteucci | Python and Hebbian Learning Example [Team 1] |
23/09/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Matteo Matteucci | Python and Hebbian Learning Example [Team 2] |
24/09/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Feed forward neural networks + Backprop |
30/09/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Matteo Matteucci | Backpropagation Example (with Hebbian parallel) [Team 1] |
30/09/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Matteo Matteucci | Backpropagation Example (with Hebbian parallel) [Team 2] |
01/10/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Error Functions and Weight decay |
07/10/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Francesco Lattari | KERAS NN - Pytorch and FFNN [Team 1] |
07/10/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Francesco Lattari | KERAS NN - Pytorch and FFNN [Team 2] |
08/10/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Facing overffitting .. and other training tricks |
14/10/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Francesco Lattari | KERAS NN - FFNN and Overfitting [Team 1] |
14/10/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Francesco Lattari | KERAS NN - FFNN and Overfitting [Team 2] |
15/10/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | -- No Lecture Today -- |
21/10/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Giacomo Boracchi | Introduction to Image classification [Team 1] |
21/10/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Giacomo Boracchi | Introduction to Image classification [Team 2] |
22/10/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Giacomo Boracchi | Convolutional Training Netwroks |
28/10/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Giacomo Boracchi | Famous CNN Architectures [Team 1] |
28/10/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Giacomo Boracchi | Famous CNN Architectures [Team 2] |
29/10/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Giacomo Boracchi | Training with data scarcity |
04/11/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Francesco Lattari | KERAS NN - Convolutional Neural Networks [Team 1] |
04/11/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Francesco Lattari | KERAS NN - Convolutional Neural Networks [Team 1] |
05/11/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Giacomo Boracchi | Fully Convolutional CNN, CNN for image segmentation |
11/11/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | --- | -- No Lecture (Prove in Itinere) -- |
12/11/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | --- | -- No Lecture (Prove in Itinere) -- |
11/11/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Giacomo Boracchi | CNN for localization and detection |
18/11/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Giacomo Boracchi | GANs [Team 1] |
18/11/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Giacomo Boracchi | GANs [Team 2] |
19/11/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Recurrent neural networks + LSTM |
25/11/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Matteo Matteucci | Word Embedding [Team 1] |
25/11/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Matteo Matteucci | Word Embedding [Team 2] |
26/11/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Sequence to sequence learning |
02/12/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Francesco Lattari | KERAS NN [Team 1] |
02/12/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Francesco Lattari | KERAS NN [Team 2] |
03/12/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Attention Mechanism and Transformer |
09/12/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | Matteo Matteucci | KERAS NN [Team 1] |
09/12/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | Matteo Matteucci | KERAS NN [Team 2] |
10/12/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | Matteo Matteucci | Questions and Answers |
16/12/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | --- | -- Spare Lecture -- |
16/12/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | --- | -- Spare Lecture -- |
17/12/2020 | Thursday | 16:15 - 19:15 | Virtual Room | Lecture | --- | -- Spare Lecture -- |
23/12/2020 | Wednesday | 15:15 - 17:15 | 2.0.2 | Lecture | --- | -- Spare Lecture -- |
23/12/2020 | Wednesday | 15:15 - 17:15 | 2.1.2 | Lecture | --- | -- Spare Lecture -- |
Course Evaluation
Course evaluation is composed of two parts:
- A written examination covering the whole program graded up to 26/32 ... may be less
- A home project in the form of a "Kaggle style" competition practicing the topics of the course graded up to 6/32 ... may be more
The final score will sum the grade of the written exam and the grade of the home project.
You can find here one example of the exam text to get a flavor of what to expect in the written examination.
Teaching Material (the textbook)
Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference you can check the following text, but keep in mind that teachers will not follow it strictly
- Deep Learning. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
Course Slides
Slides from the lectures by Matteo Matteucci
- [2019/2020] Course Introduction: introductory slides of the course with useful information about the course syllabus, grading, and the course logistics.
- [2019/2020] Machine Learning vs Deep Learning: introduction to machine learning paradigms and definition of deep learning with examples
- [2019/2020] From Perceptrons to Feed Forward Neural Networks: the original Perceptron model, Hebbian learning, feed-forward architecture, backpropagation and gradient descent, error functions and maximum likelihood estimation
- [2019/2020] Neural Networks Training: dealing with overfitting (weight decay, early stopping, dropout), vanishing gradient (ReLU and friends), batch normalization
- [2019/2020] Recurrent Neural Networks: learning with sequences, Recurrent Neural Networks, vanishing gradient, Long Short-Term Memories (LSTM).
- [2019/2020] Sequence 2 Sequence Learning: sequence to sequence learning as an encoder-decoder problem, Neural Turing Machines, attention mechanisms, the Transformer.
- [2019/2020] Word Embedding: neural autoencoders, language models, word embedding, word2vec, glove.
Slides from the lectures by Giacomo Boracchi are available in his webpage, for you
- Image Classification: Image classification and related issues, template matching, image classification via nearest neighbors methods, image classification via linear classifiers, image classification via hand-crafted features.
Slides from the practicals by Francesco Lattari
- [2019/2020] Introduction to Keras: Introduction to Keras and Tensorflow2 (slides + notebook)
- [2019/2020] Convolutional architectures in Keras: How to build, train, and evaluate convolutional models for classification and segmentation in Keras and Tensorflow2 (slides + notebook)
- [2019/2020] Recurrent architectures in Keras: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
Kaggle Homeworks
As part of the evaluation (up to 6 marks in the final grade) we are issuing 3 homeworks in the form of "Kaggle style" competitions. They are meant to practice the course topics on simple image recognition tasks.
[2020/2021]
Not yet published
[2019/2020]
- Image Classification Homework: the first homework is about image classification with convolutional neural networks. The deadline to submit the results is November the 30th.
- Image Segmentation Homework: the second homework is about image segmentation with convolutional neural networks and the like. The deadline to submit the results is December the 17th.
- Visual Question Answering Homework: the third homework is about visual question answering with convolutional and recurrent neural networks ... plus word2vec. The deadline to submit the results is January the 15th.