Difference between revisions of "Artificial Neural Networks and Deep Learning"

From Chrome
Jump to: navigation, search
(Detailed course schedule)
(Course Slides)
 
(305 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
__FORCETOC__
 
__FORCETOC__
  
The following are last minute news you should be aware of ;-)
+
The following are last-minute news you should be aware of ;-)
 +
* 13/09/2023: A new edition of the AN2DL course starts today!!!
 +
 
 +
<!--
 +
* 12/03/2023: Grades of the [[Media:AN2DL_20230217.pdf|AN2DL 17/02/2023 call]] will appear here very soon!!!
 +
* 15/02/2023: Updates of grades from the [[Media:AN2DL_20230126_bis.pdf|AN2DL 26/01/2023 call]].
 +
* 14/02/2023: Grades of the [[Media:AN2DL_20230126.pdf|AN2DL 26/01/2023 call]].
 +
* 12/02/2023: Grades of the [[Media:AN2DL_2022_2023_Challenges.pdf|AN2DL Challenges]].
 +
* 23/01/2023: added link to the detailed course syllabus
 +
* 08/01/2023: update of the teaching material in the page ... a new detailed syllabus will follow too
 +
* 21/12/2022: Grades of the [[Media:AN2DL_2022_2023_FirstChallenge.pdf|first challenge (v2)]].
 +
-->
 +
<!--
 +
07/08/2022: Grades for the [[Media:AN2DL_Grades_20220726.pdf|06/07/2022 call are available here!]]
 +
06/03/2021: Grades for the [[Media:AN2DL2122Homeworks.pdf|2021/2021 homeworks are available here!]]
 +
24/11/2021: Lecture on 09/12/2021 moved to 16/12/2021
 +
24/11/2021: Change of classroom on the 15/12 from T2.1 to B4.3
 +
12/11/2021: [https://codalab.lisn.upsaclay.fr/competitions/226 Here is the link to the First Homework!!!] (read the registration rules, they have been updated)
 +
12/11/2021: First homework is coming out! [https://docs.google.com/document/d/1T0EXf5I8knJQjK4wveewKsOxkV1IyVRlLrKs7S7yO0M/edit?usp=sharing Register to submit your solutions!]
 +
07/11/2021: We have restructured the notebooks from the labs, please check the new organization and material
 +
13/10/2021: Final grade for 2020/2021 year are [[Media:AN2DL_Grades_20210830.pdf|here]]
 +
30/09/2021: Video and slides updated + notebooks published
 +
24/09/2021: Removed last year practicals, this year they will be different!!! Python crash course moved at the end of the material
 +
22/09/2021: Added colab crash course on Python 3 in the material session
 +
18/09/2021: We skip exercising on 22/09 and added exercising on 15/12 I updated the schedule!
 +
16/09/2021: Updated material on the first lectures + added reference to python tutorials in the material section
 +
15/09/2021: Lectures start today!
 +
14/09/2021: Website under maintenance ... come back later
 +
 
 +
07/08/2021: Result from [[Media:AN2DL_Grades_20210728.pdf|28/07/2021 call with all homeworks]] are available here
 +
25/07/2021: Result from [[Media:AN2DL_Grades_20210708.pdf|08/07/2021 call with all homeworks]] are available here
 +
26/05/2021: The link to the form to request an instance of remote examination for June and July is [https://forms.office.com/r/i5fFDsPNGa here!]
 +
17/02/2021: Result from [[Media:AN2DL_Grades_20210125_7.pdf|25/01/2020 call with all homeworks]] are available here
 +
12/02/2021: Result from [[Media:AN2DL_Grades_20210125_6.pdf|25/01/2020 call with some (~85) of the third homeworks]] are available here
 +
10/02/2021: Result from [[Media:AN2DL_Grades_20210125_5.pdf|25/01/2020 call with second homeworks]] are available here
 +
09/02/2021: Result from [[Media:AN2DL_Grades_20210125_4.pdf|25/01/2020 call with some (~210) of the second homeworks]] are available here
 +
09/02/2021: Result from [[Media:AN2DL_Grades_20210125_3.pdf|25/01/2020 call with some (~120) of the second homeworks]] are available here
 +
09/02/2021: Result from [[Media:AN2DL_Grades_20210125_2.pdf|25/01/2020 call with some (~50) of the second homeworks]] are available here
 +
09/02/2021: Result from [[Media:AN2DL_Grades_20210125_1.pdf|25/01/2020 call with first homework]] are available here
 +
04/01/2021: Updated course syllabus (removed batch normalization)
 +
29/12/2020: An extra point for the students who participate to the second phase of the [https://chrome.deib.polimi.it/index.php?title=Artificial_Neural_Networks_and_Deep_Learning#.5B2020.2F2021.5D Image Segmentation task]!
 +
29/12/2020: The [https://chrome.deib.polimi.it/index.php?title=Artificial_Neural_Networks_and_Deep_Learning#.5B2020.2F2021.5D third challenge] is out! You have until 31st January 2021 to submit it ... no extensions this time ;-)         
 +
29/12/2020: Updated syllabus published
 +
29/12/2020: Past exams published
 +
11/12/2020: Uploaded the material from last week lectures!!!
 +
03/12/2020: Second course challenge is out! Check it [https://chrome.deib.polimi.it/index.php?title=Artificial_Neural_Networks_and_Deep_Learning#.5B2020.2F2021.5D here], you have to deliver it by 20th December 2020
 +
19/11/2020: Uploaded slides for today and the next days and past lectures videos
 +
18/11/2020: Updated schedule to reflect Teams merge fron today on
 +
08/11/2020: First homework challenge published [https://chrome.deib.polimi.it/index.php?title=Artificial_Neural_Networks_and_Deep_Learning#.5B2020.2F2021.5D here]
 +
18/10/2020: Updated links to lectures videos and lab notebooks
 +
14/10/2020: Requested changes of team have been authorized by Presidenza ... stay tuned I will update you on this soon
 +
13/10/2020: IMPORTANT CHANGE !!! -> Tomorrow 14/10/2020 lectures will be issued ONLY ONLINE!!!!
 +
09/10/2020: Uploaded lab notebooks and lab recordings
 +
08/10/2020: Change of schedule on the 15/10/2020 we will have online lecture
 +
08/10/2020: Today's video uploaded and Neural Networks Traning Slides updated in the cross-validation part
 +
06/10/2020: IMPORTANT CHANGE !!! -> Tomorrow 07/10/2020 lectures will be issued ONLY ONLINE!!!!
 +
02/10/2020: Published a guide to install the software which will be used in the labs [https://chrome.deib.polimi.it/index.php?title=Artificial_Neural_Networks_and_Deep_Learning#Lab_software_setup here]
 +
02/10/2020: Published fixed slides about feed forward neural networks
 +
01/10/2020: Pool to request the change of Team is [https://forms.office.com/Pages/ResponsePage.aspx?id=K3EXCvNtXUKAjjCd8ope6ztteKg6OERCsstxb4n43e9UMEw4TzNLTllJTE5UMUxUNUM1NTBYTlJFWC4u here]
 +
23/09/2020: Tomorrow 24/09/2020 we are going to have the online lecture as planned
 +
23/09/2020: Today's lectures published
 +
22/09/2020: Added slides on Perceptron, Hebbian learning and feed forward neural networks
 +
20/09/2020: Added links to the lecture recordings and uploaded slides
 +
14/09/2020: FIX - Team 1 ODD numbers, Team 2 EVEN numbers !!!
 +
14/09/2020: FIX - the hours of the second team were overlapping to the first, now they are correctly one after the other
 +
13/09/2020: The course is about to start ... stay tuned!
 +
-->
 +
 
 +
<!-- [2019/2020] 08/08/2020: Published [[Media:AN2DL_Grades_20200715_challenges.pdf|the results of the 15/07/2020 written call]] with ALL challenges included
 +
12/07/2020: Published [[Media:AN2DL_Grades_20200619_challenges.pdf|the results of the 19/06/2020 written call]] with ALL challenges included
 +
15/03/2020: Published [[Media:AN2DL_Grades_20200210_challenges.pdf|the results of the 10/02/2020 written call]] with ALL challenges included
 +
06/01/2020: Published [[Media:AN2DL_Grades_20200115_challenges.pdf|the results of the 15/01/2020 written call]] with ALL challenges included
 +
06/01/2020: Published [[Media:AN2DL_Grades_20200115.pdf|the results of the 15/01/2020 written call]] challenges included
 +
05/01/2020: Published [[Media:AN2DL_Grades_20200115_tmp.pdf|the results of the 15/01/2020 written call]] (challenges results will come tomorrow)
 +
27/01/2020: Published [[Media:AN2DL_ExamExample.pdf|an example of the exam text]] in the "Course Evaluation" session
 
  13/01/2020: You can find here a detailed [[Media:AN2DL_Syllabus.pdf|list of topics]] from the course you might expect will be requested at the exam
 
  13/01/2020: You can find here a detailed [[Media:AN2DL_Syllabus.pdf|list of topics]] from the course you might expect will be requested at the exam
 
  24/12/2019: Uploaded slides from last Lab and reference to the third and last competition!
 
  24/12/2019: Uploaded slides from last Lab and reference to the third and last competition!
Line 15: Line 89:
 
  04/10/2019: Today's lecture moved to 4.0.1!!!
 
  04/10/2019: Today's lecture moved to 4.0.1!!!
 
  26/09/2019: Uploaded slides about deep learning
 
  26/09/2019: Uploaded slides about deep learning
  24/09/2019: Thursday 26/09/2019 lecture will be in room [https://www7.ceda.polimi.it/spazi/spazi/controller/Aula.do?evn_init=event&idaula=20&jaf_currentWFID=main B21] !!!
+
  24/09/2019: Thursday 26/09/2019 lecture will be in room [https://www7.ceda.polimi.it/spazi/spazi/controller/Aula.do?evn_init=event&idaula=20&jaf_currentWFID=main B21]
 
  19/09/2019: No lecture on the 20/09/2019 ... check the detailed schedule.
 
  19/09/2019: No lecture on the 20/09/2019 ... check the detailed schedule.
 
  19/09/2019: The course starts today!
 
  19/09/2019: The course starts today!
 +
-->
  
 
==Course Aim & Organization==
 
==Course Aim & Organization==
Line 23: Line 98:
 
Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.
 
Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.
  
Nowadays, deep neural network can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).  
+
Nowadays, deep neural networks can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).  
  
The course major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.
+
The course's major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.
  
 
===Teachers===
 
===Teachers===
Line 31: Line 106:
 
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.
 
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.
  
* [https://www.deib.polimi.it/ita/personale/dettagli/267262 Matteo Matteucci]: the course teacher
+
* [https://www.deib.polimi.it/ita/personale/dettagli/267262 Matteo Matteucci] and [https://www.deib.polimi.it/ita/personale/dettagli/549640 Giacomo Boracchi]: the course teachers
* [https://www.deib.polimi.it/ita/personale/dettagli/549640 Giacomo Boracchi]: the course teacher
+
* [https://www.deib.polimi.it/ita/personale/dettagli/846174 Francesco Lattari], [https://www.deib.polimi.it/ita/personale/dettagli/920490 Eugenio Lomurno], and [https://www.deib.polimi.it/ita/personale/dettagli/1107274 Loris Giulivi]: the course teaching assistants
* [https://www.deib.polimi.it/ita/personale/dettagli/846174 Francesco Lattari]: the course teaching assistant
+
  
 
===Course Program and Syllabus===
 
===Course Program and Syllabus===
Line 44: Line 118:
 
* Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.
 
* Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.
  
We have compiled a detailed syllabus of the course student can use to doublecheck their preparation against before the exam.  
+
We have compiled a detailed syllabus of the course that students can use to double-check their preparation before the exam.
* [[Media:AN2DL_Syllabus.pdf|Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam  
+
* [https://docs.google.com/document/d/1BbcCJ7Om5LRvPCGeYkUkCk1RMjoi_ULDakkg0epsVFA/edit?usp=sharing Detailed Course Syllabus]
 +
<!--* [[Media:AN2DL_Syllabus_2022_2023.pdf|[2022/2023] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam-->
 +
<!--* [[Media:AN2DL_Syllabus_2021.pdf|[2020/2021] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam-->
 +
<!--* [[Media:AN2DL_Syllabus.pdf|[2019/2020] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam-->
  
===Detailed course schedule===
+
===Detailed course schedule and recordings===
  
A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all days we have lectures!!
+
A detailed schedule of the course is given in the form of google calendar; topics are just indicative while days and teachers are correct up to some last-minute change (I will notify you by email). Please note that not all days we have lectures!!
  
Note: Lecture timetable interpretation
+
For the [https://boracchi.faculty.polimi.it/teaching/AN2DLCalendar_CS.htm AN2DL Course Google Calendar] look here!
* On Thursday, in L26.12, starts at 16:15, ends at 18:15
+
* On Friday, in 6.0.1, starts at 14:15, ends at 17:15
+
  
 +
Note: the course is given in parallel to two sessions, the Computer Science session, and the Bioengineering + Mathematical Engineering session. Two calendars exist; lectures are the same, but the scheduling is not necessarily aligned. The previous link points to the Computer Science session, on the same page you find the link to the BIO + MTM session one.
 +
 +
Lectures will be recorded and shared afterward, no streaming of lectures is foreseen.
 +
 +
Recordings of lectures and lab sessions are linked from the google calendar events associated to the corresponding lecture.
 +
<!--
 +
For a google calendar you might [https://boracchi.faculty.polimi.it/teaching/AN2DLCalendar.htm look here!]
 +
{| border="1" align="center" style="text-align:center;"
 +
|-
 +
|Date || Day || Time || Room || Teacher || Topic
 +
|-
 +
|15/09/2021 || Wednesday || 15:15 - 17:00 || T.2.1 (Team1 ) || rowspan="2" | [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=243171f0d60ccb6507e5d097b3c55131 Course Introduction + Deep Learning Intro]
 +
|-
 +
|15/09/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|16/09/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=48eb781c6c832ebe5d1a65aa60aa80f8 From Perceptrons to Feed Forward Neural Networks]
 +
|-
 +
|22/09/2021 || Wednesday || -- || -- || rowspan="2" | -- || rowspan="2" |  -- No Lecture --
 +
|-
 +
|22/09/2021 || Wednesday || -- || --
 +
|-
 +
|23/09/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=d6326382b3d5b22cc9c82cc2094fc7d9 Feed forward neural networks and Backpropagation]
 +
|-
 +
|29/09/2021 || Wednesday || 15:15 - 17:00 || T.2.1 (Team 1)  || rowspan="2" |  [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=7e07badcc1bdb91e76d9c42fb46a44dd KERAS: Numpy, Tensorflow and FNN]
 +
|-
 +
|29/09/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|30/09/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=62460aac13f46ceb85f365ef3af13da9 Error Functions Design]
 +
|-
 +
|06/10/2021 || Wednesday || 15:15 - 17:00 || T.2.1 (Team 1) || rowspan="2" | [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=866cd6c1a612bdf9fb92e8de7630ccaa Overfitting, cross-validation, and Early Stopping]
 +
|-
 +
|06/10/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|07/10/2021 || Thursday || --- || --- || --- || No Lectures (Graduation)
 +
|-
 +
|13/10/2021 || Wednesday || 15:15 - 17:00 || T.2.1 (Team 1)|| rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=3c4494b4865929a682d8b7773deeb06b KERAS: FFNN and Overfitting]
 +
|-
 +
|13/10/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2) 
 +
|-
 +
|14/10/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=bfb3b4478f89834a3fe8908ef38a9a20 Training tricks: activation functions, network initialization, and other stuff...]
 +
|-
 +
|20/10/2021 || Wednesday || 15:15 - 17:00 || T.2.1 (Team 1)|| rowspan="2" | [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || rowspan="2" | [https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3D001aa81af4155e482b36449d5bf740a2&sa=D&source=calendar&usd=2&usg=AOvVaw27hVczu4cbwbSFkbbWSUHm The Image Classification Problem]
 +
|-
 +
|20/10/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|21/10/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3D7ce22e50e0e54055ba5c767ae7139032&sa=D&source=calendar&usd=2&usg=AOvVaw3mgWsYDB-P59366MxvLunC Convolutional Neural Networks]
 +
|-
 +
|27/10/2021 || Wednesday || 15:15 - 17:00 || T.2.1 (Team 1) || rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=b25749461c701879e3814e26b937177d KERAS: Convolutional Neural Networks]
 +
|-
 +
|27/10/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|28/10/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3De2757215c853530566cd3cf46833e75c&sa=D&source=calendar&usd=2&usg=AOvVaw3vsxZOjD-e_iXzVTPT0k88 Training with data scarcity]
 +
|-
 +
|03/11/2021 || Wednesday || 15:15 - 17:00 ||T.2.1 (Team 1)|| rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=495e5d7661d0849fb027a9451ffc4f26 KERAS: Convolutional Neural Networks]
 +
|-
 +
|03/11/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|04/11/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3Dfdfcf0d638238f240ebbd7aed018d318&sa=D&source=calendar&usd=2&usg=AOvVaw0zahxjvCI_hEMxKktO_sQ- Famous CNN architectures]
 +
|-
 +
|10/11/2021 || Wednesday || --- || --- || --- || -- No Lecture (Prove in Itinere) --
 +
|-
 +
|11/11/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3D9bfed46fbc5949d7618be69c2fa50457&sa=D&source=calendar&usd=2&usg=AOvVaw34G9ChAVef06M6iBpN9mXQ Fully Convolutional CNN, CNN for image segmentation (part 1)][https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3Dfa883d62a806513b461540a98830152c&sa=D&source=calendar&usd=2&usg=AOvVaw1A9z3vpzEAkQAO_Bjq0o8p (part 2)]
 +
|-
 +
|17/11/2021 || Wednesday || 15:15 - 17:00 ||T.2.1 (Team 1)|| rowspan="2" | [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || rowspan="2" | [https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3D997f9eb198cd2897a4f7171638a7380e&sa=D&source=calendar&usd=2&usg=AOvVaw1Wyc4KPyTxm40V8-jvU0sd GANs]
 +
|-
 +
|17/11/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|18/11/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://www.google.com/url?q=https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID%3D8879f77fc8c8d6de57149a6716c9ecc5&sa=D&source=calendar&usd=2&usg=AOvVaw2oVvMyZwpZQAnoz5D-cr2Y CNN for localization and detection]
 +
|-
 +
|24/11/2021 || Wednesday || 15:15 - 17:00 ||T.2.1 (Team 1)|| rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=ee50aa7ae9d63a876746701aec8d1947 KERAS: reconstruction and  segmentation]
 +
|-
 +
|24/11/2021 || Wednesday || 17:30 - 19:15 || T.2.1 (Team 2)
 +
|-
 +
|25/11/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=dbc4844003d98778777ca3e06e9cb961 Recurrent neural networks + LSTM]
 +
|-
 +
|01/12/2021 || Wednesday || 15:15 - 17:00 || E.Gatti - ed.20 (Team 1)|| rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=fb8a6ed405eb1812e92783631d1b9de9 KERAS: learning with time series]
 +
|-
 +
|01/12/2021 || Wednesday || 17:30 - 19:15 || E.Gatti - ed.20 (Team 2)
 +
|-
 +
|02/12/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=8f5ee66f75748e65b463160f2a06091a Sequence to sequence learning and Word Embedding]
 +
|-
 +
|08/12/2021 || Wednesday || --- || --- || --- || -- No Lecture (Holiday) --
 +
|-
 +
|09/12/2021 || Thursday || --- || --- || --- || -- No Lecture ---
 +
|-
 +
|15/12/2021 || Wednesday || 15:15 - 17:00 ||B.4.3 (Team 1)|| rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=8c0a4d5119dfc174e6eacbfa96fd3361 KERAS: learning with text]
 +
|-
 +
|15/12/2021 || Wednesday || 17:30 - 19:15 || B4.3 (Team 2)
 +
|-
 +
|16/12/2021 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci]  || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=87be019018382204732ff85b47e64ba8 Attention Mechanism and Transformer]
 +
|-
 +
|}
 +
-->
 +
<!-- 2020 - 2021
 +
{| border="1" align="center" style="text-align:center;"
 +
|-
 +
|Date || Day || Time || Room || Teacher || Topic
 +
|-
 +
|16/09/2020 || Wednesday || 15:15 - 17:00 || 2.0.2 || rowspan="2" | [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/c68da835820141b18032487d716442b8 Course Introduction]
 +
|-
 +
|16/09/2020 || Wednesday || 17:30 - 19:15 || 2.1.2
 +
|-
 +
|17/09/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/5b21d56a672d436ebc3446d2f971658a Introduction to Deep Learning] + [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/8ca39bb673a54e2d976831c99b798633 Perceptron and Hebbian Learning]
 +
|-
 +
|23/09/2020 || Wednesday || 15:15 - 17:00 || 2.0.2 || rowspan="2" | [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/3ecfb0a83eab498e88c7a6c30fef633e Hebbian Learning Example and the XOR Problem]
 +
|-
 +
|23/09/2020 || Wednesday || 17:30 - 19:15 || 2.1.2 
 +
|-
 +
|24/09/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/b276ff5f0ec24e2dad309705ab2700eb Feed forward neural networks and Backpropagation]
 +
|-
 +
|30/09/2020 || Wednesday || 15:15 - 17:00 || 2.0.2  || rowspan="2" |  [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/a2b082018a9c44ca95d4b5d4134b0a65 Backpropagation Example]
 +
|-
 +
|30/09/2020 || Wednesday || 17:30 - 19:15 || 2.1.2
 +
|-
 +
|01/10/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/cbc25ba1f9d84650b0b8d5c902281707 Error Functions Design]
 +
|-
 +
|07/10/2020 || Wednesday || 15:15 - 17:00 || Virtual Room || rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/37c8e22daab748f4925f40ec2714ae05 KERAS NN - Feed forward neural networks (Part 1)][https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/d1f2c43ed1ec4ad887064f53e7c8ec45 (Part 2)]
 +
|-
 +
|07/10/2020 || Wednesday || 17:30 - 19:15 || Virtual Room
 +
|-
 +
|08/10/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/d3c37248170341b4b128a8c795b5a288 Overffitting, cross-validation, and Early Stopping]
 +
|-
 +
|14/10/2020 || Wednesday || 15:15 - 17:00 || Virtual Room|| rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/c4634b880d2c483d90cfee40da2b385b KERAS NN - FFNN and Overfitting]
 +
|-
 +
|14/10/2020 || Wednesday || 17:30 - 19:15 || Virtual Room 
 +
|-
 +
|15/10/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/353566007b4b46d998dd4da3eb9266cc Facing overfitting, network initialization, and other stuff ...]
 +
|-
 +
|21/10/2020 || Wednesday || 15:15 - 17:00 || 2.0.2 || rowspan="2" | [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/f81e84afaa734e07b1de3404b0ce1ab3 The Image Classification Problem]
 +
|-
 +
|21/10/2020 || Wednesday || 17:30 - 19:15 || 2.1.2
 +
|-
 +
|22/10/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/60b0c685bb0544be93232406b93dc216 Convolutional Neural Networks]
 +
|-
 +
|28/10/2020 || Wednesday || 15:15 - 17:00 || 2.0.2 || rowspan="2" | [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/bf739617705646bf992949a52b4e7220 CNN Training, Transfer Learning and Visualization. Fully Convolutional CNN (Part1).]
 +
|-
 +
|28/10/2020 || Wednesday || 17:30 - 19:15 || 2.1.2
 +
|-
 +
|29/10/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/47ddba848cac4df287ef38dd5c620a47 CNN Training, Transfer Learning and Visualization. Fully Convolutional CNN (Part2).]
 +
|-
 +
|04/11/2020 || Wednesday || 15:15 - 17:00 || 2.0.2 || rowspan="2" | [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || rowspan="2" | [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/ae5d7492ccd84168820295a96f86e797 KERAS NN - Convolutional Neural Networks]
 +
|-
 +
|04/11/2020 || Wednesday || 17:30 - 19:15 || 2.1.2
 +
|-
 +
|05/11/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/playback/8a475196b91a4fb5b2ddaac94fb763dd Fully Convolutional CNN, CNN for image segmentation]
 +
|-
 +
|11/11/2020 || Wednesday || --- || --- || --- || -- No Lecture (Prove in Itinere) --
 +
|-
 +
|12/11/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=ba034f944508406bb2f8d0ed0dc2c1be CNN for localization and detection]
 +
|-
 +
|18/11/2020 || Wednesday || 16:15 - 18:15 ||Virtual Room || [https://politecnicomilano.webex.com/join/giacomo.boracchi Giacomo Boracchi] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=5855e086526f4a38b15a482bbc06aef1 GANs]
 +
|-
 +
|19/11/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/webappng/sites/politecnicomilano/recording/2ebf7788731540f1866a9554db5ef379 Recurrent neural networks + LSTM]
 +
|-
 +
|25/11/2020 || Wednesday || 16:15 - 18:15 || Virtual Room || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari] || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=5c9d857b4d1aa39d8dda3c50a797fb6d KERAS NN - Autoencoder, classification, segmentation]
 +
|-
 +
|26/11/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/webappng/sites/politecnicomilano/recording/b678aafa7fcf486e98ab293e1fc704fc Se2Seq Learning and Word Embedding]
 +
|-
 +
|02/12/2020 || Wednesday || 16:15 - 18:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/webappng/sites/politecnicomilano/recording/1931bcb8dc344c6dadcdfe1e5e6dc8a9 Attention Mechanisms]
 +
|-
 +
|03/12/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/webappng/sites/politecnicomilano/recording/8281dad4cb2a4bd8b974efc7de02066d The Transformer (and challenge presentation in the first 20 minutes)]
 +
|-
 +
|09/12/2020 || Wednesday || 16:15 - 18:15 || Virtual Room || [https://politecnicomilano.webex.com/join/francesco.lattari Francesco Lattari]  || [https://politecnicomilano.webex.com/politecnicomilano/ldr.php?RCID=a48dc7e484038a379edd4ba0e7df7ad3 KERAS NN - Recurrent Neural Networks]
 +
|-
 +
|10/12/2020 || Thursday || 16:30 - 19:15 || Virtual Room || [https://politecnicomilano.webex.com/join/matteo.matteucci Matteo Matteucci] || [https://politecnicomilano.webex.com/webappng/sites/politecnicomilano/recording/620a7db68f5b49f7956cc52141b7281b Questions and Answers]
 +
|-
 +
|16/12/2020 || Wednesday || 16:15 - 18:15 || Virtual Room || --- || -- Spare Lecture --
 +
|-
 +
|17/12/2020 || Thursday || 16:30 - 19:15 || Virtual Room || --- || -- Spare Lecture --
 +
|-
 +
|23/12/2020 || Wednesday || 16:15 - 18:15 || Virtual Room || --- ||  -- Spare Lecture --
 +
|-
 +
|}
 +
-->
 +
<!-- [2019/2020]
 
{| border="1" align="center" style="text-align:center;"
 
{| border="1" align="center" style="text-align:center;"
 
|-
 
|-
Line 111: Line 361:
 
|13/12/2019 || Friday  || 14:15 - 17:15 || 26.16 || Practicals || Francesco Lattari || Keras examples ...
 
|13/12/2019 || Friday  || 14:15 - 17:15 || 26.16 || Practicals || Francesco Lattari || Keras examples ...
 
|-
 
|-
|19/12/2019 || Thursday || 16:15 - 18:15 || 2.0.1 || Lecture || Matteo Matteucci ||  Competitiong highlights
+
|19/12/2019 || Thursday || 16:15 - 19:15 || 2.0.1 || Lecture || Matteo Matteucci ||  Competitiong highlights
 
|-
 
|-
 
|20/12/2019 || Friday  || 14:15 - 17:15 || -- || -- || -- || -- No Lecture Today --
 
|20/12/2019 || Friday  || 14:15 - 17:15 || -- || -- || -- || -- No Lecture Today --
 
|-
 
|-
|20/12/2019 || Friday  || 14:15 - 17:15 || Seminar Room || Seminar || Luigi Malago' || [[Meta Learning Seminar]]
+
|10/01/2020 || Friday  || 14:30 - 15:30 || DEIB Seminar Room || Seminar || Luigi Malago' || [[Meta Learning Seminar]]
 
|-
 
|-
 
|}
 
|}
 +
-->
  
 
===Course Evaluation===
 
===Course Evaluation===
Line 123: Line 374:
 
Course evaluation is composed of two parts:
 
Course evaluation is composed of two parts:
  
* A written examination covering the whole program graded up to 27/32
+
* A written examination covering the whole program graded up to 20/30
* A home project in the form of a Kaggle style competition practicing the topics of the course graded up to 5/32
+
* 2 home projects in the form of a "Kaggle style" challenge practicing the topics of the course graded up to 5/30 each
  
The final score will sum the grade of the written exam and the grade of the home project.
+
The final score will sum up the grade of the written exam and the grade of the home projects. Home projects are not compulsory and they are issued only once a year.
 +
<!--
 +
You can find here one [[Media:AN2DL_ExamExample.pdf|example of the exam text]] to get a flavor of what to expect in the written examination.
 +
-->
  
 
==Teaching Material (the textbook)==  
 
==Teaching Material (the textbook)==  
  
Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference you can check the following text, but keep in mind that teachers will not follow it strictly
+
Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference, you can check the following textbook, but keep in mind that teachers will not follow it strictly
  
 
* [http://www.deeplearningbook.org/ Deep Learning]. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
 
* [http://www.deeplearningbook.org/ Deep Learning]. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
 +
<!-- The remaining material about the course is available through [http://webeep.polimi.it WeBeep].-->
 +
 +
Regarding the Python programming language, we will provide you with the basics about NumPy and python scripting in case you want some introductory material you can check here
 +
 +
* [https://docs.python.org/3/tutorial/ Python tutorials]: these are the official python tutorials, we suggest 3.An Informal Introduction to Python (Numbers, Strings, Lists), 4.More Control Flow Tools (if, for, range, functions), 5.Data Structures (More on lists, Dictionaries), 9.Classes.
  
 
===Course Slides===
 
===Course Slides===
  
 
Slides from the lectures by Matteo Matteucci
 
Slides from the lectures by Matteo Matteucci
*[[Media:AN2DL_00_2019_Course_Introduction.pdf|[2019/2020] Course Introduction]]: introductory slides of the course with useful information about the course syllabus, grading, and the course logistics.  
+
*[[Media:AN2DL_00_2324_Course_Introduction.pdf|[2023/2024] Course Introduction]]: introductory slides of the course with useful information about the course syllabus, grading, and the course logistics.  
*[[Media:AN2DL_01_2019_Deep_Learning_Intro.pdf|[2019/2020] Machine Learning vs Deep Learning]]: introduction to machine learning paradigms and definition of deep learning with examples
+
*[[Media:AN2DL_01_2324_Deep_Learning_Intro.pdf|[2023/2024] Machine Learning vs Deep Learning]]: introduction to machine learning paradigms and definition of deep learning with examples.
*[[Media:AN2DL_02_2019_Prceptron_2_FeedForward.pdf|[2019/2020] From Perceptrons to Feed Forward Neural Networks]]: the original Perceptron model, Hebbian learning, feed-forward architecture, backpropagation and gradient descent, error functions and maximum likelihood estimation   
+
*[[Media:AN2DL_02_2324_Perceptron_2_FeedForward.pdf|[2023/2024] From Perceptrons to Feed Forward Neural Networks]]: the original Perceptron model, Hebbian learning, feed-forward architecture, backpropagation and gradient descent, error functions and maximum likelihood estimation.  
*[[Media:AN2DL_03_2019_NeuralNetwroksTraining.pdf|[2019/2020] Neural Networks Training]]: dealing with overfitting (weight decay, early stopping, dropout), vanishing gradient (ReLU and friends), batch normalization  
+
*[[Media:AN2DL_03_2324_NeuralNetwroksTraining.pdf|[2023/2024] Neural Networks Training]]: dealing with overfitting (weight decay, early stopping, dropout), vanishing gradient (ReLU and friends), batch normalization.
*[[Media:AN2DL_04_2019_RecurrentNeuralNetworks.pdf|[2019/2020] Recurrent Neural Networks]]: learning with sequences, Recurrent Neural Networks, vanishing gradient, Long Short-Term Memories (LSTM).   
+
*[[Media:AN2DL_04_2324_RecurrentNeuralNetworks.pdf|[2023/2024] Recurrent Neural Networks]]: learning with sequences, Recurrent Neural Networks, vanishing gradient, Long Short-Term Memories (LSTM), seq2seq model introduction.   
*[[Media:AN2DL_05_2019_Sq2SeqLearning.pdf|[2019/2020] Sequence 2 Sequence Learning]]: sequence to sequence learning as an encoder-decoder problem, Neural Turing Machines, attention mechanisms, the Transformer.
+
*[[Media:AN2DL_05_2324_Seq2SeqAndWordEmbedding.pdf|[2023/2024] Seq2Seq and Word Embedding]]: sequence 2 sequence learning, neural autoencoders, language models, word embedding, word2vec.
*[[Media:AN2DL_06_2019_WordEmbedding.pdf|[2019/2020] Word Embedding]]: neural autoencoders, language models, word embedding, word2vec, glove.
+
*[[Media:AN2DL_06_2324_AttentionAndTrasformers.pdf|[2023/2024] Beyond Sequence 2 Sequence Learning]]: Attention mechanisms in seq2seq models and Transformers.
  
Slides from the lectures by Giacomo Boracchi are available in [http://home.deib.polimi.it/boracchi/teaching/AN2DL.htm his webpage], for you  
+
Slides from the lectures by Giacomo Boracchi are available in [https://boracchi.faculty.polimi.it/teaching/AN2DL.htm his webpage]; for your convenience, I am giving pointers to the slide here for you (in case you note discrepancies please notify me)
* [http://home.deib.polimi.it/boracchi/teaching/AN2DL/2019_AN2DL_Lez1_ImageClassification.pdf Image Classification]: Image classification and related issues, template matching, image classification via nearest neighbors methods, image classification via linear classifiers, image classification via hand-crafted features.  
+
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez1_ImageClassification.pdf [2022/2023]] The Image Classification Problem
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez2_CNN.pdf [2022/2023]] Convolutional Neural Networks
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez3_CNN_TL_Data_Scarcity.pdf [2022/2023]] CNN Parameters and Training with Data Scarcity
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez4_CNN_Famous_Architectures_CNN_Visualization.pdf [2022/2023] Famous CNN architectures and CNN Visualization
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez5_CNN_for_segmentation.pdf [2022/2023]] Fully Convolutional CNN and CNN for Semantic Segmentation
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez6_Localization_Explanations.pdf [2022/2023]] CNN for Localization and CNN Explanations
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez7_CNN_for_object_detection_Metric_Learning.pdf [20223/2023]] Object Detection Networks and Metric Learning
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez8_Generative_Models.pdf [2022/2023]] Autoencoders and Generative Adversarial Networks
  
Slides from the practicals by Francesco Lattari
+
Slides from the practicals by Francesco Lattari, Eugenio Lomurno, and Loris Giulivi will be published here after each lab session: [CHECK THIS FOLDER!]
 +
<!--https://drive.google.com/drive/folders/1iKWNyTBk1KMpnnkkjLwccgOFm6EYzsry?usp=sharing-->
 +
<!--* [https://drive.google.com/drive/folders/1zO_E89H3mjc7nysBkGUt_MmCn1Mb2DzP?usp=sharing Python and convolutions]: a Colab notebook with a crash course on Python 3 to prepare for the practicals and extra material about convolutions!
 +
* [https://drive.google.com/drive/folders/1-WFu2S-MBOrNsFRy1IWEptF_hTMLmic9?usp=sharing Feed Forward Neural Networks]: NumPy Arrays, TensorFlow2 Tensors, and Feed Forward Neural Network
 +
* [https://drive.google.com/drive/folders/1-gaItYI8i01STVPrUcqZ8vxK27yGFENn?usp=sharing Dealing with Overfitting]: Hold Out, Early Stopping and Cross-Validation
 +
* [https://drive.google.com/drive/folders/10H60DRdcFCbwCKHExZdPfLdrtbnmCaRE?usp=sharing Convolutional Neural Networks]: 2D Convolutions and their meaning, Convolutional Neural Networks with Keras
 +
* [https://drive.google.com/drive/folders/10D876SqauR27hB5BUIcckKf3v4CrA_x4?usp=sharing Augmentation and Fine Tuning]: Data Augmentation, Transfer Learning, and Fine Tuning
 +
* [https://drive.google.com/drive/folders/1dxfaXo68dWnrS6xHY0yLvUDjTwDPFN6h?usp=sharing Autoencoders and Segmentation]: Exercises on Autoencoders and Multiclass Segmentation
 +
* [https://drive.google.com/drive/folders/1rZQUrs6Rb1dGg5-KLg-y9tT1yw09gr7G?usp=sharing Dealing with Time Series]: Recurrent Neural Networks, Long Short-Term Memories, and other models.
 +
-->
 +
 
 +
=== External Sources ===
 +
* [https://towardsdatascience.com/back-to-basics-deriving-back-propagation-on-simple-rnn-lstm-feat-aidan-gomez-c7f286ba973d Deriving Back Propagation on simple RNN/LSTM]: a tutorial step by step to derive the backpropagation formulas
 +
* [https://karpathy.github.io/2019/04/25/recipe/ A Recipe for Training Neural Networks]: a neat post on Andrej Karpathy's blog about do's and don'ts in training neural networks
 +
<!--*[[Media:AN2DL_Lab1_2020_KerasIntroduction.zip|[2020/2021] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 +
*[[Media:AN2DL_Lab1_2020_KerasOverfitting.zip|[2020/2021] Facing overfitting in Keras]]: Techniques to limit overfitting in Keras and Tensorboard use (slides + notebook)
 +
*[[Media:AN2DL_Lab1_2020_KerasCNN.zip|[2020/2021] Convolutional architectures in Keras]]: How to build, train, and evaluate convolutional models for image classification in Keras and Tensorflow2 (slides + notebook)
 +
*[[Media:AN2DL_Lab1_2020_KerasSegmentation.zip|[2020/2021] Image Segmentation in Keras]]: How to build, train, and evaluate convolutional models for image segmentation in Keras and Tensorflow2 (slides + notebook)
 +
*[[Media:AN2DL_Lab1_2020_KerasRecurrentNeuralNetworks.zip|[2020/2021] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)-->
 +
<!--
 
*[[Media:AN2DL_Lab1_2019_KerasIntroduction.zip|[2019/2020] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab1_2019_KerasIntroduction.zip|[2019/2020] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab2_2019_KerasCNN.zip|[2019/2020] Convolutional architectures in Keras]]: How to build, train, and evaluate convolutional models for classification and segmentation in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab2_2019_KerasCNN.zip|[2019/2020] Convolutional architectures in Keras]]: How to build, train, and evaluate convolutional models for classification and segmentation in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab3_2019_KerasRNN.zip|[2019/2020] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab3_2019_KerasRNN.zip|[2019/2020] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
 +
-->
 +
 +
===Exams===
 +
 +
Politecnico di Milano exams will be held digitally in presence via the [https://remoteexam.polimi.it/ Politecnico di Milano remote exam platform] using the Safe Exams Browser, it works only on Windows and iOS so please be prepared and test it in advance. You can have a flavor of what to expect by looking at some past exam calls.
 +
 +
*[[Media:AN2DL_20200619.pdf| 19/06/2020 Online Exam]]
 +
*[[Media:AN2DL_20200715.pdf| 15/07/2020 Online Exam]]
 +
*[[Media:AN2DL_20200903.pdf| 03/09/2020 Online Exam]]
 +
 +
'''Note''': written exams will be graded 20 points plus 10 points are given by 2 software challenges issues only during the semester. Laude will be given to students who, beside getting the highest grade, will show participation in class, will perform particularly well in the challenges (this includes the quality of the report), will submit ahead of time the written exams.
 +
 +
<!--
 +
===Lab software setup===
 +
 +
For the lab in class we suggest you install TensorFlow 2 on you machine so to be able to follow the coding examples step by step. Here what you should do:
 +
 +
* Install Anaconda according to your distro (Windows/Linux), python 3.7 from [https://www.anaconda.com/distribution]
 +
* From terminal (Anaconda Prompt in Windows):
 +
** conda create -n tf_env python=3.7 tensorflow-gpu
 +
** conda activate tf_env
 +
** pip install --upgrade pip
 +
** pip install jupyter
 +
** pip install pillow
 +
* Test your Tensorflow install
 +
** Run python from terminal (Anaconda Prompt in Windows) with «python»
 +
** import tensorflow
 +
** print(tensorflow.__version__) -> your should get version 2.1.0 or higher
 +
* Test your Jupiter install
 +
** From terminal (Anaconda Prompt in Windows) use the command jupyter notebook -> a Jupiter tab should appear in your browser
 +
** On top right click on «New» and select «Python 3» from the menu -> a Jupiter Notebook should appear in a new tab
 +
** Write code 3b and 3c in cell "In [ ]:" and execute clicking on «Run».
  
 
===Kaggle Homeworks===
 
===Kaggle Homeworks===
  
As part of the evaluation (up to 5 marks in the final grade) we are issuing 3 homeworks in the form of Kaggle competitions. They are meant to practice the course topics on simple image recognition tasks.
+
As part of the evaluation (up to 6 marks in the final grade) we are issuing 3 homeworks in the form of "Kaggle style" competitions. They are meant to practice the course topics on simple image recognition tasks.
 +
 
 +
====[2020/2021]====
 +
 
 +
* [https://www.kaggle.com/t/d475328a2ba74d5a9a49788d5c308d69 Image Classification Homework]: the first homework is about image classification with convolutional neural networks. The deadline to submit the results is November the 22nd.
 +
* [https://competitions.codalab.org/competitions/27176 Image Segmentation Homework]: the second homework is about image segmentation. This competition is open also to external participants so it lasts until January, you are not requested to participate to the two Stages, just at the Development one and your delivery is expected by December 20th. There are different datasets and leaderboards you can decide to participate to any of the leaderboard and to use any approach you prefer for segmentation.
 +
* [https://www.kaggle.com/t/cb49614dda9d4c7cac65be12451ba3cd Visual Question answering Homework]: the third homework is about visual question answering and it mixes feature extraction from images and text in order to solve an multi domain classification tas. This time we do not want to put pressure on you and the deadline is quite relaxed, you have until the 31st of January to submit your solution.
 +
 
 +
* [https://competitions.codalab.org/competitions/27176 '''Extra point ->''' Image Segmentation Homework]: from 19th January 2021 till 22nd January 2021 there will be the second phase of the image segmentation challenge where new data will be released to train you models and participate to the final leaderboard. One extra point for you if you fine tune your models and participate into this!!!
 +
 
 +
====[2019/2020]====
 
*[https://www.kaggle.com/t/5e52f27cd4fb485185e6b627c1fb0335 Image Classification Homework]: the first homework is about image classification with convolutional neural networks. The deadline to submit the results is November the 30th.
 
*[https://www.kaggle.com/t/5e52f27cd4fb485185e6b627c1fb0335 Image Classification Homework]: the first homework is about image classification with convolutional neural networks. The deadline to submit the results is November the 30th.
 
*[https://www.kaggle.com/t/ef3b29a4a5d74b2fbb3e3ec50ca4e206 Image Segmentation Homework]: the second homework is about image segmentation with convolutional neural networks and the like. The deadline to submit the results is December the 17th.
 
*[https://www.kaggle.com/t/ef3b29a4a5d74b2fbb3e3ec50ca4e206 Image Segmentation Homework]: the second homework is about image segmentation with convolutional neural networks and the like. The deadline to submit the results is December the 17th.
 
*[https://www.kaggle.com/t/6d6fd208f06f4d1abd71d47bd36586ce Visual Question Answering Homework]: the third homework is about visual question answering with convolutional and recurrent neural networks ... plus word2vec. The deadline to submit the results is January the 15th.
 
*[https://www.kaggle.com/t/6d6fd208f06f4d1abd71d47bd36586ce Visual Question Answering Homework]: the third homework is about visual question answering with convolutional and recurrent neural networks ... plus word2vec. The deadline to submit the results is January the 15th.
 +
-->

Latest revision as of 23:20, 10 December 2023


The following are last-minute news you should be aware of ;-)

* 13/09/2023: A new edition of the AN2DL course starts today!!!


Course Aim & Organization

Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.

Nowadays, deep neural networks can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).

The course's major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.

Teachers

The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.

Course Program and Syllabus

This goal is pursued in the course by:

  • Presenting major theoretical results underpinning NN (e.g., universal approx, vanishing/exploding gradient, etc.)
  • Describing the most important algorithms for NN training (e.g., backpropagation, adaptive gradient algorithms, etc.)
  • Illustrating the best practices on how to successfully train and use these models (e.g., dropout, data augmentation, etc.)
  • Providing an overview of the most successful Deep Learning architectures (e.g., CNNs, sparse and dense autoencoder, LSTMs for sequence to sequence learning, etc.)
  • Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.

We have compiled a detailed syllabus of the course that students can use to double-check their preparation before the exam.

Detailed course schedule and recordings

A detailed schedule of the course is given in the form of google calendar; topics are just indicative while days and teachers are correct up to some last-minute change (I will notify you by email). Please note that not all days we have lectures!!

For the AN2DL Course Google Calendar look here!

Note: the course is given in parallel to two sessions, the Computer Science session, and the Bioengineering + Mathematical Engineering session. Two calendars exist; lectures are the same, but the scheduling is not necessarily aligned. The previous link points to the Computer Science session, on the same page you find the link to the BIO + MTM session one.

Lectures will be recorded and shared afterward, no streaming of lectures is foreseen.

Recordings of lectures and lab sessions are linked from the google calendar events associated to the corresponding lecture.

Course Evaluation

Course evaluation is composed of two parts:

  • A written examination covering the whole program graded up to 20/30
  • 2 home projects in the form of a "Kaggle style" challenge practicing the topics of the course graded up to 5/30 each

The final score will sum up the grade of the written exam and the grade of the home projects. Home projects are not compulsory and they are issued only once a year.

Teaching Material (the textbook)

Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference, you can check the following textbook, but keep in mind that teachers will not follow it strictly

  • Deep Learning. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.

Regarding the Python programming language, we will provide you with the basics about NumPy and python scripting in case you want some introductory material you can check here

  • Python tutorials: these are the official python tutorials, we suggest 3.An Informal Introduction to Python (Numbers, Strings, Lists), 4.More Control Flow Tools (if, for, range, functions), 5.Data Structures (More on lists, Dictionaries), 9.Classes.

Course Slides

Slides from the lectures by Matteo Matteucci

Slides from the lectures by Giacomo Boracchi are available in his webpage; for your convenience, I am giving pointers to the slide here for you (in case you note discrepancies please notify me)

  • [2022/2023] The Image Classification Problem
  • [2022/2023] Convolutional Neural Networks
  • [2022/2023] CNN Parameters and Training with Data Scarcity
  • [2022/2023 Famous CNN architectures and CNN Visualization
  • [2022/2023] Fully Convolutional CNN and CNN for Semantic Segmentation
  • [2022/2023] CNN for Localization and CNN Explanations
  • [20223/2023] Object Detection Networks and Metric Learning
  • [2022/2023] Autoencoders and Generative Adversarial Networks

Slides from the practicals by Francesco Lattari, Eugenio Lomurno, and Loris Giulivi will be published here after each lab session: [CHECK THIS FOLDER!]

External Sources

Exams

Politecnico di Milano exams will be held digitally in presence via the Politecnico di Milano remote exam platform using the Safe Exams Browser, it works only on Windows and iOS so please be prepared and test it in advance. You can have a flavor of what to expect by looking at some past exam calls.

Note: written exams will be graded 20 points plus 10 points are given by 2 software challenges issues only during the semester. Laude will be given to students who, beside getting the highest grade, will show participation in class, will perform particularly well in the challenges (this includes the quality of the report), will submit ahead of time the written exams.