Difference between revisions of "Artificial Neural Networks and Deep Learning"

From Chrome
Jump to: navigation, search
(Course Slides)
(42 intermediate revisions by the same user not shown)
Line 2: Line 2:
  
 
The following are last-minute news you should be aware of ;-)
 
The following are last-minute news you should be aware of ;-)
 +
* 12/03/2023: Grades of the [[Media:AN2DL_20230217.pdf|AN2DL 17/02/2023 call]] will appear here very soon!!!
 +
* 15/02/2023: Updates of grades from the [[Media:AN2DL_20230126_bis.pdf|AN2DL 26/01/2023 call]].
 +
* 14/02/2023: Grades of the [[Media:AN2DL_20230126.pdf|AN2DL 26/01/2023 call]].
 +
* 12/02/2023: Grades of the [[Media:AN2DL_2022_2023_Challenges.pdf|AN2DL Challenges]].
 +
* 23/01/2023: added link to the detailed course syllabus
 +
* 08/01/2023: update of the teaching material in the page ... a new detailed syllabus will follow too
 +
* 21/12/2022: Grades of the [[Media:AN2DL_2022_2023_FirstChallenge.pdf|first challenge (v2)]].
 +
 +
<!--
 +
07/08/2022: Grades for the [[Media:AN2DL_Grades_20220726.pdf|06/07/2022 call are available here!]]
 +
06/03/2021: Grades for the [[Media:AN2DL2122Homeworks.pdf|2021/2021 homeworks are available here!]]
 
  24/11/2021: Lecture on 09/12/2021 moved to 16/12/2021
 
  24/11/2021: Lecture on 09/12/2021 moved to 16/12/2021
 
  24/11/2021: Change of classroom on the 15/12 from T2.1 to B4.3
 
  24/11/2021: Change of classroom on the 15/12 from T2.1 to B4.3
Line 16: Line 27:
 
  14/09/2021: Website under maintenance ... come back later
 
  14/09/2021: Website under maintenance ... come back later
  
<!--
 
 
  07/08/2021: Result from [[Media:AN2DL_Grades_20210728.pdf|28/07/2021 call with all homeworks]] are available here
 
  07/08/2021: Result from [[Media:AN2DL_Grades_20210728.pdf|28/07/2021 call with all homeworks]] are available here
 
  25/07/2021: Result from [[Media:AN2DL_Grades_20210708.pdf|08/07/2021 call with all homeworks]] are available here
 
  25/07/2021: Result from [[Media:AN2DL_Grades_20210708.pdf|08/07/2021 call with all homeworks]] are available here
Line 85: Line 95:
 
Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.
 
Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.
  
Nowadays, deep neural network can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).  
+
Nowadays, deep neural networks can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).  
  
The course major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.
+
The course's major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.
  
 
===Teachers===
 
===Teachers===
Line 93: Line 103:
 
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.
 
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.
  
* [https://www.deib.polimi.it/ita/personale/dettagli/267262 Matteo Matteucci]: the course teacher and this is his [https://politecnicomilano.webex.com/join/matteo.matteucci webex room]
+
* [https://www.deib.polimi.it/ita/personale/dettagli/267262 Matteo Matteucci] and [https://www.deib.polimi.it/ita/personale/dettagli/549640 Giacomo Boracchi]: the course teachers
* [https://www.deib.polimi.it/ita/personale/dettagli/549640 Giacomo Boracchi]: the course co-teacher and this is his [https://politecnicomilano.webex.com/join/giacomo.boracchi webex room]
+
* [https://www.deib.polimi.it/ita/personale/dettagli/846174 Francesco Lattari] and [https://www.deib.polimi.it/ita/personale/dettagli/920490 Eugenio Lomurno]: the course teaching assistants
* [https://www.deib.polimi.it/ita/personale/dettagli/846174 Francesco Lattari]: the course teaching assistant and this is his [https://politecnicomilano.webex.com/join/francesco.lattari webex room]
+
* [https://www.deib.polimi.it/ita/personale/dettagli/920490 Eugenio Lomurno]: the course teaching assistant and this is his [https://politecnicomilano.webex.com/join/eugenio.lomurno webex room]
+
  
 
===Course Program and Syllabus===
 
===Course Program and Syllabus===
Line 107: Line 115:
 
* Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.
 
* Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.
  
We have compiled a detailed syllabus of the course student can use to double check their preparation against before the exam.  
+
We have compiled a detailed syllabus of the course that students can use to double-check their preparation before the exam.
* [[Media:AN2DL_Syllabus_2021.pdf|[2020/2021] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam
+
* [https://docs.google.com/document/d/1BbcCJ7Om5LRvPCGeYkUkCk1RMjoi_ULDakkg0epsVFA/edit?usp=sharing Detailed Course Syllabus]
 +
<!--* [[Media:AN2DL_Syllabus_2022_2023.pdf|[2022/2023] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam-->
 +
<!--* [[Media:AN2DL_Syllabus_2021.pdf|[2020/2021] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam-->
 
<!--* [[Media:AN2DL_Syllabus.pdf|[2019/2020] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam-->
 
<!--* [[Media:AN2DL_Syllabus.pdf|[2019/2020] Course Syllabus]]: a detailed list of topics covered by the course and which students are expected to know when approaching the exam-->
  
===Detailed course schedule===
+
===Detailed course schedule and recordings===
  
A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all days we have lectures!!
+
A detailed schedule of the course is given in the form of google calendar; topics are just indicative while days and teachers are correct up to some last-minute change (I will notify you by email). Please note that not all days we have lectures!!
  
Note: Lecture timetable interpretation
+
For the [https://boracchi.faculty.polimi.it/teaching/AN2DLCalendar_CS.htm AN2DL Course Google Calendar] look here!
* On Wednesday, in T.2.1, Team 1, starts at 15:15, ends at 17:00
+
* On Wednesday, in T.2.1, Team 2, starts at 17:30, ends at 19:15
+
* On Thursday, in teacher webex room, starts at 16:30, ends at 19:15
+
  
Note: Teams division is based on your Codice Persona (and should minimize overlap)
+
Note: the course is given in parallel to two sessions, the Computer Science session, and the Bioengineering + Mathematical Engineering session. Two calendars exist; lectures are the same, but the scheduling is not necessarily aligned. The previous link points to the Computer Science session, on the same page you find the link to the BIO + MTM session one.
* Team 1: odd Codice Persona
+
* Team 2: even Codice Persona
+
  
For a google calendar you might [https://boracchi.faculty.polimi.it/teaching/AN2DLCalendar.htm look here!]
+
Lectures will be recorded and shared afterward, no streaming of lectures is foreseen.
  
 +
Recordings of lectures and lab sessions are linked from the google calendar events associated to the corresponding lecture.
 +
<!--
 +
For a google calendar you might [https://boracchi.faculty.polimi.it/teaching/AN2DLCalendar.htm look here!]
 
{| border="1" align="center" style="text-align:center;"
 
{| border="1" align="center" style="text-align:center;"
 
|-
 
|-
Line 211: Line 219:
 
|-
 
|-
 
|}
 
|}
 
+
-->
 
<!-- 2020 - 2021
 
<!-- 2020 - 2021
 
{| border="1" align="center" style="text-align:center;"
 
{| border="1" align="center" style="text-align:center;"
Line 293: Line 301:
 
|}
 
|}
 
-->
 
-->
 
 
<!-- [2019/2020]
 
<!-- [2019/2020]
 
{| border="1" align="center" style="text-align:center;"
 
{| border="1" align="center" style="text-align:center;"
Line 367: Line 374:
 
* 2 home projects in the form of a "Kaggle style" challenge practicing the topics of the course graded up to 5/30 each
 
* 2 home projects in the form of a "Kaggle style" challenge practicing the topics of the course graded up to 5/30 each
  
The final score will sum the grade of the written exam and the grade of the home projects. Home projects are not compulsory and they are issued only once a year.
+
The final score will sum up the grade of the written exam and the grade of the home projects. Home projects are not compulsory and they are issued only once a year.
 
<!--
 
<!--
 
You can find here one [[Media:AN2DL_ExamExample.pdf|example of the exam text]] to get a flavor of what to expect in the written examination.
 
You can find here one [[Media:AN2DL_ExamExample.pdf|example of the exam text]] to get a flavor of what to expect in the written examination.
Line 374: Line 381:
 
==Teaching Material (the textbook)==  
 
==Teaching Material (the textbook)==  
  
Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference you can check the following text, but keep in mind that teachers will not follow it strictly
+
Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference, you can check the following textbook, but keep in mind that teachers will not follow it strictly
  
 
* [http://www.deeplearningbook.org/ Deep Learning]. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
 
* [http://www.deeplearningbook.org/ Deep Learning]. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
 
<!-- The remaining material about the course is available through [http://webeep.polimi.it WeBeep].-->
 
<!-- The remaining material about the course is available through [http://webeep.polimi.it WeBeep].-->
  
Regarding the python programming language, we will provide you the basics about numpy and python scripting in case you want some introductory material you can check here
+
Regarding the Python programming language, we will provide you with the basics about NumPy and python scripting in case you want some introductory material you can check here
  
 
* [https://docs.python.org/3/tutorial/ Python tutorials]: these are the official python tutorials, we suggest 3.An Informal Introduction to Python (Numbers, Strings, Lists), 4.More Control Flow Tools (if, for, range, functions), 5.Data Structures (More on lists, Dictionaries), 9.Classes.
 
* [https://docs.python.org/3/tutorial/ Python tutorials]: these are the official python tutorials, we suggest 3.An Informal Introduction to Python (Numbers, Strings, Lists), 4.More Control Flow Tools (if, for, range, functions), 5.Data Structures (More on lists, Dictionaries), 9.Classes.
Line 386: Line 393:
  
 
Slides from the lectures by Matteo Matteucci
 
Slides from the lectures by Matteo Matteucci
*[[Media:AN2DL_00_2121_Course_Introduction.pdf|[2021/2022] Course Introduction]]: introductory slides of the course with useful information about the course syllabus, grading, and the course logistics.  
+
*[[Media:AN2DL_00_2223_Course_Introduction.pdf|[2022/2023] Course Introduction]]: introductory slides of the course with useful information about the course syllabus, grading, and the course logistics.  
*[[Media:AN2DL_01_2122_Deep_Learning_Intro.pdf|[2021/2022] Machine Learning vs Deep Learning]]: introduction to machine learning paradigms and definition of deep learning with examples
+
*[[Media:AN2DL_01_2223_Deep_Learning_Intro.pdf|[2022/2023] Machine Learning vs Deep Learning]]: introduction to machine learning paradigms and definition of deep learning with examples
*[[Media:AN2DL_02_2122_Perceptron_2_FeedForward.pdf|[2021/2022] From Perceptrons to Feed Forward Neural Networks]]: the original Perceptron model, Hebbian learning, feed-forward architecture, backpropagation and gradient descent, error functions and maximum likelihood estimation   
+
*[[Media:AN2DL_02_2223_Perceptron_2_FeedForward.pdf|[2022/2023] From Perceptrons to Feed Forward Neural Networks]]: the original Perceptron model, Hebbian learning, feed-forward architecture, backpropagation and gradient descent, error functions and maximum likelihood estimation   
*[[Media:AN2DL_03_2122_NeuralNetwroksTraining.pdf|[2021/2022] Neural Networks Training]]: dealing with overfitting (weight decay, early stopping, dropout), vanishing gradient (ReLU and friends), batch normalization  
+
*[[Media:AN2DL_03_2223_NeuralNetwroksTraining.pdf|[2022/2023] Neural Networks Training]]: dealing with overfitting (weight decay, early stopping, dropout), vanishing gradient (ReLU and friends), batch normalization  
*[[Media:AN2DL_04_2122_RecurrentNeuralNetworks.pdf|[2021/2022] Recurrent Neural Networks]]: learning with sequences, Recurrent Neural Networks, vanishing gradient, Long Short-Term Memories (LSTM), seq2seq model. 
+
*[[Media:AN2DL_05_2021_WordEmbedding.pdf|[2021/2022] Word Embedding]]: neural autoencoders, language models, word embedding, word2vec, glove.
+
*[[Media:AN2DL_06_2021_BeyondSeq2Seq.pdf|[2021/2022] Beyond Sequence 2 Sequence Learning]]: Neural Turing Machines, attention mechanisms, the Transformer.
+
  
Slides from the lectures by Giacomo Boracchi are available in [https://boracchi.faculty.polimi.it/teaching/AN2DL.htm his webpage]; for your convenience I am giving pointers to the slide here for you (in case you note discrepancies please notify me)
+
*[[Media:AN2DL_04_2223_RecurrentNeuralNetworks.pdf|[2022/2023] Recurrent Neural Networks]]: learning with sequences, Recurrent Neural Networks, vanishing gradient, Long Short-Term Memories (LSTM), seq2seq model.
* [2021/2022] [https://boracchi.faculty.polimi.it/teaching/AN2DL/2021_AN2DL_Lez1_ImageClassification.pdf The Image Classification Problem]
+
*[[Media:AN2DL_05_2223_WordEmbedding.pdf|[2022/2023] Word Embedding]]: neural autoencoders, language models, word embedding, word2vec, glove.
* [2021/2022] [https://boracchi.faculty.polimi.it/teaching/AN2DL/2021_AN2DL_Lez2_CNN.pdf Convolutional Neural Networks]
+
*[[Media:AN2DL_06_2223_BeyondSeq2Seq.pdf|[2022/2023] Beyond Sequence 2 Sequence Learning]]: Neural Turing Machines, attention mechanisms in seq2seq models (no Transformer this year!).
* [2021/2022] [https://boracchi.faculty.polimi.it/teaching/AN2DL/2021_AN2DL_Lez3_CNN_TL_Data_Scarcity.pdf CNN Parameters and Training with Data Scarcity]
+
* [2021/2022] [https://boracchi.faculty.polimi.it/teaching/AN2DL/2021_AN2DL_Lez4_CNN_for_segmentation.pdf Fully Convolutional CNN and CNN for Semantic Segmentation]
+
* [2021/2022] [https://boracchi.faculty.polimi.it/teaching/AN2DL/2021_AN2DL_Lez5_CNN_Localization_Explanations_Famous_Architectures.pdf CNN for Localization, CNN Explanations and Famous Architectures]
+
* [2021/2022] [https://boracchi.faculty.polimi.it/teaching/AN2DL/2021_AN2DL_Lez7_Generative_Models.pdf Autoencoders and Generative Adversarial Networks]
+
* [2021/2022] [https://boracchi.faculty.polimi.it/teaching/AN2DL/2021_AN2DL_Lez6_CNN_for_object_detection_Metric_Learning.pdf Object Detction Networks and Metric Learning]
+
  
Slides from the practicals by Francesco Lattari and Eugenio Lomurno
+
Slides from the lectures by Giacomo Boracchi are available in [https://boracchi.faculty.polimi.it/teaching/AN2DL.htm his webpage]; for your convenience, I am giving pointers to the slide here for you (in case you note discrepancies please notify me)
* [https://drive.google.com/drive/folders/1zO_E89H3mjc7nysBkGUt_MmCn1Mb2DzP?usp=sharing Python and convolutions]: a Colab notebook with a crash course on Python 3 to prepare for the practicals and extra material about convolutions!
+
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez1_ImageClassification.pdf [2022/2023]] The Image Classification Problem
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez2_CNN.pdf [2022/2023]] Convolutional Neural Networks
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez3_CNN_TL_Data_Scarcity.pdf [2022/2023]] CNN Parameters and Training with Data Scarcity
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez4_CNN_Famous_Architectures_CNN_Visualization.pdf [2022/2023] Famous CNN architectures and CNN Visualization
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez5_CNN_for_segmentation.pdf [2022/2023]] Fully Convolutional CNN and CNN for Semantic Segmentation
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez6_Localization_Explanations.pdf [2022/2023]] CNN for Localization and CNN Explanations
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez7_CNN_for_object_detection_Metric_Learning.pdf [20223/2023]] Object Detection Networks and Metric Learning
 +
* [https://boracchi.faculty.polimi.it/teaching/AN2DL/2022_AN2DL_Lez8_Generative_Models.pdf [2022/2023]] Autoencoders and Generative Adversarial Networks
 +
 
 +
Slides from the practicals by Francesco Lattari and Eugenio Lomurno will be published here after each lab session: [https://drive.google.com/drive/folders/1iKWNyTBk1KMpnnkkjLwccgOFm6EYzsry?usp=sharing CHECK THIS FOLDER!]
 +
<!--* [https://drive.google.com/drive/folders/1zO_E89H3mjc7nysBkGUt_MmCn1Mb2DzP?usp=sharing Python and convolutions]: a Colab notebook with a crash course on Python 3 to prepare for the practicals and extra material about convolutions!
 
* [https://drive.google.com/drive/folders/1-WFu2S-MBOrNsFRy1IWEptF_hTMLmic9?usp=sharing Feed Forward Neural Networks]: NumPy Arrays, TensorFlow2 Tensors, and Feed Forward Neural Network
 
* [https://drive.google.com/drive/folders/1-WFu2S-MBOrNsFRy1IWEptF_hTMLmic9?usp=sharing Feed Forward Neural Networks]: NumPy Arrays, TensorFlow2 Tensors, and Feed Forward Neural Network
 
* [https://drive.google.com/drive/folders/1-gaItYI8i01STVPrUcqZ8vxK27yGFENn?usp=sharing Dealing with Overfitting]: Hold Out, Early Stopping and Cross-Validation
 
* [https://drive.google.com/drive/folders/1-gaItYI8i01STVPrUcqZ8vxK27yGFENn?usp=sharing Dealing with Overfitting]: Hold Out, Early Stopping and Cross-Validation
Line 411: Line 420:
 
* [https://drive.google.com/drive/folders/1dxfaXo68dWnrS6xHY0yLvUDjTwDPFN6h?usp=sharing Autoencoders and Segmentation]: Exercises on Autoencoders and Multiclass Segmentation  
 
* [https://drive.google.com/drive/folders/1dxfaXo68dWnrS6xHY0yLvUDjTwDPFN6h?usp=sharing Autoencoders and Segmentation]: Exercises on Autoencoders and Multiclass Segmentation  
 
* [https://drive.google.com/drive/folders/1rZQUrs6Rb1dGg5-KLg-y9tT1yw09gr7G?usp=sharing Dealing with Time Series]: Recurrent Neural Networks, Long Short-Term Memories, and other models.
 
* [https://drive.google.com/drive/folders/1rZQUrs6Rb1dGg5-KLg-y9tT1yw09gr7G?usp=sharing Dealing with Time Series]: Recurrent Neural Networks, Long Short-Term Memories, and other models.
 +
-->
  
 
=== External Sources ===
 
=== External Sources ===
 
* [https://towardsdatascience.com/back-to-basics-deriving-back-propagation-on-simple-rnn-lstm-feat-aidan-gomez-c7f286ba973d Deriving Back Propagation on simple RNN/LSTM]: a tutorial step by step to derive the backpropagation formulas
 
* [https://towardsdatascience.com/back-to-basics-deriving-back-propagation-on-simple-rnn-lstm-feat-aidan-gomez-c7f286ba973d Deriving Back Propagation on simple RNN/LSTM]: a tutorial step by step to derive the backpropagation formulas
* [https://karpathy.github.io/2019/04/25/recipe/ A Recipe for Training Neural Networks]: a neat post on Andrej Karpathy blog about do's and don'ts in training neural networks
+
* [https://karpathy.github.io/2019/04/25/recipe/ A Recipe for Training Neural Networks]: a neat post on Andrej Karpathy's blog about do's and don'ts in training neural networks
 
+
 
+
 
<!--*[[Media:AN2DL_Lab1_2020_KerasIntroduction.zip|[2020/2021] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 
<!--*[[Media:AN2DL_Lab1_2020_KerasIntroduction.zip|[2020/2021] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab1_2020_KerasOverfitting.zip|[2020/2021] Facing overfitting in Keras]]: Techniques to limit overfitting in Keras and Tensorboard use (slides + notebook)
 
*[[Media:AN2DL_Lab1_2020_KerasOverfitting.zip|[2020/2021] Facing overfitting in Keras]]: Techniques to limit overfitting in Keras and Tensorboard use (slides + notebook)
Line 422: Line 430:
 
*[[Media:AN2DL_Lab1_2020_KerasSegmentation.zip|[2020/2021] Image Segmentation in Keras]]: How to build, train, and evaluate convolutional models for image segmentation in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab1_2020_KerasSegmentation.zip|[2020/2021] Image Segmentation in Keras]]: How to build, train, and evaluate convolutional models for image segmentation in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab1_2020_KerasRecurrentNeuralNetworks.zip|[2020/2021] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)-->
 
*[[Media:AN2DL_Lab1_2020_KerasRecurrentNeuralNetworks.zip|[2020/2021] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)-->
 
 
<!--
 
<!--
 
*[[Media:AN2DL_Lab1_2019_KerasIntroduction.zip|[2019/2020] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab1_2019_KerasIntroduction.zip|[2019/2020] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
Line 428: Line 435:
 
*[[Media:AN2DL_Lab3_2019_KerasRNN.zip|[2019/2020] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab3_2019_KerasRNN.zip|[2019/2020] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
 
-->
 
-->
<!--
 
===Past Online Exams===
 
  
As for the last semester, Politecnico di Milano exams will be held online. We have not yet decided the digital environment we are going to use for the exams, but you can have a flavor of what to expect by looking at past semester exam calls.  
+
===Exams===
 +
 
 +
Politecnico di Milano exams will be held digitally in presence via the [https://remoteexam.polimi.it/ Politecnico di Milano remote exam platform] using the Safe Exams Browser, it works only on Windows and iOS so please be prepared and test it in advance. You can have a flavor of what to expect by looking at some past exam calls.  
 
   
 
   
 
*[[Media:AN2DL_20200619.pdf| 19/06/2020 Online Exam]]
 
*[[Media:AN2DL_20200619.pdf| 19/06/2020 Online Exam]]
Line 437: Line 444:
 
*[[Media:AN2DL_20200903.pdf| 03/09/2020 Online Exam]]
 
*[[Media:AN2DL_20200903.pdf| 03/09/2020 Online Exam]]
  
'''Note''': this year written exams will be graded 26 points instead of 27. This is because the past year challenges were 5 points while this year they are 6 points worth. This means that the maximum mark is still 26+6=32. You can get an extra point if you participate in the second phase of the Image Segmentation Challenge thus the full mark could reach in principle 33. To get a laude it is required to pass 30.
+
'''Note''': written exams will be graded 20 points plus 10 points are given by 2 software challenges issues only during the semester. Laude will be given to students who, beside getting the highest grade, will show participation in class, will perform particularly well in the challenges (this includes the quality of the report), will submit ahead of time the written exams.  
  
 +
<!--
 
===Lab software setup===
 
===Lab software setup===
  

Revision as of 13:50, 12 March 2023


The following are last-minute news you should be aware of ;-)

* 12/03/2023: Grades of the AN2DL 17/02/2023 call will appear here very soon!!! 
* 15/02/2023: Updates of grades from the AN2DL 26/01/2023 call. 
* 14/02/2023: Grades of the AN2DL 26/01/2023 call. 
* 12/02/2023: Grades of the AN2DL Challenges. 
* 23/01/2023: added link to the detailed course syllabus
* 08/01/2023: update of the teaching material in the page ... a new detailed syllabus will follow too
* 21/12/2022: Grades of the first challenge (v2). 


Course Aim & Organization

Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.

Nowadays, deep neural networks can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).

The course's major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.

Teachers

The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.

Course Program and Syllabus

This goal is pursued in the course by:

  • Presenting major theoretical results underpinning NN (e.g., universal approx, vanishing/exploding gradient, etc.)
  • Describing the most important algorithms for NN training (e.g., backpropagation, adaptive gradient algorithms, etc.)
  • Illustrating the best practices on how to successfully train and use these models (e.g., dropout, data augmentation, etc.)
  • Providing an overview of the most successful Deep Learning architectures (e.g., CNNs, sparse and dense autoencoder, LSTMs for sequence to sequence learning, etc.)
  • Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.

We have compiled a detailed syllabus of the course that students can use to double-check their preparation before the exam.

Detailed course schedule and recordings

A detailed schedule of the course is given in the form of google calendar; topics are just indicative while days and teachers are correct up to some last-minute change (I will notify you by email). Please note that not all days we have lectures!!

For the AN2DL Course Google Calendar look here!

Note: the course is given in parallel to two sessions, the Computer Science session, and the Bioengineering + Mathematical Engineering session. Two calendars exist; lectures are the same, but the scheduling is not necessarily aligned. The previous link points to the Computer Science session, on the same page you find the link to the BIO + MTM session one.

Lectures will be recorded and shared afterward, no streaming of lectures is foreseen.

Recordings of lectures and lab sessions are linked from the google calendar events associated to the corresponding lecture.

Course Evaluation

Course evaluation is composed of two parts:

  • A written examination covering the whole program graded up to 20/30
  • 2 home projects in the form of a "Kaggle style" challenge practicing the topics of the course graded up to 5/30 each

The final score will sum up the grade of the written exam and the grade of the home projects. Home projects are not compulsory and they are issued only once a year.

Teaching Material (the textbook)

Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference, you can check the following textbook, but keep in mind that teachers will not follow it strictly

  • Deep Learning. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.

Regarding the Python programming language, we will provide you with the basics about NumPy and python scripting in case you want some introductory material you can check here

  • Python tutorials: these are the official python tutorials, we suggest 3.An Informal Introduction to Python (Numbers, Strings, Lists), 4.More Control Flow Tools (if, for, range, functions), 5.Data Structures (More on lists, Dictionaries), 9.Classes.

Course Slides

Slides from the lectures by Matteo Matteucci

Slides from the lectures by Giacomo Boracchi are available in his webpage; for your convenience, I am giving pointers to the slide here for you (in case you note discrepancies please notify me)

  • [2022/2023] The Image Classification Problem
  • [2022/2023] Convolutional Neural Networks
  • [2022/2023] CNN Parameters and Training with Data Scarcity
  • [2022/2023 Famous CNN architectures and CNN Visualization
  • [2022/2023] Fully Convolutional CNN and CNN for Semantic Segmentation
  • [2022/2023] CNN for Localization and CNN Explanations
  • [20223/2023] Object Detection Networks and Metric Learning
  • [2022/2023] Autoencoders and Generative Adversarial Networks

Slides from the practicals by Francesco Lattari and Eugenio Lomurno will be published here after each lab session: CHECK THIS FOLDER!

External Sources

Exams

Politecnico di Milano exams will be held digitally in presence via the Politecnico di Milano remote exam platform using the Safe Exams Browser, it works only on Windows and iOS so please be prepared and test it in advance. You can have a flavor of what to expect by looking at some past exam calls.

Note: written exams will be graded 20 points plus 10 points are given by 2 software challenges issues only during the semester. Laude will be given to students who, beside getting the highest grade, will show participation in class, will perform particularly well in the challenges (this includes the quality of the report), will submit ahead of time the written exams.