Difference between revisions of "Artificial Neural Networks and Deep Learning"

From Chrome
Jump to: navigation, search
(Detailed course schedule)
(Course Slides)
Line 260: Line 260:
  
 
Slides from the practicals by Francesco Lattari
 
Slides from the practicals by Francesco Lattari
 +
*[2020/2021] Introduction to Keras: Introduction to Keras and Tensorflow2 (slides + notebook)
 +
*[2020/2021] Convolutional architectures in Keras]]: How to build, train, and evaluate convolutional models for classification and segmentation in Keras and Tensorflow2 (slides + notebook)
 +
*[2020/2021] Recurrent architectures in Keras: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
 +
<!--
 
*[[Media:AN2DL_Lab1_2019_KerasIntroduction.zip|[2019/2020] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab1_2019_KerasIntroduction.zip|[2019/2020] Introduction to Keras]]: Introduction to Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab2_2019_KerasCNN.zip|[2019/2020] Convolutional architectures in Keras]]: How to build, train, and evaluate convolutional models for classification and segmentation in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab2_2019_KerasCNN.zip|[2019/2020] Convolutional architectures in Keras]]: How to build, train, and evaluate convolutional models for classification and segmentation in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab3_2019_KerasRNN.zip|[2019/2020] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
 
*[[Media:AN2DL_Lab3_2019_KerasRNN.zip|[2019/2020] Recurrent architectures in Keras]]: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)
 +
-->
  
 
===Kaggle Homeworks===
 
===Kaggle Homeworks===

Revision as of 10:43, 14 September 2020


The following are last minute news you should be aware of ;-)

14/09/2020: So far, accoding to the University rules Team 1 is defined by an EVEN Codice Persona, while Team 2 by an ODD Codice persona
13/09/2020: The course is about to start ... stay tuned!


Course Aim & Organization

Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.

Nowadays, deep neural network can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).

The course major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.

Teachers

The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.

Course Program and Syllabus

This goal is pursued in the course by:

  • Presenting major theoretical results underpinning NN (e.g., universal approx, vanishing/exploding gradient, etc.)
  • Describing the most important algorithms for NN training (e.g., backpropagation, adaptive gradient algorithms, etc.)
  • Illustrating the best practices on how to successfully train and use these models (e.g., dropout, data augmentation, etc.)
  • Providing an overview of the most successful Deep Learning architectures (e.g., CNNs, sparse and dense autoencoder, LSTMs for sequence to sequence learning, etc.)
  • Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.

We have compiled a detailed syllabus of the course student can use to doublecheck their preparation against before the exam.

  • [2019/2020] Course Syllabus: a detailed list of topics covered by the course and which students are expected to know when approaching the exam

Detailed course schedule

A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all days we have lectures!!

Note: Lecture timetable interpretation
* On Wednesday, in 2.0.2 (EX N.0.2), starts at 15:15, ends at 17:15
* On Wednesday, in 2.1.2 (EX N.1.2), starts at 17:15, ends at 19:15
* On Thursday, in teacher webex room, starts at 16:15, ends at 19:15
Date Day Time Room Teacher Type Topic
16/09/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Matteo Matteucci Course Introduction [Team 1]
16/09/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Matteo Matteucci Course Introduction [Team 2]
17/09/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Perceptron and Hebbian Learning
23/09/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Matteo Matteucci Python and Hebbian Learning Example [Team 1]
23/09/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Matteo Matteucci Python and Hebbian Learning Example [Team 2]
24/09/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Feed forward neural networks + Backprop
30/09/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Matteo Matteucci Backpropagation Example (with Hebbian parallel) [Team 1]
30/09/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Matteo Matteucci Backpropagation Example (with Hebbian parallel) [Team 2]
01/10/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Error Functions and Weight decay
07/10/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Francesco Lattari KERAS NN - Feed forward neural networks [Team 1]
07/10/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Francesco Lattari KERAS NN - Feed forward neural networks [Team 2]
08/10/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Facing overffitting .. and other training tricks
14/10/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Francesco Lattari KERAS NN - FFNN and Overfitting [Team 1]
14/10/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Francesco Lattari KERAS NN - FFNN and Overfitting [Team 2]
15/10/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci -- No Lecture Today --
21/10/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Giacomo Boracchi Introduction to Image classification [Team 1]
21/10/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Giacomo Boracchi Introduction to Image classification [Team 2]
22/10/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Giacomo Boracchi Convolutional Training Netwroks
28/10/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Giacomo Boracchi Famous CNN Architectures [Team 1]
28/10/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Giacomo Boracchi Famous CNN Architectures [Team 2]
29/10/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Giacomo Boracchi Training with data scarcity
04/11/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Francesco Lattari KERAS NN - Convolutional Neural Networks [Team 1]
04/11/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Francesco Lattari KERAS NN - Convolutional Neural Networks [Team 1]
05/11/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Giacomo Boracchi Fully Convolutional CNN, CNN for image segmentation
11/11/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture --- -- No Lecture (Prove in Itinere) --
12/11/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture --- -- No Lecture (Prove in Itinere) --
11/11/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Giacomo Boracchi CNN for localization and detection
18/11/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Giacomo Boracchi GANs [Team 1]
18/11/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Giacomo Boracchi GANs [Team 2]
19/11/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Recurrent neural networks + LSTM
25/11/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Matteo Matteucci Word Embedding [Team 1]
25/11/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Matteo Matteucci Word Embedding [Team 2]
26/11/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Sequence to sequence learning
02/12/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Francesco Lattari KERAS NN [Team 1]
02/12/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Francesco Lattari KERAS NN [Team 2]
03/12/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Attention Mechanism and Transformer
09/12/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture Matteo Matteucci KERAS NN [Team 1]
09/12/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture Matteo Matteucci KERAS NN [Team 2]
10/12/2020 Thursday 16:15 - 19:15 Virtual Room Lecture Matteo Matteucci Questions and Answers
16/12/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture --- -- Spare Lecture --
16/12/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture --- -- Spare Lecture --
17/12/2020 Thursday 16:15 - 19:15 Virtual Room Lecture --- -- Spare Lecture --
23/12/2020 Wednesday 15:15 - 17:15 2.0.2 Lecture --- -- Spare Lecture --
23/12/2020 Wednesday 15:15 - 17:15 2.1.2 Lecture --- -- Spare Lecture --


Course Evaluation

Course evaluation is composed of two parts:

  • A written examination covering the whole program graded up to 26/32 ... may be less
  • A home project in the form of a "Kaggle style" competition practicing the topics of the course graded up to 6/32 ... may be more

The final score will sum the grade of the written exam and the grade of the home project.

You can find here one example of the exam text to get a flavor of what to expect in the written examination.

Teaching Material (the textbook)

Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference you can check the following text, but keep in mind that teachers will not follow it strictly

  • Deep Learning. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.

Course Slides

Slides from the lectures by Matteo Matteucci

Slides from the lectures by Giacomo Boracchi are available in his webpage, for you

  • Image Classification: Image classification and related issues, template matching, image classification via nearest neighbors methods, image classification via linear classifiers, image classification via hand-crafted features.

Slides from the practicals by Francesco Lattari

  • [2020/2021] Introduction to Keras: Introduction to Keras and Tensorflow2 (slides + notebook)
  • [2020/2021] Convolutional architectures in Keras]]: How to build, train, and evaluate convolutional models for classification and segmentation in Keras and Tensorflow2 (slides + notebook)
  • [2020/2021] Recurrent architectures in Keras: How to build, train, and evaluate recurrent neural architectures in Keras and Tensorflow2 (slides + notebook)

Kaggle Homeworks

As part of the evaluation (up to 6 marks in the final grade) we are issuing 3 homeworks in the form of "Kaggle style" competitions. They are meant to practice the course topics on simple image recognition tasks.

[2020/2021]

Not yet published

[2019/2020]

  • Image Classification Homework: the first homework is about image classification with convolutional neural networks. The deadline to submit the results is November the 30th.
  • Image Segmentation Homework: the second homework is about image segmentation with convolutional neural networks and the like. The deadline to submit the results is December the 17th.
  • Visual Question Answering Homework: the third homework is about visual question answering with convolutional and recurrent neural networks ... plus word2vec. The deadline to submit the results is January the 15th.