Difference between revisions of "SC:Knowledge Engineering"

From Chrome
Jump to: navigation, search
(→‎Neural Networks and Evolutionary Computation)
(→‎Course Evaluation)
 
(92 intermediate revisions by 2 users not shown)
Line 2: Line 2:
  
 
The following are last minute news you should be aware of ;-)
 
The following are last minute news you should be aware of ;-)
−
09/03/2010: The course starts today!
+
  07/08/2-13: Published the text and the [[media:Grades_130726.pdf|results]] for the 26/07/2013 exam
 +
  18/07/2-13: Published the text and the [[media:Voti_20130711.pdf|results]] for the 11/07/2013 exam
 +
  13/06/2013: Update to detailed schedule for the extra lecture on 17/06/2013
 +
  05/03/2013: The course starts today!  
  
 
==Course Aim & Organization==
 
==Course Aim & Organization==
Line 28: Line 31:
  
 
Lectures are on:
 
Lectures are on:
−
* Tuesday 8:15-10:15 in room A3.8
+
* Monday 16:15-18:15 in room A3.8
−
* Wednesday 10:15-12:15 in room A3.6
+
* Tuesday 08:15-10:15 in room A3.8
 +
 
 +
A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all Tuesdays and Wednesdays are in!!
 +
 
 +
{| border="1" align="center" style="text-align:center;"
 +
|-
 +
|Date || Day || Time || Room || Teacher || Topic
 +
|-
 +
|05/03/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Course introduction: from artificial intelligence to natural computation
 +
|-
 +
|11/03/2013 || Monday || 16:15 - 18:15 || 4.1 || Matteo Matteucci || Useful background: Maximum Likelihood Estimation and Gradient descent optimization
 +
|-
 +
|12/03/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Introduction to neaural networks, the biologic neuron, the perceptron
 +
|-
 +
|18/03/2013 || Monday || 16:15 - 18:15 || 4.1 || --- || No Lecture
 +
|-
 +
|19/03/2013 || Tuesday || 08:15 - 10:15 || 3.8 || --- || No Lecture
 +
|-
 +
|25/03/2013 || Monday || 16:15 - 18:15 || 4.1 || Andrea Bonarini || Knowledge representation: introduction
 +
|-
 +
|26/03/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Andrea Bonarini || Knowledge  representation: the conceptual layer
 +
|-
 +
|01/04/2013 || Monday || 16:15 - 18:15 || 4.1 || --- || No Lecture
 +
|-
 +
|02/04/2013 || Tuesday || 08:15 - 10:15 || 3.8 || --- || No Lecture
 +
|-
 +
|08/04/2013 || Monday || 16:15 - 18:15 || 4.1 || Matteo Matteucci || Hebbian learning, Perceptron learning exercise, the XOR problem
 +
|-
 +
|09/04/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Feed forward neural networks: backpropagation
 +
|-
 +
|15/04/2013 || Monday || 16:15 - 18:15 || 4.1 || Matteo Matteucci || Demo and derivation of squared error function
 +
|-
 +
|16/04/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Cross Entropy error function, Overfitting, Crossvalidation techniques, Early stopping
 +
|-
 +
|22/04/2013 || Monday || 16:15 - 18:15 || 4.1 || --- || No Lecture because of Lauree
 +
|-
 +
|23/04/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Early Stopping and Weight Decay
 +
|-
 +
|29/04/2013 || Monday || 16:15 - 18:15 || 4.1 || Matteo Matteucci || Recurrent Architectures
 +
|-
 +
|30/04/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Radial Basis Functions
 +
|-
 +
|06/05/2013 || Monday || 16:15 - 18:15 || 4.1 || --- || No Lecture because of Lauree
 +
|-
 +
|07/05/2013 || Tuesday || 08:15 - 10:15 || 3.8 || --- || No Lecture because of Lauree
 +
|-
 +
|13/05/2013 || Monday || 16:15 - 18:15 || 4.1 || Andrea Bonarini || KR: the technique layer
 +
|-
 +
|14/05/2013 || Tuesday || 08:15 - 10:15 || 3.8 ||Andrea Bonarini || KR: knowledge modeling
 +
|-
 +
|20/05/2013 || Monday || 16:15 - 18:15 || 4.1 || Andrea Bonarini || Expert systems
 +
|-
 +
|21/05/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Andrea Bonarini || Knowledge acquisition
 +
|-
 +
|27/05/2013 || Monday || 16:15 - 18:15 || 4.1 || Matteo Matteucci || Introduction to Genetic Algoritmhs
 +
|-
 +
|28/05/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Genetic Algorithms: conding and operators (I)
 +
|-
 +
|03/06/2013 || Monday || 16:15 - 18:15 || 4.1 || Matteo Matteucci ||Genetic Algorithms: conding and operators (II)
 +
|-
 +
|04/06/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Matteo Matteucci || Estimation of distribution algorithms
 +
|-
 +
|10/06/2013 || Monday || 16:15 - 18:15 || 4.1 ||  Andrea Bonarini || Uncertainty representation
 +
|-
 +
|11/06/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Andrea Bonarini || Fuzzy sets
 +
|-
 +
|17/06/2013 || Monday || 10:15 - 12:15 || 4.2 || Matteo Matteucci || Questions and Answers
 +
|-
 +
|17/06/2013 || Monday || 16:15 - 18:15 || 4.1 || Andrea Bonarini || Fuzzy rules
 +
|-
 +
|18/06/2013 || Tuesday || 08:15 - 10:15 || 3.8 || Andrea Bonarini || Fuzzy systems
 +
|}
  
 
===Course Evaluation===
 
===Course Evaluation===
Line 37: Line 111:
 
# Topics covered by the teaching assistant during the course
 
# Topics covered by the teaching assistant during the course
 
Each part will score 16/32 and the exam is passed if the sum of the two parts, after rounding, is at least 18.
 
Each part will score 16/32 and the exam is passed if the sum of the two parts, after rounding, is at least 18.
 +
 +
As a reference for the exam format you can have a look at the following material from KE:
 +
* [[Media:2013_07_26_KE.pdf | Exam given on 26/07/2013]]
 +
* [[Media:2013_07_11_KE.pdf | Exam given on 11/07/2013]]
 +
* [[Media:2013_01_30_KE.pdf | Exam given on 30/01/2013]]
 +
* [[Media:2012_09_19_KE.pdf | Exam given on 19/09/2012]]
 +
* [[Media:2012_07_10_KE.pdf | Exam given on 10/07/2012]]
 +
* [[Media:2012_06_26_KE.pdf | Exam given on 26/06/2012]]
 +
* [[Media:2012_02_03_KE.pdf | Exam given on 03/02/2012]]
 +
* [[Media:2011_09_08_KE.pdf | Exam given on 08/09/2011]]
 +
* [[Media:2011_07_15_KE.pdf | Exam given on 15/07/2011]]
 +
* [[Media:2011_06_29_KE.pdf | Exam given on 29/06/2011]]
 +
 +
or from the Italian version (Ingegneria della Conoscenza):
 +
* [[Media:AppelloIC100204.pdf | Exam given on 04/02/2010]]
 +
* [[Media:AppelloIC090909.pdf | Exam given on 09/09/2009]]
 +
* [[Media:AppelloIC090720.pdf | Exam given on 20/07/2009]]
 +
* [[Media:AppelloIC090707.pdf | Exam given on 07/07/2009]]
 +
* [[Media:AppelloIC090228.pdf | Exam given on 28/02/2009]]
 +
* [[Media:AppelloIC090218.pdf | Exam given on 18/02/2009]]
  
 
==Teaching Material==
 
==Teaching Material==
Line 43: Line 137:
 
* [http://research.microsoft.com/en-us/um/people/cmbishop/nnpr.htm Neural Networks and Pattern Recognition], C. Bishop, Oxford University Press, 1995
 
* [http://research.microsoft.com/en-us/um/people/cmbishop/nnpr.htm Neural Networks and Pattern Recognition], C. Bishop, Oxford University Press, 1995
 
* [http://www.francoangeli.it/Ricerca/Scheda_Libro.asp?CodiceLibro=1340.70 Reti neuronali e metodi statistici], S. Ingrassia, and C. Davino. 2002.
 
* [http://www.francoangeli.it/Ricerca/Scheda_Libro.asp?CodiceLibro=1340.70 Reti neuronali e metodi statistici], S. Ingrassia, and C. Davino. 2002.
−
* [http://books.google.com/books?id=0eznlz0TF-IC&dq=melanie+mitchell+an+introduction+to+genetic+algorithms&printsec=frontcover&source=bn&hl=en&ei=U4lCS4CBEcyc_AbehcWECQ&sa=X&oi=book_result&ct=result&resnum=4&ved=0CBoQ6AEwAw#v=onepage&q=&f=false An introduction to gentic algorithms], Melanie Mitchell, MIT Press, 1998.
+
* [http://books.google.com/books?id=0eznlz0TF-IC&dq=melanie+mitchell+an+introduction+to+genetic+algorithms&printsec=frontcover&source=bn&hl=en&ei=U4lCS4CBEcyc_AbehcWECQ&sa=X&oi=book_result&ct=result&resnum=4&ved=0CBoQ6AEwAw#v=onepage&q=&f=false An introduction to genetic algorithms], Melanie Mitchell, MIT Press, 1998.
−
* ...
+
* [http://books.google.com/books?id=-DcKLlQQ1vwC&dq=Fuzzy+Logic+with+Engineering+Applications&printsec=frontcover&source=bn&hl=en&ei=ft0DTIGlIJzCmgPAu_yTAQ&sa=X&oi=book_result&ct=result&resnum=4&ved=0CCsQ6AEwAw#v=onepage&q&f=false Fuzzy Logic with Engineering Applications (Third Edition)], T. Ross, Wiley, 2010
−
* ...
+
* [http://books.google.com/books?id=Q_asI_t61F8C&printsec=frontcover#v=onepage&q&f=false Fuzzy Logic: come insegneremo alle macchine a ragionare da uomini], A. G. Pizzaleo, Castelvecchi, Roma.  
  
−
In the following you can find the lecture slides used by the teacher and the teaching assistant during classes. Some additional material that could be used to prepare the oral examination is provided as well.
+
In the following you can find the lecture slides used by the teacher and the teaching assistant during classes. Some additional material that could be used to prepare the oral examination is provided as well.  
  
−
=== Neural Networks and Evolutionary Computation ===
+
==== Neural Networks and Evolutionary Computation ====
  
−
* [Ke-handout-lecture-ml.pdf Lecture Set 1]: Introduction to Machine Learning
+
* [[Media:Ke-handout-lecture-ml.pdf | Machine Learning Lecture Set]]: Introduction to the Course and to Machine Learning
−
* [http://home.dei.polimi.it/matteucc/lectures/IC/handout-NN.pdf Lecture Set 2]: Neural Networks lectures and more
+
* [[Media:Ke-handout-lecture-nn.pdf | Neural Network Lecture Set]]: Neural Networks lectures and more
−
* [http://home.dei.polimi.it/matteucc/lectures/IC/handout-GA.pdf Lecture Set 3]: Evolutionary Computation and Genetic Algorithms
+
** [[Media:Ke-Perceptron-Exercise.pdf | Hebian Learning Exercise]]: this short exercise on Hebbian learning of perceptrons complements the one in the Neural Networks Lecture Set.
 +
** [[Media:Esempio_NN.pdf | Feedforward NN example]]: this is an example on the kind of questions about Neural Networks you could face during the exam.
 +
* [[Media:Ke-handout-lecture-ga.pdf | Genetic Algorithms Lecture Set]]: Evolutionary Computation and Genetic Algorithms
 +
** [http://davide.eynard.it/zelif/GA/gacrack_20100609.pdf  Link to the slides from the lecture on "Cracking codes with Genetic Algorithms"]
 +
** [http://medal.cs.umsl.edu/files/PMBGAs_2008.pdf  Link to the slides by Pelikan on "Evolution of Distribution Algorithms"]
 +
** [http://video.google.com/videoplay?docid=-4468676449807637547&hl=en&fs=true#  Link to a presentation by Pelikan on "Evolution of Distribution Algorithms"]
  
−
=== Knowledge Engineering ===
+
==== Knowledge Engineering ====
  
−
* ...
+
* [[Media:1-KnowledgeRepresentationHandout.pdf | Knowledge Representation Lecture Set]]: Slides about Knowledge Representation (Last Update: March 14, 2011)
 +
* [[Media:2-KnowledgeTechniquesHandout.pdf | Techniques for Knowledge Representation Lecture Set]]: Slides about Techniques for Knowledge Representation  (Last Update: March 29, 2011)
 +
* [[Media:3-KnowledgeAcquisition.pdf | Techniques for Knowledge Acquisition Lecture Set]]: Slides about Techniques for Knowledge Acquisition
 +
* [[Media:4-LifeCycleKBSHandout.pdf | Life cycle of an Expert System Lecture Set]]: Slides about the life cicle of an Expert System
 +
* [[Media:5-KBSArchitectureHandout.pdf | Architectures for KBS Lecture Set]]: Slides about architectures for Expert Systems
 +
<!--* [[Media:5-ApplicazioniIA.pdf | AI Applications Lecture Set]]: Slides about Applications of AI Techniques (they are in Italian)
 +
-->
  
−
=== Fuzzy Logic ===
+
==== Uncertainty management and Fuzzy systems ====
 +
* [[Media:6-UncertaintyManagementHandout.pdf | Management of uncertainty Lecture Set]]: Slides about management of uncertainty in KBS
 +
* [[Media:IntroFuzzySets.pdf  | Introduction to Fuzzy Sets ]]: Slides introducing to fuzzy sets
 +
* [[Media:IntroFuzzyLogic.pdf  | Introduction to Fuzzy Logic ]]: Slides introducing fuzzy logic
 +
* [[Media:FuzzyRules.pdf  | Introduction to Fuzzy Rules ]]: Slides about fuzzy rules
 +
* [[Media:FuzzyDesign.pdf  | Design of Fuzzy Systems ]]: Slides about the design of systems based on fuzzy logic
 +
* [[Media:FuzzyApplications.pdf  | Applications of Fuzzy Systems ]]: Slides about some fuzzy system application categories
  
−
* ...
+
==== Additional Lectures to complement Slides and Textbooks ====
−
 
+
−
=== Additional Lectures to complement Slides and Textbooks ===
+
  
 
* [ftp://ftp.sas.com/pub/neural/FAQ.html Neural Networks FAQ]: a great, peer-reviewed, repository of information about neural networks. You should know it by heart!
 
* [ftp://ftp.sas.com/pub/neural/FAQ.html Neural Networks FAQ]: a great, peer-reviewed, repository of information about neural networks. You should know it by heart!
−
* [http://home.dei.polimi.it/matteucc/lectures/IC/OverfittingNeuralNetwork.pdf Overfitting in Neural Networks]: a paper describing the early stopping technique and its rationale.  
+
* [http://home.dei.polimi.it/matteucc/lectures/IC/OverfittingNeuralNetworks.pdf Overfitting in Neural Networks]: a paper describing the early stopping technique and its rationale.  
 
* [http://home.dei.polimi.it/matteucc/lectures/IC/Proben1.pdf Proben 1]: read this it together with the previous paper to have a clear idea on how early stopping is related to network evaluation.
 
* [http://home.dei.polimi.it/matteucc/lectures/IC/Proben1.pdf Proben 1]: read this it together with the previous paper to have a clear idea on how early stopping is related to network evaluation.
 
* [http://home.dei.polimi.it/matteucc/lectures/IC/ga_tutorial.pdf Genetic Algorithm Tutorial]: a neat tutorial on genetic algorithms by Darrel Whitley
 
* [http://home.dei.polimi.it/matteucc/lectures/IC/ga_tutorial.pdf Genetic Algorithm Tutorial]: a neat tutorial on genetic algorithms by Darrel Whitley
 
* [http://www.obitko.com/tutorials/genetic-algorithms/ Genetic Algorithm with applets]: a nice website/tutorial about genetic algorithm I've been inspired for my lectures :-)
 
* [http://www.obitko.com/tutorials/genetic-algorithms/ Genetic Algorithm with applets]: a nice website/tutorial about genetic algorithm I've been inspired for my lectures :-)
 +
* [http://www-module.cs.york.ac.uk/evo/1011/lect/SP_EDAs_EVO.pdf Estimation of Distribution Algorithms]: slides about Estimation of Distribution Algorithms by Simon Poulding @ University of York
 +
 
* [http://home.dei.polimi.it/matteucc/lectures/IC/AppuntiMatteucci.zip Appunti (bigino)]: these are notes taken in the past by some students of mine ... they are in Italian!
 
* [http://home.dei.polimi.it/matteucc/lectures/IC/AppuntiMatteucci.zip Appunti (bigino)]: these are notes taken in the past by some students of mine ... they are in Italian!

Latest revision as of 00:16, 7 August 2013


The following are last minute news you should be aware of ;-) 

 07/08/2-13: Published the text and the results for the 26/07/2013 exam
 18/07/2-13: Published the text and the results for the 11/07/2013 exam
 13/06/2013: Update to detailed schedule for the extra lecture on 17/06/2013
 05/03/2013: The course starts today!

Course Aim & Organization

The course introduces the general principles of Artificial Intelligence and its applications. Two approaches to model building and knowledge representation will be presented: the traditional one, based on symbolic representation of knowledge (e.g., frames, rules, fuzzy logic, ...), and one inspired to biological models (e.g., neural networks and genetic algorithms).

Teachers

The course is composed by a blending of lectures and exercises by the course teacher and the teaching assistant.

Course Program and Schedule

Lectures will cover the following topics:

  • Artificial intelligence: problems, approaches and applications
  • Models for knowledge representation biologically inspired and adaptive models.
  • Machine Learning: supervised methods, unsupervised methods, reinforcement learning.
  • Artificial Neural Networks and Genetic Algorithms.
  • Knowledge representation techniques: semantic networks, frames, objects, production rules.
  • Uncertainty and approximate knowledge representation. Fuzzy systems.
  • Knowledge engineering: methods to develop knowledge based systems.
  • Knowledge acquisition, conceptualization and modeling.

Lectures are on:

  • Monday 16:15-18:15 in room A3.8
  • Tuesday 08:15-10:15 in room A3.8

A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all Tuesdays and Wednesdays are in!!

Date Day Time Room Teacher Topic
05/03/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Course introduction: from artificial intelligence to natural computation
11/03/2013 Monday 16:15 - 18:15 4.1 Matteo Matteucci Useful background: Maximum Likelihood Estimation and Gradient descent optimization
12/03/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Introduction to neaural networks, the biologic neuron, the perceptron
18/03/2013 Monday 16:15 - 18:15 4.1 --- No Lecture
19/03/2013 Tuesday 08:15 - 10:15 3.8 --- No Lecture
25/03/2013 Monday 16:15 - 18:15 4.1 Andrea Bonarini Knowledge representation: introduction
26/03/2013 Tuesday 08:15 - 10:15 3.8 Andrea Bonarini Knowledge representation: the conceptual layer
01/04/2013 Monday 16:15 - 18:15 4.1 --- No Lecture
02/04/2013 Tuesday 08:15 - 10:15 3.8 --- No Lecture
08/04/2013 Monday 16:15 - 18:15 4.1 Matteo Matteucci Hebbian learning, Perceptron learning exercise, the XOR problem
09/04/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Feed forward neural networks: backpropagation
15/04/2013 Monday 16:15 - 18:15 4.1 Matteo Matteucci Demo and derivation of squared error function
16/04/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Cross Entropy error function, Overfitting, Crossvalidation techniques, Early stopping
22/04/2013 Monday 16:15 - 18:15 4.1 --- No Lecture because of Lauree
23/04/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Early Stopping and Weight Decay
29/04/2013 Monday 16:15 - 18:15 4.1 Matteo Matteucci Recurrent Architectures
30/04/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Radial Basis Functions
06/05/2013 Monday 16:15 - 18:15 4.1 --- No Lecture because of Lauree
07/05/2013 Tuesday 08:15 - 10:15 3.8 --- No Lecture because of Lauree
13/05/2013 Monday 16:15 - 18:15 4.1 Andrea Bonarini KR: the technique layer
14/05/2013 Tuesday 08:15 - 10:15 3.8 Andrea Bonarini KR: knowledge modeling
20/05/2013 Monday 16:15 - 18:15 4.1 Andrea Bonarini Expert systems
21/05/2013 Tuesday 08:15 - 10:15 3.8 Andrea Bonarini Knowledge acquisition
27/05/2013 Monday 16:15 - 18:15 4.1 Matteo Matteucci Introduction to Genetic Algoritmhs
28/05/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Genetic Algorithms: conding and operators (I)
03/06/2013 Monday 16:15 - 18:15 4.1 Matteo Matteucci Genetic Algorithms: conding and operators (II)
04/06/2013 Tuesday 08:15 - 10:15 3.8 Matteo Matteucci Estimation of distribution algorithms
10/06/2013 Monday 16:15 - 18:15 4.1 Andrea Bonarini Uncertainty representation
11/06/2013 Tuesday 08:15 - 10:15 3.8 Andrea Bonarini Fuzzy sets
17/06/2013 Monday 10:15 - 12:15 4.2 Matteo Matteucci Questions and Answers
17/06/2013 Monday 16:15 - 18:15 4.1 Andrea Bonarini Fuzzy rules
18/06/2013 Tuesday 08:15 - 10:15 3.8 Andrea Bonarini Fuzzy systems

Course Evaluation

The course evaluation is performed by a written exam divided in two parts (lasting usually 2.5 hours in total):

  1. Topics covered by the course teacher during the course
  2. Topics covered by the teaching assistant during the course

Each part will score 16/32 and the exam is passed if the sum of the two parts, after rounding, is at least 18.

As a reference for the exam format you can have a look at the following material from KE:

or from the Italian version (Ingegneria della Conoscenza):

Teaching Material

The textbooks used for the course are:

In the following you can find the lecture slides used by the teacher and the teaching assistant during classes. Some additional material that could be used to prepare the oral examination is provided as well.

Neural Networks and Evolutionary Computation

Knowledge Engineering

Uncertainty management and Fuzzy systems

Additional Lectures to complement Slides and Textbooks

  • Appunti (bigino): these are notes taken in the past by some students of mine ... they are in Italian!
Retrieved from "http://chrome.ws.dei.polimi.it/index.php?title=SC:Knowledge_Engineering&oldid=1587"