Summary Articial Neural Networks

Arti cial Neural Networks

[Read Ch. 4]
[Recommended exercises 4.1, 4.2, 4.5, 4.9, 4.11]
 Threshold units
 Gradient descent
 Multilayer networks
 Backpropagation
 Hidden layer representations
 Example: Face Recognition
 Advanced topics




74 lecture slides for textbook Machine Learning, T. Mitchell, McGraw Hill, 1997

, Connectionist Models

Consider humans:
 Neuron switching time ~ :001 second
 Number of neurons ~ 10 10


 Connections per neuron ~ 10 ? 4 5


 Scene recognition time ~ :1 second
 100 inference steps doesn't seem like enough
! much parallel computation
Properties of arti cial neural nets (ANN's):
 Many neuron-like threshold switching units
 Many weighted interconnections among units
 Highly parallel, distributed process
 Emphasis on tuning weights automatically



75 lecture slides for textbook Machine Learning, T. Mitchell, McGraw Hill, 1997

, When to Consider Neural Networks

 Input is high-dimensional discrete or real-valued
(e.g. raw sensor input)
 Output is discrete or real valued
 Output is a vector of values
 Possibly noisy data
 Form of target function is unknown
 Human readability of result is unimportant
Examples:
 Speech phoneme recognition [Waibel]
 Image classi cation [Kanade, Baluja, Rowley]
 Financial prediction




76 lecture slides for textbook Machine Learning, T. Mitchell, McGraw Hill, 1997

, ALVINN drives 70 mph on highways




Sharp Straight Sharp
Left Ahead Right


30 Output
Units



4 Hidden
Units




30x32 Sensor
Input Retina




77 lecture slides for textbook Machine Learning, T. Mitchell, McGraw Hill, 1997