Summary Instance Based Learning

Instance Based Learning

[Read Ch. 8]
 -Nearest Neighbor
k


 Locally weighted regression
 Radial basis functions
 Case-based reasoning
 Lazy and eager learning




199 lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

, Instance-Based Learning

Key idea: just store all training examples h ( )i x i ; f xi


Nearest neighbor:
 Given query instance , rst locate nearest
xq

training example , then estimate
xn
^( )
f xq ( ) f xn


k-Nearest neighbor:
 Given , take vote among its nearest nbrs (if
xq k

discrete-valued target function)
 take mean of values of nearest nbrs (if
f k

real-valued)
P ( )
^( )
k
i=1
f xi
f xq
k




200 lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

, When To Consider Nearest Neighbor


 Instances map to points in < n




 Less than 20 attributes per instance
 Lots of training data
Advantages:
 Training is very fast
 Learn complex target functions
 Don't lose information
Disadvantages:
 Slow at query time
 Easily fooled by irrelevant attributes




201 lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997