Module V
Kernel Machines - Support Vector Machine - Optimal Separating hyper plane, Soft margin
hyperplane, Kernel trick, Kernel functions. Discrete Markov Processes, Hidden Markov models,
Three basic problems of HMMs - Evaluation problem, finding
state sequence, Learning model parameters. Combining multiple learners, Ways to
achieve diversity, Model combination schemes, Voting, Bagging, Booting
Support Vector Machine – SVM
SVM is supervised learning models for classification and regression analysis
A Support Vector Machine (SVM) performs classification by finding the hyperplane that maximizes
the margin between the two classes. The vectors that define the hyperplane are the support vectors.
Support Vectors
• Support vectors are the data points that lie closest to the decision surface (or hyperplane)
• They are the data points most difficult to classify
•Support vectors have direct influence on the optimum location of the decision surface
1
,Margin
Distance from the decision surface to the closest data point determines the margin of the
classifier.
SVM creates hyperplanes that have the largest margin to separate given data into classes.
The larger the margin, the lower is the generalization error of the classifier.
The margin (ρ) of the separator is the distance between support vectors
Maximum Margin Classifier/Optimal separating hyperplane
The hyperplane has to be as far as possible from the examples of both classes This means that
we have to maximize the margin
This is the best hyperplane because it reduces the generalization error the most.
2
, If we add new data, the Maximum Margin Classifier is the best hyperplane to correctly
classify the new data.
We can separate the red and blue objects with an infinite number of hyperplanes.
Which hyperplane is the best? Well, the best hyperplane is the one that maximizes the
margin.
Maximum Margin Classifier requires the two classes to be completely linearly separated.
3
Kernel Machines - Support Vector Machine - Optimal Separating hyper plane, Soft margin
hyperplane, Kernel trick, Kernel functions. Discrete Markov Processes, Hidden Markov models,
Three basic problems of HMMs - Evaluation problem, finding
state sequence, Learning model parameters. Combining multiple learners, Ways to
achieve diversity, Model combination schemes, Voting, Bagging, Booting
Support Vector Machine – SVM
SVM is supervised learning models for classification and regression analysis
A Support Vector Machine (SVM) performs classification by finding the hyperplane that maximizes
the margin between the two classes. The vectors that define the hyperplane are the support vectors.
Support Vectors
• Support vectors are the data points that lie closest to the decision surface (or hyperplane)
• They are the data points most difficult to classify
•Support vectors have direct influence on the optimum location of the decision surface
1
,Margin
Distance from the decision surface to the closest data point determines the margin of the
classifier.
SVM creates hyperplanes that have the largest margin to separate given data into classes.
The larger the margin, the lower is the generalization error of the classifier.
The margin (ρ) of the separator is the distance between support vectors
Maximum Margin Classifier/Optimal separating hyperplane
The hyperplane has to be as far as possible from the examples of both classes This means that
we have to maximize the margin
This is the best hyperplane because it reduces the generalization error the most.
2
, If we add new data, the Maximum Margin Classifier is the best hyperplane to correctly
classify the new data.
We can separate the red and blue objects with an infinite number of hyperplanes.
Which hyperplane is the best? Well, the best hyperplane is the one that maximizes the
margin.
Maximum Margin Classifier requires the two classes to be completely linearly separated.
3
