DEPARTMENT OF ARTIFICIAL INTELLIGENCE AND
DATA SCIENCE
REGULATION R2021
II YEAR - IV SEMESTER
CS3491 – ARTIFICIAL INTELLIGENCE AND
MACHINE LEARNING
III Unit
Prepared by,
Dr. S. RAVICHANDRAN
, UNIT III SUPERVISED LEARNING
Introduction to machine learning – Linear Regression Models: Least squares, single & multiple variables, Bayesian linear
regression, gradient descent,
◼ Linear Classification Models: Discriminant function – Probabilistic discriminative model - Logistic regression, Probabilistic
generative model – Naive Bayes, Maximum margin classifier – Support vector machine, Decision Tree, Random forests.
Introduction to machine learning
Machine learning is a field of inquiry devoted to understanding and building methods that "learn" – that
is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial
intelligence.
Machine learning is a growing technology which enables computers to learn automatically from past data. Machine learning uses
various algorithms for building mathematical models and making predictions using historical data or information. Currently, it is
being used for various tasks such as image recognition, speech recognition, email filtering, Facebook auto-tagging, recommender
system, and many more.
This machine learning tutorial gives you an introduction to machine learning along with the wide range of machine learning techniques
such as Supervised, Unsupervised, and Reinforcement learning. You will learn about regression and classification models, clustering
methods, hidden Markov models, and various sequential models.
What is Regression?
❖ Define regression with example(2M,8M)
❖ Application of regression in real life(2M)
Regression allows researchers to predict or explain the variation in one variable based on another
variable.
The variable that researchers are trying to explain or predict is called the response variable. It is also
sometimes called the dependent variable because it depends on another variable.
The variable that is used to explain or predict the response variable is called the explanatory variable. It is
also sometimes called the independent variable because it is independent of the other variable.
In regression, the order of the variables is very important. The explanatory variable (or the independent
variable) always belongs on the x-axis. The response variable (or the dependent variable) always belongs on
the y-axis.
Example:
If it is already known that there is a significant correlation between students’ GPA and their self-esteem, the
next question researchers might ask is: Can students’ scores on a self-esteem scale be predicted based on
GPA? In other words, does GPA explain self-esteem? These are the types of questions that regression
responds to.
,**Note that these questions do not imply a causal relationship. In this example, GPA is the explanatory
variable (or the independent variable) and self-esteem is the response variable (or the dependent variable).
GPA belongs on the x-axis and self-esteem belongs on the y-axis.
Regression is essential for any machine learning problem that involves continuous numbers, which includes
a vast array of real-life applications:
1. Financial forecasting, such as estimating housing or stock prices
2. Automobile testing
3. Weather analysis
4. Time series forecasting
3.4.2 Types of Regression
❖ Types of regression(2M,16M)
❖ What are the three approaches in stepwise regression?(2M)
➢ Linear Regression
➢ Logistic Regression
➢ Polynomial Regression
➢ Stepwise Regression
➢ Ridge Regression
➢ Lasso Regression
➢ Elastic Net Regression
3.4.2 .1 LINEAR REGRESSION:
Simple linear regression is useful for finding relationship between two continuous variables. One is predictor
or independent variable and other is response or dependent variable. It looks for statistical relationship but
not deterministic relationship. Relationship between two variables is said to be deterministic if one variable
can be accurately expressed by the other. For example, using temperature in degree Celsius it is possible to
, accurately predict Fahrenheit. Statistical relationship is not accurate in determining relationship between two
variables. For example, relationship between height and weight.
The core idea is to obtain a line that best fits the data. The best fit line is the one for which total prediction
error (all data points) are as small as possible. Error is the distance between the point to the regression line.
Calculate the regression coefficient and obtain the lines of regression for the following data
Solution:
Regression coefficient of X on Y
DATA SCIENCE
REGULATION R2021
II YEAR - IV SEMESTER
CS3491 – ARTIFICIAL INTELLIGENCE AND
MACHINE LEARNING
III Unit
Prepared by,
Dr. S. RAVICHANDRAN
, UNIT III SUPERVISED LEARNING
Introduction to machine learning – Linear Regression Models: Least squares, single & multiple variables, Bayesian linear
regression, gradient descent,
◼ Linear Classification Models: Discriminant function – Probabilistic discriminative model - Logistic regression, Probabilistic
generative model – Naive Bayes, Maximum margin classifier – Support vector machine, Decision Tree, Random forests.
Introduction to machine learning
Machine learning is a field of inquiry devoted to understanding and building methods that "learn" – that
is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial
intelligence.
Machine learning is a growing technology which enables computers to learn automatically from past data. Machine learning uses
various algorithms for building mathematical models and making predictions using historical data or information. Currently, it is
being used for various tasks such as image recognition, speech recognition, email filtering, Facebook auto-tagging, recommender
system, and many more.
This machine learning tutorial gives you an introduction to machine learning along with the wide range of machine learning techniques
such as Supervised, Unsupervised, and Reinforcement learning. You will learn about regression and classification models, clustering
methods, hidden Markov models, and various sequential models.
What is Regression?
❖ Define regression with example(2M,8M)
❖ Application of regression in real life(2M)
Regression allows researchers to predict or explain the variation in one variable based on another
variable.
The variable that researchers are trying to explain or predict is called the response variable. It is also
sometimes called the dependent variable because it depends on another variable.
The variable that is used to explain or predict the response variable is called the explanatory variable. It is
also sometimes called the independent variable because it is independent of the other variable.
In regression, the order of the variables is very important. The explanatory variable (or the independent
variable) always belongs on the x-axis. The response variable (or the dependent variable) always belongs on
the y-axis.
Example:
If it is already known that there is a significant correlation between students’ GPA and their self-esteem, the
next question researchers might ask is: Can students’ scores on a self-esteem scale be predicted based on
GPA? In other words, does GPA explain self-esteem? These are the types of questions that regression
responds to.
,**Note that these questions do not imply a causal relationship. In this example, GPA is the explanatory
variable (or the independent variable) and self-esteem is the response variable (or the dependent variable).
GPA belongs on the x-axis and self-esteem belongs on the y-axis.
Regression is essential for any machine learning problem that involves continuous numbers, which includes
a vast array of real-life applications:
1. Financial forecasting, such as estimating housing or stock prices
2. Automobile testing
3. Weather analysis
4. Time series forecasting
3.4.2 Types of Regression
❖ Types of regression(2M,16M)
❖ What are the three approaches in stepwise regression?(2M)
➢ Linear Regression
➢ Logistic Regression
➢ Polynomial Regression
➢ Stepwise Regression
➢ Ridge Regression
➢ Lasso Regression
➢ Elastic Net Regression
3.4.2 .1 LINEAR REGRESSION:
Simple linear regression is useful for finding relationship between two continuous variables. One is predictor
or independent variable and other is response or dependent variable. It looks for statistical relationship but
not deterministic relationship. Relationship between two variables is said to be deterministic if one variable
can be accurately expressed by the other. For example, using temperature in degree Celsius it is possible to
, accurately predict Fahrenheit. Statistical relationship is not accurate in determining relationship between two
variables. For example, relationship between height and weight.
The core idea is to obtain a line that best fits the data. The best fit line is the one for which total prediction
error (all data points) are as small as possible. Error is the distance between the point to the regression line.
Calculate the regression coefficient and obtain the lines of regression for the following data
Solution:
Regression coefficient of X on Y
