ISYE 6501 -EXAM 2 WKS 8 – 12
QUESTIONS & ANSWERS(RATED
A+)
Building simpler models with fewer factors helps avoid which problems?
A. Overfitting
B. Low prediction quality
C. Bias in the most important factors
D. Difficulty in interpretation - ANSWERA. Overfitting
D. Difficulty of interpretation
Two main reasons to limit # of factors in a model. - ANSWER1. Overfitting
2. Simplicity
When is overfitting likely to happen? - ANSWERWhen the number of factors is close
to the number of data points.
How does using a # of factors that is close to the number of data points cause
overfitting? - ANSWERThe model too closely fits the random efffects. It fits that data
set well, but fails to predict well on a new data set.
Three reasons simple models are better than complex models. - ANSWER1. Less
data is required
2. Less chance of including insignificant factors
3. Easier to interpret
Which of these is a key difference between stepwise regression and lasso
regression?
A. Lasso regression requires the data to first be scaled
B. Stepwise regression gives many models to choose from, while lasso gives just
one. - ANSWERA. If the data isn't scaled, the coefficients can have artificially
different orders of magnitude, which means they'll have unbalanced effects on the
lasso constraint.
Name three greedy approaches to variable selection. - ANSWER1. Forward
Selection
2. Backward Elimination
3. Stepwise Regression
Name two global approaches to variable selection. - ANSWER1. Lasso
2. Elastic Net
What does a greedy approach to variable selection mean? - ANSWERAt each step,
the model does one thing that looks best without taking future options into
consideration. A more classical approach.
, How does forward selection differ from backward elimination? - ANSWERForward
selection starts with zero factors and backward elimination starts with all factors.
What is stepwise regression? - ANSWERA combination of forward selection and
backward elimination. At each step, a variable is considered for addition or
elimination based on some prespecified criterion.
Name two approaches to use with stepwise regression. - ANSWER1. Start with all
factors
2. Start with no factors
What restriction does the lasso approach add? - ANSWERThe sum of the
coefficients can't be too large. "t"
What does the absolute value factor in Lasso do? - ANSWERHelps decrease the
number of factors with non zero coefficients.
What does the quadratic term in ridge regression do? - ANSWERShrink the
coefficient values.
It pushes them toward zero, or regularizes them (it's a shrinkage method)
What is the difference between the quadratic term in Ridge regression and the
absolute value term in Lasso? - ANSWERLasso's absolute value term makes some
coefficients equal zero. Ridge regressions quadratic term shrinks the coefficients, but
they won't equal zero.
How does lasso use the t value? - ANSWERIt uses that on the most important
coefficients and the others will be zero so those factors won't be part of the model.
What do you need to with the data whenever you're constraining the sum of
coefficients? - ANSWERScale the data.
The value of t in the lasso approach depends on what two things? - ANSWER1. The
number of variables you want.
2. The quality of the model as you allow more variables
What is the best approach to find the best value of t with lasso? - ANSWERTry lasso
with different values of t and pick the value that has the best tradeoff between
number of variables and quality of the model.
What constraint does Elastic Net add? - ANSWERElastic Net constrains the absolute
value of the coefficients and their squares.
Name two things similar about Lasso and Elastic Net. - ANSWER1. You have to
scale the data for both.
2. You have to pick the best value of t for both.
QUESTIONS & ANSWERS(RATED
A+)
Building simpler models with fewer factors helps avoid which problems?
A. Overfitting
B. Low prediction quality
C. Bias in the most important factors
D. Difficulty in interpretation - ANSWERA. Overfitting
D. Difficulty of interpretation
Two main reasons to limit # of factors in a model. - ANSWER1. Overfitting
2. Simplicity
When is overfitting likely to happen? - ANSWERWhen the number of factors is close
to the number of data points.
How does using a # of factors that is close to the number of data points cause
overfitting? - ANSWERThe model too closely fits the random efffects. It fits that data
set well, but fails to predict well on a new data set.
Three reasons simple models are better than complex models. - ANSWER1. Less
data is required
2. Less chance of including insignificant factors
3. Easier to interpret
Which of these is a key difference between stepwise regression and lasso
regression?
A. Lasso regression requires the data to first be scaled
B. Stepwise regression gives many models to choose from, while lasso gives just
one. - ANSWERA. If the data isn't scaled, the coefficients can have artificially
different orders of magnitude, which means they'll have unbalanced effects on the
lasso constraint.
Name three greedy approaches to variable selection. - ANSWER1. Forward
Selection
2. Backward Elimination
3. Stepwise Regression
Name two global approaches to variable selection. - ANSWER1. Lasso
2. Elastic Net
What does a greedy approach to variable selection mean? - ANSWERAt each step,
the model does one thing that looks best without taking future options into
consideration. A more classical approach.
, How does forward selection differ from backward elimination? - ANSWERForward
selection starts with zero factors and backward elimination starts with all factors.
What is stepwise regression? - ANSWERA combination of forward selection and
backward elimination. At each step, a variable is considered for addition or
elimination based on some prespecified criterion.
Name two approaches to use with stepwise regression. - ANSWER1. Start with all
factors
2. Start with no factors
What restriction does the lasso approach add? - ANSWERThe sum of the
coefficients can't be too large. "t"
What does the absolute value factor in Lasso do? - ANSWERHelps decrease the
number of factors with non zero coefficients.
What does the quadratic term in ridge regression do? - ANSWERShrink the
coefficient values.
It pushes them toward zero, or regularizes them (it's a shrinkage method)
What is the difference between the quadratic term in Ridge regression and the
absolute value term in Lasso? - ANSWERLasso's absolute value term makes some
coefficients equal zero. Ridge regressions quadratic term shrinks the coefficients, but
they won't equal zero.
How does lasso use the t value? - ANSWERIt uses that on the most important
coefficients and the others will be zero so those factors won't be part of the model.
What do you need to with the data whenever you're constraining the sum of
coefficients? - ANSWERScale the data.
The value of t in the lasso approach depends on what two things? - ANSWER1. The
number of variables you want.
2. The quality of the model as you allow more variables
What is the best approach to find the best value of t with lasso? - ANSWERTry lasso
with different values of t and pick the value that has the best tradeoff between
number of variables and quality of the model.
What constraint does Elastic Net add? - ANSWERElastic Net constrains the absolute
value of the coefficients and their squares.
Name two things similar about Lasso and Elastic Net. - ANSWER1. You have to
scale the data for both.
2. You have to pick the best value of t for both.
