4/15/26, 3:48 PM ISYE 6501 Final Flashcards | Quizlet
Social Science Economics Econometrics
ISYE 6501 Final
14 studiers today 4.9 (10 reviews)
Save
Students also studied
Flashcard sets Study guides
ISYE 6501 - Midterm 2 Isye 6501 Final exam D293 - Assessment and Learning An... ISYE 65
160 terms 323 terms Teacher 92 terms 254 term
oanise Preview Kayla_Patten2 Preview jasmine_mack3 Preview oan
Terms in this set (92)
Factor Based Models classification, clustering, regression. Implicitly assumed that we have a lot of
factors in the final model
Why limit number of factors in a model? 2 reasons overfitting: when # of factors is close to or larger than # of data points. Model
may fit too closely to random effects
simplicity: simple models are usually better
Classical variable selection approaches 1. Forward selection
2. Backwards elimination
3. Stepwise regression
greedy algorithms
Backward elimination variable selection; classical
Opposite of forward selection. Start with model with all factors, at each step find
worst factor and remove from model. Continue until no more to add, # of factor
threshold is satisfied. Remove factors at the end that were not good enough
Forward selection variable selection; classical
Start with model with no factors, at each step find best new factor to add.
Continue until none bad enough to remove, # of factor threshold is satisfied.
Remove factors at the end that were not good enough
Stepwise regression variable selection; classical
Combination of forward selection and backwards elimination. Start with all or no
factors. Each step remove/add a factor. As it continues, after adding in new factor
we eliminate right away any factors that may be good. Helps model adjust when
new factors are added, goodness values change
https://quizlet.com/458609056/isye-6501-final-flash-cards/ 1/10
, 4/15/26, 3:48 PM ISYE 6501 Final Flashcards | Quizlet
Ways of determining if factors are good enough in p-value, Rsquared, AIC, BIC
variable selection
Greedy algorithm At each step, it does the one thing that looks best
without taking future options into consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
Global variable selection approaches 1. LASSO
2. Elastic Net
Slower, but tend to give better predictive models
LASSO variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get. Budget will
be used on most important coefficients
- Method for limiting the number of variables in a model by limiting the sum of all
coefficients’ absolute values. Can be very helpful when number of data points is
less than number of factors.
Elastic Net variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get. Budget will
be used on most important coefficients
- Combination of lasso and ridge regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge regression
Ridge Regression - Method of regularization by limiting the sum of the squares of the coefficients.
Will reduce the magnitude of coefficients, not the number of variables chosen.
- The quadratic term in ridge regression
tends to shrink the coefficient values i.e Whatever the basic regression model
coefficients would be,
the quadratic constraint pushes them toward zero
or regularizes them.
https://quizlet.com/458609056/isye-6501-final-flash-cards/ 2/10
Social Science Economics Econometrics
ISYE 6501 Final
14 studiers today 4.9 (10 reviews)
Save
Students also studied
Flashcard sets Study guides
ISYE 6501 - Midterm 2 Isye 6501 Final exam D293 - Assessment and Learning An... ISYE 65
160 terms 323 terms Teacher 92 terms 254 term
oanise Preview Kayla_Patten2 Preview jasmine_mack3 Preview oan
Terms in this set (92)
Factor Based Models classification, clustering, regression. Implicitly assumed that we have a lot of
factors in the final model
Why limit number of factors in a model? 2 reasons overfitting: when # of factors is close to or larger than # of data points. Model
may fit too closely to random effects
simplicity: simple models are usually better
Classical variable selection approaches 1. Forward selection
2. Backwards elimination
3. Stepwise regression
greedy algorithms
Backward elimination variable selection; classical
Opposite of forward selection. Start with model with all factors, at each step find
worst factor and remove from model. Continue until no more to add, # of factor
threshold is satisfied. Remove factors at the end that were not good enough
Forward selection variable selection; classical
Start with model with no factors, at each step find best new factor to add.
Continue until none bad enough to remove, # of factor threshold is satisfied.
Remove factors at the end that were not good enough
Stepwise regression variable selection; classical
Combination of forward selection and backwards elimination. Start with all or no
factors. Each step remove/add a factor. As it continues, after adding in new factor
we eliminate right away any factors that may be good. Helps model adjust when
new factors are added, goodness values change
https://quizlet.com/458609056/isye-6501-final-flash-cards/ 1/10
, 4/15/26, 3:48 PM ISYE 6501 Final Flashcards | Quizlet
Ways of determining if factors are good enough in p-value, Rsquared, AIC, BIC
variable selection
Greedy algorithm At each step, it does the one thing that looks best
without taking future options into consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
Global variable selection approaches 1. LASSO
2. Elastic Net
Slower, but tend to give better predictive models
LASSO variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get. Budget will
be used on most important coefficients
- Method for limiting the number of variables in a model by limiting the sum of all
coefficients’ absolute values. Can be very helpful when number of data points is
less than number of factors.
Elastic Net variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get. Budget will
be used on most important coefficients
- Combination of lasso and ridge regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge regression
Ridge Regression - Method of regularization by limiting the sum of the squares of the coefficients.
Will reduce the magnitude of coefficients, not the number of variables chosen.
- The quadratic term in ridge regression
tends to shrink the coefficient values i.e Whatever the basic regression model
coefficients would be,
the quadratic constraint pushes them toward zero
or regularizes them.
https://quizlet.com/458609056/isye-6501-final-flash-cards/ 2/10
