ISYE 6501 MIDTERM 2 EXAM| Questions
and Answers | 2026 Update | 100% Correct.-GT
SECTION 1: VARIABLE SELECTION &
REGULARIZATION (Questions 1-14)
Question 1
When might overfitting occur in a regression model?
A. When the number of factors is much smaller than the number of data
points
B. When the number of factors is close to or larger than the number of data
points
C. When using simple models with few predictors
D. When all predictors are statistically significant
Correct Answer: B. When the number of factors is close to or larger than the
number of data points
Rationale: Overfitting occurs when the model has too many factors relative
to the number of data points, causing it to fit too closely to random effects in
the training data rather than capturing the true underlying patterns . This
results in poor generalization to new data.
Subtopic: Variable Selection - Overfitting
Question 2
Why are simple models generally better than complex ones? (Select all that
apply)
A. Less data is required for fitting
B. Less chance of including insignificant factors
,C. Easier to interpret
D. They always have higher R² values
Correct Answer: A, B, C
Rationale: Simple models require less data, have a lower chance of including
insignificant factors (reducing overfitting risk), and are easier to interpret.
Complex models may have higher training R² but often overfit and generalize
poorly .
Subtopic: Model Complexity
Question 3
What is forward selection in variable selection?
A. Start with all factors, then remove the worst ones one at a time
B. Start with no factors, add the best new factor at each step if it improves
the model
C. Randomly select a subset of factors and test performance
D. Use regularization to shrink coefficients to zero
Correct Answer: B. Start with no factors, add the best new factor at each
step if it improves the model
Rationale: Forward selection begins with an empty model (no factors). At
each step, it evaluates each candidate factor not yet in the model, selects
the one that improves the model most (based on metrics like p-value, R²,
AIC, or BIC), and adds it if it meets the threshold. The process stops when no
factor meets the inclusion criteria .
Subtopic: Variable Selection - Forward Selection
Question 4
What is backward elimination?
A. Start with no factors, add factors one at a time
B. Start with all factors, remove the worst factor at each step
C. Randomly eliminate factors and compare performance
D. Use cross-validation to select factors
, Correct Answer: B. Start with all factors, remove the worst factor at each
step
Rationale: Backward elimination begins with a model containing all
candidate factors. At each step, it identifies the "worst" factor (largest p-
value, smallest contribution) and removes it if it fails to meet a threshold
(e.g., p > 0.15). This process continues until all remaining factors meet the
retention criteria .
Subtopic: Variable Selection - Backward Elimination
Question 5
What is stepwise regression?
A. A single-pass variable selection method
B. A combination of forward selection and backward elimination
C. A regularization method using L1 penalty
D. A method that only removes variables
Correct Answer: B. A combination of forward selection and backward
elimination
Rationale: Stepwise regression is a greedy algorithm that combines forward
selection and backward elimination. It can start with no factors or all factors,
and at each step, it may add or remove a factor. After adding a new factor, it
checks whether previously added factors are still significant and may
eliminate them immediately. This allows the model to adjust if a factor is no
longer needed after new factors are added .
Subtopic: Variable Selection - Stepwise Regression
Question 6
What type of algorithms are forward selection, backward elimination, and
stepwise regression?
A. Exact optimization algorithms
B. Greedy algorithms
and Answers | 2026 Update | 100% Correct.-GT
SECTION 1: VARIABLE SELECTION &
REGULARIZATION (Questions 1-14)
Question 1
When might overfitting occur in a regression model?
A. When the number of factors is much smaller than the number of data
points
B. When the number of factors is close to or larger than the number of data
points
C. When using simple models with few predictors
D. When all predictors are statistically significant
Correct Answer: B. When the number of factors is close to or larger than the
number of data points
Rationale: Overfitting occurs when the model has too many factors relative
to the number of data points, causing it to fit too closely to random effects in
the training data rather than capturing the true underlying patterns . This
results in poor generalization to new data.
Subtopic: Variable Selection - Overfitting
Question 2
Why are simple models generally better than complex ones? (Select all that
apply)
A. Less data is required for fitting
B. Less chance of including insignificant factors
,C. Easier to interpret
D. They always have higher R² values
Correct Answer: A, B, C
Rationale: Simple models require less data, have a lower chance of including
insignificant factors (reducing overfitting risk), and are easier to interpret.
Complex models may have higher training R² but often overfit and generalize
poorly .
Subtopic: Model Complexity
Question 3
What is forward selection in variable selection?
A. Start with all factors, then remove the worst ones one at a time
B. Start with no factors, add the best new factor at each step if it improves
the model
C. Randomly select a subset of factors and test performance
D. Use regularization to shrink coefficients to zero
Correct Answer: B. Start with no factors, add the best new factor at each
step if it improves the model
Rationale: Forward selection begins with an empty model (no factors). At
each step, it evaluates each candidate factor not yet in the model, selects
the one that improves the model most (based on metrics like p-value, R²,
AIC, or BIC), and adds it if it meets the threshold. The process stops when no
factor meets the inclusion criteria .
Subtopic: Variable Selection - Forward Selection
Question 4
What is backward elimination?
A. Start with no factors, add factors one at a time
B. Start with all factors, remove the worst factor at each step
C. Randomly eliminate factors and compare performance
D. Use cross-validation to select factors
, Correct Answer: B. Start with all factors, remove the worst factor at each
step
Rationale: Backward elimination begins with a model containing all
candidate factors. At each step, it identifies the "worst" factor (largest p-
value, smallest contribution) and removes it if it fails to meet a threshold
(e.g., p > 0.15). This process continues until all remaining factors meet the
retention criteria .
Subtopic: Variable Selection - Backward Elimination
Question 5
What is stepwise regression?
A. A single-pass variable selection method
B. A combination of forward selection and backward elimination
C. A regularization method using L1 penalty
D. A method that only removes variables
Correct Answer: B. A combination of forward selection and backward
elimination
Rationale: Stepwise regression is a greedy algorithm that combines forward
selection and backward elimination. It can start with no factors or all factors,
and at each step, it may add or remove a factor. After adding a new factor, it
checks whether previously added factors are still significant and may
eliminate them immediately. This allows the model to adjust if a factor is no
longer needed after new factors are added .
Subtopic: Variable Selection - Stepwise Regression
Question 6
What type of algorithms are forward selection, backward elimination, and
stepwise regression?
A. Exact optimization algorithms
B. Greedy algorithms
