ISYE 6501 - Midterm 1 Exam
ISYE 6501 - MIDTERM 1 EXAM NEWEST 2025/2026 COMPLETE
QUESTIONS AND CORRECT DETAILED ANSWERS (VERIFIED
ANSWERS) |BRAND NEW VERSION!!
What is CART?
Classification and Regression Trees
How do you perform pruning?
For every pair of leaves created by the same branch, we use the other half of the
data to see whether the estimation error is actually improved by branching. If the
branching does improve error, the branches stay, but if the branching actually
makes the error gets or not change, we move the branches
How do we build a tree?
Start with half of the data and build a regression model on it. Then, wheenver
there's a leaf we can branch from, we can calculate the variance of the response
among all data points in the leaf. We test splitting on each factor to determine
how much lower the total variance of the two branches would be compared to
the least variance and choose the factor with the lowest variance. If the decrease
in variance is more than some threshold data, and there would be enough data
points in each branch, we make the split; otherwise, we assume there's not
enough benefit to branching and the leaf remains as is
What is the idea behind random forests?
Introduce radomness. We generate many different trees. They will have different
strengths and weaknesses. The average of all these trees is better than a single
tree with specific strengths and weaknesses
How are the steps in random forests?
1|Page
, ISYE 6501 - Midterm 1 Exam
1. Introduce randomness via bootstrapping. Branching: randomly choose a small
number of factors, set X. The common number of factors to use is log(n). Choose
the best factor within X to branch on.
What is the benefit of Random Forests
It has better overall estimates. while each tree might be over-fitting in one place
or another they don't necessarily over-fit the same way. The average overall tree
tends to fall those overreaction to random effects.
What are the drawbacks of random forests?
Harder to explain/interpret results. Can't give us a specific regression or
classification model from the data.
How is the prediction calculated in Random Forests when doing regression trees?
use the average of the predicted response
How is the prediction calculated in Random Forests when doing classification?
use the mode -- the most common predicted response
what are the similarities between Logistic Regression and Linear regression
transformation of input data, consider interaction terms, variable selection, has
trees
differences
Logistic Regression takes longer to calculate, has no closed-form solution, and
difficult to understand model quality (no r-squared value)
what is sensitivity
the fraction of category members that are correctly classified TP / (TP + FN)
what is specificity
2|Page
, ISYE 6501 - Midterm 1 Exam
the fraction of non-category member that are correctly identified TN / (TN + FP)
what does the roc curve plot
sensitivity plotted against 1 - specificity
what is the Area Under Curve
probability that the model estimates a random "yes" point higher than a random
"no" point
what does it mean when the AUC = 0.5
we are just guessing
What does ROC/AUC give you and what doesn't it
gives a quick-and-dirty estimate of quality but does not differentiate between the
coset of FN and FP
what does TP mean?
point in the category, correctly classified
what does FP mean
point not in category, model says it is
what does TN mean?
point not in category, correctly classified
what does FN mean?
point in the category model says no
how do you do KNN regression?
3|Page
, ISYE 6501 - Midterm 1 Exam
plot all the data. predict response by taking average response of k closest data
points
what are parametric methods?
the form of the predictor (linear regression)
what are non-parametric methods
we don't force any specific form onto the predictor (knn)
What is a spline?
function of polynomials that connect to each other
How does regression splines work?
Fit different functions to different parts of the data set with smooth connections
between the parts.
What is the points where the different functions connect?
they are called knots
Why do connection have to be smooth?
Otherwise you could have drastically different answers for very nearby points.
How does Bayesian Regression work?
Start with data and estimate of how regression coefficients and the random error
is distributed. Then we use Bayes theorem to update estimate.
When should you use Bayesian Regression?
When there's not much data and want to combine expert opinion.
What do descriptive questions ask?
4|Page
ISYE 6501 - MIDTERM 1 EXAM NEWEST 2025/2026 COMPLETE
QUESTIONS AND CORRECT DETAILED ANSWERS (VERIFIED
ANSWERS) |BRAND NEW VERSION!!
What is CART?
Classification and Regression Trees
How do you perform pruning?
For every pair of leaves created by the same branch, we use the other half of the
data to see whether the estimation error is actually improved by branching. If the
branching does improve error, the branches stay, but if the branching actually
makes the error gets or not change, we move the branches
How do we build a tree?
Start with half of the data and build a regression model on it. Then, wheenver
there's a leaf we can branch from, we can calculate the variance of the response
among all data points in the leaf. We test splitting on each factor to determine
how much lower the total variance of the two branches would be compared to
the least variance and choose the factor with the lowest variance. If the decrease
in variance is more than some threshold data, and there would be enough data
points in each branch, we make the split; otherwise, we assume there's not
enough benefit to branching and the leaf remains as is
What is the idea behind random forests?
Introduce radomness. We generate many different trees. They will have different
strengths and weaknesses. The average of all these trees is better than a single
tree with specific strengths and weaknesses
How are the steps in random forests?
1|Page
, ISYE 6501 - Midterm 1 Exam
1. Introduce randomness via bootstrapping. Branching: randomly choose a small
number of factors, set X. The common number of factors to use is log(n). Choose
the best factor within X to branch on.
What is the benefit of Random Forests
It has better overall estimates. while each tree might be over-fitting in one place
or another they don't necessarily over-fit the same way. The average overall tree
tends to fall those overreaction to random effects.
What are the drawbacks of random forests?
Harder to explain/interpret results. Can't give us a specific regression or
classification model from the data.
How is the prediction calculated in Random Forests when doing regression trees?
use the average of the predicted response
How is the prediction calculated in Random Forests when doing classification?
use the mode -- the most common predicted response
what are the similarities between Logistic Regression and Linear regression
transformation of input data, consider interaction terms, variable selection, has
trees
differences
Logistic Regression takes longer to calculate, has no closed-form solution, and
difficult to understand model quality (no r-squared value)
what is sensitivity
the fraction of category members that are correctly classified TP / (TP + FN)
what is specificity
2|Page
, ISYE 6501 - Midterm 1 Exam
the fraction of non-category member that are correctly identified TN / (TN + FP)
what does the roc curve plot
sensitivity plotted against 1 - specificity
what is the Area Under Curve
probability that the model estimates a random "yes" point higher than a random
"no" point
what does it mean when the AUC = 0.5
we are just guessing
What does ROC/AUC give you and what doesn't it
gives a quick-and-dirty estimate of quality but does not differentiate between the
coset of FN and FP
what does TP mean?
point in the category, correctly classified
what does FP mean
point not in category, model says it is
what does TN mean?
point not in category, correctly classified
what does FN mean?
point in the category model says no
how do you do KNN regression?
3|Page
, ISYE 6501 - Midterm 1 Exam
plot all the data. predict response by taking average response of k closest data
points
what are parametric methods?
the form of the predictor (linear regression)
what are non-parametric methods
we don't force any specific form onto the predictor (knn)
What is a spline?
function of polynomials that connect to each other
How does regression splines work?
Fit different functions to different parts of the data set with smooth connections
between the parts.
What is the points where the different functions connect?
they are called knots
Why do connection have to be smooth?
Otherwise you could have drastically different answers for very nearby points.
How does Bayesian Regression work?
Start with data and estimate of how regression coefficients and the random error
is distributed. Then we use Bayes theorem to update estimate.
When should you use Bayesian Regression?
When there's not much data and want to combine expert opinion.
What do descriptive questions ask?
4|Page
