ISYE 6414 - All Units
Linearity/Mean zero assumption - Means that the expected value (deviances) of errors is zero.
This leads to difficulties in estimating B0 and means that our model does not include a necessary
systematic component
Constant variance assumption - Means that it cannot be true that the model is more accurate for
some parts of the population, and less accurate for other parts of the populations. This can result in less
accurate parameters and poorly-calibrated prediction intervals.
response (dependent) variables - one particular variable that we are interested in understanding
or modeling (y)
What kind of variable is a response variable and why? - random, because it varies with changes in
the predictor/s along with other random changes.
What kind of variable is a predicting variable and why? - fixed, because it does not change with
the response but it is fixed before the response is measured.
linear relationship - a simple deterministic relationship between 2 factors, x and y
what are three things that a regression analysis is used for? - 1. Prediction of the response
variable, 2. Modeling the relationship between the response and explanatory variables, 3. Testing
hypotheses of association relationships
B0 = ? - intercept
B1 = ? - slope
,for our linear model where: Y = B0 + B1 + EPSILON (E), what does the epsilon represent? -
deviance of the data from the linear model (error term)
what are the 4 assumptions of linear regression? - Linearity/Mean Zero, Constant Variance,
Independence, Normality
Assumption of Independence - Means that the deviances, or in fact the response variables ys, are
independently drawn from the data-generating process. (this most often occurs in time series data) This
can result in very misleading assessments of the strength of regression.
Normality assumption - This is needed if we want to do any confidence or prediction intervals, or
hypothesis test, which we usually do. If this assumption is violated, hypothesis test and confidence and
prediction intervals and be very misleading.
what are the 3 parameters we estimated in regression? - B0, B1, sigma squared (variance of the
one pop.)
What do we mean by model parameters in statistics? - Model parameters are unknown
quantities, and they stay unknown regardless how much data are observed. We estimate those
parameters given the model assumptions and the data, but through estimation, we're not identifying the
true parameters. We're just estimating approximations of those parameters.
What is the estimated sampling distribution of s^2? - chi-square with n-1 DF
Why do we lose 1 DF for s^2? - we replace mu with zbar
what is the relationship between s^2 and sigma^2? - S^2 estimates sigma^2
What is the estimated sampling distribution of sigma^2? - chi-square with n-2 DF (~ equivalent to
MSE)
Why do we lose 2 DF for sigma^2? - we replaced two parameters, B0 and B1
,In SLR, we are interested in the behavior of which parameter? - B1
If we have a positive value for B1,.... - then that's consistent with a direct relationship between
the predicting variable x and the response variable y.
If we have a negative value for B1,.... - is consistent with an inverse relationship between x and y
When B1 is close to zero... - we interpret that there is not a significant association between
predicting variables, between the predicting variable x, and the response variable y.
How do we interpret B1? - It is the estimated expected change in the response variable associated
with one unit of change in the predicting variable.
How we interpret ^B0? - It is the estimated expected value of the response variable, when the
predicting variable equals zero.
What is the sampling distribution of ^B1? - t distribution with N-2 DF
What can we use to test for statistical significance? - t-test
What would we do if the T value is large? - Reject the null hypothesis that β1 is equal to zero. If
the null hypothesis is rejected, we interpret this that β1 is statistically significant.
what does 'statistical significance' mean? - B1 is statistically different from zero.
what is the distribution of B1? - Normal
The estimators for the regression coefficients are:
A) Biased but with small variance
B) Unbiased under normality assumptions but biased otherwise.
, C) Unbiased regardless of the distribution of the data. - C
The assumption of normality:
A) It is needed for deriving the estimators of the regression coefficients.
B) It is not needed for linear regression modeling and inference.
C) It is needed for the sampling distribution of the estimators of the regression coefficients and hence for
inference.
D) It is needed for deriving the expectation and variance of the estimators of the regression coefficients.
- C
What is 'X*'? - predictor
Where does uncertainty from estimation come from? - from estimation alone
Where does uncertainty from prediction come from? - from the estimation of regression
parameters and from the newness of the observation itself
what is the prediction interval used for? - used to provide an interval estimate for a prediction of y
for one member of the population with a particular value of x*
what is the confidence interval used for? - to provide an interval estimate for the true average
value of y for all members of the population with a particular value of x*
The estimated versus predicted regression line for a given x*:
A) Have the same variance
B) Have the same expectation
C) Have the same variance and expectation
D) None of the above - B
Linearity/Mean zero assumption - Means that the expected value (deviances) of errors is zero.
This leads to difficulties in estimating B0 and means that our model does not include a necessary
systematic component
Constant variance assumption - Means that it cannot be true that the model is more accurate for
some parts of the population, and less accurate for other parts of the populations. This can result in less
accurate parameters and poorly-calibrated prediction intervals.
response (dependent) variables - one particular variable that we are interested in understanding
or modeling (y)
What kind of variable is a response variable and why? - random, because it varies with changes in
the predictor/s along with other random changes.
What kind of variable is a predicting variable and why? - fixed, because it does not change with
the response but it is fixed before the response is measured.
linear relationship - a simple deterministic relationship between 2 factors, x and y
what are three things that a regression analysis is used for? - 1. Prediction of the response
variable, 2. Modeling the relationship between the response and explanatory variables, 3. Testing
hypotheses of association relationships
B0 = ? - intercept
B1 = ? - slope
,for our linear model where: Y = B0 + B1 + EPSILON (E), what does the epsilon represent? -
deviance of the data from the linear model (error term)
what are the 4 assumptions of linear regression? - Linearity/Mean Zero, Constant Variance,
Independence, Normality
Assumption of Independence - Means that the deviances, or in fact the response variables ys, are
independently drawn from the data-generating process. (this most often occurs in time series data) This
can result in very misleading assessments of the strength of regression.
Normality assumption - This is needed if we want to do any confidence or prediction intervals, or
hypothesis test, which we usually do. If this assumption is violated, hypothesis test and confidence and
prediction intervals and be very misleading.
what are the 3 parameters we estimated in regression? - B0, B1, sigma squared (variance of the
one pop.)
What do we mean by model parameters in statistics? - Model parameters are unknown
quantities, and they stay unknown regardless how much data are observed. We estimate those
parameters given the model assumptions and the data, but through estimation, we're not identifying the
true parameters. We're just estimating approximations of those parameters.
What is the estimated sampling distribution of s^2? - chi-square with n-1 DF
Why do we lose 1 DF for s^2? - we replace mu with zbar
what is the relationship between s^2 and sigma^2? - S^2 estimates sigma^2
What is the estimated sampling distribution of sigma^2? - chi-square with n-2 DF (~ equivalent to
MSE)
Why do we lose 2 DF for sigma^2? - we replaced two parameters, B0 and B1
,In SLR, we are interested in the behavior of which parameter? - B1
If we have a positive value for B1,.... - then that's consistent with a direct relationship between
the predicting variable x and the response variable y.
If we have a negative value for B1,.... - is consistent with an inverse relationship between x and y
When B1 is close to zero... - we interpret that there is not a significant association between
predicting variables, between the predicting variable x, and the response variable y.
How do we interpret B1? - It is the estimated expected change in the response variable associated
with one unit of change in the predicting variable.
How we interpret ^B0? - It is the estimated expected value of the response variable, when the
predicting variable equals zero.
What is the sampling distribution of ^B1? - t distribution with N-2 DF
What can we use to test for statistical significance? - t-test
What would we do if the T value is large? - Reject the null hypothesis that β1 is equal to zero. If
the null hypothesis is rejected, we interpret this that β1 is statistically significant.
what does 'statistical significance' mean? - B1 is statistically different from zero.
what is the distribution of B1? - Normal
The estimators for the regression coefficients are:
A) Biased but with small variance
B) Unbiased under normality assumptions but biased otherwise.
, C) Unbiased regardless of the distribution of the data. - C
The assumption of normality:
A) It is needed for deriving the estimators of the regression coefficients.
B) It is not needed for linear regression modeling and inference.
C) It is needed for the sampling distribution of the estimators of the regression coefficients and hence for
inference.
D) It is needed for deriving the expectation and variance of the estimators of the regression coefficients.
- C
What is 'X*'? - predictor
Where does uncertainty from estimation come from? - from estimation alone
Where does uncertainty from prediction come from? - from the estimation of regression
parameters and from the newness of the observation itself
what is the prediction interval used for? - used to provide an interval estimate for a prediction of y
for one member of the population with a particular value of x*
what is the confidence interval used for? - to provide an interval estimate for the true average
value of y for all members of the population with a particular value of x*
The estimated versus predicted regression line for a given x*:
A) Have the same variance
B) Have the same expectation
C) Have the same variance and expectation
D) None of the above - B
