ISYE6414 FINAL ASSESSMENT REVIEW
SHEET CERTIFICATION QUESTIONS AND
SOLUTIONS COMPLETE PREPARATION
VERIFIED
●● g-function.
Answer: We link the probability of success to the predicting variables
using the g link function. The g function is the s-shape function that
models the probability of success with respect to the predicting variables
The link function g is the log of the ratio of p over one minus p, where p
again is the probability of success
Logit function (log odds function) of the probability of success is a
linear model in the predicting variables
The probability of success is equal to the ratio between the exponential
of the linear combination of the predicting variables over 1 plus this
same exponential
●● Odds of a success.
Answer: This is the exponential of the Logit function
,●● Logistic Regression Assumptions.
Answer: Linearity: The relationship between the g of the probability of
success and the predicted variable, is a linear function.
Independence: The response binary variables are independently
observed
Logit: The logistic regression model assumes that the link function g is a
logit function
●● Linearity Assumption.
Answer: The Logit transformation of the probability of success is a
linear combination of the predicting variables. The relationship may not
be linear, however, and transformation may improve the fit
The linearity assumption can be evaluated by plotting the logit of the
success rate versus the predicting variables.
If there's a curvature or some non-linear pattern, it may be an indication
that the lack of fit may be due to the non-linearity with respect to some
of the predicting variables
●● Logistic Regression Coefficient.
,Answer: We interpret the regression coefficient beta as the log of the
odds ratio for an increase of one unit in the predicting variable
We do not interpret beta with respect to the response variable but with
respect to the odds of success
The estimators for the regression coefficients in logistic regression are
unbiased and thus the mean of the approximate normal distribution is
beta. The variance of the estimator does not have a closed form
expression
●● Model parameters.
Answer: The model parameters are the regression coefficients.
There is no additional parameter to model the variance since there's no
error term.
For P predictors, we have P + 1 regression coefficients for a model with
intercept (beta 0).
We estimate the model parameters using the maximum likelihood
estimation approach
●● Response variable.
, Answer: The response data are Bernoulli or binomial with one trial with
probability of success
●● MLE.
Answer: The resulting log-likelihood function to be maximized, is very
complicated and it is non-linear in the regression coefficients beta 0, beta
1, and beta p
MLE has good statistical properties under the assumption of a large
sample size i.e. large N
For large N, the sampling distribution of MLEs can be approximated by
a normal distribution
The least square estimation for the standard regression model is
equivalent with MLE, under the assumption of normality.
MLE is the most applied estimation approach
●● Parameter estimation.
Answer: Maximizing the log likelihood function with respect to beta0,
beta1 etc in closed (exact) form expression is not possible because the
log likelihood function is a non-linear function in the model parameters
i.e. we cannot derive the estimated regression coefficients in an exact
form
SHEET CERTIFICATION QUESTIONS AND
SOLUTIONS COMPLETE PREPARATION
VERIFIED
●● g-function.
Answer: We link the probability of success to the predicting variables
using the g link function. The g function is the s-shape function that
models the probability of success with respect to the predicting variables
The link function g is the log of the ratio of p over one minus p, where p
again is the probability of success
Logit function (log odds function) of the probability of success is a
linear model in the predicting variables
The probability of success is equal to the ratio between the exponential
of the linear combination of the predicting variables over 1 plus this
same exponential
●● Odds of a success.
Answer: This is the exponential of the Logit function
,●● Logistic Regression Assumptions.
Answer: Linearity: The relationship between the g of the probability of
success and the predicted variable, is a linear function.
Independence: The response binary variables are independently
observed
Logit: The logistic regression model assumes that the link function g is a
logit function
●● Linearity Assumption.
Answer: The Logit transformation of the probability of success is a
linear combination of the predicting variables. The relationship may not
be linear, however, and transformation may improve the fit
The linearity assumption can be evaluated by plotting the logit of the
success rate versus the predicting variables.
If there's a curvature or some non-linear pattern, it may be an indication
that the lack of fit may be due to the non-linearity with respect to some
of the predicting variables
●● Logistic Regression Coefficient.
,Answer: We interpret the regression coefficient beta as the log of the
odds ratio for an increase of one unit in the predicting variable
We do not interpret beta with respect to the response variable but with
respect to the odds of success
The estimators for the regression coefficients in logistic regression are
unbiased and thus the mean of the approximate normal distribution is
beta. The variance of the estimator does not have a closed form
expression
●● Model parameters.
Answer: The model parameters are the regression coefficients.
There is no additional parameter to model the variance since there's no
error term.
For P predictors, we have P + 1 regression coefficients for a model with
intercept (beta 0).
We estimate the model parameters using the maximum likelihood
estimation approach
●● Response variable.
, Answer: The response data are Bernoulli or binomial with one trial with
probability of success
●● MLE.
Answer: The resulting log-likelihood function to be maximized, is very
complicated and it is non-linear in the regression coefficients beta 0, beta
1, and beta p
MLE has good statistical properties under the assumption of a large
sample size i.e. large N
For large N, the sampling distribution of MLEs can be approximated by
a normal distribution
The least square estimation for the standard regression model is
equivalent with MLE, under the assumption of normality.
MLE is the most applied estimation approach
●● Parameter estimation.
Answer: Maximizing the log likelihood function with respect to beta0,
beta1 etc in closed (exact) form expression is not possible because the
log likelihood function is a non-linear function in the model parameters
i.e. we cannot derive the estimated regression coefficients in an exact
form
