ANOVA Linear regression Logistic regression
= used to test if there are differences in the mean of the DV as a = used to examine the relationship between a metric = to predict the likelihood that an event will or will not
result of different levels of the IV DV and one or more independent variables occur and assess variables that affect the occurrence of
the event
1.Objectives DV (Y): metric DV (Y): metric DV (Y): non-metric (binary, Y=1 or Y=0)
IV (X): non-metric IV (X): metric and non-metric (dummy coding!) IV (X): non-metric (dummy!) and metric
Example: Experiment where a person has to choose between
- Ad A, B Appeal (t-test) option 1 and option 2.
- Ad A, B, C Appeal (one-way ANOVA)
- (i) Coupons (Yes/No), (ii) Promo activity (Low, med,
high) Store sales (two-way ANOVA)
2.Design Type I error: p-value (a) Multiple LRM with interactions (in order to predict Y): Probability model (in order to predict the possibility that
Type II error: power (1-b) , which depends on: - Y = b0 + b1*X1 + b2*X2 + b3*X2*X3 + e Y=1):
- Effect size (large effect small sample size, small - Y = 1 with probability p
effect large sample size) Example: - Y = 0 with probability 1-p
- Sample size (under your control) - Satisfaction = b0 + b1*child_dummy + - Outcome between 0 and 1
- Alpha b2*wait + b3*child_dummy*wait + e - Model is S shaped
1.1 Sample size b = if the IV increases by 1 unit, the DV increases by…
20 observations per cel
1.2 Interaction
Effect of one IV on DV is dependent on another IV (moderator)
= two-way ANOVA
1.3 Covariates
Control variables (i.e. covariates ANCOVA) affect DV
separately from IV to prevent biased results, requirements:
- Continuous
- Pre-measure
- Independent of IV
- Limited number < 0.1* #observations – (#popul. - 1)
3.Assumptio Assumptions: Assumptions: Logistic function:
ns - Independence: are the observations independent - Independence
(affect estimates + std. errors) - Equality of variance exp (b 0+ b1∗X 1+ B2∗X 2 …)
- Equality of variance: is the variance equal across - Normality P ( event )=
treatment groups (affect std. error) - Linearity: is the relationship between the DV 1+ exp (b0 +b1∗X 1 + B2∗X 2 …)
- Normality: is the DV normally distributed (affect std. and IV linear (affect estimates + std. errors
errors ONLY if sample is small)
- Exp = linear predictor
, - Steepest in middle (is something is not very
low/high high effect
1.4 Independence 3.1 Independence - Flattest at min and max (if something is already
Plot residuals over time, no pattern = independent data What if violated? very low/high small effect)
1.5 Equality of variance (homoscedasticity) - Model process that make obs. dependent
Levene’s test: (e.g. adding covariates weather) P (event)
- H0 = equal variances - Consider more advanced models
odds= =exp (b0 +b 1∗x 1+ b2∗x 2 …)
1−P(event)
do not want to reject, so p> 0.05
- H1 = unequal variances 3.2 Equality of variance? - P = 0.5, log odds = 0.5/(1-0.5) = 1 (1 to 1
What if violated? success, 50/50)
What if p<.05? - Transform DV (e.g. natural logarithm) - P = 0.25, log odds are 0.25/(1-0.25) = 1/3 (1/3 to
- Sample size = similar across groups, it’s okay - Add covariate 1 success)
- If not: transform DV or add covariate re do test - P = 0.75, log odds are 0.75/(1-0.75) = 3 (3 to 1, 3
1.7 Normality op 4 kans)
1.6 Normality What if violated?
Kolmogorov-Smirnov test or Shapiro-wilk test: - Transform DV
- H0 = normally distributed - Add covariate
do not want to reject, so p> 0.05
- H1 = not normally distributed 1.8 Linearity
What if violated?
What if p<.05? - Transform DV or IV
- Large sample size, it’s okay - Directly include non-linear relationship in the
- Small sample size, transform DV to make distribution model
more symmetric
4.Estimating Key question: is signal (between groups) > noise (within 1.9 Estimation 4.1 Estimation
the model groups)? Ordinary Least Squares (OLS): find b’s that can best
F needs to be > critical F predict the DV such that the squared difference Likelihood: Choose B to maximize L (chance of occurring
between the DV and estimated DV is as small as is highest)
1. Calculate average level of all groups ^ ( y− ^y )2
possible b=
2. Check to what extent the averages differ from overall 4.2 Model fit – important part!
average 1.10 Model fit
3. Calculate SSB and MSSB - F-test: Test if model is significant Measurers:
4. Calculate differences within groups MSSW - R2 (% of variance explained in DV by the - Likelihood value Chi-square <0.05 is sign.
5. MSSB / MSSW = F - statistic ssr - Pseudo R2 / Nagelkerke R2 (cutoff time series
2
model) = r =
sst data 0.7/0.8 and cross-sectional 0.2/0.3)
- Will never decrease if you add more - Classification Table (hit rate)
variables - Homer-Lemeshow
- Adjusted R2 : used to compare models
(highest = best model) Hosmer Lemeshow: to test goodness of fit
- Ho: predicted numbers = observed numbers
= used to test if there are differences in the mean of the DV as a = used to examine the relationship between a metric = to predict the likelihood that an event will or will not
result of different levels of the IV DV and one or more independent variables occur and assess variables that affect the occurrence of
the event
1.Objectives DV (Y): metric DV (Y): metric DV (Y): non-metric (binary, Y=1 or Y=0)
IV (X): non-metric IV (X): metric and non-metric (dummy coding!) IV (X): non-metric (dummy!) and metric
Example: Experiment where a person has to choose between
- Ad A, B Appeal (t-test) option 1 and option 2.
- Ad A, B, C Appeal (one-way ANOVA)
- (i) Coupons (Yes/No), (ii) Promo activity (Low, med,
high) Store sales (two-way ANOVA)
2.Design Type I error: p-value (a) Multiple LRM with interactions (in order to predict Y): Probability model (in order to predict the possibility that
Type II error: power (1-b) , which depends on: - Y = b0 + b1*X1 + b2*X2 + b3*X2*X3 + e Y=1):
- Effect size (large effect small sample size, small - Y = 1 with probability p
effect large sample size) Example: - Y = 0 with probability 1-p
- Sample size (under your control) - Satisfaction = b0 + b1*child_dummy + - Outcome between 0 and 1
- Alpha b2*wait + b3*child_dummy*wait + e - Model is S shaped
1.1 Sample size b = if the IV increases by 1 unit, the DV increases by…
20 observations per cel
1.2 Interaction
Effect of one IV on DV is dependent on another IV (moderator)
= two-way ANOVA
1.3 Covariates
Control variables (i.e. covariates ANCOVA) affect DV
separately from IV to prevent biased results, requirements:
- Continuous
- Pre-measure
- Independent of IV
- Limited number < 0.1* #observations – (#popul. - 1)
3.Assumptio Assumptions: Assumptions: Logistic function:
ns - Independence: are the observations independent - Independence
(affect estimates + std. errors) - Equality of variance exp (b 0+ b1∗X 1+ B2∗X 2 …)
- Equality of variance: is the variance equal across - Normality P ( event )=
treatment groups (affect std. error) - Linearity: is the relationship between the DV 1+ exp (b0 +b1∗X 1 + B2∗X 2 …)
- Normality: is the DV normally distributed (affect std. and IV linear (affect estimates + std. errors
errors ONLY if sample is small)
- Exp = linear predictor
, - Steepest in middle (is something is not very
low/high high effect
1.4 Independence 3.1 Independence - Flattest at min and max (if something is already
Plot residuals over time, no pattern = independent data What if violated? very low/high small effect)
1.5 Equality of variance (homoscedasticity) - Model process that make obs. dependent
Levene’s test: (e.g. adding covariates weather) P (event)
- H0 = equal variances - Consider more advanced models
odds= =exp (b0 +b 1∗x 1+ b2∗x 2 …)
1−P(event)
do not want to reject, so p> 0.05
- H1 = unequal variances 3.2 Equality of variance? - P = 0.5, log odds = 0.5/(1-0.5) = 1 (1 to 1
What if violated? success, 50/50)
What if p<.05? - Transform DV (e.g. natural logarithm) - P = 0.25, log odds are 0.25/(1-0.25) = 1/3 (1/3 to
- Sample size = similar across groups, it’s okay - Add covariate 1 success)
- If not: transform DV or add covariate re do test - P = 0.75, log odds are 0.75/(1-0.75) = 3 (3 to 1, 3
1.7 Normality op 4 kans)
1.6 Normality What if violated?
Kolmogorov-Smirnov test or Shapiro-wilk test: - Transform DV
- H0 = normally distributed - Add covariate
do not want to reject, so p> 0.05
- H1 = not normally distributed 1.8 Linearity
What if violated?
What if p<.05? - Transform DV or IV
- Large sample size, it’s okay - Directly include non-linear relationship in the
- Small sample size, transform DV to make distribution model
more symmetric
4.Estimating Key question: is signal (between groups) > noise (within 1.9 Estimation 4.1 Estimation
the model groups)? Ordinary Least Squares (OLS): find b’s that can best
F needs to be > critical F predict the DV such that the squared difference Likelihood: Choose B to maximize L (chance of occurring
between the DV and estimated DV is as small as is highest)
1. Calculate average level of all groups ^ ( y− ^y )2
possible b=
2. Check to what extent the averages differ from overall 4.2 Model fit – important part!
average 1.10 Model fit
3. Calculate SSB and MSSB - F-test: Test if model is significant Measurers:
4. Calculate differences within groups MSSW - R2 (% of variance explained in DV by the - Likelihood value Chi-square <0.05 is sign.
5. MSSB / MSSW = F - statistic ssr - Pseudo R2 / Nagelkerke R2 (cutoff time series
2
model) = r =
sst data 0.7/0.8 and cross-sectional 0.2/0.3)
- Will never decrease if you add more - Classification Table (hit rate)
variables - Homer-Lemeshow
- Adjusted R2 : used to compare models
(highest = best model) Hosmer Lemeshow: to test goodness of fit
- Ho: predicted numbers = observed numbers
