ONE WAY ANOVA (Week 1)
• Purpose: Used to determine if there are any statistically significant differences between the means of three or more
independent groups.
• Number of Variables: Involves one independent variable (factor) with multiple levels.
• Example: Comparing the average scores of students in three different teaching methods.
• Use One-Way ANOVA when comparing means across three or more independent groups.
Steps Analysis
a) Nature of the variables (categorical PV with more than 2 groups, quantitative OV, scales)
b) Statistical assumptions:
1. Is the Data 1. Variance is homogenous across groups → Levenes test
suited for one way
ANOVA?
• ➡ Levene (df1, df2) = … p = … → equal / unequal variances
• If unequal variances use Brown-Forsythe table
2. Groups are roughly equal sized
3. Every subject can only be in one group
4. Residuals are normally distributed
a) F-test (significance of means differences)
F-value tell us how much more explained variance we have in our model compared to unexplained
variance, e.g. if F=11, we have 11 times more explained over unexplained variance, higher better
F-test hypothesis: (make sure you state H0 and H1 according to what is expected in the question)
• H0: μ1 = μ2 = ... = μi → There is no difference in mean scores across the different categories
(groups)
• H1: μi ≠ μj for some i and j → There is at least one difference in the mean scores of the
categories
Formula F-test:
2. ANOVA: Are
there differences
between means?
➡ F(dfbetween, dftotal)= …, p = … → There is (no) significant difference in mean OV scores between
the PV groups (a, b, c).
b) R2 (how much variance is explained by the model)
Formula R2
➡ R2 = …, The model explains …% of all variance in the OV.
a) Multiple comparison / Post Hoc Test (Bonferroni) (Between which groups are the differences)
3. Between which ➡ Mean Difference (a, b) = …, CI (… to …), p = …, → The groups a and b differ significantly in
groups are the mean OV scores. On the OV scale (…), the group a has a OV score that is on average … units
differences? higher than the group b.
b) Planned contrast (explores all possible pairwise comparisons)
,FACTORIAL ANOVA (Week 2)
• Purpose: Extends the One-Way ANOVA to analyze the effects of multiple independent variables (factors) and their
interactions on the dependent variable.
• Number of Variables: Involves two or more independent variables.
• Example: Examining the impact of both teaching method and gender on student scores.
• Use Factorial ANOVA when examining the effects of multiple independent variables and their interactions on the
dependent variable.
Steps Analysis
a) Nature of the variables (multiple categorical PV with more than 2 groups, quantitative OV,
independent samples / scales)
b) Statistical assumptions:
1. Is the Data suited 1. Variance is homogenous across groups → Levenes test
for factorialANOVA?
• ➡ Levene (df1, df2) = … p = … → equal / unequal variances
• If unequal variances use Brown-Forsythe table
2. Groups are roughly equal sized
3. Every subject can only be in one group
4. Residuals are normally distributed
a) Total Model F-test (significance of model)
F-test for entire model: (state H0 and H1 according to what is expected in the question)
• H0: None of the F-tests of the two PVs are significant / The different categories in the PV will
lead to similar levels in the OV
• H1: at least one of the F-tests of the PVs is significant / The different categories in the PV will
lead to varying levels in the OV
Hypothesis for interaction effect:
2. Does the total
• H0: The effect of the PV on the OV is not moderated by MedV
model explain
• H1: The effect of the PV on the OV is moderated by MedV, such that… (here specify according
variance in OV?
to how it is stated in the question
➡ F(dfbetween, dftotal)= …, p = … → At least one of the PVs or their interaction has a significant effect
on the OV. The variance of the OV is … times more explained with the model than with the
residuals. Further analysis is needed to specify.
b) R2 (how much variance is explained by the model)
➡ R2 = …, The model explains …% of all variance in the OV.
a) Main effects F-test (significance of individual PVs)
F-test for main effect:
• H0: μ1 = μ2 = ... = μi → There is no difference in mean scores across the different categories
(groups)
• H1: μi ≠ μj for some i and j. → There is at least one difference in the mean scores of the
categories (groups)
(Check for p-value, to see if there is a
significant main or interaction effect
3. Are there single ➡FPV(dfbetween, dftotal)= …, p = … → there are (no) significant differences in the mean scores of the
main or interaction OV for either group of the PV. No/significant main effect.
effects?
After observing a significant main effect in our model, we conduct a follow up test, where we
compare average scores of the individual groups, within the respective PV, against each other, to
see if their mean difference is significantly different from zero.
, b) Interaction effect F-test (significance of interaction)
F-test for interaction effect:
• H0: There is no significant interaction (effect size = zero)
• H1: There is significant interaction (effect size ≠ 0)
➡FInteraction (dfbetween, dftotal)= …, p = … → there is (no) significant interaction, the effect of PV on OV
is moderated by Moderator
After observing a significant interaction effect in our model, conduct a simple effects test, where we
compare the average scores of the individual groups/levels of one PV within each level of another
PV
a) Partial η2 (measure for effect size: tells you the proportion of variance that is associated with
the factor(s) being examined, after accounting for other factors in the model)
4. How big are the
effect sizes?
➡ η2 = SStreatment / (SStreatment + SSR) = … → According to the rule of thumb, the effect sizes is:
• 0,001 < small < 0,006
• 0,006 < medium > 0,014
• > 0,014 = large
a) Post Hoc tests / significant main effects (between which groups are the differences)
➡ Mean Difference (a1, a2) = …, CI (… to …), p = …, → The groups a1 and a2 differ significantly
in mean OV scores. On the OV scale (…), the group a1 has a OV score that is on average …
units higher than the group a2
5. Follow up analysis b) Simple effects test (between which groups of each PV is significant interaction)
(To determine Simple Effects Test is a post-hoc test which compares the mean scores of all different levels of
significance of effects) one PV WITHIN each level of another PV. It allows you to explore the differences between groups
at each level of a factor, helping to understand the nature of the interaction
➡ Mean Difference (b1 * a1, a2) = …, CI (… to …), p = …, → The groups a1 and a2 differ
significantly in mean OV scores given the condition of b1. On the OV scale (…), the group a1 has a
OV score that is on average … units higher than the group a2.
, MULTIPLE REGRESSION (Week 3)
• Purpose: Used to model the relationship between a dependent variable and one or more independent variables.
• Number of Variables: Involves at least one dependent variable and one or more independent variables.
• Example: Predicting a student's test score based on the number of hours spent studying and the number of classes
attended.
• Use Regression Analysis when exploring relationships between variables and making predictions.
• ß-coefficient: is a measure of the strength and direction of the relationship between an independent variable and
thedependent variable
Steps Analysis
a) Nature of the variables (quantitative OV, categorical PV, scale)
1. Is the Data b) Statistical assumptions:
suited for
Regression? e.g. Multicollinearity
➡ For all PVS: tolerance > 0,02, so automatically VIF < 5 → Therefore, the PVs are not strongly
related, the regression model is okay and there are no adaptions necessary
a) F-test (significance: ratio of explained to unexplained variance )
F-test for whole model:
2. Does the model • H0: β1 = β2 = ... = βk = 0
as a whole • H1: at least one ß ≠ 0
significantly
explain our ➡ With F(dfPV,dferror) = …, p = … → significant, The model significantly explains/does not explain
outcome? variance in the OV
b) R2 (how much variance is explained by the model)
➡ R2 = …, The model explains …% of all variance in the OV.
a) Partial F-test (significance of difference in expanded model)
If p-value of the 2nd model is p<0,05, the second model adds explanatory power over and above
the PVs in the first mode
➡ With partial F(dfPV,dferror) = …, p = … → model is significant
3. Comparing b) Change in R2 or adj R2 (how much more variance do we explain in our expanded model)
models
F-test for expanded model comparison:
• H0: the added PVs do not significantly explain more variance
• H1: at least one added PV significantly explains more variance
➡ R2 change = … → small/medium/large increase/decrease, therefore the expanded model with the
added PV(s) does explain more variance
a) T-tests for individual PVs
Hypothesis for t-test:
• H0: β= 0: the slope (ß-coefficent) of the relationship between PV and OV is 0.
4. Do individual • H1: ß ≠ 0: the slope (ß-coefficent) of the relationship between PV and OV is not zero.
PVs have a
significant impact? ➡ The following variables have a significant effect: betaPV = …, CI (… to …), p = …
• Purpose: Used to determine if there are any statistically significant differences between the means of three or more
independent groups.
• Number of Variables: Involves one independent variable (factor) with multiple levels.
• Example: Comparing the average scores of students in three different teaching methods.
• Use One-Way ANOVA when comparing means across three or more independent groups.
Steps Analysis
a) Nature of the variables (categorical PV with more than 2 groups, quantitative OV, scales)
b) Statistical assumptions:
1. Is the Data 1. Variance is homogenous across groups → Levenes test
suited for one way
ANOVA?
• ➡ Levene (df1, df2) = … p = … → equal / unequal variances
• If unequal variances use Brown-Forsythe table
2. Groups are roughly equal sized
3. Every subject can only be in one group
4. Residuals are normally distributed
a) F-test (significance of means differences)
F-value tell us how much more explained variance we have in our model compared to unexplained
variance, e.g. if F=11, we have 11 times more explained over unexplained variance, higher better
F-test hypothesis: (make sure you state H0 and H1 according to what is expected in the question)
• H0: μ1 = μ2 = ... = μi → There is no difference in mean scores across the different categories
(groups)
• H1: μi ≠ μj for some i and j → There is at least one difference in the mean scores of the
categories
Formula F-test:
2. ANOVA: Are
there differences
between means?
➡ F(dfbetween, dftotal)= …, p = … → There is (no) significant difference in mean OV scores between
the PV groups (a, b, c).
b) R2 (how much variance is explained by the model)
Formula R2
➡ R2 = …, The model explains …% of all variance in the OV.
a) Multiple comparison / Post Hoc Test (Bonferroni) (Between which groups are the differences)
3. Between which ➡ Mean Difference (a, b) = …, CI (… to …), p = …, → The groups a and b differ significantly in
groups are the mean OV scores. On the OV scale (…), the group a has a OV score that is on average … units
differences? higher than the group b.
b) Planned contrast (explores all possible pairwise comparisons)
,FACTORIAL ANOVA (Week 2)
• Purpose: Extends the One-Way ANOVA to analyze the effects of multiple independent variables (factors) and their
interactions on the dependent variable.
• Number of Variables: Involves two or more independent variables.
• Example: Examining the impact of both teaching method and gender on student scores.
• Use Factorial ANOVA when examining the effects of multiple independent variables and their interactions on the
dependent variable.
Steps Analysis
a) Nature of the variables (multiple categorical PV with more than 2 groups, quantitative OV,
independent samples / scales)
b) Statistical assumptions:
1. Is the Data suited 1. Variance is homogenous across groups → Levenes test
for factorialANOVA?
• ➡ Levene (df1, df2) = … p = … → equal / unequal variances
• If unequal variances use Brown-Forsythe table
2. Groups are roughly equal sized
3. Every subject can only be in one group
4. Residuals are normally distributed
a) Total Model F-test (significance of model)
F-test for entire model: (state H0 and H1 according to what is expected in the question)
• H0: None of the F-tests of the two PVs are significant / The different categories in the PV will
lead to similar levels in the OV
• H1: at least one of the F-tests of the PVs is significant / The different categories in the PV will
lead to varying levels in the OV
Hypothesis for interaction effect:
2. Does the total
• H0: The effect of the PV on the OV is not moderated by MedV
model explain
• H1: The effect of the PV on the OV is moderated by MedV, such that… (here specify according
variance in OV?
to how it is stated in the question
➡ F(dfbetween, dftotal)= …, p = … → At least one of the PVs or their interaction has a significant effect
on the OV. The variance of the OV is … times more explained with the model than with the
residuals. Further analysis is needed to specify.
b) R2 (how much variance is explained by the model)
➡ R2 = …, The model explains …% of all variance in the OV.
a) Main effects F-test (significance of individual PVs)
F-test for main effect:
• H0: μ1 = μ2 = ... = μi → There is no difference in mean scores across the different categories
(groups)
• H1: μi ≠ μj for some i and j. → There is at least one difference in the mean scores of the
categories (groups)
(Check for p-value, to see if there is a
significant main or interaction effect
3. Are there single ➡FPV(dfbetween, dftotal)= …, p = … → there are (no) significant differences in the mean scores of the
main or interaction OV for either group of the PV. No/significant main effect.
effects?
After observing a significant main effect in our model, we conduct a follow up test, where we
compare average scores of the individual groups, within the respective PV, against each other, to
see if their mean difference is significantly different from zero.
, b) Interaction effect F-test (significance of interaction)
F-test for interaction effect:
• H0: There is no significant interaction (effect size = zero)
• H1: There is significant interaction (effect size ≠ 0)
➡FInteraction (dfbetween, dftotal)= …, p = … → there is (no) significant interaction, the effect of PV on OV
is moderated by Moderator
After observing a significant interaction effect in our model, conduct a simple effects test, where we
compare the average scores of the individual groups/levels of one PV within each level of another
PV
a) Partial η2 (measure for effect size: tells you the proportion of variance that is associated with
the factor(s) being examined, after accounting for other factors in the model)
4. How big are the
effect sizes?
➡ η2 = SStreatment / (SStreatment + SSR) = … → According to the rule of thumb, the effect sizes is:
• 0,001 < small < 0,006
• 0,006 < medium > 0,014
• > 0,014 = large
a) Post Hoc tests / significant main effects (between which groups are the differences)
➡ Mean Difference (a1, a2) = …, CI (… to …), p = …, → The groups a1 and a2 differ significantly
in mean OV scores. On the OV scale (…), the group a1 has a OV score that is on average …
units higher than the group a2
5. Follow up analysis b) Simple effects test (between which groups of each PV is significant interaction)
(To determine Simple Effects Test is a post-hoc test which compares the mean scores of all different levels of
significance of effects) one PV WITHIN each level of another PV. It allows you to explore the differences between groups
at each level of a factor, helping to understand the nature of the interaction
➡ Mean Difference (b1 * a1, a2) = …, CI (… to …), p = …, → The groups a1 and a2 differ
significantly in mean OV scores given the condition of b1. On the OV scale (…), the group a1 has a
OV score that is on average … units higher than the group a2.
, MULTIPLE REGRESSION (Week 3)
• Purpose: Used to model the relationship between a dependent variable and one or more independent variables.
• Number of Variables: Involves at least one dependent variable and one or more independent variables.
• Example: Predicting a student's test score based on the number of hours spent studying and the number of classes
attended.
• Use Regression Analysis when exploring relationships between variables and making predictions.
• ß-coefficient: is a measure of the strength and direction of the relationship between an independent variable and
thedependent variable
Steps Analysis
a) Nature of the variables (quantitative OV, categorical PV, scale)
1. Is the Data b) Statistical assumptions:
suited for
Regression? e.g. Multicollinearity
➡ For all PVS: tolerance > 0,02, so automatically VIF < 5 → Therefore, the PVs are not strongly
related, the regression model is okay and there are no adaptions necessary
a) F-test (significance: ratio of explained to unexplained variance )
F-test for whole model:
2. Does the model • H0: β1 = β2 = ... = βk = 0
as a whole • H1: at least one ß ≠ 0
significantly
explain our ➡ With F(dfPV,dferror) = …, p = … → significant, The model significantly explains/does not explain
outcome? variance in the OV
b) R2 (how much variance is explained by the model)
➡ R2 = …, The model explains …% of all variance in the OV.
a) Partial F-test (significance of difference in expanded model)
If p-value of the 2nd model is p<0,05, the second model adds explanatory power over and above
the PVs in the first mode
➡ With partial F(dfPV,dferror) = …, p = … → model is significant
3. Comparing b) Change in R2 or adj R2 (how much more variance do we explain in our expanded model)
models
F-test for expanded model comparison:
• H0: the added PVs do not significantly explain more variance
• H1: at least one added PV significantly explains more variance
➡ R2 change = … → small/medium/large increase/decrease, therefore the expanded model with the
added PV(s) does explain more variance
a) T-tests for individual PVs
Hypothesis for t-test:
• H0: β= 0: the slope (ß-coefficent) of the relationship between PV and OV is 0.
4. Do individual • H1: ß ≠ 0: the slope (ß-coefficent) of the relationship between PV and OV is not zero.
PVs have a
significant impact? ➡ The following variables have a significant effect: betaPV = …, CI (… to …), p = …
