Statistics lectures 1-5
L1: Significance and Power
Null Hypothesis Significance Testing
Basic rules of significance testing:
Type I and Type II errors
Power
What affects power?
Effect size
No. of participants
Size of alpha
Problems with alpha testing
Why does this matter?
One and two-tailed tests
How this impacts data interpretation
Why does our p value change?
Which test should you use
Summary
L2: ANOVA Independent Groups
What is ANOVA?
Basic principles
Issues with multiple t-tests
Assumptions of ANOVA
Basis of the test
Between-groups ANOVA
Variance formula
Degrees of freedom
Pair-wise comparisons
Versions of ANOVA
ANOVA
MANOVA
ANCOVA
SPSS example
Worked example
Start with a plot
ANOVA in SPSS
Post-hoc tests
Glossary
L3: ANOVA Repeated Measures
Between groups vs. repeated measures
Statistics lectures 1-5 1
, F-ratio
Multi-factorial ANOVA
2x2 data examples
2x2 ANOVA
More factors
SPSS
Worked example
Possible outcomes
Plotting the data
Running the ANOVA
Output window
Reporting results
Summary
L4: Analysing Categorical Data
Chi-square test ( χ2 )
Contingency Table
What is chi-squared?
Assumptions
Violating expected frequencies
Chi-squared by hand
Critical values table
Two independent variables
Chi-square Test in SPSS
For one IV
SPSS output
For two IVs
SPSS output
Reporting the Chi-square test
Binominal test
Output and reporting
Summary
L5: Open Science and Current Issues
Current Issues
Example of an open study with notable flaws
Problems with recent practices
File drawer and publication bias
Researcher degrees of freedom
Reproducibility crisis
Can we believe published results?
Is there a reproducibility crisis?
What factors contribute to reproducibility?
Ways to fix the reproducibility issue
Statistics lectures 1-5 2
, Open science
Pre-registration
HARKing
Summary
L1: Significance and Power
Null Hypothesis Significance Testing
Basic rules of significance testing:
1. Assume the null hypothesis is true
2. Fit a statistical model/ test statistic to data
a. (conduct a t-test)
3. Consider signal/noise ratio
a. Signal - meaningful info you are trying to detect (results due to
manipulation)
b. Noise - random, unwanted variation that interferes with signal (any
confounding variables)
Different name variations for signal/ noise
Trying to measure how much variance in the data can be explained by our
model?
4. Calculate probability of getting that model assuming the null hypoth. is true
(aka. p value)
a. If p<.05, model fits data well and we can gain confidence in the alt.
hypoth.
b. Must consider how important the p value is; how likely it is we’ve found
a genuine effect; how accurate the results are
Statistics lectures 1-5 3
, Type I and Type II errors
When tests of sig. are inaccurate
ß = beta, a = alpha, 1 - ß = the power of a test
Power
☁️ Power - the probability of finding an effect assuming one exists in the
population
1-ß
ß = probability of not finding the effect, usually 0.2 (Cohen, 1992)
1 - 0.2 = 0.8
80% chance of detecting an effect assuming it genuinely exists. 0.8 is
the psychological recommendation variable
What affects power?
1. Size of alpha
2. Effect size
Statistics lectures 1-5 4
L1: Significance and Power
Null Hypothesis Significance Testing
Basic rules of significance testing:
Type I and Type II errors
Power
What affects power?
Effect size
No. of participants
Size of alpha
Problems with alpha testing
Why does this matter?
One and two-tailed tests
How this impacts data interpretation
Why does our p value change?
Which test should you use
Summary
L2: ANOVA Independent Groups
What is ANOVA?
Basic principles
Issues with multiple t-tests
Assumptions of ANOVA
Basis of the test
Between-groups ANOVA
Variance formula
Degrees of freedom
Pair-wise comparisons
Versions of ANOVA
ANOVA
MANOVA
ANCOVA
SPSS example
Worked example
Start with a plot
ANOVA in SPSS
Post-hoc tests
Glossary
L3: ANOVA Repeated Measures
Between groups vs. repeated measures
Statistics lectures 1-5 1
, F-ratio
Multi-factorial ANOVA
2x2 data examples
2x2 ANOVA
More factors
SPSS
Worked example
Possible outcomes
Plotting the data
Running the ANOVA
Output window
Reporting results
Summary
L4: Analysing Categorical Data
Chi-square test ( χ2 )
Contingency Table
What is chi-squared?
Assumptions
Violating expected frequencies
Chi-squared by hand
Critical values table
Two independent variables
Chi-square Test in SPSS
For one IV
SPSS output
For two IVs
SPSS output
Reporting the Chi-square test
Binominal test
Output and reporting
Summary
L5: Open Science and Current Issues
Current Issues
Example of an open study with notable flaws
Problems with recent practices
File drawer and publication bias
Researcher degrees of freedom
Reproducibility crisis
Can we believe published results?
Is there a reproducibility crisis?
What factors contribute to reproducibility?
Ways to fix the reproducibility issue
Statistics lectures 1-5 2
, Open science
Pre-registration
HARKing
Summary
L1: Significance and Power
Null Hypothesis Significance Testing
Basic rules of significance testing:
1. Assume the null hypothesis is true
2. Fit a statistical model/ test statistic to data
a. (conduct a t-test)
3. Consider signal/noise ratio
a. Signal - meaningful info you are trying to detect (results due to
manipulation)
b. Noise - random, unwanted variation that interferes with signal (any
confounding variables)
Different name variations for signal/ noise
Trying to measure how much variance in the data can be explained by our
model?
4. Calculate probability of getting that model assuming the null hypoth. is true
(aka. p value)
a. If p<.05, model fits data well and we can gain confidence in the alt.
hypoth.
b. Must consider how important the p value is; how likely it is we’ve found
a genuine effect; how accurate the results are
Statistics lectures 1-5 3
, Type I and Type II errors
When tests of sig. are inaccurate
ß = beta, a = alpha, 1 - ß = the power of a test
Power
☁️ Power - the probability of finding an effect assuming one exists in the
population
1-ß
ß = probability of not finding the effect, usually 0.2 (Cohen, 1992)
1 - 0.2 = 0.8
80% chance of detecting an effect assuming it genuinely exists. 0.8 is
the psychological recommendation variable
What affects power?
1. Size of alpha
2. Effect size
Statistics lectures 1-5 4
