Step by step guide T-Test
Goal: to compare means 3 tests:
1. One-sample t-test mean <-> fixed value
2. Independent samples t-test mean group 1 <-> mean group 2 in-between groups
3. Dependent samples t-test mean measurement 1 <-> mean measurement 2 within groups
Hypothesis:
Null-hypothesis H0 no difference between group means
Alternative hypothesis H1 difference between group means
Goal: How unusual is our observed difference between group means in the sample, assuming H 0 is true?
Before anything else, read the question thoroughly! Think for yourself, what is asked? What is the kind of answer
they are looking for? What is H0 in this case and what would be H 1? What are the dependent and independent
variables?
Which test do I need to use?
Important remark: If the assumption of normality is violated, following central limit theorem (= the sampling
distribution of the sample means approaches a normal distribution as the sample size becomes large) you can still
run the test. Report in the conclusion.
Important note: when reporting a non-parametric test (no normal distribution of a parameter), report the median
along with the means as well as the range. Do not report 95% CI (these revolve around the mean).
Important remark: Always report outliers. Always report effect sizes, even if there is no significant effect. How to
report T-test outcomes can be found in PU5 (APA) and in the separate document for reporting statistics guide [see
Canvas].
1. One-sample t-test (assumptions: normality, independence)
1. We want to compare the mean score of the group on some measure to a pre-determined value.
2. Explore the data. Is normality assumed? Use QQ plot or histogram for visual data. Find z-scores of skewness
and kurtosis (divide by their standard error) between -1.96 and 1.96 = normal distribution. Or use Shapiro-
Wilk test for smaller samples (closer to 1 = normal data). There is no need to check for homogeneity when
there is only one variable.
a. Normal distribution Student’s t-test
b. No normal distribution Wilcoxon rank statistics. However, if the question says to rely on CLT you use
Student’s t-test, even if normality is violated.
3. State the correct mean value for the null hypothesis in ‘hypothesis’ ‘test value’. This value will be in the
question. Check if the scale in the question matches the scale of the DV. Use a dot in jamovi for decimals.
4. Tick the boxes for: mean difference, confidence interval, effect size and descriptives.
5. Find out: significance (p-value), 95% confidence interval (lower and upper), effect size (Cohen’s d).
a. Student’s: t and degree of freedom (N – 1)
b. Wilcoxon rank statistics: W (no df)
6. Write down the conclusion in APA.
a. Significant result: data shows H0 can be rejected. Be precise in what this means.
b. No significant result: data shows H0 cannot be rejected. Be precise in what this means.
2. Independent samples t-test (assumptions: normality, independence, homogeneity of variance)
1. We want to compare the mean of two different groups to one another on a certain variable.
2. Explore the data. Is normality assumed? Use QQ plot or histogram for visual data. Find z-scores of skewness
and kurtosis (divide by their standard error) between -1.96 and 1.96 = normal distribution. Or use Shapiro-
Wilk test for smaller samples (closer to 1 = normal data).
1
Goal: to compare means 3 tests:
1. One-sample t-test mean <-> fixed value
2. Independent samples t-test mean group 1 <-> mean group 2 in-between groups
3. Dependent samples t-test mean measurement 1 <-> mean measurement 2 within groups
Hypothesis:
Null-hypothesis H0 no difference between group means
Alternative hypothesis H1 difference between group means
Goal: How unusual is our observed difference between group means in the sample, assuming H 0 is true?
Before anything else, read the question thoroughly! Think for yourself, what is asked? What is the kind of answer
they are looking for? What is H0 in this case and what would be H 1? What are the dependent and independent
variables?
Which test do I need to use?
Important remark: If the assumption of normality is violated, following central limit theorem (= the sampling
distribution of the sample means approaches a normal distribution as the sample size becomes large) you can still
run the test. Report in the conclusion.
Important note: when reporting a non-parametric test (no normal distribution of a parameter), report the median
along with the means as well as the range. Do not report 95% CI (these revolve around the mean).
Important remark: Always report outliers. Always report effect sizes, even if there is no significant effect. How to
report T-test outcomes can be found in PU5 (APA) and in the separate document for reporting statistics guide [see
Canvas].
1. One-sample t-test (assumptions: normality, independence)
1. We want to compare the mean score of the group on some measure to a pre-determined value.
2. Explore the data. Is normality assumed? Use QQ plot or histogram for visual data. Find z-scores of skewness
and kurtosis (divide by their standard error) between -1.96 and 1.96 = normal distribution. Or use Shapiro-
Wilk test for smaller samples (closer to 1 = normal data). There is no need to check for homogeneity when
there is only one variable.
a. Normal distribution Student’s t-test
b. No normal distribution Wilcoxon rank statistics. However, if the question says to rely on CLT you use
Student’s t-test, even if normality is violated.
3. State the correct mean value for the null hypothesis in ‘hypothesis’ ‘test value’. This value will be in the
question. Check if the scale in the question matches the scale of the DV. Use a dot in jamovi for decimals.
4. Tick the boxes for: mean difference, confidence interval, effect size and descriptives.
5. Find out: significance (p-value), 95% confidence interval (lower and upper), effect size (Cohen’s d).
a. Student’s: t and degree of freedom (N – 1)
b. Wilcoxon rank statistics: W (no df)
6. Write down the conclusion in APA.
a. Significant result: data shows H0 can be rejected. Be precise in what this means.
b. No significant result: data shows H0 cannot be rejected. Be precise in what this means.
2. Independent samples t-test (assumptions: normality, independence, homogeneity of variance)
1. We want to compare the mean of two different groups to one another on a certain variable.
2. Explore the data. Is normality assumed? Use QQ plot or histogram for visual data. Find z-scores of skewness
and kurtosis (divide by their standard error) between -1.96 and 1.96 = normal distribution. Or use Shapiro-
Wilk test for smaller samples (closer to 1 = normal data).
1
