ARMS CP tentamen
Knowledge clip 1
Power & effect size
Effect size: measures the proportion of total variance in the dependent variable, that is associated
with the membership of different groups as defined by an independent variable
= an objective and standardized measure of the size of and observed effect
Different ways of measuring effect size:
- R squared in an regression
- Partial eta squared (pƞ2) in an ANOVA
- Pearson r (= size of an correlation)
Partial eta squared (ƞ2):
Small effect 0.01
Medium effect 0.06
Large effect > 0.14
Pearson r (or regression β ):
Small effect 0.10
Medium effect 0.30
Large effect > 0.50
Difference between significant and relevant!
VB:
1) F(156) = 23.45, p < 0.001. ƞ2 = 0.02
2) F(56) = 23.45, p < 0.05. ƞ2 = 0.30
→ Both effects are significant since p is smaller than 0.05.
→ Effect 2 shows a more relevant effect: higher effect size (ƞ2)
VB:
→ Age is a significant
predictor, but it is NOT
associated with PTSD
because the effect size is
smaller than a small effect
size
INDEED in the analysis plan, the large
sample size was already kept in
mind: they set the criteria for
significance and relevance >
Significance level was set at α = 0.01
(instead of the usual 0.05)
→ In addition: effects were
considered to be relevant if they
were at least small
, Statistical power has an influence on the possibility of detecting an existing effect of a particular
size, therefore the opportunity to correctly reject the null hypothesis
Power = 1 – β
β = probability of a type II error (NOTE: is a different β than in the regression analysis)
Goal: minimal power of 0.80
If your sample is large and your effect size is small, you can reduce the sample to increase the effect
size
Type 1 error (α): the probability that an effect will be detected where in fact no effect exists: the H0
is rejected when in fact it is true
● H0 ten onrechte verwerpen > dacht verband, is géén verband
Type 2 error ( β ): the probability that no effect will be detected where an effect does in fact exist: the
H0 is not rejected when in fact it is false
● H0 ten onrechte NIET verwerpen > dacht geen verband, is wél verband
Statistical power influences the possibility to detect an existing effect of a given size, and so the
chance to correctly reject the null hypothesis
Type II error (β) is the chance that an effect will not be detected when in fact this effect is present;
the null hypothesis (H0) is not rejected, even though it is false
Power depends on:
- p-value = chance of a type I error (α)
- Effect size
- Sample size
Dotted line = H0
Continuing line = HA
Knowledge clip 1
Power & effect size
Effect size: measures the proportion of total variance in the dependent variable, that is associated
with the membership of different groups as defined by an independent variable
= an objective and standardized measure of the size of and observed effect
Different ways of measuring effect size:
- R squared in an regression
- Partial eta squared (pƞ2) in an ANOVA
- Pearson r (= size of an correlation)
Partial eta squared (ƞ2):
Small effect 0.01
Medium effect 0.06
Large effect > 0.14
Pearson r (or regression β ):
Small effect 0.10
Medium effect 0.30
Large effect > 0.50
Difference between significant and relevant!
VB:
1) F(156) = 23.45, p < 0.001. ƞ2 = 0.02
2) F(56) = 23.45, p < 0.05. ƞ2 = 0.30
→ Both effects are significant since p is smaller than 0.05.
→ Effect 2 shows a more relevant effect: higher effect size (ƞ2)
VB:
→ Age is a significant
predictor, but it is NOT
associated with PTSD
because the effect size is
smaller than a small effect
size
INDEED in the analysis plan, the large
sample size was already kept in
mind: they set the criteria for
significance and relevance >
Significance level was set at α = 0.01
(instead of the usual 0.05)
→ In addition: effects were
considered to be relevant if they
were at least small
, Statistical power has an influence on the possibility of detecting an existing effect of a particular
size, therefore the opportunity to correctly reject the null hypothesis
Power = 1 – β
β = probability of a type II error (NOTE: is a different β than in the regression analysis)
Goal: minimal power of 0.80
If your sample is large and your effect size is small, you can reduce the sample to increase the effect
size
Type 1 error (α): the probability that an effect will be detected where in fact no effect exists: the H0
is rejected when in fact it is true
● H0 ten onrechte verwerpen > dacht verband, is géén verband
Type 2 error ( β ): the probability that no effect will be detected where an effect does in fact exist: the
H0 is not rejected when in fact it is false
● H0 ten onrechte NIET verwerpen > dacht geen verband, is wél verband
Statistical power influences the possibility to detect an existing effect of a given size, and so the
chance to correctly reject the null hypothesis
Type II error (β) is the chance that an effect will not be detected when in fact this effect is present;
the null hypothesis (H0) is not rejected, even though it is false
Power depends on:
- p-value = chance of a type I error (α)
- Effect size
- Sample size
Dotted line = H0
Continuing line = HA
