Aantekeningen ARM 2019/2020
COLLEGE 1
Deductive reasoning: top-down logic (van algemeen naar specifiek)
Confirmation bias: the tendency to seek, interpret and remember information that confirms
their perceptions. When people overvalue evidence that confirms their hypothesis and
undervalue evidence that could disprove it.
Disconfirmation Dilemma (DD)
Evidence showing that a hypothesis/belief is not true (wanneer je resultaten krijgt die
tegenovergesteld zijn/ niet significant zijn aan je hypothese). This puts you in a dilemma.
Resolving DD is routine in ‘’normal’’ science.
Solutions: NO solutions:
- Re-evaluate theory - Reanalyse data
- Re-consider methodological choices - Change hypothesis
- Maintain a condition seeking - Ignore disconfirming results
approach: think how conditions can
disprove your hypothesis on forehand
To fight confirmation bias:
1. Method of multiple working hypotheses
o Lay out all plausible hypothesis
o Examine theoretical and empirical evidence for each one of them
o Decide which hypothesis to keep/loose
Benefits
o Objectivity: helps to separate you from your hypothesis
o Efficiency: allows experiments to be designed to distinguish among competing
hypotheses, rather than evaluating one
o Focus: reinforces a focus in falsification rather than confirmation
2. Strong Inference
o Certain systematic methods of scientific thinking may produce much more rapid
progress than others.
o See model
, Sample population
Population: the entire group of individuals that is relevant to your research question.
Sample: a properly selected part of that population: selected randomly or otherwise
Random sampling error
Sample size (underpowered sample)
Make sure to have sufficient variables which could have an impact on your results (e.g.
gender, age etc.)
Systematic sampling error
Population
When using a convenient sample which isn’t generizable to the entire population
An ideal sample should be large (enough) and aselect, but..
o Self-selection bias: sampling error caused by researcher and participant
(especially when conducting research online)
Solutions:
1. Target random selection by approaching potential respondents yourself
2. Keep your survey neutral (in copy: don’t be to specific)
3. Probability sampling: approach participants with equal chance to participate
Selected at random
Large sample
No self selection
Universality (drawing conclusions on population) is no issue
But you need huge panel, so quite costly.
Error probability: chances of encountering a statistical error (Type I of Type II)
- Type I error: the rejection of a true null hypothesis (a "false positive")
- Type II error: the non-rejection of a false null hypothesis ("false negative")
Effect size: strength of the statistical association between variables
The larger the effect size, the smaller the sample size needs to be.
Power: the ability of a statistical test to detect a true effect (reduce the chance of a false-
negative), therefore, sufficient participants are needed
Power (1 – β) = 80% or 0.8) = 80% or 0.8
Where β is chance of false-negative (Type II error) is chance of false-negative (Type II error)
Level of significance (α) = 5% or 0.05 (p-value)) = 5% or 0.05 (p-value)
COLLEGE 1
Deductive reasoning: top-down logic (van algemeen naar specifiek)
Confirmation bias: the tendency to seek, interpret and remember information that confirms
their perceptions. When people overvalue evidence that confirms their hypothesis and
undervalue evidence that could disprove it.
Disconfirmation Dilemma (DD)
Evidence showing that a hypothesis/belief is not true (wanneer je resultaten krijgt die
tegenovergesteld zijn/ niet significant zijn aan je hypothese). This puts you in a dilemma.
Resolving DD is routine in ‘’normal’’ science.
Solutions: NO solutions:
- Re-evaluate theory - Reanalyse data
- Re-consider methodological choices - Change hypothesis
- Maintain a condition seeking - Ignore disconfirming results
approach: think how conditions can
disprove your hypothesis on forehand
To fight confirmation bias:
1. Method of multiple working hypotheses
o Lay out all plausible hypothesis
o Examine theoretical and empirical evidence for each one of them
o Decide which hypothesis to keep/loose
Benefits
o Objectivity: helps to separate you from your hypothesis
o Efficiency: allows experiments to be designed to distinguish among competing
hypotheses, rather than evaluating one
o Focus: reinforces a focus in falsification rather than confirmation
2. Strong Inference
o Certain systematic methods of scientific thinking may produce much more rapid
progress than others.
o See model
, Sample population
Population: the entire group of individuals that is relevant to your research question.
Sample: a properly selected part of that population: selected randomly or otherwise
Random sampling error
Sample size (underpowered sample)
Make sure to have sufficient variables which could have an impact on your results (e.g.
gender, age etc.)
Systematic sampling error
Population
When using a convenient sample which isn’t generizable to the entire population
An ideal sample should be large (enough) and aselect, but..
o Self-selection bias: sampling error caused by researcher and participant
(especially when conducting research online)
Solutions:
1. Target random selection by approaching potential respondents yourself
2. Keep your survey neutral (in copy: don’t be to specific)
3. Probability sampling: approach participants with equal chance to participate
Selected at random
Large sample
No self selection
Universality (drawing conclusions on population) is no issue
But you need huge panel, so quite costly.
Error probability: chances of encountering a statistical error (Type I of Type II)
- Type I error: the rejection of a true null hypothesis (a "false positive")
- Type II error: the non-rejection of a false null hypothesis ("false negative")
Effect size: strength of the statistical association between variables
The larger the effect size, the smaller the sample size needs to be.
Power: the ability of a statistical test to detect a true effect (reduce the chance of a false-
negative), therefore, sufficient participants are needed
Power (1 – β) = 80% or 0.8) = 80% or 0.8
Where β is chance of false-negative (Type II error) is chance of false-negative (Type II error)
Level of significance (α) = 5% or 0.05 (p-value)) = 5% or 0.05 (p-value)
