1
2000PSY Stats 2
Self-Study:
• Scienti c Method/ structure :
1. Theory: a working model of the world, built from intuition and observation of
the world, and other people’s research to update it
2. Hypothesis: testing of the theory/ working model, by generating a hypothesis.
3. Measurement: measure facets of human bhr to produce data
4. Stats: run stats on data to understand if di erences or similarities might be
meaningful or not
5. Inferences: Use stats to make inferences about the theory to update it.
• Whenever we measure something, we get -Truth (what we’re trying to
measure) + error + bias (something that we introduce into the research) +
random error (not something that were intentionally do, just a random error
in the measurement)
• Standard normal distributions centre around 0, peak in the middle and vary
to the side
• Measurement errors can come from:
⁃ Observers
⁃ Researchers
⁃ Participants
• Reliability refers to the repeatability of measures. When we measure it
multiple times, the results must be consistent
• Validity: are we measuring what we’re trying to measure
• Reducing random error increases reliability
• Less reliability, less validity
• Reliability does not guarantee validity
Ways to test reliability:
• Inter rater/ inter observer reliability: the degree to which di erent raters/
observers give similar estimates of a phenomenon, agreement of the scores
b/w two or more judges. Especially important when the scores are
subjective and personal bias can be introduced.
• Test-retest reliability:
Type 1 error, illusion, you think there’s an e ect, but there’s no e ect
fi ff ff ff
, 2
They might give a table and ask you to report the ANOVA results, look at the
signi cance value, less than 0.05, then sign if more than not
Module 1:
Replicating science:
• Replication crisis: estimate the reproducibility of famous research in
psychology journals, checking whether they could be replicated, that they
found was that when replicated, found a wide distribution of p values, cog
psych- 50% research replicate, social psych, 36% replicable
• This has caused an increase in replication of studies
• Pre registration of studies, ppl submit a methods section, and if the
methods section is good, whether the journal is submitted or not, they will
agree to publish it regardless of the signi cance.
• In stats, the research hypothesis is framed closest to the alternative
hypothesis
• In experimental psych manipulate the independent variable and measure
the changes in dep variable
• Poor operational de nition is when, the variable measurement is so broad ir
measures multiple constructs
• A contract is the dep variable, who’s e ect is meansured after the
manipulation, a bhr or process that were intending to study
• If an experiment has good construct validity, then it has good internal
validity
• Questionnaire, before and after an event, prone to repeated testing,
• A score can only be as valid as as its reliable
• Experimental manipulations using fake participants are called stooges
Validity:
• Types of validity:
6. Construct validity/ measurement validity:
⁃ Manipulations, types, instructional: experimental conditions are de ned why
what you tell the participants, tell each grp di things
⁃ Environmental: you stage an event, present a stimulus or induce a state
⁃ Stooges: fake participants
Types of construct validity: convergent validity: do scores on the measure relate
with scores on other measure similar to this construct
Discriminant/ divergent validity: do the scores on this measure have low correlation
to scores on another unrelated measure
Face validity: do does the experimental method make sense? Is it good to
measure what its meant to measure
Content validity: does the method measure the entire range of characteristics
related to that construct
Criterion validity: two types: concurrent validity: do the scores on the measure
di er from other related scores,
ff fi
fi ff fi ff fi
2000PSY Stats 2
Self-Study:
• Scienti c Method/ structure :
1. Theory: a working model of the world, built from intuition and observation of
the world, and other people’s research to update it
2. Hypothesis: testing of the theory/ working model, by generating a hypothesis.
3. Measurement: measure facets of human bhr to produce data
4. Stats: run stats on data to understand if di erences or similarities might be
meaningful or not
5. Inferences: Use stats to make inferences about the theory to update it.
• Whenever we measure something, we get -Truth (what we’re trying to
measure) + error + bias (something that we introduce into the research) +
random error (not something that were intentionally do, just a random error
in the measurement)
• Standard normal distributions centre around 0, peak in the middle and vary
to the side
• Measurement errors can come from:
⁃ Observers
⁃ Researchers
⁃ Participants
• Reliability refers to the repeatability of measures. When we measure it
multiple times, the results must be consistent
• Validity: are we measuring what we’re trying to measure
• Reducing random error increases reliability
• Less reliability, less validity
• Reliability does not guarantee validity
Ways to test reliability:
• Inter rater/ inter observer reliability: the degree to which di erent raters/
observers give similar estimates of a phenomenon, agreement of the scores
b/w two or more judges. Especially important when the scores are
subjective and personal bias can be introduced.
• Test-retest reliability:
Type 1 error, illusion, you think there’s an e ect, but there’s no e ect
fi ff ff ff
, 2
They might give a table and ask you to report the ANOVA results, look at the
signi cance value, less than 0.05, then sign if more than not
Module 1:
Replicating science:
• Replication crisis: estimate the reproducibility of famous research in
psychology journals, checking whether they could be replicated, that they
found was that when replicated, found a wide distribution of p values, cog
psych- 50% research replicate, social psych, 36% replicable
• This has caused an increase in replication of studies
• Pre registration of studies, ppl submit a methods section, and if the
methods section is good, whether the journal is submitted or not, they will
agree to publish it regardless of the signi cance.
• In stats, the research hypothesis is framed closest to the alternative
hypothesis
• In experimental psych manipulate the independent variable and measure
the changes in dep variable
• Poor operational de nition is when, the variable measurement is so broad ir
measures multiple constructs
• A contract is the dep variable, who’s e ect is meansured after the
manipulation, a bhr or process that were intending to study
• If an experiment has good construct validity, then it has good internal
validity
• Questionnaire, before and after an event, prone to repeated testing,
• A score can only be as valid as as its reliable
• Experimental manipulations using fake participants are called stooges
Validity:
• Types of validity:
6. Construct validity/ measurement validity:
⁃ Manipulations, types, instructional: experimental conditions are de ned why
what you tell the participants, tell each grp di things
⁃ Environmental: you stage an event, present a stimulus or induce a state
⁃ Stooges: fake participants
Types of construct validity: convergent validity: do scores on the measure relate
with scores on other measure similar to this construct
Discriminant/ divergent validity: do the scores on this measure have low correlation
to scores on another unrelated measure
Face validity: do does the experimental method make sense? Is it good to
measure what its meant to measure
Content validity: does the method measure the entire range of characteristics
related to that construct
Criterion validity: two types: concurrent validity: do the scores on the measure
di er from other related scores,
ff fi
fi ff fi ff fi
