TOE Correlationeel hoorcollege 1
Surveys
Administratieve data: belastingdienst
Aspirational data: Facebook, twitter, Instagram
Transactional: wanneer je betaalt met een pinpas
(ook een bonuskaart)
Organic data => bepaalde data die een andere functie heeft, een niet onderzoek doeleinde
maar kan wel worden gebruikt voor onderzoek
We design a study & collect data to:
Describe the social reality
Study (casual) relationships
Generalize to the target population
Inferential goals:
- Description (beschrijven)
- Causation (oorzaak en gevolg)
- Prediction (voorspellen)
Survey modes:
- Face-to-face (CAPI)
- Mail (per post)
- Telephone (CATI)
- Internet
- Mixed-mode
Differences between the modes
Degree of interviewer
involvement
Degree of interaction with the
respondent
Degree of privacy
Channels of communication
Visual
Auditive
Technology use
,Types of mixed-mode design
- One mode for some respondents, another mode for others
- One mode for recruitment, another for survey administration
- One mode for data collection, another for reminders, follow-up
- One mode for main part of the interview, another for some subset of answer
(sensitive items)
Panel surveys -> panel surveys interview respondents over time (content usually the same,
but may differ)
Advantages:
- We can assess within-person change & causality
- We can disentangle age, period, and cohort effects
Potential errors:
- Attrition (drop-out or consecutive wave-nonresponse) = uitval
- Panel conditioning (learning effects)
One mode for one wave of the panel survey, another for other
(Example: first wave face-to-face, following waves online to save costs)
Coverage error (dekkingsfout)
Occurs:
- If not all members of the population have a known, nonzero change of being included
in the sample
- If persons excluded from the sampling frame, differ from those included
Sampling error (steekproeffout)
Occurs:
- From surveying only some, rather than all, members of the covered population
- Uncertainty
- We need statistics to quantify this uncertainty
,Nonresponse error (non-response fout)
Occurs:
- If people selected for a survey who do not respond differ from those who do on
relevant variables
Persons chosen for the sample refuse to
Cooperate with the survey request entirely -> unit-nonresponse
Answer a certain response -> item-nonresponse (Adjustment: imputation)
Reasons:
Technical difficulties
Lack of motivation/interest
Lack of trust with sensitive questions
Nonresponse bias
Systematic differences between respondents and non-respondents on key study variables
Adjustment error
Because of coverage and nonresponse, statistics based on respondents may depart from
those of target population.
Measurement error
Answer in a sociable acceptable way.
Measurement error occurs if respondents provide inaccurate answers due to:
- Survey mode effects
- Poor question wording
- Aspects of their behavior
Processing error
Answers to the surveys are not put right into the data
TOE Correlationeel hoorcollege 2
Construct -> Conceptual definition -> Operational definition -> variable
Reliable measurement (consistency of measurement) -> Betrouwbaarheid
- Measurement does not vary due to characteristics of how you measured or the
measurement instrument
- AKA precision
- Example: bathroom scale should give you the same weight if you step on it twice
Validity of measurement (correctness of measurement) -> Validiteit
- How well your measurement matches the concept of interest
- AKA accuracy
, Measurement validity -> Begripsvaliditeit
Face – Does the measurement appear OK?
Content – Does it measure all aspects of the construct?
Convergent – Does it correlate with another measure of the same construct?
Divergent – Does is not correlate with something that measures something different?
Criterion – Does it predict an outcome?
Measuring strength
How can we measure the strength of the relationship? -> Correlation
- A measure of strength and direction of a linear relationship between two
interval/ratio variables
- Denoted by r
- Values between -1 and +1
Reliability -> Betrouwbaarheid
- Test-retest – Test-hertest
- Interrater – Interbeoordelaarsbetrouwbaarheid
- Internal – Interne
Hawthorne effect = observer bias
People behave differently than normal
Validity
- Construct validity
- External validity
Coverage
Nonresponse
Reactivity/ Hawthorne effect (because you participate in a study you may change
your behavior)
- Recall: Under-reporting
Reliability
- Recall: Estimation
Problems with measurement
What works depends on the population and your resources
Measuring
What are you interested in?
- Facts secondary data, ask specialist
- Behaviors observe/ ask the person
- Opinions ask the person
We often ask people questions: survey!
Surveys
Administratieve data: belastingdienst
Aspirational data: Facebook, twitter, Instagram
Transactional: wanneer je betaalt met een pinpas
(ook een bonuskaart)
Organic data => bepaalde data die een andere functie heeft, een niet onderzoek doeleinde
maar kan wel worden gebruikt voor onderzoek
We design a study & collect data to:
Describe the social reality
Study (casual) relationships
Generalize to the target population
Inferential goals:
- Description (beschrijven)
- Causation (oorzaak en gevolg)
- Prediction (voorspellen)
Survey modes:
- Face-to-face (CAPI)
- Mail (per post)
- Telephone (CATI)
- Internet
- Mixed-mode
Differences between the modes
Degree of interviewer
involvement
Degree of interaction with the
respondent
Degree of privacy
Channels of communication
Visual
Auditive
Technology use
,Types of mixed-mode design
- One mode for some respondents, another mode for others
- One mode for recruitment, another for survey administration
- One mode for data collection, another for reminders, follow-up
- One mode for main part of the interview, another for some subset of answer
(sensitive items)
Panel surveys -> panel surveys interview respondents over time (content usually the same,
but may differ)
Advantages:
- We can assess within-person change & causality
- We can disentangle age, period, and cohort effects
Potential errors:
- Attrition (drop-out or consecutive wave-nonresponse) = uitval
- Panel conditioning (learning effects)
One mode for one wave of the panel survey, another for other
(Example: first wave face-to-face, following waves online to save costs)
Coverage error (dekkingsfout)
Occurs:
- If not all members of the population have a known, nonzero change of being included
in the sample
- If persons excluded from the sampling frame, differ from those included
Sampling error (steekproeffout)
Occurs:
- From surveying only some, rather than all, members of the covered population
- Uncertainty
- We need statistics to quantify this uncertainty
,Nonresponse error (non-response fout)
Occurs:
- If people selected for a survey who do not respond differ from those who do on
relevant variables
Persons chosen for the sample refuse to
Cooperate with the survey request entirely -> unit-nonresponse
Answer a certain response -> item-nonresponse (Adjustment: imputation)
Reasons:
Technical difficulties
Lack of motivation/interest
Lack of trust with sensitive questions
Nonresponse bias
Systematic differences between respondents and non-respondents on key study variables
Adjustment error
Because of coverage and nonresponse, statistics based on respondents may depart from
those of target population.
Measurement error
Answer in a sociable acceptable way.
Measurement error occurs if respondents provide inaccurate answers due to:
- Survey mode effects
- Poor question wording
- Aspects of their behavior
Processing error
Answers to the surveys are not put right into the data
TOE Correlationeel hoorcollege 2
Construct -> Conceptual definition -> Operational definition -> variable
Reliable measurement (consistency of measurement) -> Betrouwbaarheid
- Measurement does not vary due to characteristics of how you measured or the
measurement instrument
- AKA precision
- Example: bathroom scale should give you the same weight if you step on it twice
Validity of measurement (correctness of measurement) -> Validiteit
- How well your measurement matches the concept of interest
- AKA accuracy
, Measurement validity -> Begripsvaliditeit
Face – Does the measurement appear OK?
Content – Does it measure all aspects of the construct?
Convergent – Does it correlate with another measure of the same construct?
Divergent – Does is not correlate with something that measures something different?
Criterion – Does it predict an outcome?
Measuring strength
How can we measure the strength of the relationship? -> Correlation
- A measure of strength and direction of a linear relationship between two
interval/ratio variables
- Denoted by r
- Values between -1 and +1
Reliability -> Betrouwbaarheid
- Test-retest – Test-hertest
- Interrater – Interbeoordelaarsbetrouwbaarheid
- Internal – Interne
Hawthorne effect = observer bias
People behave differently than normal
Validity
- Construct validity
- External validity
Coverage
Nonresponse
Reactivity/ Hawthorne effect (because you participate in a study you may change
your behavior)
- Recall: Under-reporting
Reliability
- Recall: Estimation
Problems with measurement
What works depends on the population and your resources
Measuring
What are you interested in?
- Facts secondary data, ask specialist
- Behaviors observe/ ask the person
- Opinions ask the person
We often ask people questions: survey!
