Lecture summary Survey Research Methods

Survey Research Methods Lecture Notes – Sofie Polman
Table of Contents
Lecture 2: Introduction To Survey Research................................................................................................2
Lecture 3: Planning A Survey.......................................................................................................................5
Lecture 4: Designing A Survey – Measurement...........................................................................................8
Lecture 5: Scale Development...................................................................................................................14
Lecture 6: Designing A Survey – Questionnaire Design..............................................................................16
Lecture 7: Segmentation and Typology.....................................................................................................21
Lecture 8: Implementing A Survey............................................................................................................22
Lecture 9: Testing Relationships................................................................................................................28
Lecture 12: Wrap-Up.................................................................................................................................31

,Lecture 2: Introduction To Survey Research




What is important?
Everything related to questions
 Clear questions to be able to get a proper answer
 Avoid leading questions (questions that leads into a certain direction) so you won’t get unbiased
answers
 Making sure that scales are valid and reliable
 Have control questions to make sure that person is paying attention
 Demographics, to be able to compare different groups
 Social desirability is something to avoid

Everything related to sample
 Good sample – sample size doesn’t say anything about the sample, you have to keep in mind the
population
 Different sampling

Everything related to the analytics
 Methods should be “right”
o Exploratory factor analysis
o Regression
o ANOVA

Survey length  depends how many questions you can ask depends on motivation of participants

What is Survey Research?
A survey is a systematic method for gathering information with standardized questionnaires from (a sample
of) entities for the purpose of constructing quantitative descriptors of the attributes of the larger population
of which the entities are members. (Groves et al. 2009)
 Systematic and Standardized – to be able to quantify the information we get
o Distinguishes itself from other techniques such as qualitative research
 (A Sample of) Entities – can also be everyone in the population
o Mostly a selection of the population is included in the survey
o All entities can be included as well (e.g., employee survey)
 Quantitative Descriptors
o Statistics needed to analyze the responses

A Short History of Survey Research
 Census surveys were conducted as early as Ancient Egypt
o Census survey: surveys that are obliged to answer to, and you want to ask everyone in the
population
o Poll is survey you gather from part of the population what you generalize to the entire
population

,  Survey as a formal research method was pioneered in the 1930-40s by sociologist Paul Lazarsfeld to
examine the effects of the radio on political opinion formation of the United States
 Today, surveys provide managers with deeper insights into their customers and employees
 Surveys can contribute to generalizing experimental findings to different persons and settings

Sampling procedures of survey research
Probability sampling
Selection of sampling techniques in which the chance of each case (e.g., individual) being selected from the
population is known and not zero.
→ Randomization is used instead of deliberate choice
 European Social Survey with a sampling frame and exact data on
background of all individuals in the sample

Nonprobability sampling
Selection of sampling techniques in which the chance of each case (e.g.,
individual) being selected is NOT known.
→ The researcher deliberately picks the sample
 Fellow students selected to participate in a survey

Time Horizon of Survey Research
 Cross-Sectional Design (FOCUS)
o Opinion polls (e.g., to predict electoral results)
o Typical thesis survey (e.g., to understand consumer preferences or attitudes)
 Longitudinal Design
o Repeated cross-sectional (different sample each time)
o Fixed-sample panel design (the same sample each time)
o Cohort study (follow the same people over a longer period of time, usually after an event;
e.g., people graduating MSc Marketing in 2010 and follow their lives over a 10-year time
period)

Administration methods of survey research
 Interview surveys
o Personal surveys
o Telephone surveys
 Self-completion surveys
o Written surveys
o Online surveys
 depends on a lot of factors (time, resources) which is best/most suitable


Why so Popular? And What are the Challenges?
Main Pros
 Ease: Questionnaires are relatively easy to administer to large number of respondents
 Simplicity: coding, analysis, and interpretation of data is relatively simple
 Reliability: Standardization reduces variability in the answers

Challenges
 Asking the right questions is not easy
 Properly wording questions is not easy
 Getting the right sample (sampling) is not easy
 Motivating respondents to respond (candidly) is not easy

Publishing survey research is not easy because of certain biases about the information you can get from
surveys.

, Total Survey Error = Measurement and Representation
1. Measurement  do you actually measure what you want to
measure?
 What is the survey about or, more specifically, what does the
survey measures, and do the observed responses actually measure
what the researcher is interested in?
2. Representation (& sample when collecting data)
 Whom does the survey describe, and if the survey does not involve
self-reports, who provides the desired data?
3. Analysis and Reporting
 What can we learn from the survey?

Errors that invalidate survey results may arise in all areas.
There are also additional sources of error and bias...

Measurement in the Survey (Groves et al. 2004)
 Construct: (Latent) Variable and its relationships
o Covid infections and age
 Measurement: Form in which (latent) variable is assessed
o Question: “Did you have Covid in the past six months?”
 Response: Response provided
o Age (Year of birth)
o Open answer: Yes or No
 Edited Response
o Year of birth: 1483, 27, 201, 2200?
o Edited answer: “perhaps” is not valid?
 Population: Set of units under study
o Population: All adults in the Netherlands
 Sampling Frame
o Set of eligible units to survey
o Sampling Frame: Dutch phone no. directory
 Sample: Set of units to survey
o Sample: 10,000 Dutch phone no.
 Respondents
o Set of successfully surveyed units
o Respondents: 1,214 AI-powered interviews
o 31 percent male, 69 percent female
 Adjusted Respondents
o Final set of respondents for analysis
o Adjusted respondents: Weighted sample

A survey cannot show you causality – and sometimes does not take all factors into account (you have to
control for other factors to get to quasi-causality)

Additional survey biases: trusting and publishing results