Summary Research Skills: Design and Analysis of
Questionnaires
Tilburg University 2020-2021 – Master BDM
Overview
Summary Research Skills: Design and Analysis of Questionnaires Tilburg University 2020-2021 –
Master BDM...........................................................................................................................................1
Question & Questionnaire design – Krosnick & Presser, 2010...............................................................2
10 Examples of biased survey questions................................................................................................3
5 key messages in a survey introduction – DeFranzo, 2012...................................................................4
Ch 8. Writing effective questions – Floyd & Consenza..........................................................................10
Ch 9. Programming the survey -Toepoel (2009)...................................................................................18
Summary Research Skills: Design and Analysis of Questionnaires.........................................................1
Question & Questionnaire design – Krosnick & Presser, 2010...............................................................2
10 Examples of biased survey questions................................................................................................3
5 key messages in a survey introduction – DeFranzo, 2012...................................................................3
Ch 2. Developing the survey: questions & answers – Toepoel , 2009....................................................4
Principles for writing survey questions ( Dillman, 2007)........................................................................8
Ch 8. Writing effective questions – Floyd & Consenza..........................................................................10
Methods for testing & evaluating survey questions - Presser et al. (2004)..........................................13
Threat of satisficing in surveys – Krosnick (2000).................................................................................16
Ch 9. Programming the survey -Toepoel (2009)...................................................................................18
Factor analysis in SPSS..........................................................................................................................22
1
,Question & Questionnaire design – Krosnick & Presser,
2010
Biases
Social desirability response : form of motivated misreporting
Recall bias: form of unmotivated misreporting
Conventional wisdom
1. Use simple, familiar words (no technical terms, jargon & slang)
2. Use simple syntax
3. Avoid words with ambiguous meanings -> should be interpreted in one way
4. Strive for specific & concrete wording
5. Response options should be exhaustive & mutually exclusive
6. Avoid leading/loaded questions that push respondents towards a particular answer
7. Ask about 1 thing at the time (avoid double-barreled questions)
8. Avoid questions with single/double negations (i.e. negatively phrased words)
Optimize question order
1. Early questions > easy & pleasant to answer, built rapport between respondent & researcher
2. Beginning > explicitly address topic of survey in questions (as described prior to interview)
3. Questions on same topic should be grouped together
4. Question on same topic proceed from general to more specific
2
, 5. Questions on sensitive topics should be at the end
6. Include filter questions > avoid asking respondents questions that don’t apply
Always pretest the survey
10 Examples of biased survey questions
Survey bias example
1. Leading question: lead respondents to give “correct” answers > should use neutral words
2. Loaded question: force people to answer in a particular way, leads to dropout & unclear results
> keep respondents from explaining own opinions
3. Double-barreled question: ask 2 questions at once > should use only 1 subject per question
4. The absolute question: yes or no questions > don’t get the whole story
> should avoid using words like “always, all, ever, every”
5. Unclear question: avoid technical jargon & acronyms
6. Multiple answer questions: should be exhaustive & mutually exclusive
7. Prefer not to answer: include this option to avoid drop-outs
8. Include all possible answers: add “other” if needed
9. Use accurate scales: range from bad to excellent
10. Survey structure: personal & in-depth questions at the end
3
Questionnaires
Tilburg University 2020-2021 – Master BDM
Overview
Summary Research Skills: Design and Analysis of Questionnaires Tilburg University 2020-2021 –
Master BDM...........................................................................................................................................1
Question & Questionnaire design – Krosnick & Presser, 2010...............................................................2
10 Examples of biased survey questions................................................................................................3
5 key messages in a survey introduction – DeFranzo, 2012...................................................................4
Ch 8. Writing effective questions – Floyd & Consenza..........................................................................10
Ch 9. Programming the survey -Toepoel (2009)...................................................................................18
Summary Research Skills: Design and Analysis of Questionnaires.........................................................1
Question & Questionnaire design – Krosnick & Presser, 2010...............................................................2
10 Examples of biased survey questions................................................................................................3
5 key messages in a survey introduction – DeFranzo, 2012...................................................................3
Ch 2. Developing the survey: questions & answers – Toepoel , 2009....................................................4
Principles for writing survey questions ( Dillman, 2007)........................................................................8
Ch 8. Writing effective questions – Floyd & Consenza..........................................................................10
Methods for testing & evaluating survey questions - Presser et al. (2004)..........................................13
Threat of satisficing in surveys – Krosnick (2000).................................................................................16
Ch 9. Programming the survey -Toepoel (2009)...................................................................................18
Factor analysis in SPSS..........................................................................................................................22
1
,Question & Questionnaire design – Krosnick & Presser,
2010
Biases
Social desirability response : form of motivated misreporting
Recall bias: form of unmotivated misreporting
Conventional wisdom
1. Use simple, familiar words (no technical terms, jargon & slang)
2. Use simple syntax
3. Avoid words with ambiguous meanings -> should be interpreted in one way
4. Strive for specific & concrete wording
5. Response options should be exhaustive & mutually exclusive
6. Avoid leading/loaded questions that push respondents towards a particular answer
7. Ask about 1 thing at the time (avoid double-barreled questions)
8. Avoid questions with single/double negations (i.e. negatively phrased words)
Optimize question order
1. Early questions > easy & pleasant to answer, built rapport between respondent & researcher
2. Beginning > explicitly address topic of survey in questions (as described prior to interview)
3. Questions on same topic should be grouped together
4. Question on same topic proceed from general to more specific
2
, 5. Questions on sensitive topics should be at the end
6. Include filter questions > avoid asking respondents questions that don’t apply
Always pretest the survey
10 Examples of biased survey questions
Survey bias example
1. Leading question: lead respondents to give “correct” answers > should use neutral words
2. Loaded question: force people to answer in a particular way, leads to dropout & unclear results
> keep respondents from explaining own opinions
3. Double-barreled question: ask 2 questions at once > should use only 1 subject per question
4. The absolute question: yes or no questions > don’t get the whole story
> should avoid using words like “always, all, ever, every”
5. Unclear question: avoid technical jargon & acronyms
6. Multiple answer questions: should be exhaustive & mutually exclusive
7. Prefer not to answer: include this option to avoid drop-outs
8. Include all possible answers: add “other” if needed
9. Use accurate scales: range from bad to excellent
10. Survey structure: personal & in-depth questions at the end
3
