8
ER Working Paper Series
ISER Working Paper Series
Effects of Visual and Aural Communication
of Categorical Response Options on
Answers to Survey Questions
Peter Lynn
Institute for Social and Economic Research
University of Essex
Steven Hope
University College London
Annette Jäckle
Institute for Social and Economic Research
University of Essex
Pamela Campanelli
The Survey Coach
Gerry Nicolaas
NatCen Social Research
No. 2012-21
October 2012
ww.iser.essex.ac.uk
www.iser.essex.ac.uk
, Non-Technical Summary
The way that people answer questions can depend on whether they have had to listen to the
question being read out or have had to read it for themselves. Both of these ways of asking
questions are used in surveys. And sometimes surveys require the people taking part to both
listen and read, for example when an interviewer reads out the question but asks the
respondent to pick their answer from those listed on a card.
These different ways of being asked a question make a difference to how we think about the
question. As a result, some people may be more likely to fully understand a question if it is
asked in one way rather than another. Other people may be more likely to give a thoughtful,
considered answer rather than a hasty unconsidered reaction if the question is asked in one
way rather than another.
In this paper we investigate the effect on the answers received to survey questions of
presenting response options visually rather than relying on the respondent hearing them read
out by an interviewer. We try to identify the average effect, across a sample of people
taking part in a survey, and also to identify whether particular types of people, such as those
with greater mental capacity, are affected differently from others. We do this for several
different types of survey questions, including questions about behaviour, attitudes and
satisfaction.
For several survey questions we find evidence that presenting response options visually or
orally does indeed make a difference to the answers obtained. Furthermore, we find that this
effect is not uniform across all the respondents in the sample. Rather, it depends on the
respondent‟s mental ability and on how motivated they are to answer the questions to the
best of their ability. We also identify a need for further research as there are some aspects of
the process of asking survey questions about which we still do not know very much.
, Effects of visual and aural communication of categorical response options on
answers to survey questions
Peter Lynn
Institute for Social and Economic Research, University of Essex
Steven Hope
University College London
Annette Jäckle
Institute for Social and Economic Research, University of Essex
Pamela Campanelli
The Survey Coach
Gerry Nicolaas
NatCen Social Research
Abstract: Whether questions and answers are transmitted between interviewer and
respondent by visual or aural communication can affect the responses given. We
hypothesise that communication channel can affect either the respondent‟s understanding of
the question or the tendency to satisfice. These effects may be mediated by the cognitive
ability and motivation of the respondent and by the type and difficulty of the question. We
test our hypotheses using data from a large-scale controlled experiment. We find support for
the notion that visual presentation improves understanding of the question and reduces the
tendency to satisfice. We also find that effects are stronger for respondents of higher
cognitive ability and for motivated respondents.
Keywords: cognitive ability, end-labelled scales, primacy effect, recency effect, respondent
motivation, response order, satisficing
JEL Codes: C81, C83
Acknowledgments: This research was funded by the Survey Design and Measurement initiative of
the UK Economic and Social Research Council (ESRC) via a grant for the project “Mixed modes
and measurement error”, principal investigator Gerry Nicolaas (award no. RES-175-25-0007). We
are grateful for input to the early stages of this project from our colleagues Rebecca Taylor, Alita
Nandi and Nayantara Dutt. Preliminary versions of this paper were presented at the conference of
the European Survey Research Association, Lausanne, July 2011, at a meeting of the social statistics
section of the Royal Statistical Society, London, October 2011, and at a seminar at the University of
Essex, February 2012. We acknowledge the useful comments of participants at these events.
Contact:
, 1. Introduction
Several decades ago, survey researchers became aware that the responses provided to survey
questions could differ depending on the mode of administration of the survey. Early research
documented the nature of the differences in responses between data collection modes (Payne
1951, Sudman & Bradburn 1974), but soon researchers wanted to go further and to
understand the reasons for these differences. One possible explanation was that the way
people process information depends on whether that information is received through visual or
auditory channels (Duncan 1969). In this article we are concerned with the role in the
question-answering process of two channels of communication, visual and oral/aural. We
focus on survey items with categorical response options, either nominal or ordinal.
Previous research into the effects of communication channel on responses to such items has
shown that aural communication of response options can tend to produce recency effects,
whereby respondents have an increased tendency to choose (one of) the last option(s)
presented, while visual communication can tend to produce primacy effects, an increased
tendency to choose (one of) the first option(s) (Krosnick & Alwin, 1987), though observation
of such effects is far from universal. These effects are generally believed to represent a
reduction in the accuracy of responses, as the effects would be absent if respondents
processed fully and equally all response options. The effects are therefore a manifestation of
satisficing (Krosnick, 1991). Whether or not primacy and recency effects are observed may
depend on factors such as the time and effort that would be required to fully process all
options (Schwarz et al, 1992) the degree of uncertainty the respondent has regarding their
true state, and the degree of ambiguity regarding the distinction between response categories.
However, the role of these factors and others in influencing response-order effects is not well
understood and a number of inconsistent findings from previous research studies have not
been explained. Dillman et al (2009, p.316) conclude that “…much remains to be learned
about the conditions under which primacy and recency occur in surveys, some aspects of
which may be entangled with other effects of visual versus aural communication of survey
questions.”
Visual display of information may make the task of answering a survey question easier for
survey respondents (Schwarz et al 1991). Visual display of response options provides cues
and allows respondents to review the options at their own speed, in the order they choose, and
1
ER Working Paper Series
ISER Working Paper Series
Effects of Visual and Aural Communication
of Categorical Response Options on
Answers to Survey Questions
Peter Lynn
Institute for Social and Economic Research
University of Essex
Steven Hope
University College London
Annette Jäckle
Institute for Social and Economic Research
University of Essex
Pamela Campanelli
The Survey Coach
Gerry Nicolaas
NatCen Social Research
No. 2012-21
October 2012
ww.iser.essex.ac.uk
www.iser.essex.ac.uk
, Non-Technical Summary
The way that people answer questions can depend on whether they have had to listen to the
question being read out or have had to read it for themselves. Both of these ways of asking
questions are used in surveys. And sometimes surveys require the people taking part to both
listen and read, for example when an interviewer reads out the question but asks the
respondent to pick their answer from those listed on a card.
These different ways of being asked a question make a difference to how we think about the
question. As a result, some people may be more likely to fully understand a question if it is
asked in one way rather than another. Other people may be more likely to give a thoughtful,
considered answer rather than a hasty unconsidered reaction if the question is asked in one
way rather than another.
In this paper we investigate the effect on the answers received to survey questions of
presenting response options visually rather than relying on the respondent hearing them read
out by an interviewer. We try to identify the average effect, across a sample of people
taking part in a survey, and also to identify whether particular types of people, such as those
with greater mental capacity, are affected differently from others. We do this for several
different types of survey questions, including questions about behaviour, attitudes and
satisfaction.
For several survey questions we find evidence that presenting response options visually or
orally does indeed make a difference to the answers obtained. Furthermore, we find that this
effect is not uniform across all the respondents in the sample. Rather, it depends on the
respondent‟s mental ability and on how motivated they are to answer the questions to the
best of their ability. We also identify a need for further research as there are some aspects of
the process of asking survey questions about which we still do not know very much.
, Effects of visual and aural communication of categorical response options on
answers to survey questions
Peter Lynn
Institute for Social and Economic Research, University of Essex
Steven Hope
University College London
Annette Jäckle
Institute for Social and Economic Research, University of Essex
Pamela Campanelli
The Survey Coach
Gerry Nicolaas
NatCen Social Research
Abstract: Whether questions and answers are transmitted between interviewer and
respondent by visual or aural communication can affect the responses given. We
hypothesise that communication channel can affect either the respondent‟s understanding of
the question or the tendency to satisfice. These effects may be mediated by the cognitive
ability and motivation of the respondent and by the type and difficulty of the question. We
test our hypotheses using data from a large-scale controlled experiment. We find support for
the notion that visual presentation improves understanding of the question and reduces the
tendency to satisfice. We also find that effects are stronger for respondents of higher
cognitive ability and for motivated respondents.
Keywords: cognitive ability, end-labelled scales, primacy effect, recency effect, respondent
motivation, response order, satisficing
JEL Codes: C81, C83
Acknowledgments: This research was funded by the Survey Design and Measurement initiative of
the UK Economic and Social Research Council (ESRC) via a grant for the project “Mixed modes
and measurement error”, principal investigator Gerry Nicolaas (award no. RES-175-25-0007). We
are grateful for input to the early stages of this project from our colleagues Rebecca Taylor, Alita
Nandi and Nayantara Dutt. Preliminary versions of this paper were presented at the conference of
the European Survey Research Association, Lausanne, July 2011, at a meeting of the social statistics
section of the Royal Statistical Society, London, October 2011, and at a seminar at the University of
Essex, February 2012. We acknowledge the useful comments of participants at these events.
Contact:
, 1. Introduction
Several decades ago, survey researchers became aware that the responses provided to survey
questions could differ depending on the mode of administration of the survey. Early research
documented the nature of the differences in responses between data collection modes (Payne
1951, Sudman & Bradburn 1974), but soon researchers wanted to go further and to
understand the reasons for these differences. One possible explanation was that the way
people process information depends on whether that information is received through visual or
auditory channels (Duncan 1969). In this article we are concerned with the role in the
question-answering process of two channels of communication, visual and oral/aural. We
focus on survey items with categorical response options, either nominal or ordinal.
Previous research into the effects of communication channel on responses to such items has
shown that aural communication of response options can tend to produce recency effects,
whereby respondents have an increased tendency to choose (one of) the last option(s)
presented, while visual communication can tend to produce primacy effects, an increased
tendency to choose (one of) the first option(s) (Krosnick & Alwin, 1987), though observation
of such effects is far from universal. These effects are generally believed to represent a
reduction in the accuracy of responses, as the effects would be absent if respondents
processed fully and equally all response options. The effects are therefore a manifestation of
satisficing (Krosnick, 1991). Whether or not primacy and recency effects are observed may
depend on factors such as the time and effort that would be required to fully process all
options (Schwarz et al, 1992) the degree of uncertainty the respondent has regarding their
true state, and the degree of ambiguity regarding the distinction between response categories.
However, the role of these factors and others in influencing response-order effects is not well
understood and a number of inconsistent findings from previous research studies have not
been explained. Dillman et al (2009, p.316) conclude that “…much remains to be learned
about the conditions under which primacy and recency occur in surveys, some aspects of
which may be entangled with other effects of visual versus aural communication of survey
questions.”
Visual display of information may make the task of answering a survey question easier for
survey respondents (Schwarz et al 1991). Visual display of response options provides cues
and allows respondents to review the options at their own speed, in the order they choose, and
1
