econ stor
Make Your Publications Visible.A Service of
zbwLeibniz-Informationszentrum
Wirtschaft
Leibniz Information Centre
for Economics
Lynn, Peter; Hope, Steven; Jäckle, Annette; Campanelli, Pamela; Nicolaas,
Gerry
Working Paper
Effects of visual and aural communication of
categorical response options on answers to survey
questions
ISER Working Paper Series, No. 2012-21
Provided in Cooperation with:
Institute for Social and Economic Research (ISER), University of Essex
Suggested Citation: Lynn, Peter; Hope, Steven; Jäckle, Annette; Campanelli, Pamela; Nicolaas,
Gerry (2012) : Effects of visual and aural communication of categorical response options on
answers to survey questions, ISER Working Paper Series, No. 2012-21, University of Essex,
Institute for Social and Economic Research (ISER), Colchester
This Version is available at:
http://hdl.handle.net/10419/65994
Standard-Nutzungsbedingungen:
Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen
Zwecken und zum Privatgebrauch gespeichert und kopiert werden.
Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle
Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich
machen, vertreiben oder anderweitig nutzen.
Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen
(insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten,
gelten abweichend von diesen Nutzungsbedingungen die in der dort
genannten Lizenz gewährten Nutzungsrechte.Terms of use:
Documents in EconStor may be saved and copied for your
personal and scholarly purposes.
You are not to copy documents for public or commercial
purposes, to exhibit the documents publicly, to make them
publicly available on the internet, or to distribute or otherwise
use the documents in public.
If the documents have been made available under an Open
Content Licence (especially Creative Commons Licences), you
may exercise further usage rights as specified in the indicated
licence.
www.econstor.eu 8 Annette Jäckle University of Essex No. 2012- 21 October 2012 ISER Working Paper Series ER Working Paper Series www.iser.essex.ac.uk ww.iser.essex.ac.uk Effects of Visual and Aural Communication of Categorical R esponse Options on Answers to Survey Questions Pamela Campanelli The Survey Coach Steven Hope Institute for Social and Economic Research University College London Peter Lynn Institute for Social and Economic Research University of Essex Gerry Nicolaas NatCen Social Research Non-Technical Summary The way that people answer questions can depend on whether they have had to listen to the question being read out or have had to read it for themselves. Both of these ways of asking questions are used in surveys. And sometimes surveys require the people taking part to both listen and read, for example when an interviewer reads out the question but asks the respondent to pick their answer from those listed on a card. These different ways of being asked a question make a difference to how we think about the question. As a result, some people may be more likely to fully understand a question if it is asked in one way rather than another. Other people may be more likely to give a thoughtful, considered answer rather than a hasty unconsidered reaction if the question is asked in one way rather than another. In this paper we investigate the effect on the answers received to survey questions of presenting response options visually rather than relying on the respondent hearing them read out by an interviewer. We try to identify the average effect, across a sample of people taking part in a survey, and also to identify whether particular types of people, such as those with greater mental capacity, are affected differently from others. We do this for several different types of survey questions, including questions about behaviour, attitudes and satisfaction. For several survey questions w e find evidence that presenting response options visually or orally does indeed make a difference to the answers obtained. Furthermore, we find that this effect is not uniform across all the respondents in the sample. Rather, it depends on the respondent‟s mental ability and on how motivated they are to answer the questions to the best of their ability. We also identify a need for further research as there are some aspects of the process of asking survey questions about which we still do not know very much. Effects of visual and aural communication of categorical response options on answers to survey questions Peter Lynn Institute for Social and Economic Research, University of Essex Steven Hope University College London Annette Jäckle Institute for Social and Economic Research, University of Essex Pamela Campanelli The Survey Coach Gerry Nicolaas NatCen Social Research Abstract : Whether questions and answers are transmitted between interviewer and respondent by visual or aural communication can affect the responses given. We hypothesise that communication channel can affect either the respondent‟s understand ing of the question or the tendency to satisfice. These effects may be mediated by the cognitive ability and motivation of the respondent and by the type and difficulty of the question. We test our hypotheses using data from a large-scale controlled experiment. We find support for the notion that visual presentation improves understanding of the question and reduces the tendency to satisfice. We also find that effects are stronger for respondents of higher cognitive ability and for motivated respondents. Keywords : cognitive ability, end-labelled scales, primacy effect, recency effect, respondent motivation, response order, satisficing JEL Codes : C81, C83 Acknowledgments : This research was funded by the Survey Design and Measurement initiative of the UK Eco nomic and Social Research Council (ESRC) via a grant for the project “Mixe d modes and measurement error”, principal investigator Gerry Nicolaas (award no. RES -175-25-0007). We are grateful for input to the early stages of this project from our colleagues Rebecca Taylor, Alita Nandi and Nayantara Dutt. Preliminary versions of this paper were presented at the conference of the European Survey Research Association, Lausanne, July 2011, at a meeting of the social sta tistics section of the Royal Statistical Society, London, October 2011, and at a seminar at the University of Essex, February 2012. We acknowledge the useful comments of participants at these events. Contact:
Make Your Publications Visible.A Service of
zbwLeibniz-Informationszentrum
Wirtschaft
Leibniz Information Centre
for Economics
Lynn, Peter; Hope, Steven; Jäckle, Annette; Campanelli, Pamela; Nicolaas,
Gerry
Working Paper
Effects of visual and aural communication of
categorical response options on answers to survey
questions
ISER Working Paper Series, No. 2012-21
Provided in Cooperation with:
Institute for Social and Economic Research (ISER), University of Essex
Suggested Citation: Lynn, Peter; Hope, Steven; Jäckle, Annette; Campanelli, Pamela; Nicolaas,
Gerry (2012) : Effects of visual and aural communication of categorical response options on
answers to survey questions, ISER Working Paper Series, No. 2012-21, University of Essex,
Institute for Social and Economic Research (ISER), Colchester
This Version is available at:
http://hdl.handle.net/10419/65994
Standard-Nutzungsbedingungen:
Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen
Zwecken und zum Privatgebrauch gespeichert und kopiert werden.
Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle
Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich
machen, vertreiben oder anderweitig nutzen.
Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen
(insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten,
gelten abweichend von diesen Nutzungsbedingungen die in der dort
genannten Lizenz gewährten Nutzungsrechte.Terms of use:
Documents in EconStor may be saved and copied for your
personal and scholarly purposes.
You are not to copy documents for public or commercial
purposes, to exhibit the documents publicly, to make them
publicly available on the internet, or to distribute or otherwise
use the documents in public.
If the documents have been made available under an Open
Content Licence (especially Creative Commons Licences), you
may exercise further usage rights as specified in the indicated
licence.
www.econstor.eu 8 Annette Jäckle University of Essex No. 2012- 21 October 2012 ISER Working Paper Series ER Working Paper Series www.iser.essex.ac.uk ww.iser.essex.ac.uk Effects of Visual and Aural Communication of Categorical R esponse Options on Answers to Survey Questions Pamela Campanelli The Survey Coach Steven Hope Institute for Social and Economic Research University College London Peter Lynn Institute for Social and Economic Research University of Essex Gerry Nicolaas NatCen Social Research Non-Technical Summary The way that people answer questions can depend on whether they have had to listen to the question being read out or have had to read it for themselves. Both of these ways of asking questions are used in surveys. And sometimes surveys require the people taking part to both listen and read, for example when an interviewer reads out the question but asks the respondent to pick their answer from those listed on a card. These different ways of being asked a question make a difference to how we think about the question. As a result, some people may be more likely to fully understand a question if it is asked in one way rather than another. Other people may be more likely to give a thoughtful, considered answer rather than a hasty unconsidered reaction if the question is asked in one way rather than another. In this paper we investigate the effect on the answers received to survey questions of presenting response options visually rather than relying on the respondent hearing them read out by an interviewer. We try to identify the average effect, across a sample of people taking part in a survey, and also to identify whether particular types of people, such as those with greater mental capacity, are affected differently from others. We do this for several different types of survey questions, including questions about behaviour, attitudes and satisfaction. For several survey questions w e find evidence that presenting response options visually or orally does indeed make a difference to the answers obtained. Furthermore, we find that this effect is not uniform across all the respondents in the sample. Rather, it depends on the respondent‟s mental ability and on how motivated they are to answer the questions to the best of their ability. We also identify a need for further research as there are some aspects of the process of asking survey questions about which we still do not know very much. Effects of visual and aural communication of categorical response options on answers to survey questions Peter Lynn Institute for Social and Economic Research, University of Essex Steven Hope University College London Annette Jäckle Institute for Social and Economic Research, University of Essex Pamela Campanelli The Survey Coach Gerry Nicolaas NatCen Social Research Abstract : Whether questions and answers are transmitted between interviewer and respondent by visual or aural communication can affect the responses given. We hypothesise that communication channel can affect either the respondent‟s understand ing of the question or the tendency to satisfice. These effects may be mediated by the cognitive ability and motivation of the respondent and by the type and difficulty of the question. We test our hypotheses using data from a large-scale controlled experiment. We find support for the notion that visual presentation improves understanding of the question and reduces the tendency to satisfice. We also find that effects are stronger for respondents of higher cognitive ability and for motivated respondents. Keywords : cognitive ability, end-labelled scales, primacy effect, recency effect, respondent motivation, response order, satisficing JEL Codes : C81, C83 Acknowledgments : This research was funded by the Survey Design and Measurement initiative of the UK Eco nomic and Social Research Council (ESRC) via a grant for the project “Mixe d modes and measurement error”, principal investigator Gerry Nicolaas (award no. RES -175-25-0007). We are grateful for input to the early stages of this project from our colleagues Rebecca Taylor, Alita Nandi and Nayantara Dutt. Preliminary versions of this paper were presented at the conference of the European Survey Research Association, Lausanne, July 2011, at a meeting of the social sta tistics section of the Royal Statistical Society, London, October 2011, and at a seminar at the University of Essex, February 2012. We acknowledge the useful comments of participants at these events. Contact:
