Construction. and Analysis. or Questionnaires
Lec 1. Introduction to questionnaires
● Questionnaires are used by…
○ Governments
○ Orgnaizations
○ Companies
○ Researchers
● Definition of a questionnaire
○ Encyclopedia: ‘a questionnaire is set of printed or written questions with a choice
of answers, devised for the purpose of a survey or statistical study’
○ Questionnaires are one of the main data collection tools in social science (an in
empirical research in general)
○ There are two types of performance measured
■ Maximum performance: what a person can do (when motivated)?
● Ex. aptitude tests, achievements tests, intelligence tests
● Focus on obtaining highest scores (determeined by abilities, e.g.,
cognition)
■ Typical performance (i.e., typical behaviour): what a person will do?
● Ex. personality questionnaires, behavioural questionnaires
● Focus on obtaining representative response (no right or wrong
answers)
● Maximum vs typical performance
○ Maximum performance item (achievemnt test)
■
○ Typical performance item (questionnaire)
, ■
● What is an ‘item’?
○ Item: statement 9or question) and the response alternatives
○
● What is the purpose of a questionnaire?
○ One measurement instrument (in a questionnaire or test)... tries to isolate one
characteristic and tries to quantify to what extent someone ‘has that’
■ Gives an incomplete description of reality (on purpose)
○ Ex. measurement instrument (WAIS)
■
○ EX of a multi-itme psychometric scale as use in questionnaires
■
● Questionnaires contain multiple ‘scales’
○ Questionnaire often consists of different scales ( a coherent and often fixed set of
items):
, ■ E.g., questionnaire to measure employee satisfaction:
● Scale 1: measures work atmosphere (Items 1-20 of the
questionnaire)
● Scale 2: Measures satisfaction with work tasks (Item 21-40 of the
questionnaire)
○ In the end, we are interested in answering research questions and testing
hypoethesises
■ E.g., are intelligent people more likely to be depressed?
● ‘Ignorance is bliss’: example using questionnaires
○ Hypothesis: intelligent people are more likely to be depressed
○ We can test this by looking at the relationship between intelligence and
depression (prediction)e.g.”
■ Measuing intelligence and depression (latent → not directly observable)
■ Constructs that we cannot directly observed must be operationalized &
made measurable
■ For measuring, we can use questionnaires
● Measurement in technical sciences
○ Measuring physical characteristics such as height is not difficult
○ Often we are interested in traits that not directly observable
■ E.g., intelligence of a potential employee (HR) or mental-health related
quality of life in different countries (sociology) or the working culture in
different organizations (organization sciences)
○ Questionnaires can be seen as an approximation of ‘something’ that cannot be
precisely measured
○ Questionnaires help us to make many very important decisions! (e.g., selection of
candidates, continuation of therapy)
■ They must be of high quality → valid and reliable
● Validity
○ Validity indicates whether a studys findings are trustworthy
■ Is what we measure truely what we intend to measure?
○ I.e., intelligence meausted by ‘asking friend’ → valid?
● Reliability
○ Reliability is about the precision and repeatability of the measurement
■ Will I always get consistent results (can I replicate my result) if I measure
something repeatedly and under the same conditions?
■ Is there a lot of random measurement error?
○ I.e., bathroom scales (weight of child vs weight of child holdinga dog) →
reliable?
○ I.e., If Klass has a high IQscore based on test A, then we expect that he will have
ahigh (similar) IQ score when tested at a different moment witht the same test
, ■ If the same result can be consistently achieved by using the same methods
under the same circumstances, the measurement is considered reliable
● Content of questionnaries
○ Behaviour i.e., what people do (e.g., asking mothers with school-age chidlren; do
you work?)
○ Beliefs i.e., what people believe to be true/false (e.g., in your opinion, what
percentage of mothers of school-age children work?)
○ opinions/values/attitudes: what do people believe is desirable?
○ Demographics: get information about the respondents characteristics (gender,
age, type of employment, socio-economic status)
● Questionnaires: modes and question types
○ Some features of questionnaires
■ Mode of questionnaire administration:
● Interactional mode: with interviewer or self-adminsitered
● Technological mode: telephone, computer, internet, paper and
pencil
● Mix of adminsatation modes
■ Quesiton type: open-ended vs closed-ended questions
● open-ended questions are questions that allow someone to give a
free-form answer
● closed-ended questions can be answered with a ‘yes’ or ‘no’ or
they have a limited set of possible answers
● Questionnaires pros and cons
○ Pros
■ Very fast and efficient way to collect data
■ Cheap (especially if purchasing is doen digitally)
■ Suitable for research in large samples
○ Cons
■ Low response rates (especially for online surveys)
■ Answer to questions may deviate from facts/behaviour (e.g., due to social
desirability)
● Survey
○ technique/research method for collection information characterized by
■ Structured or systematic set of data (varaiale x case data grid)
■ Investigating way may be responsible for a particular phenomenon
○ It is possible to collect information using a questionnaire, by interviewing
(qualitative surveys) or observing (e.g., experiemetns)
■ –< survey dont equal questionnaire
Lec 1. Introduction to questionnaires
● Questionnaires are used by…
○ Governments
○ Orgnaizations
○ Companies
○ Researchers
● Definition of a questionnaire
○ Encyclopedia: ‘a questionnaire is set of printed or written questions with a choice
of answers, devised for the purpose of a survey or statistical study’
○ Questionnaires are one of the main data collection tools in social science (an in
empirical research in general)
○ There are two types of performance measured
■ Maximum performance: what a person can do (when motivated)?
● Ex. aptitude tests, achievements tests, intelligence tests
● Focus on obtaining highest scores (determeined by abilities, e.g.,
cognition)
■ Typical performance (i.e., typical behaviour): what a person will do?
● Ex. personality questionnaires, behavioural questionnaires
● Focus on obtaining representative response (no right or wrong
answers)
● Maximum vs typical performance
○ Maximum performance item (achievemnt test)
■
○ Typical performance item (questionnaire)
, ■
● What is an ‘item’?
○ Item: statement 9or question) and the response alternatives
○
● What is the purpose of a questionnaire?
○ One measurement instrument (in a questionnaire or test)... tries to isolate one
characteristic and tries to quantify to what extent someone ‘has that’
■ Gives an incomplete description of reality (on purpose)
○ Ex. measurement instrument (WAIS)
■
○ EX of a multi-itme psychometric scale as use in questionnaires
■
● Questionnaires contain multiple ‘scales’
○ Questionnaire often consists of different scales ( a coherent and often fixed set of
items):
, ■ E.g., questionnaire to measure employee satisfaction:
● Scale 1: measures work atmosphere (Items 1-20 of the
questionnaire)
● Scale 2: Measures satisfaction with work tasks (Item 21-40 of the
questionnaire)
○ In the end, we are interested in answering research questions and testing
hypoethesises
■ E.g., are intelligent people more likely to be depressed?
● ‘Ignorance is bliss’: example using questionnaires
○ Hypothesis: intelligent people are more likely to be depressed
○ We can test this by looking at the relationship between intelligence and
depression (prediction)e.g.”
■ Measuing intelligence and depression (latent → not directly observable)
■ Constructs that we cannot directly observed must be operationalized &
made measurable
■ For measuring, we can use questionnaires
● Measurement in technical sciences
○ Measuring physical characteristics such as height is not difficult
○ Often we are interested in traits that not directly observable
■ E.g., intelligence of a potential employee (HR) or mental-health related
quality of life in different countries (sociology) or the working culture in
different organizations (organization sciences)
○ Questionnaires can be seen as an approximation of ‘something’ that cannot be
precisely measured
○ Questionnaires help us to make many very important decisions! (e.g., selection of
candidates, continuation of therapy)
■ They must be of high quality → valid and reliable
● Validity
○ Validity indicates whether a studys findings are trustworthy
■ Is what we measure truely what we intend to measure?
○ I.e., intelligence meausted by ‘asking friend’ → valid?
● Reliability
○ Reliability is about the precision and repeatability of the measurement
■ Will I always get consistent results (can I replicate my result) if I measure
something repeatedly and under the same conditions?
■ Is there a lot of random measurement error?
○ I.e., bathroom scales (weight of child vs weight of child holdinga dog) →
reliable?
○ I.e., If Klass has a high IQscore based on test A, then we expect that he will have
ahigh (similar) IQ score when tested at a different moment witht the same test
, ■ If the same result can be consistently achieved by using the same methods
under the same circumstances, the measurement is considered reliable
● Content of questionnaries
○ Behaviour i.e., what people do (e.g., asking mothers with school-age chidlren; do
you work?)
○ Beliefs i.e., what people believe to be true/false (e.g., in your opinion, what
percentage of mothers of school-age children work?)
○ opinions/values/attitudes: what do people believe is desirable?
○ Demographics: get information about the respondents characteristics (gender,
age, type of employment, socio-economic status)
● Questionnaires: modes and question types
○ Some features of questionnaires
■ Mode of questionnaire administration:
● Interactional mode: with interviewer or self-adminsitered
● Technological mode: telephone, computer, internet, paper and
pencil
● Mix of adminsatation modes
■ Quesiton type: open-ended vs closed-ended questions
● open-ended questions are questions that allow someone to give a
free-form answer
● closed-ended questions can be answered with a ‘yes’ or ‘no’ or
they have a limited set of possible answers
● Questionnaires pros and cons
○ Pros
■ Very fast and efficient way to collect data
■ Cheap (especially if purchasing is doen digitally)
■ Suitable for research in large samples
○ Cons
■ Low response rates (especially for online surveys)
■ Answer to questions may deviate from facts/behaviour (e.g., due to social
desirability)
● Survey
○ technique/research method for collection information characterized by
■ Structured or systematic set of data (varaiale x case data grid)
■ Investigating way may be responsible for a particular phenomenon
○ It is possible to collect information using a questionnaire, by interviewing
(qualitative surveys) or observing (e.g., experiemetns)
■ –< survey dont equal questionnaire
