Methodology Lectures + readings 2020
LECTURE 1 INTRODUCTION H1 GETTING STARTED
Scientific research
● Attempts to describe and explain situations and behavior
First observe and see what’s happening, describing is an important part of the scientific research
and then explain
○ Different types of research questions
● Uses different types of data to do so
○ Different types of research methods
When should we believe an explanation? We never believe something 100%
● Only if the research is reliable
● Only if the research is valid
● Always provisionally (someone might come up with a better explanation)
● “In science, ‘fact’ can only mean ‘confirmed to such a degree that it would be perverse to
withhold provisional assent’” - Stephen Jay Gloud
Scientists
● Have strict rules to try and guarantee the quality of academic research
○ This course is an introduction into these rules
● Use theoretical insights to interpret observations
○ Courses like POL HCI BTI offer you theoretical insights
Methodology Q&A 1 Formulating a research question
● Not a question for which you can ‘look up’ the answer
● Clear definitions of all the parts of your question are necessary
o E.g.: how have companies changed their advertising campaigns during Covid: which
companies, what kinds of companies? Where, in which countries? What types of
campaigns, in which media? Changed compared to what / when?
● Questions that can be answered, tend to be specific questions (not the underlying, big ones)
● So research questions are specific; the data you collect / analyze in your study answer that
particular question
● The study contributes to a larger issue / question / problem
Voorbeeld vraag:
Quality of teaching
You need to define what is quality of teaching
What do I mean here and what are the definitions, you need to make choices on what you focus. For
ex you can focus on student results for quality of teching or on students opinion but its not one on
one linked because students have a bias and results do not only come from quality of teching
LECTURE 2 Worldviews, quantitative/qualitative, research purposes H1 & 2
Worldviews - People have different takes on what the purpose of research should be, which shapes
their approaches. Being aware of your own worldview allows you to refine your research questions.
● Worldview 1 / Nomothetic approach:
(from Greek, nomos = law, thetēs = one who establishes)
- Human behavior is generalizable, predictable, and motivated
- Understanding behavior is best done by isolating factors
, ● Worldview 2 / Idiographic approach:
(fromGreek, idios= own, private, graphein= towrite)
- Each person is unique, unpredictable, and self-motivated
- Understanding behavior is best done from the participant’s perspective and by considering
the whole situation
- Often qualitative
It’s not all binary choices
• More nuanced categorization of worldviews (e.g., postpositive, constructivist, transformative,
pragmatic)
• Also: research traditions, angles of approach (see list p. 26, rhetorical… critical)
• And: general tendency in many fields towards
• More attentional being paid to individual and cultural differences
• More awareness of WEIRD biases … see also week 14: the future of science
Smart starting points for research
• Look at real examples of the communicative setting / product etc. that you are interested in
• Consider your own (personal) experiences with and assumptions and preconceptions about
the topic
• Map the different stakeholders and relevant factors that you can think of
• Read existing research literature
• Define your research purpose (exploration, description, prediction, control, interpretation,
criticism)
• Formulate your research question
Research methods
Quantitative research methods > are built around the idea that human traits and behaviors can be
translated into numbers to analyze them. Things you can count
Qualitative research methods > are built around the idea that human traits and behaviors are best
understood by observing and interviewing people.
Triangulation > the act of combing research methods to neutralize the weaknesses of single research
methods. Quantitative and qualitative or quantitative data’s combined
Quantitative and qualitative research methods
Typically, quantitative methods:
Experiments – AB tests, manipulate something and measure it
Surveys
Content analysis
(Meta-analysis)
(Computational modeling)
Typically, qualitative methods:
Interviews
Focus groups
Case studies
Observational studies
Creative methods in new media design
, LECTURE 3 Theory, hypothesis, induction/deduction, variables H2
Properties of a good theory
● Can be falsified
● Generates hypotheses
o E.g., Verbal cues will have a stronger influence on impression formation in a digital
setting than in a face-to-face setting
o Really clear in what you compare and contrast - What is your exact expectation?
● Covers a broad area
o ‘people will base their impressions on the available cues in interaction’is broader
than‘people will be very critical of spelling errors in online dating profile texts’
● Is substantiated by converging evidence
o When different types of data and studies point in the same direction, the theory is
more plausible
Hypotheses
● Specify the relation we expect to find, e.g.:
o Correlation – when two things are related but the one doesn’t cause the other
o Causation – When you can claim that one thing causes the other
o One-tailed – You commit to predicting the directions of the direction of the
relationship
o two-tailed – Relationship between two variables but don’t specify the direction of
the relationship
● Are not usually said to be true or false but supported to a certain extent
o You say that your data support a certain hypothesis or is in line with the hypothesis.
● Can be tested
Variables
● In a typical study, you are interested in the relation between different concepts
● When these concepts are abstract ideas, such as ‘attitude’, ‘motivation’ or ‘appreciation’, they
are often referred to as constructs.
● In the context of a study, variables are usually measured or manipulated
Variables: Dependent (DV)
Or the outcome variable. The variable whose variation in relation to other variables the researcher is
interested in. (Typical DVs are: user satisfaction, reaction time, purchase intention, test performance,
etc. etc.)
Variables: Independent (IV)
● Variables that might cause or correlate with variation in the outcome variable. Often
manipulated by the researcher, but can also be observed (e.g., age, gender, cultural
background).
● In most studies you will read, IVs have 2 or 3 levels (but more levels are certainly possible)
Language error study: 1 IV, two levels
(errors present; errors absent
Variables
Hypothesis: profile texts with language errors
lead to more negative impressions about the
profile owner than texts that do not.
LECTURE 1 INTRODUCTION H1 GETTING STARTED
Scientific research
● Attempts to describe and explain situations and behavior
First observe and see what’s happening, describing is an important part of the scientific research
and then explain
○ Different types of research questions
● Uses different types of data to do so
○ Different types of research methods
When should we believe an explanation? We never believe something 100%
● Only if the research is reliable
● Only if the research is valid
● Always provisionally (someone might come up with a better explanation)
● “In science, ‘fact’ can only mean ‘confirmed to such a degree that it would be perverse to
withhold provisional assent’” - Stephen Jay Gloud
Scientists
● Have strict rules to try and guarantee the quality of academic research
○ This course is an introduction into these rules
● Use theoretical insights to interpret observations
○ Courses like POL HCI BTI offer you theoretical insights
Methodology Q&A 1 Formulating a research question
● Not a question for which you can ‘look up’ the answer
● Clear definitions of all the parts of your question are necessary
o E.g.: how have companies changed their advertising campaigns during Covid: which
companies, what kinds of companies? Where, in which countries? What types of
campaigns, in which media? Changed compared to what / when?
● Questions that can be answered, tend to be specific questions (not the underlying, big ones)
● So research questions are specific; the data you collect / analyze in your study answer that
particular question
● The study contributes to a larger issue / question / problem
Voorbeeld vraag:
Quality of teaching
You need to define what is quality of teaching
What do I mean here and what are the definitions, you need to make choices on what you focus. For
ex you can focus on student results for quality of teching or on students opinion but its not one on
one linked because students have a bias and results do not only come from quality of teching
LECTURE 2 Worldviews, quantitative/qualitative, research purposes H1 & 2
Worldviews - People have different takes on what the purpose of research should be, which shapes
their approaches. Being aware of your own worldview allows you to refine your research questions.
● Worldview 1 / Nomothetic approach:
(from Greek, nomos = law, thetēs = one who establishes)
- Human behavior is generalizable, predictable, and motivated
- Understanding behavior is best done by isolating factors
, ● Worldview 2 / Idiographic approach:
(fromGreek, idios= own, private, graphein= towrite)
- Each person is unique, unpredictable, and self-motivated
- Understanding behavior is best done from the participant’s perspective and by considering
the whole situation
- Often qualitative
It’s not all binary choices
• More nuanced categorization of worldviews (e.g., postpositive, constructivist, transformative,
pragmatic)
• Also: research traditions, angles of approach (see list p. 26, rhetorical… critical)
• And: general tendency in many fields towards
• More attentional being paid to individual and cultural differences
• More awareness of WEIRD biases … see also week 14: the future of science
Smart starting points for research
• Look at real examples of the communicative setting / product etc. that you are interested in
• Consider your own (personal) experiences with and assumptions and preconceptions about
the topic
• Map the different stakeholders and relevant factors that you can think of
• Read existing research literature
• Define your research purpose (exploration, description, prediction, control, interpretation,
criticism)
• Formulate your research question
Research methods
Quantitative research methods > are built around the idea that human traits and behaviors can be
translated into numbers to analyze them. Things you can count
Qualitative research methods > are built around the idea that human traits and behaviors are best
understood by observing and interviewing people.
Triangulation > the act of combing research methods to neutralize the weaknesses of single research
methods. Quantitative and qualitative or quantitative data’s combined
Quantitative and qualitative research methods
Typically, quantitative methods:
Experiments – AB tests, manipulate something and measure it
Surveys
Content analysis
(Meta-analysis)
(Computational modeling)
Typically, qualitative methods:
Interviews
Focus groups
Case studies
Observational studies
Creative methods in new media design
, LECTURE 3 Theory, hypothesis, induction/deduction, variables H2
Properties of a good theory
● Can be falsified
● Generates hypotheses
o E.g., Verbal cues will have a stronger influence on impression formation in a digital
setting than in a face-to-face setting
o Really clear in what you compare and contrast - What is your exact expectation?
● Covers a broad area
o ‘people will base their impressions on the available cues in interaction’is broader
than‘people will be very critical of spelling errors in online dating profile texts’
● Is substantiated by converging evidence
o When different types of data and studies point in the same direction, the theory is
more plausible
Hypotheses
● Specify the relation we expect to find, e.g.:
o Correlation – when two things are related but the one doesn’t cause the other
o Causation – When you can claim that one thing causes the other
o One-tailed – You commit to predicting the directions of the direction of the
relationship
o two-tailed – Relationship between two variables but don’t specify the direction of
the relationship
● Are not usually said to be true or false but supported to a certain extent
o You say that your data support a certain hypothesis or is in line with the hypothesis.
● Can be tested
Variables
● In a typical study, you are interested in the relation between different concepts
● When these concepts are abstract ideas, such as ‘attitude’, ‘motivation’ or ‘appreciation’, they
are often referred to as constructs.
● In the context of a study, variables are usually measured or manipulated
Variables: Dependent (DV)
Or the outcome variable. The variable whose variation in relation to other variables the researcher is
interested in. (Typical DVs are: user satisfaction, reaction time, purchase intention, test performance,
etc. etc.)
Variables: Independent (IV)
● Variables that might cause or correlate with variation in the outcome variable. Often
manipulated by the researcher, but can also be observed (e.g., age, gender, cultural
background).
● In most studies you will read, IVs have 2 or 3 levels (but more levels are certainly possible)
Language error study: 1 IV, two levels
(errors present; errors absent
Variables
Hypothesis: profile texts with language errors
lead to more negative impressions about the
profile owner than texts that do not.
