LECTURES EXPERIMENTAL RESEARCH 2023
Taught by: Femke van Horen
Table of content
Lecture 1 – Why experimental research? ................................................................................... 2
Lecture 2 – Experimental Research ............................................................................................ 4
Lecture 3 – Confounds Anova ..................................................................................................... 9
Lecture 4 – One-way ANOVA and two-way ANOVA ............................................................. 18
Lecture 5 – 3-way ANOVA, ANCOVA ..................................................................................... 26
Lecture 6 – Repeated measures mixed design .......................................................................... 32
Lecture 7 – Mediation ................................................................................................................. 40
Lecture 8 – Moderation with continuous variables – Spotlight Analysis .............................. 47
Lecture 9 – Power and effect size .............................................................................................. 58
Lecture 10 – Exam prep ............................................................................................................. 65
1
,Lecture 1 – Why experimental research?
Experimental research
Experimenting: describe, predict, and explain behavior of market-parties, employees, buyers.
→ break up phenomena in variables and relations between those variables.
A vs. B (treatment) testing
Conversion optimalisation
- A structured and systematic approach to improve the performance of a website
- Informed by data insights & psychology
- Taking the traffic there and making the most of it
Correlation & prediction
Behavioral research
1) Descriptive research
a. Thoughts, feelings, ideas, behaviors.
2) Correlational research
a. Identifying relationships between different observed variables: measuring
thoughts, feelings, behavior.
“Smoking mothers have more often problem children”.
b. Measures the association between 2 variables.
Correlation of -1 (negative association) to +1 (positive association).
2
, c. R = .80 → the more the mother smokes, the more often they have problem children.
R = -.80 → the more the mother smokes, the less often they have problem children.
d. Weaknesses:
Correlation ≠ causation.
Direction of the relationship?
There is a third explaining variable → spurious correlation.
3) Experimental
a. Testing causality, A → B?
“Impact of loyalty programs on sales?”
b. Only type of research that (potentially) can demonstrate that a change in one
variable causes a predictable change in another variable.
c. Most difficult: making sure that a change in Y was not caused by something else
than X.
Experimental research settings
- Field experimentation
o Real life setting
o Mundane reality: natural behavior, setting, treatment.
o Less control
- Laboratory experimentation
o More control
o Better able to manipulate variables
o No natural setting
Experimentation & causation
- Needs to be correlation between 2 variables
- Asymmetrical direction
- A (cause) → B (effect)
- Change in A is accompanied by a change in B
- No alternative explanation for the change in B than the change in A (other possible causes
are controlled for)
3
, Experimental research essence
Test specific hypotheses about relationship between cause and effect via controlled (laboratory)
conditions.
- The effect of an independent variable (IV) on a dependent variable (DV).
- Manipulating the independent variable.
- Measure effects on dependent variable.
- Control other influences (high internal but low external validity).
Importance of randomization
- Use all sorts of people with all sorts of characteristics.
- Potential confounds are under control.
- So: change in Y can be attributed to X.
Problem identification
Sources of research ideas:
- Real life experiences
- Previous research and theory
o Conflicting findings
o Boundary conditions (moderator)
o Find explanation for observed effects (mediator)
o Applying a theory to a consumer setting
Lecture 2 – Experimental Research
Full empirical cycle
Identify interesting and relevant research question – Pinpoint conceptual variables – Identify the
proper relations between them that fit your theorizing (= hypotheses) – Develop a research design
– Develop manipulations and measures for the IV’s, DV’s, mediators and moderators – Collect
data – Analyze and interpret the results – Write up study to submit
Experimental research: main steps
1) Theoretical framework
a. Problem identification
b. Hypothesis formulation
2) Experimental design
a. Manipulation of IVs
b. Measurement of DV
c. Control for confounds
3) Data analyses
a. Get familiar with your data
b. Checks
c. Conduct main analyses
d. Conduct follow up analyses
4
Taught by: Femke van Horen
Table of content
Lecture 1 – Why experimental research? ................................................................................... 2
Lecture 2 – Experimental Research ............................................................................................ 4
Lecture 3 – Confounds Anova ..................................................................................................... 9
Lecture 4 – One-way ANOVA and two-way ANOVA ............................................................. 18
Lecture 5 – 3-way ANOVA, ANCOVA ..................................................................................... 26
Lecture 6 – Repeated measures mixed design .......................................................................... 32
Lecture 7 – Mediation ................................................................................................................. 40
Lecture 8 – Moderation with continuous variables – Spotlight Analysis .............................. 47
Lecture 9 – Power and effect size .............................................................................................. 58
Lecture 10 – Exam prep ............................................................................................................. 65
1
,Lecture 1 – Why experimental research?
Experimental research
Experimenting: describe, predict, and explain behavior of market-parties, employees, buyers.
→ break up phenomena in variables and relations between those variables.
A vs. B (treatment) testing
Conversion optimalisation
- A structured and systematic approach to improve the performance of a website
- Informed by data insights & psychology
- Taking the traffic there and making the most of it
Correlation & prediction
Behavioral research
1) Descriptive research
a. Thoughts, feelings, ideas, behaviors.
2) Correlational research
a. Identifying relationships between different observed variables: measuring
thoughts, feelings, behavior.
“Smoking mothers have more often problem children”.
b. Measures the association between 2 variables.
Correlation of -1 (negative association) to +1 (positive association).
2
, c. R = .80 → the more the mother smokes, the more often they have problem children.
R = -.80 → the more the mother smokes, the less often they have problem children.
d. Weaknesses:
Correlation ≠ causation.
Direction of the relationship?
There is a third explaining variable → spurious correlation.
3) Experimental
a. Testing causality, A → B?
“Impact of loyalty programs on sales?”
b. Only type of research that (potentially) can demonstrate that a change in one
variable causes a predictable change in another variable.
c. Most difficult: making sure that a change in Y was not caused by something else
than X.
Experimental research settings
- Field experimentation
o Real life setting
o Mundane reality: natural behavior, setting, treatment.
o Less control
- Laboratory experimentation
o More control
o Better able to manipulate variables
o No natural setting
Experimentation & causation
- Needs to be correlation between 2 variables
- Asymmetrical direction
- A (cause) → B (effect)
- Change in A is accompanied by a change in B
- No alternative explanation for the change in B than the change in A (other possible causes
are controlled for)
3
, Experimental research essence
Test specific hypotheses about relationship between cause and effect via controlled (laboratory)
conditions.
- The effect of an independent variable (IV) on a dependent variable (DV).
- Manipulating the independent variable.
- Measure effects on dependent variable.
- Control other influences (high internal but low external validity).
Importance of randomization
- Use all sorts of people with all sorts of characteristics.
- Potential confounds are under control.
- So: change in Y can be attributed to X.
Problem identification
Sources of research ideas:
- Real life experiences
- Previous research and theory
o Conflicting findings
o Boundary conditions (moderator)
o Find explanation for observed effects (mediator)
o Applying a theory to a consumer setting
Lecture 2 – Experimental Research
Full empirical cycle
Identify interesting and relevant research question – Pinpoint conceptual variables – Identify the
proper relations between them that fit your theorizing (= hypotheses) – Develop a research design
– Develop manipulations and measures for the IV’s, DV’s, mediators and moderators – Collect
data – Analyze and interpret the results – Write up study to submit
Experimental research: main steps
1) Theoretical framework
a. Problem identification
b. Hypothesis formulation
2) Experimental design
a. Manipulation of IVs
b. Measurement of DV
c. Control for confounds
3) Data analyses
a. Get familiar with your data
b. Checks
c. Conduct main analyses
d. Conduct follow up analyses
4
