Lecture 1 – Causal interference “drawing the
lines between causes, outcomes and
confounders”
Segment 1.1: About this course
The course is covering quantitative and qualitative methods. The central theme is assessment and
interpretation of research: recognizing good and bad science, implication of results and the role of
theory in the application of research methods. The focus is on application an interpretation, not on
new techniques and is building on what you learned earlier.
Prior knowledge
Essential
Descriptive statistics: mean, median, standard deviation, percentiles, proportions
Measures of association: correlation, differences in means or proportions, relative risk,
coefficients
Inferential statistics: standard error, p-value, confidence interval
Statistical significance, null hypothesis significance testing
Bivariate tests: t-test, chi-squared test
Ordinary least squared regression (OLS, ‘linear regression’)
Logistic regression
What is statistical adjustment in general
Methods for statistical adjustment
Not strictly necessary
Distributions and theory (Gaussian, Student’s t-test, Chi-squared, etc.)
Mathematical explanation
Experience with software (SPSS, Stata, SAS, R, etc)
Segment 1.2: Causal theory
Learning goals this segment:
Introduction to causal inference
You will be able to…
o Explain the potential outcomes approach in causal inference and apply it in thinking
about causal effect estimation
o Define ‘causal effects’
o Apply the concepts of consistency, positivity and exchangeability in randomised and
non-randomised settings.
Questions to assess causal inference studies
What was the question?
What was the underlying question?
What was actually estimated?
Is the estimate really an answer to the question?
What is the estimate? Is that big, small, good, bad, etc?
What do the researchers conclude? Is that conclusion justified?
1
,Study: a scientific report in a skin care product
The packaging makes science claims: proven clinical results “70% less imperfections in 4
weeks. True match minerals tested under dermatological control with 41 women. Average
reduction of the most visible imperfections linked to oily skin.”
The company is implying that there is a causal relationship, improves implies a causal effect
o A leads to B
o Using True Match Minerals leads to a better skin
Is this convincing?
o No, there are a few problems with this claim.
o First of all, there was a small sample size of only 41 women. This is not always a
problem; it depends on the strength of the effect. It might be a problem in this case.
o Second, the study was performed or financed by a commercial company. It could be
a problem, but many universities have connections with commercial companies,
both could benefit but we have to protect the independence of the research, for
example the university decides what gets published and needs access to all data.
o Last, there was no use of a control group. That is an essential point missing. How
would the skin of these women look without treatment? It is impossible to
determine if these women would not have had a better skin in 4 weeks even without
the product. When you have something now, it is likely that you will not have that in
a month, we call this trend (potential) regression towards the mean.
What do we want to know? We are not interested in the outcome per se: ‘how many
imperfections’, we are interested in the role of treatment in achieving this outcome: ‘less
imperfections than without True Match Minerals’
Conclusion: no meaningful causal conclusion can be drawn from this study
Causation: towards a usable, formal framework
Formal definition (Hernàn/Robins): ‘In an individual, a treatment has a causal effect if the
outcome under treatment 1 would be different from the outcome under treatment 2.’
For the example above this would mean that:
o Woman A uses True Match Mineral: 2 bad spots
o Women A does not use True Match Mineral: 5 bad spots
o Individual treatment effect: 3 spots, thus a reduction of 60% (from 5 to 2)
o Average treatment effect: average of individual effects in a population
Potential outcomes are all observed
a=1 a=0
Causal effect: Y i ≠Y i
Y = outcome; a = treatment; 1 = yes (received treatment);
0 = no (received no treatment); i = individual; ≠ does not equal
Y a=1
K
= 1 means improvement with treatment
a=0
Y K = 0 means no improvement without treatment.
Treatment effect for K: ∆ Y K =1−0=1
Average treatment effect = average of ∆ Y K
ID a Ya=1 Ya=0 ΔYi
Patient K 1 1 0 1
Patient L 1 1a=1 1a=0 0
ID a Y Y ΔYi
Patient M 0 0 0 0
Patient K 1 1 ? ?
Patient N 0 1 0 1
Patient L 1 1 ? ?
Patient O 1 0 1 -1
Patient M 0 ? 0 ?
Patient N 0 ? 0 ?
Patient O 1 0 ? ?
2
,Not all potential outcomes are observed
Counterfactual outcome: potential outcome that is not observed because the subject did not
experience the treatment (‘counter the fact’).
Potential outcome of Y a=1 is factual for some subjects and counterfactual for others
Fundamental problem
Individual causal effect cannot be observed
o Except under extremely strong (and generally unreasonable) assumptions
o Average causal effect cannot be inferred from individual estimates
Causal inference as a missing data problem
We need a different approach to causal effects
Average causal effects can still be determined under certain conditions
Identifiability conditions
Observing the counterfactual
Based on population averages, causal effects can be estimated if three identifiability
conditions hold:
o Positivity
o Consistency
o Exchangeability
If the conditions are met, then association of exposure and outcome is an unbiased estimate
of causal effect
We can demonstrate this with a simple experiment. We go to the Lijnbaan in Rotterdam and
ask people “are you carrying a cigarette lighter?”. We note the answer and contacts and will
eventually come back after twenty years and ask “who is healthier?”. We can look into the
causal question: what is the effect of carrying a cigarette lighter on health?
Positivity: About the sample and the way it was composed.
‘Positive probability’ of being assigned to each of the treatment levels, meaning that beforehand
people should have had access to both ‘treatments’.
Units are assigned to all relevant ‘treatments’. People with and people without a cigarette
lighter. However, people with a lighter could also not have had a lighter and vice versa.
L’Oreal: 100% was assigned to True Mineral Match, 0% to comparator. Users could not not have
used it.
Consistency: Define ‘if’: is there a clear definition of ‘treatments’?
Hernan: does water kill? Drinking it or are you swimming in it?
Is it consistent? Is Broccoli consistent? If you define it well it will be, but you have to be
specific, how much broccoli, compared to what?
Carrying a cigarette lighter is quite consistent
Exchangeability: treatment groups are exchangeable
Observe ‘what would have happened if…’
3
, Treatment groups are exchangeable: it does not matter who gets treatment A and who gets
treatment B. So ‘potential outcomes’ are independent of the treatment that was actually
received’.
a
Notation: Y i ⊥ A
Are people with and without lighters exchangeable (similar in other respects)? Yes, we would
expect that.
But it may be necessary to take other factors into account (adjustment)
Exchangeability: Within the smoking group, are people with and without a lighter
exchangeable? Within the non-smoking group. Are people with and without lighter
exchangeable? Association can be ascribed to treatment effect
Positivity: units are assigned to all relevant ‘treatments’ within levels of the adjustment
factors. ‘Positive probability’ of being assigned to each of the treatment levels. We need:
o Smokers with cigarette lighter
o Smokers without a cigarette lighter
o Non-smokers with cigarette lighter
o Non-smokers without cigarette lighter
So we are going to stratify the results.
o Total sample:
Healthy with lighter (n=110): 63 (57%)
Healthy without a lighter (n=190): 165 (87%)
o Association: people with cigarette lighters less likely to be healthy? This seems
unlikely. There might be another factor involved, like smoking…
4
lines between causes, outcomes and
confounders”
Segment 1.1: About this course
The course is covering quantitative and qualitative methods. The central theme is assessment and
interpretation of research: recognizing good and bad science, implication of results and the role of
theory in the application of research methods. The focus is on application an interpretation, not on
new techniques and is building on what you learned earlier.
Prior knowledge
Essential
Descriptive statistics: mean, median, standard deviation, percentiles, proportions
Measures of association: correlation, differences in means or proportions, relative risk,
coefficients
Inferential statistics: standard error, p-value, confidence interval
Statistical significance, null hypothesis significance testing
Bivariate tests: t-test, chi-squared test
Ordinary least squared regression (OLS, ‘linear regression’)
Logistic regression
What is statistical adjustment in general
Methods for statistical adjustment
Not strictly necessary
Distributions and theory (Gaussian, Student’s t-test, Chi-squared, etc.)
Mathematical explanation
Experience with software (SPSS, Stata, SAS, R, etc)
Segment 1.2: Causal theory
Learning goals this segment:
Introduction to causal inference
You will be able to…
o Explain the potential outcomes approach in causal inference and apply it in thinking
about causal effect estimation
o Define ‘causal effects’
o Apply the concepts of consistency, positivity and exchangeability in randomised and
non-randomised settings.
Questions to assess causal inference studies
What was the question?
What was the underlying question?
What was actually estimated?
Is the estimate really an answer to the question?
What is the estimate? Is that big, small, good, bad, etc?
What do the researchers conclude? Is that conclusion justified?
1
,Study: a scientific report in a skin care product
The packaging makes science claims: proven clinical results “70% less imperfections in 4
weeks. True match minerals tested under dermatological control with 41 women. Average
reduction of the most visible imperfections linked to oily skin.”
The company is implying that there is a causal relationship, improves implies a causal effect
o A leads to B
o Using True Match Minerals leads to a better skin
Is this convincing?
o No, there are a few problems with this claim.
o First of all, there was a small sample size of only 41 women. This is not always a
problem; it depends on the strength of the effect. It might be a problem in this case.
o Second, the study was performed or financed by a commercial company. It could be
a problem, but many universities have connections with commercial companies,
both could benefit but we have to protect the independence of the research, for
example the university decides what gets published and needs access to all data.
o Last, there was no use of a control group. That is an essential point missing. How
would the skin of these women look without treatment? It is impossible to
determine if these women would not have had a better skin in 4 weeks even without
the product. When you have something now, it is likely that you will not have that in
a month, we call this trend (potential) regression towards the mean.
What do we want to know? We are not interested in the outcome per se: ‘how many
imperfections’, we are interested in the role of treatment in achieving this outcome: ‘less
imperfections than without True Match Minerals’
Conclusion: no meaningful causal conclusion can be drawn from this study
Causation: towards a usable, formal framework
Formal definition (Hernàn/Robins): ‘In an individual, a treatment has a causal effect if the
outcome under treatment 1 would be different from the outcome under treatment 2.’
For the example above this would mean that:
o Woman A uses True Match Mineral: 2 bad spots
o Women A does not use True Match Mineral: 5 bad spots
o Individual treatment effect: 3 spots, thus a reduction of 60% (from 5 to 2)
o Average treatment effect: average of individual effects in a population
Potential outcomes are all observed
a=1 a=0
Causal effect: Y i ≠Y i
Y = outcome; a = treatment; 1 = yes (received treatment);
0 = no (received no treatment); i = individual; ≠ does not equal
Y a=1
K
= 1 means improvement with treatment
a=0
Y K = 0 means no improvement without treatment.
Treatment effect for K: ∆ Y K =1−0=1
Average treatment effect = average of ∆ Y K
ID a Ya=1 Ya=0 ΔYi
Patient K 1 1 0 1
Patient L 1 1a=1 1a=0 0
ID a Y Y ΔYi
Patient M 0 0 0 0
Patient K 1 1 ? ?
Patient N 0 1 0 1
Patient L 1 1 ? ?
Patient O 1 0 1 -1
Patient M 0 ? 0 ?
Patient N 0 ? 0 ?
Patient O 1 0 ? ?
2
,Not all potential outcomes are observed
Counterfactual outcome: potential outcome that is not observed because the subject did not
experience the treatment (‘counter the fact’).
Potential outcome of Y a=1 is factual for some subjects and counterfactual for others
Fundamental problem
Individual causal effect cannot be observed
o Except under extremely strong (and generally unreasonable) assumptions
o Average causal effect cannot be inferred from individual estimates
Causal inference as a missing data problem
We need a different approach to causal effects
Average causal effects can still be determined under certain conditions
Identifiability conditions
Observing the counterfactual
Based on population averages, causal effects can be estimated if three identifiability
conditions hold:
o Positivity
o Consistency
o Exchangeability
If the conditions are met, then association of exposure and outcome is an unbiased estimate
of causal effect
We can demonstrate this with a simple experiment. We go to the Lijnbaan in Rotterdam and
ask people “are you carrying a cigarette lighter?”. We note the answer and contacts and will
eventually come back after twenty years and ask “who is healthier?”. We can look into the
causal question: what is the effect of carrying a cigarette lighter on health?
Positivity: About the sample and the way it was composed.
‘Positive probability’ of being assigned to each of the treatment levels, meaning that beforehand
people should have had access to both ‘treatments’.
Units are assigned to all relevant ‘treatments’. People with and people without a cigarette
lighter. However, people with a lighter could also not have had a lighter and vice versa.
L’Oreal: 100% was assigned to True Mineral Match, 0% to comparator. Users could not not have
used it.
Consistency: Define ‘if’: is there a clear definition of ‘treatments’?
Hernan: does water kill? Drinking it or are you swimming in it?
Is it consistent? Is Broccoli consistent? If you define it well it will be, but you have to be
specific, how much broccoli, compared to what?
Carrying a cigarette lighter is quite consistent
Exchangeability: treatment groups are exchangeable
Observe ‘what would have happened if…’
3
, Treatment groups are exchangeable: it does not matter who gets treatment A and who gets
treatment B. So ‘potential outcomes’ are independent of the treatment that was actually
received’.
a
Notation: Y i ⊥ A
Are people with and without lighters exchangeable (similar in other respects)? Yes, we would
expect that.
But it may be necessary to take other factors into account (adjustment)
Exchangeability: Within the smoking group, are people with and without a lighter
exchangeable? Within the non-smoking group. Are people with and without lighter
exchangeable? Association can be ascribed to treatment effect
Positivity: units are assigned to all relevant ‘treatments’ within levels of the adjustment
factors. ‘Positive probability’ of being assigned to each of the treatment levels. We need:
o Smokers with cigarette lighter
o Smokers without a cigarette lighter
o Non-smokers with cigarette lighter
o Non-smokers without cigarette lighter
So we are going to stratify the results.
o Total sample:
Healthy with lighter (n=110): 63 (57%)
Healthy without a lighter (n=190): 165 (87%)
o Association: people with cigarette lighters less likely to be healthy? This seems
unlikely. There might be another factor involved, like smoking…
4
