HC1: Advanced research methods
What prior knowledge do we expect?
Qualitative concepts and methods
- Interviews, observations, document
- Validity and reliability
Introduction to causal inference:
In causal inference
- We are not interested in the outcome per se.
- We are interested in the role of treatment in achieving this outcome (without true
match minerals powder, would there have been less skin imperfections).
- Conclusion
We do not have the information.
No causal claim can be made based on the L’Oréal study.
Causal effect
- Formal definition by Hernan and Robins
- In an individual, a treatment has a causal effect if the outcome under treatment 1
would be different from the outcome under treatment 2.
- To assess this, we need information on:
Potential outcome approach
Y= outcome
A= treatment
1= Yes (received treatment)
0= No treatment
I= Individual
=1 (improvement with treatment)
=0 (no improvement without treatment)
,Treatment effect for K: Delta Yk = 1-0=1 (positive effect)
Average treatment effect= average of Delta Yi
Not all potential outcomes are observed.
- Counterfactual outcome: potential outcome that is not observed because the subject
did not experience the treatment.
- Potential outcome Ya=1 is factual for some subjects, and counterfactual for others.
Fundamental problem
- Individual causal effect cannot be observed.
Expect under extremely strong assumptions.
- Average causal effect cannot be determined based on individual estimates.
Causal inference as a missing data problem.
Identifiability conditions:
Average causal effect can be determined if, and only if, three identifiability conditions are
met.
1. Positivity
2. Consistency
3. Exchangeability
- If all conditions are met the association between exposure and outcome is an
unbiased estimate of a causal effect.
Positivity
- This condition means that:
Everyone must have a positive probability of being assigned to each of the
treatment arms.
- Cigarette lighter example: people with and people without one plastic cigarette
lighter.
- In comparison:
100% was assigned to true mineral match.
0% to comparator
Consistency
- The treatment has to be well-defined.
- Does water kill: Wat kind of water, how much etc?
Exchangeability
- The individual assigned to the different treatment arms must be similar.
- It does not matter who gets treatment A and who gets B.
- People with a lighter could also not have had a lighter.
,Meeting the exchangeability condition
Four ways are possible:
1. Randomized controlled trial (RCT)
Individuals are randomly assigned to one of each treatment arms.
Differences between individuals in the different treatment arms are cancelled out
on the sample level.
Differences are independent from the treatment and outcome.
Differences are random, not systematic.
Golden standard, because typically all identifiability conditions are met in RCTs.
2. Matching
For each individual with characteristics x,y and z who gets treatment A. There is
an individual with characteristics x,y,z who gets treatment B.
Statistical methods can be applied when perfect matching is not possible.
3. Stratification
Randomly select individuals from different subsets of the larger population.
Difficult to meet the positivity condition.
Population -> strata -> random selection -> sample.
4. Adjustment
Control for factors that influence the association between the treatment and
outcome in regression analysis.
Individuals are assigned to all treatment arms within all levels of adjustment
factors.
Can also be combined with an RCT, stratification, and matching.
Complete and correct adjustment leads to exchangeability.
RCTs versus observational studies
- Golden standard however
Limited external validity
Ethical and practical considerations
- Observational studies
Real world outcomes
Availability of data
Positivity and consistency need close attention.
Internal validity threatened by lack of exchangeability.
People are often interested in causal effects, not just correlations
What is the effect of nurses’ job satisfaction on the health of ICU patients
Correlation does not imply causation
- Correlation implies association.
A statistical relationship between the treatment and outcome
Knowing the value of one variable may provide information on the value of
another variable, but that does not mean that one caused the other.
, Knowing that Zeus died after 5 days after a heart transplantation does not mean
the transplant caused Zeus’ death.
Causation= difference between potential outcomes
Correlation does not imply causation.
- A statistical association equals the difference in potential outcomes if, and only if the
identifiability conditions are met.
- For this we need:
Theory and subject knowledge
Insight into the causal structure underlying the research question.
To meet the positivity, consistency, and exchangeability conditions, and design the
study and analysis accordingly.
Design analysis
- Focus on adjustment in regression analysis.
Complete and correct adjustment leads to exchangeability.
But how do we know what to adjust for in the analysis.
Traditional selection strategies
- Correlation matrix: select variables with statistically significant association with the
outcome.
- Stepwise backward selection
Insert all variables in regression models
Remove the variable that is the least statistically significant
Run regression, remove variable that is then the least.
Or keep variable in regression model if removal leads to substantial change in the
effect estimate.
Adjust for confounders, which are traditionally defined as being
Associated with the exposure
Problems with traditional strategies
- These strategies are still applied
These methods rely on the available data, rather than on theory/subject
knowledge
Selected strategy may increase, rather than reduce bias
Stepwise selection may result in false certainty
These methods are increasingly considered as outdate
Cigarette lighter example in DAG terms
What prior knowledge do we expect?
Qualitative concepts and methods
- Interviews, observations, document
- Validity and reliability
Introduction to causal inference:
In causal inference
- We are not interested in the outcome per se.
- We are interested in the role of treatment in achieving this outcome (without true
match minerals powder, would there have been less skin imperfections).
- Conclusion
We do not have the information.
No causal claim can be made based on the L’Oréal study.
Causal effect
- Formal definition by Hernan and Robins
- In an individual, a treatment has a causal effect if the outcome under treatment 1
would be different from the outcome under treatment 2.
- To assess this, we need information on:
Potential outcome approach
Y= outcome
A= treatment
1= Yes (received treatment)
0= No treatment
I= Individual
=1 (improvement with treatment)
=0 (no improvement without treatment)
,Treatment effect for K: Delta Yk = 1-0=1 (positive effect)
Average treatment effect= average of Delta Yi
Not all potential outcomes are observed.
- Counterfactual outcome: potential outcome that is not observed because the subject
did not experience the treatment.
- Potential outcome Ya=1 is factual for some subjects, and counterfactual for others.
Fundamental problem
- Individual causal effect cannot be observed.
Expect under extremely strong assumptions.
- Average causal effect cannot be determined based on individual estimates.
Causal inference as a missing data problem.
Identifiability conditions:
Average causal effect can be determined if, and only if, three identifiability conditions are
met.
1. Positivity
2. Consistency
3. Exchangeability
- If all conditions are met the association between exposure and outcome is an
unbiased estimate of a causal effect.
Positivity
- This condition means that:
Everyone must have a positive probability of being assigned to each of the
treatment arms.
- Cigarette lighter example: people with and people without one plastic cigarette
lighter.
- In comparison:
100% was assigned to true mineral match.
0% to comparator
Consistency
- The treatment has to be well-defined.
- Does water kill: Wat kind of water, how much etc?
Exchangeability
- The individual assigned to the different treatment arms must be similar.
- It does not matter who gets treatment A and who gets B.
- People with a lighter could also not have had a lighter.
,Meeting the exchangeability condition
Four ways are possible:
1. Randomized controlled trial (RCT)
Individuals are randomly assigned to one of each treatment arms.
Differences between individuals in the different treatment arms are cancelled out
on the sample level.
Differences are independent from the treatment and outcome.
Differences are random, not systematic.
Golden standard, because typically all identifiability conditions are met in RCTs.
2. Matching
For each individual with characteristics x,y and z who gets treatment A. There is
an individual with characteristics x,y,z who gets treatment B.
Statistical methods can be applied when perfect matching is not possible.
3. Stratification
Randomly select individuals from different subsets of the larger population.
Difficult to meet the positivity condition.
Population -> strata -> random selection -> sample.
4. Adjustment
Control for factors that influence the association between the treatment and
outcome in regression analysis.
Individuals are assigned to all treatment arms within all levels of adjustment
factors.
Can also be combined with an RCT, stratification, and matching.
Complete and correct adjustment leads to exchangeability.
RCTs versus observational studies
- Golden standard however
Limited external validity
Ethical and practical considerations
- Observational studies
Real world outcomes
Availability of data
Positivity and consistency need close attention.
Internal validity threatened by lack of exchangeability.
People are often interested in causal effects, not just correlations
What is the effect of nurses’ job satisfaction on the health of ICU patients
Correlation does not imply causation
- Correlation implies association.
A statistical relationship between the treatment and outcome
Knowing the value of one variable may provide information on the value of
another variable, but that does not mean that one caused the other.
, Knowing that Zeus died after 5 days after a heart transplantation does not mean
the transplant caused Zeus’ death.
Causation= difference between potential outcomes
Correlation does not imply causation.
- A statistical association equals the difference in potential outcomes if, and only if the
identifiability conditions are met.
- For this we need:
Theory and subject knowledge
Insight into the causal structure underlying the research question.
To meet the positivity, consistency, and exchangeability conditions, and design the
study and analysis accordingly.
Design analysis
- Focus on adjustment in regression analysis.
Complete and correct adjustment leads to exchangeability.
But how do we know what to adjust for in the analysis.
Traditional selection strategies
- Correlation matrix: select variables with statistically significant association with the
outcome.
- Stepwise backward selection
Insert all variables in regression models
Remove the variable that is the least statistically significant
Run regression, remove variable that is then the least.
Or keep variable in regression model if removal leads to substantial change in the
effect estimate.
Adjust for confounders, which are traditionally defined as being
Associated with the exposure
Problems with traditional strategies
- These strategies are still applied
These methods rely on the available data, rather than on theory/subject
knowledge
Selected strategy may increase, rather than reduce bias
Stepwise selection may result in false certainty
These methods are increasingly considered as outdate
Cigarette lighter example in DAG terms
