Summary Advanced Research Methods
Exam
- 8 questions (including subquestions), 3 hours
- plan strategically (varying points per Q)
- answer concisely and to the point, but answer all parts
- Open questions
- Not simply reproducing definition … but applying your knowledge to examples
- Explaining why (show that you understand)
- Spss output - understanding the technique
- Assignments = practice questions
- The interviewing lecture and assignment 7 were not part of the exam.
—------------------------------------------------------------------------------------------------------------------------
Structure of the lectures
Causal mechanisms
● Intro & Causal mechanisms
● Causal mechanisms & macro/micro relations
● Multilevel modeling
● Longitudinal data analysis: time series, panel and TSCS data
● Reliability and Factor Analysis: Measuring latent concepts
Experiments
● Endogeneity, causal inference and the experimental approach
Advanced qualitative methods
● Content and Discourse Analysis
● Process-tracing: uncovering (ideational) causal mechanisms
● Interviewing
---------------------------------------------------------------------------------------------------------------------------
,Meeting 1. Intro & Causal mechanisms
INTRODUCTION
Goldthorpe (2001) three understandings of causation
1. Causation as Robust Dependence
2. Causation as Consequential Manipulation (experimental logic)
3. Causation as Generative Process (mechanism-based approach)
4. Methodological individualism
Main take-away points:
- causal mechanisms + specifying them
- Macro / Micro idea, and how these are related.
—------------------------------------------------------------------------------------------------------------------------
What is ARM?
Equip you with the tools necessary to:
● construct methodologically sound research designs
➢ Especially for your own master’s thesis
● critically judge the quality of analysis in articles, research papers and policy briefs
➢ Especially when it comes to causality
What are we going to do (and what not)
● ARM provides you with a broad overview of relevant advanced research methods
● ARM aims at teaching you a basic understanding of these methods
➢ principles - how does it work
➢ usefulness - when can we apply it?
➢ strengths & limitations
● ARM does not aim at:
➢ Teaching you full mastery of the methods (you get to know the basics, not
become an expert)
➢ Providing you with specific practical skills in applying the methods
Theory & inference
- A set of logically interrelated propositions about empirical reality (Schutt, 2012)
- A systematic explanation for the observed facts and laws that relate to a specific
aspect of life (Babbie, 1989)
- A supposition or a system of ideas intended to explain something, especially one
based on general principles independent of the thing to be explained (Oxford
Dictionary)
Inference
, - The goal of social science is “to infer beyond the immediate data to something
broader that is not directly observed (King et al., 1994, p. 8)
Quantitative & qualitative inference
- from sample to population - quantitative
- from case to a broader set of cases - qualitative
Inductive & deductive reference
- theory generating → inductive (specific information → general conclusions)
- theory testing → deductive (general information → specific conclusions)
Descriptive & causal inference
- Focus on explanation (X → Y) - causal inference:
- But: description necessary first step → To show what is happening they attempt to
explain why it is happening (Goldthorpe 2001, p.11) - descriptive
Descriptive questions
- How high is election turnout in different states of the US?
- How did election turnout in the US develop over the past three decades?
- How is the war in Ukraine framed by right-wing media?
- To what extent do citizens of different EU countries believe in conspiracy theories?
Causal questions
- How do economic conditions impact cross-state differences in turnout
- What is the effect of social media use on the decision to vote?
- Through which mechanisms does social media influence people’s voting decisions?
THREE UNDERSTANDINGS OF CAUSATION (GOLDTHORPE, 2001)
1. Robust dependence approach
While correlation* does not imply causation, causation must in some way or other imply association
(Goldthorpe 2001, p. 2)
*correlation is also referred to as covariance or association.
- Correlation (between X and Y) is crucial in this approach. Correlation means two variables
are associated (for example: X goes up, Y goes up = positive correlation)
- Robust dependence: the correlation remains after controlling for possible other explanations
(confounders).
- confounders: variables that correlate to both X and Y.
- If the effect between X and Y still holds after controlling for the possible confounders there is
a true causal effect.
- If it does not: it is a spurious association or omitted variable bias
- Typically: this approach is associated to regression-based models on large datasets
, Critique 1#: No strict causal test: You can never make sure that you controlled for all possible
confounders. When are you done controlling?
Critique 2#: What is the mechanism in between X and Y? How X and Y are connected. What is the
process from x leading to Y (mechanism). We want to know why X leads to Y. There can be multiple
stories/mechanisms.
Alternative approach (critique 1#)
2. Consequential manipulation approach (Experimental logic)
Consequential manipulation (experimental logic)
➢ Experimental logic is based on counterfactual reasoning*:
*To establish causality you are going to observe Y in the presence and absence (counter
factual) of X. By comparing these two situations you can see what the actual effect of X on Y
is.
What is the effect of having a coffee on how well I perform in this lecture? Counting the number of
times I say uhh in this lecture. What is the effect of drinking one cup of coffee? the counterfactual
would be having the exact same erika with the exact breakfast, sleep, etc. Copy two situations except
for that coffee. If there was a parallel world in which I could do this, then I would have the perfect
counterfactual for myself and could see the effect of the coffee on her performance (number of eh).
this is not possible.. there is no parallel world.
Critiques (2)
➢ Effects of causes: causes must be manipulable. Structural characteristics can in this logic not
be a cause anymore.
➢ Problem of agency: people have a free will. They want to do well in a lecture and avoid saying
uhh. Or maybe I decided to say a lot of uhh today. This also falls outside the goal of the
researcher.
Experimental research has solved this by random assignment. A group of people taking part in an
experiment and randomly dividing them. Because it is random you assume that the groups are equal
on all relevant characteristics, and if they are equal you can have one group all drink one group of
Exam
- 8 questions (including subquestions), 3 hours
- plan strategically (varying points per Q)
- answer concisely and to the point, but answer all parts
- Open questions
- Not simply reproducing definition … but applying your knowledge to examples
- Explaining why (show that you understand)
- Spss output - understanding the technique
- Assignments = practice questions
- The interviewing lecture and assignment 7 were not part of the exam.
—------------------------------------------------------------------------------------------------------------------------
Structure of the lectures
Causal mechanisms
● Intro & Causal mechanisms
● Causal mechanisms & macro/micro relations
● Multilevel modeling
● Longitudinal data analysis: time series, panel and TSCS data
● Reliability and Factor Analysis: Measuring latent concepts
Experiments
● Endogeneity, causal inference and the experimental approach
Advanced qualitative methods
● Content and Discourse Analysis
● Process-tracing: uncovering (ideational) causal mechanisms
● Interviewing
---------------------------------------------------------------------------------------------------------------------------
,Meeting 1. Intro & Causal mechanisms
INTRODUCTION
Goldthorpe (2001) three understandings of causation
1. Causation as Robust Dependence
2. Causation as Consequential Manipulation (experimental logic)
3. Causation as Generative Process (mechanism-based approach)
4. Methodological individualism
Main take-away points:
- causal mechanisms + specifying them
- Macro / Micro idea, and how these are related.
—------------------------------------------------------------------------------------------------------------------------
What is ARM?
Equip you with the tools necessary to:
● construct methodologically sound research designs
➢ Especially for your own master’s thesis
● critically judge the quality of analysis in articles, research papers and policy briefs
➢ Especially when it comes to causality
What are we going to do (and what not)
● ARM provides you with a broad overview of relevant advanced research methods
● ARM aims at teaching you a basic understanding of these methods
➢ principles - how does it work
➢ usefulness - when can we apply it?
➢ strengths & limitations
● ARM does not aim at:
➢ Teaching you full mastery of the methods (you get to know the basics, not
become an expert)
➢ Providing you with specific practical skills in applying the methods
Theory & inference
- A set of logically interrelated propositions about empirical reality (Schutt, 2012)
- A systematic explanation for the observed facts and laws that relate to a specific
aspect of life (Babbie, 1989)
- A supposition or a system of ideas intended to explain something, especially one
based on general principles independent of the thing to be explained (Oxford
Dictionary)
Inference
, - The goal of social science is “to infer beyond the immediate data to something
broader that is not directly observed (King et al., 1994, p. 8)
Quantitative & qualitative inference
- from sample to population - quantitative
- from case to a broader set of cases - qualitative
Inductive & deductive reference
- theory generating → inductive (specific information → general conclusions)
- theory testing → deductive (general information → specific conclusions)
Descriptive & causal inference
- Focus on explanation (X → Y) - causal inference:
- But: description necessary first step → To show what is happening they attempt to
explain why it is happening (Goldthorpe 2001, p.11) - descriptive
Descriptive questions
- How high is election turnout in different states of the US?
- How did election turnout in the US develop over the past three decades?
- How is the war in Ukraine framed by right-wing media?
- To what extent do citizens of different EU countries believe in conspiracy theories?
Causal questions
- How do economic conditions impact cross-state differences in turnout
- What is the effect of social media use on the decision to vote?
- Through which mechanisms does social media influence people’s voting decisions?
THREE UNDERSTANDINGS OF CAUSATION (GOLDTHORPE, 2001)
1. Robust dependence approach
While correlation* does not imply causation, causation must in some way or other imply association
(Goldthorpe 2001, p. 2)
*correlation is also referred to as covariance or association.
- Correlation (between X and Y) is crucial in this approach. Correlation means two variables
are associated (for example: X goes up, Y goes up = positive correlation)
- Robust dependence: the correlation remains after controlling for possible other explanations
(confounders).
- confounders: variables that correlate to both X and Y.
- If the effect between X and Y still holds after controlling for the possible confounders there is
a true causal effect.
- If it does not: it is a spurious association or omitted variable bias
- Typically: this approach is associated to regression-based models on large datasets
, Critique 1#: No strict causal test: You can never make sure that you controlled for all possible
confounders. When are you done controlling?
Critique 2#: What is the mechanism in between X and Y? How X and Y are connected. What is the
process from x leading to Y (mechanism). We want to know why X leads to Y. There can be multiple
stories/mechanisms.
Alternative approach (critique 1#)
2. Consequential manipulation approach (Experimental logic)
Consequential manipulation (experimental logic)
➢ Experimental logic is based on counterfactual reasoning*:
*To establish causality you are going to observe Y in the presence and absence (counter
factual) of X. By comparing these two situations you can see what the actual effect of X on Y
is.
What is the effect of having a coffee on how well I perform in this lecture? Counting the number of
times I say uhh in this lecture. What is the effect of drinking one cup of coffee? the counterfactual
would be having the exact same erika with the exact breakfast, sleep, etc. Copy two situations except
for that coffee. If there was a parallel world in which I could do this, then I would have the perfect
counterfactual for myself and could see the effect of the coffee on her performance (number of eh).
this is not possible.. there is no parallel world.
Critiques (2)
➢ Effects of causes: causes must be manipulable. Structural characteristics can in this logic not
be a cause anymore.
➢ Problem of agency: people have a free will. They want to do well in a lecture and avoid saying
uhh. Or maybe I decided to say a lot of uhh today. This also falls outside the goal of the
researcher.
Experimental research has solved this by random assignment. A group of people taking part in an
experiment and randomly dividing them. Because it is random you assume that the groups are equal
on all relevant characteristics, and if they are equal you can have one group all drink one group of
