Lecture 11: Qualitative research
“Qualitative work is expressed in natural language, whereas quantitative work is expressed in numbers and in statistical models”
(Gerring 2017: 18)
Strengths and weaknesses of qualitative research
What are the strengths of qualitative research?
Allows for detailed understanding of the process / causal mechanism
Gives room for meaning, interpretation & voice
o With a qualitative case study, you can open the black box, and understand what people believe, how they feel,
what the causal mechanism is exactly.
Allows for surprise / unexpected findings
o ‘Front loading’ in quantitative research (Yanow 2014: 144)
If you do quantitative research, you have to decide and design in advance (e.g. surveys), you have to make
all these choices in advance
o In qualitative research: more room for unexpected findings, you can go in the field and allow for change,
uncertainty, varying choices.
What are the weaknesses of qualitative research?
Small and non-randomized samples (E.g. Documents, people) limited generalizability
No statistical calculations, but interpretation by the researcher limited replicability
Complementarity of qualitative and quantitative research
Qualitative research in political science
Qualitative research in political science
Qualitative research has always played an important role in political science, for example in:
o Public administration, international relations, comparative politics, legal studies
o But less in political behaviour
Research paradigms and qualitative research
Recall distinction between:
o Positivism / objectivism
o Realism
o Interpretivism / constructivism
Qualitative research is broadly interpretivist, but not in
Political Science
Ontology and epistemology
Ontology: What is the nature of reality?
Epistemology: What can we know about reality?
Quality criteria for qualitative research
Contrary to statistical analysis, no one ‘cookbook’ for qualitative research. But, three sets of criteria
1) The ‘rigor’ criteria (validity, reliability) for positivist (or realist) research:
Internal validity: is the relationship really causal
, External validity: generalizability, applicability in other contexts
Reliability: consistency of the inquiry, stability, replicability
Objectivity: neutrality, free of bias/values
2) The ‘parallel’ criteria for interpretive (or realist) research (Guba & Lincoln 1989)
Rigor criteria of limited use when no ‘real’ world/law-like relationship is assumed. But not anything goes Instead, Guba & Lincoln
(1989)
Credibility
o Parallel to internal validity
o ‘isomorphism between constructed realities of respondents and the reconstructions attributed to them’ (Guba &
Lincoln 237)
o Guba & Lincoln offer six ways to check credibility:
1) Prolonged engagement
Substantial involvement at the site of the inquiry, in order to overcome the effects of misinformation,
distortion, or presented “fronts,” to establish the rapport and build the trust necessary to uncover
constructions, and to facilitate immersing oneself in and understanding the context’s culture
2) Persistent observation
Sufficient observation to enable the evaluator to “identify those characteristics and elements in the
situation that are most relevant to the problem or issue being pursued and [to focus] on them in detail
Add depth
3) Peer debriefing
The process of engaging, with a disinterested peer, in extended and extensive discussions of one’s
findings, conclusions, tentative analyses, and, occasionally, field stresses, the purpose of which is both
“testing out” the findings with someone who has no contractual interest in the situation and also helping
to make propositional that tacit and implicit information that the evaluator might possess.
4) Negative case analysis
The process of revising working hypotheses in the light of hindsight, with an eye toward developing and
refining a given hypothesis (or set of them) until it accounts for all known cases.
5) Progressive subjectivity
The process of monitoring the evaluator’s (or any inquirer’s) own developing construction. The inquirer’s
construction cannot be given privilege over that of anyone else (except insofar as he or she may be able to
introduce a wider range of information and a higher level of sophistication than may any other single
respondent).
Prior to engaging in any activity at the site or in the context in which the investigation is to proceed, the
inquirer records his or her a priori construction—what he or she expects to find once the study is under
way—and archives that record. At regular intervals throughout the study the inquirer again records his or
her developing construction
6) Member checks
The process of testing hypotheses, data, preliminary categories, and interpretations with members of the
stakeholding groups from whom the original constructions were collected.
This is the single most crucial technique for establishing credibility
This process occurs continuously, both during the data collection and analysis stage, and, again, when
(and if) a narrative case study is prepared.
Member checks can be formal and informal, and with individuals (for instance, after interviews, in order
to verify that what was written down is what was intended to be communicated) or with groups (for
instance, as portions of the case study are written, members of stakeholding groups are asked to react to
what has been presented as representing their construction).
Transferability
o Parallel to external validity
o Always relative: ‘depends entirely on the degree to which salient conditions overlap or match’ (G&L: 241)
, o Empirical process for checking the degree of similarity between sending and receiving contexts. Further, the burden of
proof for claimed generalizability is on the inquirer, while the burden of proof for claimed transferability is on the
receiver.
o How: thick description of findings within context (time, place, culture)
o Often a relative weakness of qualitative research
o Is linked to case selection / sampling
o Bryman distinguishes:
Theoretical / analytical generalization
Qualitative research can critically test existing theories (most- and least-likely case selection)
Or it can contribute to the formulation of new theories, because it:
Explicates mechanisms that can have wide-ranging application
Brings new ways of seeing and understanding into plain view (see further Wedeen 2010:
268)
Generalisation to broader population
Two steps (reverse of two steps of data selection):
From sample to case -> can we say something about the case, based on the sample of
respondents / documents / etc.?
From case to broader population -> can we say something about the broader population,
based on the case
Generalisations are necessarily bounded and contingent
o Example: Cramer (2012)
o Two steps:
From sample to case
From case to broader population
Dependability
o Parallel to reliability, ‘concerned with the stability of data over time’ (G&L: 242)
o How: ‘process audit’ -> that outsider reviewers ‘can explore the process, judge the decisions that were made, and
understand what salient factors […] led the evaluator to the decisions and interpretations made’ (G&L: 242)
Confirmability
o Parallel to objectivity
o ‘assuring that data, interpretations, and outcomes of inquiries are rooted in contexts and persons apart from the evaluator
and are not simply figments of the evaluator’s imagination’ (G&L: 243)
o How: audit of relationship between data and outcomes by outsider reviewers
o Note: not really tenable in interpretivist research
3) Reflexivity (for all research?) Corlett, Sandra and Mavin, Sharon (2015) “Reflexivity and Researcher Positionality”
Methodological reflexivity
o Methodological reflexivity accepts that the researcher makes methodological and method choices (what is
included and what is left out), and acknowledges that research methods, as used by researchers, are not
neutral tools – each have ‘philosophical baggage’
o Reflexive researchers make explicit this baggage to an audience and provide a convincing account of the
knowledge ‘manufacturing conditions’
Data are produced by the researcher, not collected
And their involvement is taken as a source of data in its own right
Self-reflexivity
o Identity: The researcher’s epistemological position and assumptions influence understandings of researcher
field roles, and the related concept of identities.
o Motivations: which interests, values, experiences or political commitments do I bring to the research? (Corlett
& Main 2015: 385)
, How should researchers deal with their partiality/ biases?
Positivist researchers: try to minimize (feminist empiricism: “add women”)
Realist / Interpretivist researchers:
o Reflect and/or make it part of your analysis
o Aim for multiple perspectives (feminist standpoint theory)
Critical researchers: Let a normative engagement guide your research
o Positionality: how does my identity affect the research process? ‘how are race/gender/class made meaningful
in this relationship’ (Day cited in ibid: 387)
Mostly relevant for obstrusive data collection
Sometimes called interviewer effect (Lecture 13)
Requires reflection, shapes what people tell you in context dependent ways
o Power: what is the power relationship between the researcher and the researched?; What role do
positionality, identity, and power play in the process of knowledge production?;
o Voice: who speaks for whom? Who has a voice in the case that certain social groups do not have a voice?
Lecture 12: On feminist epistemology and the importance of diversity
Epistemology
Definition
“the study of knowledge and justified belief.” Stanford Encyclopedia of Philosophy,
How do we decide what is true, who do we believe, etc.
Epistemology is fundamental to academic practice
Feminist epistemologies
Science is a social and political practice shaped by power relations
Feminist perspectives on objectivity
o Situated knowledges
o Standpoint
o Reflexivity
Embrace your bias!
Positivist vs reflexive epistemologies
Positivist epistemology
A metaphor: “Standard Fishbowl Model of Science” (Schliesser 2015)
o He says that the scientist is a cat which stands outside a fish bowl, observing from a distance what is happening in
the close environment of the fish bowl. And scientists should be the objective cat.
Scientist should be:
o Objective, impartial, neutral
o Dispassionate
o Replicable
o Disinterested truth seeker
People studied:
o Data, phenomena, object of study
Reflexive epistemology
Scientists are part of the world they study; science is a social and political practice
o “The observer and the observed are in the same causal scientific plane” (Harding 1991: 11)
“Qualitative work is expressed in natural language, whereas quantitative work is expressed in numbers and in statistical models”
(Gerring 2017: 18)
Strengths and weaknesses of qualitative research
What are the strengths of qualitative research?
Allows for detailed understanding of the process / causal mechanism
Gives room for meaning, interpretation & voice
o With a qualitative case study, you can open the black box, and understand what people believe, how they feel,
what the causal mechanism is exactly.
Allows for surprise / unexpected findings
o ‘Front loading’ in quantitative research (Yanow 2014: 144)
If you do quantitative research, you have to decide and design in advance (e.g. surveys), you have to make
all these choices in advance
o In qualitative research: more room for unexpected findings, you can go in the field and allow for change,
uncertainty, varying choices.
What are the weaknesses of qualitative research?
Small and non-randomized samples (E.g. Documents, people) limited generalizability
No statistical calculations, but interpretation by the researcher limited replicability
Complementarity of qualitative and quantitative research
Qualitative research in political science
Qualitative research in political science
Qualitative research has always played an important role in political science, for example in:
o Public administration, international relations, comparative politics, legal studies
o But less in political behaviour
Research paradigms and qualitative research
Recall distinction between:
o Positivism / objectivism
o Realism
o Interpretivism / constructivism
Qualitative research is broadly interpretivist, but not in
Political Science
Ontology and epistemology
Ontology: What is the nature of reality?
Epistemology: What can we know about reality?
Quality criteria for qualitative research
Contrary to statistical analysis, no one ‘cookbook’ for qualitative research. But, three sets of criteria
1) The ‘rigor’ criteria (validity, reliability) for positivist (or realist) research:
Internal validity: is the relationship really causal
, External validity: generalizability, applicability in other contexts
Reliability: consistency of the inquiry, stability, replicability
Objectivity: neutrality, free of bias/values
2) The ‘parallel’ criteria for interpretive (or realist) research (Guba & Lincoln 1989)
Rigor criteria of limited use when no ‘real’ world/law-like relationship is assumed. But not anything goes Instead, Guba & Lincoln
(1989)
Credibility
o Parallel to internal validity
o ‘isomorphism between constructed realities of respondents and the reconstructions attributed to them’ (Guba &
Lincoln 237)
o Guba & Lincoln offer six ways to check credibility:
1) Prolonged engagement
Substantial involvement at the site of the inquiry, in order to overcome the effects of misinformation,
distortion, or presented “fronts,” to establish the rapport and build the trust necessary to uncover
constructions, and to facilitate immersing oneself in and understanding the context’s culture
2) Persistent observation
Sufficient observation to enable the evaluator to “identify those characteristics and elements in the
situation that are most relevant to the problem or issue being pursued and [to focus] on them in detail
Add depth
3) Peer debriefing
The process of engaging, with a disinterested peer, in extended and extensive discussions of one’s
findings, conclusions, tentative analyses, and, occasionally, field stresses, the purpose of which is both
“testing out” the findings with someone who has no contractual interest in the situation and also helping
to make propositional that tacit and implicit information that the evaluator might possess.
4) Negative case analysis
The process of revising working hypotheses in the light of hindsight, with an eye toward developing and
refining a given hypothesis (or set of them) until it accounts for all known cases.
5) Progressive subjectivity
The process of monitoring the evaluator’s (or any inquirer’s) own developing construction. The inquirer’s
construction cannot be given privilege over that of anyone else (except insofar as he or she may be able to
introduce a wider range of information and a higher level of sophistication than may any other single
respondent).
Prior to engaging in any activity at the site or in the context in which the investigation is to proceed, the
inquirer records his or her a priori construction—what he or she expects to find once the study is under
way—and archives that record. At regular intervals throughout the study the inquirer again records his or
her developing construction
6) Member checks
The process of testing hypotheses, data, preliminary categories, and interpretations with members of the
stakeholding groups from whom the original constructions were collected.
This is the single most crucial technique for establishing credibility
This process occurs continuously, both during the data collection and analysis stage, and, again, when
(and if) a narrative case study is prepared.
Member checks can be formal and informal, and with individuals (for instance, after interviews, in order
to verify that what was written down is what was intended to be communicated) or with groups (for
instance, as portions of the case study are written, members of stakeholding groups are asked to react to
what has been presented as representing their construction).
Transferability
o Parallel to external validity
o Always relative: ‘depends entirely on the degree to which salient conditions overlap or match’ (G&L: 241)
, o Empirical process for checking the degree of similarity between sending and receiving contexts. Further, the burden of
proof for claimed generalizability is on the inquirer, while the burden of proof for claimed transferability is on the
receiver.
o How: thick description of findings within context (time, place, culture)
o Often a relative weakness of qualitative research
o Is linked to case selection / sampling
o Bryman distinguishes:
Theoretical / analytical generalization
Qualitative research can critically test existing theories (most- and least-likely case selection)
Or it can contribute to the formulation of new theories, because it:
Explicates mechanisms that can have wide-ranging application
Brings new ways of seeing and understanding into plain view (see further Wedeen 2010:
268)
Generalisation to broader population
Two steps (reverse of two steps of data selection):
From sample to case -> can we say something about the case, based on the sample of
respondents / documents / etc.?
From case to broader population -> can we say something about the broader population,
based on the case
Generalisations are necessarily bounded and contingent
o Example: Cramer (2012)
o Two steps:
From sample to case
From case to broader population
Dependability
o Parallel to reliability, ‘concerned with the stability of data over time’ (G&L: 242)
o How: ‘process audit’ -> that outsider reviewers ‘can explore the process, judge the decisions that were made, and
understand what salient factors […] led the evaluator to the decisions and interpretations made’ (G&L: 242)
Confirmability
o Parallel to objectivity
o ‘assuring that data, interpretations, and outcomes of inquiries are rooted in contexts and persons apart from the evaluator
and are not simply figments of the evaluator’s imagination’ (G&L: 243)
o How: audit of relationship between data and outcomes by outsider reviewers
o Note: not really tenable in interpretivist research
3) Reflexivity (for all research?) Corlett, Sandra and Mavin, Sharon (2015) “Reflexivity and Researcher Positionality”
Methodological reflexivity
o Methodological reflexivity accepts that the researcher makes methodological and method choices (what is
included and what is left out), and acknowledges that research methods, as used by researchers, are not
neutral tools – each have ‘philosophical baggage’
o Reflexive researchers make explicit this baggage to an audience and provide a convincing account of the
knowledge ‘manufacturing conditions’
Data are produced by the researcher, not collected
And their involvement is taken as a source of data in its own right
Self-reflexivity
o Identity: The researcher’s epistemological position and assumptions influence understandings of researcher
field roles, and the related concept of identities.
o Motivations: which interests, values, experiences or political commitments do I bring to the research? (Corlett
& Main 2015: 385)
, How should researchers deal with their partiality/ biases?
Positivist researchers: try to minimize (feminist empiricism: “add women”)
Realist / Interpretivist researchers:
o Reflect and/or make it part of your analysis
o Aim for multiple perspectives (feminist standpoint theory)
Critical researchers: Let a normative engagement guide your research
o Positionality: how does my identity affect the research process? ‘how are race/gender/class made meaningful
in this relationship’ (Day cited in ibid: 387)
Mostly relevant for obstrusive data collection
Sometimes called interviewer effect (Lecture 13)
Requires reflection, shapes what people tell you in context dependent ways
o Power: what is the power relationship between the researcher and the researched?; What role do
positionality, identity, and power play in the process of knowledge production?;
o Voice: who speaks for whom? Who has a voice in the case that certain social groups do not have a voice?
Lecture 12: On feminist epistemology and the importance of diversity
Epistemology
Definition
“the study of knowledge and justified belief.” Stanford Encyclopedia of Philosophy,
How do we decide what is true, who do we believe, etc.
Epistemology is fundamental to academic practice
Feminist epistemologies
Science is a social and political practice shaped by power relations
Feminist perspectives on objectivity
o Situated knowledges
o Standpoint
o Reflexivity
Embrace your bias!
Positivist vs reflexive epistemologies
Positivist epistemology
A metaphor: “Standard Fishbowl Model of Science” (Schliesser 2015)
o He says that the scientist is a cat which stands outside a fish bowl, observing from a distance what is happening in
the close environment of the fish bowl. And scientists should be the objective cat.
Scientist should be:
o Objective, impartial, neutral
o Dispassionate
o Replicable
o Disinterested truth seeker
People studied:
o Data, phenomena, object of study
Reflexive epistemology
Scientists are part of the world they study; science is a social and political practice
o “The observer and the observed are in the same causal scientific plane” (Harding 1991: 11)
