MIXING QUALITATIVE AND QUANTITATIVE METHODS IN PURSUIT OF
RICHER ANSWERS TO REAL-WORLD QUESTIONS
Sharon Gilad
ABSTRACT
Qualitative methods are relatively scarce in public administration research. This imbalance
between qualitative and quantitative methods poses three significant concerns. First, there is a
risk that measurement hurdles, coupled with the distance that quantitative methodology fosters
between academics and administrative practice and practitioners, undermines our inclination and
capacity to study policy-meaningful research questions that matter in the real world. Second, and
related, the causality underlying the real problems that policymakers and public organizations
face is often much too complex to be captured by one type of methodology, whether quantitative
or qualitative. Third, quantitative methodology is most conducive to testing already available
theories and hypotheses, as opposed to theory building. I propose that the answer to these
concerns lays in denouncing commitment to abstract philosophical divisions and advancing
collaboration between qualitative and quantitative researchers and versions of mixed methods
that transcend mere triangulation. These arguments are illustrated in relation to the study of
bureaucratic discrimination of minorities.
-0-
,INTRODUCTION
Social-science methodology textbooks, and research-methods courses, commonly mirror the
taken-for-granted division between qualitative and quantitative research. This division is said to
embody not only different technical tools for data collection and analysis, but incommensurable,
that is logically incompatible, philosophical traditions, pertaining to divergent beliefs about
ontology and epistemology. Crudely stated, these textbook distinctions associate quantitative
research with a positivist paradigm that conceives of reality as consisting of generalizable
patterns of cause and effect, which can be objectively known through deductive hypotheses
generation and their empirical verification. A related philosophical approach, post-positivism,
which is amenable to both qualitative and quantitative methods, is associated with a commitment
to objectivity, alongside recognition of the difficulties to fully access and explain the objective
reality, and thereby with a preference for hypotheses falsification over verification. Still,
qualitative research is most often associated with a constructivist paradigm, which stresses the
multiplicity in social constructions of reality, and the limitations and subjectivity of coming to
know them. Constructivists aim to inductively unpack participants’ actions based on their
situated, shared, interpretations of their social environment, which the researcher investigates and
construes, employing her distinct, non-replicable, vantage point.1
Reiterating and espousing the rationale of the above institutionalized distinctions, a recent
systematic review of the state of qualitative methodology in public administration (PA),
published in Public Administration Review, cautions qualitative researchers to pay closer and
careful attention to where they stand on the postpositivist-interpretivist divide (Ospina et al.,
1
See Riccuci (2008, 2010) for a nuanced depiction of alternative paradigms and their implications for the study
public administration.
-1-
,2018: 2). The authors go on to criticize qualitative researchers, in public administration, for their
failure to explicitly self-define their orientation and commitment to the interpretivist versus post-
positivist traditions (ibid, 6). Their analysis of 129 qualitative papers, published between 2010
and 2014, in six key public administration journals, indicated that most authors did not signal
their fundamental ontological and epistemological convictions, and that only 24% adopted what
the authors coded as an interpretivist approach. The authors acknowledge both issues with
lament, offering that this omission may be a function of ignorance or carelessness among the
community of qualitative public administration researchers.
The above paper, by Ospina et al’s (2018) provides us with important systematic
mapping of the state of qualitative methodology in our field, as elaborated below. However,
some of their guidance regarding future research requires further consideration. Contrary to the
above criticism and lament, I offer that in today’s public administration research, which is
dominated by quantitative studies, it would be artificial and unwarranted for qualitative
researchers to avow decisive allegiance to abstract notions of ontology and epistemology and to
distinct methodological camps. Instead, drawing on Morgan (2007), I propose that the choice of
research methods, in PA, should be led by the epistemological challenges that are posed by
concrete policy-relevant research questions, as opposed to general philosophical assumptions
about the nature of reality and how to know it. This approach, which questions the very
usefulness of the linkage between methodological choices and overarching philosophical
paradigms, differs from that of Yang et al. (2008) who offer that research methods, in public
administration, may draw upon alternative paradigms, so long as researchers appreciate the
distinct ontological and epistemological underpinnings of different methods.
-2-
, I further contend that the problem to be addressed, in current PA, is not qualitative
researchers’ failure to signals their ontological and epistemological convictions, but the near
exclusivity of quantitative research in our leading journals. This scarcity of qualitative research
likely undermines our collective endeavor to address real-world problems that governments face
(Milward et al, 2016; Moynihan, 2017; 2018; Roberts, 2018), and may lead to our provision of
overly reductionist explanations for what are often complex, wicked, problems. Moreover, it
hinders theoretical innovation in our field, leading us to focus on more of the same, relying on
existing indices and datasets. Drawing on the developments in adjacent fields and promising
changes in ours (Hendren et al. 2018; Honig, 2018; Mele and Belardinelli, 2018) this paper
offers versions of mixed-methods that transcend mere triangulation as having the greatest
potential to ameliorate these concerns. I start, however, with some building blocks,
conceptualizing what I mean by qualitative, quantitative and mixed methods.
CONCEPUTALIZING QUALITATIVE, QUANTITATIVE AND MIXED METHODS
Qualitative and quantitative research methods both involve an array of data types and techniques
of analysis. Qualitative methods are associated with case studies, which may be historically and
geographically bound events, organizational units or individuals, among others. Data regarding
these cases may be drawn from participant or non-participant observations, from semi-structured
interviews, focus groups or from narrative analysis of text. Quantitative data collection and
analysis comprises observational data, indexing human behavior or non-human factors, survey
data reflecting human perceptions, and experiments, with the latter involving a variety of
laboratory, survey-based, field-based and natural designs.
-3-
RICHER ANSWERS TO REAL-WORLD QUESTIONS
Sharon Gilad
ABSTRACT
Qualitative methods are relatively scarce in public administration research. This imbalance
between qualitative and quantitative methods poses three significant concerns. First, there is a
risk that measurement hurdles, coupled with the distance that quantitative methodology fosters
between academics and administrative practice and practitioners, undermines our inclination and
capacity to study policy-meaningful research questions that matter in the real world. Second, and
related, the causality underlying the real problems that policymakers and public organizations
face is often much too complex to be captured by one type of methodology, whether quantitative
or qualitative. Third, quantitative methodology is most conducive to testing already available
theories and hypotheses, as opposed to theory building. I propose that the answer to these
concerns lays in denouncing commitment to abstract philosophical divisions and advancing
collaboration between qualitative and quantitative researchers and versions of mixed methods
that transcend mere triangulation. These arguments are illustrated in relation to the study of
bureaucratic discrimination of minorities.
-0-
,INTRODUCTION
Social-science methodology textbooks, and research-methods courses, commonly mirror the
taken-for-granted division between qualitative and quantitative research. This division is said to
embody not only different technical tools for data collection and analysis, but incommensurable,
that is logically incompatible, philosophical traditions, pertaining to divergent beliefs about
ontology and epistemology. Crudely stated, these textbook distinctions associate quantitative
research with a positivist paradigm that conceives of reality as consisting of generalizable
patterns of cause and effect, which can be objectively known through deductive hypotheses
generation and their empirical verification. A related philosophical approach, post-positivism,
which is amenable to both qualitative and quantitative methods, is associated with a commitment
to objectivity, alongside recognition of the difficulties to fully access and explain the objective
reality, and thereby with a preference for hypotheses falsification over verification. Still,
qualitative research is most often associated with a constructivist paradigm, which stresses the
multiplicity in social constructions of reality, and the limitations and subjectivity of coming to
know them. Constructivists aim to inductively unpack participants’ actions based on their
situated, shared, interpretations of their social environment, which the researcher investigates and
construes, employing her distinct, non-replicable, vantage point.1
Reiterating and espousing the rationale of the above institutionalized distinctions, a recent
systematic review of the state of qualitative methodology in public administration (PA),
published in Public Administration Review, cautions qualitative researchers to pay closer and
careful attention to where they stand on the postpositivist-interpretivist divide (Ospina et al.,
1
See Riccuci (2008, 2010) for a nuanced depiction of alternative paradigms and their implications for the study
public administration.
-1-
,2018: 2). The authors go on to criticize qualitative researchers, in public administration, for their
failure to explicitly self-define their orientation and commitment to the interpretivist versus post-
positivist traditions (ibid, 6). Their analysis of 129 qualitative papers, published between 2010
and 2014, in six key public administration journals, indicated that most authors did not signal
their fundamental ontological and epistemological convictions, and that only 24% adopted what
the authors coded as an interpretivist approach. The authors acknowledge both issues with
lament, offering that this omission may be a function of ignorance or carelessness among the
community of qualitative public administration researchers.
The above paper, by Ospina et al’s (2018) provides us with important systematic
mapping of the state of qualitative methodology in our field, as elaborated below. However,
some of their guidance regarding future research requires further consideration. Contrary to the
above criticism and lament, I offer that in today’s public administration research, which is
dominated by quantitative studies, it would be artificial and unwarranted for qualitative
researchers to avow decisive allegiance to abstract notions of ontology and epistemology and to
distinct methodological camps. Instead, drawing on Morgan (2007), I propose that the choice of
research methods, in PA, should be led by the epistemological challenges that are posed by
concrete policy-relevant research questions, as opposed to general philosophical assumptions
about the nature of reality and how to know it. This approach, which questions the very
usefulness of the linkage between methodological choices and overarching philosophical
paradigms, differs from that of Yang et al. (2008) who offer that research methods, in public
administration, may draw upon alternative paradigms, so long as researchers appreciate the
distinct ontological and epistemological underpinnings of different methods.
-2-
, I further contend that the problem to be addressed, in current PA, is not qualitative
researchers’ failure to signals their ontological and epistemological convictions, but the near
exclusivity of quantitative research in our leading journals. This scarcity of qualitative research
likely undermines our collective endeavor to address real-world problems that governments face
(Milward et al, 2016; Moynihan, 2017; 2018; Roberts, 2018), and may lead to our provision of
overly reductionist explanations for what are often complex, wicked, problems. Moreover, it
hinders theoretical innovation in our field, leading us to focus on more of the same, relying on
existing indices and datasets. Drawing on the developments in adjacent fields and promising
changes in ours (Hendren et al. 2018; Honig, 2018; Mele and Belardinelli, 2018) this paper
offers versions of mixed-methods that transcend mere triangulation as having the greatest
potential to ameliorate these concerns. I start, however, with some building blocks,
conceptualizing what I mean by qualitative, quantitative and mixed methods.
CONCEPUTALIZING QUALITATIVE, QUANTITATIVE AND MIXED METHODS
Qualitative and quantitative research methods both involve an array of data types and techniques
of analysis. Qualitative methods are associated with case studies, which may be historically and
geographically bound events, organizational units or individuals, among others. Data regarding
these cases may be drawn from participant or non-participant observations, from semi-structured
interviews, focus groups or from narrative analysis of text. Quantitative data collection and
analysis comprises observational data, indexing human behavior or non-human factors, survey
data reflecting human perceptions, and experiments, with the latter involving a variety of
laboratory, survey-based, field-based and natural designs.
-3-
