Quant vs Qual Is the Wrong Debate

Every IS PhD student I know has sat through at least one seminar where someone said something like "well, that is just a qualitative study" as if the word "qualitative" were a verdict. Usually said by someone whose own work involves a five-construct structural equation model with a survey of 200 MBA students. Usually said about a study that spent two years inside an organization watching how people actually used a system. The contempt goes in both directions. Interpretive researchers have their own version of the dismissal, usually involving the phrase "causal obsession" or "black-box thinking." I have been in rooms where both were said, sometimes in the same hour.

The thing that bothers me about this debate is that almost everyone in it is arguing about the wrong thing.

Lee (1991) showed something that the IS field absorbed intellectually and then largely ignored in practice. Positivist and interpretive approaches can be integrated. His framework moves through three levels of understanding: subjective understanding, which captures how actors make sense of their own situation; interpretive understanding, which is the researcher's field-based interpretation of those meanings; and positivist understanding, which involves formal propositions assessed by falsifiability, logical consistency, relative explanatory power, and survival in competition with alternatives. These are not separate paradigmatic camps. They are levels of a unified analytical process. A researcher can and often should move through all three.

The fact that this argument was made in 1991 and we are still treating quant and qual as opposing tribes in 2026 is, to me, a sign that the debate was never really about epistemology. It was about identity.

Klein and Myers (1999) gave IS researchers the sharpest set of criteria for evaluating interpretive field research: the hermeneutic circle, contextualization, interaction between researcher and subjects, abstraction and generalization, dialogical reasoning, multiple interpretations, and the principle of suspicion. These are not loose criteria. They are demanding ones. A study that does interviews but does not engage with how the researcher's presence shapes what is said, and does not situate the findings in historical and organizational context, and does not account for the possibility that different participants hold genuinely different interpretations, is not interpretive research in Klein and Myers' sense. It is qualitative data collection without paradigmatic commitment.

That last sentence contains the distinction I want to spend more time on. Method and paradigm are not the same thing. A case study can be positivist: Yin (2018) designs positivist case studies with construct validity, internal validity, external validity, and reliability criteria borrowed from the experimental tradition. A survey can be interpretive, at least in principle, if the constructs are grounded in actors' subjective meanings and the analysis preserves that meaning rather than reducing it to a distribution. The study-hub note I kept returning to while preparing for comps was this: "Paradigms differ in assumptions about reality, knowledge, inquiry, and evaluation, not merely in preferred methods."

Eisenhardt (1989) built her multi-case theory-building framework on a positivist logic. Her design assumes that patterns across cases generalize, that replication logic applies, and that within-case analysis followed by cross-case comparison can produce propositions that look like the outputs of positivist research. Sarker, Sarker, and Sidorova (2006), using actor-network theory, did something interpretive: they explained a specific business process change failure by following the translations and betrayals in a network of humans and artifacts, without claiming that their findings would replicate in a different context. Both used qualitative data. The paradigmatic assumptions were completely different.

This matters for how we evaluate studies. Positivist research is evaluated by reliability, validity, and the possibility of replication. Interpretive research is evaluated by Klein and Myers' seven principles. Critical realist research is evaluated by explanatory power and the logic of retroduction: inferring from observable events the underlying mechanisms that generated them. If you evaluate a Klein-and-Myers interpretive study by asking whether its findings replicate across samples, you are applying the wrong criteria. If you evaluate a Yin-style positivist case study by asking whether the researcher adequately reflected on their own role in shaping the data, you are applying the wrong criteria. The evaluation criteria are part of the paradigm.

What I find genuinely interesting about the current moment in IS is that machine learning research is rediscovering these issues from a different direction. Large-scale pattern-finding in big data looks superficially like positivism: find the relationship in the data, test whether it generalizes, report the coefficient. But the way many ML researchers actually work, especially in prompt engineering and evaluation of language models, is closer to something interpretive. They are asking what the model "means" by an output, how context shapes what the model produces, whether a given response is appropriate for a given situation. These are meaning-making questions, not hypothesis-testing questions. The methods are often quantitative but the epistemological logic is something else.

Goles and Hirschheim (2000) argued for moving beyond paradigm wars through methodological pluralism, paradigm interplay, and pragmatism. Their version of pragmatism is not the abandonment of epistemological standards but the choice of approach based on what the research question actually requires rather than paradigmatic loyalty. I think this is right in principle, though it requires a level of philosophical self-awareness that most researchers, including me on a bad day, do not consistently maintain. It is easy to say "I am being pragmatic" when what you mean is "I am using the method I already know."

The strongest version of the argument I want to make is this: calling your study "qualitative" tells me almost nothing about your paradigmatic assumptions. It tells me something about your data collection approach and perhaps your analysis technique. It does not tell me whether you believe in an objective reality that your interpretations are trying to approximate, whether you believe meaning is socially constructed, whether you are looking for causal mechanisms beneath the surface of observable events, or whether you see yourself as an advocate for the people you are studying. All of those positions are compatible with interviews and observation. The paradigm is the set of answers to those deeper questions. The method is the tool.

When I hear someone say "this field is too quantitative" or "this field is too qualitative," I want to ask: compared to what paradigmatic standard? If you think IS has too many SEM studies, the argument you are actually making is that most IS research is operating within positivist assumptions, and that interpretive or critical realist questions are being underserved. That is a substantive argument worth having. But the debate needs to name the paradigm, not the method. Otherwise, we are fighting about the hammer when the real question is what kind of house we are trying to build.

For the fuller argument about what paradigm choice actually implies, the post on your paradigm is not neutral goes deeper into what different ontological commitments require of a researcher. And for what counts as theoretical evidence, the question of what a paradigm takes to be evidence connects directly to what theory is and is not, where the mechanism question sits at the center of the argument.