Case Study Research in IS: What Yin Actually Said

IS research papers use the phrase "case study" to describe a wide range of things. Sometimes it means a detailed story about one company that illustrates a point. Sometimes it means a year of embedded observation with systematic data collection across multiple sources. Sometimes it means three interviews at a company that happens to have an interesting system. These are not the same thing, and conflating them produces a field that talks past itself when it talks about case study methodology.

Robert Yin's work on case study research is the most cited reference for case study design in IS, and the reason is that he gave the method a rigorous framework for when to use it and what rigor looks like in that context. The framework is worth knowing precisely because it rules out a lot of what gets labeled case study research in practice.

Yin's first move is definitional. Case studies are appropriate when you want to answer "how" and "why" questions, when you have little or no control over events, and when the focus is on a contemporary phenomenon situated in a real-world context. This is not just a description of a method. It is a diagnostic for when the method fits the question. If your question is "how much" or "how many," you have a survey or an experiment, not a case study. If you want to compare adoption rates across a thousand firms, case study is the wrong tool. But if you want to understand how a specific ERP implementation went wrong, or why a hospital's staff responded to a new clinical decision support system in ways the designers did not anticipate, case study is not just defensible. It is probably the most appropriate choice.

The distinction between single-case and multiple-case designs is one of the things people most often get wrong when they cite Yin. Single-case designs are not weaker than multiple-case designs. They are different. A single case is appropriate when it is a critical case that tests a well-formulated theory, when it is an extreme or unique case, or when it is a revelatory case that gives access to a phenomenon that was previously inaccessible. Using a single case as a pilot for a multiple-case study is also legitimate, though it is different from the single case as the main contribution. The logic in all of these is that the case is chosen purposively, not randomly, because of what it can reveal.

Multiple-case designs follow replication logic, and this is where I think the most confusion lives in IS. Replication logic is not sampling logic. When you have four cases in your study, you are not trying to represent a population. You are trying to replicate findings across different contexts in the same way a scientist runs an experiment multiple times to confirm a result. Yin distinguishes literal replication, where cases are chosen to produce the same findings, from theoretical replication, where cases are chosen under conditions that theory predicts should produce different results but for specifiable reasons. This logic implies that the number of cases needed is not determined by statistical power. It is determined by how confident you want to be in the pattern of replication. Two or three cases for literal replication, more for theoretical replication.

The validity framework Yin lays out for case study research maps onto the four criteria that quantitative researchers use but requires completely different techniques for each one. Construct validity is the most commonly violated in IS case studies. It requires that you measure what you claim to measure, which in a case study means using multiple sources of evidence for each construct and maintaining a chain of evidence from the raw data to the findings. An IS case study that draws all its data from interviews with a few managers has a construct validity problem, because managers' accounts of what happened are shaped by memory, self-interest, and organizational context. Triangulating across interviews, documents, observational data, and archival records is not methodological pedantry. It is what construct validity requires.

Internal validity is about whether you have established a causal relationship when you claim one. In a case study this means ruling out rival explanations. If I claim that a new project management system improved delivery times at a firm, I need to show that delivery times did not improve for other reasons, that there was not a new project manager hired at the same time, that there was no shift in the types of projects the firm was taking on. Pattern matching against a theoretical prediction is one technique Yin recommends. Explanation building is another, where you iteratively refine the causal story as you encounter disconfirming evidence.

External validity is the one that makes IS survey researchers skeptical of case studies. It is also the one that requires the most conceptual care. Yin's argument is that case studies do not generalize to populations the way survey samples do. They generalize to theory. The question is not whether this hospital's experience can be projected to all hospitals. The question is whether the theoretical propositions the study develops can be applied to similar situations. This is analytic generalization, not statistical generalization. The difference matters, and conflating them produces the false conclusion that case studies cannot generalize at all.

Kathleen Eisenhardt's work on building theory from case studies is the other reference point that IS researchers use alongside Yin. Eisenhardt's process is specifically designed for theory development rather than theory testing. It starts with a research question that is broad enough to allow new constructs to emerge, selects cases to maximize the opportunity for theoretical insight, and relies on within-case analysis followed by cross-case pattern searching. The outputs are theoretical propositions that can be tested in subsequent work. Eisenhardt is explicit that you should enter the field with a research focus but without hypotheses, so that the theory is genuinely grounded in what you find rather than in what you expected to find.

The tension between Yin and Eisenhardt is real and worth naming. Yin's framework is most useful when you have enough theory to specify the propositions you are testing or the pattern you expect to see. Eisenhardt's process is most useful when the theory is underdeveloped and you need the cases to generate it. IS researchers sometimes treat them as interchangeable, but they have different epistemological postures. Yin leans closer to verification. Eisenhardt leans closer to discovery. Both are rigorous. Knowing which one fits your question matters.

What I notice most when I read IS case studies that are not working is that they have collected data at a site and then told a story about it. The story may be accurate. It may be interesting. But it does not carry theoretical weight because the data collection was not designed around any theoretical propositions, the rival explanations were not addressed, and the external validity claim was either absent or naively statistical. A case study that does not engage Yin's validity framework is not rigorous case study research. It is an organizational narrative with academic formatting.

IS has used case studies to produce some of its most influential work. The method can carry serious theoretical weight. But it requires the same discipline that any other method requires: choosing the right design for the question, collecting data in a way that supports your claims, being honest about what the findings do and do not support, and situating the contribution in terms that others can build on.