What Makes a Good IS Paper?

I used to think a good IS paper was one that got accepted. That made sense as a working definition early in my PhD, because acceptance was the only concrete outcome I could observe. Then I started reading the papers that did get accepted, noticing which ones I kept coming back to and which ones I filed away and never opened again. The gap between publishable and genuinely useful is wider than I expected.

Whetten (1989) asks a deceptively hard question: what constitutes a theoretical contribution? His answer is structured around four elements. What names the constructs. How specifies the relationships among them. Why provides the argument that justifies why those constructs relate the way they do. And the boundary conditions say when, where, and for whom the explanation should hold. The "why" is what Whetten calls the theoretical glue. A paper can have constructs, relationships, and hypotheses and still not have theory, because theory without the why is just a list with arrows. Whetten also frames how to evaluate contributions: by scope and degree. Scope is how broadly the theory applies. Degree is how deeply it explains. The strongest contributions are either very broad or very deep, and ideally both.

I think about this whenever I see a paper that extends TAM to a new context, adds two moderators, and claims a theoretical contribution. The constructs are named. The relationships are specified. The hypotheses are tested. But the why has not changed from Davis (1989). Nothing about why perceived usefulness predicts intention has been explained more deeply. The scope has changed, because the context is new, but degree has not moved. That kind of paper passes review constantly, and it is not nothing, repetition in different contexts does matter for accumulating evidence, but calling it a theoretical contribution overstates what happened.

The "so what" test is the informal version of Whetten's formal standard. I first heard it from a faculty member who used it to dismiss a paper summary I had prepared. "So what does this tell us that we did not already know?" I had no good answer. The paper I had summarized introduced a model, tested it, confirmed the hypotheses, and stopped there. There was no moment where the findings changed how you would think about the phenomenon. The model explained variance, which is good, but it did not re-explain anything or open any new questions. A paper that passes the "so what" test adds something to the conversation that was not there before. It shifts how you would design future research, or how you would interpret an existing finding, or what constructs you would choose to measure.

Gregor and Hevner (2013) give a sharper version of this in the design science context. Their contribution matrix has four quadrants organized by problem maturity and solution maturity. Invention is a new solution for a new problem. Improvement is a better solution for a known problem. Exaptation is a known solution applied to a new problem. Routine design is a known solution for a known problem, and it is not research at all. The matrix is diagnostic. If you can place your paper clearly in a quadrant, you know what kind of contribution you are claiming. Most published IS work lands in improvement and exaptation. Pure invention is rare. But the matrix also exposes papers that are claiming invention while actually doing routine design, which is where the problem is.

The review process at IS journals does not directly ask Whetten's questions or use Gregor and Hevner's matrix. Reviewers ask their own questions, and these vary enough that what gets emphasized is partly a function of who happens to be reviewing your paper. But there are things that show up consistently in good reviews. Strong motivation: why does the question matter, and why does it matter now, in a way that is specific rather than generic? Clear theoretical gap: not just "this has not been studied" but "existing theory cannot explain this and here is why." Appropriate method: a method that fits the research question rather than the method the researcher is most comfortable with. Credible analysis: results that would survive a skeptic running the same analysis. And honest limitations that do not just list every possible threat to validity but actually say which ones matter most for interpreting the specific findings.

The honest limitations section is underrated. I read papers where the limitations paragraph is three sentences about sample size and common method bias, buried at the end, clearly written to satisfy the reviewer who will ask about it rather than to tell the reader something. A real limitations section changes what conclusions you can draw. It says: given what I found, you should believe claim A strongly, claim B with more caution, and you should not generalize claim C to this other context because of this specific feature of my research design. That kind of honesty is actually more convincing than the false confidence of a paper that presents every finding as equally certain.

What separates a technically correct paper from one that advances the field is often whether it has a genuine antagonist. I mean this not in a combative sense but in an intellectual one. The best IS papers are in dialogue with a specific inadequacy in existing theory. Markus and Robey (1988) are doing this when they argue that IS research cannot keep treating technology as either the sole cause of organizational outcomes or as merely a passive input. Their paper has something to push against. Baird and Maruping (2021) are doing this when they argue that "use" as the foundational IS construct cannot capture what happens when artifacts become agentic. That paper also has something to push against, and the push is precise enough that you understand exactly what the prior literature got wrong and why.

Papers without an antagonist tend to describe a gap rather than address a failure. "No one has studied X in the context of Y" is a gap. "Prior theory predicted Z, but X in context Y produces a contradictory result that requires us to revisit the mechanism" is an antagonist. Gaps invite incremental extensions. Antagonists invite reformulation. The field publishes both, but the papers that stay in the conversation are usually the ones with the antagonist.

For a PhD student, this is harder than it sounds because identifying a genuine antagonist requires knowing the literature well enough to know what it actually predicted and what it actually found, not just what it claimed to contribute. That takes time. Reading papers once for comprehension is not the same as reading them critically enough to see where the mechanism breaks down. I am still learning to do the second kind of reading, and the difference between the two is, I think, most of what separates a junior researcher from a senior one.