56% of IS Papers Do Not Theorize Technology

I have a yellow highlighter mark on page 704 of the Sarker et al. (2019) PDF that I do not remember making. I found it tonight while flipping through old study notes from last semester. The highlighted sentence reads: approximately 56% of reviewed studies may be categorized as Type I. I must have read that sentence twice before: once during the BCIS 6670 class discussion when Dr. Sidorova walked us through the sociotechnical axis, and once when I was writing my study hub summaries for Day 1. Both times I understood it in the shallow way you understand a number when you are moving fast. Fifty six percent. More than half. OK, got it. Move on.

Tonight, on the third read, it stopped me cold.

I was sitting at my desk with the paper open on one monitor and my old day1.html notes on the other. The Transport Test is what did it. Sarker et al. describe it as a thought experiment based on Lee (2001). You take the entire research model from a paper and transport it from an IT context to a non-IT context. Say you move it from IT security to physical security in an elementary school with no computers. If everything still works and nothing in the model needs to change, the paper is Type I. IT was not doing anything. It was just background.

I started applying it to papers I know. A study about how personality traits predict career advancement of IT employees. Take out the IT and the model still holds perfectly in any industry. A study about team cohesion in software development teams. The same model would work for a marketing team. A study about what drives user satisfaction with an ERP system. Change ERP to any workplace tool and the logic does not break. I kept going through my mental library of IS papers and I kept failing them.

The first two times I read Sarker et al., I treated the six types as a taxonomy to memorize. Type I is predominantly social. Type II is social imperative. Type III is additive antecedents. Type IV is interplay, the gold standard. Type V is technical imperative. Type VI is predominantly technical. That is the kind of memorization that gets you through a class discussion but not through comps, because it does not force you to see what is actually at stake.

What is at stake is whether the IS discipline has a reason to exist. I have written before about Markus and Robey's emergent causal structure and how it shapes IS causal agency, but Sarker et al. give the identity argument an empirical backbone that the earlier conceptual arguments lacked.

Sarker et al. frame the sociotechnical perspective as the axis of cohesion for the discipline. That is Abbott's term. An axis of cohesion is the shared frame that gives a discipline its common language and broadly accepted research orientation. For political science it is power. For anthropology it is the ethnographic method. For IS it should be the sociotechnical perspective, the idea that social and technical factors interact to produce outcomes and neither gets priority. But the review of MISQ and ISR from 2000 to 2016 shows that the discipline is not actually sociotechnical in practice. It clusters at the edges: 56% Type I on the social side, about 6% Type VI on the technical side, and only about 13% Type IV where genuine interplay happens.

The three dangers finally clicked for me on this read. The first is uneven emphasis. Most research gathers at the two extremes, leaving the middle dangerously thin. The second is limited variety in relationship types. When social and technical factors do appear together, researchers almost only study fit or joint optimization. Sarker et al. argue we should also study contextual, inscribed, imbricated, disharmonious, and role-reversal relationships. The third danger is instrumental focus. About 91% of IS research looks only at efficiency and productivity outcomes. Only 9% considers humanistic outcomes like well-being, fairness, or ethics. I had read that 91% number before too, but I had not connected it to the 56% number. They are the same problem expressed differently. When you do not theorize technology, you also do not theorize its consequences for people.

The frustrating thing is that I could have understood all of this on the first read. The paper says it plainly. But I was not ready because I did not have the context yet. I had not read Orlikowski and Iacono (2001) closely enough to appreciate their five views of the IT artifact. I had written earlier about why the IT artifact is not optional, but I had not fully absorbed the scale of the problem that Orlikowski and Iacono documented. I had not sat with Benbasat and Zmud (2003) long enough to feel the identity crisis as a real threat rather than an abstract debate. I had not tried to explain the variance versus process distinction from Mohr (1982) to myself out loud. All of those readings converged tonight in a way they had not before. The identity crisis is not resolved. The evidence is sitting in the numbers.

I think what changed on this third read is that I stopped reading the paper as a source of information and started reading it as a diagnostic tool. Not what does Sarker et al. say, but what does their framework reveal about the papers I have already studied. When you apply the Transport Test to your own mental library, the 56% becomes personal. You realize how many papers you have highlighted and annotated and written summaries for are Type I. You realize that your own understanding of IS research is shaped by a literature that mostly does not do what the discipline claims to do.

There is something unsettling about that. But there is also something clarifying. The three dangers are not abstract warnings for journal editors. They are warnings for me about what kind of researcher I would become if I only reproduce the dominant pattern. The paper explicitly says that the dominance of Type I research is partly explained by the fact that data from human subjects is easier to collect, borrowed theories are easier to apply, and Type I papers are easier to publish because editors and reviewers already know what to expect. Those are career incentives. They do not make the research bad. But they do mean that producing genuinely sociotechnical research requires swimming against a current that most of the discipline follows.

I closed the PDF and looked at my yellow highlighter mark again. I still do not remember making it. But I will remember what it means now.