The Real Test Is Month Seven

Six months after a major enterprise system launch, someone always publishes the success metrics. User registrations: 94%. Training completion: 87%. The project is declared a win. Then, eighteen months later, in a meeting nobody wanted to schedule, someone asks why only 22% of users are actually running reports from the system, why the rest are still requesting data from the analytics team directly, and why the helpdesk tickets have shifted from "how do I use this" to "can you pull this for me."

The IS field spent a long time studying adoption and almost no time studying what happens after. TAM predicts whether people will start using a system. UTAUT tells you what drives initial intention. Both treat adoption as the dependent variable and stop there. Bhattacherjee (2001) asked the question the field had mostly been avoiding: after people start using a system, what determines whether they keep using it?

He adapted Oliver's (1977) expectation-confirmation model from consumer satisfaction research and built what he called the expectation-confirmation model of IS continuance. The logic runs like this. Before using a system, users form expectations about what it will do for them. After using it, they compare their experience against those expectations. If the experience confirms or exceeds expectations, satisfaction rises. If the experience disconfirms expectations, satisfaction drops. And it is satisfaction, not the original intention to use, that drives continuance.

The source files I work from confirm the core mechanism: "continuance intention is determined by satisfaction and perceived usefulness; satisfaction is determined by expectation confirmation and perceived usefulness." That is the ECM in one sentence. Confirmation is the pivot. A system that is genuinely good but was sold as transformational will produce disconfirmation because the expectations were set wrong. A system that is genuinely mediocre but was sold as exactly what it is will produce confirmation and satisfaction.

The finding I keep thinking about is this: discontinuance can happen even when adoption was high. High initial use is not evidence that the system works for users. It is evidence that the conditions for initial adoption were present: training was available, managers encouraged it, the system was new and people were curious. None of those conditions predict what happens at month seven when the training sessions are a distant memory, the novelty has worn off, and the system is just another part of the daily workflow. At that point, what matters is whether users feel the system delivered on what it promised.

DeLone and McLean's IS success model, updated in 2003, made a related point that is easy to miss. They distinguish continued use as a separate construct from initial use. System quality, information quality, and service quality lead to use and user satisfaction, which lead to net benefits. But continued use is not the same outcome as initial use, and it requires different conditions to sustain. The model implies that IS success evaluation should happen at multiple time points, not just at launch.

The practical failure mode is what I think of as the demo gap. The system looks excellent in a controlled demonstration. The vendor's team has prepared the environment, selected the favorable examples, and eliminated the rough edges. The people who see the demo form expectations based on that experience. Then they start using the actual system in production, with real data, real edge cases, real workflow complexity, and real integration failures. The gap between demo expectations and production reality is the expectation gap that ECM predicts will drive dissatisfaction and discontinuance.

SaaS companies know this. That is why the most important metric for subscription software is often called time-to-value: how quickly does a new user reach the moment where the software demonstrably does something useful for them. If that moment happens before the trial ends, confirmation occurs, satisfaction rises, and conversion follows. If it does not happen, no amount of feature marketing changes the outcome. The user's expectations were set at signup, they were not confirmed during trial, and they churn. ECM predicts this precisely.

Enterprise implementations fail the time-to-value test systematically. Implementation takes months. Training happens before the system is configured for production use. The first real experience of the system, for most users, is after the implementation team has left and the training context has faded. The expectations set during implementation, which were often optimistic because the implementation team was trying to build support, meet a reality that is more complicated. Disconfirmation sets in slowly and quietly, manifesting not as dramatic rejection but as gradual workaround accumulation and reduced engagement.

The post-mortem that most IS implementations never get is the honest expectation audit: what did we tell users this system would do, and did it do that? Not what the system can do in theory. Not what it does for users with advanced training. What did the average user expect, and what did they experience? The gap between those two answers is the real explanation for month-seven abandonment. The system did not fail. The promise failed.

I think this is why IS implementation research that focuses only on adoption rates is systematically incomplete. Adoption measures whether people started. ECM measures whether they stayed, and it points at the mechanism that determines whether staying or leaving happens. Researchers and practitioners who design evaluation frameworks only around initial adoption are measuring the easy question and missing the important one.