Clinical Decision Support: When the Algorithm Disagrees With the Doctor

The alert fires and the physician clicks through it. This is a daily occurrence in most hospitals. Electronic health record systems generate alerts for drug interactions, duplicate orders, abnormal lab values, and potential contraindications. Some of those alerts matter. Many of them, in most health systems, are so frequent and of such variable clinical importance that physicians have learned to dismiss them almost reflexively. Clinical informaticists have a name for this: alert fatigue. The phenomenon is well-documented in the health IT literature, though I want to be careful about citing specific numbers because the rates vary considerably across institutions, EHR systems, and clinical settings.

The thing that bothers me about alert fatigue is not the operational problem, though it is real. It is the accountability question hiding underneath it.

Clinical decision support systems, usually abbreviated as CDSS, are a broad category of health IT tools that provide clinicians with patient-specific information or recommendations at the point of care. The simplest versions are the drug interaction alerts that have been in EHR systems for decades. The more sophisticated versions include diagnostic support tools, sepsis prediction algorithms, early warning systems for patient deterioration, and increasingly, AI-driven tools that analyze imaging or pathology data and suggest diagnoses. The category spans an enormous range of technical complexity, clinical stakes, and regulatory treatment.

The basic CDSS functionality, drug interaction checking, allergy alerts, duplicate order detection, is mature. It is integrated into essentially every major EHR platform, and few people argue seriously that hospitals were better off without it. The problem is that maturity created its own failure mode. When an alert fires for every low-severity interaction alongside the high-severity ones, and when there are dozens of alerts per shift, the clinician's ability to sort signal from noise degrades. The alert that eventually gets dismissed as routine might be the one that mattered. Alert fatigue is not a sign that CDSS is bad. It is a sign that CDSS implementation was not designed with the cognitive load of the clinician in mind, which is a recognizable IS design failure.

The regulatory boundary is where things get more interesting. The FDA has authority over some clinical decision support tools as medical devices, depending on the intended use and the degree to which a clinician can independently review the basis for the recommendation. The 21st Century Cures Act, signed into law in 2016, included provisions that tried to define the line between regulated and non-regulated clinical decision support. The general logic is that if a tool provides a recommendation and a clinician can independently verify the basis for it, the tool is lower regulatory risk than one that provides a recommendation without surfacing the underlying reasoning. A drug interaction alert that shows the patient's medication list and the specific pharmacological mechanism is different, regulatorily, from a black-box algorithm that says "this patient is at high risk" without explaining why. The specifics of that regulatory line have been contested and clarified through FDA guidance documents in the years since the Act. I am hedging here deliberately because the regulatory landscape around AI-based CDSS is genuinely evolving and I do not want to state as settled what is still moving.

What strikes me as under-discussed in the health IT space is the accountability question when the recommendation is wrong and followed. There are several parties involved when a CDSS is deployed in a clinical setting. The vendor who built the tool. The hospital that licensed it, configured it, and decided how to present its outputs. The EHR system it is integrated into. The institution's clinical informatics team that made implementation decisions. And the physician who saw the alert and made a clinical decision. When an adverse event occurs and an investigation finds that the CDSS generated a flawed recommendation that the physician followed, who bears legal and ethical responsibility for the outcome?

The honest answer is that this is contested territory and differs significantly by jurisdiction. In most US malpractice frameworks, the physician's clinical judgment remains the standard of care, meaning a physician is generally expected to exercise independent clinical judgment rather than simply follow algorithmic outputs. But this creates a paradox. If physicians are expected to override flawed algorithms and their malpractice liability follows their independent judgment, then the algorithm serves mostly as a safety net below that judgment. But if alert fatigue has made the alerts effectively invisible, the safety net is not actually functioning. The hospital that deployed the system may have created an accountability structure where the physician is expected to catch the algorithm's errors in an environment that the hospital itself designed to make error-catching harder.

Gartner has tracked AI augmentation in clinical workflows as an emerging and growing area of health IT investment, though I want to be careful not to attribute specific adoption figures from sources I have not directly verified. The directional claim, that health systems are investing in more sophisticated CDSS and that AI-driven tools are being integrated into more clinical decision points, is consistent with what I see in industry reporting. The accountability infrastructure has not kept pace with the technology deployment, and that gap is not unique to healthcare. It shows up anywhere that consequential algorithmic decisions are made by systems embedded in professional workflows without clear governance for who is responsible when the system fails.

The version of this problem that I think about most is the case where the CDSS is right and the physician overrides it. Overtrust is the more discussed failure mode: physician follows a wrong recommendation. But undertrust has its own costs. A physician who has learned through experience that a particular alert type is usually not clinically significant will develop a heuristic to dismiss it, and that heuristic can fail when the alert is eventually correct. The system gave the right signal. The human missed it because the signal-to-noise ratio had conditioned them to ignore that channel. Neither the physician nor the system failed by any obvious individual standard. The failure is in the sociotechnical design of the whole.

The design principle that seems right to me is that clinical decision support should surface not just the recommendation but the basis for it, with enough transparency that a clinician can make a genuine independent judgment rather than a reflexive accept or dismiss. This is exactly what the 21st Century Cures Act tried to encode in law. Whether deployed systems actually achieve it is a different empirical question, and the answer probably varies considerably across institutions and system types.