AI diagnostic errors raise hospital blame unless doctors stay deeply involved

by · News-Medical

New research suggests hospitals may need more than token oversight, showing that comprehensive physician review can soften public backlash when AI-assisted diagnosis causes harm.

Paper: Public reactions to hospitals after adverse events involving AI. Image Credit: Antonio Marca / Shutterstock

Artificial intelligence (AI) is increasingly being used in healthcare for tasks such as diagnostics, treatment planning, patient communication, and clinical operations. However, if it is involved in diagnostic errors or adverse events that harm patients, public reactions may be negative, experimental findings suggest, according to a recent study published in the journal npj Digital Public Health. The authors also note that this is less pronounced when physicians remain substantively and interactively involved in AI-assisted decision-making.

Background

Along with expected gains in efficiency and performance, the use of AI in healthcare involves the risk of AI-related adverse events, “unintended injuries caused by medical management,” specifically when incorrect or incomplete AI-generated outputs are applied in patient care. These may be followed by diminished patient trust in the hospital, damage to its reputation, and legal action.

Little research has been done on how the public responds to such events, particularly in attributing responsibility and blame to hospitals. Hospitals are an obvious target because they oversee the implementation and oversight of AI systems through their organizational structure, and are often considered the primary providers of care, as in the UK.

Assessing Public Reaction to Mistakes Involving AI

The current report describes two studies. The first examined public reactions to hospitals in the context of hypothetical adverse events involving a missed diagnosis. This was varied by the source of diagnostic interpretation and the level of physician involvement.

The first study, which included 299 online participants, showed that adverse events involving AI were more likely to elicit negative reactions directed at the hospital, even when an endoscopist reviewed the AI result. However, in the latter case, the attribution of responsibility was significantly lower than in the AI-only condition, although still higher than in the human-only condition.

Substantive Physician Involvement Reduces Hospital Responsibility Attribution

The second study, which included 602 online participants, examined whether collaboration between a physician and AI influences the public reaction.

Unlike the first study, which examined broad levels of collaboration, the second study explored structurally distinct ways of collaboration, as they occur in real life. The participants were randomly assigned to one of the following conditions:

• AI-only (AI diagnosis)
• Human-only (radiologist diagnosis without AI assistance)
• Human-AI collaboration:
o autonomous (AI-led, low human involvement)
o sequential (radiologist reviewed areas flagged by the AI, moderate involvement)
o interactive (radiologist reviewed both AI-flagged areas and the full image, integrating AI input into an independent clinical assessment, high involvement)

The scenario was the same across all conditions: a pneumonia diagnosis was missed, resulting in patient harm.

Across the comparisons, public responses towards the hospital were more unfavorable when AI was used alone or with autonomous or sequential physician involvement. Participants attributed more responsibility to the hospital and were more likely to file a complaint or consider legal action when AI was involved without substantive physician involvement, compared to meaningful involvement by a human physician. Sequential collaboration did not significantly change the negative response compared to AI-only interpretation.

In the physician-only model or the interactive AI-physician condition, with high physician involvement, participants were less likely to attribute responsibility to the hospital or file a complaint against it. Legal action was less likely in the interactive collaboration condition than in the AI-only condition.

These findings emphasize the potentially protective and reassuring value of ensuring human oversight with a high level of involvement in AI-integrated tasks in healthcare. This suggests that visible and comprehensive physician oversight may be valuable for reassuring patients about AI-assisted healthcare.

Limitations

The researchers used hypothetical situations since real-world adverse scenarios cannot ethically be induced. Participants were primarily recruited from the UK, although supplementary analyses reported that the observed patterns did not differ systematically across countries.

Conclusions

This is among the earliest studies examining public reactions to adverse events associated with AI use in hospitals.

The study demonstrates that public responses towards hospitals associated with adverse events involving AI are more negative than for comparable human-only adverse events, especially without high levels of human physician involvement. Participants attributed more responsibility to such hospitals and reported higher likelihoods of filing complaints and pursuing legal action, and hospitals were still held accountable for AI-assisted decisions.

Although AI may be used as part of clinical workflows, hospitals were still perceived as being responsible for the adoption, implementation, and oversight of AI use. The role of clear communication and workflow design in preserving AI-associated efficiency without risking loss of patient confidence remains to be assessed.

Future research should also examine the varying degrees of responsibility that the public assigns to physicians, AI developers, and hospitals for mistakes made during AI-supported care. Another area of research involves differences in reaction with varying degrees of patient outcomes.

Download your PDF copy by clicking here.

Journal reference: