Physicians often trust incorrect AI treatment recommendations, study finds

AI systems can help physicians categorize patients according to their different care needs, such as whether a patient is more or less likely to benefit from a certain treatment. Since these systems are not perfect, they are meant to be used as suggestions, with potential errors caught and corrected by physicians.

Prior research has shown that, in general, people struggle to notice and correct mistakes made by AI. To explore how this challenge may extend to physicians, Vinas and colleagues analyzed data from 223 physicians who anonymously participated in online experiments.

These findings highlight potential challenges for incorporating AI-based classification into healthcare. Future research could build on this study, such as by developing and testing strategies and protocols that could increase human critical thinking and detection of AI errors, in order to maximize the benefits of the human-AI collaboration while minimizing potential errors."

Lead author Aranzazu Vinas notes: "In both experiments, physicians mostly trusted the AI's classifications and had trouble learning from the feedback. Furthermore, in the second experiment, professionals did not notice that the treatment was completely ineffective."

Co-author Fernando Blanco summarizes: "It is important to investigate the errors that humans (including doctors) make when working with algorithms, in order to learn how to minimize the problems that arise from them."

Source:

PLOS

Journal reference: