New patient-specific AI model improves precision in minimally invasive surgeries

Researchers created a new technique that accurately and rapidly matches X-rays captured during surgery with a patient's preoperative 3D medical scan. This method could make it easier for clinicians to precisely pilot minimally invasive surgical tools, leading to faster and safer procedures.

Clinicians perform many minimally invasive surgeries using real-time X-rays to help them steer devices like catheters and endoscopes through tiny incisions. But since X-rays are flat images, it can be challenging to determine exactly where surgical tools are located and oriented within the patient's body, increasing the risk of complications.

To help localize surgical devices, clinicians may manually align X-rays with preoperative 3D medical images, such as CT scans or MRIs. Artificial intelligence tools designed to streamline this process struggle to align images robustly for all patients, making them infeasible in practice.

This new system, developed by scientists and clinicians at MIT and collaborating institutions, uses an AI model that adapts to each patient in only about five minutes. The model automatically matches one patient's X-rays with 3D scans in a matter of seconds, and with sub-millimeter precision.

Named xvr (which stands for X-ray volume registration), it outperformed existing AI methods by an order of magnitude across a wide range of patients, body parts, and medical procedures.

He is joined on the paper by his advisor Polina Golland, the Sunlin and Priscilla Chou Professor of Electrical Engineering and Computer Science (EECS), a principal investigator in CSAIL, the leader of the Medical Vision Group, and co-senior author of the paper; and Neel Dey, a former postdoc in the Medical Vision Group who is now an investigator at Harvard Medical School and Massachusetts General Hospital as well as co-senior author on the paper. Additional co-authors include David-Dimitris Chlorogiannis, a researcher and clinician at Harvard Medical School; Andrew Abumoussa, a neurosurgeon at St. Luke's Marion Bloch Neuroscience Institute; Anna M. Larson, a pediatric clinician at Shriners Children's Hospital; Nazim Haouchine, an assistant professor of radiology at Harvard and Brigham and Women's Hospital; Darren B. Orbach, a physician and scientist at Boston Children's Hospital; and Sarah Frisken, an associate professor of radiology at Harvard.

Making X-rays more informative

Vivek Gopalakrishnan, postdoc, MIT Computer Science and Artificial Intelligence LaboratoryIt takes decades of training for a clinician to become skilled enough to see grainy, 2D images and understand how everything is oriented. We want to make these 2D X-rays more informative, so it becomes safer and easier to do these life-saving procedures."

A lack of high-quality annotated medical image data makes it difficult to train a deep-learning model robust enough to adapt to many patients, Gopalakrishnan says.

Rather than trying to make a machine-learning model that can be applied to all patients, the researchers built a model designed to adapt extremely well for the specific patient.

"We tailor this one specific model for this one specific patient, and it doesn't matter if it works on other people because there will be different models for those people," Gopalakrishnan adds.

Patient-specific machine learning

Xvr takes one patient's preoperative 3D scan, like an MRI or CT, and uses it to generate thousands of synthetic X-rays from many angles, producing about 1,000 images each second. It uses a physics-based simulation of the X-ray process to ensure these synthetic images are realistic.

"Instead of generating data from nothing, like some types of generative AI, this physics simulation is entirely based on the CT scan or MRI from this patient. Because xvr creates patient-specific data in a purely physics-based manner, there is no room for hallucinations," Gopalakrishnan says.

The xvr framework uses these simulated data to train an AI model that can accurately align this patient's 2D X-rays with their 3D image scan in a matter of seconds.

They collected whole-body 3D medical scans from more than 2,000 patients covering a wide range of ages, image modalities, and regions. Xvr used these diverse data to generate synthetic X-rays and train a foundation model to perform 2D/3D registration.

"So now you can get patient-specific accuracy but also in a very rapid time frame," Gopalakrishnan says.

"For the past two years, we've been carefully developing this algorithm and validating it. Now, we are collaborating closely with surgical robotics companies and clinical groups to turn this research into useful tools for navigation or deployment," Gopalakrishnan says.

This work was funded, in part, but the National Institutes of Health (NIH), the MIT CSAIL-Wistron Program, the MIT-IBM Computing Research Lab, the MIT Jameel Clinic, the MIT Health and Life Sciences Collaborative, and the Chou Family Transformative Research Fund.

Source:

Massachusetts Institute of Technology

Journal reference: