Single photo predicts your chances of surviving cancer treatment

Researchers at Mass General Brigham Cancer Institute have developed an AI tool that determines your biological age and your chances of surviving challenging cancer treatment – all from a single photograph of your face.

Currently, clinicians intuitively assess how well a patient might fare when they undergo treatment. Older patients might also need to undergo rigorous tests when they're already not feeling great. These tools could supplement the former, and potentially reduce reliance on the latter.

"As the population ages, physicians need better ways to estimate physical reserve and frailty before recommending intensive treatments such as radiation, surgery or systemic therapy," said corresponding author Raymond Mak, MD, a radiation oncologist at the institute and a corresponding author on the study.

"To our knowledge, this is the first demonstration that an AI model can directly predict cancer survival from a simple photograph," said Hugo Aerts, PhD, co-senior author and director of the institute's AI in medicine program.

The team first built an AI foundation model called FAHR-Face (Foundation Artificial Intelligence for Health Recognition) to assess health-related attributes from regular ol' photos of people's faces. This goes far beyond showing an algorithm a few pictures of ill patients and asking if a person in a new photo also looks sick.

Using a self-supervised learning method called masked autoencoders, it was trained on more than 42 million photos of faces and commanded to reconstruct partially obscured photos to get a holistic understanding of facial structure and features. This helped the model generate robust digital biomarkers for various health assessment tasks.

The model was then fine-tuned to create two separate tools. FAHR-FaceAge estimates a patient’s biological age – a measure of physiological changes associated with aging, rather than a chronological account of birthdays. FAHR-FaceSurvival directly predicts survival outcomes in patients with cancer.

Interestingly, the FaceAge model didn't get tripped up by makeup, facelifts, glasses, or different lighting in photos, though it did make small errors when it looked at images of people who had nose jobs.

The FaceAge model estimated people's chronological age more accurately than other existing models out there, with a low average error of 5.1 years (compared to another model's average error of 6.92 years).

This model is useful for determining how well – or how poorly – a person might be aging. Patients who appeared five or more years older than their actual age had a 21% higher risk of mortality compared to those who looked their actual age. Conversely, looking five years younger than one's real age was associated with 17% higher survival rates.

The other model, FaceSurvival, was further trained on facial photographs of more than 34,000 cancer patients undergoing radiotherapy, as well as two separate groups of nearly 4,000 and 5,000 cancer patients, respectively. The idea was to test the model's ability to determine survival risk for patients from photos.

FaceSurvival successfully identified patients at high risk of mortality; those it placed in the highest-risk category had more than triple the mortality rate, compared to those in the lowest-risk category. What's more, in the highest-risk score group, 52% of patients died within 90 days, compared to only 2.1% in the lowest-risk group.

When combined, the two models provided complementary insights while capturing different information about a patient's health from their photo. FAHR-FaceAge focuses on the nasolabial folds, the forehead area just above the orbits, and the temples, while FAHR-FaceSurvival evaluates the region below the eyes and around the nasal bridge.

You can see these regions highlighted in yellow in the header image at the top of this article. The image on the left depicts the features most important to FaceSurvival's predictions of cancer survival outcomes, while the one on the right shows the features most important to FaceAge's estimates of biological age.

According to the team, this shows that consumer-grade facial photos can serve as a reliable, non-invasive biomarker to help clinicians gauge patients' overall health and treatment risk.

"This foundation-model approach gives us a scalable way to develop multiple facial health biomarkers rather than training each model from scratch," noted Aerts.

If you have 10 minutes and a photo to spare, you can participate in an expanded study the team has opened to the public via this page, and receive your own FaceAge readout.

Source: Mass General Brigham

This research was published in The Lancet Digital Health