Potential for racial bias raises concerns for radiology AI

In July 2021, a team of 20 researchers from four countries led by Dr. Judy Gichoya from Emory University posted a preprint paper detailing how deep-learning models demonstrated consistently high performance for predicting a patient’s racial identity across multiple imaging modalities and anatomical locations.

Significant changes were made from the original preprint after several rounds of peer review, including polishing of the message and the addition of new experiments, according to a spokesperson from MIT’s Computer Science and Artificial Intelligence Laboratory.

However, the researchers still concluded that AI algorithms could readily learn to identify patients as Black, white, or Asian from medical imaging data alone — even when radiologists could not — and that this capability was generalizable across multiple imaging modalities and to external environments.

Despite repeated experiments and attempts to understand why the algorithms could achieve these results, the researchers were still unable to find an explanation.

“We strongly recommend that all developers, regulators, and users who are involved in medical image analysis consider the use of deep learning models with extreme caution as such information could be misused to perpetuate or even worsen the well documented racial disparities that exist in medical practice,” the authors wrote.

Highly accurate performance

The researchers initially developed a deep-learning model using three large datasets: Emory CXR, MIMIC-CXR, and CheXpert. The algorithm was tested on both an unseen subset of the dataset used to train the model and a completely different dataset.

To assess whether this capability was limited to chest x-rays, they also developed other algorithms that analyzed nonchest x-ray images from other body locations, such as digital radiography (DR), lateral cervical spine radiographs, or chest CT exams. These algorithms…