HYDERABAD: A team of researchers at the International Institute of Information Technology-Hyderabad (IIIT-H) has found that heatmaps generated by medical artificial intelligence (AI) models to indicate disease in chest X-rays may not always correspond to the areas radiologists consider clinically relevant.
The study, by IIIT-H’s Language Technologies Research Centre (LTRC) and led by Prof Parameswari Krishnamurthy, examined four vision-language models (VLMs) — MAIRA-2, MedGemma-4B, LLaVA-Med-1.5 and LLaVA-1.5 — against thousands of publicly available chest X-rays. The findings have been accepted at MICCAI 2026, the International Conference on Medical Image Computing and Computer Assisted Intervention.
The researchers sought to examine a question that goes beyond whether an AI model arrives at the correct diagnosis: does the area highlighted by the model correspond to where a radiologist would actually locate the disease?
To answer this, the team compared AI-generated heatmaps with regions identified by radiologists and conducted a reader study involving two radiologists.
The results showed that a model could appear to identify the correct part of an X-ray without necessarily locating the disease in the same way as a radiologist.
Dr Syed Faizan, principal investigator, said the models may diagnose first and then use that information to place the heatmap. Removing diagnostic information reduced localisation performance, suggesting the diagnosis may influence the model’s visual reasoning.
Interpretation is key: Docs
The human assessment also revealed a notable difference from the automated audit. While MAIRA-2 was ranked higher for overlap with radiologist-identified regions, the two radiologists rated MedGemma higher.
Researchers said this difference could stem from how radiologists interpret visual information. A radiologist may need to examine not only the precise site of an abnormality but also the surrounding area to understand the extent of disease. A broader heatmap could therefore be more clinically useful than one that tightly overlaps with the suspected lesion.