An artificial intelligence tool detected brain aneurysms that radiologists missed in routine clinical practice, increasing the number of identified cases by 39%, but it also produced substantially more false-positive findings than radiologists.
The findings, published in the Journal of the American College of Radiology, suggest that AI could serve as a second set of eyes for radiologists, while highlighting limitations that could affect its value in different clinical settings.
Researchers at Northwell Health evaluated an FDA-cleared algorithm from Aidoc designed to identify suspected intracranial aneurysms on CT angiography (CTA). The deep learning software analyzes scans for findings suggestive of an aneurysm and flags suspicious cases for radiologist attention.
The study included 3,856 CTA examinations across a large health system. The AI operated in “shadow mode,” meaning it processed scans without showing its results to radiologists or influencing patient care. Independent neuroradiologists reviewed cases where the AI and clinical interpretations disagreed.
Radiologists and the AI agreed on more than 96% of examinations, but each caught aneurysms the other missed. The AI detected 55 confirmed aneurysms that radiologists had not reported, increasing the number of cases found by 39% compared with radiologists alone. Radiologists, however, detected 30 confirmed aneurysms that the AI missed.
Read Also: Smartphone tool boosts cataract screening in remote India
The AI’s greater sensitivity came with a tradeoff. It detected 84.6% of true aneurysms compared with 71.8% for radiologists, but its positive predictive value was considerably lower—78.2% versus 92.7%. In other words, more than one in five aneurysms flagged by the AI were not confirmed as true aneurysms.
This limitation was more apparent among findings identified only by the AI. Of 101 AI-only findings, 55 were confirmed aneurysms and 46 were false positives.
The AI’s performance varied by clinical setting. Among inpatients, it added 18 true aneurysm detections while producing seven false-positive alerts. Results were also favorable in the emergency department. In outpatient care, however, the AI added only four true detections and generated more false-positive than true-positive findings.
Many of the additional aneurysms detected by the AI were small, offering opportunities for risk assessment and monitoring before rupture.
Overall, the findings indicate that AI may help radiologists identify more aneurysms, but its added sensitivity comes at the cost of false-positive findings requiring physician review. The differences across care settings highlight the need to evaluate medical AI in the specific clinical environments where it will be used.
