Use of AI in mammography has been slower than other radiological specialities, but it’s catching up now.
You may not be ready to entrust the scrutiny of your own or a loved one’s boobs to an algorithm just yet, but it’s getting harder to argue that AI shouldn’t at least play a supporting role.
This image is entirely the fault of an AI called Midjourney. We asked for mammograms.
The NHS uses a quality assurance test called PERFORMS to keep its army of mammogram readers on its toes.
In this study in Radiology a team analysed readings from 552 humans and one commercially available AI – Lunit, which backed the study – on two small test sets of PERFORMS images. These are “enriched with challenging malignant cases” (mmm … extra cancer) and therefore not representative of typical screening populations.
The AI performed very similarly to the humans overall, coming out with an area under the ROC curve (a measure of test usefulness) of 0.93 compared with the humans’ 0.88 (a non-statistically significant difference).
Using the recall threshold recommended by the developer (analogous to the point where a radiologist is concerned enough to recall the patient for further testing) the AI beat the humans on specificity (89% to 76%).
Using a recall threshold set to mimic human readers (intuitively weird given that’s what you’re comparing it with, but that may just be a Back Page brain deficit), there was no significant difference in sensitivity or specificity – but the authors note the study was a little underpowered.
Professor Yan Chen, who led the trial, said it was too early to know exactly how AIs should be used in practice just yet, but that AIs would ultimately have to be continually assessed – just like NHS radiologists – to ensure their performance wasn’t drifting.
In an accompanying editorial, Yale professor of radiology Liane Philpotts notes that double reading is recommended in Europe and common practice in the UK but difficult to sustain amid staff shortages.
Mammography being one of the hardest types of image to read, she says, it’s great that AI can keep up … but you’d kind of hope it’d be doing better than humans by now.
That in fact was the result of a study last month in Lancet Oncology, in which 80,000 Swedish women were randomised to have their imaging read by two radiologists or with the help of AI.
The AI-assisted reading found 20% more cancers than the human duos, with a cancer detection rate of 6.1 per 1000 and 5.1 per 1000 respectively.
Of the women recalled for further testing in the AI group, 28% were diagnosed with cancer, compared with 25% of the human group.
Neither of these studies registered subjects’ ethnicity so there may be limits to generalisability.
But given scores like these, it seems to the Back Page that if we don’t start sending the algorithms down the breast imaging mines soon we’ll be missing a trick – or worse.
Send sensitive and specific story tips to penny@medicalrepublic.com.au.
