The Netherlands has an extensive national screening program for breast cancer. Every year, about 1 million breast X-rays are reviewed. A mammogram is routinely examined by not one, but two radiologists. Nowadays, AI systems are increasingly being used to assess mammogram images. And how is that working out? In many cases, the computer can do this faster and better than a specialized radiologist. Will the mammogram radiologist eventually become redundant? Or will the computer remain the specialist’s assistant?
“The latest AI systems work with deep learning techniques and the results are improving rapidly,” says Ritse Mann, mammary radiologist at the Radboudumc university hospital in Nijmegen, the Netherlands. He is a proponent of the use of artificial intelligence within his field and already uses it on a regular basis. “A number of AI programs are now just as good or even better at detecting tumors on mammograms than I or my colleagues. We are on the verge of a revolution and I expect AI in breast cancer detection to really take off.”
The use of computer systems in breast cancer diagnostics is not new: Computer Aided Detection, or CAD systems, have been used by radiologists for much longer. This the of system is able to recognize abnormalities in breast x-rays. However, these are not self-learning programs. Instead they are software fed with knowledge from radiologists. Mann: “The beauty of the new AI programs is that they learn to recognize tumors themselves on the basis of a large number of mammograms. And they are very good at doing that.”
Screenpoint Medical
At the moment, there are a handful of commercial companies around the world that make AI systems for assessing mammograms which are already as good or even better than the average mammogram radiologist. One such company is Screenpoint Medical, a spin-off from Radboudumc. The company developed the AI program Transpara that can assess 2D and 3D mammograms. This program is now used in more than twenty countries and its development is proceeding at lightning speed. The program detects over a quarter more tumors than a year ago. Mann also works with Transpara at the Radboudumc. “We do research together. Screenpoint Medical has become one of the leading companies in this field.”
National breast cancer screening program
By far the largest proportion of mammograms in the Netherlands are performed as part of the national breast cancer screening program. The rest are made in hospitals, for example, for a lump in the breast. It would save a lot of time and human resources if the computer would help assess the huge number of mammograms. Mann: “I can read 300 mammograms in a morning, so it goes pretty fast.”
Each mammogram is reviewed by two radiologists, sometimes even a third radiologist. “Despite this, we miss quite a few tumors. Sometimes that’s because the tumor is very small. But also because it is basically work done by people and you can overlook something,” says Mann. “If a tumor is found in a woman between two rounds of screening who has taken part in the national screening study, it turns out in a quarter of the cases that the tumor was actually already visible during the last screening, but we missed it. That is a painful fact. But it happens.”
Breast X-ray gets a report card
AI applications such as Transpara can be used to reduce the number of missed tumors. The program scores mammograms from 1 to 10. With 10 indicating the highest risk of cancer and 1 representing the lowest risk. As it happens, with a score of 10, the likelihood of breast cancer is still only about 5 percent. With a score of 1, that chance is even much lower, albeit that the chance is still there.
If a tumor is clearly visible, both a human being and a computer will be able to detect it. If nothing can really be seen, then this means that it is usually clear as well. The gain but also the pitfalls lie within those grey areas, Mann maintains. “Suppose you let the AI system filter out all mammograms that score higher than a 5. The radiologist then has to assess these personally, yet those with a lower score are no longer examined. We then automatically miss about 5 percent of all tumors, as they are in the group which has a low risk assessment (1-5).”
We miss quite a few tumors. Sometimes that’s because the tumor is very small. But also because it is basically work done by people and you can overlook something,” Ritse Mann, mammogram radiologist at Radboudumc
“On the other hand, we could find more tumors in the group with a higher score than we do at present. On a net basis, it should not matter how many tumors are found in time. But how do you explain to the patient that even an AI program makes mistakes from time to time? The bar for a computer system is set higher, so the radiologist will also have to take a look at those low-risk images to see if the computer really has not missed anything.”
AI as an aid
For now, reviewing mammograms remains primarily the work of humans, because we are not ( yet) comfortable with the idea that computers can do it better. But for Mann, the question is not if but rather how best to use AI software in detecting breast cancer. Nearly 75 percent of radiologists now believe that AI should be given an independent role in assessment. Without involving humans, as it were. A quarter would rather use AI as an aid on the side.
Research shows that an AI system has as much sensitivity as a single radiologist. Mann: “But it is not yet as good as two radiologists working together. You could, of course, have the computer and the radiologist work together to make an assessment. Then you halve the workload and find more tumors. That’ s when you would use the AI system as a valued colleague. So, I can imagine something like: ‘How about it?'”
Extra diagnoses
The AI system regularly finds tumors that a radiologist often will not spot in an image until two years later. ” Evidently, the computer is better at recognizing a tumor at an early stage. Consequently, it would be good to take another look at things that the computer finds particularly suspicious. Because there are lives that can be saved in doing that,” Mann says. “Suppose that the system gives a score of 9 or 10. Then we can offer these women an MRI or some other additional screening test. They will most likely benefit from this more intensive screening or some other type,” Mann points out.
Do it straight away, you might think. But there are drawbacks, too, according to Mann. “If you are called back after a screening, the chance that you have cancer is about 20 to 25 percent. So many women are called back in error. That then leads to stress, extra diagnostics and significantly higher costs. We undoubtedly find more tumors, but it is not certain that the chance of survival is increased. We do eventually find many of those tumors, even if at a later stage. That is a trade-off we have to make.”
Accountable
Who is accountable if a mistake is made? Will we accept higher costs for additional diagnostics? How many additional tumors do you find and how many do you miss if you start using AI on a large scale to detect breast cancer? These are major questions as far as Artificial Intelligence in healthcare is concerned. “I expect it to become more and more integrated. Soon a risk analysis will roll right out of the screening,” says Mann. ” It is still in its infancy. There are commercial companies behind it, but there is as yet not a proper quality standard in place that a program needs to meet.”
At the moment, it is still unclear how AI systems behave in the even of mass screenings. This is because the programs are not yet authorized to be linked to the screening database. Mann: “We run into problems with privacy laws, while the screening itself is totally anonymous.”
Mann expects that the door is now slightly ajar and will soon swing wide open. “I think AI software will eventually assess mammograms on their own. We are simply no match for it. In the Netherlands, we now have a shortage of lab technicians in particular, not so much of radiologists. This means that taking mammograms is the main hurdle, not the assessment. But take Great Britain as an example: There are not enough radiologists and the incentives are greater. Or look at Africa, where there is no radiologist on hand. That is when you immediately switch to an AI solution.”
Also interesting: Radboudumc deploys AI to recognize COVID-19 on lung images
Related Posts
