On this page
High-Level Overview of Federated Learning x Medical Imaging
If you follow Artificial Intelligence and Machine Learning, youâve probably heard a lot about the applications of these emerging technologies in the healthcare space. This entire hype train has stemmed from the raw processing power that machine learning models are able to employ. Recently, Machine Learning models have been utilized and leveraged to generate and build statistical models from medical data. These models have had various use cases, ex. AI has been used to assist radiologists in performing diagnoses using Convolutional Neural Networks. Despite this, there has been an evident hesitation to exploit medical data to develop efficient and effective machine learning models. These struggles are characterized by the fact that good models cannot be generated with the small datasets that exist at each site. To add salt to the wound, many medical images are not properly labeled and radiologists need to spend time and resources in labeling the data to train the model. Taking a step back, adding this work onto the plates of radiologists does not seem logistically sustainable considering the increasing amount of data being received by sites every year.
In order to avoid this problem growing unsustainably, a new machine learning framework has been proposed, Federated Learning. Federated learning is a new subset of AI that is being researched currently, it was initially discovered a couple of years back, in 2017.
This framework allows for local sites to train a universal model and iterate on the model to increase efficiency and accuracy.
The core feature behind Federated Learning is the fact that this model can be shared and used worldwide without sharing datasets, which addresses a huge problem around data sharing. Data sharing is not systematic in healthcare as collecting, curating and maintaining a high-quality data set takes time, effort and expense.
Understanding The Need đ§
Research in the Artificial Intelligence and Machine Learning spaces have led to major breakthroughs and innovations in radiology, pathology, and genomics. As aforementioned, AI has generated lots of success in detection and diagnosis by using medical images as inputs. Using images such as radiography and computed tomography (CT) scans allow the AI to self-train and learn to identify patterns and similarities to provide an accurate detection/diagnosis. Advancements have allowed AI to learn from raw images via network algorithms. In doing so, the outputs generated have been accurate and consistent, showing the potential role it plays in assisting radiologists.
The biggest example of this was when Stanford employed a convolutional neural network with 121 layers (CheXNet) which managed to outperform 4 radiologists after being trained on a big dataset.
Without a doubt, the AI model showed promise, but it had a major shortcoming that cannot go unnoticed: the dependency on a large amount of data. Deep-learning models require a ton of data to ensure that the model is trained to a high level of accuracy. Despite imaging centers having a large database, the images are often unlabeled which doesnât allow the model to learn effectively. Unlabeled images can only be labelled and annotated by a trained radiologist, which is a waste of time and resources.
As such, producing a well-trained and annotated dataset is a time-consuming process and it is one of the major drawbacks of developing autonomous AI solutions for medical imaging.
To get around this huge problem, Federated Learning allows for a universal model to be trained across a large range of healthcare institutions to optimize for a highly accurate and efficient AI model. At a high level, an algorithm is employed to gather labelled data from these healthcare institutions to expand access to data and as a result, improve the ability of the AI model to detect and diagnose. This âcollaborative learningâ model has raised concerns about data privacy and security and it may endanger patient privacy. However, to combat these issues, federated learning allows for algorithms to learn from local data without extracting the data from the local site. Thus, the patientâs data remains private and secure.
Understanding Federated Learning đ¤
Before jumping right into how these federated learning models will be employed in the healthcare/medical imaging space, itâs important to understand what Federated Learning is and how it works.
With Federated Learning, organizations/institutions are able to work with each other to solve a data-based/machine learning problem by leveraging a central server/service provider. To achieve this, a deep-learning model is sustained and iterated upon within a central server. As aforementioned, the central server is distributed to every remote silo data center; in doing so, the sites are able to keep their data localized and private. The data from each site/collaborator is never shared during training; instead of moving the data to the central server, also known as conventional and centralized machine/convolutional deep learning, the central server has a universal model. As such, each institution has its own personalized model based on the data that is available at the said institution. After training the model with the data, each data center provides feedback on how to iterate/improve the model â this is usually done by weight or the error gradient of the model. The central server processes all the improvements and recommendations and incorporates the changes that will only better the model. In the instances where the feedback isnât valuable, the central model will simply dismiss the edge cases.
In other words, Federated Learning is essentially decentralized Machine Learning
If youâd like to read more about Federated Learning, check out this article!
Benefits of FL âŹď¸
Federated Learning is a rather new concept being applied in the medical field. However, it comes with a wide range of benefits.
First, FL allows the central model to learn/train from a large and diverse dataset obtained from the many data centers. Patient data and images at individual institutions are obtained from a very specific subset. This makes it highly unlikely that the data is seen or shared with other institutions/hospitals. This is extremely beneficial for hospitals that are located in different geographic areas, where patients face different problems. The gender ratios, age distributions, and ethnicities of the patients tend to differ based on the hospital. As a result of these differences, if an atypical patient was to be evaluated at a specific hospital, using a model that was trained solely on that hospitalâs data may be inaccurate, as the model hasnât learned from atypical cases. But these problems are eradicated by deploying a federated learning model. By doing so, healthcare institutions will be able to increase their external validity. Thus, the results generated will be more accurate, even for edge/atypical cases. Furthermore, this cooperation will allow for the advancement of precision medicine in the healthcare space.
On the other hand, the advancement of AI models requires periodic training and upgrading. This requirement may add unnecessary pressure to radiologists, who will need to constantly label data to retrain the model and make it more accurate. Breaking this problem down further, there are times when patient volume is at its peak and it becomes really difficult for radiologists to produce annotated data. However, this is where Federated Learning comes in; since peak volumes will differ from hospital to hospital, federated learning will allow radiologists at less busy hospitals to annotate data while others may be busier. In doing this, each hospital/institution can download and use the most up-to-date model year-round.
Lastly, incorporating the federated learning framework allows for hospitals to auto-scale without any additional costs. As the federated learning model gets more participants in the server (new hospitals/medical institutions), more data is added to the data pool. As the federated learning model continues its cycle, an ever-growing dataset is being fed into the model. This global model will continue to grow and increase in accuracy after users train their individual models, which requires minimal resources to aggregate models and, as a result, makes deployment more economical.
Applications
With efforts to further research and develop Federated Learning increasing, it is currently being explored as a viable application in the healthcare space. Many reports have shown a proof of concept model with federated learning for medical imaging.
Back in 2018, Intel collaborated with the Centre for Biomedical Image Computing and Analytics, at the University of Pennsylvania to further assess the use of federated learning for brain-image segmentation. In the assessment, they used a publicly available dataset from the brain tumour segmentation challenge (BraTS) from 2018. The data used was a collection of multi-model MRI brain scans from patients with gliomas. Gliomas is a very common type of tumour originating in the brain; approximately 33% of all brain tumours are gliomas. It originates from the glial cells that support the neurons in the brain. All the findings that are considered âabnormalâ, were annotated by radiologists âusing three distinct labels that corresponded to either peritumoral edematous/invaded tissue, non-enhancing/solid, necrotic/cystic tumour core, or enhancing tumourâ (PubMed). In this elegant experiment, the U-Net deep CNN model was used, while deploying the FL algorithm to perform model validation. In order to test the models' accuracy and efficacy, theoretical institutions were created to simulate the experiment. The data was split in 2 ways; the first split distributed the data randomly to each institution, but the second way distributed the data to where they were collected from. During the entire implementation process, each institution received the original model and it was collectively trained, allowing the central model to increase in performance. In doing so, during the next round, clients would receive a better model and the entire cycle would repeat. When comparing the results, they were extremely similar to traditional centralized machine learning models.
Another research experiment was conducted by NVIDIA when they worked with Kingâs College London and they displayed their results during the Medical Image Computing and Computer Assisted Intervention Society conference. The FL model was trained once again using the BraTS 2018 dataset and they attempted to protect patient privacy by applying the differential-privacy technique. This technique that was employed encodes the data before sharing; complex mathematical algorithms are used to prevent any potential data-breaches. As a result, NVIDIA was able to achieve similar, if not better results using a FL model without any data being shared.
Conclusion
Conclusively, it can be observed that training models on individual sights is extremely inefficient as they have a limited quantity of labeled datasets to build and train their own AI models. In turn, this lack of data often limits the external validity of models, hence why these models are not being implemented at a global scale currently. However, with federated learning, this major drawback can be eradicated completely. Federated learning utilizes a collaboration-based training approach which allows multiple institutions to pool their data together to train a single global model that has greater accuracy and efficacy. In doing so, no data is directly shared as federated learning is a decentralized approach to machine learning. The entire process offers a clean and smooth approach to deploying AI solutions in the healthcare space. Although this topic is still proof of concept, as research efforts increase, Federated Learning will definitely assist in advancing the healthcare field forward.
References
- Darzidehkalani, Erfan, et al. âFederated Learning in Medical Imaging: Part I: Toward Multicentral Health Care Ecosystems.â Journal of the American College of Radiology, Elsevier, 26 Apr. 2022, https://www.sciencedirect.com/science/article/pii/S1546144022002800.
- Flores, Mona. âMedical AI Needs Federated Learning, so Will Every Industry.â NVIDIA Blog, 15 Sept. 2021, https://blogs.nvidia.com/blog/2021/09/15/federated-learning-nature-medicine/.
- Ng, Dianwen, et al. âFederated Learning: A Collaborative Effort to Achieve Better Medical Imaging Models for Individual Sites That Have Small Labelled Datasets.â Quantitative Imaging in Medicine and Surgery, AME Publishing Company, Feb. 2021, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7779924/.
- Rieke, Nicola, et al. âThe Future of Digital Health with Federated Learning.â Nature News, Nature Publishing Group, 14 Sept. 2020, https://www.nature.com/articles/s41746-020-00323-1.