A committee of experts from leading U.S. medical centers and research institutes is leveraging NVIDIA-powered federated learning to evaluate the impact of federated learning and AI-assisted annotation to train AI models for tumor segmentation.
Federated learning is a technique for developing more accurate and generalizable AI models trained on data from diverse data sources without mitigating data security or privacy. It allows multiple organizations to collaborate on developing an AI model without sensitive data leaving their servers.
“Due to privacy and data management limitations, it’s becoming increasingly difficult to share data from one site to another and aggregate it in one place, and AI for imaging is developing faster than research institutes can establish data-sharing contracts,” said John Garrett, associate professor of radiology at the University of Wisconsin-Madison. “Embracing federated learning to build and test models across multiple sites at once is the only way, practically speaking, to keep up. It’s an indispensable tool.”
Garrett is part of the Society for Imaging Informatics and Medicine (SIIM) Machine Learning Tools and Research Subcommittee, a group of physicians, researchers, and engineers that aims to promote the development and application of AI for medical imaging. NVIDIA is a member of SIIM and has been collaborating with the committee on federated learning projects since 2019.
“Federated learning techniques enable improved data privacy and security in compliance with privacy regulations such as GDPR, HIPAA, and others,” said committee chair Khaled Younis. “In addition, we see improved accuracy and generalizability of the models.”
To support their latest project, the team, which includes collaborators from Case Western, Georgetown University, the Mayo Clinic, the University of California at San Diego, the University of Florida and Vanderbilt University, turned to NVIDIA FLARE (NVFlare), an open source framework that includes robust security features, advanced privacy protection techniques, and a flexible system architecture.
Through the NVIDIA Academic Scholarship ProgramThe committee received four NVIDIA RTX A5000 GPUwhich were distributed among the participating research institutes to configure their workstations for federated learning. Other contributors used NVIDIA GPUs in the cloud and on local servers, highlighting the flexibility of NVFLare.
Cracking the code of federated learning
Each of the six participating medical centers provided data from about 50 medical imaging studies for the project, which focused on renal cell carcinoma, a type of kidney cancer.
“The idea of federated learning is that during training we exchange the model instead of exchanging the data,” said Yuankai Huo, assistant professor of computer science and director of the Biomedical Data Representation and Learning Laboratory at Vanderbilt University.
In a federated learning framework, an initial global model transmits model parameters to client servers. Each server uses those parameters to configure a local version of the model that is trained on the organization's proprietary data. Updated parameters from each of the local models are then sent back to the global model, where they are aggregated to produce a new global model. The cycle repeats until the model's predictions no longer improve with each round of training.
The group experimented with model architectures and hyperparameters to optimize training speed, accuracy, and the number of imaging studies required to train the model to the desired level of accuracy.
AI-assisted annotation with NVIDIA MONAI
In the first phase of the project, the training data used for the model was manually labeled. For the next phase, the team is using NVIDIA MONAI for AI-assisted annotation to evaluate how the model’s performance differs with AI-assisted segmented training data compared to traditional annotation methods.
“The biggest problem with federated learning activities is that the data across sites isn’t terribly uniform. People use different imaging equipment, have different protocols, and simply label their data differently,” Garrett said. “By training the federated learning model a second time with MONAI incorporated, we aim to find out if that improves the overall annotation accuracy.”
The team is using MONAI Tagan image labeling tool that allows users to develop custom AI annotation applications, reducing the time and effort required to create new datasets. Experts will validate and refine AI-generated segmentations before they are used for model training.
Data from both manual and AI-assisted annotation phases are hosted on Flywheel, a leading medical imaging and AI data platform that features NVIDIA MONAI integrated in their offers.
Once the project is complete, the team plans to publish their methodology, annotated datasets, and pre-trained model to support future work.
“We are interested in not only exploring these tools,” Garrett said, “but also in publishing our work so that others can learn and use these tools across the medical field.”
Apply for an NVIDIA Academic Scholarship
He NVIDIA Academic Scholarship Program Promotes academic research by providing world-class computing resources and access to researchers. Applications are now open for full-time faculty at accredited academic institutions who use NVIDIA technology to accelerate projects in Simulation and modeling, Generative AI and large language models.
Future application cycles will focus on data science, graph and vision, and edge AI projects, including federated learning.
