A medical image such as an MRI or X-ray can seem like a confusing array of black-and-white shapes to those unfamiliar with radiology. Identifying specific structures—like tumors—within these images can be a daunting challenge. However, AI systems trained to recognize biological boundaries can assist in segmenting regions of interest that healthcare professionals monitor for diseases and other anomalies. By automating the outline process, AI can save valuable time, allowing medical personnel to focus on more complex tasks rather than manually tracing anatomy across multiple images.
The significant hurdle, though, is that AI systems require extensive labeled datasets to train properly. For example, to enable an AI model to recognize the cerebral cortex, thousands of MRI scans need to be labeled, indicating the varying shapes the cortex may take in different brains. To address this challenge, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts General Hospital (MGH), and Harvard Medical School have introduced the interactive “ScribblePrompt” framework. This innovative tool can rapidly segment any medical image, including unfamiliar types, thus sidestepping the tedious data labeling that typically necessitates extensive manual effort.
To create this advanced tool, the research team simulated user annotations across over 50,000 scans, from MRIs and ultrasounds to photographs, encompassing biological structures like eyes, cells, brains, bones, and skin. Instead of relying solely on human inputs, the researchers utilized algorithms to mimic how users would scribble or click to highlight specific regions in medical images. This method included the use of superpixel algorithms to identify areas within images that share similar values. By leveraging synthetic data, ScribblePrompt effectively prepares itself for real-world segmentation tasks.
“AI has tremendous promise in enhancing image analysis and other complex data tasks, enabling humans to work more efficiently,” remarks Hallee Wong, an MIT Ph.D. student and lead author of the recent paper detailing ScribblePrompt. Wong emphasizes that the aim is to augment the capabilities of medical professionals, not to replace them. With its interactive system, ScribblePrompt helps doctors concentrate on more critical aspects of their analyses. The system demonstrates impressive efficiency compared to existing interactive segmentation techniques, reducing annotation time by 28 percent against Meta’s Segment Anything Model (SAM).
The ScribblePrompt interface is intuitive, allowing users to highlight the desired segment by either scribbling over it or clicking on it. For example, researchers can click on individual veins in a retinal scan, and ScribblePrompt will automatically detect and outline the entire structure or background as directed. If the tool misses any sections, users can correct it via additional markings or exclude areas using a “negative scribble” feature, enhancing the model’s accuracy.
A user study at MGH showed ScribblePrompt’s effectiveness, where 93.8 percent of neurology researchers preferred its performance over SAM in improving segments after user corrections, while 87.5 percent favored ScribblePrompt for click-based edits. The tool was trained on simulated interactions encompassing 54,000 images from 65 datasets, which included various types of medical scans, such as CT scans, X-rays, MRIs, ultrasounds, and more.
“Traditional methods often struggle with user interactions like scribbling, making it difficult to replicate such dynamics during training. Through our synthetic segmentation tasks, ScribblePrompt effectively learns to respond to diverse inputs,” Wong explains. The team focused on developing a generalized foundation model capable of adapting to new images and tasks based on extensive training data.
Following the training phase, researchers assessed ScribblePrompt across 12 new datasets it had not encountered before. Despite the unfamiliar data, the tool outperformed four existing segmentation methods, achieving higher efficiency and accuracy in addressing user-specified regions of interest.
“Segmentation is a pivotal task in biomedical image analysis, integral to both clinical practice and research,” states Adrian Dalca, a CSAIL research scientist and assistant professor at MGH and Harvard Medical School. He underscores ScribblePrompt’s design as being extremely relevant for clinicians and researchers, significantly expediting the segmentation process.
Bruce Fischl, a professor of radiology at Harvard Medical School and a neuroscientist at MGH, highlights the limitations of existing segmentation algorithms, particularly in the medical imaging realm where images often involve complex 3D volumes. He notes, “ScribblePrompt revolutionizes manual annotation by enabling faster and more precise interactions, resulting in a user-friendly interface that enhances productivity for annotators.”
Wong collaborated with three other CSAIL affiliates—John Guttag, a professor at MIT and principal investigator at CSAIL, and Ph.D. student Marianne Rakic—on this innovative project. Their research received support from several organizations, including Quanta Computer Inc., the Eric and Wendy Schmidt Center at the Broad Institute, Wistron Corp., and the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health.
The research team plans to present their findings at the upcoming 2024 European Conference on Computer Vision and recently showcased their work at the DCAMI workshop during the Computer Vision and Pattern Recognition Conference, where they received the Bench-to-Bedside Paper Award for ScribblePrompt based on its potential impact in clinical settings. Through this groundbreaking work, they hope to pave the way for more efficient and effective medical image analysis, ultimately benefiting healthcare professionals and patients alike.
Source link
