6 research outputs found
Transformation of intracranial direct electrical stimulation sites to postoperative MRI in awake glioma surgery
No full textEvaluating the Interpretability of Prototype Networks for Medical Image Analysis
No full textArtificial intelligence (AI) holds great potential for assisting physicians in medical image analysis, yet current models provide predictions without transparent explanations of their decision-making process. Explainable AI (XAI) aims to address this issue by providing tools to enhance model interpretability. However, existing XAI methods, such as post-hoc explanation techniques, often produce coarse heatmaps that lack the precision needed for medical applications. In contrast, self-explaining models like prototype networks offer a promising approach by learning characteristic image patterns, called prototypes. These prototypes could aid physicians in their diagnostic process. In this work, we evaluate the interpretability of prototype networks in the context of medical image classification using a mammography and a colorectal polyp (CP) dataset. We assess the quality and relevance of the learned prototypes by comparing them to domain-specific knowledge and physician-annotated bounding boxes. Our key contribution involves linking these prototypes to textual descriptions of CPs. Our experiments demonstrate that learned prototypes visually correspond to clinically relevant regions of interest (ROIs), obtaining a mean intersection over union (IoU) of 0.25 ± 0.29. Challenges remain in improving spatial accuracy and linking prototypes to textual descriptions. We discuss these limitations and propose future directions for integrating prototype networks into clinical workflows, contributing to the development of transparent and trustworthy XAI systems for clinical decision support.</p
Diffusion-based lung nodule synthesis for advanced evaluation of deep learning models
No full textThe accurate detection of lung nodules in computed tomography (CT) scans is critical for the early diagnosis and treatment of lung cancer. Despite the advances of computer aided detection algorithms for nodule detection, biases in training data often compromise the generalization of deep learning models trained for this task. To address this, we present a novel methodology for synthesizing realistic lung nodules, using a fine-tuned version of the Instruct Pix2Pix framework, based on Stable Diffusion. The proposed approach enables precise control over the size and placement of the nodule within CT slices, facilitating the creation of synthetic datasets tailored for systematic bias evaluation and model validation. By generating paired CT slices with and without nodules, we provide a supervised baseline for training the diffusion model. This work demonstrates the realism of the created nodules through a clinician evaluation study, where observers often misclassified synthetic nodules as real. In fact, clinicians have mistaken synthesized nodules for real nodules in 41% of cases, a rate close to the probability level of 50%, underscoring the high fidelity of the generated images. This pioneering work shows the applicability of diffusion models for evaluating deep learning models and opens doors for future research to extend this methodology to 3D and high-quality scans. The resulting dataset and code will be made publicly available to promote further research in this domain.</p
Evaluating task-specific augmentations in self-supervised pre-training for 3D medical image analysis
Get PDFSelf-supervised learning (SSL) has become a crucial approach for pre-training deep learning models in natural and medical image analysis. However, applying transformations designed for natural images to three-dimensional (3D) medical data poses challenges. This study explores the efficacy of specific augmentations in the context of self-supervised pre-training for volumetric medical images. A 3D non-contrastive framework is proposed for in-domain self-supervised pre-training on 3D gray-scale thorax CT data, incorporating four spatial and two intensity augmentations commonly used in 3D medical image analysis. The pre-trained models, adapted versions of ResNet-50 and Vision Transformer (ViT)-S, are evaluated on lung nodule classification and lung tumor segmentation tasks. The results indicate a significant impact of SSL, with a remarkable increase in AUC and DSC as compared to training from scratch. For classification, random scalings and random rotations play a fundamental role in achieving higher downstream performance, while intensity augmentations show limited contribution and may even degrade performance. For segmentation, random intensity histogram shifting enhances robustness, while other augmentations have marginal or negative impacts. These findings underscore the necessity of tailored data augmentations within SSL for medical imaging, emphasizing the importance of task-specific transformations for optimal model performance in complex 3D medical datasets
Identifying Key Challenges in Ovarian Tumor Classification:A Comparative Study Using Deep Learning and Radiomics
No full textOvarian tumor malignancy classification is a challenging task for radiologists using current algorithms, as they do not reach the required accuracy for clinical decision-making. The heterogeneous nature of ovarian tumors makes differentiation more difficult compared to other classification tasks, such as e.g. lung-nodule classification, even for experienced radiologists. The integration of computed tomography (CT) data with deep learning holds potential to improve accuracy, but its application remains in its infancy, where most previous work relies on radiomic feature extraction. This study compares a radiomics-based model and two deep learning models across both ovarian and lung datasets. The results demonstrate that although these models are promising, there is considerable variation in performance across the classification tasks, which reflects their level of difficulty. The performed feature analysis shows that ovarian tumors exhibit high feature heterogeneity and lack feature robustness, underscoring the need for novel methods to enhance feature differentiation. However, it is also shown that it is possible to gain performance for a majority of cases, by identifying features that can filter out outliers. Furthermore, the obtained results suggest that tumor volume plays a clear role in ovarian tumor classification, where large malignant tumors are most difficult to classify. These findings highlight that there is no one-size-fits-all solution to deep learning for tumor classification in CT, as model performance depends heavily on the task. Tailoring models to address dataset-specific challenges is critical to advancing their clinical utility. The code for our work is published on GitHub at: https://github.com/EloySchultz/OvaCADx_SPIE2025.</p
Multi-center Ovarian Tumor Classification Using Hierarchical Transformer-Based Multiple-Instance Learning
Get PDFMalignant ovarian tumors (OTs) are a leading cause of gynecological cancer deaths, and often remain asymptomatic until advanced stages, making early and accurate diagnosis crucial for effective treatment and good patient outcome. Current diagnostic methods often fall short due to the heterogeneous nature of OTs and the complexities in distinguishing benign from malignant forms. To overcome these limitations, this study proposes a novel framework leveraging transformer-based multiple-instance learning (MIL) and hierarchical self-supervised pre-training. To validate the model, a comprehensive multi-center dataset has been compiled, encompassing diverse patient demographics and imaging protocols. Benchmarking against conventional radiomics methods and other deep learning approaches, the hierarchical MIL model demonstrates superior performance with a median AUROC of 0.84 and high recall of 0.91. These results highlight significant improvements in sensitivity, essential for minimizing false negatives in clinical settings. The performed study emphasizes the importance of multi-center validation and external dataset testing to ensure generalization of the proposed model and obtain a higher robustness. The encountered complexity of multi-center data is found significant, since various clinical factors play an influential role. This makes baseline comparisons virtually impossible and the need for more multi-center research increasingly compelling and encouraging
