1,720,987 research outputs found
Deep learning approach to discontinuity-preserving image registration
Get PDFImage registration is an indispensable tool in medical image analysis. Traditionally, registration algorithms are aimed at aligning image pairs using regularizers to impose smoothness restrictions on unknown deformation fields. The majority of these methods assume global smoothness in the image domain, which pose issues for scenarios where motion discontinuities exist. Examples where local motion discontinuities happen are the sliding motion between adjacent organ tissues, and the pushing motion of the lungs against the chest wall during the respiratory cycle. Furthermore, an objective function must be optimized for each given pair of images. Thus registration of multiple image sets becomes very time-consuming and poorly scale with higher resolution image volumes. Using recent developments in deep learning, we propose an unsupervised learning-based image registration model. This model is trained over a loss function with a custom regularizer that preserves local discontinuities while simultaneously respecting the smoothness assumption in homogeneous regions of image volumes. In following a learning-based model, the image registration process can be completed within seconds, which is significantly quicker than optimization-based registration algorithms. The proposed model will be evaluated qualitatively and quantitatively on datasets of chest computed tomography (CT) 3D image volumes.University of Ontario Institute of Technolog
Computational methods for medical image registration
Get PDFA significant amount of research has been dedicated to the improvement of techniques in the field of medical image analysis. Imaging modalities have been improved and new acquisition methods have been introduced to reveal greater anatomical detail and to allow for more information to be extracted from medical images. However, certain challenges remain when processing and analyzing information from medical images. Image registration is a technique to find a reasonable transformation that best aligns a pair or group of images. Of particular interest in this thesis is the use of image registration in three main categories: cardiac fiber atlas construction from healthy porcine hearts, motion correction in contrast-enhanced image sequences, and the development of novel computational techniques that improve the performance of existing medical image registration methods. We provide an overview of each of the problems that we tackled, followed by a discussion of the underlying motivation and theory behind the proposed methods, and extensive validations. Most importantly, this work highlights the central role that image registration plays in biomedical research - from producing clinically relevant image-based predictive models, to enabling accurate diagnosis of diseases and the analysis of treatment response.University of Ontario Institute of Technolog
Generative models for multi-modality image inpainting and resolution enhancement
No full textRecently, deep learning methods specifically generative adversarial networks (GANs) have been used to rapidly improve a wide range of image enhancement tasks including image inpainting and image resolution enhancement also known as super-resolution. Image-to-image translation methods convert an image provided in a source modality (e.g., a nighttime image) to an image of a target modality (e.g., a daytime image) by learning an image generation function. These methods can be applied to a wide variety of problems in image processing and computer vision. The use of GANs for image-to-image translation has also been extensively studied. We propose the problem of combining the image-enhancement tasks (e.g., image inpainting or super-resolution) with the image-to-image translation task in a joint formulation. Given a distorted nighttime image of a scene can one recover a restored daytime image of the same scene? Two models to address the joint problem will be presented. Our models are validated on night-to-day joint image translation and enhancement for both super-resolution and inpainting. Promising qualitative and quantitative results will be reported.University of Ontario Institute of Technolog
An optimization-based approach to image fusion using structural similarity
Get PDFThis thesis examines a framework for solving optimization problems involving structural similarity index measure Mean. Image fusion involves combining multiple images to preserve their most desirable characteristics. The objective is to create a single enhanced image that can be utilized for various applications, including human visual perception, object detection, and target recognition. A general framework is introduced for the formulation of image fusion, which incorporates the SSIM, one of the most effective measures of visual proximity that is consistent with the human visual system. The concept of image fusion is revisited, its importance is emphasized, and an overview of methods commonly used for image fusion is provided. An alternative expression of SSIM using Mean values and vector norms is proposed, along with the calculation of its derivative. Next, the SSIM Mean is used to formulate image fusion as data fidelity of an optimization problem. Gradient-based methods are applied to solve it. A comparison is conducted between the proposed method and the method presented by Brunet in his Ph.D. thesis at UWaterloo. The experiment use test images in MATLAB. The evaluation metrics MSE, PSNR, and SSIM show expected results in assessing the experimental outcomes. The significant advantage of the proposed formulation is its flexibility in accommodating different degradation operators and regularization terms.University of Ontario Institute of Technolog
Hybrid ConVIRT - enhancing medical image-text representation learning of vision language models
Get PDFChallenges in medical imaging are being addressed through advancements in image-text representation learning, as demonstrated by Hybrid-ConVIRT, which builds on contrastive learning frameworks such as ConVIRT and MedCLIP. These medical contrastive learning models trained on domain-specific datasets, have tackled issues related to the costly, expert-annotated datasets typically required in traditional medical imaging. These models serve as foundational, general purpose models that, once trained on vast amounts of data, can be adapted for various downstream tasks. Hybrid-ConVIRT further improves upon this by incorporating image transformations during training and employing CXR-BERT-specialized as the text encoder, specifically tailored for chest Xray reports. To optimize feature alignment, it introduces a Fused Similarity Matrix (FSM), which serves as a more refined target, leveraging the learned capabilities of Med-CLIP and ConVIRT. Evaluation across four datasets (COVID, Tuberculosis, Pneumonia/TB Mix, and CheXpert) using linear probe and zero-shot classification demonstrates Hybrid-ConVIRT’s enhanced discriminative power and feature robustness, particularly on COVID and CheXpert datasets. Although MedCLIP shows a slight advantage on the Tuberculosis dataset, this is likely due to its larger training set. The Hybrid model’s performance supports the potential of combining the learned representations from both preceding models
Single magnetic resonance image super-resolution using generative adversarial network
Get PDFSuper-Resolution is the process of converting given low-resolution images into corresponding high-resolution ones. The resolution enhancement process applied to medical images can potentially improve diagnostic accuracy for a variety of conditions and reduce imaging scan time. Recent improvements in computational tools have made deep learning methods popular for various image processing techniques including resolution enhancement. In this thesis, we consider the image super-resolution problem of the brain and cardiac MRI datasets. To increase the spatial resolution of these medical images, we have adapted a Generative Adversarial Network (GAN) model where the generator has a DenseNet type structure and the discriminator is based on the U-Net model. We have used a combination of loss functions to ensure the generated images are consistent with ground truth. To train and validate the model, we have used four different datasets consisting of brain and cardiac MRI. Promising qualitative and quantitative results are provided.University of Ontario Institute of Technolog
Comparative analysis of deep learning and graph cut algorithms for cell image segmentation
Get PDFImage segmentation is a commonly used technique in digital image processing with many applications in the area of computer vision and medical image analysis. The goal of image segmentation is to partition an image into multiple regions, normally based on the characteristics of pixels in a given image. Image segmentation could involve separating the foreground from background in an image, or clustering image regions based on similarities in intensity, color, or shape.
In this thesis, we consider the problem of cell image segmentation and evaluate the performance of two major techniques on a dataset of cell image sequences. First, we apply a traditional segmentation algorithm based on the so-called graph cut that addresses the segmentation problem using an energy minimization scheme defined on a weighted graph. Second, we use modern techniques based on deep neural networks, namely U-Net and LSTM that have a time-consuming training and a relatively quick testing phase.
Performance of each technique will be analyzed qualitatively and quantitatively based on various standard measures and will be compared statistically.University of Ontario Institute of Technolog
Multi-scale local explanation approach for image analysis using model-agnostic explainable artificial intelligence (XAI)
Get PDFThe recent success of deep neural networks has generated a remarkable growth in Artificial Intelligence (AI) research, and it received much interest over the past few years. However, one of the main challenges for the broad adoption of deep learning based models such as Convolutional Neural Networks (CNN) is the lack of understanding of their decisions. Local Interpretable Model-agnostic Explanations (LIME) is an explanation method which produces a coarse heatmap as a visual explanation highlighting the most important superpixels affecting the CNN’s decision. This thesis aims to explore and develop a multi-scale scheme of LIME to explain decisions made by CNN models through heatmaps of coarse to finer scales. More precisely, when LIME highlights large superpixels from the coarse scale, there are some tiny regions in the corresponding superpixel that influenced the model’s prediction at the finer scale. Therefore, we propose a multi-scale scheme of LIME and two weighting approaches based on Gaussian distribution and a parameter-free framework to produce visual explanations observed from different scales. We investigated the proposed multi-scale scheme on Flower Dataset from TensorFlow and a biological dataset, Camelyon 16. The results prove that the explanations are faithful to the underlying model, and our visualizations are reasonably interpretable.University of Ontario Institute of Technolog
Structure guided image restoration : a deep learning approach
Get PDFImage restoration aims at recovery of degraded images and estimating the original. Over the past few years, computer vision research has been dominated by deep learning techniques in part due to advances in computing infrastructure, algorithms and image capturing devices. As a result, deep neural networks currently set the state-of-the-art in image restoration problems. However, many of these techniques fail to reconstruct reasonable structures as they are commonly over-smoothed and/or blurry.
In this dissertation, we develop models based on deep convolutional neural networks to address two image restoration problems: image inpainting and image super-resolution. We develop a new approach for image inpainting that does a better job of reproducing missing regions exhibiting fine details. Furthermore, we extend this method to image superresolution by reformulating the problem as an in-between pixels inpainting task. We propose a two-stage adversarial model that comprises of an edge generator followed by an image completion network. The edge generator hallucinates edges of the missing region of the image, and the image completion network fills in the missing regions using hallucinated edges as a priori. We evaluate our model over the publicly available datasets and show that it outperforms current state-of-the-art techniques quantitatively and qualitatively.University of Ontario Institute of Technolog
Mathematical methods for 2D-3D cardiac image registration
Get PDFWe propose a mathematical formulation aimed at intensity-based slice-to-volume registration,
aligning a cross-sectional slice of a 3D volume to a 2D image. The approach is
flexible and can accommodate various regularization schemes, similarity measures, and
optimizers. We evaluate the framework by registering 2D and 3D cardiac magnetic resonance
(MR) images obtained in vivo, aimed at image-guided surgery applications that
utilise real-time MR imaging as a visualization tool. Rigid-body and affine transformations
are used to validate the parametric model. Target registration error (TRE),
Jaccard, and Dice indices are used to evaluate the algorithm and demonstrate the accuracy
of the registration scheme on both simulated and clinical data. Registration with the
affine model appeared to be more robust than the rigid model in controlled registration
experiments. By simply extending the rigid model to an affine model, alignment of the
cardiac region generally improved, without the need for complex dissimilarity measures
or regularizers.University of Ontario Institute of Technolog
- …
