123 research outputs found
857628_supp_mat_table – Supplemental material for Robotic technology for palliative and supportive care: Strengths, weaknesses, opportunities and threats
No full textSupplemental material, 857628_supp_mat_table for Robotic technology for palliative and supportive care: Strengths, weaknesses, opportunities and threats by Amara Callistus Nwosu, Bethany Sturgeon, Tamsin McGlinchey, Christian DG Goodwin, Ardhendu Behera, Stephen Mason, Sarah Stanley and Terry R Payne in Palliative Medicine</p
857628_supp_mat – Supplemental material for Robotic technology for palliative and supportive care: Strengths, weaknesses, opportunities and threats
No full textSupplemental material, 857628_supp_mat for Robotic technology for palliative and supportive care: Strengths, weaknesses, opportunities and threats by Amara Callistus Nwosu, Bethany Sturgeon, Tamsin McGlinchey, Christian DG Goodwin, Ardhendu Behera, Stephen Mason, Sarah Stanley and Terry R Payne in Palliative Medicine</p
A visual signature-based identification method of low-resolution document images and its exploitation to automate indexing of multimodal recordings
No full textCette thèse porte sur le développement d’un système complet pour l’indexation automatique, centrée sur le document (DocMIR), de données multimédias issues d’environnements multimodaux tels que réunions, conférences, etc. Tant des méthodes de traitement d’images que de segmentation vidéo et d’analyse de document sont utilisées pour mettre en relation les données temporelles de réunions avec les documents. Le système proposé s’articule autour de deux tâches principales : une segmentation vidéo basée sur le document et l’identification d’images de documents à basse résolution. Plusieurs heures de données audio-visuelles doivent être fragmentées en segments de taille raisonnable afin de faciliter une navigation ultérieure. Durant une présentation, les documents projetés sont souvent capturés en tant que flux vidéo et peuvent être utilisés comme des pointeurs sémantiques pertinents, du fait qu’ils apparaissent à un instant spécifique, durent un temps déterminé et résument le discours courant de l’orateur. Les approches existantes ne sont pas applicables dans le cas où les vidéos proviennent d’appareils à basse résolution tels que des webcams. Pour remédier à ces inconvénients, la technique de segmentation proposée considère la stabilité plutôt que le changement dans les séquences vidéo et ne nécessite en outre aucune identification du document pour confirmer le changement. Par ailleurs, une identification des documents à basse résolution est requise pour lier les documents électroniques originaux aux données multimédias segmentées. La méthode proposée utilise une signature visuelle du document composée des signatures de couleur et de mise en page. Les caractéristiques visuelles telles que les couleurs, leur distribution spatiale et la mise en page sont extraites puis structurées hiérarchiquement dans la signature. Cette approche permet une mise en correspondance rapide et efficace, afin de répondre aux besoins d’applications réelles. Elle résout par ailleurs les problèmes de faible résolution de l’image, des arrière-plans bruités et texturés ainsi que des conditions de luminosité variables de l’environnement de capture. Les méthodes de comparaison de la signature appliquent une fusion multi-niveaux séquentielle, linéaire ou non-linéaire des diverses caractéristiques visuelles. Cette nouvelle méthode d’identification a été comparée aux approches classiques au moyen de données réelles enregistrées lors de réunions et conférences, et s’est montrée significativement plus performante. Les performances des différentes techniques développées dans cette thèse prouvent l’utilité des documents en tant que modalité additionnelle et interfaces naturelles pour interagir avec des données multimédias capturées dans des environnements multimodaux.This thesis investigates methods for building an efficient application system for the document-based automatic indexing and retrieval (DocMIR) of multimedia data captured from multimodal environments such as meetings, conferences, etc. Both empirical image processing, video segmentation methods and document analysis approaches are studied to bridge the gap between temporal data and static information. The proposed system focuses on two major tasks: document-based video segmentation and low-resolution document image identification. The captured audio-visual data of several hours should be fragmented into reasonable distinct smaller segments in order to provide useful access points. During a presentation, projected documents are often captured as a video stream and can be used as meaningful semantic pointers because they appear at specific time, remain in visual focus for a definite duration and summarize presenter’s discourse at that time. The existing approaches for video segmentation are not applicable in this scenario since videos are captured from low-resolution devices, such as web-cams. In order to overcome these drawbacks, the proposed feature-based segmentation technique considers the stability rather than changes in video sequences. The technique does not require any document identification methods to confirm the change. The identification of low-resolution documents is also required to link original electronic documents with the temporally segmented captured multimedia data. The proposed identification method uses a Visual Signature consisting of Layout Signature and Color Signature. This signature-based approach is considered for fast and efficient matching in order to fulfill the needs of real-time applications. It also overcomes the problem of poor resolution, noisy, complex backgrounds and varying lighting conditions of the capture environment. The visual features such as colors, their spatial distribution and layout features are extracted and structured hierarchically to form the Color Signature and Layout Signature, respectively. The matching of signature is based on both, sequential as well as multi-level linear and non-linear fusion of various visual features. The performance of the proposed technique has been compared with existing approaches using real data recorded from meetings and conferences and found to be significantly better. The high-quality performances of the above-mentioned techniques prove the usefulness of documents as an additional modality and natural interface, to interact with multimedia data captured from multimodal environments
Deep learning models to automate the scoring of hand radiographs for rheumatoid arthritis
No full textThe van der Heijde modification of the Sharp (SvdH) score is a widely used radiographic scoring method to quantify damage in Rheumatoid Arthritis (RA) in clinical trials. However, its complexity with a necessity to score each individual joint, and the expertise required limit its application in clinical practice, especially in disease progression measurement. In this work, we addressed this limitation by developing a bespoke, automated pipeline that is capable of predicting the SvdH score and RA severity from hand radiographs without the need to localise the joints first. Using hand radiographs from RA and suspected RA patients, we first investigated the performance of the state-of-the-art architectures in predicting the total SvdH score for hands and wrists and its corresponding severity class. Secondly, we leveraged publicly available data sets to perform transfer learning with different finetuning schemes and ensemble learning, which resulted in substantial improvement in model performance being on par with an experienced human reader. The best model for RA scoring achieved a Pearson’s correlation coefficient (PCC) of 0.925 and root mean squared error (RMSE) of 18.02, while the best model for RA severity classification achieved an accuracy of 0.358 and PCC of 0.859. Our score prediction model attained almost comparable accuracy with experienced radiologists (PCC = 0.97, RMSE = 18.75). Finally, using Grad-CAM, we showed that our models could focus on the anatomical structures in hands and wrists which clinicians deemed as relevant to RA progression in the majority of cases
Generating Chest Radiology Report Findings Using a Multimodal Method
No full textAutomatic report generation from chest x-ray imaging (CXR) could potentially alleviate the workload of radiologists and improve clinical efficacy. We introduce a multimodal approach, that integrates radiology images with text describing patients’ indications, to generate the findings section in radiology reports. We instantiate this approach by building on two existing methods, R2Gen and CvT2DistilGPT2. We report experiments on two public datasets, MIMIC-CXR and IU X-ray, using evaluation metrics for natural language generation and clinical efficacy assessment. Results show that improvements across all metrics are obtained through the incorporation of indications text. For example, we obtain 35% and 8% increases in BLEU-4 and F1 scores, respectively.</p
Multimodal 3D Brain Tumor Segmentation with Adversarial Training and Conditional Random Field
No full textAccurate brain tumor segmentation remains a challenging task due to structural complexity and great individual differences of gliomas. Leveraging the pre-eminent detail resilience of CRF and spatial feature extraction capacity of V-net, we propose a multimodal 3D Volume Generative Adversarial Network (3D-vGAN) for precise segmentation. The model utilizes Pseudo-3D for V-net improvement, adds conditional random field after generator and use original image as supplemental guidance. Results, using the BraTS-2018 dataset, show that 3D-vGAN outperforms classical segmentation models, including U-net, Gan, FCN and 3D V-net, reaching specificity over 99.8%
Multimodal deformable image registration for long-COVID analysis based on progressive alignment and multi-perspective loss
No full textLong COVID is characterized by persistent symptoms, particularly pulmonary impairment, which necessitates advanced imaging for accurate diagnosis. Hyperpolarised Xenon-129 MRI (XeMRI) offers a promising avenue by visualising lung ventilation, perfusion, as well as gas transfer. Integrating functional data from XeMRI with structural data from Computed Tomography (CT) is crucial for comprehensive analysis and effective treatment strategies in long COVID, requiring precise data alignment from those complementary imaging modalities. To this end, CT-MRI registration is an essential intermediate step, given the significant challenges posed by the direct alignment of CT and Xe-MRI. Therefore, we proposed an end-to-end multimodal deformable image registration method that achieves superior performance for aligning long-COVID lung CT and proton density MRI (pMRI) data. Moreover, our method incorporates a novel Multi-perspective Loss (MPL) function, enhancing state-of-the-art deep learning methods for monomodal registration by making them adaptable for multimodal tasks. The registration results achieve a Dice coefficient score of 0.913, indicating a substantial improvement over the state-of-the-art multimodal image registration techniques. Since the XeMRI and pMRI images are acquired in the same sessions and can be roughly aligned, our results facilitate subsequent registration between XeMRI and CT, thereby potentially enhancing clinical decision-making for long COVID management
Latent Body-Pose guided DenseNet for Recognizing Driver’s Fine-grained Secondary Activities
No full textLatent Body-Pose guided DenseNet for Recognizing Driver’s Fine-grained Secondary Activities
Get PDFOver the past two decades, there has been an increasing research in developing self-driving vehicles, with many industries pushing the bounds alongside academia. Automatic recognition of in-vehicle activities plays a key role in developing such vehicles. In this work, we propose a novel human-pose driven approach for video-based monitoring of driver’s state/activity and is inspired by the recent success of deep Convolutional Neural Network (CNN) in visual recognition tasks. The approach infers the driver’s state/activity from a single frame and thus, could operate in real-time. We also bring together ideas from recent works on human pose detection and transfer learning for visual recognition. The adapted DenseNet integrates these ideas under one framework, where one stream is focused on the latent body pose and the other stream is on appearance information. The proposed method is extensively evaluated on two challenging datasets consisting various secondary nondriving activities. Our experimental results demonstrate that the driver activity recognition performance improves significantly when the latent body-pose is integrated into the existing deep networks
Latent Body-Pose guided DenseNet for Recognizing Driver’s Fine-grained Secondary Activities
Get PDFOver the past two decades, there has been an increasing research in developing self-driving vehicles, with many industries pushing the bounds alongside academia. Automatic recognition of in-vehicle activities plays a key role in developing such vehicles. In this work, we propose a novel human-pose driven approach for video-based monitoring of driver’s state/activity and is inspired by the recent success of deep Convolutional Neural Network (CNN) in visual recognition tasks. The approach infers the driver’s state/activity from a single frame and thus, could operate in real-time. We also bring together ideas from recent works on human pose detection and transfer learning for visual recognition. The adapted DenseNet integrates these ideas under one framework, where one stream is focused on the latent body pose and the other stream is on appearance information. The proposed method is extensively evaluated on two challenging datasets consisting various secondary nondriving activities. Our experimental results demonstrate that the driver activity recognition performance improves significantly when the latent body-pose is integrated into the existing deep networks
- …
