118 research outputs found
{HMRN}: heat map regression network to detect and track small objects in wide-area motion imagery
No full textWe propose HMRN, a deep heat map regression network to detect and track small moving objects in wide-area motion imagery (WAMI) by modifying a deep multi-object tracker. Object detection in WAMI images is challenging, because they cover large geographical areas and contain many small vehicles that do not have sufficient appearance-based cues for effective detection. Typically, background subtraction is applied to detect changed regions in WAMI image sequences. However, these methods suffer from high number of false detections. In this paper, we represent objects in WAMI images as heat maps and develop a deep regression network that predicts the object heat maps from current image, previous image and the predicted heat map of the previous image. Experiments are performed on Wright–Patterson Air Force Base (WPAFB) 2009 dataset and results show that the proposed method is almost ten times faster than its competitors while achieving state-of-the-art detection and tracking accuracy as well. We achieve significant reduction in false positives leading to an increase in average precision and F1 scores
Improved YOLOv4 for aerial object detection
No full textDrones equipped with cameras are being used for surveillance purposes. These surveillance systems need vision-based object detection of ground objects which look very small because of the altitude of drones. We propose an improved YOLOv4 model targeted for vision-based small object detection. We investigated the performance of state of the art YOLOv4 object detector on the VisDrone dataset. We enhanced the features of small objects by connecting Upsampling layers and concatenating the upsampled features with the original features to obtain more refined and grained features for small objects. Experiments showed that the modified YOLOv4 achieved 2 percent better mAP results as compared to the original YOLOv4 at different image resolutions on the VisDrone dataset while running at the same speed as the original YOLOv4
Improved homographic adaptation for keypoint generation in cross-spectral registration of thermal and optical imagery
No full textAutonomous navigation is an important area of research for aerial vehicles. Visual odometry and simultaneous localization and mapping algorithms are critical for the three-dimensional understanding of the environment. For that purpose, consistent multi-spectral maps of the environment should be generated. Existing pixel-based image registration methods are accurate but too slow to operate in real-time. Recently deep learning is used to develop feature-based data-driven methods for generating interest points and associated descriptors for registering multi-spectral image pairs. These methods are fast and perform better than existing methods for optical images. However, the results are less convincing for thermal image registration. In this work, we propose an improved multi-spectral homographic adaptation technique to generate highly repeatable ground truth interest points that are invariant across viewpoint changes in both spectra. These interest points are used to train the MultiPoint image registration network. Simulation results show that our improved model outperforms existing techniques for feature-based image alignment of optical and thermal images
X2V: 3D organ volume reconstruction from a planar X-Ray image with neural implicit methods
No full textIn this work, an innovative approach is proposed for three-dimensional (3D) organ volume reconstruction from a single planar X-ray, namely X2V network. Such capability holds pivotal clinical potential, especially in real-time image-guided radiotherapy, computer-aided surgery, and patient follow-up sessions. Traditional methods for 3D volume reconstruction from X-rays often require the utilization of statistical 3D organ templates, which are employed in 2D/3D registration. However, these methods may not accurately account for the variation in organ shapes across different subjects. Our X2V model overcomes this problem by leveraging neural implicit representation. A vision transformer model is integrated as an encoder network, specifically designed to direct and enhance attention to particular regions within the X-ray image. The reconstructed meshes exhibit a similar topology to the ground truth organ volume, demonstrating the ability of X2V in accurately capturing the 3D structure from a 2D image. The effectiveness of X2V is evaluated on lung X-rays using several metrics, including volumetric Intersection over Union (IoU). X2V outperforms the state-of-the-art method in the literature for lungs (DeepOrganNet) by about 7-9% achieving IoU's between 0.892-0.942 versus DeepOrganNet's IoU of 0.815-0.888.Publisher versio
Dual camera based high spatio-temporal resolution video generation for wide area surveillance
No full textWide area surveillance (WAS) requires high spatiotemporal resolution (HSTR) video for better precision. As an alternative to expensive WAS systems, low-cost hybrid imaging systems can be used. This paper presents the usage of multiple video feeds for the generation of HSTR video as an extension of reference based super resolution (RefSR). One feed captures video at high spatial resolution with low frame rate (HSLF) while the other captures low spatial resolution and high frame rate (LSHF) video simultaneously for the same scene. The main purpose is to create an HSTR video from the fusion of HSLF and LSHF videos. In this paper we propose an end-to-end trainable deep network that performs optical flow (OF) estimation and frame reconstruction by combining inputs from both video feeds. The proposed architecture provides significant improvement over existing video frame interpolation and RefSR techniques in terms of PSNR and SSIM metrics and can be deployed on drones with dual cameras
Hstr-net: Reference based video super-resolution with dual cameras
No full textHigh-spatio-temporal resolution (HSTR) video recording plays a crucial role in enhancing various imagery tasks that require fine-detailed information. State-of-the-art cameras provide this required high frame-rate and high spatial resolution together, albeit at a high cost. To alleviate this issue, this paper proposes a dual camera system for the generation of HSTR video using reference-based super-resolution (RefSR). One camera captures high spatial resolution low frame rate (HSLF) video while the other captures low spatial resolution high frame rate (LSHF) video simultaneously for the same scene. A novel deep learning architecture is proposed to fuse HSLF and LSHF video feeds and synthesize HSTR video frames. The proposed model combines optical flow estimation and (channel-wise and spatial) attention mechanisms to capture the fine motion and complex dependencies between frames of the two video feeds. Simulations show that the proposed model provides significant improvement over existing reference-based SR techniques in terms of PSNR and SSIM metrics. The method also exhibits sufficient frames per second (FPS) for aerial monitoring when deployed on a power-constrained drone equipped with dual cameras. The source code is publicly available at https://github.com/umutsuluhan/HSTRNet. © The Author(s) 2025.TÜBİTAKPublisher versio
Deep learning-based blind image super-resolution with iterative kernel reconstruction and noise estimation
No full textBlind single image super-resolution (SISR) is a challenging task in image processing due to the ill-posed nature of the inverse problem. Complex degradations present in real life images make it difficult to solve this problem using naive deep learning approaches, where models are often trained on synthetically generated image pairs. Most of the effort so far has been focused on solving the inverse problem under some constraints, such as for a limited space of blur kernels and/or assuming noise-free input images. Yet, there is a gap in the literature to provide a well-generalized deep learning-based solution that performs well on images with unknown and highly complex degradations. In this paper, we propose IKR-Net (Iterative Kernel Reconstruction Network) for blind SISR. In the proposed approach, kernel and noise estimation and high-resolution image reconstruction are carried out iteratively using dedicated deep models. The iterative refinement provides significant improvement in both the reconstructed image and the estimated blur kernel even for noisy inputs. IKR-Net provides a generalized solution that can handle any type of blur and level of noise in the input low-resolution image. IKR-Net achieves state-of-the-art results in blind SISR, especially for noisy images with motion blur
Finslerian Viewpoint to the Rectifying, Normal and Osculating Curves
No full textThe theory of Finsler metric was introduced by Paul Finsler, in 1918. The author defines this metric using the Minkowski norm instead of the inner product. Therefore, this geometry is a more general metric and includes the Riemannian metric. In the present work, using the Finsler metric, we investigate the position vector of the rectifying, normal and osculating curves in Finslerian 3-space F-3. We obtain the general characterizations of these curves in F-3. Furthermore, we show that rectifying curves are extremal curves derived from the Finslerian spherical curve. We also plotted various examples by using the Randers metrics
A dual-input deep learning architecture for classification and latency estimation in ABR signals
No full textIntroduction Auditory brainstem response (ABR) is an objective neurophysiological evaluation designed to measure the electrical activity originating from the auditory nerve and brainstem in response to auditory stimulation. This assessment objectively records synchronous neural activity as it propagates along the auditory pathway. It is characterized by several distinct waves, most notably waves I, III, and V. Wave V plays a central clinical role since its presence and latency are routinely used to assess a patient's hearing status. However, manual identification and localization of wave V are time consuming and subjective. Previous work has explored automated detection methods to reduce this burden.Methods In this paper, we make two primary contributions. First, we propose a multi-task deep learning pipeline that simultaneously (i) detects the presence of wave V and (ii) predicts its latency, thus eliminating the need for separate manual interpretation steps and enhancing clinical usability. Second, inspired by the audiologist's practice of comparing responses at multiple click sound intensities-specifically, using responses at high intensities, where waves are more prominent, as reference-we introduce a paired-signal approach. Each input to our deep learning model consists of the test signal together with its corresponding 80 dB reference from the same recording session. This provides the model with richer contextual information, and we show that the paired-signal approach improves over the single input approach. For multi-task learning, we design a network that consists of a backbone and two branches, one for latency prediction and the other for classification of whether wave V exists or not. We first train a latency-prediction network and then freeze its feature extraction layers to initialize a classification branch. Finally, we fine-tune the entire network using a joint loss function that balances classification and regression objectives.Results Experimental results demonstrate that our joint model1 outperforms conventional single-task approaches. For classification, it achieves an F1-score of 0.92; for latency regression, it attains an R2 of 0.90.Discussion Our findings highlight the promise of convolutional neural networks for enhancing ABR analysis and underscore their potential to streamline clinical workflows in the diagnosis of auditory disorders.TÜBİTAKPublisher versio
HM-net: A regression network for object center detection and tracking on wide area motion imagery
No full textWide Area Motion Imagery (WAMI) yields high resolution images with a large number of extremely small objects. Target objects have large spatial displacements throughout consecutive frames. This nature of WAMI images makes object tracking and detection challenging. In this paper, we present our deep neural network-based combined object detection and tracking model, namely, Heat Map Network (HM-Net). HM-Net is significantly faster than state-of-the-art frame differencing and background subtraction-based methods, without compromising detection and tracking performances. HM-Net follows object center-based joint detection and tracking paradigm. Simple heat map-based predictions support unlimited number of simultaneous detections. The proposed method uses two consecutive frames and the object detection heat map obtained from the previous frame as input, which helps HM-Net monitor spatio-temporal changes between frames and keep track of previously predicted objects. Although reuse of prior object detection heat map acts as a vital feedback-based memory element, it can lead to unintended surge of false positive detections. To increase robustness of the method against false positives and to eliminate low confidence detections, HM-Net employs novel feedback filters and advanced data augmentations. HM-Net outperforms state-of-the-art WAMI moving object detection and tracking methods on WPAFB dataset with its 96.2% F1 and 94.4% mAP detection scores, while achieving 61.8 % mAP tracking score on the same dataset. This performance corresponds to an improvement of 2.1% for F1, 6.1% for mAP scores on detection, and 9.5% for mAP score on tracking over state-of-the-art
- …
