40 research outputs found
Potential evaluation of visible-thermal UAV image fusion for individual tree detection based on convolutional neural network
Get PDFUnmanned aerial vehicles (UAVs) outfitted with thermal and visible sensors are already a popular platform in precision agriculture thanks to recent advances in remote sensing. Many researchers have studied integrating data from sensors with different spectral characteristics to achieve higher-level properties and, consequently, detect the trees accurately. In this research, visible and thermal images, as well as normalized digital surface models resulting from UAVs with high spatial resolution, are employed to accurately extract trees from two studied urban areas with complex backgrounds. In the thermal image, trees can be detected in hidden areas based on their brightness temperature difference compared to other features. In contrast, the visible image has a higher spatial resolution, and fusing this data with thermal images can resolve the complexity of the problem. In the proposed method, first, a deep learning network based on visible-thermal data is evaluated in terms of detecting trees with various data approaches. These evaluations include comparison tests on four types of data input to the convolutional network of the visible images, thermal images, fusing visible-thermal images, and also fusing visible-thermal- normalized digital surface model images. Results of evaluation parameters indicate maximum precision in the fourth approach (intersection-over-union = 91.72, F-score = 95.67). Then, the output binary map with the highest accuracy approach and Canny edge detection operator is utilized to accurately identify tree boundaries, count, and estimate the area and diameter of the tree canopy. Finally, the findings revealed the root mean square error (RMSE) first and second areas are 0.21 m2, 0.08 m and 0.24 m2, 0.11 m respectively for the area and diameter of the tree crown
Precise 3D extraction of building roofs by fusion of UAV-based thermal and visible images
Get PDFThermography is an efficient way of detecting the thermal problems of the roof as a major part of a building’s energy dissipation. Thermal images have a low spatial resolution, making it a challenge to produce a three-dimensional thermal model using aerial images. This paper proposes a combination of thermal and visible point clouds to generate a higher-resolution thermal point cloud from roofs of buildings. For this purpose, after obtaining the internal orientation parameters through camera calibration, visible and thermal point clouds were generated and then registered to each other using ground control points. The roofs of buildings were then extracted from the visible point cloud in four steps. First ground points were removed using cloth simulation filter (CSF), and then vegetation points were eliminated by applying entropy and red-green-blue vegetation index (RGBVI). Geometric features and a segmentation step were considered to filter roofs from other parts. Finally, by combining visible and thermal point clouds, the generated point had a high spatial resolution along with thermal information. In the achieved results, the thermal camera calibration was performed with an accuracy of 0.31 pixels, and the thermal and visible point clouds were registered with an absolute distance of < 0.3 m. The experimental results showed an accuracy of 18 cm for automated extraction of building roofs and 0.6 pixel for production of a high-resolution thermal point cloud, which was five times the density of the primary thermal point cloud and free from distortions
A modified YOLOv4 deep learning network for vision-based UAV recognition
Get PDFThe use of drones in various applications has now increased, and their popularity among the general public has increased. As a result, the possibility of their misuse and their unauthorized intrusion into important places such as airports and power plants are increasing, threatening public safety. For this reason, accurate and rapid recognition of their types is very important to prevent their misuse and the security problems caused by unauthorized access to them. Performing this operation in visible images is always associated with challenges, such as the small size of the drone, confusion with birds, the presence of hidden areas, and crowded backgrounds. In this paper, a novel and accurate technique with a change in the YOLOv4 network is presented to recognize four types of drones (multirotors, fixed-wing, helicopters, and VTOLs) and to distinguish them from birds using a set of 26,000 visible images. In this network, more precise and detailed semantic features were extracted by changing the number of convolutional layers. The performance of the basic YOLOv4 network was also evaluated on the same dataset, and the proposed model performed better than the basic network in solving the challenges. Compared to the basic YOLOv4 network, the proposed model provides better performance in solving challenges. Additionally, it can perform automated vision-based recognition with a loss of 0.58 in the training phase and 83% F1-score, 83% accuracy, 83% mean Average Precision (mAP), and 84% Intersection over Union (IoU) in the testing phase. These results represent a slight improvement of 4% in these evaluation criteria over the YOLOv4 basic mode
CMGFNet: A deep cross-modal gated fusion network for building extraction from very high-resolution remote sensing images
Get PDFThe extraction of urban structures such as buildings from very high-resolution (VHR) remote sensing imagery has improved dramatically, thanks to recent developments in deep multimodal fusion models. However, Due to the variety of colour intensities with complex textures of building objects in VHR images and the low quality of the digital surface model (DSM), it is challenging to develop the optimal cross-modal fusion network that takes advantage of these two modalities. This research presents an end-to-end cross-modal gated fusion network (CMGFNet) for extracting building footprints from VHR remote sensing images and DSMs data. The CMGFNet extracts multi-level features from RGB and DSM data by using two separate encoders. We offer two methods for fusing features in two modalities: Cross-modal and multi-level feature fusion. For cross-modal feature fusion, a gated fusion module (GFM) is proposed to combine two modalities efficiently. The multi-level feature fusion fuses the high-level features from deep layers with shallower low-level features through a top-down strategy. Furthermore, a residual-like depth-wise separable convolution (R-DSC) is introduced to enhance the performance of the up-sampling process and decrease the parameters and time complexity in the decoder section. Experimental results from challenging datasets show that the CMGFNet outperforms other state-of-the-art models. The efficacy of all significant elements is also confirmed by the extensive ablation study
Architectural heritage 3D modelling using unmanned aerial vehicles multi-view imaging
Get PDFToday, Architectural Heritage 3D models are created using Unmanned Aerial Vehicles (UAV) imagery and processing through Computer Vision (CV) methods. They are becoming more acceptable as reliable sources for study, documentation, diagnostics, intervention planning, monitoring, and management decision-making. The Deir-e-Kaj caravanserai, located in Qom, Iran, is a massive and half-destroyed architectural heritage that belongs to the Seljuk era. The obstructed access due to illegal deep excavations and the extensive demolished structure did not allow for a complete mapping using traditional and terrestrial techniques. Considering the condition and vulnerability of the artifact, it looks necessary to use a safe, non-destructive, and remote method for 3D documenting. The literature review shows in most of the research UAVs are used for acquiring nadir images, which are combined with the terrestrially acquired data for complete 3D modelling. In this case, a multi-view UAV imaging strategy is considered for the as-is 3D modelling of Deire-e-Kaj. Three different imaging angles are designed and used to carry out the comprehensive and all-needed data acquisition. The nadir images are acquired to cover the plan and enclosure, and the horizontal and oblique images cover the façades and interior spaces of the artifact. Adopting a suitable photogrammetric process based on the SfM workflow allows for obtaining an accurate, high-quality, and textured 3D model of the caravanserai. Accuracy evaluation of the result using Ground Control Points shows a total accuracy of ±1 cm. This study demonstrates the efficiency of multi-view UAV photogrammetry as a rapid, safe, and precise method to create a complete 3D model of massive, hard-to-access, and vulnerable Architectural Heritage.</p
DECISION-BASED FUSION OF PANSHARPENED VHR SATELLITE IMAGES USING TWO-LEVEL ROLLING SELF-GUIDANCE FILTERING AND EDGE INFORMATION
Get PDFPan-sharpening (PS) fuses low-resolution multispectral (LR MS) images with high-resolution panchromatic (HR PAN) bands to produce HR MS data. Current PS methods either better maintain the spectral information of MS images, or better transfer the PAN spatial details to the MS bands. In this study, we propose a decision-based fusion method that integrates two basic pan-sharpened very-high-resolution (VHR) satellite imageries taking advantage of both images simultaneously. It uses two-level rolling self-guidance filtering (RSGF) and Canny edge detection. The method is tested on Worldview (WV)-2 and WV-4 VHR satellite images on the San Fransisco and New York areas, using four PS algorithms. Results indicate that the proposed method increased the overall spectral-spatial quality of the base pan-sharpened images by 7.2% and 9.8% for the San Fransisco and New York areas, respectively. Our method therefore effectively addresses decision-level fusion of different base pan-sharpened images
Fusion of UAV-based infrared and visible images for thermal leakage map generation of building facades
Get PDFTo make the best use of available energy resources and reduce costs, improving the energy efficiency of buildings has become a critical issue for the construction industry. Today, developing a three-dimensional model of the energy consumption rates in buildings based on thermal infrared images is essential to visualize, identify and increase energy efficiency. The purpose of this study is to suggest a methodology for generating a thermal leakage map of building facades utilizing the fusion of thermal infrared and visible images captured by Unmanned Aerial Vehicles (UAVs). In general, the proposed method involves three basic steps: the generation of thermal infrared and visible dense point clouds from the building’s facade using Structure from Motion (SfM) and Multi-View Stereo (MVS) algorithms; the fusion of visible and thermal infrared dense point clouds using the Iterative Closest Point (ICP) algorithm to overcome thermal infrared point cloud constraints; the use of edge extraction and region-based segmentation methods to determine the location of the thermal leakage of building facade’s. To that end, two datasets obtained for separate building facades are used to assess the proposed strategy. The results of the data analyses for the extraction of the desired components and determination of thermal leakage locations on the building facets provided a Precision and Recall score of 87 and 90% for the first dataset and 87 and 88 for the second dataset. Examining the outcomes of calculating thermal leakage zones indicates improving Precision and Recall
Varying weighted spatial quality assessment for high resolution satellite image pan-sharpening
Get PDFThis paper focuses on spatial quality assessment of pan-sharpened imagery that contains valuable information of input images. Its aim is to show that fusion functions respond differently to different types of landscapes. It compares a quality assessment of an object-level procedure with that of a conventional pixel-level-based procedure which assigns uniform quality scores to all image pixels of pan-sharpened images. To do so, after performing a series of pan-sharpening evaluations, a weighted procedure for spatial quality assessments of pan-sharpening products, allocating spatially varying weight factors to the image pixels proportional to their level of spatial information content is proposed. All experiments are performed using five high-resolution image datasets using fusion products produced by three common pan-sharpening algorithms. The datasets are acquired from WorldView-2, QuickBird, and IKONOS. Experimental results show that the spatial distortion of fused images for the class vegetation cover exceeds that of man-made structures, reaching more than 4% in some cases. Our procedure can preclude illogical fidelity estimations occurring when pan-sharpened images contain different land covers. Since particular image structures are of high importance in remote sensing applications, our procedure provides a purpose-oriented estimation of the spatial quality for pan-sharpened images in comparison with conventional procedures
Citrus orchard mapping in Juybar, Iran: Analysis of NDVI time series and feature fusion of multi-source satellite imageries
Get PDFNowadays crop mapping as an interdisciplinary hot topic attracted both agriculture and remote sensing researchers' interests. This study proposed an automatic method to map citrus orchards in Juybar, Iran, where planting citrus trees is booming there. In this regard, 148 Sentinel-1, Sentinel-2, and ALOS Digital Surface Model (DSM) tiles are processed in Google Earth Engine to provide a hybrid feature set including initial satellite images, Gray Level Co-occurrence Matrix (GLCM) textural features, and spectral features such as vegetation, built-up, bare-soil indices, and the proposed Vegetation Dynamic Index (VDI). A semi-automatic sample selection paradigm is also developed based on a time-series analysis of 12 monthly Normalized Difference Vegetation Indices (NDVIs), Otsu thresholding, multi-level thresholding (MLT), and using two proposed indices called Evergreenness Index (EGI) and Water-covered or No-vegetation (WCNV) index, and finally human post-revision. The output of the Support Vector Machine (SVM) classification using 60,000 samples and the post-classification operation showed that the classified map has an average overall accuracy (OA) and an average kappa coefficient (KC) equal to 99.7% and 0.992, respectively. The results show that multispectral bands lonely extracted orchards with high accuracy (OA: 99.55%, KC: 0.986), and the rest of the features only made a slight improvement to that. For the year 2019, an area of about 4351 ha is estimated as citrus orchards, which is in accordance with the agriculture department's reports (~4700 ha). The results indicate that from 2016 to 2019, despite a “citrus to non-citrus” land-use conversion of about 754 ha, the citrus orchards area was totally expanded by about 17%. Comparing the results with the Google Earth images indicates the precise extraction of orchards with a 10 m spatial resolution. To use the proposed method for extensive cases or areas with other types of evergreen trees, it is recommended to use high-resolution normalized DSMs (nDSMs) and textural features
ARCHITECTURAL HERITAGE 3D MODELLING USING UNMANNED AERIAL VEHICLES MULTI-VIEW IMAGING
Get PDFToday, Architectural Heritage 3D models are created using Unmanned Aerial Vehicles (UAV) imagery and processing through Computer Vision (CV) methods. They are becoming more acceptable as reliable sources for study, documentation, diagnostics, intervention planning, monitoring, and management decision-making. The Deir-e-Kaj caravanserai, located in Qom, Iran, is a massive and half-destroyed architectural heritage that belongs to the Seljuk era. The obstructed access due to illegal deep excavations and the extensive demolished structure did not allow for a complete mapping using traditional and terrestrial techniques. Considering the condition and vulnerability of the artifact, it looks necessary to use a safe, non-destructive, and remote method for 3D documenting. The literature review shows in most of the research UAVs are used for acquiring nadir images, which are combined with the terrestrially acquired data for complete 3D modelling. In this case, a multi-view UAV imaging strategy is considered for the as-is 3D modelling of Deire-e-Kaj. Three different imaging angles are designed and used to carry out the comprehensive and all-needed data acquisition. The nadir images are acquired to cover the plan and enclosure, and the horizontal and oblique images cover the façades and interior spaces of the artifact. Adopting a suitable photogrammetric process based on the SfM workflow allows for obtaining an accurate, high-quality, and textured 3D model of the caravanserai. Accuracy evaluation of the result using Ground Control Points shows a total accuracy of ±1 cm. This study demonstrates the efficiency of multi-view UAV photogrammetry as a rapid, safe, and precise method to create a complete 3D model of massive, hard-to-access, and vulnerable Architectural Heritage
