1,721,007 research outputs found
Deep Semantic Segmentation of Built Heritage Point Clouds
Full text linkL'abstract è presente nell'allegato / the abstract is in the attachmen
Fine-tuning and data augmentation techniques for semantic segmentation of heritage point clouds
Full text linkThis topic of this contribution falls within the broader debate on Digital Humanities. Experiencing and testing an approach that combines geomatics and its production of three-dimensional data of the built cultural heritage (CH) with information technology is the core point. In the digital CH domain, the ever-increasing availability of three-dimensional data, provides the opportunity to rapidly generate detailed 3D scenes to support restoration and conservation activities of built heritage.
Concurrently, the recent research trends in geomatics are facing the issue of managing these heritage data to enrich the geometrical representation of the asset, creating a complete informative data collector. HBIM (Historic Building Information Modeling) constitutes a reference, and they typically rely on point clouds to perform the scan-to-BIM processes.
These processes are still mostly manually carried out by domain experts, making the workflow very time-consuming, not fully exploiting the potential of point clouds and wasting an uncountable amount of data. In fact, parametric objects can be described through a few relevant points or contours. The use of Artificial Intelligence algorithms, in particular Deep Learning (DL) techniques, for the automatic recognition of architectural elements from point clouds can therefore provide valuable support through the semantic segmentation task. A proposal to tackle this framework was outlined in previous works, and the methodology here proposed constitutes a development of their results. Starting from those former tests obtained with the Dynamic Graph Convolutional Neural Network (DGCNN), close attention is paid to: i) transfer learning techniques, ii) the combination with external classifiers, such as Random Forest (RF), iii) the evaluation of data augmentation techniques on a domain-specific dataset (ArCH dataset). Besides, an investigation on how to make the whole workflow more functional
and "friendly" for external users is carried out too. With regard to transfer learning techniques, the fine-tuning approach is proposed to understand if, also in the CH domain, it can lead to performances improvement, introducing a new scene in a pre-trained network. In fact, the peculiarities of each scene do not guarantee certain and definite results, as for other domains. This section is divided into two subsections: a classic fine-tuning and a fine-tuning with the addition of the RF in the final part of the prediction. In the latter case, the choice of adding the RF is due to the results obtained in some stateof-the-art works, where this classifier provides excellent results in a short time and even in the presence of relatively limited data. In this hybrid approach, the network weights are employed as well as in the classic fine-tuning technique. Then, the final part of the DGCNN performing the segmentation of the points is excluded, leading the network to be used as a feature extractor method; afterwards, a scene of the dataset never seen by the network is chosen and divided into one part for
training and one for the test. Finally, the features of both parts are extracted, using the feature extractor, and exploited as input for training the RF classifier. Tests conducted on data augmentation show that it does not significantly affect overall performances, but still provide proper support for those categories with fewer points. On the other side, the tests on the fine-tuning have given rise to manifold considerations. Firstly, the standard fine-tuning can achieve performances almost equal to those where only the DGCNN is used, considerably improving some categories. Thus, they confirm that, once the
DNN is pre-trained, data processing and prediction times can be significantly reduced (from ca. 48 to 0.5 h), in the case of heritage point clouds too. Then, performances similar to the reference tests are obtained also with the use of the DGCNN as a feature extractor and the RF as a classifier, demonstrating that the final classifier does not affect the prediction
Semi-automatic integration of GIS data into BIM environment for urban district modeling / Integrazione semi-automatica di dati GIS in ambiente BIM per la modellazione dei distretti urbani
No full textThis research investigates a methodology for exchanging GIS-BIM data to generate urban portions through semi-automatic processes with visual programming.
La ricerca indaga una metodologia per lo scambio dati GIS-BIM per generare porzioni urbane attraverso processi semi-automatici con programmazione visuale
Transferencia de técnicas de aprendizaje y mejora del rendimiento en la segmentación semántica profunda de nubes de puntos del patrimonio construido
Full text link[EN] The growing availability of three-dimensional (3D) data, such as point clouds, coming from Light Detection and Ranging (LiDAR), Mobile Mapping Systems (MMSs) or Unmanned Aerial Vehicles (UAVs), provides the opportunity to rapidly generate 3D models to support the restoration, conservation, and safeguarding activities of cultural heritage (CH). The so-called scan-to-BIM process can, in fact, benefit from such data, and they can themselves be a source for further analyses or activities on the archaeological and built heritage. There are several ways to exploit this type of data, such as Historic Building Information Modelling (HBIM), mesh creation, rasterisation, classification, and semantic segmentation. The latter, referring to point clouds, is a trending topic not only in the CH domain but also in other fields like autonomous navigation, medicine or retail. Precisely in these sectors, the task of semantic segmentation has been mainly exploited and developed with artificial intelligence techniques. In particular, machine learning (ML) algorithms, and their deep learning (DL) subset, are increasingly applied and have established a solid state-of-the-art in the last half-decade. However, applications of DL techniques on heritage point clouds are still scarce; therefore, we propose to tackle this framework within the built heritage field. Starting from some previous tests with the Dynamic Graph Convolutional Neural Network (DGCNN), in this contribution close attention is paid to: i) the investigation of fine-tuned models, used as a transfer learning technique, ii) the combination of external classifiers, such as Random Forest (RF), with the artificial neural network, and iii) the evaluation of the data augmentation results for the domain-specific ArCH dataset. Finally, after taking into account the main advantages and criticalities, considerations are made on the possibility to profit by this methodology also for non-programming or domain experts.[ES] La creciente disponibilidad de datos tridimensionales (3D), como nubes de puntos, provenientes de la detección de la luz y distancia (LiDAR), sistemas de mapeado móvil (MMS) o vehículos aéreos no tripulados (UAV), brinda la oportunidad de generar rápidamente modelos 3D para apoyar las actividades de restauración, conservación y salvaguardia del patrimonio cultural (CH). El llamado proceso de escaneado-a-BIM puede, de hecho, beneficiarse de dichos datos, y ellos mismos pueden ser una fuente para futuros análisis o actividades sobre el patrimonio arqueológico y el construido. Hay varias formas de explotar este tipo de datos, como el modelado de información de edificios históricos (HBIM), la creación de mallas, la rasterización, la clasificación y la segmentación semántica. Este último, referido a las nubes de puntos, es un tema de máxima actualidad no solo en el dominio del PC sino también en otros campos como la navegación autónoma, la medicina o el comercio minorista. Precisamente en estos sectores, la tarea de la segmentación semántica se ha explotado y desarrollado principalmente con técnicas de inteligencia artificial. En particular, los algoritmos de aprendizaje automático (AA) y su subconjunto de aprendizaje profundo (AP) se aplican cada vez más y han establecido un sólido estado de la técnica en la última media década. Sin embargo, las aplicaciones de las técnicas de AP en las nubes de puntos tradicionales son todavía escasas; por tanto, nos proponemos abordar este marco dentro del ámbito del patrimonio construido. Partiendo de algunas pruebas anteriores con la Red Neural Convolucional de Gráfico Dinámico (DGCNN), en esta contribución se presta atención a: i) la investigación de modelos afinados, utilizados como técnica de aprendizaje por transferencia, ii) la combinación de clasificadores externos, como Random Forest (RF), con la red neuronal artificial, y iii) la evaluación de los resultados de aumentación de datos para el conjunto de datos específico del dominio ArCH. Finalmente, después de tener en cuenta las principales ventajas y criticidades, se hace una consideración sobre la posibilidad de beneficiarse de esta metodología también a expertos no programadores o del campo.Matrone, F.; Martini, M. (2021). Transfer learning and performance enhancement techniques for deep semantic segmentation of built heritage point clouds. Virtual Archaeology Review. 12(25):73-84. https://doi.org/10.4995/var.2021.15318OJS73841225Armeni, I., Sener, O., Zamir, A. R., Jiang, H., Brilakis, I., Fischer, M., & Savarese, S. (2016). 3D semantic parsing of large-scale indoor spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1534-1543. https://doi.org/10.1109/CVPR.2016.170Baraldi, L., Cornia, M., Grana, C., & Cucchiara, R. (2018). Aligning text and document illustrations: towards visually explainable digital humanities. In 24th International Conference on Pattern Recognition (ICPR), 1097-1102. IEEE. https://doi.org/10.1109/ICPR.2018.8545064Bassier, M., Yousefzadeh, M., & Vergauwen, M. (2020). Comparison of 2D and 3D wall reconstruction algorithms from point cloud data for as-built BIM. Journal of Information Technology in Construction (ITcon), 25(11), 173-192. https://doi.org/10.36680/j.itcon.2020.011Boulch, A., Guerry, J., Le Saux, B., & Audebert, N. (2018). SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks. Computers & Graphics, 71, 189-198. https://doi.org/10.1016/j.cag.2017.11.010Chadwick, J., (2020). Google launches hieroglyphics translator that uses AI to decipher images of Ancient Egyptian script. Available at https://www.dailymail.co.uk/sciencetech/article-8540329/Google-launches-hieroglyphics-translator-uses-AI-decipher-Ancient-Egyptian-script.html Last access 24/11/2020Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., & James, S. (2020). Machine learning for cultural heritage: a survey. Pattern Recognition Letters, 133, 102-108. https://doi.org/10.1016/j.patrec.2020.02.017Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013). Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research, 32(11), 1231-1237. https://doi.org/10.1177/0278364913491297Grilli, E., & Remondino, F. (2019). Classification of 3D digital heritage. Remote Sensing, 11(7), 847. https://doi.org/10.3390/rs11070847Grilli, E., & Remondino, F. (2020). Machine learning generalisation across different 3D architectural heritage. ISPRS International Journal of Geo-Information, 9(6), 379. https://doi.org/10.3390/ijgi9060379Grilli, E., Özdemir, E., & Remondino, F. (2019a). Application Of Machine And Deep Learning Strategies For The Classification Of Heritage Point Clouds. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-4/W18, 447-454, 2019. https://doi.org/10.5194/isprs-archives-XLII-4-W18-447-2019Grilli, E., Farella, E. M., Torresani, A., & Remondino, F. (2019b). Geometric features analysis for the classification of cultural heritage point clouds. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W15, 541-548, 2019 https://doi.org/10.5194/isprs-archives-XLII-2-W15-541-2019Hackel, T., Savinov, N., Ladicky, L., Wegner, J. D., Schindler, K., & Pollefeys, M. (2017). Semantic3d.net: A new large-scale point cloud classification benchmark. arXiv:1704.03847He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770-778. arXiv:1512.03385Korc, F., & Förstner, W. (2009). eTRIMS Image Database for interpreting images of man-made scenes. Dept. of Photogrammetry, University of Bonn, Tech. Rep. TR-IGG-P-2009-01.Landrieu, L., & Simonovsky, M. (2018). Large-scale point cloud semantic segmentation with superpoint graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4558-4567. arXiv:1711.09869Llamas, J., M Lerones, P., Medina, R., Zalama, E., & Gómez-García-Bermejo, J. (2017). Classification of architectural heritage images using deep learning techniques. Applied Sciences, 7(10), 992. https://doi.org/10.3390/app7100992Mathias, M., Martinovic, A., Weissenberg, J., Haegler, S., & VanGool, L. (2011). Automatic architectural style recognition. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XXXVIII-5/W16, 171-176 3. https://doi.org/10.3390/app7100992Matrone, F., Grilli, E., Martini, M., Paolanti, M., Pierdicca, R., & Remondino, F. (2020a). Comparing machine and deep learning methods for large 3D heritage semantic segmentation. ISPRS International Journal of Geo-Information, 9(9), 535. https://doi.org/10.3390/ijgi9090535Matrone, F., Lingua, A., Pierdicca, R., Malinverni, E. S., Paolanti, M., Grilli, E., Remondino, F., Murtiyoso, A., & Landes, T. (2020b). A benchmark for large-scale heritage point cloud semantic segmentation. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B2-2020, 1419-1426. https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-1419-2020Murtiyoso, A., & Grussenmeyer, P. (2019a). Automatic heritage building point cloud segmentation and classification using geometrical rules. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W15, 821-827. https://doi.org/10.5194/isprs-archives-XLII-2-W15-821-2019Murtiyoso, A., & Grussenmeyer, P. (2019b). Point cloud segmentation and semantic annotation aided by GIS data for heritage complexes. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W9, 523-528, 2019. https://doi.org/10.5194/isprs-archives-XLII-2-W9-523-2019Oses, N., Dornaika, F., & Moujahid, A. (2014). Image-based delineation and classification of built heritage masonry. Remote Sensing, 6(3), 1863-1889. https://doi.org/10.3390/rs6031863Park, Y., & Guldmann, J. M. (2019). Creating 3D city models with building footprints and LIDAR point cloud classification: A machine learning approach. Computers, Environment and Urban Systems, 75, 76-89. https://doi.org/10.1016/j.compenvurbsys.2019.01.004Pierdicca, R., Paolanti, M., Matrone, F., Martini, M., Morbidoni, C., Malinverni, E. S. & Lingua, A. M. (2020). Point cloud semantic segmentation using a deep learning framework for cultural heritage. Remote Sensing, 12(6), 1005. https://doi.org/10.3390/rs12061005Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 652-660. arXiv:1612.00593Sharafi, S., Fouladvand, S., Simpson, I., & Alvarez, J. A. B. (2016). Application of pattern recognition in detection of buried archaeological sites based on analysing environmental variables, Khorramabad Plain, West Iran. Journal of Archaeological Science: Reports, 8, 206-215. https://doi.org/10.1016/j.jasrep.2016.06.024Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556Stathopoulou, E. K., & Remondino, F. (2019). Semantic photogrammetry: boosting image-based 3D reconstruction with semantic labeling. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42(2), W9. https://doi.org/10.5194/isprs-archives-XLII-2-W9-685-2019Teboul, O., Kokkinos, I., Simon, L., Koutsourakis, P., & Paragios, N. (2012). Parsing facades with shape grammars and reinforcement learning. IEEE transactions on pattern analysis and machine intelligence, 35(7), 1744-1756. https://doi.org/10.1109/TPAMI.2012.252.Teruggi, S., Grilli, E., Russo, M., Fassi, F., & Remondino, F. (2020). A hierarchical machine learning approach for multi-level and multi-resolution 3D point cloud classification. Remote Sensing, 12(16), 2598. https://doi.org/10.3390/rs12162598Tyleček, R., & Šára, R. (2013). Spatial pattern templates for recognition of objects with regular structure. In German Conference on Pattern Recognition, Springer, Berlin, Heidelberg, 364-374. https://doi.org/10.1007/978-3-642-40602-7_39Verschoof-van der Vaart, W. B., & Lambers, K. (2019). Learning to Look at LiDAR: the use of R-CNN in the automated detection of archaeological objects in LiDAR data from the Netherlands. Journal of Computer Applications in Archaeology, 2(1). https://doi.org/10.5334/jcaa.32Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., &Solomon, J. M. (2019). Dynamic graph CNN for learning on point clouds. ACM Transactions On Graphics, 38(5), 1-12. arXiv:1801.07829Weinmann, M., Jutzi, B., Hinz, S., & Mallet, C. (2015). Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers. ISPRS Journal of Photogrammetry and Remote Sensing, 105, 286-304. https://doi.org/10.1016/j.isprsjprs.2015.01.016Xie, Y., Tian, J., & Zhu, X. X. (2019). Linking points with labels in 3D: a review of point cloud semantic segmentation. arXiv:1908.08854Yan, H., Ding, Y., Li, P., Wang, Q., Xu, Y., & Zuo, W. (2017). Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 2272-2281). arXiv:1705.00609 https://doi.org/10.1109/CVPR.2017.10
Initial Experiments on the Use of Radiance Fields for Underwater 3D Reconstruction
Full text linkUnderwater photogrammetry presents unique challenges, including light attenuation, refraction, and turbidity, that affect the accuracy and quality of 3D reconstructions. This study investigates the performance of novel neural rendering techniques, Neural Radiance Fields (NeRF), SeaThru-NeRF, and 3D Gaussian Splatting (3DGS), in comparison to conventional Structure-from-Motion (SfM) workflows. Using a dataset acquired during the SIFET benchmark campaign on a submerged Roman archaeological site, we processed image data via Nerfacto, SeaThru, and Jawset Postshot (3DGS) and compared outputs against a reference model produced in Agisoft Metashape. Evaluation criteria included processing time, geometric accuracy (via M3C2 analysis), point cloud density and roughness, and point cloud completeness. Results show that radiance fields-based methods significantly reduce processing time while providing competitive visual results. SeaThru-NeRF demonstrated the highest geometric accuracy, benefiting from underwater-specific corrections, while 3DGS offered photorealistic rendering. These findings highlight the potential of neural methods for underwater cultural heritage documentation, though further improvements are needed in data fidelity and robustness under challenging underwater conditions
Enhancing explainability of deep learning models for point cloud analysis: a focus on semantic segmentation
Full text linkSemantic segmentation of point clouds plays a critical role in various applications, such as urban planning, infrastructure management, environmental analyses and autonomous navigation. Understanding the behaviour of deep neural networks (DNNs) in analysing point cloud data is essential for improving segmentation accuracy and developing effective network architectures and acquisition strategies. In this paper, we investigate the traits of some state-of-the-art neural networks using indoor and urban outdoor point cloud datasets. We compare PointNet, DGCNN, and BAAF-Net on specifically selected datasets, including synthetic and real-world environments. The chosen datasets are S3DIS, SynthCity, Semantic3D, and KITTI. We analyse the impact of different factors such as dataset type (synthetic vs. real), scene type (indoor vs. outdoor), and acquisition system (static vs. mobile sensors). Through detailed analyses and comparisons, we provide insights into the strengths and limitations not only of different network architectures in handling urban point clouds but also of their data structure. This study contributes to going beyond the mere and unconditional use of AI algorithms, trying to explain DNNs behaviour in point cloud analysis and paving the way for future research to enhance segmentation accuracy and develop possible guidelines both for network design and data acquisition in the geomatics field
From scan-to-BIM to a structural finite elements model of built heritage for dynamic simulation
Full text linkThe progress in information technology allows an innovative transformation of practices commonly involved in the engineering and construction field, especially in relation to the existing architectural heritage’s control and management activities. The proposed methodology takes advantage of an integrated 3D metric survey as a basis for an HBIM (Historic Building Information Modelling) model to be exploited for the definition of a Finite Elements Model (FEM). This paper aims to show the applicability of a digital process, stemmed from the integration in Rhinoceros 3D of a BIM structural model, leading to the dynamic simulation of the analytical FEM through PRO_SAP® (a PROfessional Structural Analysis Program). The described workflow investigates the interoperability issues, along with the difficulties in the Scan-to-HBIM processes, demonstrating how HBIM
models can anyhow support operations aimed at maintaining and preserving existing historical assets, also from a structural point of view, even if with still persistent criticalities
Armonizzazione di standard spaziali e normativa antisismica: Una proposta per la rappresentazione semantica 3D del complesso architettonico di Tolentino
Full text linkThe need of effectively sharing information about architectural heritage affectedby disaster event, in order to foster its preservation, requires the adoption of acommon language and standards among the involved actors and stakeholders.The application of spatial and geographical databases, enabling to connect architectural heritage representation with the data useful for hazard and risk analysis, could facilitate the pre and post event estimation of vulnerability.This paper outlines a methodology to represent 3D models of the architecturalheritage according to some existing standards data models and thesauri inventories (INSPIRE, CityGML, UNESCO, CIDOC-CRM, MONDIS, Getty). Inaddition, as a consequence of the collaboration between the Geomatics group and the Structural and Seismic Engineering group of the Polytechnic of Turin, an integration of the database for a correlation between the geometric entities of structural components and their related earthquake damage mechanisms was tested.Urban Data Scienc
Exploitation of the Number of Return Echoes for DTM Extraction from Point Clouds Acquired by LiDAR UAS DJI Zenmuse L1
Full text linkFollowing the enormous technological developments of LiDAR (Light Detection And Ranging) sensors, it is currently easier to find them commercially in the UA (Uncrewed Aerial Systems) sector. In particular, with the Zenmuse L1 by DJI (Dà-Jiāng Innovations) the market has grown globally, mainly due to the compactness of the product that is easily compatible with UAS. The L1 sensor can record up to three returns of the emanating signal, so it can acquire a larger amount of points, such as those below the vegetation. Therefore, in addition to the geometric information of the points, the Zenmuse L1 point clouds also provide other information, such as the number of echo returns from 1 to 3. This data could be exploited to improve the automatic extraction of the digital terrain model (DTM) from the point clouds, hopefully leading to the avoidance of manual correction. This research aims to focus on evaluating whether the addition of the return number feature can affect the identification of the ground points through different computational methods and can improve the time efficiency of state-of-the-art algorithms
NEW DEVELOPMENTS IN LIDAR UAS SURVEYS. PERFORMANCE ANALYSES AND VALIDATION OF THE DJI ZENMUSE L1
Full text linkThanks to the latest technological developments LiDAR (Light Detection And Ranging) sensors are no longer an exclusive feature of manned airborne platforms but they are close to becoming a commercial solution in the UAS (Uncrewed Aerial Systems) domain. The release on the market of the Zenmuse L1 by DJI (Dà-Jiāng Innovations) is a step further in this direction, thanks also to a substantial work of enhancement made by the Chinese company not only on the hardware side, but also on the software one. The research presented in this work is focused on the use of the L1 LiDAR for the 3D survey of built heritage, analysing the results of different tests to highlight first considerations on its performances and the point cloud quality. Considering its recent release, this sensor is still yet to be thoroughly analysed and validated and its performances to be assessed. LiDAR data has been acquired on a selected test site, documented also with traditional Terrestrial Laser Scanner (TLS) and UAS photogrammetry. The latter techniques (supported also by a topographic survey) will thus be exploited to generate the ground reference and to assess the quality and accuracy of the L1 dataset
- …
