1,720,964 research outputs found
AUTOMATIC MASKS GENERATION FOR THE DIGITIZATION OF A WOODEN COLLECTION OF MAQUETTES USING DEEP LEARNING
Get PDFSegmentación semántica multiclase en la digitalización del patrimonio mueble utilizando técnicas de aprendizaje profundo
Get PDF[EN] Digitisation processes of movable heritage are becoming increasingly popular to document the artworks stored in our museums. An increasing number of strategies for the three-dimensional (3D) acquisition and modelling of these invaluable assets have been developed in the last few years, to efficiently respond to this documentation need and contribute to deepening the knowledge of the masterpieces investigated constantly by researchers operating in many fieldworks. Nowadays, one of the most effective solutions is represented by the development of image-based techniques, usually connected to a Structure-from-Motion (SfM) photogrammetric approach. However, while the acquisition of the images is relatively rapid, it is the processes connected to the data processing that are very time-consuming and require substantial manual involvement of the operator. The development of deep learning-based strategies can be an effective solution to enhance the level of automatism. In the case of the current research, which has been carried out in the framework of the digitisation of a collection of wooden maquettes stored in the ‘Museo Egizio di Torino’ using a photogrammetric approach, an automatic masking strategy using deep learning techniques is proposed, to increase the level of automatism and therefore, optimise the photogrammetric pipeline. Starting from a manually annotated dataset a neural network has been trained to automatically perform a semantic classification with the aim to isolate the maquettes from the background. The proposed methodology has allowed obtaining automatically segmented masks with a high degree of accuracy. The followed workflow is described (as regards acquisition strategies, dataset processing, and neural network training), and the accuracy of the results is evaluated and discussed. In addition, the possibility of performing a multiclass segmentation on the digital images to recognise different categories of objects in the images and define a semantic hierarchy is proposed to perform automatic classification of different elements in the acquired images.[ES] Los procesos de digitalización del patrimonio mueble son cada vez más populares para documentar las obras de arte almacenadas en nuestros museos. En los últimos años se han desarrollado un número creciente de estrategias de adquisición y modelado tridimensional (3D) de estos activos de valor incalculable, que responden de manera eficiente a esta necesidad de documentación y contribuyen a profundizar en el conocimiento de las obras maestras investigadas constantemente por investigadores que operan en muchos trabajos de campo. Hoy en día, una de las soluciones más efectivas está relacionada con el desarrollo de técnicas basadas en imágenes, generalmente conectadas a un enfoque fotogramétrico de estructura-y-movimiento (SfM). Sin embargo, si bien la adquisición de las imágenes es relativamente rápida, son los procesos relacionados con el procesamiento de los datos los que consumen mucho tiempo y requieren una participación manual sustancial del operador. El desarrollo de estrategias basadas en el aprendizaje profundo puede ser una solución eficaz para mejorar el nivel de automatismo. En el caso de la presente investigación, que se ha llevado a cabo en el marco de la digitalización de una colección de maquetas de madera almacenadas en el 'Museo Egizio di Torino' mediante un enfoque fotogramétrico, se propone una estrategia de enmascaramiento automático mediante técnicas de aprendizaje profundo, que incrementa el nivel de automatismo y por tanto optimiza el flujo fotogramétrico. A partir de un conjunto de datos anotados manualmente, se ha entrenado una red neuronal que realiza automáticamente una clasificación semántica con el objetivo de aislar las maquetas del fondo. La metodología propuesta ha permitido obtener más caras segmentadas automáticamente con alto grado de precisión. Se describe el flujo de trabajo seguido (en cuanto a estrategias de toma, procesamiento del conjuntos de datos y entrenamiento de las redes neuronales), y se evalúa y discute la precisión de los resultados. Además, se propone la posibilidad de realizar una segmentación multiclase sobre las imágenes digitales que permitan reconocer diferentes categorías de objetos en las imágenes y definir una jerarquía semántica que clasifique automáticamente diferentes elementos en la toma de las imágenes.The authors thank Volta® A.I. (and in particular Silvio Revelli) for the contribution to this work and for providing high-end hardware for neural network training.
In addition, they would like to thank Alessia Fassone of Museo Egizio di Torino and all the people involved in the B.A.C.K. TO T.H.E. F.U.T.U.RE. project (in particular, Fulvio Rinaudo, who coordinated the Geomatic team).
Finally, they wish to express their gratitude to Nannina Spanò and Filiberto Chiabrando for the helpful confrontation during the presented research.Patrucco, G.; Setragno, F. (2021). Multiclass semantic segmentation for digitisation of movable heritage using deep learning techniques. Virtual Archaeology Review. 12(25):85-98. https://doi.org/10.4995/var.2021.15329OJS85981225Adami, A., Balletti, C., Fassi, F., Fregonese, L., Guerra, F., Taffurelli, L., Vernier, P. (2015). The bust of Francesco II Gonzaga: From digital documentation to 3D printing. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, II-5/W3, 9-15. https://doi.org/10.5194/isprsannals-II-5-W3-9-2015Badrinarayanan, V., Kendall, A., Cipolla, R. (2017). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615Balletti, C., Ballarin, M., & Guerra, F. (2017). 3D printing: state of the art and future perspectives. Journal of Cultural Heritage, 26,172-182. https://doi.org/10.1016/j.culher.2017.02.010Balletti, C., & Ballarin, M. (2019). An application of integrated 3D technologies for replicas in Cultural Heritage. International Journal of Geo-Information, 8(6), 285. https://doi.org/10.3390/ijgi8060285Barbieri, L., Bruno, F., & Muzzupappa, M. (2018). User-centered design of a virtual reality exhibit for archaeological museums. International Journal on Interactive Design and Manufacturing (IJIDeM), 12, 561-571. https://doi.org/10.1007/s12008-017-0414-zCaruana, R., Lawrence, S., & Giles, C. L. (2001). Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. Advances in Neural Information Processing Systems (pp. 402-408). https://doi.org/10.1109/IJCNN.2000.857823Cermelli, F., Mancini, M., Bulo, S. R., Ricci, E., & Caputo, B. (2020). Modeling the background for incremental learning in semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9233-9242. https://doi.org/10.1109/CVPR42600.2020.00925Condorelli, F., Rinaudo, F., Salvadore, F., & Tagliaventi, S. (2020). A neural network approach to detecting lost heritage in historical video. International Journal of Geo-Information, 9(5), 297. https://doi.org/10.3390/ijgi9050297Chiabrando, F., Sammartano, G., Spanò, A., & Spreafico, A. (2019). Hybrid 3D models: When Geomatics innovations meet extensive built heritage complexes. International Journal of Geo-Information, 8(3), 124. https://doi.org/10.3390/ijgi8030124Dall'Asta, E., Bruno, N., Bigliardi, G., Zerbi, A., & Roncella, R.(2016). Photogrammetric techniques for promotion of archaeological Heritage: the Archaeological Museum of Parma (Italy). International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLI-B5, 243-250. https://doi.org/10.5194/isprs-archives-XLI-B5-243-2016Felicetti, A., Paolanti, M., Zingaretti, P., Pierdicca, R., & Malinverni, E. S. (2020). Mo.Se.: Mosaic image segmentation based on deep cascading learning. Virtual Archaeology Review, 12(24), 25-38. https://doi.org/10.4995/var.2021.14179Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., & James, S. (2020). Machine Learning for Cultural Heritage: A Survey. Pattern Recognition Letters, 133, 102-108. https://doi.org/10.1016/j.patrec.2020.02.017Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., & Garcia-Rodriguez, J. (2017). A survey on deep learning techniques for image and video semantic segmentation. Applied Soft Computing, 70, 41-65. https://doi.org/10.1016/j.asoc.2018.05.018George, D., Xie, X., & Tam, G. K. (2018). 3D mesh segmentation via multi-branch 1D convolutional neural networks. Graphical Models, 96, 1-10. https://doi.org/10.1016/j.gmod.2018.01.001Giuffrida, D., Mollica Nardo, V., Giacobello, F., Adinolfi, O., Mastelloni, M. A., Toscano, G., & Ponterio, R. S. (2019). Combined 3D surveying and Raman Spectroscopy Techniques on artifacts preserved at Archaeological Musem of Lipari. Heritage, 2(3), 2017-2027. https://doi.org/10.3390/heritage2030121Grilli, E., Farella, E. M., Torresani, A., & Remondino, F. (2019). Geometric features analysis for the classification of Cultural Heritage point clouds. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W15, 541-548. https://doi.org/10.5194/isprs-archives-XLII-2-W15-541-2019Grilli, E., Özdemir, E., & Remondino, F. (2019). Application of machine and deep learning strategies for the classification of Heritage point clouds. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-4/W18, 447-454. https://doi.org/10.5194/isprs-archives-XLII-4-W18-447-2019Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.Gu, J., Wang, Z., Kuen, J., Ma., L., Shahroudy, A., Shuai, B., & Chen., T. (2018). Recent advances in convolutional neural networks. Pattern Recognition, 77, 354-377. https://doi.org/10.1016/j.patcog.2017.10.013Guidi, G., Malik, U. S., Frischer, B., Barandoni, C., & Paolucci, F. (2017). The Indiana University-Uffizi project: Metrological challenges and workflow for massive 3D digitization of sculptures. 23rd International Conference on Virtual System & Multimedia (VSMM), 1-8. https://doi.org/10.1109/VSMM.2017.8346268He, T., Shen, C., Tian, Z., Gong, D., Sun, C., & Yan, Y. (2019). Knowledge adaptation for efficient semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 578-587. https://doi.org/10.1109/CVPR.2019.00067Jégou, S., Drozdzal, M., Vazquez, D., Romero, A., & Bengio, Y. (2017). The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 11-19). https://doi.org/10.1109/CVPRW.2017.156Kersten, T. P., Tschirschwitz, F., & Deggim, S. (2017). Development of a virtual museum including a 4D presentation of building history in Virtual Reality. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W3, 361-367. https://doi.org/10.5194/isprs-archives-XLII-2-W3-361-2017Knyaz, A. V., Kniaz, V. V., Remondino, F., Zheltov, S. Y., & Gruen, A. (2020). 3D reconstruction of a complex grid structure combining UAS images and deep learning. Remote Sensing, 12(19), 3128. https://doi.org/10.3390/rs12193128Lin, P., Sun, P., Cheng, G., Xie, S., Li, X., & Shi, J. (2020). Graph-guided architecture search for real-time semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4203-4212. https://doi.org/10.1109/CVPR42600.2020.00426Llamas, J., Lerones, P. M., Medina, R., Zalama, E., & Gómez-García-Bermejo, J. (2017). Classification of architectural heritage images using deep learning techniques. Applied Science, 7(10), 992. https://doi.org/10.3390/app7100992Lo Turco, M., Piumatti, P., Rinaudo, F., Tamborrino, R., & González-Aguilera, D., (2018). B.A.C.K. TO T.H.E. F.U.T.U.RE. − BIM acquisition as cultural key to transfer heritage of ancient Egypt for many uses to many users replayed. In S. Bertocci (Ed.), Programmi Multidisciplinari Per L'internazionalizzazione Della Ricerca. Patrimonio Culturale, Architettura e Paesaggio (pp. 107-109). DIDA Press.Lo Turco, M., Piumatti, P., Rinaudo, F., Calvano, M., Spreafico, A., & Patrucco, G. (2018). The digitisation of museum collections for research, management and enhancement of tangible and intangible heritage. 3rd Digital Heritage International Congress (DigitalHERITAGE) held jointly with 24th International Conference on Virtual Systems & Multimedia (VSMM 2018), San Francisco, CA, USA. https://doi.org/10.1109/DigitalHeritage.2018.8810128Mafrici, N., & Giovannini, E. C. (2020). Digitalizing data: From the historical research to data modelling for a (digital) collection documentation. In M. Lo Turco, E. C. Giovannini, , & N. Mafrici (Eds.), Digital & Documentation. Digital Strategies for Cultural Heritage (Vol. 2, pp. 38-51). Pavia University Press. https://doi.org/10.5194/isprs-archives-XLII-2-W15-519-2019Malik, U. S., Guidi, G. (2018). Massive 3D digitization of sculptures: Methodological approaches for improving efficiency. IOP Conference Series: Material Science and Engineering, 364. https://doi.org/10.1088/1757-899X/364/1/012015Minto, S., & Remondino, F. (2014). Online access and sharing of reality-based 3D models. SCIRES-IT-SCIentific RESearch and Information Technology, 4(2), 17-28. http://doi.org/10.2423/i22394303v4n2p17Patrucco, G., Chiabrando, F., Dondi, P, & Malagodi, M. (2018). Image and range-based 3D acquisition and modeling of popular musical instruments. Proceedings from the Document Academy, 5(2), 9. https://doi.org/10.35492/docam/5/2/9Patrucco, G., Rinaudo, F., & Spreafico, A. (2019). A new handheld scanner for 3D survey of small artifacts: The Stonex F6. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W15, 895-901. https://doi.org/10.5194/isprs-archives-XLII-2-W15-895-2019Pierdicca, R., Paolanti, M., Matrone, F., Martini, M., Morbidoni, C., Malinverni, E. S., Frontoni, E., & Lingua, A. M. (2020). Point cloud semantic segmentation using a deep learning framework for Cultural Heritage. Remote Sensing, 12(6), 1005. https://doi.org/10.3390/rs12061005Salvador-García, E., Viñals, M. J., & García-Valldecabres, J. L. (2020). Potential of HBIM to improve the efficiency of visitor flow management in Heritage sites. Towards smart heritage management. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIV-M-1-2020, 451-456. https://doi.org/10.5194/isprs-archives-XLIV-M-1-2020-451-2020Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 1-48. https://doi.org/10.1186/s40537-019-0197-0Stathopoulou, E. K., & Remondino, F. (2019). Semantic photogrammetry: Boosting image-based 3D reconstruction with semantic labeling. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W9, 685-690. https://doi.org/10.5194/isprs-archives-XLII-2-W9-685-2019UNESCO. (1979). Recommendation for the Protection of Movable Cultural Property, Records of the General Conference, 20th Session, I: Resolutions. Paris: UNESCO.Vargas, R., Mosavi, A., & Ruiz, R. (2018). Deep learning: A review. Advances in Intelligent Systems and Computing, 29(8), 232-244. https://doi.org/10.20944/PREPRINTS201810.0218.V1Yazan, E., & Talu, M. F. (2017). Comparison of the stochastic gradient descent based optimization techniques. 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), 1-5. https://doi.org/10.1109/IDAP.2017.809029
ENHANCING AUTOMATION OF HERITAGE PROCESSES: GENERATION OF ARTIFICIAL TRAINING DATASETS FROM PHOTOGRAMMETRIC 3D MODELS
Get PDFNowadays, many efficient technologies have been developed with the aim of collecting digital images and other metric data, greatly optimising the acquisition procedures and techniques. However, processing this data can be onerous and time-consuming, and increasingly often, there is a need to develop new strategies to enhance the level of automation of these processes. Using artificial intelligence, and particularly Convolutional Neural Networks, it is possible to automate processing tasks such as classification and segmentation. However, a significant challenge is represented by the necessity of obtaining sufficient training data to properly train a deep learning model. These datasets are composed of a significant amount of data and need to be annotated, which sometimes represents an onerous and challenging task. Synthetic data can represent an effective solution to this problem, significantly reducing the time and effort required to manually create annotated datasets and can be particularly useful when studying objects characterised by specific features and high complexity, requiring tailored solutions and ad hoc training. The presented research explores the opportunity of using synthetic datasets – generated from photogrammetric 3D models – for deep-learning-based heritage digitisation applications. The use of synthetic data generated from textured 3D models derived from SfM photogrammetric processes is proposed, with the aim of enhancing automatic procedures in the framework of heritage processes
Synthetic Training Datasets for Architectural Conservation: A Deep Learning Approach for Decay Detection
Get PDFArchitectural heritage conservation increasingly relies on innovative tools for detecting and monitoring degradation. The study presented in the current paper explores the use of synthetic datasets—namely, rendered images derived from photogrammetric models—to train convolutional neural networks (CNNs) for the automated detection of deterioration in historical reinforced concrete structures. The primary objective is to assess the effectiveness of synthetic images for deep learning training, comparing their performance with models trained on traditional datasets. The research focuses on a significant case study: the parabolic concrete arch of Morano sul Po. Two classification scenarios were tested: a single-class model for structure recognition and a multi-class model for identifying degradation patterns, such as exposed reinforcement bars. The findings indicate that synthetic datasets can effectively support structure identification, achieving results comparable to those obtained with real-world imagery. However, challenges arise in multi-class classification, particularly in distinguishing fine-grained degradation features. This study highlights the potential of artificial datasets in overcoming the limitations of annotated data availability in heritage conservation. The proposed approach represents a promising step toward automating documentation and damage assessment, ultimately contributing to more efficient and scalable heritage monitoring strategies
The Violin Ontology
No full textBowed musical instruments have been the subject of scientific investigations for centuries. Yet, the physical phenomena that are behind their timbral quality are still far from being fully understood. This is one of the reasons why the art of violin making is still so strongly tied to tradition. This manuscript describes early results in a study of the relations that exist between timbral and acoustic characteristics of such instruments and their high-level descriptors. In particular, we propose a suitable ontology for a timbral characterization of violins, where every resource is connected and provided with formally defined semantics. Semantic web technologies have taught us how ontologies can become a powerful tool for gathering and managing knowledge in specific areas of interest, where resources are connected and described with formally defined semantics. This, in fact, represents a crucial step for building applications that reason over Web data. In this paper we present an ontology for knowledge representation of violins, as part of a wider ontology of bowed instruments. With this ontology we capture timbral and acoustic aspects of violins as well as violin making and properties of the materials used for their production. We collected and organized semantic descriptors used by numerous violin makers (from natural language) to describe sound proprieties of musical instruments. We also developed an initial model of the relation between semantic descriptors and low-level audio features. The ontology that we present in this study formalizes the semantics of the high-level descriptors and investigates the relation with low- level features. The terminology has been collected through a series of interviews with violin makers in the city of Cremona (Italy), world heritage site for the practice of violin makers. Through listening tests and a feature extraction we study the correlation between high-level descriptors and objective properties of sound
Training-based semantic descriptors modeling for violin quality sound characterization
Get PDFViolin makers and musicians describe the timbral qualities of violins using semantic terms coming from natural language. In this study we use regression techniques of machine intelligence and audio features to model in a training-based fashion a set of high-level (semantic) descriptors for the automatic annotation of musical instruments. The most relevant semantic descriptors are collected through interviews to violin makers. These descriptors are then correlated with objective features extracted from a set of violins from the historical and contemporary collections of the Museo del Violino and of the International School of Luthiery both in Cremona. As sound description can vary throughout a performance, our approach also enables the modelling of time-varying (evolutive) semantic annotation
Feature-Based Analysis of the Impact of Ground Coat and Varnish on Violin Tone Qualities
Get PDFAssessing the tonal qualities of an acoustic musical instrument is a challenge that has long been pursued by
researchers in musical acoustics. This is a topic of particular interest when it comes to discussing the case of
violins. Historical violins are often believed to owe a great deal of their celebrated timbral qualities to the choice
and the make of their finishings, particularly the ground coat and the varnish. The impact of such finishings
on the instrument’s tonal qualities, however, is not so well understood. In this paper we investigate the impact
of the finishing process on the instrument’s timbre through a joint analysis of the characteristics of the materials
involved for this process (ground coat and varnish) and audio features extracted from the sound produced by such
violins at various stages of their finishing. Some of the results are compared with those found in the literature for
validation purposes. The characterization of the impact of ground coat and varnish has been conducted during
the finishing process of a new violin
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
