1,721,045 research outputs found
Ebb & Flow: Uncovering Costantino Nivola's Olivetti Sandcast through 3D Fabrication and Virtual Exploration
No full textPractical Free-form RTI Acquisition with Local Spot Lights
No full textWe present an automated light calibration pipeline for free-form acquisition of shape and reflectance of objects using common
off-the-shelf illuminators, such as LED lights, that can be placed arbitrarily close to the objects. We acquire multiple digital
photographs of the studied object shot from a stationary camera. In each photograph, a light is freely positioned around the
object in order to cover a wide variety of illumination directions. While common free-form acquisition approaches are based
on the simplifying assumptions that the light sources are either sufficiently far from the object that all incoming light can be
modeled using parallel rays, or that lights are local points emitting uniformly in space, we use the more realistic model of
a scene lit by a moving local spot light with exponential fall-off depending on the cosine of the angle between the spot light
optical axis and the illumination direction, raised to the power of the spot exponent. We recover all spot light parameters
using a multipass numerical method. First, light positions are determined using standard methods used in photometric stereo
approaches. Then, we exploit measures taken on a Lambertian reference planar object to recover the spot light exponent and the
per-image spot light optical axis; we minimize the difference between the observed reflectance and the reflectance synthesized
by using the near-field Lambertian equation. The optimization is performed in two passes, first generating a starting solution
and then refining it using a Levenberg-Marquardt iterative minimizer. We demonstrate the effectiveness of the method based on
an error analysis performed on analytical datasets, as well as on real-world experiments
Web-based Exploration of Annotated Multi-Layered Relightable Image Models
No full textWe introduce a novel approach for exploring image-based shape and material models registered with structured descriptive information fused in multi-scale overlays. We represent the objects of interest as a series of registered layers of image-based shape and material data. These layers are represented at different scales and can come out of a variety of pipelines. These layers can include both Reflectance Transformation Imaging representations, and spatially varying normal and Bidirectional Reflectance Distribution Function fields, possibly as a result of fusing multi-spectral data. An overlay image pyramid associates visual annotations to the various scales. The overlay pyramid of each layer is created at data preparation time by either one of the three subsequent methods: (1) by importing it from other pipelines, (2) by creating it with the simple annotation drawing toolkit available within the viewer, and (3) with external image editing tools. This makes it easier for the user to seamlessly draw annotations over the region of interest. At runtime, clients can access an annotated multi-layered dataset by a standard web server. Users can explore these datasets on a variety of devices; they range from small mobile devices to large-scale displays used in museum installations. On all these aforementioned platforms, JavaScript/WebGL2 clients running in browsers are fully capable of performing layer selection, interactive relighting, enhanced visualization, and annotation display. We address the problem of clutter by embedding interactive lenses. This focus-and-context-aware (multiple-layer) exploration tool supports exploration of more than one representation in a single view. That allows mixing and matching of presentation modes and annotation display. The capabilities of our approach are demonstrated on a variety of cultural heritage use-cases. That involves different kinds of annotated surface and material models
Fast and accurate neural reflectance transformation imaging through knowledge distillation
Full text linkReflectance Transformation Imaging (RTI) is very popular for its ability to visually analyze surfaces by enhancing surface details through interactive relighting, starting from only a few tens of photographs taken with a fixed camera and variable illumination. Traditional methods like Polynomial Texture Maps (PTM) and Hemispherical Harmonics (HSH) are compact and fast, but struggle to accurately capture complex reflectance fields using few per-pixel coefficients and fixed bases, leading to artifacts, especially in highly reflective or shadowed areas. The NeuralRTI approach, which exploits a neural autoencoder to learn a compact function that better approximates the local reflectance as a function of light directions, has been shown to produce superior quality at comparable storage cost. However, as it performs interactive relighting with custom decoder networks with many parameters, the rendering step is computationally expensive and not feasible at full resolution for large images on limited hardware. Earlier attempts to reduce costs by directly training smaller networks have failed to produce valid results. For this reason, we propose to reduce its computational cost through a novel solution based on Knowledge Distillation (DISK-NeuralRTI). Starting from a teacher network that can be one of the original Neural RTI methods or a more complex solution, DISK-NeuralRTI can create a student architecture with a simplified decoder network that preserves image quality and has computational cost compatible with real-time web-based visualization of large surfaces. Experimental results show that we can obtain a student prediction that is on par or more accurate than the existing NeuralRTI solutions with up to 80% parameter reduction. Using a novel benchmark of high-resolution Multi-Light image collections (RealRTIHR), we also tested the usability of a web-based visualization tool based on our simplified decoder for realistic surface inspection tasks. The results show that the solution reaches interactive frame rates without the necessity of using progressive rendering with image quality loss
Multispectral RTI Analysis of Heterogeneous Artworks
Full text linkWe propose a novel multi-spectral reflectance transformation imaging (MS-RTI) framework for the acquisition and direct analysis of the reflectance behavior of heterogeneous artworks. Starting from free-form acquisitions, we compute per-pixel calibrated multi-spectral appearance profiles, which associate a reflectance value to each sampled light direction and frequency. Visualization, relighting, and feature extraction is performed directly on appearance profile data, applying scattered data interpolation based on Radial Basis Functions to estimate per-pixel reflectance from novel lighting directions. We demonstrate how the proposed solution can convey more insights on the object materials and geometric details compared to classical multi-light methods that rely on low-frequency analytical model fitting eventually mixed with a separate handling of high-frequency components, hence requiring constraining priors on material behavior. The flexibility of our approach is illustrated on two heterogeneous case studies, a painting and a dark shiny metallic sculpture, that showcase feature extraction, visualization, and analysis of high-frequency properties of artworks using multi-light, multi-spectral (Visible, UV and IR) acquisitions.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091the DSURF (PRIN 2015) project funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa
Crack Detection in Single- and Multi-Light Images of Painted Surfaces using Convolutional Neural Networks
Full text linkCracks represent an imminent danger for painted surfaces that needs to be alerted before degenerating into more severe aging effects, such as color loss. Automatic detection of cracks from painted surfaces' images would be therefore extremely useful for art conservators; however, classical image processing solutions are not effective to detect them, distinguish them from other lines or surface characteristics. A possible solution to improve the quality of crack detection exploits Multi-Light Image Collections (MLIC), that are often acquired in the Cultural Heritage domain thanks to the diffusion of the Reflectance Transformation Imaging (RTI) technique, allowing a low cost and rich digitization of artworks' surfaces. In this paper, we propose a pipeline for the detection of crack on egg-tempera paintings from multi-light image acquisitions and that can be used as well on single images. The method is based on single or multi-light edge detection and on a custom Convolutional Neural Network able to classify image patches around edge points as crack or non-crack, trained on RTI data. The pipeline is able to classify regions with cracks with good accuracy when applied on MLIC. Used on single images, it can give still reasonable results. The analysis of the performances for different lighting directions also reveals optimal lighting directions
A DICOM-Inspired Metadata Architecture for Managing Multimodal Acquisitions in Cultural Heritage
No full textQuantitative and qualitative analyses of cultural heritage (CH) assets need to interconnect individual pieces of information, including a variety of multimodal acquisitions, to form a holistic compounded view of studied objects. The need for joint acquisition brings with it the requirement for defining a protocol to store, structure and support the interoperability of the multisource data. In our work, we are performing multiple imaging studies in order to analyze the material, to monitor the behavior and to diagnose the status of CH objects. In particular, we employ, in addition to coarse 3D scanning, two high-resolution surface data capture techniques: reflectance transformation imaging and microprofilometry. Given this multivariate input, we have defined a hierarchical data organization, similar to the one used in the medical field by the Digital Imaging and Communications in Medicine (DICOM) protocol, that supports pre-alignment of local patches with respect to a global model. Furthermore, we have developed two supporting tools for multi-modal data handling: one for metadata annotation and another one for image registration. In this work, we illustrate our approach and discuss its practical application in a case study on a real CH object - a bronze bas-relief.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 66509
- …
