1,721,204 research outputs found
Joint MAM - MANER Conference - Material Appearance Network for Education and Research: Frontmatter
No full textWorkshop on Material Appearance ModelingJoint MAM - MANER Conference - Material Appearance Network for Education and Researc
Can we Grasp the Color of Translucent Objects?
No full textWhile colorimetry is traditionally measuring point colors, there is an increasing need to quantify colors of 3D objects in real-world scenes. 3D objects, especially translucent ones, exhibit high spatio-temporal variation in color. This raises multiple questions on how to measure color of 3D translucent objects, how to describe their color appearance, and how to quantify color differences among them. Or are these ill-posed problems in the first place? We discuss the first steps on this topic and suggest the future directions for color and appearance research.Workshop on Material Appearance ModelingJoint MAM - MANER Conference - Material Appearance Network for Education and ResearchPerception / Color and Spectra / Reproductio
Material Appearance for Conservation and Restoration- Capturing and modelling the appearance of gilded surfaces
No full textThe human visual system is extremely efficient at identifying and recognising different materials. Rarely do humans mistake one material for another, and the appearance of a material tells us many important cues about the environment we live in. However, the mechanisms behind material appearance and perception are not fully understood.
Cultural heritage objects exist in a wide range of materials, each with its own particular appearance. While it is common to evaluate the colour of cultural heritage objects before and after restoration, this is not always representative of perceptual differences that the restoration may cause. Gilding is a form of polychromy commonly used in the Middle Ages which poses many technical challenges to acquire its appearance. The gold leaf, of metallic nature, creates a particular appearance which changes depending on the fabrication method and has a strong angular-dependence.
This thesis aims to address some of these issues, by acquiring, modelling and analysing the appearance of complex cultural heritage materials. Focus is given to gilded surfaces since they exhibit interesting appearance properties such as gloss, metallicity, and in some cases even translucency. The research is divided into two sub-objectives, one concerning methods for appearance capture, and the second deals with applying material appearance analysis for conservation and restoration.
The first research domain of this PhD thesis deals with methods for material appearance capture. First, the challenges and limitations of using conventional methods for appearance capture are explored. As an alternative, an imaging-based methodology is developed and evaluated for bi- directional reflectance measurements, catered for cultural heritage materials of challenging appearance. This method is then applied to satisfy the second sub-objective.
The second research domain concerns the application of material appearance analysis for conservation. For this, the appearance of different types of gilding is acquired, modelled and evaluated. By using perceptual gloss metrics the different types of gilding can be described and characterised in terms of perceptual differences. Moreover, varnish removal methods are evaluated in terms of appearance change, to guide the conservation of a 15th century painted panel
On the Appearance of Translucent Objects: Perception and Assessment by Human Observers
No full textAppearance characterizes visual features of objects and materials. It is a multiplex psychovisual phenomenon that is usually broken into several appearance attributes for simplification of its measurement and communication, and for studying its nature. Color, texture, gloss, and translucency are considered the major appearance attributes. Significant research work has been done in metrology for accurate instrumental measurement of optical properties of materials, and considerable advances have been made in computer graphics, permitting the generation of highly photorealistic visual stimuli. Nevertheless, the knowledge remains limited on how humans perceive appearance, how we behave to assess appearance, what factors impact our perception, how different attributes interact with each other, and all in all how optical properties relate with their perceptual counterparts.
In this thesis, we explore various aspects of appearance perception with a focus on the appearance of translucent objects. For this purpose, we conducted a series of social and psychophysical experiments with real and synthetic visual stimuli. Elucidating appearance perception of translucent objects has implications for industrial, academic and artistic applications alike.
In the initial stage of the study, we organized a social experiment in order to collect qualitative observations on the process of appearance assessment, construct a qualitative model of material appearance and generate relevant research hypotheses. The hypotheses have been analyzed in context of the state-of-the-art.
Afterwards, we tested the most interesting hypotheses quantitatively, in order to assess their generalization prospects. The experimental results have provided indications in support of the hypotheses. We have observed that translucency of an object impacts perception of glossiness, while detection of translucency difference depends on geometric thickness of the objects and optical thickness of the materials they are made of. Additionally, we examined a potential role of several cues in translucency perception that are present in the image detected by either a camera or a human observer. We found that blurriness of the image and the presence of caustics can impact apparent translucency.
Finally, we conducted a comprehensive survey on translucency perception, advancing the state-of-the-art with our findings, and outlining unanswered questions for future research.Sammendrag
Utseende karakteriserer visuelle egenskaper ved gjenstander og materialer. Det er et mangfoldig psykovisuelt fenomen som vanligvis blir brutt ned til flere utseendeattributter, for å forenkle dets måling og kommunikasjon, og studering av dets natur. Farge, tekstur, glans og gjennomskinnelighet anses som de viktigste utseendeattributtene. Det er gjort betydelig forskningsarbeid innen metrologi for nøyaktig instrumentell måling av materialers optiske egenskaper, og betydelige fremskritt innen datagrafikk som tillater generering av meget fotorealistiske visuelle stimuli. Likevel er kunnskapen fortsatt begrenset om hvordan mennesker oppfatter utseende, hvordan vi oppfører oss for å vurdere utseende, hvilke faktorer som påvirker vår oppfatning, hvordan forskjellige attributter innvirker på hverandre, og alt i alt hvordan optiske egenskaper relateres til deres perseptuelle motstykker.
I denne avhandlingen utforsker vi ulike persepsjonsaspekter med fokus på utseendet til gjennomskinnelige objekter. For dette formålet gjennomførte vi en serie sosiale og psykofysiske eksperimenter med ekte og syntetiske visuelle stimuli. Kunnskap om uteseende til gjennomskinnelige gjenstander har implikasjoner for både industrielle, akademiske og kunstneriske anvendelser.
I den innledende fasen av studien gjennomførte vi et sosialt eksperiment for å samle kvalitative observasjoner om prosessen med utseendevurdering, konstruere en kvalitativ modell for materialutseende og frembringe relevante forskningshypoteser. Hypotesene er analysert i sammenheng med kunnskapsfronten.
Etterpå testet vi de mest interessante hypotesene kvantitativt, for å vurdere deres muligheter for generalisering. De eksperimentelle resultatene har gitt indikasjoner til støtte for hypotesene. Vi har observert at et objekts gjennomskinnelighet påvirker oppfatningen av glans, mens deteksjon av gjennomskinnelighetsforskjeller avhenger av gjenstandenes geometriske tykkelse og materialene de er laget av sin optiske tetthet. I tillegg har vi undersøkt rollen til flere potensielle perseptuelle indikatorer for gjennomskinnelighet, som kan finnes i bilder som er registrert enten av et kamera eller av en menneskelig observator. Vi har funnet at bildeuskarphet og kaustikk kan påvirke oppfattelsen av gjennomskinnelighet.
Til slutt gjennomførte vi en omfattende undersøkelse om perseptuell gjennomskinnelighet, oppdaterte kunnskapsfronten med våre funn, og skisserte ubesvarte spørsmål for fremtidig forskning
Hyperspectral Imaging of Stained Glass
Get PDFSince its first applications in cultural heritage, hyperspectral imaging (HSI) has become avaluable tool for documenting and analyzing many kinds of artworks, thanks to the possibility of obtaining spectral information regarding relatively large areas in a non-invasive way. In the last decades, HSI has been successfully used to study paintings on panels, canvas and plaster, manuscripts, and photographic materials, and has allowed for successfully characterizing the distribution of pigments and colorants in the artwork under study.
The work presented in this thesis focuses on evaluating the advantages and limitations of performing HSI on stained-glass windows. Compared to the abovementioned types of artworks, prior HSI applications on stained-glass windows are very limited due to the numerous challenges related to the optical properties of the glass and external factors that can negatively impact the quality of the image acquisition. For example, since stained-glass windows are mainly transparent, a setup for spectral transmittance measurements is necessary. If the stainedglass windows are still part of a building and cannot be removed, the intensity of sunlight (used as the light source) can vary throughout the day, and the presence of vegetation or buildings in the background can affect the actual color of the glass. In addition, accessing the stained glass with the instrument could also be an issue if no support is available (e.g., scaffolding).
One of the research project’s main objectives is thus to propose new acquisition methodologies that allow for carrying out HSI of stained glass in different situations, from relatively small, detached panels, to stained-glass windows in situ. The characteristics of each setup will be thoroughly discussed to highlight advantages and limitations.
The second objective is to validate the data obtained from the proposed setups to demonstrate HSI’s capabilities in characterizing the materials used in stained glass. This validation process has been carried out from two perspectives; first, the correctness of the obtained spectra has been verified by comparing them with results from UV-VIS-NIR spectroscopy, which is extensively used for chromophore identification. Second, X-Ray fluorescence spectroscopy (XRF) has been used as a complementary analytical technique to determine the elemental composition of the glass and verify the presence of the chromophore identified by HSI.
Image analysis solutions for automatically identifying and mapping stained glass components were also explored. Besides the traditional classification methodologies, unsupervised unmixing approaches were also investigated, which showed promising results
Multispectral and Hyperspectral Imaging of Art: Quality, Calibration and Visualization
Get PDFMultispectral and hyperspectral imaging can be powerful tools for analyzing and documenting works of art due to their ability to simultaneously capture both accurate spectral and spatial information. The data can be used for a wide range of diagnostic and analytical purposes, including materials identification, pigment mapping, the detection of hidden features or areas of lost material, for colorimetric analysis or for precise quantitative documentation.
However, a number of technical challenges exist which have prevented multispectral and hyperspectral imaging from realizing their full potential and which have prevented the technologies from becoming more widely used and routine analytical tools. Both multispectral and hyperspectral imaging systems require careful and precise acquisition workflows in order to produce useful data. In addition, processing and calibration of the acquired data can be a challenge for many cultural heritage users. Hyperspectral imaging, in particular, can produce vast quantities of raw data that require complex processing and the ability to manage the large resulting volumes of data. Moreover, the final high resolution and multidimensional data that is produced can be difficult to use or to visualize.
This thesis, therefore, seeks to address some of these issues and seeks to analyze and quantify potential problems and then propose tools, workflows and methodologies to resolve and mitigate them. The research presented here focuses on two main areas. The first research area concerns the quality of spectral data and how to measure, quantify and improve it. To do so, it is necessary to first establish exactly what spectral quality is and what methods can be used to quantify it. These methods are then applied to ascertain the levels of quality seen in data acquired under routine operating conditions with an evaluation of data from an extensive round-robin test of hyperspectral imaging systems.
In order to improve the quality of spectral data, the various elements that contribute to and affect spectral quality within a system are then analyzed. Ac quisition and calibration pipelines are then defined for both multispectral and hyperspectral equipment with practical guidelines and workflows provided that aim to help users produce the best quality data possible.
The second research area concerns the visualization of such data and examines ways to facilitate and make large and complex image data accessible online. For this, an architecture, visualization techniques and a full software platform are presented for the efficient distribution and visualization of high resolution multi-modal and multispectral or hyperspectral image data. This work is then extended in order to push the technology to the limits and to apply the techniques to the field of astronomy where image sizes are at their most extreme.Multispektral og hyperspektral avbildning kan være kraftige verktøy for å analysere og dokumentere kunstverk, på grunn av dets evne til samtidig å fange både nøyaktig spektral og romlig informasjon. Dataene kan brukes til et bredt spekter av diagnostiske og analytiske formål, inkludert materialidentifikasjon, pigmentkartlegging, påvisning av skjulte egenskaper eller områder med tapt materiale, for kolorimetrisk analyse eller for presis kvantitativ dokumentasjon.
Imidlertid eksisterer det en rekke tekniske utfordringer som har forhindret multispektral og hyperspektral avbildning fra å utnytte sitt fulle potensiale, og som har forhindret teknologiene fra å bli mer brukt og fra å bli standard analytiske verktøy. Både multispektrale og hyperspektrale bildesystemer krever nøye og presise avbildningsarbeidsflyter for å produsere nyttige data. I tillegg kan behandling og kalibrering av innhentede data være en utfordring for mange brukere i kulturarvsektoren. Spesielt hyperspektral avbildning kan produsere store mengder rådata som krever kompleks behandling og muligheten til å håndtere de store resulterende datamengdene. Videre kan det endelige høyoppløselige og flerdimensjonale datasettet som produseres være vanskelig å bruke eller å visualisere.
Denne avhandlingen søker derfor å adressere noen av disse problemene og søker å analysere og kvantifisere potensielle problemer, og deretter foreslå verktøy, arbeidsflyter og metoder for å redusere og løse dem. Forskningen som presenteres her fokuserer på to hovedområder. Det første forskningsområdet gjelder selve kvaliteten på spektraldataene og hvordan man måler, kvantifiserer og forbedrer denne kvaliteten. For å gjøre dette er det nødvendig å først fastslå nøyaktig hva spektral kvalitet er og hvilke metoder som kan brukes til å kvantifisere den. Disse metodene blir deretter brukt for å fastslå kvalitetsnivåene til i datasett innhentet under rutinemessige driftsforhold, gjennom en evaluering av data fra en omfattende «round-robin test» av hyperspektrale bildesystemer.
For å forbedre kvaliteten på spektraldata analyseres de forskjellige elementene som bidrar til og påvirker spektralkvaliteten i et system. Avbildnings- og kalibreringssarbeidsflyter defineres deretter for både multispektralt og hyperspektralt utstyr, med foreslåtte praktiske retningslinjer og arbeidsflyterfor at brukerne skal produsere data med best mulig kvalitet.
Det andre forskningsområdet gjelder visualisering av slike data og undersøker måter å legge til rette for og gjøre store og komplekse bildedata tilgjengelige online. For dette presenteres en arkitektur, visualiseringsteknikker og en full programvareplattform for effektiv distribusjon og visualisering av høyoppløselige multimodale og multispektrale eller hyperspektrale bildedata. Dette arbeidet er også utvidet for å presse teknologien til det ytterste og til å anvende teknikkene på astronomi-feltet der bildestørrelsene er på sitt mest ekstreme
Spectral filter array cameras as a diagnostic skin imaging tool
Get PDFSkin is the human’s largest organ and has many vital functions including protection from pathogens, temperature regulation, touch sensation, vitamin D synthesis and protecting the water inside the body. It does not only protect the body from the environment but also carries information about the health of individuals.
Diagnosis and monitoring of skin and vital functions measured in non-contact is a broad field of research. Measuring vital signs, monitoring oxygenation and skin diagnosis can benefit from spatially resolved images of the tissue. Standard three channel colour imaging provides the ease of use and acquisition speed for a clinical setup but lacks the spectral resolution to identify specific narrow bands of interest containing the essential information. Spectral imaging has been used to quantify diagnostically relevant physical properties of living tissue but suffers from slow acquisition speed and time delays between the acquisition of different bands. Recent sensor development has led to so-called spectral filter array (SFA) cameras, which combine the acquisition speed and ease of use of standard RGB imaging with the spectral resolution of spectral cameras. To utilise all the benefits of this new imaging modality, additional processing steps are required. This thesis explores SFA imaging in the context of skin diagnosis, and the imaging is enhanced with physical skin simulation models.
Skin models based on Monte Carlo simulation allow change and control over optical properties and resulting spectral reflectance from skin can be recorded. The simulated spectral reflectance with known optical properties is used in three different ways within this research. First, they are tested, by studying the impact of the optical properties on a resulting colour patch. This provides a better understanding of the relationship between colour shade and different combinations of optical properties. Secondly, the simulations are used to enhance the interpretability of spectral measurements regarding important physical skin properties with diagnostic value. This approach is applied to two different spectral filter array cameras and an RGB imager in conjunction with multiple LEDs. Thirdly, skin simulations are performed to generate an exhaustive spectral reflectance database, for training and enhancement of spectral reconstructions of skin reflectance. This specialised database covers a wide range of physically, but not physiologically possible optical properties.
Different spectral imagers in the visual and near-infrared spectrum are applied to measuring oxygenation spatially resolved in living tissue. Additionally, a processing framework is proposed for spectral filter array cameras. This framework combines several SFA camera-specific processing steps and shows transferability to other cameras. It is tested by comparing oxygenation estimations from both a visual range (VIS) and a near-infrared (NIRS) spectral filter array camera with the de facto clinical standard in an upper arm occlusion test.
Finally, this work proposes a framework for selecting and testing SFA cameras for skin diagnosis tasks without the need of (extensive) clinical studies. This framework could aid in the development of SFA cameras for specific tasks and explores currently commercially available models for skin oxygenation measurements. In the future, it can be expected that spectral filter array cameras will become cheaper and more common. This work establishes a solid foundation for applying this new versatile form of spectral imaging in the context of skin diagnosis. Both general practitioners, dermatologists and, anesthesiologists can benefit from an easy to use, spatially resolved, real-time oxygenation measurement tool
Importance of Multi-modal Reflection Data for Predictive Rendering
No full textAccurate modeling of material appearance is crucial for achieving predictive rendering. To improve reflection models, databases with measured reflection behaviors of various material and surface types are of great importance. However, current databases provide limited information, focusing either on the spectral or spatial domain and often lack microgeometry details. To address this limitation, a multi-modal database that includes reflection behavior in both spectral and spatial domains, along with comprehensive microgeometry information, is essential. Additionally, scatter information simulated with microgeometry enhances the understanding of light-matter interactions. This paper discusses the importance of multi-modal reflection data in developing more realistic and computationally efficient reflection models for predictive rendering.Workshop on Material Appearance ModelingJoint MAM - MANER Conference - Material Appearance Network for Education and ResearchRepresentation / Capture / Wave Optic
Neural Texture Block Compression
No full textBlock compression is a widely used technique to compress textures in real-time graphics applications, offering a reduction in storage size. However, their storage efficiency is constrained by the fixed compression ratio, which substantially increases storage size when hundreds of high-quality textures are required. In this paper, we propose a novel block texture compression method with neural networks, Neural Texture Block Compression (NTBC). NTBC learns the mapping from uncompressed textures to block-compressed textures, which allows for significantly reduced storage costs without any change in the shaders. Our experiments show that NTBC can achieve reasonable-quality results with up to about 45% less storage footprint, preserving real-time performance with a modest computational overhead at the texture loading phase in the graphics pipeline.Workshop on Material Appearance ModelingJoint MAM - MANER Conference - Material Appearance Network for Education and ResearchRepresentation / Capture / Wave Optic
Detect manipulated face Images using deep learning tools
Get PDFAnsiktmanipuleringsangrep er en sårbarhet som er tilstede i mange ansiktgjenkjenningssystemer. Et manipulert ansikt kan bli generert ved manipulering av identitetsinformasjonen fra to ansiktsbilder. Dette bildet kan også bli referet til som en morf. Det manipulerte bilde kan så bli utstedt som et identitetsreisedokument, som for eksempel et pass. Dersom myndighetene aksepterer passet, kan dette bli brukt av to forskjellig personer for å komme seg inn i et land. Basert på en forespørsel fra MOBAI AS, vil denne rapporten utforske alle aspektene ved prosessflyten for å avgjøre hvor mulige forbedringer kan utføres. Rapporten vil først starte med å utforske mulighetene ved å trene modellen på morfer laget av en moderne morfealgoritme. De nye morfene laget av denne algoritmen vil så bli brukt til å teste modellens egenskaper til å klassifisere denne dataen riktig. Deretter vil ulike metoder for å hente ut features bli vurdert. Features blir hentet ut og sammenliknet ved bruk av moderne algoritmer. Så vil ulike klassifiseringsalgoritmer bli testet. Her vil bruken av "support vector machines" og nevrale nettverk bli vurdert. Sammenslåing av ulike klassifiseringsarkitekturer blir også vurdert. Til slutt vil alle resultatene bli testet mot en tredje morfealgoritme, som gjøres i et forsøkt på å fastslå modellens ytelse og generaliseringskapabiliteter mot ukjente angrep. Rapporten vil så konkludere ved å fremheve et forslag på en løsning hvor sammenslåing av score på resultatnivå blir utført mellom to klassifiseringsalgoritmer. Her vil ulike ekstraheringsalgoritmer for features være grunnlaget. Dette forslaget reduserer den ekvivalente feilraten på test utført på samme dataset fra 8.08% til 2.72%, og oppnår sammenliknbare resultater på kryss-dataset-testing.Face manipulation attacks are a security vulnerability present across many face recognition systems. One manipulated image can be generated by manipulating the identity information from two face images. This image can also be referred to as a morph. The morphed image can be issued as an identity travel document, such as a passport. If the authority accepts the passport, two persons may use a single ID travel document to enter the country. Based on an inquiry from MOBAI AS, this thesis investigates the possibilities of improving their morphing attack detection pipeline. The thesis explores all aspects of the pipeline to investigate where possible improvements can be made. Firstly, the thesis investigates the possibility of training the model on morphs created using a new state-of-the-art morphing tool. The new morphs test the algorithm's ability to classify unseen data correctly. Afterwards, different methods for extracting features are evaluated. Features are extracted and compared using state-of-the-art face recognition algorithms. Then, different classifier approaches are tested. Here, the utilization of support vector machines and neural networks is evaluated. Fusion between different classifier architectures is also considered. Finally, all results are tested against a third morphing algorithm, to ascertain performance and generalization against unknown attacks. The thesis concludes by presenting a proposed approach for score-level classifier fusion of two models based on different features. The proposed approach reduces the equivalent error rate on the same-dataset test from 8.08% to 2.72% while keeping comparable cross-dataset results
- …
