164 research outputs found
Better Optical Triangulation through Spacetime Analysis
No full textThe standard methods for extracting range data from optical triangulationscanners are accurate only for planar objects of uniform reflectance illuminated by an incoherent source. Using these methods, curved surfaces, discontinuous surfaces, and surfaces of varying reflectance cause systematic distortions of the range data. Coherent light sources such as lasers introduce speckle artifacts that further degrade the data. We present a new ranging method based on analyzing the time evolution of the structured light reflections. Using our spacetime analysis, we can correct for each of these artifacts, thereby attaining significantly higher accuracy using existing technology. We present results that demonstrate the validity of our method using a commercial laser stripe triangulation scanner. Keywords: Active and real-time vision, low level processing, optical triangulation, laser rangefinding, 3-D scanning i Copyright c fl 1995 Brian Curless and Marc Levoy Better Optical Triangulation thr..
Interactive Playspaces for Object Assembly and Digital Storytelling
No full textThesis (Ph.D.)--University of Washington, 2013Today we observe a consistent shift towards doing our tasks virtually through machines. This mode of work ensures that the users are not tied by lack of resources required for the task, and get additional advantages like ability to make quick corrections and share the result remotely. Researchers in the field of human computer interaction have constantly pushed towards tangible user interfaces which allow the users to get a sense of doing the task physically while it happens virtually. Designing interfaces for 3-dimensional tasks poses interesting challenges. The traditional desktop or tabletop setups do not work very well here because it is hard for the user to visualize the 3D virtual world using 2-dimensional displays and control them using un-intuitive devices like keyboard/mouse/joystick etc. Researchers and industry have explored augmented reality-style or immersive environments-based interfaces to let users interact with the virtual world. However, most of these interfaces are too specialized and hard to set up. In this thesis, I explore an easy to set up interactive environment, called a playspace, for a variety of 3D tasks. The user performs the task while a color+depth camera observes and understands the task in real-time. It then presents context-specific feedback and automatically reflects the inferred activity in a virtual world on a screen in front of the user. The playspace also integrates other input modalities such as gestures, voice commands and standard devices like keyboard and mouse. The modular nature of the framework allows different applications to plug into the playspace environment easily. Playspaces allow users to do a task physically while it is virtually replicated on the fly. The virtual result can then be post-processed or edited in real-time, again through physical props. The framework also opens the opportunities to assist users in real-time. I have developed and evaluated three applications in the playspace environment -- 1. Block model assembly - The system automatically learns and builds a virtual replica of a Duplo block model by observing the user build it. It also assists the user in creating a predefined model in a novel way while detecting any mistakes and assisting in making any corrections on the fly. I report on a user study that shows that the proposed guidance method is better than the traditional figure-based guidance method. 2. Digital storytelling - The system allows a user to act out a story using rigid puppets and automatically converts that into an animation. Further, it also allows the user to record multiple takes for the same story and merge them automatically after the user has roughly annotated them based on his liking. This is helpful when the user wants to try out different styles and later merge them. I report on a user study to test this utility. 3. Designing 3D environment prototypes - The system allows the user to easily manipulate virtual objects in a scene by “attaching” them to a physical object of user's choice. The user can add, move, scale, clone or delete objects from a database, thus creating simple 3D virtual environments. The user can also paint the terrain in the virtual world by using textures from his surroundings
Realistically Editing Indoor Scenes
No full textThesis (Ph.D.)--University of Washington, 2021Mixed reality is an exciting application of computer graphics that seamlessly combines the real and the virtual. Many of the most compelling mixed reality applications involve modifying the contents of the surrounding real-world scene, whether it is adding a virtual game character running around your room, replacing your existing couch with a new one in a furniture retail app, or changing the style of your clothing in a fashion app. For most applications, especially those for which the primary goal is aesthetic evaluation, these edits need to be done in a visually realistic way: something that looks like a bad Photoshop job is nowhere near as useful as something that looks like you had physically placed or moved something in your room. In this thesis, I describe methods for reconstructing models of an indoor scene’s lighting and materials in a way that enables realistic, physically consistent edits to the scene. These edits include not only inserting virtual objects into a room, but also removing existing objects, as well as relighting and retexturing. I first outline a method to scan scenes to acquire a 3D mesh textured with a high dynamic range representation of the scene’s appearance. I then show how to use this captured data to infer parametric models of the room’s base geometry (e.g walls, floor, doors, windows). I present an inverse rendering framework to use the high dynamic range data to simultaneously infer light intensity distributions and diffuse reflectances across an entire indoor scene that enables realistic visualizations of emptying, refurnishing, and relighting a room. I subsequently derive a method to solve for not just the intensities but also the number and locations of unobserved local light emitters in a scene. Finally, I describe a neural rendering method to reconcile the differences between a scene’s true appearance and its reconstructed scene model
Methods and Applications for Portrait Selection from Video, Light Field Photography, and Light Field Video Editing
No full textThesis (Ph.D.)--University of Washington, 2022Computational photography applications can assist photographers in capturing images that more closely represent a moment in time, a memory, or an artistic vision, compared to a traditional photograph. At the time of capture, computational cameras can record more information than traditional cameras. In post-processing, computational photography applications can selectively combine or edit down this information. In this thesis, I describe computational photography applications of this type. First, I describe a method to automatically select frames from video of a person that would work well as candid portraits. Next, I describe methods to render light field images from a consumer light field camera. In light field imaging, the scene is simultaneously photographed from multiple angles with a small baseline. The plane of focus and viewing angle can be decided in post-production, and small foreground objects or occluders can be removed. Finally, I describe possible applications of a future light field video camera, based on interviews with film industry professionals
Learning Clinical Body Composition Metrics from 2D and 3D Optical Imaging
No full textThesis (Ph.D.)--University of Washington, 2023Accurate human body shape representation has many applications within computer graphics including 3D animation, virtual tailoring, ergonomic engineering, and virtual reality reconstruction. For clinical researchers working at the intersection of computer graphics, machine learning, and obesity-related epidemiology, computational modeling of human body shape presents a novel and accessible pathway to quantifying, classifying, and monitoring risk factors associated with premature mortality caused by metabolic syndrome. Total and regional body composition and body shape are strongly correlated with progression of metabolic syndrome as well as degenerative conditions such as sarcopenia and osteoporosis. Estimates of body composition from optically measured human body geometry are cheap, safe, and non-invasive relative to current reference methods that require exposure to ionizing radiation.In this thesis, I present a body of work that thoroughly investigates predicting body composition from optical images of human body shape with clinically significant precision and accuracy. This thesis contributes the following:
1. Introduces a model that predicts 3D body shape and total and regional body composition metrics from monocular 2D images.
2. Develops a method that automatically standardizes 3D human body scans to watertight manifold mesh templates with consistent topology and anatomical correspondence and demonstrates the viability of this tool for constructing new shape and regression models that predict body composition with agnosticism towards input scanning devices.
3. Extends the automatic mesh templating method to create the first autoencoded shape model for a pediatric cohort paired with body composition prediction from shape parameters derived with unsupervised learning.
4. Performs a systematic review of deep 3D shape autoencoders for total human body geometry with the goal of identifying the current state of the art methods and architectures in reconstruction accuracy while also suggesting standards and best practices for future work in this research field.
5. Performs a novel estimation of body composition from nonlinear features extracted by a deep autoencoder with nonlinear Gaussian process regression and comprehensively compares marginal contributions of linear and nonlinear shape and regression algorithms against the linear baselines of prior works with systematic ablation studies
Photo-Realistic Scene Modeling and Visualization using Online Photo Collections
No full textThesis (Ph.D.)--University of Washington, 2015Reconstructing 3D scenes from online photo collections has attracted a tremendous amount of interest from both academia and industry. The progress in the past decade has been exceptional, in terms of scale and reconstruction quality. Yet, we are still far from creating 3D models that support consumer level graphics applications. The challenge is twofold. First, modern geometry reconstruction and illumination/reflectance estimation techniques are generating low quality 3D models that are severely contaminated by visual artifacts, i.e., geometry holes, over-inflated boundaries, noisy surface details, low-resolution texture, etc. These artifacts are often extremely noticeable, thus largely limit the applicability of these 3D reconstruction approaches. Second, the real-world is dynamic, but very little research has been devoted to modeling and visualizing transient objects in photos. Therefore, typically we see ghost town 3D models in even the best-of-the-breed work. In this thesis, I first introduce the Visual Turing Test with two of the first relight-able city-scale MVS models. The results show that poor geometry reconstruction and the lack of transient scene elements significantly reduce the photorealism of the rendered images. Our grand vision is that eventually the 3D reconstruction research will be able to pass the Visual Turing Test. While we are still far from that, this dissertation proposes new approaches to photo-realistic scene modeling and visualization. This line of research addresses both of the two aspects of the challenge, i.e., reducing severe artifacts and incorporating some transient objects (people) in renderings, by improving various key components of modern 3D reconstruction pipelines. To be more specific, our work pushes the limit of (1) Structure-from-Motion research by solving the ground-to-aerial geo-registration with pixel level accuracy; (2) Multi-View Stereo by incorporating occluding contour information, and show dramatically improved geometry; (3) lighting/texture estimation by explicitly modeling outdoor illumination, and optimizing for lighting parameters and scene albedo, (4) image-based rendering to improve visualization of a scene with erroneous geometry, and (5) modeling transient objects. This dissertation describes work that can be considered as early effort towards the goal of making 3D reconstruction technologies widely applicable in real-world graphics applications
Manufacturing-Aware Reconstruction
No full textThesis (Ph.D.)--University of Washington, 2024Most of the objects that surround us begin their life as designs. The machines and people involved in the manufacturing process must communicate with a high degree of precision, demanding information-rich designs capable of specifying geometry down to individual mathematically-defined surfaces, as well as other relevant details such as functionality, materials, and assembly instructions. Consequently, designs are a concise representation for the built objects in our environment. In this work, we explore various techniques to reconstruct man-made objects from measured data using geometric priors inherent in these design representations to enhance the accuracy and robustness of our methods.Some key questions we address are: Can we reconstruct designs when our observations of the world are partial? And can we achieve this without significantly sacrificing generality? We will explore methods for using prior knowledge about the manufacturing domain to extract complete carpentry designs from partial visual information. We will see that knowledge of the manufacturing process, even in the absence of the full procedure, allows us to more effectively leverage visual data for reconstruction. We will also present work on reconstructing partially-observed objects in terms of a broader class of geometries associated with computer-aided design (CAD) representations, achieving high precision reconstructions of objects in the wild across many different fabrication domains. Looking beyond geometry, we also present work on augmenting purely geometric designs with functionality by inferring the motions of parts using deep learning. In summation, we demonstrate new techniques to reconstruct various types of designs from partial observations, exploring geometry- and functionality-focused methods, by leveraging manufacturing priors
- …
