1,720,978 research outputs found
Acquisition and Processing of ToF and Stereo data
Full text linkProviding a computer the capability to estimate the three-dimensional geometry of a scene is a fundamental problem in computer vision. A classical systems that has been adopted for solving this problem is the so-called stereo vision system (stereo system). Such a system is constituted by a couple of cameras and it exploits the principle of triangulation in order to provide an estimate of the framed scene. In the last ten years, new devices based on the time-of-flight principle have been proposed in order to solve the same problem, i.e., matricial Time-of-Flight range cameras (ToF cameras).
This thesis focuses on the analysis of the two systems (ToF and stereo cam- eras) from a theoretical and an experimental point of view. ToF cameras are introduced in Chapter 2 and stereo systems in Chapter 3. In particular, for the case of the ToF cameras, a new formal model that describes the acquisition process is derived and presented. In order to understand strengths and weaknesses of such different systems, a comparison methodology is introduced and explained in Chapter 4. From the analysis of ToF cameras and stereo systems it is possible to understand the complementarity of the two systems and it is intuitive to figure that a synergic fusion of their data might provide an improvement in the quality of the measurements preformed by the two devices. In Chapter 5 a method for fusing ToF and stereo data based on a probability approach is presented. In Chapter 6 a method that exploits color and three-dimensional geometry information for solving the classical problem of scene segmentation is explaine
Hand Gesture Recognition for 3D Interfaces
No full textThe recent introduction of many new three dimensional applications and display technologies has created the need for new human-computer interfaces in order to interact with them in a simpler and more natural way compared to what is possible with traditional devices such as the keyboard and mouse. In this work we describe a novel interface for interactive 3D browsing that relies only on the direct acquisition of hand gestures exploiting data from depth cameras such as the Microsoft Kinect. We do not aim at introducing a complete working system for this task, but instead all the various building blocks and available techniques will be presented in order to construct a framework inside which the various components can be fitted
Fusion of Geometry and Color Information for Scene Segmentation
Full text linkScene segmentation is a well-known problem in computer vision traditionally tackled by exploiting only the color information from a single scene view. Recent hardware and software developments allow to estimate in real-time scene geometry and open the way for new scene segmentation approaches based on the fusion of both color and depth data. This paper follows this rationale and proposes a novel segmentation scheme where multidimensional vectors are used to jointly represent color and depth data and normalized cuts spectral clustering is applied to them in order to segment the scene. The critical issue of how to balance the two sources of information is solved by an automatic procedure based on an unsupervised metric for the segmentation quality. An extension of the proposed approach based on the exploitation of both images in stereo vision systems is also proposed. Different acquisition setups, like time-of-flight cameras, the Microsoft Kinect device and stereo vision systems have been used for the experimental validation. A comparison of the effectiveness of the different depth imaging systems for segmentation purposes is also presented. Experimental results show how the proposed algorithm outperforms scene segmentation algorithms based on geometry or color data alone and also other approaches that exploit both clues
A probabilistic approach to ToF and stereo data fusion
No full textCurrent 3D video applications require the availability of depth information, that can be acquired real-time by stereo vision systems and ToF cameras. In this paper, a heterogeneous acquisition system is considered, made of two high resolution standard cameras (stereo pair) and one ToF camera. The stereo system and the ToF camera must be properly calibrated together in order to operate jointly. Therefore this work introduces first a generalized multi-camera calibration technique which does not exploit only the luminance (color) information, but also the depth information extracted by the ToF camera. A probabilistic algorithm is then derived in order to obtain high quality depth information from the information of both the ToF camera and the stereo-pair. Experimental results show that the proposed calibration algorithm leads to a very accurate calibration suitable for the fusion algorithm, that allows for precise extraction of the depth information
Combined use of ToF sensors and standard cameras for 3D video acquisition
No full text3D video applications usually require the availability of
high resolution depth and color information. Depth information
can be acquired at video rate by Time-of-Flight matrix
sensors, but these devices usually have a limited resolution
and image quality. A common solution to this issue is the
combined use of ToF sensors and color cameras. This paper
firstly presents a generalized multi-camera calibration technique
that aims at calibrating together the ToF sensor with
two synchronized cameras exploiting the color information
from both type of sensors but also the depth measures of the
ToF sensor. In the second part of the work we will present an
ad-hoc interpolation technique to obtain high resolution depth
information exploiting side information from the color camera
together with the ToF measures and a novel surface prediction
scheme. Finally we will show how the high resolution
depth map obtained with the proposed approach can be used
in order to warp the available video streams to arbitrary viewpoints
in 3D video applications. Experimental results have
shown how the proposed method allows to obtain a more accurate
calibration and to improve the quality of the depth data
and warped views if compared with standard approaches
Probabilistic ToF and Stereo Data Fusion Based on Mixed Pixel Measurement Models
Full text linkThis paper proposes a method for fusing data acquired by a ToF camera and a stereo pair based on a model for depth measurement by ToF cameras which accounts also for depth discontinuity artifacts due to the mixed pixel effect. Such model is exploited within both a ML and a MAP-MRF frameworks for ToF and stereo data fusion. The proposed MAP-MRF framework is characterized by site-dependent range values, a rather important feature since it can be used both to improve the accuracy and to decrease the computational complexity of standard MAP-MRF approaches. This paper, in order to optimize the site dependent global cost function characteristic of the proposed MAP-MRF approach, also introduces an extension to Loopy Belief Propagation which can be used in other contexts. Experimental data validate the proposed ToF measurements model and the effectiveness of the proposed fusion techniques
A Novel Interpolation Scheme for Range Data with Side Information
Full text linkTime-of-Flight matrix sensors currently available allow for the acquisition of range maps at video rate but usually have a limited resolution. At the same time high resolution color cameras are widely available. This makes highly desirable methods that are able to exploit the combined use of ToF sensors and color cameras to obtain high resolution range maps. This work presents a novel interpolation technique that exploits side information from a standard color camera to increase the resolution of range maps. A segmented version of the high resolution color image is used in order to identify the main objects in the scene, while a novel surface prediction scheme is used to interpolate the available depth samples. Critical issues like the joint calibration of the two devices and the unreliability of the acquired data have also been taken into account with ad- hoc solutions. The performance of the proposed scheme has been verified with both synthetic and real-world data and experimental results have shown how the proposed method allows to obtain a more accurate interpolation with sharper edges if compared with standard approaches
Scene Segmentation Assisted by Stereo Vision
Full text linkStereo vision systems for 3D reconstruction have been deeply studied and are nowadays capable to provide a reasonably accurate estimate of the 3D geometry of a framed scene. They are commonly used to merely extract the 3D structure of the scene. However, a great variety of applications is not interested in the geometry itself, but rather in scene analysis operations, among which scene segmentation is a very important one. Classically, scene segmentation has been tackled by means of color information only, but it turns out to be a badly conditioned image processing operation which remains very challenging. This paper proposes a new framework for scene segmentation where color information is assisted by 3D geometry data, obtained by stereo vision techniques. This approach resembles in some way what happens inside our brain, where the two different views coming from the eyes are used to recognize the various object in the scene and by exploiting a pair of images instead of just one allows to greatly improve the segmentation quality and robustness. Clearly the performance of the approach is dependent on the specific stereo vision algorithm used in order to extract the geometry information. This paper investigates which stereo vision algorithms are best suited to this kind of analysis. Experimental results confirm the effectiveness of the proposed framework and allow to properly rank stereo vision systems on the basis of their performances when applied to the scene segmentation problem
- …
