1,720,989 research outputs found
Exploring Upper Limb Segmentation with Deep Learning for Augmented Virtuality
No full textSense of presence, immersion, and body ownership are among the main challenges concerning Virtual Reality (VR) and freehand-based interaction methods. Through specific hand tracking devices, freehand-based methods can allow users to use their hands for VE interaction. To visualize and make easy the freehand methods, recent approaches take advantage of 3D meshes to represent the user's hands in VE. However, this can reduce user immersion due to their unnatural correspondence with the real hands. We propose an augmented virtuality (AV) pipeline allows users to visualize their limbs in VE to overcome this limit. In particular, they were captured by a single monocular RGB camera placed in an egocentric perspective, segmented using a deep convolutional neural network (CNN), and streamed in the VE. In addition, hands were tracked through a Leap Motion controller to allow user interaction. We introduced two case studies as a preliminary investigation for this approach. Finally, both quantitative and qualitative evaluations of the CNN results were provided and highlighted the effectiveness of the proposed CNN achieving remarkable results in several real-life unconstrained scenarios
Human segmentation in surveillance video with deep learning
Full text linkAdvanced intelligent surveillance systems are able to automatically analyze video of surveillance data without human intervention. These systems allow high accuracy of human activity recognition and then a high-level activity evaluation. To provide such features, an intelligent surveillance system requires a background subtraction scheme for human segmentation that captures a sequence of images containing moving humans from the reference background image. This paper proposes an alternative approach for human segmentation in videos through the use of a deep convolutional neural network. Two specific datasets were created to train our network, using the shapes of 35 different moving actors arranged on background images related to the area where the camera is located, allowing the network to take advantage of the entire site chosen for video surveillance. To assess the proposed approach, we compare our results with an Adobe Photoshop tool called Select Subject, the conditional generative adversarial network Pix2Pix, and the fully-convolutional model for real-time instance segmentation Yolact. The results show that the main benefit of our method is the possibility to automatically recognize and segment people in videos without constraints on camera and people movements in the scene (Video, code and datasets are available at http://graphics.unibas.it/www/HumanSegmentation/index.md.html)
Egocentric upper limb segmentation in unconstrained real-life scenarios
No full textThe segmentation of bare and clothed upper limbs in unconstrained real-life environments has been less explored. It is a challenging task that we tackled by training a deep neural network based on the DeepLabv3+ architecture. We collected about 46 thousand real-life and carefully labeled RGB egocentric images with a great variety of skin tones, clothes, occlusions, and lighting conditions. We then widely evaluated the proposed approach and compared it with state-of-the-art methods for hand and arm segmentation, e.g., Ego2Hands, EgoArm, and HGRNet. We used our test set and a subset of the EgoGesture dataset (EgoGestureSeg) to assess the model generalization level on challenging scenarios. Moreover, we tested our network on hand-only segmentation since it is a closely related task. We made a quantitative analysis through standard metrics for image segmentation and a qualitative evaluation by visually comparing the obtained predictions. Our approach outperforms all comparing models in both tasks and proving the robustness of the proposed approach to hand-to-hand and hand-to-object occlusions, dynamic user/camera movements, different lighting conditions, skin colors, clothes, and limb/hand poses
A Low-Cost Full Body Tracking System in Virtual Reality Based on Microsoft Kinect
No full textWe present an approach based on a natural user interface and virtual reality that allows the user’s body to be visualized and tracked inside a virtual environment. Our aim is to improve the sensation of virtual reality immersion through low-cost technology such as HTC Vive and Microsoft Kinect 2. The system has been developed using the Unity 3D game engine and C# language. Our approach has been validated through the implementation of an application for 3D mesh painting where the user is able to interact through hand gestures to select a color from the 3D color palette, rotate the 3D mesh and paint it
StreamflowVL: A virtual fieldwork laboratory that supports traditional hydraulics engineering learning
Full text linkThis paper describes an innovative virtual laboratory for students of Hydraulic Engineering at an Italian university that shows water discharge measurement techniques applied in open-channel flows. Such new technology, which supports traditional practical classes, has the potential to increase students' motivation and improve their skills, as well as simultaneously reducing the costs, time, and possible dangers that continuous field experiments would involve. Thanks to this immersive and interactive experience that is carried out indoors, students learn to move around a fluvial environment, as well as work more safely and with reduced risks of accidents. Besides, the virtual lab can boost learners' interest by combining education with pleasure and making knowledge more fun. Collaboration with a group of students enrolled in the Master's degree course of the Civil and Environmental Engineering program at Basilicata University at the early stages of developing the educational tool led to improvements in its performance and features. Also, a preliminary testing procedure carried out on a student sample, verified the achievement of the students' learning objectives in terms of knowledge and skills. Such analysis indicated that students took more active role in the teaching/learning process and they showed greater interest in the topic dealt with through the new technology compared to the involvement of students observed during traditional lessons in previous years. The architecture and operational modes of the virtual laboratory as well as the results of the preliminary analysis are discussed
Sustainable water management: Virtual reality training for open-channel flow monitoring
Full text linkThe estimated population growth in the next decades will create severe scarcity of water and will have a tremendous impact on the natural environment. Both the developed and developing countries will have to face increasing challenges to match the greater demand of clean and safe water, looking for supplies far from the residential area. This situation will be furtherly exasperated by the effects of climate change which, increasing the frequency and intensity of extreme events, will reduce the availability and the quality of water resources and will subject the population to serious and ongoing hazards. In such context, an accurate and continuous monitoring of surface waters represents a fundamental step to reduce the contamination status and plan actions for a sustainable management of this resource. In the last years, the development of advanced methodologies and high-tech equipment able to lower the times and costs of the field surveys has not been associated with an appropriate training of the technical staff of public and private bodies responsible for the control of the territory. In most cases, unable to outsource highly qualified personnel due to lack of funding, such bodies tend to reduce the monitoring activities, leaving the areas even more subject to the risk of disastrous events. The present paper proposes an innovative educational tool based on the virtual reality in support to technical and non-technical workforces in field activities. The tool represents a Virtual Laboratory able to train on the standard techniques for the accurate monitoring of the water discharge in open-channel flows and was successfully tested on a sample of people from the private and public water sector. According to the results, its use increased the fieldworkers' ability to quickly move within the river as well as to easily and correctly manage the measurement equipment and methodology, so reducing the costs and times of surveys in situ
Solid and Effective Upper Limb Segmentation in Egocentric Vision
Full text linkUpper limb segmentation in egocentric vision is a challenging and nearly unexplored task that extends the well-known hand localization problem and can be crucial for a realistic representation of users' limbs in immersive and interactive environments, such as VR/MR applications designed for web browsers that are a general-purpose solution suitable for any device. Existing hand and arm segmentation approaches require a large amount of well-annotated data. Then different annotation techniques were designed, and several datasets were created. Such datasets are often limited to synthetic and semi-synthetic data that do not include the whole limb and differ significantly from real data, leading to poor performance in many realistic cases. To overcome the limitations of previous methods and the challenges inherent in both egocentric vision and segmentation, we trained several segmentation networks based on the state-of-the-art DeepLabv3+ model, collecting a large-scale comprehensive dataset. It consists of 46 thousand real-life and well-labeled RGB images with a great variety of skin colors, clothes, occlusions, and lighting conditions. In particular, we carefully selected the best data from existing datasets and added our EgoCam dataset, which includes new images with accurate labels. Finally, we extensively evaluated the trained networks in unconstrained real-world environments to find the best model configuration for this task, achieving promising and remarkable results in diverse scenarios. The code, the collected egocentric upper limb segmentation dataset, and a video demo of our work will be available on the project page1
AvatarizeMe: A Fast Software Tool for Transforming Selfies into Animatable Lifelike Avatars Using Machine Learning
No full textCreating realistic avatars that faithfully replicate facial features from single-input images is a challenging task in computer graphics, virtual communication, and interactive entertainment. These avatars have the potential to revolutionize virtual experiences by enhancing user engagement and personalization. However, existing methods, such as 3D facial capture systems, are costly and complex. Our approach adopts the 3D Morphable Face Model (3DMM) method to create avatars with remarkably realistic features in a bunch of seconds, using only a single input image. Our method extends beyond facial shape resemblance; it meticulously generates both facial and bodily textures, enhancing overall likeness. Within Unreal Engine 5, our avatars come to life with real-time body and facial animations. This is made possible through a versatile skeleton for body and head movements and a suite of 52 face blendshapes, enabling the avatar to convey emotions and expressions with fidelity. This poster presents our approach, bridging the gap between reality and virtual representation, and opening doors to immersive virtual experiences with lifelike avatars
StreamFlowVR: A Tool for Learning Methodologies and Measurement Instruments for River Flow Through Virtual Reality
No full textVirtual Reality potentialities can be exploited for several types of contexts beyond the mainstream video game industry. One of the most interesting application fields concerns its use in education and teaching especially in the contexts where security and safety are essential elements. The measurement of river flow is required for river management purposes including water resources planning, pollution prevention, and, flood control. The teaching of this complex task requires to transfer well-known methodologies as well as careful attention while performing the training in situ. In this paper, we propose a virtual reality tool called StreamFlowVR to improve the learning process for the river flow instruments and measuring methodologies. The basic idea is to create a rich user experience in a secure environment where students can understand the correct methodologies. We believe that the use of virtual reality benefits the students in learning hydraulic phenomenon faster and more confident respect to the traditional approach on a real river
Visualising a software system as a city through virtual reality
No full textWe describe a technique developed using C++ language and Unreal Engine 4 that allows users to visualise software systems written in object-oriented Java through virtual reality and using the city metaphor. Our aim is to use virtual reality to visualise the metrics of classes and packages of a software system. In this paper, we present a prototype. The ultimate goal will be to demonstrate that it is possible to use virtual reality to better understand software
- …
