64 research outputs found
Efficient and Robust Deep Learning for Robotics
No full textL'abstract è presente nell'allegato / the abstract is in the attachmen
MPOSE2021: a Dataset for Short-Time Pose-Based Human Action Recognition
No full textThis repository contains the MPOSE2021 Dataset for short-time pose-based Human Action Recognition (HAR). MPOSE2021 is specifically designed to perform short-time Human Action Recognition.
MPOSE2021 is developed as an evolution of the MPOSE Dataset [1-3]. It is made by human pose data detected by OpenPose [4] and Posenet [11] on popular datasets for HAR, i.e. Weizmann [5], i3DPost [6], IXMAS [7], KTH [8], UTKinetic-Action3D (RGB only) [9] and UTD-MHAD (RGB only) [10], alongside original video datasets, i.e. ISLD and ISLD-Additional-Sequences [1]. Since these datasets have heterogenous action labels, each dataset labels are remapped to a common and homogeneous list of actions. Generated sequences have a number of frames between 20 and 30. Sequences are obtained by cutting the so-called Precursor videos (video from the above-mentioned datasets), with non-overlapping sliding windows. Frames where OpenPose/PoseNet cannot detect any subject are automatically discarded. Resulting samples contain one subject at the time, performing a fraction of a single action. Overall, MPOSE2021 contains 15429 samples, divided into 20 actions, performed by 100 subjects.
More information about the dataset can be found in the MPOSE2021 repository, also providing a user-friendly Python package to import and use the dataset by just running the command
pip install mpose
Data Structure
The repository contains 3 datasets for each pose extractor (namely 1, 2 and 3) which consist of the same data divided in different train/test splits. Each dataset contains X and y numpy arrays for both training and testing. X has the following shape:
(B, T, K, C)
where
B is the batch number;
T (= 30) is the duration of the sequences in frames (zero-padded in the case of shorter sequences);
K (= 17 for PoseNet and 25 for OpenPose) is the number of pose keypoints;
C (= 3) is the number of channels, comprehending 2D keypoint coordinates (x,y) in the original video reference frame and the keypoint confidence (p <= 1)
The .txt files specifying the metadata associated with the split samples are also included.
References
MPOSE2021 is part of a paper published by the Pattern Recognition Journal (Elsevier), and is intended for scientific research purposes. If you want to use MPOSE2021 for your research work, please also cite [1-11].
@article{mazzia2021action,
title={Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition},
author={Mazzia, Vittorio and Angarano, Simone and Salvetti, Francesco and Angelini, Federico and Chiaberge, Marcello},
journal={Pattern Recognition},
pages={108487},
year={2021},
publisher={Elsevier}
}
[1] Angelini, F., Fu, Z., Long, Y., Shao, L., & Naqvi, S. M. (2019). 2D Pose-Based Real-Time Human Action Recognition With Occlusion-Handling. IEEE Transactions on Multimedia, 22(6), 1433-1446.
[2] Angelini, F., Yan, J., & Naqvi, S. M. (2019, May). Privacy-preserving Online Human Behaviour Anomaly Detection Based on Body Movements and Objects Positions. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 8444-8448). IEEE.
[3] Angelini, F., & Naqvi, S. M. (2019, July). Joint RGB-Pose Based Human Action Recognition for Anomaly Detection Applications. In 2019 22th International Conference on Information Fusion (FUSION) (pp. 1-7). IEEE.
[4] Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2019). OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE transactions on pattern analysis and machine intelligence, 43(1), 172-186.
[5] Gorelick, L., Blank, M., Shechtman, E., Irani, M., & Basri, R. (2007). Actions as Space-Time Shapes. IEEE transactions on pattern analysis and machine intelligence, 29(12), 2247-2253.
[6] Starck, J., & Hilton, A. (2007). Surface Capture for Performance-Based Animation. IEEE computer graphics and applications, 27(3), 21-31.
[7] Weinland, D., Özuysal, M., & Fua, P. (2010, September). Making Action Recognition Robust to Occlusions and Viewpoint Changes. In European Conference on Computer Vision (pp. 635-648). Springer, Berlin, Heidelberg.
[8] Schuldt, C., Laptev, I., & Caputo, B. (2004, August). Recognizing Human Actions: a Local SVM Approach. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. (Vol. 3, pp. 32-36). IEEE.
[9] Xia, L., Chen, C. C., & Aggarwal, J. K. (2012, June). View Invariant Human Action Recognition using Histograms of 3D Joints. In 2012 IEEE computer society conference on computer vision and pattern recognition workshops (pp. 20-27). IEEE.
[10] Chen, C., Jafari, R., & Kehtarnavaz, N. (2015, September). UTD-MHAD: A Multimodal Dataset for Human Action Recognition utilizing a Depth Camera and a Wearable Inertial Sensor. In 2015 IEEE International conference on image processing (ICIP) (pp. 168-172). IEEE.
[11] Papandreou, G., Zhu, T., Chen, L. C., Gidaris, S., Tompson, J., & Murphy, K. (2018). Personlab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 269-286)
Position-Agnostic Autonomous Navigation in Vineyards with Deep Reinforcement Learning
No full textPrecision agriculture is rapidly attracting research to efficiently introduce
automation and robotics solutions to support agricultural activities. Robotic
navigation in vineyards and orchards offers competitive advantages in
autonomously monitoring and easily accessing crops for harvesting, spraying and
performing time-consuming necessary tasks. Nowadays, autonomous navigation
algorithms exploit expensive sensors which also require heavy computational
cost for data processing. Nonetheless, vineyard rows represent a challenging
outdoor scenario where GPS and Visual Odometry techniques often struggle to
provide reliable positioning information. In this work, we combine Edge AI with
Deep Reinforcement Learning to propose a cutting-edge lightweight solution to
tackle the problem of autonomous vineyard navigation without exploiting precise
localization data and overcoming task-tailored algorithms with a flexible
learning-based approach. We train an end-to-end sensorimotor agent which
directly maps noisy depth images and position-agnostic robot state information
to velocity commands and guides the robot to the end of a row, continuously
adjusting its heading for a collision-free central trajectory. Our extensive
experimentation in realistic simulated vineyards demonstrates the effectiveness
of our solution and the generalization capabilities of our agent
Waypoint Generation in Row-Based Crops with Deep Learning and Contrastive Clustering
No full textThe development of precision agriculture has gradually introduced automation in the agricultural process to support and rationalize all the activities related to field management. In particular, service robotics plays a predominant role in this evolution by deploying autonomous agents able to navigate in fields while executing different tasks without the need for human intervention, such as monitoring, spraying and harvesting. In this context, global path planning is the first necessary step for every robotic mission and ensures that the navigation is performed efficiently and with complete field coverage. In this paper, we propose a learning-based approach to tackle waypoint generation for planning a navigation path for row-based crops, starting from a top-view map of the region-of-interest. We present a novel methodology for waypoint clustering based on a contrastive loss, able to project the points to a separable latent space. The proposed deep neural network can simultaneously predict the waypoint position and cluster assignment with two specialized heads in a single forward pass. The extensive experimentation on simulated and real-world images demonstrates that the proposed approach effectively solves the waypoint generation problem for both straight and curved row-based crops, overcoming the limitations of previous state-of-the-art methodologies
Generative Adversarial Super-Resolution at the Edge with Knowledge Distillation
No full textSingle-Image Super-Resolution can support robotic tasks in environments where
a reliable visual stream is required to monitor the mission, handle
teleoperation or study relevant visual details. In this work, we propose an
efficient Generative Adversarial Network model for real-time Super-Resolution,
called EdgeSRGAN (code available at https://github.com/PIC4SeR/EdgeSRGAN). We
adopt a tailored architecture of the original SRGAN and model quantization to
boost the execution on CPU and Edge TPU devices, achieving up to 200 fps
inference. We further optimize our model by distilling its knowledge to a
smaller version of the network and obtain remarkable improvements compared to
the standard training approach. Our experiments show that our fast and
lightweight model preserves considerably satisfying image quality compared to
heavier state-of-the-art models. Finally, we conduct experiments on image
transmission with bandwidth degradation to highlight the advantages of the
proposed system for mobile robotic applications
Local Planners with Deep Reinforcement Learning for Indoor Autonomous Navigation
No full textAutonomous indoor navigation requires an elab- orated and accurate algorithmic stack, able to guide robots through cluttered, unstructured, and dynamic environments. Global and local path planning, mapping, localization, and decision making are only some of the required layers that undergo heavy research from the scientific community to achieve the requirements for fully functional autonomous navigation. In the last years, Deep Reinforcement Learning (DRL) has proven to be a competitive short-range guidance system solution for power-efficient and low computational cost point-to-point local planners. One of the main strengths of this approach is the possibility to train a DRL agent in a simulated environment that encapsulates robot dynamics and task constraints and then deploy its learned point-to-point navigation policy in a real setting. However, despite DRL easily integrates complex mechanical dynamics and multimodal signals into a single model, the effect of different sensor data on navigation performance has not been investigated yet. In this paper, we compare two different DRL navigation solutions that leverage LiDAR and depth camera information, respectively. The agents are trained in the same simulated environment and tested on a common benchmark to highlight the strengths and criticalities of each technique
Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition
No full textDeep neural networks based purely on attention have been successful across several domains, relying on minimal architectural priors from the designer. In Human Action Recognition (HAR), attention mechanisms have been primarily adopted on top of standard convolutional or recurrent layers, improving the overall generalization capability. In this work, we introduce Action Transformer (AcT), a simple, fully, self-attentional architecture that consistently outperforms more elaborated networks that mix convolutional, recurrent, and attentive layers. In order to limit computational and energy requests, building on previous human action recognition research, the proposed approach exploits 2D pose representations over small temporal windows, providing a low latency solution for accurate and effective real-time performance. Moreover, we open-source MPOSE2021, a new large-scale dataset, as an attempt to build a formal training and evaluation benchmark for real-time, short-time HAR. The proposed methodology was extensively tested on MPOSE2021 and compared to several state-of-the-art architectures, proving the effectiveness of the AcT model and laying the foundations for future work on HAR
Back-to-Bones: Rediscovering the role of backbones in domain generalization
No full textDomain Generalization (DG) studies the capability of a deep learning model to generalize to out-of-training
distributions. In the last decade, literature has been massively filled with training methodologies that claim
to obtain more abstract and robust data representations to tackle domain shifts. Recent research has provided
a reproducible benchmark for DG, pointing out the effectiveness of naive empirical risk minimization (ERM)
over existing algorithms. Nevertheless, researchers persist in using the same outdated feature extractors, and
little to no attention has been given to the effects of different backbones yet. In this paper, we go ‘‘back to the
backbones’’, proposing a comprehensive analysis of their intrinsic generalization capabilities, which so far have
been overlooked by the research community. We evaluate a wide variety of feature extractors, from standard
residual solutions to transformer-based architectures, finding an evident linear correlation between large-scale
single-domain classification accuracy and DG capability. Our extensive experimentation shows that by adopting
competitive backbones in conjunction with effective data augmentation, plain ERM outperforms recent DG
solutions and achieves state-of-the-art accuracy. Moreover, our additional qualitative studies reveal that novel
backbones give more similar representations to same-class samples, separating different domains in the feature
space. This boost in generalization capabilities leaves marginal room for DG algorithms. It suggests a new
paradigm for investigating the problem, placing backbones in the spotlight and encouraging the development
of consistent algorithms on top of them. The code is available at https://github.com/PIC4SeR/Back-to-Bone
A Deep Learning Driven Algorithmic Pipeline for Autonomous Navigation in Row-Based Crops
No full textExpensive sensors and inefficient algorithmic pipelines significantly affect the overall cost of autonomous machines. However, affordable robotic solutions are essential to practical usage, and their financial impact constitutes a fundamental requirement for employing service robotics in most fields of application. Among all, researchers in the precision agriculture domain strive to devise robust and cost-effective autonomous platforms in order to provide genuinely large-scale competitive solutions. In this article, we present a complete algorithmic pipeline for row-based crops autonomous navigation, specifically designed to cope with low-range sensors and seasonal variations. Firstly, we build on a robust data-driven methodology to generate a viable path for the autonomous machine, covering the full extension of the crop with only the occupancy grid map information of the field. Moreover, our solution leverages on latest advancement of deep learning optimization techniques and synthetic generation of data to provide an affordable solution that efficiently tackles the well-known Global Navigation Satellite System unreliability and degradation due to vegetation growing inside rows. Extensive experimentation and simulations against computer-generated environments and real-world crops demonstrated the robustness and intrinsic generalizability to different factors of variations of our methodology that open the possibility of highly affordable and fully autonomous machines
Robust ultra-wideband range error mitigation with deep learning at the edge
No full textUltra-wideband (UWB) is the state-of-the-art and most popular technology for wireless localization. Nevertheless, precise ranging and localization in non-line-of-sight (NLoS) conditions is still an open research topic. Indeed, multipath effects, reflections, refractions, and complexity of the indoor radio environment can easily introduce a positive bias in the ranging measurement, resulting in highly inaccurate and unsatisfactory position estimation. This article proposes an efficient representation learning methodology that exploits the latest advancement in deep learning and graph optimization techniques to achieve effective ranging error mitigation at the edge. Channel Impulse Response (CIR) signals are directly exploited to extract high semantic features to estimate corrections in either NLoS or LoS conditions. Extensive experimentation with different settings and configurations has proved the effectiveness of our methodology and demonstrated the feasibility of a robust and low computational power UWB range error mitigation
- …
