6,220 research outputs found
Advances in System Identification: Gaussian Regression and Robot Inverse Dynamics Learning
No full textNonparametric Gaussian regression models are powerful tools for supervised learning problems. Recently they have been introduced in the field of system identification as an alternative to classical parametric models used in prediction error methods. The focus of this thesis is the analysis and the extension of linear Gaussian regression models and their applications to the identification of the inverse dynamics of robotic platforms.
When Gaussian processes are applied to linear systems identification, according to the Bayesian paradigm the impulse response is modeled a priori with a Gaussian distribution encoding the desired structural properties of the dynamical system (e.g. smoothness, BIBO stability, sparsity, etc.). The inference on the impulse response estimate is obtained through the posterior distribution which combines the information of the a priori distribution together with the information given by the data.
The Bayesian framework naturally allows the adaptation of the model class and its complexity while also accounting for uncertainty and noise, thus providing a robust mean to trade bias versus variance. On the other hand, one disadvantage of these nonparametric methods is that their aim to identify directly the impulse response of the predictor model does not guarantee the stability of the forward model.
These general advantages and disadvantages inspired the research on this manuscript.
A COMPARISON BETWEEN GAUSSIAN REGRESSION AND PARAMETRIC PEM. The term of comparison for these Gaussian regression models will be the classical parametric technique. In addition to an analysis of the two approaches in terms of error in fitting the impulse response estimates, we are interested in comparing the confidence intervals around these estimates. A new definition of the confidence intervals is proposed in order to pave the way for a fair comparison between the two approaches. Numerical simulations show that the Bayesian estimates have higher prediction performance.
ONLINE GAUSSIAN REGRESSION. In an on-line system identification setting, new data become available at given time steps and real-time estimation requirements have to be satisfied. The goal is to compute the model estimate with low and fixed computational complexity and a reduced memory storage. We developed a tailored Bayesian procedure which updates the quantities to compute the marginal likelihood and the impulse response estimate iteratively and performs the estimation of the hyperparameters by computing only one iteration of a suitable optimization algorithm to maximize the marginal likelihood. Both quasi-Newton methods and EM algorithm are adopted as optimization algorithms. When time-varying systems are considered, the property of ‘forgetting the past data’ is required. Accordingly we propose two schemes: the usage of a temporal window which slides over the data and the usage of a forgetting factor which is a variable that exponentially decreases the weight of the old data. In particular, we propose to consider the forgetting factor both as a fixed constant or as an estimating variable. The proposed nonparametric procedures have satisfactory performances compared to the batch algorithm and outperform the classical parametric approaches both in terms of computational time and adherence of the model estimate to the true one.
ENFORCING MODEL STABILITY IN NONPARAMETRIC GAUSSIAN REGRESSION. The main idea of the Bayesian approach is to frame linear system identification as predictor estimation in an infinite dimensional space with the aid of regularization techniques. This approach is based on the prediction error minimization and can guarantee the identification of stable predictors. Unfortunately, the stability of the predictors does not guarantee the stability of the impulse response of the forward model in general. Various techniques are successfully proposed to guarantee the stability of this model.
ONLINE SEMIPARAMETRIC LEARNING FOR INVERSE DYNAMICS MODELING. Dynamic models can be obtained from the first principles of mechanics, using the so called Rigid Body Dynamics. This approach results in a parametric model in which the values of physically meaningful parameters must be provided in order to complete the fixed structure of the model. Alternatively, the nonparametric Gaussian regression modeling can be employed extrapolating the dynamics directly from the experimental data, without making any unrealistic approximation on the physical system (e.g. assuming linear frictions models, ignoring the dynamics of the hydraulic actuators, etc.). Nevertheless, nonparametric models deteriorate their performance when predicting unseen data that are not in the ``neighbourhood'' of the training dataset. In order to exploit the advantages of both techniques, semiparametric models which combine the parametric and the nonparametric models are analyzed
Online semi-parametric learning for inverse dynamics modeling
No full textThis paper presents a semi-parametric algorithm for online learning of a robot inverse dynamics model. It combines the strength of the parametric and non-parametric modeling. The former exploits the rigid body dynamics equation, while the latter exploits a suitable kernel function. We provide an extensive comparison with other methods from the literature using real data from the iCub humanoid robot. In doing so we also compare two different techniques, namely cross validation and marginal likelihood optimization, for estimating the hyperparameters of the kernel function
Classical vs. Bayesian methods for linear system identification: Point estimators and confidence sets
No full text
A Black-Box Physics-Informed Estimator Based on Gaussian Process Regression for Robot Inverse Dynamics Identification
No full textLearning the inverse dynamics of robots directly from data, adopting a black-box approach, is interesting for several real-world scenarios where limited knowledge about the system is available. In this article, we propose a black-box model based on Gaussian process (GP) regression for the identification of the inverse dynamics of robotic manipulators. The proposed model relies on a novel multidimensional kernel, called Lagrangian Inspired Polynomial (LIP) kernel. The LIP kernel is based on two main ideas. First, instead of directly modeling the inverse dynamics components, we model as GPs the kinetic and potential energy of the system. The GP prior on the inverse dynamics components is derived from those on the energies by applying the properties of GPs under linear operators. Second, as regards the energy prior definition, we prove a polynomial structure of the kinetic and potential energy, and we derive a polynomial kernel that encodes this property. As a consequence, the proposed model allows also to estimate the kinetic and potential energy without requiring any label on these quantities. Results on simulation and on two real robotic manipulators, namely a 7 DOF Franka Emika Panda, and a 6 DOF MELFA RV4FL, show that the proposed model outperforms state-of-the-art black-box estimators based both on Gaussian processes and neural networks in terms of accuracy, generality, and data efficiency. The experiments on the MELFA robot also demonstrate that our approach achieves performance comparable to fine-tuned model-based estimators, despite requiring less prior information. The code of the proposed model is publicly available
Derivative-Free Online Learning of Inverse Dynamics Models
No full textThis paper discusses online algorithms for inverse dynamics modeling in robotics. Several model classes, including rigid body dynamics models, data-driven models and semiparametric models (which are combination of the previous two classes), are placed in a common framework. While model classes used in the literature typically exploit joint velocities and accelerations, which needs to be approximated resorting to numerical differentiation schemes, in this paper, a new 'derivative-free' (DF) framework is proposed, which does not require this preprocessing step. An extensive experimental study with real data from the right arm of the iCub robot is presented, comparing different model classes and estimation procedures, showing that the proposed DF methods outperform existing methodologies
Semiparametrical Gaussian Processes Learning of Forward Dynamical Models for Navigating in a Circular Maze
No full textThis paper presents a problem of model learning for the purpose of learning how to navigate a ball to a goal state in a circular maze environment with two degrees of freedom. The motion of the ball in the maze environment is influenced by several non-linear effects such as dry friction and contacts, which are difficult to model physically. We propose a semiparametric model to estimate the motion dynamics of the ball based on Gaussian Process Regression equipped with basis functions obtained from physics first principles. The accuracy of this semiparametric model is shown not only in estimation but also in prediction at n-steps ahead and its compared with standard algorithms for model learning. The learned model is then used in a trajectory optimization algorithm to compute ball trajectories. We propose the system presented in the paper as a benchmark problem for reinforcement and robot learning, for its interesting and challenging dynamics and its relative ease of reproducibility
Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application
No full textIn this paper, we present a Model-Based Reinforcement Learning (MBRL)
algorithm named \emph{Monte Carlo Probabilistic Inference for Learning COntrol}
(MC-PILCO). The algorithm relies on Gaussian Processes (GPs) to model the
system dynamics and on a Monte Carlo approach to estimate the policy gradient.
This defines a framework in which we ablate the choice of the following
components: (i) the selection of the cost function, (ii) the optimization of
policies using dropout, (iii) an improved data efficiency through the use of
structured kernels in the GP models. The combination of the aforementioned
aspects affects dramatically the performance of MC-PILCO. Numerical comparisons
in a simulated cart-pole environment show that MC-PILCO exhibits better data
efficiency and control performance w.r.t. state-of-the-art GP-based MBRL
algorithms. Finally, we apply MC-PILCO to real systems, considering in
particular systems with partially measurable states. We discuss the importance
of modeling both the measurement system and the state estimators during policy
optimization. The effectiveness of the proposed solutions has been tested in
simulation and on two real systems, a Furuta pendulum and a ball-and-plate rig.Comment: Accepted in IEEE Transactions on Robotics. MC-PILCO code is publicly
available at https://www.merl.com/research/license/MC-PILC
- …
