300 research outputs found
Towards Feature Selection In Actor-Critic Algorithms
No full textChoosing features for the critic in actor-critic algorithms with function approximation is known to be a challenge. Too few critic features can lead to degeneracy of the actor gradient, and too many features may lead to slower convergence of the learner. In this paper, we show that a well-studied class of actor policies satisfy the known requirements for convergence when the actor features are selected carefully. We demonstrate that two popular representations for value methods - the barycentric interpolators and the graph Laplacian proto-value functions - can be used to represent the actor in order to satisfy these conditions. A consequence of this work is a generalization of the proto-value function methods to the continuous action actor-critic domain. Finally, we analyze the performance of this approach using a simulation of a torque-limited inverted pendulum
A hypothesis-based algorithm for planning and control in non-Gaussian belief spaces
No full textWe consider the partially observable control problem where it is potentially necessary to perform complex information-gathering operations in order to localize state. One approach to solving these problems is to create plans in belief-space, the space of probability distributions over the underlying state of the system. The belief-space plan encodes a strategy for performing a task while gaining information as necessary. Most approaches to belief-space planning rely upon representing belief state in a particular way (typically as a Gaussian). Unfortunately, this can lead to large errors between the assumed density representation and the true belief state. We propose a new computationally efficient algorithm for planning in non-Gaussian belief spaces. We propose a receding horizon re-planning approach where planning occurs in a low-dimensional sampled representation of belief state while the true belief state of the system is monitored using an arbitrary accurate high-dimensional representation. Our key contribution is a planning problem that, when solved optimally on each re-planning step, is guaranteed, under certain conditions, to enable the system to gain information. We prove that when these conditions are met, the algorithm converges with probability one. We characterize algorithm performance for different parameter settings in simulation and report results from a robot experiment that illustrates the application of the algorithm to robot grasping
Supplemental Material - Certified polyhedral decompositions of collision-free configuration space
No full textSupplemental Material for Certified polyhedral decompositions of collision-free configuration space by Hongkai Dai, Alexandre Amice, Peter Werner, Annan Zhang, and Russ Tedrake in The International Journal of Robotics Research</p
Mixed-Integer Convex Formulations for Planning Nonlinear Dynamics in Complex Environments
No full textPresented on October 19, 2016 from 12:00 p.m.-1:00 p.m. in the Marcus Nanotechnology Building, Rooms 1116-1118 on the Georgia Tech campus.Russ Tedrake is a professor of Electrical Engineering and Computer Science, Aeronautics and
Astronautics, and Mechanical Engineering at MIT; the director of the Center for Robotics at the Computer
Science and Artificial Intelligence Lab; and the leader of Team MIT’s entry in the DARPA Robotics
Challenge. Additionally, Tedrake is the director of Simulation and Control at the new Toyota Research
Institute. He is a recipient of the NSF CAREER Award, the MIT Jerome Saltzer Award for undergraduate
teaching, the DARPA Young Faculty Award in Mathematics, the 2012 Ruth and Joel Spira Teaching
Award, and was honored as a Microsoft Research New Faculty Fellow. Tedrake received his B.S.E. in Computer Engineering from the University of Michigan, Ann Arbor, in
1999, and his Ph.D. in Electrical Engineering and Computer Science from MIT in 2004, working with
Sebastian Seung. After graduation, he joined the MIT Brain and Cognitive Sciences Department as a
postdoctoral associate. During his education, he also spent time at Microsoft, Microsoft Research, and the
Santa Fe Institute.Runtime: 53:37 minutesHumanoid robots walking across intermittent terrain, robotic arms grasping multifaceted objects, or UAVs
darting left or right around a tree — many of the dynamics and control problems we face today have an
inherently combinatorial structure. In this talk, I’ll review some recent work on planning and control
methods that address this combinatorial structure without sacrificing the rich underlying nonlinear
dynamics. I’ll present some details of our explorations with mixed-integer convex- and SDP-relaxations
applied to hard problems in legged locomotion over rough terrain, grasp optimization, and UAVs flying
through highly cluttered environments
Approximate hybrid model predictive control for multi-contact push recovery in complex environments
No full textFeedback control of robotic systems interacting with the environment through contacts is a central topic in legged robotics. One of the main challenges posed by this problem is the choice of a model sufficiently complex to capture the discontinuous nature of the dynamics but simple enough to allow online computations. Linear models have proved to be the most effective and reliable choice for smooth systems; we believe that piecewise affine (PWA) models represent their natural extension when contact phenomena occur. Discrete-time PWA systems have been deeply analyzed in the field of hybrid Model Predictive Control (MPC), but the straightforward application of MPC techniques to complex systems, such as a humanoid
robot, leads to mixed-integer optimization problems which are not solvable at real-time rates. Explicit MPC methods can
construct the entire control policy offline, but the resulting policy becomes too complex to compute for systems at the scale of a humanoid robot. In this paper we propose a novel algorithm which splits the computational burden between an offline sampling phase and a limited number of online convex optimizations, enabling the application of hybrid predictive controllers to higher-dimensional systems. In doing so we are willing to partially sacrifice feedback optimality, but we set stability of the system as an inviolable requirement. Simulation results of a simple planar humanoid that balances by making contact with its environment are presented to validate the proposed controller
Simulation-based LQR-trees with input and state constraints
No full textWe present an algorithm that probabilistically covers a bounded region of the state space of a nonlinear system with a sparse tree of feedback stabilized trajectories leading to a goal state. The generated tree serves as a lookup table control policy to get any reachable initial condition within that region to the goal. The approach combines motion planning with reasoning about the set of states around a trajectory for which the feedback policy of the trajectory is able to stabilize the system. The key idea is to use a random sample from the bounded region for both motion planning and approximation of the stabilizable sets by falsification; this keeps the number of samples and simulations needed to generate covering policies reasonably low. We simulate the nonlinear system to falsify the stabilizable sets, which allows enforcing input and state constraints. Compared to the algebraic verification using sums of squares optimization in our previous work, the simulation-based approximation of the stabilizable set is less exact, but considerably easier to implement and can be applied to a broader range of nonlinear systems. We show simulation results obtained with model systems and study the performance and robustness of the generated policies
Minimalistic Control of a Compass Gait Robot in Rough Terrain
No full textAlthough there has been an increasing interest in dynamic bipedal locomotion for significant improvement of energy efficiency and dexterity of mobile robots in the real world, their locomotion capabilities are still mostly restricted on flat surfaces. The difficulty of dynamic locomotion in rough terrain is mainly originated in the stability and controllability of gait patterns while exploiting the natural mechanical dynamics of the robots. For a systematic investigation of the challenging problem, this paper presents the simplest control architecture for the compass gait model which can be used for locomotion in rough terrain. Locomotion of the model is mainly achieved by an open-loop oscillator which induces self-stabilizing gait patterns, and we test the proposed control architecture in a real-world robotic platform. In addition, we also found that this controller is capable of varying stride length with a minimum change of control parameters, which enables locomotion in rough terrains. By using these basic principles of self-stability and gait variability, we extended the proposed controller with a simple sensory feedback about the location in the environment, which makes the robot possible to control gait patterns autonomously for traversing a rough terrain. We describe a set of experimental results and discuss how the proposed minimalistic control architecture can be enhanced for dynamic locomotion control in more complex environment.National Science Foundation (U.S.) (Grant No. 0746194)Swiss National Science Foundation (Grant No. PBZH2-114461
A Quadratic Regulator-Based Heuristic for Rapidly Exploring State Space
No full textKinodynamic planning algorithms like Rapidly-Exploring Randomized Trees (RRTs) hold the promise of finding feasible trajectories for rich dynamical systems with complex, nonconvex constraints. In practice, these algorithms perform very well on configuration space planning, but struggle to grow efficiently in systems with dynamics or differential constraints. This is due in part to the fact that the conventional distance metric, Euclidean distance, does not take into account system dynamics and constraints when identifying which node in the existing tree is capable of producing children closest to a given point in state space. We show that an affine quadratic regulator (AQR) design can be used to approximate the exact minimum-time distance pseudometric at a reasonable computational cost. We demonstrate improved exploration of the state spaces of the double integrator and simple pendulum when using this pseudometric within the RRT framework, but this improvement drops off as systems' nonlinearity and complexity increase. Future work includes exploring methods for approximating the exact minimum-time distance pseudometric that can reason about dynamics with higher-order terms
L[subscript 2]-gain optimization for robust bipedal walking on unknown terrain
No full textIn this paper we seek to quantify and explicitly optimize the robustness of a control system for a robot walking on terrain with uncertain geometry. Geometric perturbations to the terrain enter the equations of motion through a relocation of the hybrid event “guards” which trigger an impact event; these perturbations can have a large effect on the stability of the robot and do not fit into the traditional robust control analysis and design methodologies without additional machinery. We attempt to provide that machinery here. In particular, we quantify the robustness of the system to terrain perturbations by defining an L[subscript 2] gain from terrain perturbations to deviations from the nominal limit cycle. We show that the solution to a periodic dissipation inequality provides a sufficient upper bound on this gain for a linear approximation of the dynamics around the limit cycle, and we formulate a semidefinite programming problem to compute the L[subscript 2] gain for the system with a fixed linear controller. We then use either binary search or an iterative optimization method to construct a linear robust controller and to minimize the L[subscript 2] gain. The simulation results on canonical robots suggest that the L[subscript 2] gain is closely correlated to the actual number of steps traversed on the rough terrain, and our controller can improve the robot's robustness to terrain disturbances.National Science Foundation (U.S.) (Contract CNS-0960061)United States. Defense Advanced Research Projects Agency. Maximum Mobility and Manipulation Program (BAA-10-65-M3-FP-024
Metastable Walking Machines
No full textLegged robots that operate in the real world are inherently subject to stochasticity in their dynamics and uncertainty about the terrain. Owing to limited energy budgets and limited control authority, these “disturbances” cannot always be canceled out with high-gain feedback. Minimally actuated walking machines subject to stochastic disturbances no longer satisfy strict conditions for limit-cycle stability; however, they can still demonstrate impressively long-living periods of continuous walking. Here, we employ tools from stochastic processes to examine the “stochastic stability” of idealized rimless-wheel and compass-gait walking on randomly generated uneven terrain. Furthermore, we employ tools from numerical stochastic optimal control to design a controller for an actuated compass gait model which maximizes a measure of stochastic stability—the mean first-passage time—and compare its performance with a deterministic counterpart. Our results demonstrate that walking is well characterized as a metastable process, and that the stochastic dynamics of walking should be accounted for during control design in order to improve the stability of our machines
- …
