497 research outputs found
Robust SLAM and motion segmentation under long-term dynamic large occlusions
Full text linkVisual sensors are key to robot perception, which can not only help robot localisation but also enable robots to interact with the environment. However, in new environments, robots can fail to distinguish the static and dynamic components in the visual input. Consequently, robots are unable to track objects or localise themselves. Methods often require precise robot proprioception to compensate for camera movement and separate the static background from the visual input. However, robot proprioception, such as \ac{IMU} or wheel odometry, usually faces the problem of drift accumulation. The state-of-the-art methods demonstrate promising performance but either (1) require semantic segmentation, which is inaccessible in unknown environments, or (2) treat dynamic components as outliers -- which is unfeasible when dynamic objects occupy a large proportion of the visual input.
This research work systematically unifies camera and multi-object tracking problems in indoor environments by proposing a multi-motion tracking system; and enables robots to differentiate the static and dynamic components in the visual input with the understanding of their own movements and actions. Detailed evaluation of both simulation environments and robotic platforms suggests that the proposed method outperforms the state-of-the-art dynamic SLAM methods when the majority of the camera view is occluded by multiple unmodeled objects over a long period of time
Stochastic optimal control with learned dynamics models
Full text linkThe motor control of anthropomorphic robotic systems is a challenging computational
task mainly because of the high levels of redundancies such systems exhibit. Optimality
principles provide a general strategy to resolve such redundancies in a task driven
fashion. In particular closed loop optimisation, i.e., optimal feedback control (OFC),
has served as a successful motor control model as it unifies important concepts such
as costs, noise, sensory feedback and internal models into a coherent mathematical
framework.
Realising OFC on realistic anthropomorphic systems however is non-trivial: Firstly,
such systems have typically large dimensionality and nonlinear dynamics, in which
case the optimisation problem becomes computationally intractable. Approximative
methods, like the iterative linear quadratic gaussian (ILQG), have been proposed to
avoid this, however the transfer of solutions from idealised simulations to real hardware
systems has proved to be challenging. Secondly, OFC relies on an accurate description
of the system dynamics, which for many realistic control systems may be unknown,
difficult to estimate, or subject to frequent systematic changes. Thirdly, many (especially
biologically inspired) systems suffer from significant state or control dependent
sources of noise, which are difficult to model in a generally valid fashion. This thesis
addresses these issues with the aim to realise efficient OFC for anthropomorphic
manipulators.
First we investigate the implementation of OFC laws on anthropomorphic hardware.
Using ILQG we optimally control a high-dimensional anthropomorphic manipulator
without having to specify an explicit inverse kinematics, inverse dynamics
or feedback control law. We achieve this by introducing a novel cost function that
accounts for the physical constraints of the robot and a dynamics formulation that resolves
discontinuities in the dynamics. The experimental hardware results reveal the
benefits of OFC over traditional (open loop) optimal controllers in terms of energy
efficiency and compliance, properties that are crucial for the control of modern anthropomorphic
manipulators.
We then propose a new framework of OFC with learned dynamics (OFC-LD) that,
unlike classic approaches, does not rely on analytic dynamics functions but rather updates
the internal dynamics model continuously from sensorimotor plant feedback. We
demonstrate how this approach can compensate for unknown dynamics and for complex
dynamic perturbations in an online fashion.
A specific advantage of a learned dynamics model is that it contains the stochastic
information (i.e., noise) from the plant data, which corresponds to the uncertainty in
the system. Consequently one can exploit this information within OFC-LD in order
to produce control laws that minimise the uncertainty in the system. In the domain of
antagonistically actuated systems this approach leads to improved motor performance,
which is achieved by co-contracting antagonistic actuators in order to reduce the negative
effects of the noise. Most importantly the shape and source of the noise is unknown
a priory and is solely learned from plant data. The model is successfully tested on an
antagonistic series elastic actuator (SEA) that we have built for this purpose.
The proposed OFC-LD model is not only applicable to robotic systems but also
proves to be very useful in the modelling of biological motor control phenomena and
we show how our model can be used to predict a wide range of human impedance
control patterns during both, stationary and adaptation tasks
Presence AI Version 2.0 – Unified Consciousness Simulation Framework (An Extension of TCSS + Ψ(U) Framework by Sethu Krishnan, 2025)
No full textPresence AI Version 2.0 expands the mathematical framework for conscious presence simulation in AI systems. It introduces multi-dimensional awareness, tracking memory, physiology, empathy, and ego-reactivity. This paper outlines the new formula (Ψₘ(U)) with its components, interprets scoring under extreme scenarios, and presents a roadmap for future versions leading up to a simulated soul in AI.Use of AI Assistance:
This document was developed using ChatGPT by OpenAI as a tool for organizing, formatting, and expanding upon the core concepts of the author’s original theory — Ego Safe Selection and Consciousness Mapping (TCSS + Ψ(U)).
All key ideas, equations, scoring logic, and the foundational consciousness model originated from the author. ChatGPT was used to assist with:
Structuring the Presence AI Version 2.0 framework
Generating formal mathematical expressions
Simulating test scenarios
Drafting clear, consistent academic language
Designing the version roadmap (v1 → v12)
Refining analogies and scoring interpretations
The AI did not originate any independent theory or claim authorship. All intellectual ownership remains with the author, Sethu Krishnan
Novel approach for representing, generalising, and quantifying periodic gaits
Full text linkOur goal is to introduce a novel method for representing, generalising, and comparing
gaits; particularly, walking gait. Human walking gaits are a result of complex, interdependent
factors that include variations resulting from embodiments, environment and
tasks, making techniques that use average template frameworks suboptimal for systematic
analysis or corrective interventions. The proposed work aims to devise methodologies
for being able to represent gaits and gait transitions such that optimal policies that
eliminate the inter-personal variations from tasks and embodiment may be recovered.
Our approach is built upon (i) work in the domain of null-space policy recovery and
(ii) previous work in generalisation for point-to-point movements. The problem is formalised
using a walking phase model, and the null-space learning method is used to
generalise a consistent policy from multiple observations with rich variations. Once
recovered, the underlying policies (mapped to different gait phases) can serve as reference
guideline to quantify and identify pathological gaits while being robust against
interpersonal and task variations.
To validate our methods, we have demonstrated robustness of our method with simulated
sagittal 2-link gait data with multiple ground truth constraints and policies. Pathological
gait identification was then tested on real-world human gait data with induced
gait abnormality, with the proposed method showing significant robustness to variations
in speed and embodiment compared to template based methods. Future work will
extend this to kinetic features and higher degree-of-freedom
Online receding horizon planning of multi-contact locomotion
Full text linkLegged robots can traverse uneven terrain by using multiple contacts between their limbs and the environment. Nevertheless, to enable reliable operation in the real world, legged robots necessarily require the capability to online re-plan their motions in response to changing conditions, such as environment changes, or state deviations due to external force perturbations. To approach this goal, Receding Horizon Planning (RHP) can be a promising solution. RHP refers to the planning framework that can constantly update the motion plan for immediate execution. To achieve successful RHP, we typically need to consider an extended planning horizon, which consists of an execution horizon that plans the motion to be executed, and a prediction horizon that foresees the future. Although the prediction horizon is never executed, it is important to the success of RHP. This is because the prediction horizon serves as a value function approximation that evaluates the feasibility and the future effort required for accomplishing the given task starting from a chosen robot state. Having such value information can guide the execution horizon toward the states that are beneficial for the future.
Nevertheless, computing such multi-contact motions for a legged robot to traverse uneven terrain can be time-consuming, especially when considering a long planning horizon. The computation complexity typically comes from the simultaneous resolution of the following two sub-problems: 1) selecting a gait pattern that specifies the sequence in which the limbs break and make contact with the environment; 2)synthesizing the contact and motion plan that determines the robot state trajectory along with the contact plan, i.e., contact locations and contact timings. The issue of gait pattern selection introduces combinatorial complexity into the planning problem,while the computation of the contact and motion plan brings high-dimensionality and non-convexity due to the consideration of complex non-linear dynamics constraints.
To facilitate online RHP of multi-contact motions, in this thesis, we focus on exploring novel methods to address these two sub-problems efficiently. To give more detail, we firstly consider the problem of planning contact and motion plans in an online receding horizon fashion. In this case, we pre-specifying the gait pattern as a priori. Although this helps us to avoid the combinatorial complexity, the resulting planning problem is still high-dimensional and non-convex, which can hinder online computation. To improve the computation speed, we propose to simplify the modeling of the value function approximation that is required for guiding the RHP. This leads to 1) Receding Horizon Planning with Multiple Levels of Model Fidelity, where we compute the prediction horizon with a convex relaxed model; 2) Locally- Guided Receding Horizon Planning—where we propose to learn an oracle to predict local objectives (intermediate goals) for completing a given task, and then we use these local objectives to construct local value functions to guide a short-horizon RHP. We evaluate our methods for planning centroidal trajectories of a humanoid robot walking on moderate slopes as well as large slopes where static stability cannot be maintained.The result of multi-fidelity RHP demonstrates that we can accelerate the computation speed by relaxing the model accuracy in the prediction horizon. However, the relaxation cannot be arbitrary. Furthermore, owing to the shortened planning horizon, we find that locally-guided RHP demonstrates the best computation efficiency (95%-98.6%cycles converge online). This computation advantage enables us to demonstrate online RHP for our real-world humanoid robot Talos walking in dynamic environments that change on-the-fly.
To handle the combinatorial complexity that arises from the gait pattern selection issue, we propose the idea of constructing a map from the task specifications to the gait pattern selections for a given environment model and performance objective(cost). We show that for a 2D half-cheetah model and a quadruped robot, a direct mapping between a given task and an optimal gait pattern can be established. We use supervised learning to capture the structure of this map in the form of gait regions.Furthermore, we also find that the trajectories in each gait region are qualitatively similar. We utilize this property to construct a warm-starting trajectory for each gait region, i.e., the mean of the trajectories discovered in each region. We empirically show that these warm-starting trajectories can improve the computation speed of our trajectory optimization problem up to 60 times when compared with random initial guesses. Moreover, we also conduct experimental trials on the ANYmal robot to validate our method
Motion synthesis for high degree-of-freedom robots in complex and changing environments
Full text linkThe use of robotics has recently seen significant growth in various domains such as
unmanned ground/underwater/aerial vehicles, smart manufacturing, and humanoid
robots. However, one of the most important and essential capabilities required for
long term autonomy, which is the ability to operate robustly and safely in real-world
environments, in contrast to industrial and laboratory setup is largely missing. Designing
robots that can operate reliably and efficiently in cluttered and changing
environments is non-trivial, especially for high degree-of-freedom (DoF) systems, i.e.
robots with multiple actuators. On one hand, the dexterity offered by the kinematic
redundancy allows the robot to perform dexterous manipulation tasks in complex
environments, whereas on the other hand, such complex system also makes controlling
and planning very challenging. To address such two interrelated problems, we
exploit robot motion synthesis from three perspectives that feed into each other: end-pose
planning, motion planning and motion adaptation. We propose several novel
ideas in each of the three phases, using which we can efficiently synthesise dexterous
manipulation motion for fixed-base robotic arms, mobile manipulators, as well as
humanoid robots in cluttered and potentially changing environments.
Collision-free inverse kinematics (IK), or so-called end-pose planning, a key prerequisite
for other modules such as motion planning, is an important and yet unsolved
problem in robotics. Such information is often assumed given, or manually provided
in practice, which significantly limiting high-level autonomy. In our research, by using
novel data pre-processing and encoding techniques, we are able to efficiently
search for collision-free end-poses in challenging scenarios in the presence of uneven
terrains.
After having found the end-poses, the motion planning module can proceed. Although
motion planning has been claimed as well studied, we find that existing algorithms
are still unreliable for robust and safe operations in real-world applications,
especially when the environment is cluttered and changing. We propose a novel
resolution complete motion planning algorithm, namely the Hierarchical Dynamic
Roadmap, that is able to generate collision-free motion trajectories for redundant
robotic arms in extremely complicated environments where other methods would fail.
While planning for fixed-base robotic arms is relatively less challenging, we also investigate
into efficient motion planning algorithms for high DoF (30 - 40) humanoid
robots, where an extra balance constraint needs to be taken into account. The result
shows that our method is able to efficiently generate collision-free whole-body trajectories
for different humanoid robots in complex environments, where other methods
would require a much longer planning time.
Both end-pose and motion planning algorithms compute solutions in static environments,
and assume the environments stay static during execution. While human
and most animals are incredibly good at handling environmental changes, the state-of-the-art robotics technology is far from being able to achieve such an ability. To
address this issue, we propose a novel state space representation, the Distance Mesh
space, in which the robot is able to remap the pre-planned motion in real-time and
adapt to environmental changes during execution.
By utilizing the proposed end-pose planning, motion planning and motion adaptation
techniques, we obtain a robotic framework that significantly improves the
level of autonomy. The proposed methods have been validated on various state-of-the-art robot platforms, such as UR5 (6-DoF fixed-base robotic arm), KUKA LWR
(7-DoF fixed-base robotic arm), Baxter (14-DoF fixed-base bi-manual manipulator),
Husky with Dual UR5 (15-DoF mobile bi-manual manipulator), PR2 (20-DoF mobile
bi-manual manipulator), NASA Valkyrie (38-DoF humanoid) and many others, showing
that our methods are truly applicable to solve high dimensional motion planning
for practical problems
Concurrent design and motion planning in robotics using differentiable optimal control
Full text linkRobot design optimization (what the robot is) and motion planning (how the robot
moves) are two problems that are connected. Robots are limited by their design in
terms of what motions they can execute – for instance a robot with a heavy base has
less payload capacity compared to the same robot with a lighter base. On the other
hand, the motions that the robot executes guide which design is best for the task.
Concurrent design (co-design) is the process of performing robot design and motion
planning together. Although traditionally co-design has been viewed as an offline
process that can take hours or days, we view interactive co-design tools as the next
step as they enable quick prototyping and evaluation of designs across different tasks
and environments.
In this thesis we adopt a gradient-based approach to co-design. Our baseline
approach embeds the motion planning into bi-level optimization and uses gradient
information via finite differences from the lower motion planning level to optimize
the design in the upper level. Our approach uses the full rigid-body dynamics of the
robot and allows for arbitrary upper-level design constraints, which is key for finding
physically realizable designs. Our approach is also between 1.8 and 8.4 times faster on
a quadruped trotting and jumping co-design task as compared to the popular genetic
algorithm covariance matrix adaptation evolutionary strategy (CMA-ES). We further
demonstrate the speed of our approach by building an interactive co-design tool that
allows for optimization over uneven terrain with varying height.
Furthermore, we propose an algorithm to analytically take the derivative of nonlinear
optimal control problems via differential dynamic programming (DDP). Analytical
derivatives are a step towards addressing the scalability and accuracy issues of finite
differences. We further compared with a simultaneous approach for co-design that
optimizes both motion and design in one nonlinear program. On a co-design task for
the Kinova robotic arm we observed a 54-times improvement in computational speed.
We additionally carry out hardware validation experiments on the quadruped robot
Solo. We designed longer lower legs for the robot, which minimize the peak torque
used during trotting. Although we always observed an improvement in peak torque,
it was less than in simulation (7.609% versus 28.271%). We discuss some of the sim-toreal
issues including the structural stability of joints and slipping of feet that need to
be considered and how they can be addressed using our framework.
In the second part of this thesis we propose solutions to some open problems
in motion planning. Firstly, in our co-design approach we assumed fixed contact
locations and timings. Ideally we would like the motion planner to choose the contacts
instead. We solve a related, but simpler problem, which is the control of satellite
thrusters, which are similar to robot feet but do not have the constraint of having to be
in contact with the ground to exert force on the robot. We introduce a sparse, L1 cost
on control inputs (thrusters) and implement optimization via DDP-style solvers. We
use full rigid-body dynamics and achieve bang-bang control via optimization, which
is a difficult problem due to the discrete switching nature of the thrusters.
Lastly, we present a method for planning and control of a hybrid, wheel-legged
robot. This is a difficult problem, as the robot needs to always actively balance on
the wheel even when not driving or jumping forward. We propose the variablelength
wheeled inverted pendulum (VL-WIP) template model that captures only the
necessary dynamic interactions between wheels and base. We embedded this into
a model-predictive controller (MPC) and demonstrated highly dynamic behaviors,
including swinging-up and jumping over a gap.
Both of these motion planning problems expand the ability of our motion planning
tools to new domains, which is an integral part also of the co-design algorithms, as
co-design aims to optimize both design, and motion, together
Enhancing gait rehabilitation using robotic assistance and functional electrical stimulation
No full textWearable robots and assistive exoskeletons have great potential as tools for
rehabilitation and assisted living. By providing support to dedicated joints and
body segments, these devices can foster the independence of people suffering
from neurological diseases and improve quality of life. However, ensuring
these devices respond appropriately to the unique needs of each patient is
crucial, as it can play a decisive role in whether neural plasticity is induced.
This is particularly challenging as patients suffering from stroke or incomplete
spinal cord injury start regaining control of their limbs, where rigid robotdriven
interventions are no longer adequate to facilitate recovery and more
adaptive patient-driven interventions are required.
Motivated by the principles of neuroplasticity, this thesis delves into the
integration of wearable robots in neurological gait rehabilitation, with a central
focus on the design of personalised interventions for ambulatory patients.
More specifically, this thesis explores methods for the optimisation of robotic
controllers and collaborative functional electrical stimulation (FES) controllers,
such that assistance can be provided as needed, encouraging the patient to use
their residual strength and actively take part in gait training.
Firstly, we formulate the problem of providing assistance ‘as needed’ as
an optimisation problem and propose an offline model-based optimisation
method for the design of personalised rehabilitation interventions. Using
motion capture and high-fidelity musculoskeletal models, we construct a personalised
model of the human interacting with the robot, and we optimise the
controller of the robot using forward dynamics. We describe how this method
can be applied for both the design of novel near-optimal surrogate controllers
in the real-world, as well as the fine-tuning of parameterised controllers with a
known control structure. The effect of the offline model-based optimisation
method is evaluated both in simulation and experimentally, highlighting the
need for personalisation and the importance of capturing the inter-personal
and intra-personal variability in human behaviour.
An alternative to model-based optimisation is human-in-the-loop optimisation,
where the human response to different levels of assistance can be obtained
in real time, reducing uncertainties due to modelling bias. Human-in-the-loop
optimisation has proven to be an effective method for reducing the metabolic
cost in robot-assisted locomotion, but its potential in enhancing rehabilitation
has not been explored. We hypothesise that with the use of the Covariance
Matrix Adaptation Evolution Strategy (CMA-ES), time-dependent variations in
gait may be captured, facilitating the detection of time-varying local minima.
Using continuous optimisation over a two-day experimental protocol we carry
out a preliminary study of human-in-the-loop optimisation and present the
results obtained from healthy subjects.
Beyond the deployment of robotics in neural rehabilitation, numerous
physiological benefits can be achieved with the use of functional electrical stimulation
(FES). Particularly interesting is the integration of robotics with FES,
as the two have several complementary characteristics. However, due to the
increased complexity of hybrid robot-FES systems, controller personalisation
through model-based or human-in-the-loop optimisation becomes increasingly
demanding. To promote the triadic collaboration between human, robot and
FES, we propose a novel hierarchical and adaptive controller. We demonstrate
a hybrid system that can prioritise the voluntary contributions of the human
and effectively distribute the necessary assistive forces between the robot and
the FES, in order to delay the onset of muscle fatigue and provide assistance
as needed.
These methods contribute towards the advancement of techniques for designing
personalised interventions for gait rehabilitation, which could lead to
improved functional outcomes and accelerated recovery after training
Robustness to external disturbances for legged robots using dynamic trajectory optimisation
Full text linkIn robotics, robustness is an important and desirable attribute of any system, from
perception to planning and control. Robotic systems need to handle numerous factors
of uncertainty when they are deployed, and the more robust a method is, the fewer
chances there are of something going wrong. In planning and control, being robust
is crucial to deal with uncertain contact timings and positions, mismatches in the
dynamics model of the system, noise in the sensor readings and communication
delays. In this thesis, we focus on the problem of dealing with uncertainty and
external disturbances applied to the robot.
Reactive robustness can be achieved at the control stage using a variety of control
schemes. For example, model predictive control approaches are robust against external
disturbances thanks to the online high-frequency replanning of the motion being
executed. However, taking robustness into account in a proactive way, i.e., during the
planning stage itself, enables the adoption of kinematic configurations that allow the
system as a whole to better deal with uncertainty and disturbances.
To this end, we propose a novel trajectory optimisation framework for robotic
systems, ranging from fixed-base manipulators to legged robots, such as humanoids
or quadrupeds equipped with arms. We tackle the problem from a first-principles
perspective, and define a robustness metric based on the robot’s capabilities, such as
the torques available to the system (considering actuator torque limits) and contact
stability constraints. We compare our results with other existing approaches and,
through simulation and experiments on the real robot, we show that our method is
able to plan trajectories that are more robust against external disturbances
Understanding the fundamentals of bipedal locomotion in humans and robots
Full text linkWalking is a robust and efficient method of moving around the world, which would greatly enhance the capabilities of humanoid robots, although they cannot match the performance of their biological counterparts. The highly nonlinear dynamics of locomotion create a vast state-action space, which makes model-based control difficult, yet biological humans are highly proficient and robust in their motion while operating under similar constraints. This disparity in performance naturally leads to the question: what can we learn about locomotion control by observing humans, and how can this be used to develop bio-inspired locomotion control in mechatronic humanoids? This thesis investigates bio-inspired locomotion control, but also explores the limitations of this approach and how we can use robotic platforms to move towards a better understanding of locomotion.
We first present a methodology for measuring and analysing human locomotion behaviour, specifically disturbance recovery, and fit models to this complex behaviour to represent it in as simple as possible such that it can be easily translated into a simple controller for reactive motion. A minimum-jerk Model Predictive Control algorithm at the Centre of Mass (CoM) best captured human motion during multiple recovery strategies instead of using one controller for each strategy, which is common in this area. Capturing this simple CoM model of complex human behaviour shows that bio-inspiration can be an important tool for controller development, but behaviour varies between and even within individuals given similar initial conditions, which manifests as stochastic behaviour. Coupled with the ability to only measure expressed behaviours instead of direct control policies, this stochasticity presents a fundamental limit to using bio-inspiration for control purposes, as only indirect inferences can be made about a complex, stochastic system.
To overcome these barriers, we investigate the use of mechatronic humanoid robots as a means to explore invariant aspects of the vast dynamic state-space of locomotion which are described by physical laws, and are therefore not subject to the stochastic behaviour of individual humans, that apply to both biological and mechatronic humanoid forms. We present a pipeline to explore the invariant energetics of humanoid robots during stepping for push recovery, where the most efficient stepping parameters are identified for a given initial CoM velocity and desired step length. Using this to explore the stepping state-space, our analysis finds a region of attraction between disturbance magnitude and optimal step length surrounded by a region of similarly efficient alternatives which corresponds to the stochastic behavior observed in humans during push recovery, which we would be unable to identify without reproducibility, direct access to internal measurements and known full body dynamics, which is not available in humans.
We expand this paradigm further to investigate the invariant energetics of continuous walking using a full-body humanoid by exploring the state-space of step-length and step-timing to identify the most efficient sub-spaces of these parameters which describes the most efficient way to walk. Through analysis of this state-space, we provide evidence that the humanoid morphology exhibits a passive tendency towards energy-optimal motion and its dynamics follow a region of attraction towards Cost of Transport-optimal motion.
Overall, these findings demonstrate the utility of robotics as a tool with which to explore certain aspects of legged locomotion and the results gained from our methodology suggest that humans do not need to explore a vast state-action space to learn to walk, they need only internalise simple heuristics for the natural dynamics of stepping that are easy to learn and can produce rapid, reactive and efficient stepping without costly decision-making processes
- …
