1,720,971 research outputs found
Dynamical low-rank approximation strategies for nonlinear feedback control problems
No full textThis paper addresses the stabilization of dynamical systems in the infinite horizon optimal control setting using nonlinear feedback control based on State-Dependent Riccati Equations (SDREs). While effective, the practical implementation of such feedback strategies is often constrained by the high dimensionality of state spaces and the computational challenges associated with solving SDREs, particularly in parametric scenarios. To mitigate these limitations, we introduce the Dynamical Low-Rank Approximation (DLRA) methodology, which provides an efficient and accurate framework for addressing high-dimensional feedback control problems. DLRA dynamically constructs a low-dimensional representation that evolves with the problem, enabling the simultaneous resolution of multiple parametric instances in real-time. We propose two novel algorithms to enhance numerical performances: the cascade-Newton-Kleinman method and Riccati-based DLRA (R-DLRA). The cascade-Newton-Kleinman method accelerates convergence by leveraging Riccati solutions from the nearby parameter or time instance, supported by a theoretical sensitivity analysis. R-DLRA integrates Riccati information into the DLRA basis construction to improve the quality of the solution. These approaches are validated through nonlinear one-dimensional and two-dimensional test cases showing transport-like behavior, demonstrating that R-DLRA outperforms standard DLRA and Proper Orthogonal Decomposition-based model order reduction in both speed and accuracy, offering a superior alternative to Full Order Model solutions
Error Estimates for a Tree Structure Algorithm Solving Finite Horizon Control Problems
Full text linkIn the dynamic programming approach to optimal control problems a crucial role is played by the value function that is characterized as the unique viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. It is well known that this approach suffers from the "curse of dimensionality" and this limitation has reduced its use in real world applications. Here, we analyze a dynamic programming algorithm based on a tree structure to mitigate the "curse of dimensionality". The tree is built by the discrete time dynamics avoiding the use of a fixed space grid which is the bottleneck for highdimensional problems, this also drops the projection on the grid in the approximation of the value function. In this work, we present first order error estimates for the the approximation of the value function based on the tree-structure algorithm. The estimate turns out to have the same order of convergence of the numerical method used for the approximation of the dynamics. Furthermore, we analyze a pruning technique for the tree to reduce the complexity and minimize the computational effort. Finally, we present some numerical tests to show the theoretical results
Approximation of optimal control problems for the Navier-Stokes equation via multilinear HJB-POD
No full textWe consider the approximation of some optimal control problems for the Navier-Stokes equation via a Dynamic Programming approach. These control problems arise in many industrial applications and are very challenging from the numerical point of view since the semi-discretization of the dynamics corresponds to an evolutive system of ordinary differential equations in very high-dimension. The typical approach is based on the Pontryagin maximum principle and leads to a two point boundary value problem. Here we present a different approach based on the value function and the solution of a Bellman equation, a challenging problem in high-dimension. We mitigate the curse of dimensionality via a recent multilinear approximation of the dynamics coupled with a dynamic programming scheme on a tree structure. We discuss several aspects related to the implementation of this new approach and we present some numerical examples to illustrate the results on classical control problems studied in the literature.We consider the approximation of some optimal control problems for the Navier-Stokes equation via a Dynamic Programming approach. These control problems arise in many industrial applications and are very challenging from the numerical point of view since the semi-discretization of the dynamics corresponds to an evolutive system of ordinary differential equations in very high-dimension. The typical approach is based on the Pontryagin maximum principle and leads to a two point boundary value problem. Here we present a different approach based on the value function and the solution of a Bellman equation, a challenging problem in high-dimension. We mitigate the curse of dimensionality via a recent multilinear approximation of the dynamics coupled with a dynamic programming scheme on a tree structure. We discuss several aspects related to the implementation of this new approach and we present some numerical examples to illustrate the results on classical control problems studied in the lite..
FEEDBACK RECONSTRUCTION TECHNIQUES FOR OPTIMAL CONTROL PROBLEMS ON A TREE STRUCTURE
Full text linkThe computation of feedback control using Dynamic Programming equation is a difficult task due the curse of dimensionality. The tree structure algorithm is one the methods introduced recently that mitigate this problem. The method computes the value function avoiding the construction of a space grid and the need for interpolation techniques using a discrete set of controls. However, the computation of the control is strictly linked to control set chosen in the computation of the tree. Here, we extend and complete the method selecting a finer control set in the computation of the feedback. This requires to use an interpolation method for scattered data which allows us to reconstruct the value function for nodes not belonging to the tree. The effectiveness of the method is shown via a numerical example
Separable Approximations of Optimal Value Functions Under a Decaying Sensitivity Assumption
No full textAn efficient approach for the construction of separable approximations of optimal value functions from interconnected optimal control problems is presented. The approach is based on assuming decaying sensitivities between subsystems, enabling a curse-of-dimensionality free approximation, for instance by deep neural networks
Data-driven Tensor Train Gradient Cross Approximation for Hamilton-Jacobi-Bellman Equations
Full text linkA gradient-enhanced functional tensor train cross approximation method for
the resolution of the Hamilton-Jacobi-Bellman (HJB) equations associated to
optimal feedback control of nonlinear dynamics is presented. The procedure uses
samples of both the solution of the HJB equation and its gradient to obtain a
tensor train approximation of the value function. The collection of the data
for the algorithm is based on two possible techniques: Pontryagin Maximum
Principle and State Dependent Riccati Equations. Several numerical tests are
presented in low and high dimension showing the effectiveness of the proposed
method and its robustness with respect to inexact data evaluations, provided by
the gradient information. The resulting tensor train approximation paves the
way towards fast synthesis of the control signal in real-time applications
A HJB-POD approach for the control of nonlinear PDEs on a tree structure
Full text linkThe Dynamic Programming approach allows to compute a feedback control for nonlinear problems, but suffers from the curse of dimensionality. The computation of the control relies on the resolution of a nonlinear PDE, the Hamilton-Jacobi-Bellman equation, with the same dimension of the original problem. Recently, a new numerical method to compute the value function on a tree structure has been introduced. The method allows to work without a structured grid and avoids any interpolation. Here, we aim at testing the algorithm for nonlinear two dimensional PDEs. We apply model order reduction to decrease the computational complexity since the tree structure algorithm requires to solve many PDEs. Furthermore, we prove an error estimate which guarantees the convergence of the proposed method. Finally, we show efficiency of the method through numerical tests
An efficient DP algorithm on a tree-structure for finite horizon optimal control problems
Full text linkThe classical dynamic programming (DP) approach to optimal control problems is based on the characterization of the value function as the unique viscosity solution of a Hamilton-Jacobi-Bellman equation. The DP scheme for the numerical approximation of viscosity solutions of Bellman equations is typically based on a time discretization which is projected on a fixed state-space grid. The time discretization can be done by a one-step scheme for the dynamics and the projection on the grid typically uses a local interpolation. Clearly the use of a grid is a limitation with respect to possible applications in high-dimensional problems due to the curse of dimensionality. Here, we present a new approach for finite horizon optimal control problems where the value function is computed using a DP algorithm with a tree structure algorithm constructed by the time discrete dynamics. In this way there is no need to build a fixed space triangulation and to project on it. The tree will guarantee a perfect matching with the discrete dynamics and drop off the cost of the space interpolation allowing for the solution of very high-dimensional problems. Numerical tests will show the effectiveness of the proposed method
High-order approximation of the finite horizon control problem via a tree structure algorithm
Full text linkSolving optimal control problems via Dynamic Programming is a difficult task that suffers for the”curse of dimensionality”. This limitation has reduced its practical impact in real world applications since the construction of numerical methods for nonlinear PDEs in very high dimension is practically unfeasible. Recently, we proposed a new numerical method to compute the value function avoiding the construction of a space grid and the need for interpolation techniques. The method is based on a tree structure that mimics the continuous dynamics and allows to solve optimal control problems in high-dimension. This property is particularly useful to attack control problems with PDE constraints. We present a new high-order approximation scheme based on the tree structure and show some numerical results
A tree structure algorithm for optimal control problems with state constraints
Full text linkWe present a tree structure algorithm for optimal control problems with state constraints. We prove a convergence result for a discrete time approximation of the value function based on a novel formulation in the case of convex constraints. Then the Dynamic Programming approach is developed by a discretization in time leading to a tree structure in space derived by the controlled dynamics, taking into account the state constraints to cut several branches of the tree. Moreover, an additional pruning allows for the reduction of the tree complexity as for the case without state constraints. Since the method does not use an a priori space grid, no interpolation is needed for the reconstruction of the value function and the accuracy essentially relies on the time step h. These features permit a reduction in CPU time and in memory allocations. The synthesis of optimal feedback controls is based on the values on the tree and an interpolation on the values obtained on the tree will be necessary if a different discretization in the control space is adopted, e.g. to improve the accuracy of the method in the reconstruction of the optimal trajectories. Several examples show how this algorithm can be applied to problems in low dimension and compare it to a classical DP method on a grid
- …
