1,720,983 research outputs found
Error Estimates for a Tree Structure Algorithm Solving Finite Horizon Control Problems
Full text linkIn the dynamic programming approach to optimal control problems a crucial role is played by the value function that is characterized as the unique viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. It is well known that this approach suffers from the "curse of dimensionality" and this limitation has reduced its use in real world applications. Here, we analyze a dynamic programming algorithm based on a tree structure to mitigate the "curse of dimensionality". The tree is built by the discrete time dynamics avoiding the use of a fixed space grid which is the bottleneck for highdimensional problems, this also drops the projection on the grid in the approximation of the value function. In this work, we present first order error estimates for the the approximation of the value function based on the tree-structure algorithm. The estimate turns out to have the same order of convergence of the numerical method used for the approximation of the dynamics. Furthermore, we analyze a pruning technique for the tree to reduce the complexity and minimize the computational effort. Finally, we present some numerical tests to show the theoretical results
An accelerated value/policy iteration scheme for optimal control problems and games
No full textWe present an accelerated algorithm for the solution of static Hamilton-Jacobi-Bellman equations related to optimal control problems and differential games. The new scheme combines the advantages of value iteration and policy iteration methods by means of an efficient coupling. The method starts with a value iteration phase on a coarse mesh and then switches to a policy iteration procedure over a finer mesh when a fixed error threshold is reached. We present numerical tests assessing the performance of the scheme
Piecewise DMD for oscillatory and Turing spatio-temporal dynamics
Full text linkDynamic Mode Decomposition (DMD) is an equation-free method that aims at reconstructing the best linear fit from temporal datasets. In this paper, we show that DMD does not provide accurate approximation for datasets describing oscillatory dynamics, like spiral waves, relaxation oscillations and spatio-temporal Turing instability. Inspired by the classical “divide and conquer” approach, we propose a piecewise version of DMD (pDMD) to overcome this problem. The main idea is to split the original dataset in N submatrices and then apply the exact (randomized) DMD method in each subset of the obtained partition. We describe the pDMD algorithm in detail and we introduce some error indicators to evaluate its performance when N is increased. Numerical experiments show that very accurate reconstructions are obtained by pDMD for datasets arising from time snapshots of
certain reaction-diffusion PDE systems, like the FitzHugh-Nagumo model, a lambda-omega system and the DIB morpho-chemical system for battery modeling. Finally, a discussion about the overall computational load and the future prediction features of the new algorithm is also provided
A PINN Approach for the Online Identification and Control of Unknown PDEs
Full text linkPhysics-Informed Neural Networks (PINNs) have revolutionized solving differential equations by integrating physical laws into neural networks training. This paper explores PINNs for open-loop optimal control problems with incomplete information, such as sparse initial and boundary data and partially unknown system parameters. We derive optimality conditions from the Lagrangian multipliers and use PINNs to predict the state, adjoint, and control variables. In contrast with previous methods, our approach integrates these elements into a single neural network and addresses scenarios with consistently limited data. In addition, we address the study of partially unknown equations identifying underlying parameters online by searching for the optimal solution recurring to a 2-in-series architecture of PINNs, in which scattered data of the uncontrolled solution is used. Numerical examples show the effectiveness of the proposed method even in scenarios characterized by a considerable lack of information
Understanding Mass Transfer Directions via Data-Driven Models with Application to Mobile Phone Data
No full textThe aim of this paper is to solve an inverse problem which regards a mass moving in a bounded domain. We assume that the mass moves following an unknown velocity field and that the evolution of the mass density can be described by a partial differential equation, which is also unknown. The input data of the problems are given by some snapshots of the mass distribution at certain times, while the sought output is the velocity field that drives the mass along its displacement. To this aim, we put in place an algorithm based on the combination of two methods: first, we use the dynamic mode decomposition to create a mathematical model describing the mass transfer; second, we use the notion of Wasserstein distance (also known as earth mover's distance) to reconstruct the underlying velocity field that is responsible for the displacement. Finally, we consider a real-life application: the algorithm is employed to study the travel flows of people in large populated areas using, as input data, density profiles (i.e., the spatial distribution) of people in given areas at different time instants. These kinds of data are provided by the Italian telecommunication company TIM and are derived by mobile phone usage
Online identification and control of PDEs via reinforcement learning methods
No full textWe focus on the control of unknown partial differential equations (PDEs). The system dynamics is unknown, but we assume we are able to observe its evolution for a given control input, as typical in a reinforcement learning framework. We propose an algorithm based on the idea to control and identify on the fly the unknown system configuration. In this work, the control is based on the state-dependent Riccati approach, whereas the identification of the model on Bayesian linear regression. At each iteration, based on the observed data, we obtain an estimate of the a-priori unknown parameter configuration of the PDE and then we compute the control of the correspondent model. We show by numerical evidence the convergence of the method for infinite horizon control problems
A HJB-POD approach for the control of nonlinear PDEs on a tree structure
Full text linkThe Dynamic Programming approach allows to compute a feedback control for nonlinear problems, but suffers from the curse of dimensionality. The computation of the control relies on the resolution of a nonlinear PDE, the Hamilton-Jacobi-Bellman equation, with the same dimension of the original problem. Recently, a new numerical method to compute the value function on a tree structure has been introduced. The method allows to work without a structured grid and avoids any interpolation. Here, we aim at testing the algorithm for nonlinear two dimensional PDEs. We apply model order reduction to decrease the computational complexity since the tree structure algorithm requires to solve many PDEs. Furthermore, we prove an error estimate which guarantees the convergence of the proposed method. Finally, we show efficiency of the method through numerical tests
FEEDBACK RECONSTRUCTION TECHNIQUES FOR OPTIMAL CONTROL PROBLEMS ON A TREE STRUCTURE
Full text linkThe computation of feedback control using Dynamic Programming equation is a difficult task due the curse of dimensionality. The tree structure algorithm is one the methods introduced recently that mitigate this problem. The method computes the value function avoiding the construction of a space grid and the need for interpolation techniques using a discrete set of controls. However, the computation of the control is strictly linked to control set chosen in the computation of the tree. Here, we extend and complete the method selecting a finer control set in the computation of the feedback. This requires to use an interpolation method for scattered data which allows us to reconstruct the value function for nodes not belonging to the tree. The effectiveness of the method is shown via a numerical example
HJB-RBF Based Approach for the Control of PDEs
Full text linkSemi-Lagrangian schemes for the discretization of the dynamic programming principle are
based on a time discretization projected on a state-space grid. The use of a structured grid
makes this approach not feasible for high-dimensional problems due to the curse of dimensionality.
Here, we present a new approach for infinite horizon optimal control problems
where the value function is computed using radial basis functions by the Shepard moving
least squares approximationmethod on scattered grids.We propose a newmethod to generate
a scattered mesh driven by the dynamics and the selection of the shape parameter in the RBF
using an optimization routine. This mesh will help to localize the problem and approximate
the dynamic programming principle in high dimension. Error estimates for the value function
are also provided. Numerical tests for high dimensional problems will show the effectiveness
of the proposed method
- …
