320 research outputs found
Duality-based a posteriori error estimates for some approximation schemes for optimal investment problems
No full textWe consider a Markov chain approximation scheme for utility maximization problems in continuous time, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauß-Hermite approximation of the Gaußian increments. The error estimates previously derived in Picarelli and Reisinger (2018) are asymmetric between lower and upper bounds due to the control approximation and improve on known results in the literature in the lower case only. In the present paper, we use duality results to obtain a posteriori upper error bounds which are empirically of the same order as the lower bounds. The theoretical results are confirmed by our numerical tests
Probabilistic error analysis for some approximation schemes to optimal control problems
No full textWe introduce a class of numerical schemes for optimal stochastic control problems based on a novel Markov chain approximation, which uses, in turn, a piecewise constant policy approximation, Euler– Maruyama time stepping, and a Gauß-Hermite approximation of the Gaußian increments. We provide lower error bounds of order arbitrarily close to 1/2 in time and 1/3 in space for Lipschitz viscosity solutions, coupling probabilistic arguments with regularization techniques as introduced by Krylov. The corresponding order of the upper bounds is 1/4 in time and 1/5 in space. For sufficiently regular solutions, the order is 1 in both time and space for both bounds. Finally, we propose techniques for further improving the accuracy of the individual components of the approximation
Some regularity and convergence results for parabolic Hamilton-Jacobi-Bellman equations in bounded domains
No full textWe study parabolic Hamilton-Jacobi-Bellman (HJB) equations in bounded domains with strong Dirichlet boundary conditions. We work under the assumption of the existence of a sufficiently regular barrier function for the problem to obtain well-posedness and regularity of a related switching system and the convergence of its components to the HJB equation. In particular, we show existence of a viscosity solution to the switching system by a novel construction of sub- and supersolutions and application of Perron’s method. Error bounds for monotone schemes for the HJB equation are then derived from estimates near the boundary, where the standard regularisation procedure for viscosity solutions is not applicable, and are found to be of the same order as known results for the whole space. We deduce error bounds for some common finite difference and truncated semi-Lagrangian schemes
Improved order 1/4 convergence for piecewise constant policy approximation of stochastic control problems
No full textIn N. V. Krylov, "Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies", Electron. J. Probab., 4(2), 1999, it is proved under standard assumptions that the value functions of controlled diffusion processes can be approximated with order 1/6 error by those with controls which are constant on uniform time intervals. In this note we refine the proof and show that the provable rate can be improved to 1/4, which is optimal in our setting. Moreover, we demonstrate the improvements this implies for error estimates derived by similar techniques for approximation schemes, bringing these in line with the best available results from the PDE literature
Deep xVA solver - A neural network based counterparty credit risk management framework
No full textIn this paper, we present a novel computational framework for portfolio-wide risk management problems wherethe presence of a potentially large number of risk factors makes traditional numerical techniques ineffective.The new method utilises a coupled system of BSDEs for the valuation adjustments (xVA) and solves these by a recursive application of a neural network based BSDE solver.This not only makes the computation of xVA for high-dimensional problems feasible, but also produces hedge ratios and dynamic risk measures for xVA, and allows simulations of the collateral account
Boundary Mesh Refinement for Semi-Lagrangian Schemes
No full textWe study semi-Lagrangian schemes for the Dirichlet problem for second-order degenerate elliptic PDEs. Like other wide stencil schemes, these schemes have to be truncated near the boundaries to avoid " over-stepping ". The various modifications proposed in the literature lead to either reduced consistency orders for those points, or even a loss of consistency with the differential operator in the usual sense. We propose a local mesh refinement strategy near domain boundaries which achieves a uniform order of consistency up to the boundary in the first case, and in both cases reduces the width of the region where overstepping occurs, so that the practically observed convergence order is unaffected by overstepping. We demonstrate this numerically for a linear parabolic equation and a second order HJB equation
Analysis of various estimators for multi-dimensional Zakai equations
No full textWe first consider a one-dimensional stochastic partial differential equation
(SPDE) of Zakai type describing a large credit portfolio. Specifically, we construct
estimators of linear functionals of the solution from an implicit Milstein
scheme on a space-time mesh. We compare the complexity of a multi-index
Monte Carlo (MIMC) approach with the multilevel Monte Carlo (MLMC)
method of Giles and Reisinger (2012), and find, by means of Fourier analysis,
that the MIMC method has slightly improved complexity O(ε-2|log ε|) for a
root mean square error (RMSE) ε if a carefully adapted discretisation is used.
Then, we propose an implicit finite difference scheme for a two-dimensional
parabolic SPDE of Zakai type, based on a Milstein approximation to the
stochastic integral and an alternating direction implicit (ADI) discretisation
of the elliptic term. We prove its mean-square stability and convergence in
L2 of first order in time and second order in space, by Fourier analysis, in the
presence of Dirac initial data.
Next, we analyse the accuracy and computational complexity of estimators
for linear functionals of the solution to the 2-d SPDE, coupled with the sparse
combination technique and MLMC. We find, by detailed Fourier analysis, that
for a RMSE ε, MLMC with sparse combination has the optimal complexity
O(ε-2), whereas MLMC on regular grids has O(ε-2(log ε)2), standard MC with
sparse combination O(ε-7/2(|log ε|)5/2), and MC on regular grids O(ε-4). We
give a discussion of the higher-dimensional setting without detailed proofs,
which suggests that MLMC with sparse combination always leads to the optimal
complexity.
Finally, we consider a particular two-dimensional SPDE with fast mean-reverting
volatility on a timescale O(ε-1), and study the small ε asymptotics. We find
an asymptotic expansion of the solution to the SPDE as ε â 0 and conclude
from numerical experiments the convergence order 1=2 of the leading term and
order 1 after inclusion of the first correction term.</p
Optimal control and reinforcement learning for formula one lap simulation
No full textLap simulation in a Formula One context is a subclass of optimal control
problems and describes the computation of optimal trajectories around
racing circuits. The results of lap simulation are primarily used
for vehicle setup and strategic racing decisions. The optimal
lap problem is solved using two classes of algorithms. The first algorithm
uses direct collocation to compute optimal trajectories and the second
algorithm uses specially constructed reinforcement learning environments
and generalised function approximation to compute desirable system inputs.
Historically direct collocation methods were considered impractical for minimum
lap time simulations, due to their high computational costs. The
exponential increase in computational performance has enabled the practical
application of these algorithms.
These lap time simulations require a vehicle model, as well as a track
discretisation. As an example for this, the classical bicycle model along with
a curvilinear track model are introduced. To solve the resulting direct
collocation problems, algorithms for non-linear optimisation problems are
presented and performance critical aspects are discussed. The optimisation
algorithm is accelerated by utilising highly parallel computer architectures,
such as graphics processing units (GPUs). An analytical gradient approximation
is presented to achieve approximations of projection systems which constitute
one most performance critical components of the solution process. Mesh
refinement algorithms are discussed and a novel mesh refinement heuristic based
on optimal polynomial approximation in an sense is discussed. The
approximation is improved by detecting singularities and using Clenshaw--Curtis
quadrature on intermediary intervals.
In Chapter 4 of this work, the lap time optimisation
problem is reformulated as a reinforcement learning environments. For this,
the relevant background literature on reinforcement learning is discussed
and a translation of a training optimisation environment is constructed.
Details of this environment are discussed in the form of reward signals,
terminal conditions, and observation features. A series of learning models
is discussed with increasing feature fidelity leading to an algorithm that
can generalise well across representations of circuits from the 2022 Formula One
calendar. This work expands on the current literature by providing novel, physically
motivated, reinforcement learning environments for lap time optimisation tasks.
The results of both approaches are combined by using strategy extraction to
initialise the collocation optimisation algorithm and optimise the underlying mesh
Analysis of multi-index Monte Carlo estimators for a Zakai SPDE
No full textIn this article, we propose a space-time Multi-Index Monte Carlo (MIMC) estimator for a onedimensional parabolic stochastic partial differential equation (SPDE) of Zakai type. We compare the complexity with the Multilevel Monte Carlo (MLMC) method of Giles and Reisinger (2012), and find, by means of Fourier analysis, that the MIMC method: (i) has suboptimal complexity of O(ε −2 | log ε| 3 ) for a root mean square error (RMSE) ε if the same spatial discretisation as in the MLMC method is used; (ii) has a better complexity of O(ε −2 | log ε|) if a carefully adapted discretisation is used; (iii) has to be adapted for non-smooth functionals. Numerical tests confirm these findings empirically
DUALITY-BASED A POSTERIORI ERROR ESTIMATES FOR SOME APPROXIMATION SCHEMES FOR CONVEX OPTIMAL CONTROL PROBLEMS
No full textWe consider a Markov chain approximation scheme for utility maximization in continuous time optimal investment problems, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauss-Hermite approximation of the Gaussian increments.The error estimates previously derived in A. Picarelli and C. Reisinger, Probabilistic error analysis for some approximation schemes to optimal control problems, arXiv:1810.04691, are asymmetric between lower and upper bounds due to the control approximation and improve on known results in the literature in the lower case only. In the present paper, we use duality results to obtain a posteriori upper error bounds which are empirically of the same order as the lower bounds. The theoretical results are confirmed by our numerical tests
- …
