1,720,974 research outputs found
Optimal control of continuous-time markov chains with noise-free observation
No full textWe consider an infinite horizon optimal control problem for a continuous-time Markov chain in a finite set with noise-free partial observation. The observation process is defined as , , where is a given map defined on . The observation is noise-free in the sense that the only source of randomness is the process itself. The aim is to minimize a discounted cost functional and study the associated value function . After transforming the control problem with partial observation into one with complete observation (the separated problem) using filtering equations, we provide a link between the value function associated with the latter control problem and the original value function . Then, we present two different characterizations of (and indirectly of ): on one hand as the unique fixed point of a suitably defined contraction mapping and on the other hand as the unique constrained viscosity solution (in the sense of Soner) of a HJB integro-differential equation. Under suitable assumptions, we finally prove the existence of an optimal control
Stochastic filtering and optimal control of pure jump Markov processes with noise-free partial observation
No full textWe consider an infinite horizon optimal control problem for a pure jump Markov process , taking values in a complete and separable metric space , with noise-free partial observation. The observation process is defined as , , where is a given map defined on . The observation is noise-free in the sense that the only source of randomness is the process itself. The aim is to minimize a discounted cost functional. In the first part of the paper we write down an explicit filtering equation and characterize the filtering process as a Piecewise Deterministic Process. In the second part, after transforming the original control problem with partial observation into one with complete observation (the separated problem) using filtering equations, we prove the equivalence of the original and separated problems through an explicit formula linking their respective value functions. The value function of the separated problem is also characterized as the unique fixed point of a suitably defined contraction mapping
Nonlinear Filtering of Partially Observed Systems arising in Singular Stochastic Optimal Control
Full text linkThis paper deals with a nonlinear filtering problem in which a
multi-dimensional signal process is additively affected by a process
whose components have paths of bounded variation. The presence of the process
prevents from directly applying classical results and novel estimates
need to be derived. By making use of the so-called reference probability
measure approach, we derive the Zakai equation satisfied by the unnormalized
filtering process, and then we deduce the corresponding Kushner-Stratonovich
equation. Under the condition that the jump times of the process do not
accumulate over the considered time horizon, we show that the unnormalized
filtering process is the unique solution to the Zakai equation, in the class of
measure-valued processes having a square-integrable density. Our analysis paves
the way to the study of stochastic control problems where a decision maker can
exert singular controls in order to adjust the dynamics of an unobservable
It\^o-process
Risk Measures and Progressive Enlargement of Filtration: A BSDE Approach
Full text linkWe consider dynamic risk measures induced by backward stochastic differential equations (BSDEs) in an enlargement of filtration setting. On a fixed probability space, we are given a standard Brownian motion and a pair of random variables , with , that enlarge the reference filtration, i.e., the one generated by the Brownian motion. These random variables can be interpreted financially as a default time and an associated mark. After introducing a BSDE driven by the Brownian motion and the random measure associated to , we define the dynamic risk measure , for a fixed time T > 0, induced by its solution. We prove that can be decomposed in a pair of risk measures, acting before and after , and we characterize its properties giving suitable assumptions on the driver of the BSDE. Furthermore, we prove an inequality satisfied by the penalty term associated to the robust representation of and we discuss the dynamic entropic risk measure case, providing examples where it is possible to write explicitly its decomposition and simulate it numerically
HJB equations and stochastic control on half-spaces of Hilbert spaces
No full textIn this paper, we study a first extension of the theory of mild solutions for Hamilton–Jacobi–Bellman (HJB) equations in Hilbert spaces to the case where the domain is not the whole space. More precisely, we consider a half-space as domain, and a semilinear HJB equation. Our main goal is to establish the existence and the uniqueness of solutions to such HJB equations, which are continuously differentiable in the space variable. We also provide an application of our results to an exit-time optimal control problem, and we show that the corresponding value function is the unique solution to a semilinear HJB equation, possessing sufficient regularity to express the optimal control in feedback form. Finally, we give an illustrative example
HJB Equations and Stochastic Control on Half-Spaces of Hilbert Spaces
No full textIn this paper, we study a first extension of the theory of mild solutions for Hamilton–Jacobi–Bellman (HJB) equations in Hilbert spaces to the case where the domain is not the whole space. More precisely, we consider a half-space as domain, and a semilinear HJB equation. Our main goal is to establish the existence and the uniqueness of solutions to such HJB equations, which are continuously differentiable in the space variable. We also provide an application of our results to an exit-time optimal control problem, and we show that the corresponding value function is the unique solution to a semilinear HJB equation, possessing sufficient regularity to express the optimal control in feedback form. Finally, we give an illustrative example
State constrained control problems in Banach lattices and applications
Full text linkThis paper aims to study a family of deterministic optimal control problems
in infinite dimensional spaces. The peculiar feature of such problems is the
presence of a positivity state constraint, which often arises in economic
applications. To deal with such constraints, we set up the problem in a Banach
space with a Riesz space structure (i.e., a Banach lattice) not necessarily
reflexive: a typical example is the space of continuous functions on a compact
set. In this setting, which seems to be new in this context, we are able to
find explicit solutions to the Hamilton-Jacobi-Bellman (HJB) equation
associated to a suitable auxiliary problem and to write the corresponding
optimal feedback control. Thanks to a type of infinite dimensional
Perron-Frobenius Theorem, we use these results to get information about the
optimal paths of the original problem. This was not possible in the infinite
dimensional setting used in earlier works on this subject, where the state
space was an space
A simple planning problem for COVID-19 lockdown: a dynamic programming approach
Full text linkA large number of recent studies consider a compartmental SIR model to study optimal control policies aimed at containing the diffusion of COVID-19 while minimizing the economic costs of preventive measures. Such problems are non-convex and standard results need not to hold. We use a Dynamic Programming approach and prove some continuity properties of the value function of the associated optimization problem. We study the corresponding Hamilton-Jacobi-Bellman equation and show that the value function solves it in the viscosity sense. Finally, we discuss some optimality conditions. Our paper represents a first contribution towards a complete analysis of non-convex dynamic optimization problems, within a Dynamic Programming approach
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
