196,116 research outputs found

    Convergence results for an averaged LQR problem with applications to reinforcement learning

    No full text
    In this paper, we will deal with a linear quadratic optimal control problem with unknown dynamics. As a modeling assumption, we will suppose that the knowledge that an agent has on the current system is represented by a probability distribution π on the space of matrices. Furthermore, we will assume that such a probability measure is opportunely updated to take into account the increased experience that the agent obtains while exploring the environment, approximating with increasing accuracy the underlying dynamics. Under these assumptions, we will show that the optimal control obtained by solving the “average” linear quadratic optimal control problem with respect to a certain π converges to the optimal control driven related to the linear quadratic optimal control problem governed by the actual, underlying dynamics. This approach is closely related to model-based reinforcement learning algorithms where prior and posterior probability distributions describing the knowledge on the uncertain system are recursively updated. In the last section, we will show a numerical test that confirms the theoretical results

    Convergence of the Value Function in Optimal Control Problems with Unknown Dynamics

    No full text
    We deal with the convergence of the value function of an approximate control problem with uncertain dynamics to the value function of a nonlinear optimal control problem. The assumptions on the dynamics and the costs are rather general and we assume to represent uncertainty in the dynamics by a probability distribution. The proposed framework aims to describe and motivate some model-based Reinforcement Learning algorithms where the model is probabilistic. We also show some numerical experiments which confirm the theoretical results
    corecore