1,721,015 research outputs found
General Lq Problem For Infinite Jump Linear Systems And The Minimal Solution Of Algebraic Riccati Equations
No full textThe paper addresses the LQ control problem for systems with countable Markov jump parameters, and the associated coupled algebraic Riccati equations. The problem is considered in a general optimization setting in which the solution is not required to be stabilizing in any sense. We show that a necessary and sufficient condition for a solution to the control problem to exist is that the Riccati equations have a nonempty set of solutions, which generalizes previous known results requiring stabilizability as a sufficient condition.We clarify the connection between the minimal solution of the Riccati equation and the control problem, showing that the minimal solution provides the synthesis of the optimal control. The derived results strengthen the relations of the theory of Markov jump systems with the one of linear deterministic systems. An illustrative example is included. Copyright © 2005 IFAC.1616Athans, M., Castanon, D., Dunn, K.P., Greene, C.S., Lee, W.H., Sandell, N.R., Willsky, A.S., The stochastic control of the F- 8C aircraft using a multiple model adaptive control (MMAC) method - Part i: Equilibrium flight (1977) IEEE Transactions on Automatic Control, 22, pp. 768-780Costa, E.F., Do Val, J.B.R., On the detectability and observability of continuous-time Markov jump linear systems (2002) SIAM Journal on Control and Optimization, 41 (4), pp. 1295-1314Costa, E.F., Do Val, J.B.R., Weak detectability and the linear quadratic control problem of discrete-time Markov jump linear systems (2002) International Journal of Control. Special Issue on Switched and Polytopic Linear Systems, 75, pp. 1282-1292Costa, E.F., Do Val, J.B.R., An algorithm for solving a perturbed algebraic Riccati equation European Journal of Control, , To appear in theCosta, E.F., Do Val, J.B.R., Fragoso, M.D., A new approach to detectability of discrete-time infinite Markov jump linear systems SIAM Journal on Control and Optimization, , To appear in theCosta, E.F., Do Val, J.B.R., Fragoso, M.D., On a detectability concept of discrete-time infinite Markov jump linear systems Stochastic Analysis and Applications, , To appear in theCosta, O.L.V., Fragoso, M.D., Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems (1995) IEEE Transactions on Automatic Control, AC-40, pp. 2076-2088Costa, O.L.V., Marques, R.P., Robust H 2-control for discrete-time Markovian jump linear systems (2000) International Journal of Control, 73 (1), pp. 11-21Costa, O.L.V., Val, J.B.R.D., Geromel, J.C., A convex programming approach to H 2 control of discrete-time Markovian jump linear systems (1997) Int. J. Control, 66 (4), pp. 557-579Costa, O.L.V., Do Val, J.B.R., Geromel, J.C., Continuous-time state-feedback H 2-control of Markovian jump linear systems via convex analysis (1999) Automatica, 35, pp. 259-268Do Val, J.B.R., Costa, E.F., Numerical solution for the linear-quadratic control problem of Markov jump linear systems and a weak detectability concept (2002) Journal of Optimization Theory and Applications, 114 (1), pp. 69-96Do Val, J.B.R., Geromel, J.C., Costa, O.L.V., Solutions for the linear quadratic control problem of Markov jump linear systems (1999) Journal of Optimization Theory and Applications, 103, pp. 283-311Fragoso, M., Baczynski, J., Optimal control for continuous time LQ problems with infinite Markov jump parameters (2001) SIAM Journal on Control and Optimization, 40 (1), pp. 270-297Ji, Y., Chizeck, H.J., Controlability, stabilizability and continuous time Markovian jump linear quadratic control (1990) IEEE Transactions on Automatic Control, 35 (7), pp. 777-788Kucera, V., On nonnegative definite solutions to matrix quadratic equations (1972) Automatica, 8, pp. 413-423Martensson, K., On the matrix Riccati equation (1971) Inform. Sci., 3, pp. 17-49Molinari, B.P., The time-invariant linear-quadratic optimal control problem (1977) Automatica, 13, pp. 347-357Rami, M.A., Ghaoui, L.E., LMI optimization for nonstandard Riccati equations arising in stochastic control (1996) IEEE Transactions on Automatic Control, 41 (11), pp. 1666-1671Sworder, D.D., Rogers, R.O., An LQ-solution to a control problem associated with a solar thermal central receiver (1983) IEEE Transactions on Automatic Control, 28 (10), pp. 971-97
Stationary Policies For The Second Moment Stability In A Class Of Stochastic Systems
No full textThis paper presents a study on the uniform second moment stability for a class of stochastic control system. The main result states that the existence of the long-run average cost under a stationary policy is equivalent to the uniform second moment stability of the corresponding stochastic control system. To illustrate the result, a numerical example is developed to verify the uniform second moment stability of a simultaneous state-feedback control system. © 2011 IEEE.12641268Vargas, A.N., Do Val, J.B.R., Average cost and stability of time-varying linear systems (2010) IEEE Trans. Automat. Control, 55, pp. 714-720Vargas, A.N., Do Val, J.B.R., A controllability condition for the existence of average optimal stationary policies of linear stochastic systems (2009) Proc. European Control Conference, pp. 32-37. , Budapest, HungaryVargas, A.N., Do Val, J.B.R., Average optimal stationary policies: Convexity and convergence conditions in linear stochastic control systems (2009) Proc. 48th IEEE Conf. Decision Control and 28th Chinese Control Conference, pp. 3388-3393. , Shangai, ChinaVargas, A.N., Do Val, J.B.R., Minimum second moment state for the existence of average optimal stationary policies in linear stochastic systems (2010) Proc. American Control Conference, pp. 373-377. , Baltimore, MD, USAYang, X.M., Yang, X.Q., Teo, K.L., A matrix trace inequality (2001) J. Math. Anal. Appl., 263, pp. 327-331Herńandez-Lerma, O., Lasserre, J.B., (1996) Discrete-time Markov Control Processes: Basic Optimality Criteria, , Springer-Verlag, New YorkAnderson, B.D.O., Moore, J.B., (1979) Optimal Filtering, , Prentice-Hall, Englewood Cliffs, N.JKozin, F., A survey of stability of stochastic systems (1969) Automatica, 5, pp. 95-112Cho, Y., Lam, J., A computational method for simultaneous LQ optimal control design via piecewise constant output feedback (2001) IEEE Trans. Systems Man Cybernetics Part B, 31, pp. 836-842Howitt, G.D., Luus, R., Control of a collection of linear systems by linear state feedback control (1993) Int. Journal Control, 58 (1), pp. 79-96Luke, R.A., Dorato, P., Abdallah, C.T., Linear-quadratic simultaneous performance design (1997) Proc. American Control Conf., pp. 3602-3605Lavaei, J., Aghdam, A.G., Simultaneous LQ control of a set of LTI systems using constrained generalized sampled-data hold functions (2007) Automatica, 43 (2), pp. 274-280Saadatjooa, F., Derhami, V., Karbassi, S.M., Simultaneous control of linear systems by state feedback (2009) Computers Math. Appl., 58 (1), pp. 154-160Wu, J., Lee, T., Optimal static output feedback simultaneous regional pole placement (2005) IEEE Trans. Systems Man Cybernetics Part B, 35, pp. 881-893Geromel, J.C., Peres, P.L.D., Souza, S.R., H 2-guaranteed cost control for uncertain discrete-time linear systems (1993) Int. Journal of Control, 57, pp. 853-86
On The Observability And Detectability Of Continuous-time Markov Jump Linear Systems
No full textThe paper introduces a new detectability concept for continuous-time Markov jump linear systems with finite Markov space that generalizes previous concepts found in the literature. The detectability in the weak sense is characterized as mean square detectability of a certain related stochastic system, making both detectability senses directly comparable. The concept can also ensure that the solution of the coupled algebraic Riccati equation associated to the quadratic control problem is unique and stabilizing, making other concepts redundant. The paper also obtains a set of matrices that plays the role of the observability matrix for deterministic linear systems, and it allows geometric and qualitative properties. Tests for weak observability and detectability of a system are provided, the first consisting of a simple rank test, similar to the usual observability test for deterministic linear systems. The complete results are presented in [3].219941999Brewer, J.W., Kronecker products and matrix calculus in system theory (1979) IEEE Trans. Circuits Systems I Fund. Theory Appl., 25, pp. 772-781Costa, E.F., Do Val, J.B.R., On the detectability and observability of discrete-time Markov jump linear systems (2001) Systems Control Lett., 44, pp. 135-145Costa, E.F., Do Val, J.B.R., On the detectability and observability of continuous-time Markov jump linear systems (2002) SIAM J. Control Optim., 41 (4), pp. 1295-1314Costa, O.L.V., Do Val, J.B.R., Geromel, J.C., Continuous-time state-feedback H2-control of Markovian jump linear systems via convex analysis (1999) Automatica J. IFAC, 35, pp. 259-268Costa, O.L.V., Fragoso, M., Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems (1995) IEEE Trans. Automat. Control, 40, pp. 2076-2088Do Val, J.B.R., Geromel, J.C., Costa, O.L.V., Solutions for the linear quadratic control problem of Markov jump linear systems (1999) J. Optim. Theory Appl., 103, pp. 283-311Ji, Y., Chizeck, H.J., Controllability, stabilizability and continuous time Markovian jump linear quadratic control (1990) IEEE Trans. Automat. Control, 35, pp. 777-788Morozan, T., Stability and control for linear systems with jump Markov perturbations (1995) Stochastic Anal. Appl., 13, pp. 91-11
A Finite-time Stability Concept And Conditions For Finite-time And Exponential Stability Of Controlled Nonlinear Systems
No full textThis paper introduces a finite-time stability concept for nonlinear systems and the corresponding notion of stabilizability of controls. The concept generalizes previous finite-time stability as it requires that the state trajectory satisfies a given bound at some time instant in a certain interval, whereas previous notions considers that the system stays within that bound over the entire interval. Here, the bound on the trajectory may represent a region arbitrarily close to the origin, and thus it is in tune with situations where contractive trajectories are required. This feature allows us to relate the proposed concept with the usual exponential stability concept, and simultaneously, with the previous finite-time concepts, thus clarifying the relations among them. As regards to controlled systems, we derive a sufficient condition for stabilizability, with the interpretation that the size of the time interval demanded by the control to drive the trajectory into a specified region is in inverse proportion with the size of the region. Moreover, we present a simple moving horizon implementation for a stabilizing (in the new sense) control that provides an exponentially stable controlled system. For stationary controls we can connect stabilizability in the sense here with the classical exponential sense. We show that the control is exponentially stabilizing whenever it is finite-time stabilizing and stationary. An illustrative example is included. ©2006 IEEE.200623092314Amato, F., Ariola, M., Dorato, P., Finite-time control of linear systems subject to parametric uncertainties and disturbances (2001) Automatica, 37, pp. 1459-1463Amato, F., Ariola, M., Dorato, P., Finite-time control of linear systems subject to parametric uncertainties and disturbances (2001) Automatica, 37 (9), pp. 1459-1463Anderson, B.D.O., Moore, J.B., Detectability and stabilizability of time-varying discrete-time linear systems (1981) SIAM Journal on Control and Optimization, 19 (1), pp. 20-32Bhat, S.P., Bernstein, D.S., Finite-time stability of continuous autonomous systems (2000) SIAM J. Control Optim, 38 (3), pp. 751-766Costa, E.F., do Val, J.B.R., On the detectability and observability of continuous-time Markov jump linear systems (2002) SIAM Journal on Control and Optimization, 41 (4), pp. 1295-1314Costa, E.F., do Val, J.B.R., Optimal cost convergence with respect to the time horizon (2003) ECC'03 European Control Conference, pp. 1-6. , Cambridge, United KingdomCosta, E.F., do Val, J.B.R., Stability of receding horizon control of nonlinear systems (2003) 42st IEEE Conference on Decision and Control, pp. 2077-2081. , Maui, Hawaii, USA, IEEECosta, E.F., do Val, J.B.R., Obtaining stabilizing stationary controls via finite horizon cost (2006) Submitted to the ACC'06 American Control ConferenceCosta, E.F., do Val, J.B.R., Fragoso, M.D., A new approach to detectability of discrete-time infinite Markov jump linear systems (2005) SIAM Journal on Control and Optimization, 43, pp. 2132-2156Dorato, P., Short time stability in linear time-varying systems (1961) IRE International Convention Record, Part 4, 83-87Hager, W.W., Horowitz, L.L., Convergence and stability properties of the discrete Riccati operator equation and the associated optimal control and filtering problems (1976) SIAM Journal on Control and Optimization, 14 (2), pp. 295-312Hong, Y., Huang, J., Xu, Y., On an output feedback finite-time stabilization problem (2001) IEEE Transactions on Automatic Control, 46 (2), pp. 305-309Mastellone, S., Dorato, P., Abdallah, C.T., Finite-time stochastic stability of discrete-time nonlinear systems: Analysis and design (2004) 43th Conference on Decision and ControlSastry, S., (1999) Nonlinear Systems: Analysis, stability and control, , Springer New YorkSathananthan, S., Keel, L.H., Optimal practical stabilization and controllability of systems with Markovian jumps (2003) Nonlinear Analysis, 54 (6), pp. 1011-1027Weiss, L., Infante, E.F., Finite time stability under perturbating forces and on product spaces (1967) IEEE Trans. on Automatic Control, 1, pp. 54-5
Weak Detectability And The Lq Problem Of Discrete-time Infinite Markov Jump Linear Systems
No full textThe paper deals with a concept of weak detectability for discrete-tine infinite Markov jump linear systems, which relates the stochastic convergence of the output with the stochastic convergence of the state and generalizes previous concepts. Certain invariant sets are introduced, which allow us to find a related system that is stochastically detectable if and only if the original system is weakly detectable. This provides the necessary tools to show, via an additional assumption, that the weak detectability concept is invariant with respect to linear state feedback control. As an immediate extension, the result provides that linear state feedback controls are stabilizing whenever the associated cost functional is bounded. In addition, it is shown that the detectability concept assures that the solution of the JLQ is stabilizing and the solution of the associated algebraic Riccati equation is unique and stabilizing, thus retrieving the usual role that detectability concepts play in finite dimensional MJLS and linear deterministic systems. Finally, regarding the assumption, the paper shows that: it is not related to the detectability concept, it always holds for finite dimensional Markov jump linear systems, and it holds under a condition of uniform observability on trajectories associated with non-convergent output.657895794Costa, E.F., Do Val, J.B.R., On the detectability and observability of discrete-time Markov jump linear systems (2001) System Control Lett., 44, pp. 135-145Costa, E.F., Do Val, J.B.R., On the detectability and observability of continuous-time Markov jump linear systems (2002) SIAM J. Control Optim., 41 (4), pp. 1295-1314Costa, E.F., Do Val, J.B.R., Weak detectability and the linear quadratic control problem of discrete-time Markov jump linear systems (2002) Internat. J. Control, 75 (16-17), pp. 1282-1292Costa, E.F., Do Val, J.B.R., Fragoso, M.D., On a detectability concept of discrete-time infinite Markov jump linear systems (2002) IFAC 15th World Congress, pp. 2660-2665. , BarcelonaCosta, O.L.V., Fragoso, M.D., Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems (1995) IEEE Trans. Automat. Control, AC-40, pp. 2076-2088Fragoso, M., Baczynski, J., Optimal control for continuous time LQ problems with infinite Markov jump parameters (2001) SIAM J. Control Optim., 40 (1), pp. 270-297Fragoso, M.D., Baczynski, J., Stochastic versus mean square stability in continuous time linear infinite Markov jump parameter systems (2002) Stochastic Anal. Appl., 20 (20), pp. 347-356Golub, G.H., Loan, C.V., (1996) Matrix Computation, , Johns Hopkins Press, London, third editio
Finite Approximation Of The Optimal Average Cost For A Class Of Stochastic Control Systems
No full textThis paper provides conditions for which the optimal finite-stage cost, divided by the number of stages, converges to the optimal long-run average cost as the number of stages goes to infinity. The main condition is based on a controllability to the origin property. The discrete-time stochastic system is linear with respect to the system state but the control possess a general structure, possibly nonlinear. To illustrate the effectiveness of the result, an application to the simultaneous state-feedback control problem is considered. © 2011 IFAC.18PART 11242712431Anderson, B.D.O., Moore, J.B., (1979) Optimal Filtering, , Prentice-Hall, Englewood Cliffs, N.JArapostathis, A., Borkar, V.S., Fernández-Gaucherand, E., Ghosh, M.K., Marcus, S.I., Discrete-time controlled Markov processes with average cost criterion: A survey (1993) SIAM J. Control Optim., 31 (2), pp. 282-344Bertsekas, D.P., Shreve, S.E., (1978) Stochastic Optimal Control: The Discrete Time Case, , Academic PressCho, Y.-Y., Lam, J., A computational method for simultaneous LQ optimal control design via piecewise constant output feedback (2001) IEEE Trans. Systems Man Cybernetics Part B, 31, pp. 836-842Do Val, J.B.R., Başar, T., Receding horizon control of jump linear systems and a macroeconomic policy problem (1999) J. Econom. Dynam. Control, 23, pp. 1099-1131Geromel, J.C., Peres, P.L.D., Souza, S.R., H 2- guaranteed cost control for uncertain discrete-time linear systems. Int (1993) Journal of Control, 57, pp. 853-864Hernández-Lerma, O., Lasserre, J.B., (1996) Discrete-Time Markov Control Processes: Basic Optimality Criteria, , Springer-Verlag, New YorkHowitt, G.D., Luus, R., Control of a collection of linear systems by linear state feedback control (1993) Int. Journal Control, 58 (1), pp. 79-96Lavaei, J., Aghdam, A.G., Simultaneous LQ control of a set of LTI systems using constrained generalized sampled-data hold functions (2007) Automatica, 43 (2), pp. 274-280Luke, R.A., Dorato, P., Abdallah, C.T., Linear-quadratic simultaneous performance design (1997) Proc. American Control Conf., pp. 3602-3605Meyn, S.P., The policy iteration algorithm for average reward Markov decision processes with general state space (1997) IEEE Trans. Automat. Control, 42 (12), pp. 1663-1680Saadatjooa, F., Derhami, V., Karbassi, S.M., Simultaneous control of linear systems by state feedback (2009) Computers Math. Appl., 58 (1), pp. 154-160Schal, M., Average optimality in dynamic programming with general state space (1993) Math. Oper. Res., 18, pp. 163-172Sennott, L.I., The convergence of value iteration in average cost Markov decision chains (1996) Oper. Res. Lett., 19, pp. 11-16Vargas, A.N., Do Val, J.B.R., Minimum second moment state for the existence of average optimal stationary policies in linear stochastic systems (2010) Proc. American Control Conference, pp. 373-377. , Baltimore, MD, USAVargas, A.N., Do Val, J.B.R., Average optimal stationary policies: Convexity and convergence conditions in linear stochastic control systems (2009) Proc. 48th IEEE Conf. Decision Control and 28th Chinese Control Conference, pp. 3388-3393. , Shangai, ChinaVargas, A.N., Do Val, J.B.R., A controllability condition for the existence of average optimal stationary policies of linear stochastic systems (2009) Proc. European Control Conference, pp. 32-37. , Budapest, HungaryVargas, A.N., Do Val, J.B.R., Costa, E.F., Receding horizon control of Markov jump linear systems subject to noise and unobservable state chain (2004) Proc. 43th IEEE Conf. Decision Control, pp. 4381-4386Wu, J.-L., Lee, T.-T., Optimal static output feedback simultaneous regional pole placement (2005) IEEE Trans. Systems Man Cybernetics Part B, 35, pp. 881-89
Obtaining Stabilizing Stationary Controls Via Finite Horizon Cost
No full textThis paper focus on the stabilizing properties of stationary feedback controls for general nonlinear systems that are obtained by minimizing a finite horizon cost, in a receding horizon control basis. The main result is to establish exponential stability for stationary controls obtained from minimization of sufficiently large but finite time horizon cost. The approach requires a previously defined notion of closed-loop detectability of nonlinear systems, and in the present paper we introduce conditions under which the aforementioned detectability sense is verified from the open-loop system data, as is usual in linear systems. In connection, we verify that stabilizable and detectable linear time-invariant systems satisfy each of the work assumptions. © 2006 IEEE.200642974302Anderson, B.D.O., Moore, J.B., Detectability and stabilizability of time-varying discrete-time linear systems (1981) SIAM Journal on Control and Optimization, 19 (1), pp. 20-32Costa, E.F., do Val, J.B.R., Optimal cost convergence with respect to the time horizon (2003) ECC'03 European Control Conference, pp. 1-6. , Cambridge, United KingdomCosta, E.F., do Val, J.B.R., Stability of receding horizon control of nonlinear systems (2003) 42st IEEE Conference on Decision and Control, pp. 2077-2081. , Maui, Hawaii, USA, IEEECosta, E.F., do Val, J.B.R., A finite-time stability concept and conditions for finite-time and exponential stability of controlled nonlinear systems (2006) Submitted to the ACC'06 American Control ConferenceDe Nicolao, G., Strada, S., On the stability of receding-horizon LQ Control with zero-state terminal contraint (1997) IEEE Transactions on Automatic Control, 42 (2), pp. 257-260Hager, W.W., Horowitz, L.L., Convergence and stability properties of the discrete Riccati operator equation and the associated optimal control and filtering problems (1976) SIAM Journal on Control and Optimization, 14 (2), pp. 295-312Ito, K., Kunisch, K., Asymptotic properties of receding horizon optimal control problems (2002) Siam Journal on Control and Optimization, 40 (5), pp. 1585-1610Jadbabaie, A., Hauser, J., On the stablity of unconstrained receding horizon control with a general terminal cost (2001) Conference on Decicion and Control 2001Kailath, T., (1980) Linear Systems, , Prentice-HallKothare, M.V., Balakrishnan, V., Morari, M., Robust constrained model predictive control using linear matrix inequalities (1996) Automatica, 32 (10), pp. 1361-1379Lee, J.W., Kwon, W.H., Choi, J., On stability of constrained receding horizon control with finite terminal weighting matrix (1998) Automatica, 34 (12), pp. 1607-1612Lee, Y.I., Kouvaritakis, B., Constrained receding horizon predictive control for systems with disturbances (1999) International Journal of Control, 72 (11), pp. 1027-1032Mayne, D.Q., Michalska, H., Receeding horizon control of nonlinear systems (1990) IEEE Transactions on Automatic Control, 35, pp. 814-824Mosca, E., Zhang, J., Stable redesign of predictive control (1992) Automatica, 28 (6), pp. 1229-1233Primbs, J.A., Nevistic, V., A new approach to stability analysis for constrained finite receding horizon control without end constraints (2000) IEEE Transactions on Automatic Control, 45, pp. 1507-1512Rawlings, J.B., Muske, K.R., The stability of constrained receding horizon control (1993) IEEE Transactions on Automatic Control, 38 (10), pp. 1512-151
Weak Controllability And Weak Stabilizability Concepts For Linear Systems With Markov Jump Parameters
No full textThe paper introduces weak controllability and weak stabilizability concepts for discrete-time Markov jump linear system with finite Markov space. We introduce a collection of matrices ℂ that resembles controllability matrices of deterministic linear systems. The collection of matrices ℂ allows us to define a weak controllability concept by requiring that the matrices are full rank, as well as to introduce a weak stabilizability concept that is a dual of the weak detectability concept found in the literature of Markov jump systems. An important feature of the introduced concept is that it generalizes the concept of mean square stabilizability. The role that this concept plays in the scenario of the filtering problem is investigated through case studies, which suggest that weak stabilizability together with mean square detectability ensure that the state estimator is mean square stable. Illustrative examples are included. © 2006 IEEE.2006905910Costa, E.F., do Val, J.B.R., On the detectability and observability for continuous-time Markov jump linear systems (2002) SIAM Journal on Control and Optimization, 41 (4), pp. 1295-1314Costa, E.F., do Val, J.B.R., Weak detectability and the linear-quadratic control problem of discrete-time Markov jump linear systems (2002) International Journal of Control, 75, pp. 1282-1292Costa, E.F., do Val, J.B.R., Fragoso, M.D., A new approach to detectability of discrete-time infinite Markov jump linear systems (2005) SIAM Journal on Control and Optimization, Philadelphia, 43 (6), pp. 2132-2156Costa, E.F., do Val, J.B.R., Fragoso, M.D., On a detectability concept of discrete-time infinite Markov jump linear systems (2005) Stochastic Analysis and Applications, 23 (1), pp. 1-5Costa, O.L.V., Fragoso, M.D., Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems (1995) IEEE Transactions on Automatic Control, AC-40, pp. 2076-2088Costa, O.L.V., Tuesta, E.F., H2-control and the separation principle for discrete-time Markovian jump linear systems (2004) Mathematics of Control, Signals and Systems, 16, pp. 320-350Costa, O.L.V., Fragoso, M.D., Stability results for discrete-time linear systems with Markovian jumping parameters (1993) Journal of Mathematical Analysis and Applications, 179, pp. 154-178Fragoso, M., Baczynski, J., Optimal control for continuous time LQ problems with infinite Markov jump parameters (2001) SIAM Journal on Control and Optimization, 40 (1), pp. 270-297Ji, Y., Chizeck, H.J., Jump linear quadratic Gaussian control: Steady-state solution and testable conditions (1990) Control Theory and Advanced Technology, 6 (3), pp. 289-319Ji, Y., Chizeck, H.J., Jump linear quadratic gaussian conrol in continuous time (1992) IEEE Transactions on Automatic Control, 37 (12), pp. 1884-1892Ji, Y., Chizeck, H.J., Feng, X., Loparo, K.A., Stability and control of discrete-time jump linear systems (1991) Control Theory and Advanced Tecnology, 7 (2), pp. 247-269Morozan, T., Stability and control for linear systems with jump Markov perturbations (1995) Stochastic Analysis and Applications, 13 (1), pp. 91-11
Approximate dynamic programming via direct search in the space of value function approximations
No full textThis paper deals with approximate value iteration (AVI) algorithms applied to discounted dynamic programming (DP) problems. For a fixed control policy, the span semi-norm of the so-called Bellman residual is shown to be convex in the Banach space of candidate solutions to the DP problem. This fact motivates the introduction of an AVI algorithm with local search that seeks to minimize the span semi-norm of the Bellman residual in a convex value function approximation space. The novelty here is that the optimality of a point in the approximation architecture is characterized by means of convex optimization concepts and necessary and sufficient conditions to local optimality are derived. The procedure employs the classical AVI algorithm direction (Bellman residual) combined with a set of independent search directions, to improve the convergence rate. It has guaranteed convergence and satisfies, at least, the necessary optimality conditions over a prescribed set of directions. To illustrate the method, examples are presented that deal with a class of problems from the literature and a large state space queueing problem setting.</p
Quadratic Costs And Second Moments Of Jump Linear Systems With General Markov Chain
No full textThis paper presents an analytic, systematic approach to handle quadratic functionals associated with Markov jump linear systems with general jumping state. The Markov chain is finite state, but otherwise general, possibly reducible and periodic. We study how the second moment dynamics are affected by the additive noise and the asymptotic behaviour, either oscillatory or invariant, of the Markov chain. The paper comprises a series of evaluations that lead to a tight two-sided bound for quadratic cost functionals. A tight two-sided bound for the norm of the second moment of the system is also obtained. These bounds allow us to show that the long-run average cost is well defined for system that are stable in the mean square sense, in spite of the periodic behaviour of the chain and taking into consideration that it may not be unique, as it may depend on the initial distribution. We also address the important question of approximation of the long-run average cost via adherence of finite horizon costs. © 2011 Springer-Verlag London Limited.2301/03/15141157Costa, O.L.V., Fragoso, M.D., Stability results for discrete-time linear systems with Markovian jumping parameters (1993) J Math Anal Appl, 179, pp. 154-178. , 1244955 0790.93108 10.1006/jmaa.1993.1341Ji, Y., Chizeck, H.J., Jump linear quadratic Gaussian control: Steady-state solution and testable conditions (1990) Control Theory Adv Technol, 6 (3), pp. 289-319. , 1080011Costa, O.L.V., Fragoso, M.D., Marques, R.P., (2005) Discrete-time Markovian Jump Linear Systems, , Springer New YorkZampolli, F., Optimal monetary policy in a regime-switching economy: The response to abrupt shifts in exchange rate dynamics (2006) Journal of Economic Dynamics and Control, 30 (9-10), pp. 1527-1567. , DOI 10.1016/j.jedc.2005.10.013, PII S0165188906000522, Computing in Economics and FinanceCosta, O.L.V., De Paulo, W.L., Indefinite quadratic with linear costs optimal control of Markov jump with multiplicative noise systems (2007) Automatica, 43 (4), pp. 587-597. , DOI 10.1016/j.automatica.2006.10.022, PII S0005109806004584Khanbaghi, M., Malhame, R.P., Perrier, M., Optimal white water and broke recirculation policies in paper mills via jump linear quadratic control (2002) IEEE Transactions on Control Systems Technology, 10 (4), pp. 578-588. , DOI 10.1109/TCST.2002.1014677, PII S1063653602053599Hernández-Lerma, O., Lasserre, J.B., (1996) Discrete-time Markov Control Processes: Basic Optimality Criteria, , Springer New YorkArapostathis, A., Borkar, V., Fernandez-Gaucherand, E., Ghosh, M.K., Marcus, S.I., Discrete-time controlled Markov processes with average cost criterion: A survey (1993) SIAM Journal on Control and Optimization, 31 (2), pp. 282-344Grimm, G., Messina, M.J., Tuna, S.E., Teel, A.R., Model predictive control: For want of a local control Lyapunov function, all is not lost (2005) IEEE Transactions on Automatic Control, 50 (5), pp. 546-558. , DOI 10.1109/TAC.2005.847055Costa, E.F., Do Val, J.B.R., Uniform approximation of infinite horizon control problems for nonlinear systems and stability of the approximating controls (2009) IEEE Trans Automat Control, 54 (4), pp. 881-886. , 2514828 10.1109/TAC.2008.2010970Costa, E.F., Do Val, J.B.R., Obtaining stabilizing stationary controls via finite horizon cost (2006) Proceedings of the American Control Conference, 2006, pp. 4297-4302. , 1657394, Proceedings of the 2006 American Control ConferenceJadbabaie, A., Hauser, J., On the stability of receding horizon control with a general terminal cost (2005) IEEE Transactions on Automatic Control, 50 (5), pp. 674-678. , DOI 10.1109/TAC.2005.846597Costa, E.F., Do Val, J.B.R., Fragoso, M.D., A new approach to detectability of discrete-time infinite Markov jump linear systems (2005) SIAM Journal on Control and Optimization, 43 (6), pp. 2132-2156. , DOI 10.1137/S036301290342992XÇinlar, E., (1975) Introduction to Stochastic Processes, , Prentice Hall New York 0341.60019Costa, O.L.V., Fragoso, M.D., Discrete-time LQ-optimal control problems for finite Markov jump parameters systems (1995) IEEE Trans Automat Control, 40, pp. 2076-2088. , 1364955 0843.93091 10.1109/9.47832
- …
