Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Linear–quadratic–Gaussian control
open-in-new
<h2 id="mathematical-description-of-the-problem-and-solution">Mathematical description of the problem and solution</h2> <h3>Continuous time</h3> <p>Consider the <a href="/facts/Continuous-time/RR8wpS9e">continuous-time</a> linear dynamic system </p> x ˙ ( t ) = A ( t ) x ( t ) + B ( t ) u ( t ) + v ( t ) , {\displaystyle {\dot {\mathbf {x} }}(t)=A(t)\mathbf {x} (t)+B(t)\mathbf {u} (t)+\mathbf {v} (t),} y ( t ) = C ( t ) x ( t ) + w ( t ) , {\displaystyle \mathbf {y} (t)=C(t)\mathbf {x} (t)+\mathbf {w} (t),} <p>where x {\displaystyle {\mathbf {x} }} represents the vector of state variables of the system, u {\displaystyle {\mathbf {u} }} the vector of control inputs and y {\displaystyle {\mathbf {y} }} the vector of measured outputs available for feedback. Both additive white Gaussian system noise v ( t ) {\displaystyle \mathbf {v} (t)} and additive white Gaussian measurement noise w ( t ) {\displaystyle \mathbf {w} (t)} affect the system. Given this system the objective is to find the control input history u ( t ) {\displaystyle {\mathbf {u} }(t)} which at every time t {\displaystyle {\mathbf {} }t} may depend linearly only on the past measurements y ( t ′ ) , 0 ≤ t ′ < t {\displaystyle {\mathbf {y} }(t'),0\leq t'<t} such that the following cost function is minimized: </p> J = E [ x T ( T ) F x ( T ) + ∫ 0 T x T ( t ) Q ( t ) x ( t ) + u T ( t ) R ( t ) u ( t ) d t ] , {\displaystyle J=\mathbb {E} \left[{\mathbf {x} ^{\mathrm {T} }}(T)F{\mathbf {x} }(T)+\int _{0}^{T}{\mathbf {x} ^{\mathrm {T} }}(t)Q(t){\mathbf {x} }(t)+{\mathbf {u} ^{\mathrm {T} }}(t)R(t){\mathbf {u} }(t)\,dt\right],} F ≥ 0 , Q ( t ) ≥ 0 , R ( t ) > 0 , {\displaystyle F\geq 0,\quad Q(t)\geq 0,\quad R(t)>0,} <p>where E {\displaystyle \mathbb {E} } denotes the <a href="/facts/Expected_value/1XV0JKL8">expected value</a>. The final time (horizon) T {\displaystyle {\mathbf {} }T} may be either finite or infinite. If the horizon tends to infinity the first term x T ( T ) F x ( T ) {\displaystyle {\mathbf {x} }^{\mathrm {T} }(T)F{\mathbf {x} }(T)} of the cost function becomes negligible and irrelevant to the problem. Also to keep the costs finite the cost function has to be taken to be J / T {\displaystyle {\mathbf {} }J/T} . </p><p>The LQG controller that solves the LQG control problem is specified by the following equations: </p> x ^ ˙ ( t ) = A ( t ) x ^ ( t ) + B ( t ) u ( t ) + L ( t ) ( y ( t ) − C ( t ) x ^ ( t ) ) , x ^ ( 0 ) = E [ x ( 0 ) ] , {\displaystyle {\dot {\hat {\mathbf {x} }}}(t)=A(t){\hat {\mathbf {x} }}(t)+B(t){\mathbf {u} }(t)+L(t)\left({\mathbf {y} }(t)-C(t){\hat {\mathbf {x} }}(t)\right),\quad {\hat {\mathbf {x} }}(0)=\mathbb {E} \left[{\mathbf {x} }(0)\right],} u ( t ) = − K ( t ) x ^ ( t ) . {\displaystyle {\mathbf {u} }(t)=-K(t){\hat {\mathbf {x} }}(t).} <p>The matrix L ( t ) {\displaystyle {\mathbf {} }L(t)} is called the Kalman gain of the associated <a href="/facts/Kalman_filter/wNK7rnbk">Kalman filter</a> represented by the first equation. At each time t {\displaystyle {\mathbf {} }t} this filter generates estimates x ^ ( t ) {\displaystyle {\hat {\mathbf {x} }}(t)} of the state x ( t ) {\displaystyle {\mathbf {x} }(t)} using the past measurements and inputs. The Kalman gain L ( t ) {\displaystyle {\mathbf {} }L(t)} is computed from the matrices A ( t ) , C ( t ) {\displaystyle {\mathbf {} }A(t),C(t)} , the two intensity matrices V ( t ) , W ( t ) {\displaystyle \mathbf {} V(t),W(t)} associated to the white Gaussian noises v ( t ) {\displaystyle \mathbf {v} (t)} and w ( t ) {\displaystyle \mathbf {w} (t)} and finally E [ x ( 0 ) x T ( 0 ) ] {\displaystyle \mathbb {E} \left[{\mathbf {x} }(0){\mathbf {x} }^{\mathrm {T} }(0)\right]} . These five matrices determine the Kalman gain through the following associated matrix Riccati differential equation: </p> P ˙ ( t ) = A ( t ) P ( t ) + P ( t ) A T ( t ) − P ( t ) C T ( t ) W − 1 ( t ) C ( t ) P ( t ) + V ( t ) , {\displaystyle {\dot {P}}(t)=A(t)P(t)+P(t)A^{\mathrm {T} }(t)-P(t)C^{\mathrm {T} }(t){\mathbf {} }W^{-1}(t)C(t)P(t)+V(t),} P ( 0 ) = E [ x ( 0 ) x T ( 0 ) ] . {\displaystyle P(0)=\mathbb {E} \left[{\mathbf {x} }(0){\mathbf {x} }^{\mathrm {T} }(0)\right].} <p>Given the solution P ( t ) , 0 ≤ t ≤ T {\displaystyle P(t),0\leq t\leq T} the Kalman gain equals </p> L ( t ) = P ( t ) C T ( t ) W − 1 ( t ) . {\displaystyle {\mathbf {} }L(t)=P(t)C^{\mathrm {T} }(t)W^{-1}(t).} <p>The matrix K ( t ) {\displaystyle {\mathbf {} }K(t)} is called the feedback gain matrix. This matrix is determined by the matrices A ( t ) , B ( t ) , Q ( t ) , R ( t ) {\displaystyle {\mathbf {} }A(t),B(t),Q(t),R(t)} and F {\displaystyle {\mathbf {} }F} through the following associated matrix Riccati differential equation: </p> − S ˙ ( t ) = A T ( t ) S ( t ) + S ( t ) A ( t ) − S ( t ) B ( t ) R − 1 ( t ) B T ( t ) S ( t ) + Q ( t ) , {\displaystyle -{\dot {S}}(t)=A^{\mathrm {T} }(t)S(t)+S(t)A(t)-S(t)B(t)R^{-1}(t)B^{\mathrm {T} }(t)S(t)+Q(t),} S ( T ) = F . {\displaystyle {\mathbf {} }S(T)=F.} <p>Given the solution S ( t ) , 0 ≤ t ≤ T {\displaystyle {\mathbf {} }S(t),0\leq t\leq T} the feedback gain equals </p> K ( t ) = R − 1 ( t ) B T ( t ) S ( t ) . {\displaystyle {\mathbf {} }K(t)=R^{-1}(t)B^{\mathrm {T} }(t)S(t).} <p>Observe the similarity of the two matrix Riccati differential equations, the first one running forward in time, the second one running backward in time. This similarity is called duality. The first matrix Riccati differential equation solves the linear–quadratic estimation problem (LQE). The second matrix Riccati differential equation solves the <a href="/facts/Linear-quadratic_regulator/8YybEFpU">linear–quadratic regulator</a> problem (LQR). These problems are dual and together they solve the linear–quadratic–Gaussian control problem (LQG). So the LQG problem separates into the LQE and LQR problem that can be solved independently. Therefore, the LQG problem is called separable. </p><p>When A ( t ) , B ( t ) , C ( t ) , Q ( t ) , R ( t ) {\displaystyle {\mathbf {} }A(t),B(t),C(t),Q(t),R(t)} and the noise intensity matrices V ( t ) {\displaystyle \mathbf {} V(t)} , W ( t ) {\displaystyle \mathbf {} W(t)} do not depend on t {\displaystyle {\mathbf {} }t} and when T {\displaystyle {\mathbf {} }T} tends to infinity the LQG controller becomes a time-invariant dynamic system. In that case the second matrix Riccati differential equation may be replaced by the associated <a href="/facts/Algebraic_Riccati_equation/lVWtEIQ9">algebraic Riccati equation</a>. </p> <h3>Discrete time</h3> <p>Since the <a href="/facts/Discrete-time/RR8wpS9e">discrete-time</a> LQG control problem is similar to the one in continuous-time, the description below focuses on the mathematical equations. </p><p>The discrete-time linear system equations are </p> x i + 1 = A i x k + B i u i + v i , {\displaystyle {\mathbf {x} }_{i+1}=A_{i}\mathbf {x} _{k}+B_{i}\mathbf {u} _{i}+\mathbf {v} _{i},} y i = C i x i + w i . {\displaystyle \mathbf {y} _{i}=C_{i}\mathbf {x} _{i}+\mathbf {w} _{i}.} <p>Here i {\displaystyle \mathbf {} i} represents the discrete time index and v i , w i {\displaystyle \mathbf {v} _{i},\mathbf {w} _{i}} represent discrete-time Gaussian white noise processes with covariance matrices V i , W i {\displaystyle \mathbf {} V_{i},W_{i}} , respectively, and are independent of each other. </p><p>The quadratic cost function to be minimized is </p> J = E [ x N T F x N + ∑ i = 0 N − 1 ( x i T Q i x i + u i T R i u i ) ] , {\displaystyle J=\mathbb {E} \left[{\mathbf {x} }_{N}^{\mathrm {T} }F{\mathbf {x} }_{N}+\sum _{i=0}^{N-1}(\mathbf {x} _{i}^{\mathrm {T} }Q_{i}\mathbf {x} _{i}+\mathbf {u} _{i}^{\mathrm {T} }R_{i}\mathbf {u} _{i})\right],} F ≥ 0 , Q i ≥ 0 , R i > 0. {\displaystyle F\geq 0,Q_{i}\geq 0,R_{i}>0.\,} <p>The discrete-time LQG controller is </p> x ^ i + 1 = A i x ^ i + B i u i + L i + 1 ( y i + 1 − C i + 1 { A i x ^ i + B i u i } ) , x ^ 0 = E [ x 0 ] {\displaystyle {\hat {\mathbf {x} }}_{i+1}=A_{i}{\hat {\mathbf {x} }}_{i}+B_{i}{\mathbf {u} }_{i}+L_{i+1}\left({\mathbf {y} }_{i+1}-C_{i+1}\left\{A_{i}{\hat {\mathbf {x} }}_{i}+B_{i}\mathbf {u} _{i}\right\}\right),\qquad {\hat {\mathbf {x} }}_{0}=\mathbb {E} [{\mathbf {x} }_{0}]} , u i = − K i x ^ i . {\displaystyle \mathbf {u} _{i}=-K_{i}{\hat {\mathbf {x} }}_{i}.\,} <p>and x ^ i {\displaystyle {\hat {\mathbf {x} }}_{i}} corresponds to the predictive estimate x ^ i = E [ x i | y i , u i − 1 ] {\displaystyle {\hat {\mathbf {x} }}_{i}=\mathbb {E} [\mathbf {x} _{i}|\mathbf {y} ^{i},\mathbf {u} ^{i-1}]} . </p><p>The Kalman gain equals </p> L i = P i C i T ( C i P i C i T + W i ) − 1 , {\displaystyle {\mathbf {} }L_{i}=P_{i}C_{i}^{\mathrm {T} }(C_{i}P_{i}C_{i}^{\mathrm {T} }+W_{i})^{-1},} <p>where P i {\displaystyle {\mathbf {} }P_{i}} is determined by the following matrix Riccati difference equation that runs forward in time: </p> P i + 1 = A i ( P i − P i C i T ( C i P i C i T + W i ) − 1 C i P i ) A i T + V i , P 0 = E [ ( x 0 − x ^ 0 ) ( x 0 − x ^ 0 ) T ] . {\displaystyle P_{i+1}=A_{i}\left(P_{i}-P_{i}C_{i}^{\mathrm {T} }\left(C_{i}P_{i}C_{i}^{\mathrm {T} }+W_{i}\right)^{-1}C_{i}P_{i}\right)A_{i}^{\mathrm {T} }+V_{i},\qquad P_{0}=\mathbb {E} [\left({\mathbf {x} }_{0}-{\hat {\mathbf {x} }}_{0}\right)\left({\mathbf {x} }_{0}-{\hat {\mathbf {x} }}_{0}\right)^{\mathrm {T} }].} <p>The feedback gain matrix equals </p> K i = ( B i T S i + 1 B i + R i ) − 1 B i T S i + 1 A i {\displaystyle {\mathbf {} }K_{i}=(B_{i}^{\mathrm {T} }S_{i+1}B_{i}+R_{i})^{-1}B_{i}^{\mathrm {T} }S_{i+1}A_{i}} <p>where S i {\displaystyle {\mathbf {} }S_{i}} is determined by the following matrix Riccati difference equation that runs backward in time: </p> S i = A i T ( S i + 1 − S i + 1 B i ( B i T S i + 1 B i + R i ) − 1 B i T S i + 1 ) A i + Q i , S N = F . {\displaystyle S_{i}=A_{i}^{\mathrm {T} }\left(S_{i+1}-S_{i+1}B_{i}\left(B_{i}^{\mathrm {T} }S_{i+1}B_{i}+R_{i}\right)^{-1}B_{i}^{\mathrm {T} }S_{i+1}\right)A_{i}+Q_{i},\quad S_{N}=F.} <p>If all the matrices in the problem formulation are time-invariant and if the horizon N {\displaystyle {\mathbf {} }N} tends to infinity the discrete-time LQG controller becomes time-invariant. In that case the matrix Riccati difference equations may be replaced by their associated discrete-time <a href="/facts/Algebraic_Riccati_equation/lVWtEIQ9">algebraic Riccati equations</a>. These determine the time-invariant linear–quadratic estimator and the time-invariant <a href="/facts/Linear-quadratic_regulator/8YybEFpU">linear–quadratic regulator</a> in discrete-time. To keep the costs finite instead of J {\displaystyle {\mathbf {} }J} one has to consider J / N {\displaystyle {\mathbf {} }J/N} in this case. </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Stochastic_control/ETvJLl24">Stochastic control</a></li> <li><a href="/facts/Separation_principle_in_stochastic_control/6QUrIBAn">Separation principle in stochastic control</a></li> <li><a href="/facts/Witsenhausen%2527s_counterexample/4m2m7T00">Witsenhausen's counterexample</a></li></ul> <h2 id="further-reading">Further reading</h2> <ul><li>Stengel, Robert F. (1994). <a href="https://books.google.com/books?id=jDjPxqm7Lw0C"><i>Optimal Control and Estimation</i></a>. New York: Dover. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-486-68200-5.</li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Karl Johan Astrom (1970). Introduction to Stochastic Control Theory. Vol. 58. Academic Press. ISBN 0-486-44531-3. <a href="0-486-44531-3" target="_blank">0-486-44531-3</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Anders Lindquist (1973). "On Feedback Control of Linear Stochastic Systems". SIAM Journal on Control. 11 (2): 323–343. doi:10.1137/0311025.. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Tryphon T. Georgiou and Anders Lindquist (2013). "The Separation Principle in Stochastic Control, Redux". IEEE Transactions on Automatic Control. 58 (10): 2481–2494. arXiv:1103.3005. doi:10.1109/TAC.2013.2259207. S2CID 12623187. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Van Willigenburg L.G.; De Koning W.L. (2000). "Numerical algorithms and issues concerning the discrete-time optimal projection equations". European Journal of Control. 6 (1): 93–100. doi:10.1016/s0947-3580(00)70917-4. Associated software download from Matlab Central. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Van Willigenburg L.G.; De Koning W.L. (1999). "Optimal reduced-order compensators for time-varying discrete-time systems with deterministic and white parameters". Automatica. 35: 129–138. doi:10.1016/S0005-1098(98)00138-1. Associated software download from Matlab Central. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Zigic D.; Watson L.T.; Collins E.G.; Haddad W.M.; Ying S. (1996). "Homotopy methods for solving the optimal projection equations for the H2 reduced order model problem". International Journal of Control. 56 (1): 173–191. doi:10.1080/00207179208934308. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Collins Jr. E.G; Haddad W.M.; Ying S. (1996). "A homotopy algorithm for reduced-order dynamic compensation using the Hyland-Bernstein optimal projection equations". Journal of Guidance, Control, and Dynamics. 19 (2): 407–417. doi:10.2514/3.21633. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Hyland D.C; Bernstein D.S. (1984). "The optimal projection equations for fixed order dynamic compensation" (PDF). IEEE Transactions on Automatic Control. AC-29 (11): 1034–1037. doi:10.1109/TAC.1984.1103418. hdl:2027.42/57875. <a href="https://deepblue.lib.umich.edu/bitstream/2027.42/57875/1/OptimalProjectionRedOrdDynCompTAC1984.pdf" target="_blank">https://deepblue.lib.umich.edu/bitstream/2027.42/57875/1/OptimalProjectionRedOrdDynCompTAC1984.pdf</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Bernstein D.S.; Davis L.D.; Hyland D.C. (1986). "The optimal projection equations for reduced-order discrete-time modeling estimation and control" (PDF). Journal of Guidance, Control, and Dynamics. 9 (3): 288–293. Bibcode:1986JGCD....9..288B. doi:10.2514/3.20105. hdl:2027.42/57880. <a href="https://deepblue.lib.umich.edu/bitstream/2027.42/57880/1/DTReduced-OrderDiscrete-TimeModelingEstimationandControl.pdf" target="_blank">https://deepblue.lib.umich.edu/bitstream/2027.42/57880/1/DTReduced-OrderDiscrete-TimeModelingEstimationandControl.pdf</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>Van Willigenburg L.G.; De Koning W.L. (2000). "Numerical algorithms and issues concerning the discrete-time optimal projection equations". European Journal of Control. 6 (1): 93–100. doi:10.1016/s0947-3580(00)70917-4. Associated software download from Matlab Central. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>Doyle, John C. (1978). "Guaranteed Margins for LQG Regulators" (PDF). IEEE Transactions on Automatic Control. 23 (4): 756–757. doi:10.1109/TAC.1978.1101812. ISSN 0018-9286. <a href="https://murray.cds.caltech.edu/images/murray.cds/b/b4/Guaranteed_margins_for_LQG_regulators_-_doyle.pdf" target="_blank">https://murray.cds.caltech.edu/images/murray.cds/b/b4/Guaranteed_margins_for_LQG_regulators_-_doyle.pdf</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Green, Michael; Limebeer, David J. N. (1995). Linear Robust Control. Englewood Cliffs: Prentice Hall. p. 27. ISBN 0-13-102278-4. <a href="0-13-102278-4" target="_blank">0-13-102278-4</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>Van Willigenburg L.G.; De Koning W.L. (1999). "Optimal reduced-order compensators for time-varying discrete-time systems with deterministic and white parameters". Automatica. 35: 129–138. doi:10.1016/S0005-1098(98)00138-1. Associated software download from Matlab Central. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Matsakis, Demetrios (March 8, 2019). "The effects of proportional steering strategies on the behavior of controlled clocks". Metrologia. 56 (2): 025007. Bibcode:2019Metro..56b5007M. doi:10.1088/1681-7575/ab0614. <a href="https://doi.org/10.1088%2F1681-7575%2Fab0614" target="_blank">https://doi.org/10.1088%2F1681-7575%2Fab0614</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>Athans M. (1971). "The role and use of the stochastic Linear-Quadratic-Gaussian problem in control system design". IEEE Transactions on Automatic Control. AC-16 (6): 529–552. doi:10.1109/TAC.1971.1099818. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> </ol>