Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Autocovariance
open-in-new
<h2 id="auto-covariance-of-stochastic-processes">Auto-covariance of stochastic processes</h2> <h3>Definition</h3> <p>With the usual notation E {\displaystyle \operatorname {E} } for the <a href="/facts/Expected_value/1XV0JKL8">expectation</a> operator, if the stochastic process { X t } {\displaystyle \left\{X_{t}\right\}} has the <a href="/facts/Mean/swcEd4Pg">mean</a> function μ t = E [ X t ] {\displaystyle \mu _{t}=\operatorname {E} [X_{t}]} , then the autocovariance is given by<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a>: p. 162 </p> <table><tbody><tr><td> K X X ( t 1 , t 2 ) = cov [ X t 1 , X t 2 ] = E [ ( X t 1 − μ t 1 ) ( X t 2 − μ t 2 ) ] = E [ X t 1 X t 2 ] − μ t 1 μ t 2 {\displaystyle \operatorname {K} _{XX}(t_{1},t_{2})=\operatorname {cov} \left[X_{t_{1}},X_{t_{2}}\right]=\operatorname {E} [(X_{t_{1}}-\mu _{t_{1}})(X_{t_{2}}-\mu _{t_{2}})]=\operatorname {E} [X_{t_{1}}X_{t_{2}}]-\mu _{t_{1}}\mu _{t_{2}}} </td> <td></td> <td>Eq.1</td></tr></tbody></table> <p>where t 1 {\displaystyle t_{1}} and t 2 {\displaystyle t_{2}} are two instances in time. </p> <h3>Definition for weakly stationary process</h3> <p>If { X t } {\displaystyle \left\{X_{t}\right\}} is a <a href="/facts/Weak-sense_stationarity/vozdgfEX">weakly stationary (WSS) process</a>, then the following are true:<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a>: p. 163 </p> μ t 1 = μ t 2 ≜ μ {\displaystyle \mu _{t_{1}}=\mu _{t_{2}}\triangleq \mu } for all t 1 , t 2 {\displaystyle t_{1},t_{2}} <p>and </p> E [ | X t | 2 ] < ∞ {\displaystyle \operatorname {E} [|X_{t}|^{2}]<\infty } for all t {\displaystyle t} <p>and </p> K X X ( t 1 , t 2 ) = K X X ( t 2 − t 1 , 0 ) ≜ K X X ( t 2 − t 1 ) = K X X ( τ ) , {\displaystyle \operatorname {K} _{XX}(t_{1},t_{2})=\operatorname {K} _{XX}(t_{2}-t_{1},0)\triangleq \operatorname {K} _{XX}(t_{2}-t_{1})=\operatorname {K} _{XX}(\tau ),} <p>where τ = t 2 − t 1 {\displaystyle \tau =t_{2}-t_{1}} is the lag time, or the amount of time by which the signal has been shifted. </p><p>The autocovariance function of a WSS process is therefore given by:<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a>: p. 517 </p> <table><tbody><tr><td> K X X ( τ ) = E [ ( X t − μ t ) ( X t − τ − μ t − τ ) ] = E [ X t X t − τ ] − μ t μ t − τ {\displaystyle \operatorname {K} _{XX}(\tau )=\operatorname {E} [(X_{t}-\mu _{t})(X_{t-\tau }-\mu _{t-\tau })]=\operatorname {E} [X_{t}X_{t-\tau }]-\mu _{t}\mu _{t-\tau }} </td> <td></td> <td>Eq.2</td></tr></tbody></table> <p>which is equivalent to </p> K X X ( τ ) = E [ ( X t + τ − μ t + τ ) ( X t − μ t ) ] = E [ X t + τ X t ] − μ 2 {\displaystyle \operatorname {K} _{XX}(\tau )=\operatorname {E} [(X_{t+\tau }-\mu _{t+\tau })(X_{t}-\mu _{t})]=\operatorname {E} [X_{t+\tau }X_{t}]-\mu ^{2}} . <h3>Normalization</h3> <p>It is common practice in some disciplines (e.g. statistics and <a href="/facts/Time_series_analysis/fSXPR817">time series analysis</a>) to normalize the autocovariance function to get a time-dependent <a href="/facts/Pearson_correlation_coefficient/Igf0xDPc">Pearson correlation coefficient</a>. However in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "autocorrelation" and "autocovariance" are used interchangeably. </p><p>The definition of the normalized auto-correlation of a stochastic process is </p> ρ X X ( t 1 , t 2 ) = K X X ( t 1 , t 2 ) σ t 1 σ t 2 = E [ ( X t 1 − μ t 1 ) ( X t 2 − μ t 2 ) ] σ t 1 σ t 2 {\displaystyle \rho _{XX}(t_{1},t_{2})={\frac {\operatorname {K} _{XX}(t_{1},t_{2})}{\sigma _{t_{1}}\sigma _{t_{2}}}}={\frac {\operatorname {E} [(X_{t_{1}}-\mu _{t_{1}})(X_{t_{2}}-\mu _{t_{2}})]}{\sigma _{t_{1}}\sigma _{t_{2}}}}} . <p>If the function ρ X X {\displaystyle \rho _{XX}} is well-defined, its value must lie in the range [ − 1 , 1 ] {\displaystyle [-1,1]} , with 1 indicating perfect correlation and −1 indicating perfect <a href="/facts/Anti-correlation/kTjs0r6G">anti-correlation</a>. </p><p>For a WSS process, the definition is </p> ρ X X ( τ ) = K X X ( τ ) σ 2 = E [ ( X t − μ ) ( X t + τ − μ ) ] σ 2 {\displaystyle \rho _{XX}(\tau )={\frac {\operatorname {K} _{XX}(\tau )}{\sigma ^{2}}}={\frac {\operatorname {E} [(X_{t}-\mu )(X_{t+\tau }-\mu )]}{\sigma ^{2}}}} . <p>where </p> K X X ( 0 ) = σ 2 {\displaystyle \operatorname {K} _{XX}(0)=\sigma ^{2}} . <h3>Properties</h3> <h4>Symmetry property</h4> K X X ( t 1 , t 2 ) = K X X ( t 2 , t 1 ) ¯ {\displaystyle \operatorname {K} _{XX}(t_{1},t_{2})={\overline {\operatorname {K} _{XX}(t_{2},t_{1})}}} <a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a>: p.169 <p>respectively for a WSS process: </p> K X X ( τ ) = K X X ( − τ ) ¯ {\displaystyle \operatorname {K} _{XX}(\tau )={\overline {\operatorname {K} _{XX}(-\tau )}}} <a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a>: p.173 <h4>Linear filtering</h4> <p>The autocovariance of a linearly filtered process { Y t } {\displaystyle \left\{Y_{t}\right\}} </p> Y t = ∑ k = − ∞ ∞ a k X t + k {\displaystyle Y_{t}=\sum _{k=-\infty }^{\infty }a_{k}X_{t+k}\,} <p>is </p> K Y Y ( τ ) = ∑ k , l = − ∞ ∞ a k a l K X X ( τ + k − l ) . {\displaystyle K_{YY}(\tau )=\sum _{k,l=-\infty }^{\infty }a_{k}a_{l}K_{XX}(\tau +k-l).\,} <h2 id="calculating-turbulent-diffusivity">Calculating turbulent diffusivity</h2> <p>Autocovariance can be used to calculate <a href="/facts/Turbulent_diffusivity/NpgFUhG5">turbulent diffusivity</a>.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> Turbulence in a flow can cause the fluctuation of velocity in space and time. Thus, we are able to identify turbulence through the statistics of those fluctuations. </p><p><a href="/facts/Reynolds_decomposition/Vjp3SRWk">Reynolds decomposition</a> is used to define the velocity fluctuations u ′ ( x , t ) {\displaystyle u'(x,t)} (assume we are now working with 1D problem and U ( x , t ) {\displaystyle U(x,t)} is the velocity along x {\displaystyle x} direction): </p> U ( x , t ) = ⟨ U ( x , t ) ⟩ + u ′ ( x , t ) , {\displaystyle U(x,t)=\langle U(x,t)\rangle +u'(x,t),} <p>where U ( x , t ) {\displaystyle U(x,t)} is the true velocity, and ⟨ U ( x , t ) ⟩ {\displaystyle \langle U(x,t)\rangle } is the <a href="/facts/Reynolds_decomposition/Vjp3SRWk">expected value of velocity</a>. If we choose a correct ⟨ U ( x , t ) ⟩ {\displaystyle \langle U(x,t)\rangle } , all of the stochastic components of the turbulent velocity will be included in u ′ ( x , t ) {\displaystyle u'(x,t)} . To determine ⟨ U ( x , t ) ⟩ {\displaystyle \langle U(x,t)\rangle } , a set of velocity measurements that are assembled from points in space, moments in time or repeated experiments is required. </p><p>If we assume the turbulent flux ⟨ u ′ c ′ ⟩ {\displaystyle \langle u'c'\rangle } ( c ′ = c − ⟨ c ⟩ {\displaystyle c'=c-\langle c\rangle } , and <i>c</i> is the concentration term) can be caused by a random walk, we can use <a href="/facts/Fick%2527s_laws_of_diffusion/2guvKD4T">Fick's laws of diffusion</a> to express the turbulent flux term: </p> J turbulence x = ⟨ u ′ c ′ ⟩ ≈ D T x ∂ ⟨ c ⟩ ∂ x . {\displaystyle J_{{\text{turbulence}}_{x}}=\langle u'c'\rangle \approx D_{T_{x}}{\frac {\partial \langle c\rangle }{\partial x}}.} <p>The velocity autocovariance is defined as </p> K X X ≡ ⟨ u ′ ( t 0 ) u ′ ( t 0 + τ ) ⟩ {\displaystyle K_{XX}\equiv \langle u'(t_{0})u'(t_{0}+\tau )\rangle } or K X X ≡ ⟨ u ′ ( x 0 ) u ′ ( x 0 + r ) ⟩ , {\displaystyle K_{XX}\equiv \langle u'(x_{0})u'(x_{0}+r)\rangle ,} <p>where τ {\displaystyle \tau } is the lag time, and r {\displaystyle r} is the lag distance. </p><p>The turbulent diffusivity D T x {\displaystyle D_{T_{x}}} can be calculated using the following 3 methods: </p> <ol><li>If we have velocity data along a <i><a href="/facts/Turbulent_diffusion/088p0LcS">Lagrangian trajectory</a></i>: D T x = ∫ τ ∞ u ′ ( t 0 ) u ′ ( t 0 + τ ) d τ . {\displaystyle D_{T_{x}}=\int _{\tau }^{\infty }u'(t_{0})u'(t_{0}+\tau )\,d\tau .} </li><li>If we have velocity data at one fixed (<a href="/facts/Turbulent_diffusion/088p0LcS">Eulerian</a>) location: D T x ≈ [ 0.3 ± 0.1 ] [ ⟨ u ′ u ′ ⟩ + ⟨ u ⟩ 2 ⟨ u ′ u ′ ⟩ ] ∫ τ ∞ u ′ ( t 0 ) u ′ ( t 0 + τ ) d τ . {\displaystyle D_{T_{x}}\approx [0.3\pm 0.1]\left[{\frac {\langle u'u'\rangle +\langle u\rangle ^{2}}{\langle u'u'\rangle }}\right]\int _{\tau }^{\infty }u'(t_{0})u'(t_{0}+\tau )\,d\tau .} </li><li>If we have velocity information at two fixed (Eulerian) locations: D T x ≈ [ 0.4 ± 0.1 ] [ 1 ⟨ u ′ u ′ ⟩ ] ∫ r ∞ u ′ ( x 0 ) u ′ ( x 0 + r ) d r , {\displaystyle D_{T_{x}}\approx [0.4\pm 0.1]\left[{\frac {1}{\langle u'u'\rangle }}\right]\int _{r}^{\infty }u'(x_{0})u'(x_{0}+r)\,dr,} where r {\displaystyle r} is the distance separated by these two fixed locations.</li></ol> <h2 id="auto-covariance-of-random-vectors">Auto-covariance of random vectors</h2> <p class="note">Main article: <a href="/facts/Auto-covariance_matrix/kLhgjHC6">Auto-covariance matrix</a></p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Autoregressive_process/LpboAQ0W">Autoregressive process</a></li> <li><a href="/facts/Correlation/egkluAEm">Correlation</a></li> <li><a href="/facts/Cross-covariance/nBl7HhGY">Cross-covariance</a></li> <li><a href="/facts/Cross-correlation/Fs1hY3br">Cross-correlation</a></li> <li><a href="/facts/Kalman_filter/wNK7rnbk">Noise covariance estimation</a> (as an application example)</li></ul> <h2 id="further-reading">Further reading</h2> <ul><li>Hoel, P. G. (1984). <i>Mathematical Statistics</i> (Fifth ed.). New York: Wiley. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0-471-89045-4.</li> <li><a href="https://web.archive.org/web/20060428122150/http://w3eos.whoi.edu/12.747/notes/lect06/l06s02.html">Lecture notes on autocovariance from WHOI</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Hsu, Hwei (1997). Probability, random variables, and random processes. McGraw-Hill. ISBN 978-0-07-030644-8. <a href="978-0-07-030644-8" target="_blank">978-0-07-030644-8</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Hsu, Hwei (1997). Probability, random variables, and random processes. McGraw-Hill. ISBN 978-0-07-030644-8. <a href="978-0-07-030644-8" target="_blank">978-0-07-030644-8</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Lapidoth, Amos (2009). A Foundation in Digital Communication. Cambridge University Press. ISBN 978-0-521-19395-5. <a href="978-0-521-19395-5" target="_blank">978-0-521-19395-5</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3 <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3 <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Taylor, G. I. (1922-01-01). "Diffusion by Continuous Movements". Proceedings of the London Mathematical Society. s2-20 (1): 196–212. Bibcode:1922PLMS..220S.196T. doi:10.1112/plms/s2-20.1.196. ISSN 1460-244X. <a href="https://zenodo.org/record/1433523" target="_blank">https://zenodo.org/record/1433523</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> </ol>