Autocovariance

In probability theory and statistics, given a stochastic process, the autocovariance is a function that gives the covariance of the process with itself at pairs of time points. Autocovariance is closely related to the autocorrelation of the process in question.

Auto-covariance of stochastic processes

Definition

With the usual notation $\operatorname {E}$ for the expectation operator, if the stochastic process $\left\{X_{t}\right\}$ has the mean function $\mu _{t}=\operatorname {E} [X_{t}]$ , then the autocovariance is given by^{: p. 162}

\operatorname {K} _{XX}(t_{1},t_{2})=\operatorname {cov} \left[X_{t_{1}},X_{t_{2}}\right]=\operatorname {E} [(X_{t_{1}}-\mu _{t_{1}})(X_{t_{2}}-\mu _{t_{2}})]=\operatorname {E} [X_{t_{1}}X_{t_{2}}]-\mu _{t_{1}}\mu _{t_{2}}

Eq.1

where $t_{1}$ and $t_{2}$ are two instances in time.

Definition for weakly stationary process

If $\left\{X_{t}\right\}$ is a weakly stationary (WSS) process, then the following are true:^{: p. 163}

\mu _{t_{1}}=\mu _{t_{2}}\triangleq \mu

for all

t_{1},t_{2}

and

\operatorname {E} [|X_{t}|^{2}]<\infty

for all

t

and

\operatorname {K} _{XX}(t_{1},t_{2})=\operatorname {K} _{XX}(t_{2}-t_{1},0)\triangleq \operatorname {K} _{XX}(t_{2}-t_{1})=\operatorname {K} _{XX}(\tau ),

where $\tau =t_{2}-t_{1}$ is the lag time, or the amount of time by which the signal has been shifted.

The autocovariance function of a WSS process is therefore given by:^{: p. 517}

\operatorname {K} _{XX}(\tau )=\operatorname {E} [(X_{t}-\mu _{t})(X_{t-\tau }-\mu _{t-\tau })]=\operatorname {E} [X_{t}X_{t-\tau }]-\mu _{t}\mu _{t-\tau }

Eq.2

which is equivalent to

\operatorname {K} _{XX}(\tau )=\operatorname {E} [(X_{t+\tau }-\mu _{t+\tau })(X_{t}-\mu _{t})]=\operatorname {E} [X_{t+\tau }X_{t}]-\mu ^{2}

.

Normalization

It is common practice in some disciplines (e.g. statistics and time series analysis) to normalize the autocovariance function to get a time-dependent Pearson correlation coefficient. However in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "autocorrelation" and "autocovariance" are used interchangeably.

The definition of the normalized auto-correlation of a stochastic process is

\rho _{XX}(t_{1},t_{2})={\frac {\operatorname {K} _{XX}(t_{1},t_{2})}{\sigma _{t_{1}}\sigma _{t_{2}}}}={\frac {\operatorname {E} [(X_{t_{1}}-\mu _{t_{1}})(X_{t_{2}}-\mu _{t_{2}})]}{\sigma _{t_{1}}\sigma _{t_{2}}}}

.

If the function $\rho _{XX}$ is well-defined, its value must lie in the range $[-1,1]$ , with 1 indicating perfect correlation and −1 indicating perfect anti-correlation.

For a WSS process, the definition is

\rho _{XX}(\tau )={\frac {\operatorname {K} _{XX}(\tau )}{\sigma ^{2}}}={\frac {\operatorname {E} [(X_{t}-\mu )(X_{t+\tau }-\mu )]}{\sigma ^{2}}}

.

where

\operatorname {K} _{XX}(0)=\sigma ^{2}

.

Properties

Symmetry property

\operatorname {K} _{XX}(t_{1},t_{2})={\overline {\operatorname {K} _{XX}(t_{2},t_{1})}}

^: p.169

respectively for a WSS process:

\operatorname {K} _{XX}(\tau )={\overline {\operatorname {K} _{XX}(-\tau )}}

^: p.173

Linear filtering

The autocovariance of a linearly filtered process $\left\{Y_{t}\right\}$

Y_{t}=\sum _{k=-\infty }^{\infty }a_{k}X_{t+k}\,

is

K_{YY}(\tau )=\sum _{k,l=-\infty }^{\infty }a_{k}a_{l}K_{XX}(\tau +k-l).\,

Calculating turbulent diffusivity

Autocovariance can be used to calculate turbulent diffusivity. Turbulence in a flow can cause the fluctuation of velocity in space and time. Thus, we are able to identify turbulence through the statistics of those fluctuations^{[citation needed]}.

Reynolds decomposition is used to define the velocity fluctuations $u'(x,t)$ (assume we are now working with 1D problem and $U(x,t)$ is the velocity along $x$ direction):

U(x,t)=\langle U(x,t)\rangle +u'(x,t),

where $U(x,t)$ is the true velocity, and $\langle U(x,t)\rangle$ is the expected value of velocity. If we choose a correct $\langle U(x,t)\rangle$ , all of the stochastic components of the turbulent velocity will be included in $u'(x,t)$ . To determine $\langle U(x,t)\rangle$ , a set of velocity measurements that are assembled from points in space, moments in time or repeated experiments is required.

If we assume the turbulent flux $\langle u'c'\rangle$ ( $c'=c-\langle c\rangle$ , and c is the concentration term) can be caused by a random walk, we can use Fick's laws of diffusion to express the turbulent flux term:

J_{{\text{turbulence}}_{x}}=\langle u'c'\rangle \approx D_{T_{x}}{\frac {\partial \langle c\rangle }{\partial x}}.

The velocity autocovariance is defined as

K_{XX}\equiv \langle u'(t_{0})u'(t_{0}+\tau )\rangle

or

K_{XX}\equiv \langle u'(x_{0})u'(x_{0}+r)\rangle ,

where $\tau$ is the lag time, and $r$ is the lag distance.

The turbulent diffusivity $D_{T_{x}}$ can be calculated using the following 3 methods:

If we have velocity data along a Lagrangian trajectory:
$D_{T_{x}}=\int _{\tau }^{\infty }u'(t_{0})u'(t_{0}+\tau )\,d\tau .$
If we have velocity data at one fixed (Eulerian) location^{[citation needed]}:
$D_{T_{x}}\approx [0.3\pm 0.1]\left[{\frac {\langle u'u'\rangle +\langle u\rangle ^{2}}{\langle u'u'\rangle }}\right]\int _{\tau }^{\infty }u'(t_{0})u'(t_{0}+\tau )\,d\tau .$
If we have velocity information at two fixed (Eulerian) locations^{[citation needed]}:
$D_{T_{x}}\approx [0.4\pm 0.1]\left[{\frac {1}{\langle u'u'\rangle }}\right]\int _{r}^{\infty }u'(x_{0})u'(x_{0}+r)\,dr,$
where $r$ is the distance separated by these two fixed locations.

Auto-covariance of random vectors

References

1 2 Hsu, Hwei (1997). Probability, random variables, and random processes. McGraw-Hill. ISBN 978-0-07-030644-8.
↑ Lapidoth, Amos (2009). A Foundation in Digital Communication. Cambridge University Press. ISBN 978-0-521-19395-5.
1 2 Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3
↑ Taylor, G. I. (1922-01-01). "Diffusion by Continuous Movements". Proceedings of the London Mathematical Society. s2-20 (1): 196–212. Bibcode:1922PLMS..220S.196T. doi:10.1112/plms/s2-20.1.196. ISSN 1460-244X.