Information source (mathematics)

In mathematics, an information source is a sequence of random variables ranging over a finite alphabet ฮ“, having a stationary distribution.

The uncertainty, or entropy rate, of an information source is defined as

H{๐—}=limnโ†’โˆžH(Xn|X0,X1,โ€ฆ,Xnโˆ’1)

where

X0,X1,โ€ฆ,Xn

is the sequence of random variables defining the information source, and

H(Xn|X0,X1,โ€ฆ,Xnโˆ’1)

is the conditional information entropy of the sequence of random variables. Equivalently, one has

H{๐—}=limnโ†’โˆžH(X0,X1,โ€ฆ,Xnโˆ’1,Xn)n+1.

See also

References