G-test

In statistics, G-tests are likelihood-ratio or maximum likelihood statistical significance tests that are increasingly being used in situations where chi-squared tests were previously recommended.

Formulation

The general formula for test statistics of the G-test is

G=2∑iOi⋅ln⁡(OiEi),

where Oi≥0 is the observed count in a cell, Ei>0 is the expected count under the null hypothesis, ln denotes the natural logarithm, and the sum is taken over all non-empty cells. The resulting G is asymptotically chi-squared distributed as the total number of observations tends to infinity (convergence in distribution).

Furthermore, the total observed count must be equal to the total expected count:

∑iOi=∑iEi=N,

where N is the total number of observations.

Both, the G-test statistics G and the chi-square test statistics χ2 are special cases of a general family of power divergence statistics by Cressie and Read. For λ∉{0,−1} set

CRλ=2λ(λ+1)∑iOi((OiEi)λ−1).

Then,

G=limλ→0CRλ,χ2=CR1.

Derivation

We can derive the value of the G-test from the log-likelihood ratio test where the underlying model is a multinomial model.

Suppose we had a sample O=(O1,…,Om) where each Oi is the number of times that an object of type i was observed. Furthermore, let N=∑i=1mOi be the total number of observations. If we assume that the underlying model is multinomial, then the test statistic is defined by

ln⁡(L(p~|O)L(p̂|O))=ln⁡(∏i=1mp~iOi∏i=1mp̂iOi),

where p~=(p~1,…,p~m) is the null hypothesis and p̂=(p̂1,…,p̂m) is the maximum likelihood estimate (MLE) of the parameters given the data. Recall that for the multinomial model, the MLE of p̂i given some data is given by

p̂i=OiN.

Furthermore, we may represent each null hypothesis parameter p~i as

p~i=EiN,

where Ei is the expected count of objects of type i under the null hypothesis. Thus, by substituting the representations of p~i and p̂i in the log-likelihood ratio, the equation simplifies to

ln⁡(L(p~|O)L(p̂|O))=ln⁡(∏i=1m(EiOi)Oi)=∑i=1mOiln⁡(EiOi)

Finally, multiply by a factor of −2 (used to make the G-test formula asymptotically equivalent to the Pearson's chi-squared test statistics) to achieve the form

G=−2∑i=1mOiln⁡(EiOi)=2∑i=1mOiln⁡(OiEi)

Heuristically, one can imagine Oi as continuous and approaching zero, in which case Oiln⁡Oi→0, and terms with zero observations can simply be dropped. However the expected count in each cell must be strictly greater than zero for each cell (Ei>0 for all i) to apply the method.

Distribution and use

Given the null hypothesis that the observed frequencies result from random sampling from a distribution with the given expected frequencies, the distribution of the test statistics G is approximately a chi-squared distribution, with the same number of degrees of freedom as in the corresponding chi-squared test.

For very small samples the multinomial test for goodness of fit, and Fisher's exact test for contingency tables, or even Bayesian hypothesis selection are preferable to the G-test. McDonald recommends to always use an exact test (exact test of goodness-of-fit, Fisher's exact test) if the total sample size is less than 1 000 .

There is nothing magical about a sample size of 1 000, it's just a nice round number that is well within the range where an exact test, chi-square test, and G–test will give almost identical p values. Spreadsheets, web-page calculators, and SAS shouldn't have any problem doing an exact test on a sample size of 1 000 .
— John H. McDonald (2014)

G-tests have been recommended at least since the 1981 edition of Biometry, a statistics textbook by Robert R. Sokal and F. James Rohlf.

Relation to other metrics

Relation to the chi-squared test

The commonly used chi-squared tests for goodness of fit to a distribution and for independence in contingency tables are in fact approximations of the log-likelihood ratio on which the G-tests are based.

The general formula for Pearson's chi-squared test statistic is

χ2=∑i(Oi−Ei)2Ei.

The approximation of the G-test statistics by chi-squared test statistics is obtained by a second order Taylor expansion of the natural logarithm around 1 (see the derivation below). We have G≈χ2 when the observed counts Oi are close to the expected counts Ei. When this difference is large, however, the approximation by the chi-squared test statistics begins to break down. Here, the effects of outliers in data will be more pronounced, and this explains the why chi-squared tests fail in situations with little data.

For samples of a reasonable size, the G-test and the chi-squared test will lead to the same conclusions. However, the approximation to the theoretical chi-squared distribution for the G-test is better than for the Pearson's chi-squared test. In cases where Oi>2⋅Ei for some cell case the G-test is always better than the chi-squared test.[citation needed]

For testing goodness-of-fit the G-test is infinitely more efficient than the chi-squared test in the sense of Bahadur, but the two tests are equally efficient in the sense of Pitman or in the sense of Hodges and Lehmann.

Derivation (chi-squared)

Consider

G=2∑iOiln⁡(OiEi),

and let Oi=Ei+δi with ∑iδi=0, so that the total number of counts remains the same. Assume that δi=Oi−Ei is small in comparison to Ei for all i. To be more precise, notice that Ei=Θ(n) using big Θ notation. If Oi=Ei+𝒪(n1/2) using big O notation for large n, which should be true under the null hypothesis because of the central limit theorem, then δi=𝒪(n1/2) and

δi3Ei2=𝒪(n3/2n2)=𝒪(n−1/2)

follow, which will be used later.

Upon substitution we find,

G=2∑i(Ei+δi)ln⁡(1+δiEi).

Using the Taylor expansion ln⁡(1+x)=x−12x2+𝒪(x3) yields

G=2∑i(Ei+δi)(δiEi−12δi2Ei2+𝒪(δi3Ei3)),

and distributing terms we find,

G=2∑i(δi+12δi2Ei+𝒪(δi3Ei2)).

Now, using ∑iδi=0 and δi=Oi−Ei and 𝒪(δi3/Ei2)=𝒪(n−1/2) for large n, we can write the result,

G≈∑i(Oi−Ei)2Ei.

Relation to Kullback–Leibler divergence

The G-test statistic is proportional to the Kullback–Leibler divergence of the theoretical distribution p~=(p~1,…,p~m) of the null hypothesis from the empirical distribution p̂=(p̂1,…,p̂m) of the observed data:

G=2∑iOi⋅ln⁡(OiEi)=2N∑ip̂i⋅ln⁡(p̂ip~i)=2NDKL(p̂‖p~),

where N is the total number of observations and p~i=EiN and p̂i=OiN are the theoretical and empirical probabilities of objects of type i, respectively.

Relation to mutual information

For analysis of contingency tables the value of the G-test statistics can also be expressed in terms of mutual information.

In this case objects with two-dimensional types (i,j) are considered. Let Oij be the count of objects of type (i,j), i.e., Oij is the entry in the contingency table in row i and column j. Set

N=∑ijOij,p̂ij=OijN,p̂i∙=∑jOijN,p̂∙j=∑iOijN.

Then the estimated expected count of objects of type (i,j) assuming independence is given by

Eij=Np̂i∙p̂∙j.

Finally, the G-test statistics in this case is given by

G=2∑ijOijln⁡(OijEij)

Let X,Y be random variables with joint distribution given by the empirical distribution p̂ij of the contingency table, i.e.,

P(X=i,Y=j)=p̂ij,P(X=i)=p̂i∙,P(Y=j)=p̂∙j.

Then the G-test statistics can be expressed in several alternative forms:

G=2N⋅∑ijp̂ij(ln⁡(p̂ij)−ln⁡(p̂i∙)−ln⁡(p̂∙j))=2N⋅(H(X)+H(Y)−H(X,Y))=2N⋅MI⁡(X,Y),

where the entropies H(X) and H(Y) are given

H(X)=−∑ip̂i∙ln⁡(p̂i∙),H(Y)=−∑jp̂∙jln⁡(p̂∙j)

and the joint entropy H(X,Y) is given by

H(X,Y)=−∑ijp̂ijln⁡(p̂ij)

and the mutual information of X and Y is

MI⁡(X,Y)=H(X)+H(Y)−H(X,Y).


It can also be shown[citation needed] that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G-test statistic.[citation needed]

Application

Statistical software