Information matrix test

In econometrics, the information matrix test is used to determine whether a regression model is misspecified. The test was developed by Halbert White, who observed that in a correctly specified model and under standard regularity assumptions, the Fisher information matrix can be expressed in either of two ways: as the outer product of the gradient of the log-likelihood function, or as a function of its Hessian matrix.

Consider a linear model ๐ฒ=๐—๐œท+๐ฎ, where the errors ๐ฎ are assumed to be distributed N(0,ฯƒ2๐ˆ). If the parameters ฮฒ and ฯƒ2 are stacked in the vector ๐œฝ๐–ณ=[ฮฒฯƒ2], the resulting log-likelihood function is

โ„“(๐œฝ)=โˆ’n2logโกฯƒ2โˆ’12ฯƒ2(๐ฒโˆ’๐—๐œท)๐–ณ(๐ฒโˆ’๐—๐œท)

The information matrix can then be expressed as

๐ˆ(๐œฝ)=Eโก[(โˆ‚โ„“(๐œฝ)โˆ‚๐œฝ)(โˆ‚โ„“(๐œฝ)โˆ‚๐œฝ)๐–ณ]

that is the expected value of the outer product of the gradient or score. Second, it can be written as the negative of the Hessian matrix of the log-likelihood function

๐ˆ(๐œฝ)=โˆ’Eโก[โˆ‚2โ„“(๐œฝ)โˆ‚๐œฝโˆ‚๐œฝ๐–ณ]

If the model is correctly specified, both expressions should be equal. Combining the equivalent forms yields

๐œŸ(๐œฝ)=โˆ‘i=1n[โˆ‚2โ„“(๐œฝ)โˆ‚๐œฝโˆ‚๐œฝ๐–ณ+โˆ‚โ„“(๐œฝ)โˆ‚๐œฝโˆ‚โ„“(๐œฝ)โˆ‚๐œฝ]

where ๐œŸ(๐œฝ) is an (rร—r) random matrix, where r is the number of parameters. White showed that the elements of nโˆ’1/2๐œŸ(๐œฝฬ‚), where ๐œฝฬ‚ is the MLE, are asymptotically normally distributed with zero means when the model is correctly specified. In small samples, however, the test generally performs poorly.

References

  1. โ†‘ White, Halbert (1982). "Maximum Likelihood Estimation of Misspecified Models". Econometrica. 50 (1): 1โ€“25. doi:10.2307/1912526. JSTORย 1912526.
  2. โ†‘ Godfrey, L. G. (1988). Misspecification Tests in Econometrics. Cambridge University Press. pp.ย 35โ€“37. ISBNย 0-521-26616-5.
  3. โ†‘ Orme, Chris (1990). "The Small-Sample Performance of the Information-Matrix Test". Journal of Econometrics. 46 (3): 309โ€“331. doi:10.1016/0304-4076(90)90012-I.

Further reading