Influential observation

In statistics, an influential observation is an observation for a statistical calculation whose deletion from the dataset would noticeably change the result of the calculation. In particular, in regression analysis an influential observation is one whose deletion has a large effect on the parameter estimates.

Various methods have been proposed for measuring influence. Assume an estimated regression ${\displaystyle \mathbf {y} =\mathbf {X} \mathbf {b} +\mathbf {e} }$, where ${\displaystyle \mathbf {y} }$is an n×1 column vector for the response variable, ${\displaystyle \mathbf {X} }$is the n×k design matrix of explanatory variables (including a constant), ${\displaystyle \mathbf {e} }$is the n×1 residual vector, and ${\displaystyle \mathbf {b} }$is a k×1 vector of estimates of some population parameter ${\displaystyle \mathbf {\beta } \in \mathbb {R} ^{k}}$. Also define ${\displaystyle \mathbf {H} \equiv \mathbf {X} \left(\mathbf {X} ^{\mathsf {T}}\mathbf {X} \right)^{-1}\mathbf {X} ^{\mathsf {T}}}$, the projection matrix of ${\displaystyle \mathbf {X} }$. Then we have the following measures of influence:

1. ${\displaystyle {\text{DFBETA}}_{i}\equiv \mathbf {b} -\mathbf {b} _{(-i)}={\frac {\left(\mathbf {X} ^{\mathsf {T}}\mathbf {X} \right)^{-1}\mathbf {x} _{i}^{\mathsf {T}}e_{i}}{1-h_{ii}}}}$, where ${\displaystyle \mathbf {b} _{(-i)}}$denotes the coefficients estimated with the i-th row ${\displaystyle \mathbf {x} _{i}}$of ${\displaystyle \mathbf {X} }$deleted, ${\displaystyle h_{ii}=\mathbf {x} _{i}\left(\mathbf {X} ^{\mathsf {T}}\mathbf {X} \right)^{-1}\mathbf {x} _{i}^{\mathsf {T}}}$denotes the i-th value of matrix's ${\displaystyle \mathbf {H} }$main diagonal. Thus DFBETA measures the difference in each parameter estimate with and without the influential point. There is a DFBETA for each variable and each observation (if there are N observations and k variables there are N·k DFBETAs). Table shows DFBETAs for the third dataset from Anscombe's quartet (bottom left chart in the figure):

An outlier may be defined as a data point that differs markedly from other observations. A high-leverage point are observations made at extreme values of independent variables. Both types of atypical observations will force the regression line to be close to the point. In Anscombe's quartet, the bottom right image has a point with high leverage and the bottom left image has an outlying point.

• Influence function (statistics)
• Outlier
• Leverage
  • Partial leverage
• Regression analysis
• Cook's distance § Detecting highly influential observations
• Anomaly detection

• Dehon, Catherine; Gassner, Marjorie; Verardi, Vincenzo (2009). "Beware of 'Good' Outliers and Overoptimistic Conclusions". Oxford Bulletin of Economics and Statistics. 71 (3): 437–452. doi:10.1111/j.1468-0084.2009.00543.x. S2CID 154376487.
• Kennedy, Peter (2003). "Robust Estimation". A Guide to Econometrics (Fifth ed.). Cambridge: The MIT Press. pp. 372–388. ISBN 0-262-61183-X.