Scale-invariant feature operator

In the fields of computer vision and image analysis, the scale-invariant feature operator (or SFOP) is an algorithm to detect local features in images. The algorithm was published by Förstner et al. in 2009.

The scale-invariant feature operator (SFOP) is based on two theoretical concepts:

• spiral model
• feature operator

Desired properties of keypoint detectors:

• Invariance and repeatability for object recognition
• Accuracy to support camera calibration
• Interpretability: Especially corners and circles, should be part of the detected keypoints (see figure).
• As few control parameters as possible with clear semantics
• Complementarity to known detectors

scale-invariant corner/circle detector.

Maximize the weight ${\displaystyle w}$= 1/variance of a point ${\displaystyle p}$

${\displaystyle w(\mathbf {p} ,\alpha ,\tau ,\sigma )=\left(N(\sigma )-2\right){\frac {\lambda _{min}(M(\mathbf {p} ,\alpha ,\tau ,\sigma ))}{\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )}}}$  

comprising:

1. the image model

• Distance d of an edge from a reference point p in a spiral feature
• 
• 
• 

${\displaystyle {\begin{aligned}\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )&=\sum _{n=1}^{N(\sigma )}[(\mathbf {q} _{n}-\mathbf {p} )^{T}\mathbf {R} _{\alpha }\mathbf {\nabla } _{T}g(\mathbf {q} _{n})]^{2}G_{\sigma }(\mathbf {q} _{n}-\mathbf {p} )\\&=N(\sigma )\mathbf {tr} \left\{R_{\alpha }\mathbf {\nabla } _{\tau }\mathbf {\nabla } _{\tau }^{T}R_{\alpha }^{T}*\mathbf {p} \mathbf {p} ^{T}G_{\sigma }(\mathbf {p} )\right\}\end{aligned}}}$

2. the smaller eigenvalue of the structure tensor ${\displaystyle \underbrace {M(\mathbf {p} ,\alpha ,\tau ,\sigma )} _{\text{structure tensor}}=\underbrace {G_{\sigma }(\mathbf {p} )} _{\text{weighted summation}}*\underbrace {(R_{\sigma }\nabla _{\tau }\nabla _{\tau }^{T}R_{\sigma }^{T})} _{\text{squared rotated gradients}}}$

Reduce the 5-dimensional search space by

• linking the differentiation scale ${\displaystyle \tau }$to the integration scale

${\displaystyle \tau =\sigma /3}$

• solving for the optimal ${\displaystyle {\hat {\alpha }}}$using the model

${\displaystyle \Omega (\alpha )=a-b\cos(2\alpha -2\alpha _{0})}$

• and determining the parameters from three angles, e. g.

${\displaystyle \Omega (0^{\circ }),\Omega (60^{\circ }),\Omega (120^{\circ })\quad \rightarrow \quad a,b,\alpha _{0}\quad \rightarrow \quad {\hat {\alpha }}}$

• pre-selection possible:

${\displaystyle \alpha =0^{\circ }\,\rightarrow \,{\mbox{junctions}},\quad \alpha =90^{\circ }\,\rightarrow \,{\mbox{circular features}}}$

• non-maxima suppression over scale, space and angle

• thresholding the isotropy ${\displaystyle \lambda _{2(M)}}$:
  eigenvalues characterize the shape of the keypoint, smallest eigenvalue has to be larger than threshold ${\displaystyle T_{\lambda }}$
  derived from noise variance ${\displaystyle V(n)}$and significance level ${\displaystyle S}$:

${\displaystyle T_{\lambda }(V(n),\tau ,\sigma ,S)={\frac {N(\sigma )}{16\pi \tau ^{4}}}V(n)\chi _{2,S}^{2}}$

• Results of different detectors on a Siemens star
• 
  Sfop: junctions red, circular features cyan
• 
  Edge-based Regions
• 
  Intensity-based Regions
• 
  MSER
• 
  Harris affine
• 
  Hessian affine
• 
  Lowe

• Corner detection
• Feature detection (computer vision)

• Project website at University of Bonn