Ridge function

In mathematics, a ridge function is any function f:ℝd→ℝ that can be written as the composition of an univariate function g:ℝ→ℝ, that is called a profile function, with an affine transformation, given by a direction vector a∈ℝd with shift b∈ℝ.

Then, the ridge function reads f(x)=g(x⊤a+b) for x∈ℝd.

Coinage of the term 'ridge function' is often attributed to B.F. Logan and L.A. Shepp.

Relevance

A ridge function is not susceptible to the curse of dimensionality[clarification needed], making it an instrumental tool in various estimation problems. This is a direct result of the fact that ridge functions are constant in d−1 directions: Let a1,…,ad−1 be d−1 independent vectors that are orthogonal to a, such that these vectors span d−1 dimensions. Then

f(𝒙+∑k=1d−1ck𝒂k)=g(𝒙⋅𝒂+∑k=1d−1ck𝒂k⋅𝒂)=g(𝒙⋅𝒂+∑k=1d−1ck0)=g(𝒙⋅𝒂)=f(𝒙)

for all ci∈ℝ,1≤i<d. In other words, any shift of 𝒙 in a direction perpendicular to 𝒂 does not change the value of f.

Ridge functions play an essential role in amongst others projection pursuit, generalized linear models, and as activation functions in neural networks. For a survey on ridge functions, see. For books on ridge functions, see.

References

  1. ↑ Logan, B.F.; Shepp, L.A. (1975). "Optimal reconstruction of a function from its projections". Duke Mathematical Journal. 42 (4): 645–659. doi:10.1215/S0012-7094-75-04256-8.
  2. ↑ Konyagin, S.V.; Kuleshov, A.A.; Maiorov, V.E. (2018). "Some Problems in the Theory of Ridge Functions". Proc. Steklov Inst. Math. 301: 144–169. doi:10.1134/S0081543818040120. S2CID 126211876.
  3. ↑ Pinkus, Allan (August 2015). Ridge functions. Cambridge: Cambridge Tracts in Mathematics 205. Cambridge University Press. 215 pp. ISBN 9781316408124.
  4. ↑ Ismailov, Vugar (December 2021). Ridge functions and applications in neural networks. Providence, RI: Mathematical Surveys and Monographs 263. American Mathematical Society. 186 pp. ISBN 978-1-4704-6765-4.