Ridge function

☆ Save On Wikipedia ↗

In mathematics, a ridge function is any function f : R d → R {\displaystyle f:\mathbb {R} ^{d}\rightarrow \mathbb {R} } {\displaystyle f:\mathbb {R} ^{d}\rightarrow \mathbb {R} } that can be written as the composition of an univariate function g : R → R {\displaystyle g:\mathbb {R} \rightarrow \mathbb {R} } {\displaystyle g:\mathbb {R} \rightarrow \mathbb {R} }, that is called a profile function, with an affine transformation, given by a direction vector a ∈ R d {\displaystyle a\in \mathbb {R} ^{d}} {\displaystyle a\in \mathbb {R} ^{d}} with shift b ∈ R {\displaystyle b\in \mathbb {R} } {\displaystyle b\in \mathbb {R} }.

Then, the ridge function reads f ( x ) = g ( x ⊤ a + b ) {\displaystyle f(x)=g(x^{\top }a+b)} {\displaystyle f(x)=g(x^{\top }a+b)} for x ∈ R d {\displaystyle x\in \mathbb {R} ^{d}} {\displaystyle x\in \mathbb {R} ^{d}}.

Coinage of the term 'ridge function' is often attributed to B.F. Logan and L.A. Shepp.[1]

Relevance

A ridge function is not susceptible to the curse of dimensionality, making it an instrumental tool in various estimation problems. This is a direct result of the fact that ridge functions are constant in d − 1 {\displaystyle d-1} {\displaystyle d-1} directions: Let a 1 , … , a d − 1 {\displaystyle a_{1},\dots ,a_{d-1}} {\displaystyle a_{1},\dots ,a_{d-1}} be d − 1 {\displaystyle d-1} {\displaystyle d-1} independent vectors that are orthogonal to a {\displaystyle a} {\displaystyle a}, such that these vectors span d − 1 {\displaystyle d-1} {\displaystyle d-1} dimensions. Then

f ( x + ∑ k = 1 d − 1 c k a k ) = g ( x ⋅ a + ∑ k = 1 d − 1 c k a k ⋅ a ) = g ( x ⋅ a + ∑ k = 1 d − 1 c k 0 ) = g ( x ⋅ a ) = f ( x ) {\displaystyle f\left({\boldsymbol {x}}+\sum _{k=1}^{d-1}c_{k}{\boldsymbol {a}}_{k}\right)=g\left({\boldsymbol {x}}\cdot {\boldsymbol {a}}+\sum _{k=1}^{d-1}c_{k}{\boldsymbol {a}}_{k}\cdot {\boldsymbol {a}}\right)=g\left({\boldsymbol {x}}\cdot {\boldsymbol {a}}+\sum _{k=1}^{d-1}c_{k}0\right)=g({\boldsymbol {x}}\cdot {\boldsymbol {a}})=f({\boldsymbol {x}})} {\displaystyle f\left({\boldsymbol {x}}+\sum _{k=1}^{d-1}c_{k}{\boldsymbol {a}}_{k}\right)=g\left({\boldsymbol {x}}\cdot {\boldsymbol {a}}+\sum _{k=1}^{d-1}c_{k}{\boldsymbol {a}}_{k}\cdot {\boldsymbol {a}}\right)=g\left({\boldsymbol {x}}\cdot {\boldsymbol {a}}+\sum _{k=1}^{d-1}c_{k}0\right)=g({\boldsymbol {x}}\cdot {\boldsymbol {a}})=f({\boldsymbol {x}})}

for all c i ∈ R , 1 ≤ i < d {\displaystyle c_{i}\in \mathbb {R} ,1\leq i<d} {\displaystyle c_{i}\in \mathbb {R} ,1\leq i<d}. In other words, any shift of x {\displaystyle {\boldsymbol {x}}} {\displaystyle {\boldsymbol {x}}} in a direction perpendicular to a {\displaystyle {\boldsymbol {a}}} {\displaystyle {\boldsymbol {a}}} does not change the value of f {\displaystyle f} {\displaystyle f}.

Ridge functions play an essential role in amongst others projection pursuit, generalized linear models, and as activation functions in neural networks. For a survey on ridge functions, see.[2] For books on ridge functions, see.[3][4]

References

  1. Logan, B.F.; Shepp, L.A. (1975). "Optimal reconstruction of a function from its projections". Duke Mathematical Journal. 42 (4): 645–659. doi:10.1215/S0012-7094-75-04256-8.
  2. Konyagin, S.V.; Kuleshov, A.A.; Maiorov, V.E. (2018). "Some Problems in the Theory of Ridge Functions". Proc. Steklov Inst. Math. 301: 144–169. doi:10.1134/S0081543818040120. S2CID 126211876.
  3. Pinkus, Allan (August 2015). Ridge functions. Cambridge: Cambridge Tracts in Mathematics 205. Cambridge University Press. 215 pp. ISBN 9781316408124.
  4. Ismailov, Vugar (December 2021). Ridge functions and applications in neural networks. Providence, RI: Mathematical Surveys and Monographs 263. American Mathematical Society. 186 pp. ISBN 978-1-4704-6765-4.