191.125.52.120: /* Definitions */fixing sign mistake

2024-08-08T03:45:38Z

Definitions: fixing sign mistake

New page

{{Short description|Piecewise function that clamps its input to be non-negative}}
{{Refimprove|date=January 2017}}
[[File:Ramp function.svg|thumb|325px|[[Graph of a function|Graph]] of the ramp function]]

The '''ramp function''' is a [[unary function|unary]] [[real function]], whose [[graph of a function|graph]] is shaped like a [[ramp]]. It can be expressed by numerous [[#Definitions|definitions]], for example "0 for negative inputs, output equals input for non-negative inputs". The term "ramp" can also be used for other functions obtained by [[scaling and shifting]], and the function in this article is the ''unit'' ramp function (slope 1, starting at 0).

In mathematics, the ramp function is also known as the [[positive part]].

In [[machine learning]], it is commonly known as a [[Rectifier_(neural_networks)|ReLU]] [[activation function]]<ref name='brownlee'>{{cite web |last1=Brownlee |first1=Jason |title=A Gentle Introduction to the Rectified Linear Unit (ReLU) |url=https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/ |website=Machine Learning Mastery |access-date=8 April 2021 |date=8 January 2019}}</ref><ref name="medium-relu">{{cite web |last1=Liu |first1=Danqing |title=A Practical Guide to ReLU |url=https://medium.com/@danqing/a-practical-guide-to-relu-b83ca804f1f7 |website=Medium |access-date=8 April 2021 |language=en |date=30 November 2017}}</ref> or a [[Rectifier (neural networks)|rectifier]] in analogy to [[half-wave rectification]] in [[electrical engineering]]. In [[statistics]] (when used as a [[likelihood function]]) it is known as a [[tobit model]].

This function has numerous [[#Applications|applications]] in mathematics and engineering, and goes by various names, depending on the context. There are [[Rectifier_(neural_networks)#Other_non-linear_variants|differentiable variants]] of the ramp function.

== Definitions ==
The ramp function ({{math|''R''(''x'') : '''R''' → '''R'''0+}}) may be defined analytically in several ways. Possible definitions are:
* A [[piecewise function]]: <math display="block">R(x) := \begin{cases}
x, & x \ge 0; \\
0, & x<0
\end{cases} </math>
* Using the [[Iverson bracket]] notation: <math display="block">R(x) := x \cdot [x \geq 0]</math> or <math display="block">R(x) := x \cdot [x > 0]</math>
* The [[Maxima and minima|max function]]: <math display="block">R(x) := \max(x,0) </math>
* The [[arithmetic mean|mean]] of an [[independent variable]] and its [[absolute value]] (a straight line with unity gradient and its modulus): <math display="block">R(x) := \frac{x+|x|}{2} </math> this can be derived by noting the following definition of {{math|max(''a'', ''b'')}}, <math display="block"> \max(a,b) = \frac{a + b + |a - b|}{2} </math> for which {{math|1=''a'' = ''x''}} and {{math|1=''b'' = 0}}
* The [[Heaviside step function]] multiplied by a straight line with unity gradient: <math display="block">R\left( x \right) := x H(x)</math>
* The [[convolution]] of the Heaviside step function with itself: <math display="block">R\left( x \right) := H(x) * H(x)</math>
* The [[integral]] of the Heaviside step function:<ref>{{MathWorld|title=Ramp Function|id=RampFunction}}</ref> <math display="block">R(x) := \int_{-\infty}^{x} H(\xi)\,d\xi</math>
* [[Macaulay brackets]]: <math display="block">R(x) := \langle x\rangle</math>
* The [[positive and negative parts|positive part]] of the [[identity function]]: <math display="block">R := \operatorname{id}^+</math>
* As a limit function: <math display="block">R\left( x \right) := \lim_{a\to \infty} \begin{cases} \frac{1}{a} ,\quad x=0 \\ \dfrac{x}{1-e^{-ax}},\quad x\neq 0\end{cases} </math>
It could approximated as close as desired by choosing an increasing positive value <math> a>0 </math>.

== Applications ==
The ramp function has numerous applications in engineering, such as in the theory of [[digital signal processing]].

[[File:Long call option.svg|thumb|Payoff and profits from buying a [[call option]].]]
In [[finance]], the payoff of a [[call option]] is a ramp (shifted by ''strike price''). Horizontally flipping a ramp yields a [[put option]], while vertically flipping (taking the negative) corresponds to ''selling'' or being "short" an option. In finance, the shape is widely called a "[[hockey stick]]", due to the shape being similar to an [[ice hockey stick]].

[[File:Friedmans mars hinge functions.png|thumb|A mirrored pair of [[Multivariate adaptive regression splines#Hinge functions|hinge functions]] with a knot at x=3.1]]
In [[statistics]], [[Multivariate adaptive regression splines#Hinge functions|hinge functions]] of [[multivariate adaptive regression splines]] (MARS) are ramps, and are used to build [[regression model]]s.

== Analytic properties ==

=== Non-negativity ===

In the whole [[domain of a function|domain]] the function is non-negative, so its [[absolute value]] is itself, i.e.
<math display="block">\forall x \in \Reals: R(x) \geq 0 </math>
and
<math display="block">\left| R (x) \right| = R(x)</math>
{{math proof | by the mean of definition 2, it is non-negative in the first quarter, and zero in the second; so everywhere it is non-negative.}}

=== Derivative ===
Its derivative is the [[Heaviside step function]]:
<math display="block">R'(x) = H(x)\quad \mbox{for } x \ne 0.</math>

=== Second derivative ===
The ramp function satisfies the differential equation:
<math display="block"> \frac{d^2}{dx^2} R(x - x_0) = \delta(x - x_0), </math>
where {{math|''δ''(''x'')}} is the [[Dirac delta]]. This means that {{math|''R''(''x'')}} is a [[Green's function]] for the second derivative operator. Thus, any function, {{math|''f''(''x'')}}, with an integrable second derivative, {{math|''f''″(''x'')}}, will satisfy the equation:
<math display="block"> f(x) = f(a) + (x-a) f'(a) + \int_{a}^b R(x - s) f''(s) \,ds \quad \mbox{for }a < x 

=== [[Fourier transform]] ===
<math display="block"> \mathcal{F}\big\{ R(x) \big\}(f) = \int_{-\infty}^{\infty} R(x) e^{-2\pi ifx} \, dx = \frac{i\delta '(f)}{4\pi}-\frac{1}{4 \pi^2 f^2}, </math>
where {{math|''δ''(''x'')}} is the [[Dirac delta]] (in this formula, its [[derivative]] appears).

=== [[Laplace transform]] ===
The single-sided [[Laplace transform]] of {{math|''R''(''x'')}} is given as follows,<ref>{{Cite web| url=https://lpsa.swarthmore.edu/LaplaceXform/FwdLaplace/LaplaceFuncs.html#Ramp| title=The Laplace Transform of Functions| website=lpsa.swarthmore.edu |access-date=2019-04-05}}</ref>
<math display="block"> \mathcal{L}\big\{R(x)\big\} (s) = \int_{0}^{\infty} e^{-sx}R(x)dx = \frac{1}{s^2}. </math>

== Algebraic properties ==

=== Iteration invariance ===
Every [[iterated function]] of the ramp mapping is itself, as
<math display="block"> R \big( R(x) \big) = R(x) .</math>
{{math proof | proof =
<math display="block"> R \big( R(x) \big) := \frac{R(x)+|R(x)|}{2} = \frac{R(x)+R(x)}{2} = R(x) .</math>
This applies the [[#Non-negativity|non-negative property]].}}

==See also==
* [[Tobit model]]
* [[Rectifier (neural networks)]]

== References ==
{{Reflist}}

[[Category:Real analysis]]
[[Category:Special functions]]

Ramp function - Revision history

191.125.52.120: /* Definitions */fixing sign mistake