Generalized Pareto distribution

In statistics, the generalized Pareto distribution (GPD) is a family of continuous probability distributions. It is often used to model the tails of another distribution. It is specified by three parameters: location $\mu$, scale $\sigma$, and shape $\xi$. Sometimes it is specified by only scale and shape and sometimes only by its shape parameter. Some references give the shape parameter as $\kappa =-\xi$.

With shape $\xi >0$ and location $\mu =\sigma /\xi$, the GPD is equivalent to the Pareto distribution with scale $x_m=\sigma /\xi$ and shape $\alpha =1/\xi$.

Definition

The cumulative distribution function of $X\sim GPD(\mu ,\sigma ,\xi )$ ($\mu \in \mathbb{ℝ}$, $\sigma >0$, and $\xi \in \mathbb{ℝ}$) is

$F_{(\mu ,\sigma ,\xi )}(x)=\{\begin{array}{ll}1-{(1+\frac{\xi (x-\mu )}{\sigma })}^{-1/\xi }& \text{for }\xi \ne 0,\\ 1-\exp (-\frac{x-\mu }{\sigma })& \text{for }\xi =0,\end{array}$

where the support of $X$ is $x⩾\mu$ when $\xi ⩾0$, and $\mu ⩽x⩽\mu -\sigma /\xi$ when $\xi <0$.

The probability density function (pdf) of $X\sim GPD(\mu ,\sigma ,\xi )$ is

$f_{(\mu ,\sigma ,\xi )}(x)=\frac{1}{\sigma }{(1+\frac{\xi (x-\mu )}{\sigma })}^{(-\frac{1}{\xi }-1)}$,

again, for $x⩾\mu$ when $\xi ⩾0$, and $\mu ⩽x⩽\mu -\sigma /\xi$ when $\xi <0$.

The pdf is a solution of the following differential equation:

$\left\{\begin{array}{l}{f}^{\prime }(x)(-\mu \xi +\sigma +\xi x)+(\xi +1)f(x)=0,\\ f(0)=\frac{{(1-\frac{\mu \xi }{\sigma })}^{-\frac{1}{\xi }-1}}{\sigma }\end{array}\right\}$

The standard cumulative distribution function (cdf) of the GPD is defined using $z=\frac{x-\mu }{\sigma }$

$F_{\xi }(z)=\{\begin{array}{ll}1-{(1+\xi z)}^{-1/\xi }& \text{for }\xi \ne 0,\\ 1-e^{-z}& \text{for }\xi =0.\end{array}$

where the support is $z\ge 0$ for $\xi \ge 0$ and $0\le z\le -1/\xi$ for $\xi <0$. The corresponding probability density function (pdf) is

$f_{\xi }(z)=\{\begin{array}{ll}(1+\xi z{)}^{-\frac{\xi +1}{\xi }}& \text{for }\xi \ne 0,\\ e^{-z}& \text{for }\xi =0.\end{array}$

Special cases

• If the shape $\xi$ and location $\mu$ are both zero, the GPD is equivalent to the exponential distribution.
• With shape $\xi =-1$, the GPD is equivalent to the continuous uniform distribution $U(0,\sigma )$.
• With shape $\xi >0$ and location $\mu =\sigma /\xi$, the GPD is equivalent to the Pareto distribution with scale $x_m=\sigma /\xi$ and shape $\alpha =1/\xi$.
• If $X$ $\sim$ $GPD$ $($$\mu =0$, $\sigma$, $\xi$ $)$, then $Y=\log (X)\sim exGPD(\sigma ,\xi )$ [1]. (exGPD stands for the exponentiated generalized Pareto distribution.)
• GPD is similar to the Burr distribution.

Generating generalized Pareto random variables

Generating GPD random variables

If U is uniformly distributed on (0, 1], then

$X=\mu +\frac{\sigma (U^{-\xi }-1)}{\xi }\sim GPD(\mu ,\sigma ,\xi \ne 0)$

and

$X=\mu -\sigma \ln (U)\sim GPD(\mu ,\sigma ,\xi =0).$

Both formulas are obtained by inversion of the cdf.

The Pareto package in R and the gprnd command in the Matlab Statistics Toolbox can be used to generate generalized Pareto random numbers.

GPD as an Exponential-Gamma Mixture

A GPD random variable can also be expressed as an exponential random variable, with a Gamma distributed rate parameter.

$X|\mathrm{\Lambda }\sim \mathsf{E}\mathsf{x}\mathsf{p}(\mathrm{\Lambda })$

and

$\mathrm{\Lambda }\sim \mathsf{G}\mathsf{a}\mathsf{m}\mathsf{m}\mathsf{a}(\alpha ,\beta )$

then

$X\sim \mathsf{G}\mathsf{P}\mathsf{D}(\xi =1/\alpha ,\sigma =\beta /\alpha )$

Notice however, that since the parameters for the Gamma distribution must be greater than zero, we obtain the additional restrictions that $\xi$ must be positive.

In addition to this mixture (or compound) expression, the generalized Pareto distribution can also be expressed as a simple ratio. Concretely, for $Y\sim \mathsf{E}\mathsf{x}\mathsf{p}\mathsf{o}\mathsf{n}\mathsf{e}\mathsf{n}\mathsf{t}\mathsf{i}\mathsf{a}\mathsf{l}(1)$ and $Z\sim \mathsf{G}\mathsf{a}\mathsf{m}\mathsf{m}\mathsf{a}(1/\xi ,1),$ we have $\mu +\frac{\sigma Y}{\xi Z}\sim \mathsf{G}\mathsf{P}\mathsf{D}(\mu ,\sigma ,\xi ).$ This is a consequence of the mixture after setting $\beta =\alpha$ and taking into account that the rate parameters of the exponential and gamma distribution are simply inverse multiplicative constants.

Exponentiated generalized Pareto distribution

The exponentiated generalized Pareto distribution (exGPD)

File:ExGPDpdf.png

The pdf of the $exGPD(\sigma ,\xi )$ (exponentiated generalized Pareto distribution) for different values $\sigma$ and $\xi$.

If $X\sim GPD$ $($$\mu =0$, $\sigma$, $\xi$ $)$, then $Y=\log (X)$ is distributed according to the exponentiated generalized Pareto distribution, denoted by $Y$ $\sim$ $exGPD$ $($$\sigma$, $\xi$ $)$.

The probability density function(pdf) of $Y$ $\sim$ $exGPD$ $($$\sigma$, $\xi$ $)(\sigma >0)$ is

$g_{(\sigma ,\xi )}(y)=\{\begin{array}{l}\frac{e^y}{\sigma }(1+\frac{\xi e^y}{\sigma }{)}^{-1/\xi -1}\text{for }\xi \ne 0,\\ \frac{1}{\sigma }{e}^{y-e^y/\sigma }\text{for }\xi =0,\end{array}$

where the support is $-\mathrm{\infty }<y<\mathrm{\infty }$ for $\xi \ge 0$, and $-\mathrm{\infty }<y\le \log (-\sigma /\xi )$ for $\xi <0$.

For all $\xi$, the $\log \sigma$ becomes the location parameter. See the right panel for the pdf when the shape $\xi$ is positive.

The exGPD has finite moments of all orders for all $\sigma >0$ and $-\mathrm{\infty }<\xi <\mathrm{\infty }$.

File:Var exGPD.png

The variance of the $exGPD(\sigma ,\xi )$ as a function of $\xi$. Note that the variance only depends on $\xi$. The red dotted line represents the variance evaluated at $\xi =0$, that is, ${\psi }^{\prime }(1)={\pi }^2/6$.

The moment-generating function of $Y\sim exGPD(\sigma ,\xi )$ is

$M_Y(s)=E[e^{sY}]=\{\begin{array}{l}-\frac{1}{\xi }(-\frac{\sigma }{\xi }{)}^{s}B(s+1,-1/\xi )\text{for }s\in (-1,\mathrm{\infty }),\xi <0,\\ \frac{1}{\xi }(\frac{\sigma }{\xi }{)}^{s}B(s+1,1/\xi -s)\text{for }s\in (-1,1/\xi ),\xi >0,\\ {\sigma }^s\mathrm{\Gamma }(1+s)\text{for }s\in (-1,\mathrm{\infty }),\xi =0,\end{array}$

where $B(a,b)$ and $\mathrm{\Gamma }(a)$ denote the beta function and gamma function, respectively.

The expected value of $Y$ $\sim$ $exGPD$ $($$\sigma$, $\xi$ $)$ depends on the scale $\sigma$ and shape $\xi$ parameters, while the $\xi$ participates through the digamma function:

$E[Y]=\{\begin{array}{l}\log (-\frac{\sigma }{\xi })+\psi (1)-\psi (-1/\xi +1)\text{for }\xi <0,\\ \log \left(\frac{\sigma }{\xi }\right)+\psi (1)-\psi (1/\xi )\text{for }\xi >0,\\ \log \sigma +\psi (1)\text{for }\xi =0.\end{array}$

Note that for a fixed value for the $\xi \in (-\mathrm{\infty },\mathrm{\infty })$, the $\log \sigma$ plays as the location parameter under the exponentiated generalized Pareto distribution.

The variance of $Y$ $\sim$ $exGPD$ $($$\sigma$, $\xi$ $)$ depends on the shape parameter $\xi$ only through the polygamma function of order 1 (also called the trigamma function):

$Var[Y]=\{\begin{array}{l}{\psi }^{\prime }(1)-{\psi }^{\prime }(-1/\xi +1)\text{for }\xi <0,\\ {\psi }^{\prime }(1)+{\psi }^{\prime }(1/\xi )\text{for }\xi >0,\\ {\psi }^{\prime }(1)\text{for }\xi =0.\end{array}$

See the right panel for the variance as a function of $\xi$. Note that ${\psi }^{\prime }(1)={\pi }^2/6\approx 1.644934$.

Note that the roles of the scale parameter $\sigma$ and the shape parameter $\xi$ under $Y\sim exGPD(\sigma ,\xi )$ are separably interpretable, which may lead to a robust efficient estimation for the $\xi$ than using the $X\sim GPD(\sigma ,\xi )$ [2]. The roles of the two parameters are associated each other under $X\sim GPD(\mu =0,\sigma ,\xi )$ (at least up to the second central moment); see the formula of variance $Var(X)$ wherein both parameters are participated.

The Hill's estimator

Assume that $X_{1:n}=(X_1,\cdots ,X_n)$ are $n$ observations (need not be i.i.d.) from an unknown heavy-tailed distribution $F$ such that its tail distribution is regularly varying with the tail-index $1/\xi$ (hence, the corresponding shape parameter is $\xi$). To be specific, the tail distribution is described as

$\overline{F}(x)=1-F(x)=L(x)\cdot x^{-1/\xi },\text{for some }\xi >0,\text{where }L\text{ is a slowly varying function.}$

It is of a particular interest in the extreme value theory to estimate the shape parameter $\xi$, especially when $\xi$ is positive (so called the heavy-tailed distribution).

Let $F_u$ be their conditional excess distribution function. Pickands–Balkema–de Haan theorem (Pickands, 1975; Balkema and de Haan, 1974) states that for a large class of underlying distribution functions $F$, and large $u$, $F_u$ is well approximated by the generalized Pareto distribution (GPD), which motivated Peak Over Threshold (POT) methods to estimate $\xi$: the GPD plays the key role in POT approach.

A renowned estimator using the POT methodology is the Hill's estimator. Technical formulation of the Hill's estimator is as follows. For $1\le i\le n$, write $X_{(i)}$ for the $i$-th largest value of $X_1,\cdots ,X_n$. Then, with this notation, the Hill's estimator (see page 190 of Reference 5 by Embrechts et al [3]) based on the $k$ upper order statistics is defined as

${\hat{\xi }}_k^{\text{Hill}}={\hat{\xi }}_k^{\text{Hill}}(X_{1:n})=\frac{1}{k-1}\sum _{j=1}^{k-1}\log \left(\frac{X_{(j)}}{X_{(k)}}\right),\text{for }2\le k\le n.$

In practice, the Hill estimator is used as follows. First, calculate the estimator ${\hat{\xi }}_k^{\text{Hill}}$ at each integer $k\in \{2,\cdots ,n\}$, and then plot the ordered pairs $\{(k,{\hat{\xi }}_k^{\text{Hill}}){\}}_{k=2}^n$. Then, select from the set of Hill estimators $\{{\hat{\xi }}_k^{\text{Hill}}{\}}_{k=2}^n$ which are roughly constant with respect to $k$: these stable values are regarded as reasonable estimates for the shape parameter $\xi$. If $X_1,\cdots ,X_n$ are i.i.d., then the Hill's estimator is a consistent estimator for the shape parameter $\xi$ [4].

Note that the Hill estimator ${\hat{\xi }}_k^{\text{Hill}}$ makes a use of the log-transformation for the observations $X_{1:n}=(X_1,\cdots ,X_n)$. (The Pickand's estimator ${\hat{\xi }}_k^{\text{Pickand}}$ also employed the log-transformation, but in a slightly different way [5].)

See also

• Burr distribution
• Pareto distribution
• Generalized extreme value distribution
• Exponentiated generalized Pareto distribution
• Pickands–Balkema–de Haan theorem

Further reading

• Chapter 20, Section 12: Generalized Pareto Distributions.

External links

• Mathworks: Generalized Pareto distribution