View source for Beta-binomial distribution

{{Short description|Discrete probability distribution}}
{{Probability distribution
  | pdf_image  =[[Image:Beta-binomial distribution pmf.png|325px|Probability mass function for the beta-binomial distribution]]|

 | cdf_image  =[[Image:Beta-binomial cdf.png|325px|Cumulative probability distribution function for the beta-binomial distribution]]|

  | type       = mass
  | notation   = <math>\mathrm{BetaBin}(n,\alpha,\beta)</math>
  | parameters = ''n'' ∈ [[Natural numbers|'''N'''<sub>0</sub>]] — number of trials<br /><math>\alpha > 0</math> ([[real number|real]]) <br /><math>\beta > 0</math> ([[real number|real]])
  | support    = ''x'' ∈ {&thinsp;0, …, ''n''&thinsp;}
  | pdf        =<math>\binom{n}{x} \frac{\mathrm{B}(x+\alpha,n-x+\beta)} {\mathrm{B}(\alpha,\beta)}\!</math> <br/><br/> where <math> \Beta(x,y)=\frac{\Gamma(x)\,\Gamma(y)}{\Gamma(x+y)}</math> is the [[beta function]]|
  | cdf        =<math>\begin{cases} 0,& x < 0 \\ \binom{n}{x} \tfrac{\mathrm{B}(x+\alpha,n-x+\beta)} {\mathrm{B}(\alpha,\beta)} {}_3\!F_2(\boldsymbol{a};\boldsymbol{b};x), & 0 \le x < n \\ 1,& x \geq n \end{cases}</math> <br/><br/>where <big> <sub>3</sub>''F''<sub>2</sub>('''a''';'''b''';x)</big> is the [[Generalized hypergeometric series|generalized hypergeometric function]]<br/> <small><math>{}_3\!F_2(1, -x, n\! -\! x\! +\! \beta; n\! -\! x\! +\! 1, 1\! -\! x\! -\! \alpha; 1)\!</math></small>|
  | mean       =<math>\frac{n\alpha}{\alpha+\beta}\!</math>|
  | median     =|
  | mode       = |
  | variance   =<math>\frac{n\alpha\beta(\alpha+\beta+n)}{(\alpha+\beta)^2(\alpha+\beta+1)}\!</math>|
  | skewness   =<math>\tfrac{(\alpha+\beta+2n)(\beta-\alpha)}{(\alpha+\beta+2)}\sqrt{\tfrac{1+\alpha+\beta}{n\alpha\beta(n+\alpha+\beta)}}\!</math>|
  | kurtosis   = See text|
  | entropy    =|
  | mgf        =<math>_{2}F_{1}(-n,\alpha;\alpha+\beta;1-e^{t})\!</math> where  <math>_{2}F_{1}</math> is the [[hypergeometric function]]|
  | char       =<math>_{2}F_{1}(-n,\alpha;\alpha+\beta;1-e^{it})\!</math>|
  | pgf        =<math>_{2}F_{1}(-n,\alpha;\alpha+\beta;1-z)\!</math>|

   }}

In [[probability theory]] and [[statistics]], the '''beta-binomial distribution''' is a family of discrete [[probability distribution]]s on a finite [[Support (mathematics)|support]] of non-negative integers arising when the probability of success in each of a fixed or known number of [[Bernoulli trial]]s is either unknown or random. The beta-binomial distribution is the [[binomial distribution]] in which the probability of success at each of ''n'' trials is not fixed but randomly drawn from a [[beta distribution]]. It is frequently used in [[Bayesian statistics]], [[empirical Bayes methods]] and [[classical statistics]] to capture [[overdispersion]] in [[binomial type]] distributed data.

The beta-binomial is a one-dimensional version of the [[Dirichlet-multinomial distribution]] as the binomial and beta distributions are univariate versions of the [[multinomial distribution|multinomial]] and [[Dirichlet distribution]]s respectively. The special case where ''α'' and ''β'' are integers is also known as the [[negative hypergeometric distribution]].

==Motivation and derivation==

===As a compound distribution===
The [[Beta distribution]] is a [[conjugate prior|conjugate distribution]] of the [[binomial distribution]]. This fact leads to an analytically tractable [[compound distribution]] where one can think of the <math> p </math> parameter in the binomial distribution as being randomly drawn from a beta distribution. 
Suppose we were interested in predicting the number of heads, <math>x</math> in <math>n</math> future trials. This is given by 

:<math>
 \begin{align}  f(x\mid n,\alpha,\beta) & = \int_0^1 \mathrm{Bin}(x|n,p)\mathrm{Beta}(p\mid \alpha, \beta) \, dp \\[6pt]
                           & = {n\choose x}\frac{1}{\mathrm{B}(\alpha,\beta)}
                               \int_0^1 p^{x+\alpha-1}(1-p)^{n-x+\beta-1} \, dp \\[6pt]
                           & = {n\choose x}\frac{\mathrm{B}(x+\alpha,n-x+\beta)} {\mathrm{B}(\alpha,\beta)}. 
 \end{align}
</math>
Using the properties of the [[beta function]], this can alternatively be written
:<math>
   f(x\mid n,\alpha,\beta) = \frac{\Gamma(n+1)\Gamma(x+\alpha)\Gamma(n-x+\beta)}{\Gamma(n+\alpha+\beta)\Gamma(x+1)\Gamma(n-x+1)} \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}

</math>

===As an urn model===
The beta-binomial distribution can also be motivated via an [[urn model]] for positive [[integer]] values of ''α'' and ''β'', known as the [[Pólya urn model]]. Specifically, imagine an urn containing ''α'' red balls and ''β'' black balls, where random draws are made. If a red ball is observed, then two red balls are returned to the urn. Likewise, if a black ball is drawn, then two black balls are returned to the urn. If this is repeated ''n'' times, then the probability of observing ''x'' red balls follows a beta-binomial distribution with parameters ''n'', ''α'' and&nbsp;''β''.

By contrast, if the random draws are with simple replacement (no balls over and above the observed ball are added to the urn), then the distribution follows a binomial distribution and if the random draws are made without replacement, the distribution follows a [[hypergeometric distribution]].

==Moments and properties==
The first three raw [[moment (mathematics)|moments]] are
::<math> 
 \begin{align} 
   \mu_1 & =\frac{n\alpha}{\alpha+\beta} \\[8pt]
   \mu_2 & =\frac{n\alpha[n(1+\alpha)+\beta]}{(\alpha+\beta)(1+\alpha+\beta)}\\[8pt]
   \mu_3 & =\frac{n\alpha[n^{2}(1+\alpha)(2+\alpha)+3n(1+\alpha)\beta+\beta(\beta-\alpha)]}{(\alpha+\beta)(1+\alpha+\beta)(2+\alpha+\beta)}
 \end{align}
</math>

and the [[kurtosis]] is
::<math> 
   \beta_2 = \frac{(\alpha + \beta)^2 (1+\alpha+\beta)}{n \alpha \beta( \alpha + \beta + 2)(\alpha + \beta + 3)(\alpha + \beta + n) } \left[ (\alpha + \beta)(\alpha + \beta - 1 + 6n) + 3 \alpha\beta(n - 2) + 6n^2 -\frac{3\alpha\beta n(6-n)}{\alpha + \beta} - \frac{18\alpha\beta n^{2}}{(\alpha+\beta)^2} \right].
</math>

Letting <math>p=\frac{\alpha}{\alpha+\beta} \!</math> we note, suggestively, that the mean can be written as
::<math>
\mu = \frac{n\alpha}{\alpha+\beta}=np
\!</math>

and the variance as

::<math>
\sigma^2 = \frac{n\alpha\beta(\alpha+\beta+n)}{(\alpha+\beta)^2(\alpha+\beta+1)}
 = np(1-p) \frac{\alpha + \beta + n}{\alpha + \beta + 1} = np(1-p)[1+(n-1)\rho]
\!</math>

where <math>\rho= \tfrac{1}{\alpha+\beta+1}\!</math>. The parameter <math>\rho \; \!</math>  is known as the "intra class" or "intra cluster" correlation. It is this positive correlation which gives rise to overdispersion. Note that when <math>n=1</math>, no information is available to distinguish between the beta and binomial variation, and the two models have equal variances.

===Factorial moments===
The {{math|''r''}}-th [[factorial moment]] of a Beta-binomial random variable {{math|''X''}} is

:<math>\operatorname{E}\bigl[(X)_r\bigr] = \frac{n!}{(n-r)!}\frac{B(\alpha+r,\beta)}{B(\alpha,\beta)} =
(n)_r \frac{B(\alpha+r,\beta)}{B(\alpha,\beta)} </math>.

==Point estimates==

===Method of moments===
The [[Method of moments (statistics)|method of moments]] estimates can be gained by noting the first and second moments of the beta-binomial and setting those equal to the sample moments <math>m_1</math> and <math>m_2</math>. We find

::<math>
 \begin{align} 
   \widehat{\alpha} & =\frac{nm_1-m_2}{n(\frac{m_2}{m_1}-m_1-1)+m_1} \\[5pt]
   \widehat{\beta} & =\frac{(n-m_1)(n-\frac{m_2}{m_1})}{n(\frac{m_2}{m_1}-m_1 - 1)+m_1}.
 \end{align}
</math>
These estimates can be non-sensically negative which is evidence that the data is either undispersed or underdispersed relative to the binomial distribution. In this case, the binomial distribution and the [[hypergeometric distribution]] are alternative candidates respectively.

===Maximum likelihood estimation===

While closed-form [[Maximum likelihood|maximum likelihood estimates]] are impractical, given that the pdf consists of common functions ([[gamma function]] and/or Beta functions), they can be easily found via direct numerical optimization. Maximum likelihood estimates from empirical data can be computed using general methods for fitting multinomial Pólya distributions, methods for which are described in [[#Minka2003|(Minka 2003).]]
The [[R (programming language)|R]] package VGAM through the function vglm, via maximum likelihood, facilitates the fitting of [[generalized linear model|glm]] type models with responses distributed according to the beta-binomial distribution. There is no requirement that n is fixed throughout the observations.

===Example: Sex ratio heterogeneity===

The following data gives the number of male children among the first 12 children of family size 13 in 6115 families taken from hospital records in 19th century [[Saxony]] (Sokal and Rohlf, p.&nbsp;59 from Lindsey). The 13th child is ignored to blunt the effect of families non-randomly stopping when a desired gender is reached.
{| class="wikitable"
|-
| '''Males'''|| 0|| 1|| 2|| 3|| 4|| 5|| 6|| 7|| 8|| 9|| 10|| 11|| 12
|-
| '''Families'''|| 3|| 24|| 104|| 286|| 670|| 1033|| 1343|| 1112|| 829|| 478|| 181|| 45|| 7
|}

The first two sample moments are
::<math> 
 \begin{align} 
   m_1 & = 6.23\\
   m_2 & = 42.31 \\
     n & = 12
 \end{align}
</math>
and therefore the method of moments estimates are

::<math> 
 \begin{align} 
   \widehat{\alpha} & = 34.1350\\
   \widehat{\beta} & = 31.6085.
 \end{align}
</math>

The [[maximum likelihood]] estimates can be found numerically
::<math> 
 \begin{align} 
   \widehat\alpha_\mathrm{mle} & = 34.09558\\
   \widehat\beta_\mathrm{mle} & = 31.5715
 \end{align}
</math>
and the maximized log-likelihood is

::<math>
   \log \mathcal{L} = -12492.9
</math>

from which we find the [[Akaike information criterion|AIC]]

::<math>
   \mathit{AIC}=24989.74.
</math>

The AIC for the competing binomial model is AIC&nbsp;=&nbsp;25070.34 and thus we see that the beta-binomial model provides a superior fit to the data i.e. there is evidence for overdispersion. [[Trivers&ndash;Willard hypothesis|Trivers and Willard]] postulate a theoretical justification for heterogeneity in gender-proneness among [[mammal|mammalian]] offspring.

The superior fit is evident especially among the tails

{| class="wikitable"
|-
| '''Males'''|| 0|| 1|| 2|| 3|| 4|| 5|| 6|| 7|| 8|| 9|| 10|| 11|| 12
|-
| '''Observed Families'''|| 3|| 24|| 104|| 286|| 670|| 1033|| 1343|| 1112|| 829|| 478|| 181|| 45|| 7
|-
| '''Fitted Expected (Beta-Binomial)'''|| 2.3|| 22.6|| 104.8|| 310.9|| 655.7|| 1036.2|| 1257.9|| 1182.1|| 853.6|| 461.9|| 177.9|| 43.8|| 5.2
|-
| '''Fitted Expected (Binomial ''p'' = 0.519215)'''|| 0.9|| 12.1|| 71.8|| 258.5|| 628.1|| 1085.2|| 1367.3|| 1265.6|| 854.2|| 410.0|| 132.8|| 26.1|| 2.3
|}

==Role in Bayesian statistics==

The beta-binomial distribution plays a prominent role in the Bayesian estimation of a Bernoulli success probability <math>p</math> which we wish to estimate based on data. Let <math>\mathbf{X}=\{X_1, X_2, \cdots X_{n_1}\} </math> be a [[sample (statistics)|sample]] of [[independent and identically distributed]] Bernoulli random variables <math>X_i \sim \text{Bernoulli}(p)</math>. Suppose, our knowledge of <math>p</math> - in Bayesian fashion - is uncertain and is modeled by the [[prior distribution]] <math>p \sim \text{Beta}(\alpha,\beta)</math>. If <math>Y_1=\sum_{i=1}^{n_1} X_i</math> then through [[compound distribution|compounding]], the [[Posterior_predictive_distribution#Prior_vs._posterior_predictive_distribution | prior predictive distribution]] of 
:<math>
Y_1 \sim \text{BetaBin}(n_1, \alpha,\beta)
</math>.

After observing <math>Y_1</math> we note that the [[posterior distribution]] for <math>p</math>

:<math>
\begin{align}
f(p|\mathbf{X},\alpha,\beta) & \propto \left(\prod_{i=1}^{n_1} p^{x_i}(1-p)^{1-x_i} \right)p^{\alpha-1}(1-p)^{\beta-1}\\
 & = Cp^{\sum x_i +\alpha-1}(1-p)^{n_1 -\sum x_i +\beta-1} \\
& = Cp^{y_1 +\alpha-1}(1-p)^{n_1-y_1 +\beta-1} 
\end{align}
</math>

where <math>C</math> is a [[normalizing constant]]. We recognize the posterior distribution as a <math>\mathrm{Beta}(y_1+\alpha,n_1-y_1+\beta)</math>.

Thus, again through compounding, we find that the [[posterior predictive distribution]] of a sum of a future sample of size <math>n_2</math> of <math>\mathrm{Bernoulli}(p)</math> random variables is 

:<math>
Y_2 \sim \mathrm{BetaBin}(n_2, y_1+\alpha, n_1-y_1+\beta)
</math>.

==Generating random variates==
 
To draw a beta-binomial random variate <math>X \sim \mathrm{BetaBin}(n, \alpha,\beta)</math> simply draw <math>p \sim \mathrm{Beta}(\alpha,\beta) </math> and then draw <math>X \sim \mathrm{B}(n,p)</math>.

==Related distributions==
* <math>\mathrm{BetaBin}(1, \alpha, \beta) \sim \mathrm{Bernoulli}(p)\,</math> where <math>p=\frac{\alpha}{\alpha+\beta}\,</math>.
* <math>\mathrm{BetaBin}(n, 1, 1) \sim U(0,n)\,</math> where <math>U(a,b)\,</math> is the [[Uniform distribution (discrete)|discrete uniform distribution]].
* If <math>X \sim \mathrm{BetaBin}(n, \alpha, \beta) \,</math> then <math> (n-X) \sim \mathrm{BetaBin}(n,\beta, \alpha) \,</math> 
* <math> \lim_{s \rightarrow \infty} \mathrm{BetaBin}(n, ps, (1-p)s) \sim \mathrm{B}(n,p)\,</math> where <math>p=\frac{\alpha}{\alpha+\beta}\,</math> and <math>s=\alpha+\beta\,</math> and <math>\mathrm{B}(n,p)\,</math> is the [[binomial distribution]].
* <math>\lim_{n \rightarrow \infty} \mathrm{BetaBin}(n, n \lambda , n^2) \sim \mathrm{Pois}(\lambda)\,</math> where <math>\mathrm{Pois}(\lambda)\,</math> is the [[Poisson distribution]].
* <math>\lim_{n \rightarrow \infty} \mathrm{BetaBin}(n, 1, \frac{np}{(1-p)}) \sim \mathrm{Geom}(p)\,</math> where <math>\mathrm{Geom}(p)\,</math> is the [[geometric distribution]].
* <math>\lim_{n \rightarrow \infty} \mathrm{BetaBin}(n, r, \frac{np}{(1-p)}) \sim \mathrm{NB}(r,p)\,</math> where <math>\mathrm{NB}(r,p)\,</math> is the [[negative binomial distribution]].

==See also==
* [[Dirichlet-multinomial distribution]]

== References ==
* {{anchor|Minka2003}}Minka, Thomas P. (2003). [http://research.microsoft.com/~minka/papers/dirichlet/ Estimating a Dirichlet distribution]. Microsoft Technical Report.

==External links==
* [http://it.stlawu.edu/~msch/biometrics/papers.htm Using the Beta-binomial distribution to assess performance of a biometric identification device]
* [http://research.microsoft.com/~minka/software/fastfit/ Fastfit] contains Matlab code for fitting Beta-Binomial distributions (in the form of two-dimensional Pólya distributions) to data.
* Interactive graphic: [http://www.math.wm.edu/~leemis/chart/UDR/UDR.html Univariate Distribution Relationships]
* [https://cran.r-project.org/web/packages/VGAM/index.html Beta-binomial functions in VGAM R package]
* [http://foundry.sandia.gov/releases/latest/javadoc-api/gov/sandia/cognition/statistics/distribution/BetaBinomialDistribution.html Beta-binomial distribution in Sandia National Labs Cognitive Foundry Java library]

{{Probability distributions}}

{{DEFAULTSORT:Beta-Binomial Distribution}}
[[Category:Discrete distributions]]
[[Category:Compound probability distributions]]
[[Category:Conjugate prior distributions]]