View source for Marginal distribution

{{Short description|Aspect of probability and statistics}}

In [[probability theory]] and [[statistics]], the '''marginal distribution''' of a [[subset]] of a [[indexed family|collection]] of [[random variable]]s is the [[probability distribution]] of the variables contained in the subset. It gives the probabilities of various values of the variables in the subset without reference to the values of the other variables. This contrasts with a [[conditional distribution]], which gives the probabilities contingent upon the values of the other variables.

'''Marginal variables''' are those variables in the subset of variables being retained. These concepts are "marginal" because they can be found by summing values in a table along rows or columns, and writing the sum in the margins of the table.<ref>{{cite book | author = Trumpler, Robert J. | author2 = Harold F. Weaver | name-list-style = amp | title = Statistical Astronomy | year = 1962 | publisher = Dover Publications |pages=32–33 }}</ref> The distribution of the marginal variables (the marginal distribution) is obtained by '''marginalizing''' (that is, focusing on the sums in the margin) over the distribution of the variables being discarded, and the discarded variables are said to have been '''marginalized out'''.

The context here is that the theoretical studies being undertaken, or the [[data analysis]] being done, involves a wider set of random variables but that attention is being limited to a reduced number of those variables. In many applications, an analysis may start with a given collection of random variables, then first extend the set by defining new ones (such as the sum of the original random variables) and finally reduce the number by placing interest in the marginal distribution of a subset (such as the sum). Several different analyses may be done, each treating a different subset of variables as the marginal distribution.

== Definition ==
=== Marginal probability mass function ===
Given a known [[joint distribution]] of two '''discrete''' [[random variable]]s, say, {{mvar|X}} and {{mvar|Y}}, the marginal distribution of either variable – {{mvar|X}} for example – is the [[probability distribution]] of {{mvar|X}} when the values of {{mvar|Y}} are not taken into consideration. This can be calculated by summing the [[joint probability]] distribution over all values of {{mvar|Y}}. Naturally, the converse is also true: the marginal distribution can be obtained for {{mvar|Y}} by summing over the separate values of {{mvar|X}}.

:<math>p_X(x_i)=\sum_{j}p(x_i,y_j)</math>, and <math> p_Y(y_j)=\sum_{i}p(x_i,y_j)</math>

{| class="wikitable" style="text-align: center; margin: 1em auto;"
! {{diagonal split header|''Y''|''X''}} || ''x''<sub>1</sub> || ''x''<sub>2</sub> || ''x''<sub>3</sub> || ''x''<sub>4</sub> || ''p<sub>Y</sub>''(''y'') ↓
|-
! ''y''<sub>1</sub>
| {{sfrac|4|32}}  || {{sfrac|2|32}} || {{sfrac|1|32}} || {{sfrac|1|32}}
! {{sfrac|8|32}}
|-
! ''y''<sub>2</sub>
| {{sfrac|3|32}} || {{sfrac|6|32}} || {{sfrac|3|32}} || {{sfrac|3|32}}
! {{sfrac|15|32}}
|-
! ''y''<sub>3</sub>
| {{sfrac|9|32}} || 0 || 0 || 0
! {{sfrac|9|32}}
|-
! ''p<sub>X</sub>''(''x'') →
! {{sfrac|16|32}} || {{sfrac|8|32}} || {{sfrac|4|32}} || {{sfrac|4|32}}
! {{sfrac|32|32}}
|-
|+ style="caption-side: bottom;" |{{nobold|Joint and marginal distributions of a pair of discrete random variables, ''X'' and ''Y'', dependent, thus having nonzero [[mutual information]] {{math|''I''(''X''; ''Y'')}}. The values of the joint distribution are in the 3×4 rectangle; the values of the marginal distributions are along the right and bottom margins.}}
|}
A '''marginal probability''' can always be written as an [[expected value]]:
<math display="block">p_X(x) = \int_y p_{X \mid Y}(x \mid y) \, p_Y(y) \, \mathrm{d}y = \operatorname{E}_{Y} [p_{X \mid Y}(x \mid Y)]\;.</math>

Intuitively, the marginal probability of ''X''  is computed by examining the conditional probability of ''X'' given a particular value of ''Y'', and then averaging this conditional probability over the distribution of all values of ''Y''.

This follows from the definition of [[expected value]] (after applying the [[law of the unconscious statistician]])
<math display="block">\operatorname{E}_Y [f(Y)] = \int_y f(y) p_Y(y) \, \mathrm{d}y.</math>

Therefore, marginalization provides the rule for the transformation of the probability distribution of a random variable ''Y'' and another random variable {{math|''X''{{=}}''g''(''Y'')}}:
<math display="block">p_X(x) = \int_y p_{X \mid Y}(x \mid y) \, p_Y(y) \, \mathrm{d}y = \int_y \delta\big(x - g(y)\big) \, p_Y(y) \, \mathrm{d}y.</math>

=== Marginal probability density function ===
Given two '''continuous''' [[random variable]]s ''X'' and ''Y'' whose [[joint distribution]] is known, then the marginal [[probability density function]] can be obtained by integrating the [[joint probability]] density, {{mvar|f}}, over ''Y,'' and vice versa. That is

:<math>f_X(x) = \int_{c}^{d} f(x,y) \, dy  </math>   
:<math>f_Y(y) = \int_{a}^{b} f(x,y) \, dx  </math>

where <math>x\in[a,b]</math>, and <math>y\in[c,d]</math>.

=== Marginal cumulative distribution function ===
Finding the marginal [[cumulative distribution function]] from the joint cumulative distribution function is easy. Recall that:

* For '''discrete [[random variable]]s''', <math display="block">F(x,y) = P(X\leq x, Y\leq y)</math>
* For '''continuous random variables''', <math display="block">F(x,y) = \int_{a}^{x} \int_{c}^{y} f(x',y') \, dy' dx'</math>

If ''X'' and ''Y'' jointly take values on [''a'', ''b''] × [''c'', ''d''] then

:<math>F_X(x)=F(x,d)</math> and <math>F_Y(y)=F(b,y)</math>

If ''d'' is ∞, then this becomes a limit <math display="inline">F_X(x) = \lim_{y \to \infty} F(x,y)</math>. Likewise for <math>F_Y(y)</math>.

== Marginal distribution vs. conditional distribution ==

=== Definition ===
The '''marginal probability''' is the probability of a single event occurring, independent of other events. A '''[[Conditional probability distribution|conditional probability]]''', on the other hand, is the probability that an event occurs given that another specific event ''has already'' occurred. This means that the calculation for one variable is dependent on another variable.<ref>{{Cite web|url=https://study.com/academy/lesson/marginal-conditional-probability-distributions-definition-examples.html|title=Marginal & Conditional Probability Distributions: Definition & Examples|website=Study.com|language=en|access-date=2019-11-16}}</ref>

The conditional distribution of a variable given another variable is the joint distribution of both variables divided by the marginal distribution of the other variable.<ref>{{Cite web|url=https://www.math.fsu.edu/~paris/Pexam/|title=Exam P [FSU Math]|website=www.math.fsu.edu|access-date=2019-11-16}}</ref> That is,

* For '''discrete [[random variable]]s''',<math display="block">p_{Y|X}(y|x) = P(Y=y \mid X=x) = \frac{P(X=x,Y=y)}{P_X(x)}</math>
* For '''continuous random variables''',<math display="block">f_{Y|X}(y|x)=\frac{f_{X,Y}(x,y)}{f_X(x)}</math>

=== Example ===
Suppose there is data from a classroom of 200 students on the amount of time studied (''X'') and the percentage of correct answers (''Y'').<ref>{{Citation|title=Marginal and conditional distributions|url=https://www.khanacademy.org/math/ap-statistics/analyzing-categorical-ap/distributions-two-way-tables/v/marginal-distribution-and-conditional-distribution|language=en|access-date=2019-11-16}}</ref> Assuming that ''X'' and ''Y'' are discrete random variables, the joint distribution of ''X'' and ''Y'' can be described by listing all the possible values of ''p''(''x<sub>i</sub>'',''y<sub>j</sub>''), as shown in Table.3.
{| class="wikitable" style="text-align: center; width:560px; margin: 1em auto;"
! {{diagonal split header|''Y''|''X''}} 
! colspan="6" |Time studied (minutes)
|-
| rowspan="7" {{vertical header|cellstyle=background-color:#ececec|'''% correct'''}}
! 
! scope="col" |''x''<sub>1</sub> (0-20)
! scope="col" |''x''<sub>2</sub> (21-40)
!''x''<sub>3</sub> (41-60)
!''x''<sub>4</sub>(>60)
!''p''<sub>''Y''</sub>(''y'') ↓
|-
|{{rh|align=center}} |''y''<sub>1</sub> (0-20)
|{{sfrac|2|200}}
|0
|0
|{{sfrac|8|200}}
|{{rh|align=center}} |{{sfrac|10|200}}
|-
|{{rh|align=center}} |''y''<sub>2</sub> (21-40)
|{{sfrac|10|200}}
|{{sfrac|2|200}}
|{{sfrac|8|200}}
|0
|{{rh|align=center}} |{{sfrac|20|200}}
|-
|{{rh|align=center}} |''y''<sub>3</sub> (41-59)
|{{sfrac|2|200}}
|{{sfrac|4|200}}
|{{sfrac|32|200}}
|{{sfrac|32|200}}
|{{rh|align=center}} |{{sfrac|70|200}}
|-
|{{rh|align=center}} |''y''<sub>4</sub> (60-79)
|0
|{{sfrac|20|200}}
|{{sfrac|30|200}}
|{{sfrac|10|200}}
|{{rh|align=center}} |{{sfrac|60|200}}
|-
|{{rh|align=center}} |''y''<sub>5</sub> (80-100)
| align="center" | 0
| align="center" | {{sfrac|4|200}}
| align="center" |{{sfrac|16|200}}
| align="center" |{{sfrac|20|200}}
| {{rh|align=center}} |{{sfrac|40|200}}
|-
|{{rh|align=center}} |''p<sub>X</sub>''(''x'') →
! align="center" |{{sfrac|14|200}}
! align="center" |{{sfrac|30|200}}
! align="center" |{{sfrac|86|200}}
! align="center" |{{sfrac|70|200}}
! align="center" |1
|-
|+ style="caption-side: bottom;" |{{nobold|[[Two-way table]] of dataset of the relationship in a classroom of 200 students between the amount of time studied and the percentage correct}}
|}
The '''marginal distribution''' can be used to determine how many students scored 20 or below: <small><math>p_Y(y_1) = P_Y(Y=y_1) = \sum_{i=1}^4 P(x_i,y_1) = \frac{2}{200} + \frac{8}{200} = \frac{10}{200}</math></small>, meaning 10 students or 5%.

The '''[[conditional distribution]]''' can be used to determine the probability that a student that studied 60 minutes or more obtains a scored of 20 or below: <small><math>p_{Y|X}(y_1|x_4) = P(Y=y_1|X=x_4) = \frac{P(X=x_4,Y=y_1)}{P(X=x_4)} = \frac{8/200}{70/200} = \frac{8}{70} = \frac{4}{35}</math></small>, meaning there is about a 11% probability of scoring 20 after having studied for at least 60 minutes.

==Real-world example==
Suppose that the probability that a pedestrian will be hit by a car, while crossing the road at a pedestrian crossing, without paying attention to the traffic light, is to be computed. Let H be a [[discrete random variable]] taking one value from {Hit, Not&nbsp;Hit}. Let L (for traffic light) be a discrete random variable taking one value from {Red, Yellow, Green}.

Realistically, H will be dependent on L. That is, P(H&nbsp;= Hit) will take different values depending on whether L is red, yellow or green (and likewise for P(H&nbsp;= Not&nbsp;Hit)). A person is, for example, far more likely to be hit by a car when trying to cross while the lights for perpendicular traffic are green than if they are red. In other words, for any given possible pair of values for H and L, one must consider the [[joint probability distribution]] of H and L to find the probability of that pair of events occurring together if the pedestrian ignores the state of the light.

However, in trying to calculate the '''marginal probability''' P(H&nbsp;= Hit), what is being sought is the probability that H&nbsp;= Hit in the situation in which the particular value of L is unknown and in which the pedestrian ignores the state of the light. In general, a pedestrian can be hit if the lights are red OR if the lights are yellow OR if the lights are green. So, the answer for the marginal probability can be found by summing P(H&nbsp;|&nbsp;L) for all possible values of L, with each value of L weighted by its probability of occurring.

Here is a table showing the conditional probabilities of being hit, depending on the state of the lights. (Note that the columns in this table must add up to 1 because the probability of being hit or not hit is 1 regardless of the state of the light.)

{| class="wikitable"
|+ Conditional distribution: <math>P(H\mid L)</math>
|-
! {{diagonal split header|H|L}}
! width="60"|Red
! width="60"|Yellow
! width="60"|Green
|-
! Not Hit
| align="center" | 0.99
| align="center" | 0.9
| align="center" | 0.2
|-
! Hit
| align="center" | 0.01
| align="center" | 0.1
| align="center" | 0.8
|}

To find the joint probability distribution, more data is required. For example, suppose P(L&nbsp;=&nbsp;red)&nbsp;= 0.2, P(L&nbsp;=&nbsp;yellow)&nbsp;= 0.1, and P(L&nbsp;=&nbsp;green)&nbsp;= 0.7. Multiplying each column in the conditional distribution by the probability of that column occurring results in the joint probability distribution of H and L, given in the central 2×3 block of entries. (Note that the cells in this 2×3 block add up to&nbsp;1).

{| class="wikitable"
|+ Joint distribution: {{tmath|P(H,L)}}
|-
! {{diagonal split header|H|L}}
! width="60"|Red
! width="60"|Yellow
! width="60"|Green
! width="60"|Marginal probability P(''H'')
|-
! Not Hit
| align="left" | 0.198
| align="left" | 0.09
| align="left" | 0.14
| align="left" | 0.428
|-
! Hit
| align="left" | 0.002
| align="left" | 0.01
| align="left" | 0.56
| align="left" | 0.572
|-
! Total
| align="left" | 0.2
| align="left" | 0.1
| align="left" | 0.7
| align="left" | 1
|}

The marginal probability P(H&nbsp;= Hit) is the sum 0.572 along the H&nbsp;= Hit row of this joint distribution table, as this is the probability of being hit when the lights are red OR yellow OR green. Similarly, the marginal probability that P(H&nbsp;= Not&nbsp;Hit) is the sum along the H&nbsp;= Not&nbsp;Hit row.

==Multivariate distributions==

[[File:MultivariateNormal.png|thumb|300px|Many samples from a bivariate normal distribution. The marginal distributions are shown in red and blue. The marginal distribution of X is also approximated by creating a histogram of the X coordinates without consideration of the Y coordinates.]]

For [[multivariate distribution]]s, formulae similar to those above apply with the symbols ''X'' and/or ''Y'' being interpreted as vectors. In particular, each summation or integration would be over all variables except those contained in ''X''.<ref name=":1">{{Cite book|title=A modern introduction to probability and statistics : understanding why and how|date=2005|publisher=Springer|others=Dekking, Michel, 1946-|isbn=9781852338961|location=London|oclc=262680588}}</ref>

That means, If ''X''<sub>1</sub>,''X''<sub>2</sub>,…,''X<sub>n</sub>'' are '''discrete [[random variable]]s''', then the marginal [[probability mass function]] should be
<math display="block">p_{X_i}(k)=\sum p(x_1,x_2,\dots,x_{i-1},k,x_{i+1},\dots,x_n);</math>
if ''X''<sub>1</sub>,''X''<sub>2</sub>,…,''X<sub>n</sub>'' are '''continuous random variables''', then the marginal [[probability density function]] should be
<math display="block">f_{X_i}(x_i)=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty} f(x_1,x_2,\dots,x_n) dx_1 dx_2 \cdots dx_{i-1} dx_{i+1} \cdots dx_n .</math>

==See also==
* [[Compound probability distribution]]
* [[Joint probability distribution]]
* [[Marginal likelihood]]
* [[Wasserstein metric]]
* [[Conditional probability distribution|Conditional distribution]]

==References==
{{Reflist}}

==Bibliography==
* {{cite book | author1-last = Everitt | author1-first= B. S. | author2-last = Skrondal | author2-first= A. | year = 2010 | title = Cambridge Dictionary of Statistics | publisher = [[Cambridge University Press]] }}
*{{Cite book|title=A modern introduction to probability and statistics|author=Dekking, F. M.|author2=Kraaikamp, C.|author3=Lopuhaä, H. P.|author4=Meester, L. E.|publisher=London : Springer|year=2005|isbn=9781852338961}}

{{Authority control}}

{{DEFAULTSORT:Marginal Distribution}}
[[Category:Theory of probability distributions]]