imported>Arjayay: Duplicate word removed

2024-12-26T20:40:12Z

Duplicate word removed

New page

{{Short description|Theorem in probability and statistics}}
In [[probability theory]] and [[statistics]], the '''law of the unconscious statistician''', or '''LOTUS''', is a theorem which expresses the [[expected value]] of a [[Function (mathematics)|function]] {{math|''g''(''X'')}} of a [[random variable]] {{mvar|X}} in terms of {{mvar|g}} and the [[probability distribution]] of {{mvar|X}}.

The form of the law depends on the type of random variable {{mvar|X}} in question. If the distribution of {{mvar|X}} is [[discrete probability distribution|discrete]] and one knows its [[probability mass function]] {{math|''pX''}}, then the expected value of {{math|''g''(''X'')}} is
<math display="block">\operatorname{E}[g(X)] = \sum_x g(x) p_X(x), \,</math>
where the sum is over all possible values {{mvar|x}} of {{mvar|X}}. If instead the distribution of {{mvar|X}} is [[continuous probability distribution|continuous]] with [[probability density function]] {{math|''f''''X''}}, then the expected value of {{math|''g''(''X'')}} is
<math display="block">\operatorname{E}[g(X)] = \int_{-\infty}^\infty g(x) f_X(x) \, \mathrm{d}x </math>

Both of these special cases can be expressed in terms of the [[cumulative probability distribution function]] {{math|''F''''X''}} of {{mvar|X}}, with the expected value of {{math|''g''(''X'')}} now given by the [[Lebesgue–Stieltjes integral]]
<math display="block">\operatorname{E}[g(X)] = \int_{-\infty}^\infty g(x) \, \mathrm{d}F_X(x). </math>

In even greater generality, {{mvar|X}} could be a [[random element]] in any [[measurable space]], in which case the law is given in terms of [[measure theory]] and the [[Lebesgue integral]]. In this setting, there is no need to restrict the context to [[probability measure]]s, and the law becomes a general theorem of [[mathematical analysis]] on Lebesgue integration relative to a [[pushforward measure]].

==Etymology==
This proposition is (sometimes) known as the ''law of the unconscious statistician'' because of a purported tendency to think of the aforementioned law as the very definition of the expected value of a function {{math|''g''(''X'')}} and a random variable {{math|''X''}}, rather than (more formally) as a consequence of the true definition of expected value.{{sfnm|1a1=DeGroot|1a2=Schervish|1y=2014|1pp=213−214}} The naming is sometimes attributed to [[Sheldon Ross]]' textbook ''Introduction to Probability Models'', although he removed the reference in later editions.{{sfnm|1a1=Casella|1a2=Berger|1y=2001|1loc=Section 2.2|2a1=Ross|2y=2019}} Many statistics textbooks do present the result as the definition of expected value.{{sfnm|1a1=Casella|1a2=Berger|1y=2001|1loc=Section 2.2}}

==Joint distributions==
A similar property holds for [[joint distribution]]s, or equivalently, for [[random vector]]s. For discrete random variables ''X'' and ''Y'', a function of two variables ''g'', and joint probability mass function <math>p_{X, Y}(x, y)</math>:{{sfnm|1a1=Ross|1y=2019}}
<math display="block">\operatorname{E}[g(X, Y)] = \sum_y \sum_x g(x, y) p_{X, Y}(x, y)</math>
In the [[Absolute continuity#Absolute continuity of functions|absolutely continuous]] case, with <math>f_{X, Y}(x, y)</math> being the joint probability density function,
<math display="block">\operatorname{E}[g(X, Y)] = \int_{-\infty}^\infty \int_{-\infty}^\infty g(x, y) f_{X, Y}(x, y) \, \mathrm{d}x \, \mathrm{d}y</math>

==Special cases==
A number of special cases are given here. In the simplest case, where the random variable {{mvar|X}} takes on countably many values (so that its distribution is discrete), the proof is particularly simple, and holds without modification if {{mvar|X}} is a discrete [[random vector]] or even a discrete [[random element]].

The case of a [[continuous random variable]] is more subtle, since the proof in generality requires subtle forms of the change-of-variables formula for integration. However, in the framework of [[measure theory]], the discrete case generalizes straightforwardly to general (not necessarily discrete) [[random element]]s, and the case of a continuous random variable is then a special case by making use of the [[Radon–Nikodym theorem]].

===Discrete case===
Suppose that {{mvar|X}} is a random variable which takes on only finitely or countably many different values {{math|''x''1, ''x''2, ...}}, with probabilities {{math|''p''1, ''p''2, ...}}. Then for any function {{mvar|g}} of these values, the random variable {{math|''g''(''X'')}} has values {{math|''g''(''x''1), ''g''(''x''2), ...}}, although some of these may coincide with each other. For example, this is the case if {{math|''X''}} can take on both values {{math|1}} and {{math|−1}} and {{math|''g''(''x'') {{=}} ''x''2}}.

Let {{math|''y''1, ''y''2, ...}} enumerate the possible ''distinct'' values of <math>g(X)</math>, and for each {{mvar|i}} let {{math|''I''''i''}} denote the collection of all {{mvar|j}} with {{math|''g''(''x''''j'') {{=}} ''y''''i''}}. Then, according to the definition of expected value, there is
<math display="block">\operatorname{E}[g(X)]=\sum_i y_i p_{g(X)}(y_i).</math>

Since a <math>y_i</math> can be the image of multiple, distinct <math>x_j</math>, it holds that
<math display="block">p_{g(X)}(y_i) = \sum_{j \in I_i} p_X(x_j).</math>

Then the expected value can be rewritten as
<math display="block">\sum_i y_i p_{g(X)}(y_i) = \sum_i y_i \sum_{j \in I_i} p_X(x_j) = \sum_i \sum_{j \in I_i} g(x_j) p_X(x_j) = \sum_x g(x)p_X(x).</math>
This equality relates the average of the outputs of {{math|''g''(''X'')}} as weighted by the probabilities of the outputs themselves to the average of the outputs of {{math|''g''(''X'')}} as weighted by the probabilities of the outputs of {{mvar|X}}.

If {{mvar|X}} takes on only finitely many possible values, the above is fully rigorous. However, if {{mvar|X}} takes on countably many values, the last equality given does not always hold, as seen by the [[Riemann series theorem]]. Because of this, it is necessary to assume the [[absolute convergence]] of the sums in question.{{sfnm|1a1=Feller|1y=1968|1loc=Section IX.2}}

===Continuous case===
Suppose that {{mvar|X}} is a random variable whose distribution has a continuous density {{mvar|f}}. If {{mvar|g}} is a general function, then the probability that {{math|''g''(''X'')}} is valued in a set of real numbers {{mvar|K}} equals the probability that {{mvar|X}} is valued in {{math|''g''−1(''K'')}}, which is given by
<math display="block">\int_{g^{-1}(K)} f(x)\,\mathrm{d}x.</math>
Under various conditions on {{mvar|g}}, the [[Integration by substitution|change-of-variables formula for integration]] can be applied to relate this to an integral over {{mvar|K}}, and hence to identify the density of {{math|''g''(''X'')}} in terms of the density of {{mvar|X}}. In the simplest case, if {{mvar|g}} is differentiable with nowhere-vanishing derivative, then the above integral can be written as
<math display="block">\int_K f(g^{-1}(y))(g^{-1})'(y)\,\mathrm{d}y,</math>
thereby identifying {{math|''g''(''X'')}} as possessing the density {{math|''f'' (''g''−1(''y''))(''g''−1)′(''y'')}}. The expected value of {{math|''g''(''X'')}} is then identified as
<math display="block">\int_{-\infty}^\infty yf(g^{-1}(y))(g^{-1})'(y)\,\mathrm{d}y=\int_{-\infty}^\infty g(x)f(x)\,\mathrm{d}x,</math>
where the equality follows by another use of the change-of-variables formula for integration. This shows that the expected value of {{math|''g''(''X'')}} is encoded entirely by the function {{mvar|g}} and the density {{mvar|f}} of {{mvar|X}}.{{sfnm|1a1=Papoulis|1a2=Pillai|1y=2002|1loc=Chapter 5}}

The assumption that {{mvar|g}} is differentiable with nonvanishing derivative, which is necessary for applying the usual change-of-variables formula, excludes many typical cases, such as {{math|''g''(''x'') {{=}} ''x''2}}. The result still holds true in these broader settings, although the proof requires more sophisticated results from [[mathematical analysis]] such as [[Sard's theorem]] and the [[coarea formula]]. In even greater generality, using the [[Lebesgue integral|Lebesgue theory]] as below, it can be found that the identity
<math display="block">\operatorname{E}[g(X)]=\int_{-\infty}^\infty g(x)f(x)\,\mathrm{d}x</math>
holds true whenever {{mvar|X}} has a density {{mvar|f}} (which does not have to be continuous) and whenever {{mvar|g}} is a [[measurable function]] for which {{math|''g''(''X'')}} has finite expected value. (Every continuous function is measurable.) Furthermore, without modification to the proof, this holds even if {{mvar|X}} is a [[random vector]] (with density) and {{mvar|g}} is a multivariable function; the integral is then taken over the multi-dimensional range of values of {{mvar|X}}.

===Measure-theoretic formulation===
An abstract and general form of the result is available using the framework of [[measure theory]] and the [[Lebesgue integral]]. Here, the setting is that of a [[measure space]] {{math|(Ω, ''μ'')}} and a [[measurable map]] {{math|''X''}} from {{math|Ω}} to a [[measurable space]] {{math|Ω'}}. The theorem then says that for any measurable function {{mvar|g}} on {{math|Ω'}} which is valued in [[real numbers]] (or even the [[extended real number line]]), there is
<math display="block"> \int_\Omega g \circ X \, \mathrm{d}\mu = \int_{\Omega'} g \, \mathrm{d}(X_\sharp \mu),</math>
(interpreted as saying, in particular, that either side of the equality exists if the other side exists). Here {{math|''X''♯ ''μ''}} denotes the [[pushforward measure]] on {{math|Ω′}}. The 'discrete case' given above is the special case arising when {{mvar|X}} takes on only countably many values and {{mvar|μ}} is a [[probability measure]]. In fact, the discrete case (although without the restriction to probability measures) is the first step in proving the general measure-theoretic formulation, as the general version follows therefrom by an application of the [[monotone convergence theorem]].{{sfnm|1a1=Bogachev|1y=2007|1loc=Section 3.6|2a1=Cohn|2y=2013|2loc=Section 2.6|3a1=Halmos|3y=1950|3loc=Section 39}} Without any major changes, the result can also be formulated in the setting of [[outer measure]]s.{{sfnm|1a1=Federer|1y=1969|1loc=Section 2.4}}

If {{mvar|μ}} is a [[σ-finite measure]], the theory of the [[Radon–Nikodym derivative]] is applicable. In the special case that the measure {{math|''X''♯ ''μ''}} is [[absolutely continuous]] relative to some background σ-finite measure {{mvar|ν}} on {{math|Ω′}}, there is a real-valued function {{math|''f''''X''}} on {{math|Ω'}} representing the [[Radon–Nikodym derivative]] of the two measures, and then
<math display="block">\int_{\Omega'} g \, \mathrm{d}(X_\sharp \mu)=\int_{\Omega'}gf_X\,\mathrm{d}\nu.</math>
In the further special case that {{math|Ω′}} is the [[real number line]], as in the contexts discussed above, it is natural to take {{math|''ν''}} to be the [[Lebesgue measure]], and this then recovers the 'continuous case' given above whenever {{math|''μ''}} is a [[probability measure]]. (In this special case, the condition of σ-finiteness is vacuous, since Lebesgue measure and every probability measure are trivially σ-finite.){{sfnm|1a1=Halmos|1y=1950|1loc=Section 39}}

==References==
{{Reflist|30em}}
{{refbegin}}
* {{cite book|mr=2267655|last1=Bogachev|first1=V. I.|title=Measure theory. Volume I|publisher=[[Springer-Verlag]]|location=Berlin|year=2007|isbn=978-3-540-34513-8|author-link1=Vladimir Bogachev|zbl=1120.28001|doi=10.1007/978-3-540-34514-5}}
*{{cite book|last1=Casella|first1=George|last2=Berger|first2=Roger L.|title=Statistical inference|series=Duxbury Advanced Series|publisher=Duxbury|location=Pacific Grove, CA|year=2001|edition=Second edition of 1990 original|isbn=0-534-11958-1|author-link1=George Casella|author-link2=Roger Lee Berger|zbl=0699.62001}}
* {{cite book|mr=3098996|last1=Cohn|first1=Donald L.|title=Measure theory|edition=Second edition of 1980 original|series=Birkhäuser Advanced Texts: Basler Lehrbücher|publisher=[[Birkhäuser|Birkhäuser/Springer]]|location=New York|year=2013|isbn=978-1-4614-6955-1|doi=10.1007/978-1-4614-6956-8|zbl=1292.28002}}
*{{cite book | last1 = DeGroot| first1 = Morris H. | last2 = Schervish | first2 = Mark J. | title = Probability and statistics | publisher = [[Pearson Education]] | edition = Fourth edition of 1975 original | year = 2014|mr=0373075|author-link1=Morris H. DeGroot|zbl=0619.62001|isbn=0-321-50046-6}}
*{{cite book| last = Federer| first = Herbert| author-link1=Herbert Federer|title = Geometric measure theory| place= Berlin–Heidelberg–New York| publisher = [[Springer-Verlag]]| series = Die Grundlehren der mathematischen Wissenschaften| volume = 153| year = 1969| isbn = 978-3-540-60656-7| mr=0257325| zbl= 0176.00801 | doi=10.1007/978-3-642-62010-2}}
*{{cite book|last1=Feller|first1=William|title=An introduction to probability theory and its applications. Volume I|edition=Third edition of 1950 original|publisher=[[John Wiley & Sons, Inc.]]|location=New York–London–Sydney|year=1968|author-link1=William Feller|mr=0228020|zbl=0155.23101}}
* {{cite book|mr=0033869|last1=Halmos|first1=Paul R.|title=Measure theory|publisher=[[D. Van Nostrand Co., Inc.]]|location=New York|year=1950|author-link1=Paul Halmos|doi=10.1007/978-1-4684-9440-2|zbl=0040.16802}}
*{{cite book|last1=Papoulis|first1=Athanasios|last2=Pillai|first2=S. Unnikrishna|title=Probability, random variables, and stochastic processes|edition=Fourth edition of 1965 original|author-link1=Athanasios Papoulis|author-link2=Unnikrishna Pillai|year=2002|publisher=[[McGraw-Hill]]|location=New York|isbn=0-07-366011-6}}
*{{cite book|last1=Ross|first1=Sheldon M.|title=Introduction to probability models|edition=Twelfth edition of 1972 original|publisher=[[Academic Press]]|location=London|year=2019|isbn=978-0-12-814346-9|mr=3931305|author-link1=Sheldon M. Ross|doi=10.1016/C2017-0-01324-1|zbl=1408.60002}}
{{refend}}

[[Category:Theory of probability distributions]]
[[Category:Statistical laws]]

Law of the unconscious statistician - Revision history

imported>Arjayay: Duplicate word removed