Likelihood function - Revision history

imported>Olexa Riznyk: Improving references

2026-07-04T07:17:09Z

Improving references

imported>WikiEditor50: Lowercase "rule"

2025-12-26T04:20:51Z

Lowercase "rule"

imported>Jonathan Teagan: linked converse

2025-10-09T20:25:18Z

linked converse

Show changes

imported>Hellacioussatyr: /* Example: the gamma distribution */

2025-03-03T13:13:20Z

Example: the gamma distribution

Show changes

← Previous revision		Revision as of 07:17, 4 July 2026
Line 2:		Line 2:
	{{Technical\|date=August 2025}}		{{Technical\|date=August 2025}}
	{{Bayesian statistics}}		{{Bayesian statistics}}
	A '''likelihood function''' (often simply called the '''likelihood''') measures how well a [[statistical model]] explains [[Realization (probability)\|observed data]] by calculating the probability of seeing that data under different [[Statistical parameter\|parameter]] values of the model. It is constructed from the [[joint probability distribution]] of the [[random variable]] that (presumably) generated the observations.<ref>{{cite book \|first1=George \|last1=Casella \|first2=Roger L. \|last2=Berger \|title=Statistical Inference \|location= \|publisher=Duxbury \|edition=2nd \|year=2002 \|isbn=0-534-24312-6 \|page=290 }}</ref><ref>{{cite book \|first=Jon \|last=Wakefield \|title=Frequentist and Bayesian Regression Methods \|location= \|publisher=Springer \|edition=1st \|year=2013 \|isbn=978-1-4419-0925-1 \|page=36 }}</ref><ref>{{cite book \|first1 = Erich L. \|last1=Lehmann \| first2 = George \|last2 = Casella \|title=Theory of Point Estimation \|location= \|publisher=Springer \|edition=2nd \|year=1998 \|isbn= 0-387-98502-6 \|page=444 }}</ref> When evaluated on the actual data points, it becomes a function solely of the model parameters.		A '''likelihood function''' (often simply called the '''likelihood''') measures how well a [[statistical model]] explains [[Realization (probability)\|observed data]] by calculating the probability of seeing that data under different [[Statistical parameter\|parameter]] values of the model. It is constructed from the [[joint probability distribution]] of the [[random variable]] that (presumably) generated the observations.<ref>{{cite book \|author1-link = George Casella \|author2-link=Roger Lee Berger \|first1=George \|last1=Casella \|first2=Roger L. \|last2=Berger \|title=Statistical Inference \|location= \|publisher=Duxbury \|edition=2nd \|year=2002 \|isbn=0-534-24312-6 \|page=290 }}</ref><ref>{{cite book \|first=Jon \|last=Wakefield \|title=Frequentist and Bayesian Regression Methods \|location= \|publisher=Springer \|edition=1st \|year=2013 \|isbn=978-1-4419-0925-1 \|page=36 }}</ref><ref>{{cite book \|first1 = Erich L. \|last1=Lehmann \| first2 = George \|last2 = Casella \|title=Theory of Point Estimation \|location= \|publisher=Springer \|edition=2nd \|year=1998 \|isbn= 0-387-98502-6 \|page=444 }}</ref> When evaluated on the actual data points, it becomes a function solely of the model parameters.

	In [[maximum likelihood estimation]], the model parameter(s) or [[arg max\|argument that maximizes]] the likelihood function serves as a [[Point estimation\|point estimate]] for the unknown parameter, while the [[Fisher information]] (often approximated by the likelihood's [[Hessian matrix]] at the maximum) gives an indication of the estimate's [[Precision (statistics)\|precision]].		In [[maximum likelihood estimation]], the model parameter(s) or [[arg max\|argument that maximizes]] the likelihood function serves as a [[Point estimation\|point estimate]] for the unknown parameter, while the [[Fisher information]] (often approximated by the likelihood's [[Hessian matrix]] at the maximum) gives an indication of the estimate's [[Precision (statistics)\|precision]].
Line 31:		Line 31:

	====Example====		====Example====
	[[Image:likelihoodFunctionAfterHH.png\|thumb\|400px\|Figure 1.~~ ~~ The likelihood function (<math display="inline">p_\text{H}^2</math>) for the probability of a coin landing heads-up (without prior knowledge of the coin's fairness), given that we have observed HH.]]		[[Image:likelihoodFunctionAfterHH.png\|thumb\|400px\|Figure 1. The likelihood function (<math display="inline">p_\text{H}^2</math>) for the probability of a coin landing heads-up (without prior knowledge of the coin's fairness), given that we have observed HH.]]
	[[Image:likelihoodFunctionAfterHHT.png\|thumb\|400px\|Figure 2.~~ ~~ The likelihood function (<math display="inline">p_\text{H}^2(1-p_\text{H})</math>) for the probability of a coin landing heads-up (without prior knowledge of the coin's fairness), given that we have observed HHT.]]		[[Image:likelihoodFunctionAfterHHT.png\|thumb\|400px\|Figure 2. The likelihood function (<math display="inline">p_\text{H}^2(1-p_\text{H})</math>) for the probability of a coin landing heads-up (without prior knowledge of the coin's fairness), given that we have observed HHT.]]
	Consider a simple statistical model of a coin flip: a single parameter <math display="inline">p_\text{H}</math> that expresses the "fairness" of the coin. The parameter is the probability that a coin lands heads up ("H") when tossed. <math display="inline">p_\text{H}</math> can take on any value within the range 0.0 to 1.0. For a perfectly [[fair coin]], <math display="inline">p_\text{H} = 0.5</math>.		Consider a simple statistical model of a coin flip: a single parameter <math display="inline">p_\text{H}</math> that expresses the "fairness" of the coin. The parameter is the probability that a coin lands heads up ("H") when tossed. <math display="inline">p_\text{H}</math> can take on any value within the range 0.0 to 1.0. For a perfectly [[fair coin]], <math display="inline">p_\text{H} = 0.5</math>.

Line 56:		Line 56:

	===Continuous probability distribution===		===Continuous probability distribution===
	Let <math display="inline">X</math> be a [[random variable]] following an [[Probability distribution#~~Continuous~~ probability distribution\|absolutely continuous probability distribution]] with [[probability density function\|density function]] <math display="inline">f</math> (a function of <math display="inline">x</math>) which depends on a parameter <math display="inline">\theta</math>. Then the function		Let <math display="inline">X</math> be a [[random variable]] following an [[Probability distribution#Absolutely continuous probability distribution\|absolutely continuous probability distribution]] with [[probability density function\|density function]] <math display="inline">f</math> (a function of <math display="inline">x</math>) which depends on a parameter <math display="inline">\theta</math>. Then the function

	<math display="block">\mathcal{L}(\theta \mid x) = f_\theta (x), </math>		<math display="block">\mathcal{L}(\theta \mid x) = f_\theta (x), </math>
Line 122:		Line 122:
	exist for all <math display="inline">\, r, s, t = 1, 2, \ldots, k \,</math> in order to ensure the existence of a [[Taylor expansion]]. Second, for almost all <math display="inline">x</math> and for every <math display="inline">\, \theta \in \Theta \,</math> it must be that		exist for all <math display="inline">\, r, s, t = 1, 2, \ldots, k \,</math> in order to ensure the existence of a [[Taylor expansion]]. Second, for almost all <math display="inline">x</math> and for every <math display="inline">\, \theta \in \Theta \,</math> it must be that
	<math display="block"> \left\| \frac{\partial f}{\partial \theta_r} \right\| < F_r(x) \,, \quad \left\| \frac{\partial^2 f}{\partial \theta_r \, \partial \theta_s} \right\| < F_{rs}(x) \,, \quad \left\| \frac{\partial^3 f}{\partial \theta_r \, \partial \theta_s \, \partial \theta_t} \right\| < H_{rst}(x) </math>		<math display="block"> \left\| \frac{\partial f}{\partial \theta_r} \right\| < F_r(x) \,, \quad \left\| \frac{\partial^2 f}{\partial \theta_r \, \partial \theta_s} \right\| < F_{rs}(x) \,, \quad \left\| \frac{\partial^3 f}{\partial \theta_r \, \partial \theta_s \, \partial \theta_t} \right\| < H_{rst}(x) </math>
	where <math display="inline">H</math> is such that <math display="inline">\, \int_{-\infty}^{\infty} H_{rst}(z) \~~mathrm{d}z~~ \leq M < \infty \;.</math> This boundedness of the derivatives is needed to allow for [[differentiation under the integral sign]]. And lastly, it is assumed that the [[information matrix]],		where <math display="inline">H</math> is such that <math display="inline">\, \int_{-\infty}^{\infty} H_{rst}(z) \, dz \leq M < \infty \;.</math> This boundedness of the derivatives is needed to allow for [[differentiation under the integral sign]]. And lastly, it is assumed that the [[information matrix]],
	<math display="block">\mathbf{I}(\theta) = \int_{-\infty}^{\infty} \frac{\partial \log f}{\partial \theta_r}\ \frac{\partial \log f}{\partial \theta_s}\ f\ ~~\mathrm{d}z~~ </math>		<math display="block">\mathbf{I}(\theta) = \int_{-\infty}^{\infty} \frac{\partial \log f}{\partial \theta_r}\ \frac{\partial \log f}{\partial \theta_s}\ f\, dz </math>
	is [[positive definite]] and <math display="inline">\, \left\| \mathbf{I}(\theta) \right\| \,</math> is finite. This ensures that the [[Score (statistics)\|score]] has a finite variance.<ref>{{cite book \|first1=Edward \|last1=Greenberg \|first2=Charles E. Jr. \|last2=Webster \|title=Advanced Econometrics: A Bridge to the Literature \|location=New York, NY \|publisher=John Wiley & Sons \|year=1983 \|isbn=0-471-09077-8 \|pages=~~24–25~~ }}</ref>		is [[positive definite]] and <math display="inline">\, \left\| \mathbf{I}(\theta) \right\| \,</math> is finite. This ensures that the [[Score (statistics)\|score]] has a finite variance.<ref>{{cite book \|first1=Edward \|last1=Greenberg \|first2=Charles E. Jr. \|last2=Webster \|title=Advanced Econometrics: A Bridge to the Literature \|url=https://archive.org/details/advancedeconomet00unse \|location=New York, NY \|publisher=John Wiley & Sons \|year=1983 \|isbn=0-471-09077-8 \|pages=[https://archive.org/details/advancedeconomet00unse/page/24 24]–25 }}</ref>

	The above conditions are sufficient, but not necessary. That is, a model that does not meet these regularity conditions may or may not have a maximum likelihood estimator of the properties mentioned above. Further, in case of non-independently or non-identically distributed observations additional properties may need to be assumed.		The above conditions are sufficient, but not necessary. That is, a model that does not meet these regularity conditions may or may not have a maximum likelihood estimator of the properties mentioned above. Further, in case of non-independently or non-identically distributed observations additional properties may need to be assumed.
Line 139:		Line 139:
	The likelihood ratio is central to [[likelihoodist statistics]]: the ''[[law of likelihood]]'' states that the degree to which data (considered as evidence) supports one parameter value versus another is measured by the likelihood ratio.		The likelihood ratio is central to [[likelihoodist statistics]]: the ''[[law of likelihood]]'' states that the degree to which data (considered as evidence) supports one parameter value versus another is measured by the likelihood ratio.

	In [[frequentist inference]], the likelihood ratio is the basis for a [[test statistic]], the so-called [[likelihood-ratio test]]. By the [[Neyman–Pearson lemma]], this is the most [[Statistical power\|powerful]] test for comparing two [[simple hypothesis\|simple hypotheses]] at a given [[significance level]]. Numerous other tests can be viewed as likelihood-ratio tests or approximations thereof.<ref>{{cite journal \|first=A. \|last=Buse \|title=The Likelihood Ratio, Wald, and Lagrange Multiplier Tests: An Expository Note \|journal=[[The American Statistician]] \|volume=36 \|issue=3a \|year=1982 \|pages=153–157 \|doi=10.1080/00031305.1982.10482817 }}</ref> The asymptotic distribution of the log-likelihood ratio, considered as a test statistic, is given by [[Wilks' theorem]].		In [[frequentist inference]], the likelihood ratio is the basis for a [[test statistic]], the so-called [[likelihood-ratio test]]. By the [[Neyman–Pearson lemma]], this is the most [[Statistical power\|powerful]] test for comparing two [[simple hypothesis\|simple hypotheses]] at a given [[significance level]]. Numerous other tests can be viewed as likelihood-ratio tests or approximations thereof.<ref>{{cite journal \|first=A. \|last=Buse \|title=The Likelihood Ratio, Wald, and Lagrange Multiplier Tests: An Expository Note \|url=https://archive.org/details/sim_american-statistician_1982-08_36_3/page/153 \|journal=[[The American Statistician]] \|volume=36 \|issue=3a \|year=1982 \|pages=153–157 \|doi=10.1080/00031305.1982.10482817 }}</ref> The asymptotic distribution of the log-likelihood ratio, considered as a test statistic, is given by [[Wilks' theorem]].

	The likelihood ratio is also of central importance in [[Bayesian inference]], where it is known as the [[Bayes factor]], and is used in [[Bayes' rule]]. Stated in terms of [[odds]], Bayes' rule states that the ''posterior'' odds of two alternatives, {{tmath\|A_1}} and {{tmath\|A_2}}, given an event {{tmath\|B}}, is the ''prior'' odds, times the likelihood ratio. As an equation:		The likelihood ratio is also of central importance in [[Bayesian inference]], where it is known as the [[Bayes factor]], and is used in [[Bayes' rule]]. Stated in terms of [[odds]], Bayes' rule states that the ''posterior'' odds of two alternatives, {{tmath\|A_1}} and {{tmath\|A_2}}, given an event {{tmath\|B}}, is the ''prior'' odds, times the likelihood ratio. As an equation:
Line 241:		Line 241:
	The equations defined by the stationary point of the score function serve as [[estimating equations]] for the maximum likelihood estimator.		The equations defined by the stationary point of the score function serve as [[estimating equations]] for the maximum likelihood estimator.
	<math display="block">s_{n}(\theta) = \mathbf{0}</math>		<math display="block">s_{n}(\theta) = \mathbf{0}</math>
	In that sense, the maximum likelihood estimator is implicitly defined by the value at <math display="inline">\mathbf{0}</math> of the [[inverse function]] <math display="inline">s_{n}^{-1}: \mathbb{E}^{d} \to \Theta</math>, where <math display="inline">\mathbb{E}^{d}</math> is the <var>d</var>-dimensional [[Euclidean space]], and <math display="inline">\Theta</math> is the parameter space. Using the [[inverse function theorem]], it can be shown that <math display="inline">s_{n}^{-1}</math> is [[well-defined]] in an [[open neighborhood]] about <math display="inline">\mathbf{0}</math> with probability going to one, and <math display="inline">\hat{\theta}_{n} = s_{n}^{-1}(\mathbf{0})</math> is a consistent estimate of <math display="inline">\theta</math>. As a consequence there exists a sequence <math display="inline">\left\{ \hat{\theta}_{n} \right\}</math> such that <math display="inline">s_{n}(\hat{\theta}_{n}) = \mathbf{0}</math> asymptotically [[almost surely]], and <math display="inline">\hat{\theta}_{n} \xrightarrow{\text{p}} \theta_{0}</math>.<ref>{{cite journal \|first=Robert V. \|last=Foutz \|title=On the Unique Consistent Solution to the Likelihood Equations \|journal=[[Journal of the American Statistical Association]] \|volume=72 \|year=1977 \|issue=357 \|pages=147–148 \|doi=10.1080/01621459.1977.10479926 }}</ref> A similar result can be established using [[Rolle's theorem]].<ref>{{cite journal \|first1=Robert E. \|last1=Tarone \|first2=Gary \|last2=Gruenhage \|title=A Note on the Uniqueness of Roots of the Likelihood Equations for Vector-Valued Parameters \|journal=Journal of the American Statistical Association \|volume=70 \|year=1975 \|issue=352 \|pages=903–904 \|doi=10.1080/01621459.1975.10480321 }}</ref><ref>{{cite journal \|first1=Kamta \|last1=Rai \|first2=John \|last2=Van Ryzin \|title=A Note on a Multivariate Version of Rolle's Theorem and Uniqueness of Maximum Likelihood Roots \|journal=Communications in Statistics \|series=Theory and Methods \|volume=11 \|year=1982 \|issue=13 \|pages=1505–1510 \|doi=10.1080/03610928208828325 }}</ref>		In that sense, the maximum likelihood estimator is implicitly defined by the value at <math display="inline">\mathbf{0}</math> of the [[inverse function]] <math display="inline">s_{n}^{-1}: \mathbb{E}^{d} \to \Theta</math>, where <math display="inline">\mathbb{E}^{d}</math> is the <var>d</var>-dimensional [[Euclidean space]], and <math display="inline">\Theta</math> is the parameter space. Using the [[inverse function theorem]], it can be shown that <math display="inline">s_{n}^{-1}</math> is [[well-defined]] in an [[open neighborhood]] about <math display="inline">\mathbf{0}</math> with probability going to one, and <math display="inline">\hat{\theta}_{n} = s_{n}^{-1}(\mathbf{0})</math> is a consistent estimate of <math display="inline">\theta</math>. As a consequence there exists a sequence <math display="inline">\left\{ \hat{\theta}_{n} \right\}</math> such that <math display="inline">s_{n}(\hat{\theta}_{n}) = \mathbf{0}</math> asymptotically [[almost surely]], and <math display="inline">\hat{\theta}_{n} \xrightarrow{\text{p}} \theta_{0}</math>.<ref>{{cite journal \|first=Robert V. \|last=Foutz \|title=On the Unique Consistent Solution to the Likelihood Equations \|url=https://archive.org/details/sim_journal-of-the-american-statistical-association_1977-03_72_357/page/147 \|journal=[[Journal of the American Statistical Association]] \|volume=72 \|year=1977 \|issue=357 \|pages=147–148 \|doi=10.1080/01621459.1977.10479926 }}</ref> A similar result can be established using [[Rolle's theorem]].<ref>{{cite journal \|first1=Robert E. \|last1=Tarone \|first2=Gary \|last2=Gruenhage \|title=A Note on the Uniqueness of Roots of the Likelihood Equations for Vector-Valued Parameters \|journal=Journal of the American Statistical Association \|volume=70 \|year=1975 \|issue=352 \|pages=903–904 \|doi=10.1080/01621459.1975.10480321 }}</ref><ref>{{cite journal \|first1=Kamta \|last1=Rai \|first2=John \|last2=Van Ryzin \|title=A Note on a Multivariate Version of Rolle's Theorem and Uniqueness of Maximum Likelihood Roots \|journal=Communications in Statistics \|series=Theory and Methods \|volume=11 \|year=1982 \|issue=13 \|pages=1505–1510 \|doi=10.1080/03610928208828325 }}</ref>

	The second derivative evaluated at <math display="inline">\hat{\theta}</math>, known as [[Fisher information]], determines the curvature of the likelihood surface,<ref>{{citation \|first=B. Raja \|last=Rao \|title=A formula for the curvature of the likelihood surface of a sample drawn from a distribution admitting sufficient statistics \|journal=[[Biometrika]] \|volume=47 \|issue=1–2 \|year=1960 \|pages=203–207 \|doi=10.1093/biomet/47.1-2.203 \|mode=cs1 }}</ref> and thus indicates the [[Precision (statistics)\|precision]] of the estimate.<ref>{{citation \|first1=Michael D. \|last1=Ward \|first2=John S. \|last2=Ahlquist \|title=Maximum Likelihood for Social Science : Strategies for Analysis \|publisher= [[Cambridge University Press]] \|year=2018 \|pages=25–27 \|mode=cs1 }}</ref>		The second derivative evaluated at <math display="inline">\hat{\theta}</math>, known as [[Fisher information]], determines the curvature of the likelihood surface,<ref>{{citation \|first=B. Raja \|last=Rao \|title=A formula for the curvature of the likelihood surface of a sample drawn from a distribution admitting sufficient statistics \|journal=[[Biometrika]] \|volume=47 \|issue=1–2 \|year=1960 \|pages=203–207 \|doi=10.1093/biomet/47.1-2.203 \|mode=cs1 }}</ref> and thus indicates the [[Precision (statistics)\|precision]] of the estimate.<ref>{{citation \|first1=Michael D. \|last1=Ward \|first2=John S. \|last2=Ahlquist \|title=Maximum Likelihood for Social Science : Strategies for Analysis \|publisher= [[Cambridge University Press]] \|year=2018 \|pages=25–27 \|mode=cs1 }}</ref>

← Previous revision		Revision as of 04:20, 26 December 2025
Line 19:		Line 19:
	<math display="block">\mathcal{L}(\theta \mid x). </math>		<math display="block">\mathcal{L}(\theta \mid x). </math>

	In other words, when <math display="inline">f(x\mid\theta)</math> is viewed as a function of <math display="inline">x</math> with <math display="inline">\theta</math> fixed, it is a probability density function, and when viewed as a function of <math display="inline">\theta</math> with <math display="inline">x</math> fixed, it is a likelihood function. In the [[Frequentist_probability\|frequentist paradigm]], the notation <math display="inline">f(x\mid\theta)</math> is often avoided and instead <math display="inline">f(x;\theta)</math> or <math display="inline">f(x,\theta)</math> are used to indicate that <math display="inline">\theta</math> is regarded as a fixed unknown quantity rather than as a [[random variable]] being conditioned on.		In other words, when <math display="inline">f(x\mid\theta)</math> is viewed as a function of <math display="inline">x</math> with <math display="inline">\theta</math> fixed, it is a probability density function, and when viewed as a function of <math display="inline">\theta</math> with <math display="inline">x</math> fixed, it is a likelihood function. In the [[Frequentist_probability\|frequentist paradigm]], the notation <math display="inline">f(x\mid\theta)</math> is often avoided and instead <math display="inline">f(x;\theta)</math> or <math display="inline">f(x,\theta)</math> are used to indicate that <math display="inline">\theta</math> is regarded as a fixed unknown quantity rather than as a [[random variable]] being conditioned on.

	The likelihood function does ''not'' specify the probability that <math display="inline">\theta</math> is the truth, given the observed sample <math display="inline">X = x</math>. Such an interpretation is a common error, with potentially disastrous consequences (see [[prosecutor's fallacy]]).		The likelihood function does ''not'' specify the probability that <math display="inline">\theta</math> is the truth, given the observed sample <math display="inline">X = x</math>. Such an interpretation is a common error, with potentially disastrous consequences (see [[prosecutor's fallacy]]).
Line 26:		Line 26:
	Let <math display="inline">X</math> be a discrete [[random variable]] with [[probability mass function]] <math display="inline">p</math> depending on a parameter <math display="inline">\theta</math>. Then the function		Let <math display="inline">X</math> be a discrete [[random variable]] with [[probability mass function]] <math display="inline">p</math> depending on a parameter <math display="inline">\theta</math>. Then the function

	<math display="block">\mathcal{L}(\theta \mid x) = p_\theta (x) = P_\theta (X=x), </math>		<math display="block">\mathcal{L}(\theta \mid x) = p_\theta (x) = P_\theta (X=x) = \text{Pr}\{ X=x \mid \Theta=\theta \} , </math>

	considered as a function of <math display="inline">\theta</math>, is the ''likelihood function'', given the [[Outcome (probability)\|outcome]] <math display="inline">x</math> of the random variable <math display="inline">X</math>. Sometimes the probability of "the value <math display="inline">x</math> of <math display="inline">X</math> for the parameter value <math display="inline">\theta</math>" is written as {{math\|''P''(''X'' {{=}} ''x'' {{!}} ''θ'')}} or {{math\|''P''(''X'' {{=}} ''x''; ''θ'')}}. The likelihood is the probability that a particular outcome <math display="inline">x</math> is observed when the true value of the parameter is <math display="inline">\theta</math>, equivalent to the probability mass on <math display="inline">x</math>; it is ''not'' a probability density over the parameter <math display="inline">\theta</math>. The likelihood, <math display="inline">\mathcal{L}(\theta \mid x) </math>, should not be confused with <math display="inline">P(\theta \mid x)</math>, which is the posterior probability of <math display="inline">\theta</math> given the data <math display="inline">x</math>.		considered as a function of <math display="inline">\theta</math>, a possible value of the deterministic but unknown parameter <math display="inline">\Theta</math>, is the ''likelihood function'', given the [[Outcome (probability)\|outcome]] <math display="inline">x</math> of the random variable <math display="inline">X</math>. Sometimes the probability of "the value <math display="inline">x</math> of <math display="inline">X</math> for the parameter value <math display="inline">\theta</math>" is written as {{math\|''P''(''X'' {{=}} ''x'' {{!}} ''θ'')}} or {{math\|''P''(''X'' {{=}} ''x''; ''θ'')}}. The likelihood is the probability that a particular outcome <math display="inline">x</math> is observed when the true value of the parameter is <math display="inline">\theta</math>, equivalent to the probability mass on <math display="inline">x</math>; it is ''not'' a probability density over the parameter <math display="inline">\theta</math>. The likelihood, <math display="inline">\mathcal{L}(\theta \mid x) </math>, should not be confused with <math display="inline">P(\theta \mid x)</math>, which is the posterior probability of <math display="inline">\theta</math> given the data <math display="inline">x</math>.

	====Example====		====Example====
Line 43:		Line 43:
	<math display="block">\mathcal{L}(p_\text{H}=0.5 \mid \text{HH}) = 0.25.</math>		<math display="block">\mathcal{L}(p_\text{H}=0.5 \mid \text{HH}) = 0.25.</math>

	This is not the same as saying that <math display="inline">P(p_\text{H} = 0.5 \mid \text{HH}) = 0.25</math>, a conclusion which could only be reached via [[Bayes' theorem]] given knowledge about the marginal probabilities <math display="inline">P(p_\text{H} = 0.5)</math> and <math display="inline">P(\text{HH})</math>.		This is not the same as saying that <math display="inline">P(p_\text{H} = 0.5 \mid \text{HH}) = 0.25</math>, a conclusion which could only be reached via [[Bayes' theorem]] given knowledge about the marginal probabilities <math display="inline">P(p_\text{H} = 0.5)</math> and <math display="inline">P(\text{HH})</math>.

	Now suppose that the coin is not a fair coin, but instead that <math display="inline">p_\text{H} = 0.3</math>. Then the probability of two heads on two flips is		Now suppose that the coin is not a fair coin, but instead that <math display="inline">p_\text{H} = 0.3</math>. Then the probability of two heads on two flips is
Line 150:		Line 150:
	===Relative likelihood function===		===Relative likelihood function===
	{{See also\|Relative likelihood}}		{{See also\|Relative likelihood}}
	Since the actual value of the likelihood function depends on the sample, it is often convenient to work with a standardized measure. Suppose that the [[maximum likelihood estimate]] for the parameter {{mvar\|θ}} is <math display="inline">\hat{\theta}</math>. Relative plausibilities of other {{mvar\|θ}} values may be found by comparing the likelihoods of those other values with the likelihood of <math display="inline">\hat{\theta}</math>. The ''relative likelihood'' of {{mvar\|θ}} is defined to be<ref name='Kalbfleisch'>{{citation \| author-link= James G. Kalbfleisch \| last= Kalbfleisch \| first= J. G. \| year=1985 \| title= Probability and Statistical Inference \| publisher= Springer}} (§9.3).</ref><ref>{{citation\| last= Azzalini \| first= A. \| title= Statistical Inference—Based on the likelihood \| year= 1996 \| publisher= [[Chapman & Hall]] \| url= https://books.google.com/books?id=hyN6gXHvSo0C \| isbn= 9780412606502 }} (§1.4.2).</ref><ref name='Sprott'>Sprott, D. A. (2000), ''Statistical Inference in Science'', Springer (chap. 2).</ref><ref>Davison, A. C. (2008), ''Statistical Models'', [[Cambridge University Press]] (§4.1.2).</ref><ref>{{citation\|first1= L. \| last1= Held \| first2= D. S. \| last2= Sabanés Bové \| title= Applied Statistical Inference—Likelihood and Bayes \| year= 2014 \| publisher= Springer}} (§2.1).</ref>		Since the actual value of the likelihood function depends on the sample, it is often convenient to work with a standardized measure. Suppose that the [[maximum likelihood estimate]] for the parameter {{mvar\|θ}} is <math display="inline">\hat{\theta}</math>. Relative plausibilities of other {{mvar\|θ}} values may be found by comparing the likelihoods of those other values with the likelihood of <math display="inline">\hat{\theta}</math>. The ''relative likelihood'' of {{mvar\|θ}} is defined to be<ref name='Kalbfleisch'>{{citation \| author-link= James G. Kalbfleisch \| last= Kalbfleisch \| first= J. G. \| year=1985 \| title= Probability and Statistical Inference \| publisher= Springer}} (§9.3).</ref><ref>{{citation\| last= Azzalini \| first= A. \| title= Statistical Inference—Based on the likelihood \| year= 1996 \| publisher= [[Chapman & Hall]] \| url= https://books.google.com/books?id=hyN6gXHvSo0C \| isbn= 9780412606502 }} (§1.4.2).</ref><ref name='Sprott'>Sprott, D. A. (2000), ''Statistical Inference in Science'', Springer (chap. 2).</ref><ref>Davison, A. C. (2008), ''Statistical Models'', [[Cambridge University Press]] (§4.1.2).</ref><ref>{{citation\|first1= L. \| last1= Held \| first2= D. S. \| last2= Sabanés Bové \| title= Applied Statistical Inference—Likelihood and Bayes \| year= 2014 \| publisher= Springer}} (§2.1).</ref>
	<math display="block">R(\theta) = \frac{\mathcal{L}(\theta \mid x)}{\mathcal{L}(\hat{\theta} \mid x)}.</math>		<math display="block">R(\theta) = \frac{\mathcal{L}(\theta \mid x)}{\mathcal{L}(\hat{\theta} \mid x)}.</math>
	Thus, the relative likelihood is the likelihood ratio (discussed above) with the fixed denominator <math display="inline"> \mathcal{L}(\hat{\theta})</math>. This corresponds to standardizing the likelihood to have a maximum of 1.		Thus, the relative likelihood is the likelihood ratio (discussed above) with the fixed denominator <math display="inline"> \mathcal{L}(\hat{\theta})</math>. This corresponds to standardizing the likelihood to have a maximum of 1.
Line 337:		Line 337:

	====Bayesian interpretation====		====Bayesian interpretation====
	In [[Bayesian inference]], although one can speak about the likelihood of any proposition or [[random variable]] given another random variable: for example the likelihood of a parameter value or of a [[statistical model]] (see [[marginal likelihood]]), given specified data or other evidence,<ref name='good1950'>I. J. Good: ''Probability and the Weighing of Evidence'' (Griffin 1950), §6.1</ref><ref name='jeffreys1983'>H. Jeffreys: ''Theory of Probability'' (3rd ed., Oxford University Press 1983), §1.22</ref><ref name='jaynes2003'>E. T. Jaynes: ''Probability Theory: The Logic of Science'' (Cambridge University Press 2003), §4.1</ref><ref name='lindley1980'>D. V. Lindley: ''Introduction to Probability and Statistics from a Bayesian Viewpoint. Part 1: Probability'' (Cambridge University Press 1980), §1.6</ref> the likelihood function remains the same entity, with the additional interpretations of (i) a [[Conditional probability distribution\|conditional density]] of the data given the parameter (since the parameter is then a random variable) and (ii) a measure or amount of information brought by the data about the parameter value or even the model.<ref name='good1950'/><ref name='jeffreys1983'/><ref name='jaynes2003'/><ref name='lindley1980'/><ref name='gelmanetal2014'>A. Gelman, J. B. Carlin, H. S. Stern, D. B. Dunson, A. Vehtari, D. B. Rubin: ''Bayesian Data Analysis'' (3rd ed., Chapman & Hall/CRC 2014), §1.3</ref> Due to the introduction of a probability structure on the parameter space or on the collection of models, it is possible that a parameter value or a statistical model have a large likelihood value for given data, and yet have a low ''probability'', or vice versa.<ref name='jaynes2003'/><ref name='gelmanetal2014'/> This is often the case in medical contexts.<ref>{{citation \|first1=H. C. \|last1=Sox \|first2=M. C. \|last2=Higgins \|first3=D. K. \|last3=Owens \|title=Medical Decision Making \|edition=2nd \|publisher=Wiley \|year=2013 \|doi=10.1002/9781118341544 \|isbn=9781118341544 \|at=chapters 3–4 }}</ref> Following [[Bayes' ~~Rule~~]], the likelihood when seen as a conditional density can be multiplied by the [[prior probability]] density of the parameter and then normalized, to give a [[posterior probability]] density.<ref name='good1950'/><ref name='jeffreys1983'/><ref name='jaynes2003'/><ref name='lindley1980'/><ref name="gelmanetal2014"/> More generally, the likelihood of an unknown quantity <math display="inline">X</math> given another unknown quantity <math display="inline">Y</math> is proportional to the ''probability of <math display="inline">Y</math> given <math display="inline">X</math>''.<ref name='good1950'/><ref name='jeffreys1983'/><ref name='jaynes2003'/><ref name='lindley1980'/><ref name='gelmanetal2014'/>		In [[Bayesian inference]], although one can speak about the likelihood of any proposition or [[random variable]] given another random variable: for example the likelihood of a parameter value or of a [[statistical model]] (see [[marginal likelihood]]), given specified data or other evidence,<ref name='good1950'>I. J. Good: ''Probability and the Weighing of Evidence'' (Griffin 1950), §6.1</ref><ref name='jeffreys1983'>H. Jeffreys: ''Theory of Probability'' (3rd ed., Oxford University Press 1983), §1.22</ref><ref name='jaynes2003'>E. T. Jaynes: ''Probability Theory: The Logic of Science'' (Cambridge University Press 2003), §4.1</ref><ref name='lindley1980'>D. V. Lindley: ''Introduction to Probability and Statistics from a Bayesian Viewpoint. Part 1: Probability'' (Cambridge University Press 1980), §1.6</ref> the likelihood function remains the same entity, with the additional interpretations of (i) a [[Conditional probability distribution\|conditional density]] of the data given the parameter (since the parameter is then a random variable) and (ii) a measure or amount of information brought by the data about the parameter value or even the model.<ref name='good1950'/><ref name='jeffreys1983'/><ref name='jaynes2003'/><ref name='lindley1980'/><ref name='gelmanetal2014'>A. Gelman, J. B. Carlin, H. S. Stern, D. B. Dunson, A. Vehtari, D. B. Rubin: ''Bayesian Data Analysis'' (3rd ed., Chapman & Hall/CRC 2014), §1.3</ref> Due to the introduction of a probability structure on the parameter space or on the collection of models, it is possible that a parameter value or a statistical model have a large likelihood value for given data, and yet have a low ''probability'', or vice versa.<ref name='jaynes2003'/><ref name='gelmanetal2014'/> This is often the case in medical contexts.<ref>{{citation \|first1=H. C. \|last1=Sox \|first2=M. C. \|last2=Higgins \|first3=D. K. \|last3=Owens \|title=Medical Decision Making \|edition=2nd \|publisher=Wiley \|year=2013 \|doi=10.1002/9781118341544 \|isbn=9781118341544 \|at=chapters 3–4 }}</ref> Following [[Bayes' rule]], the likelihood when seen as a conditional density can be multiplied by the [[prior probability]] density of the parameter and then normalized, to give a [[posterior probability]] density.<ref name='good1950'/><ref name='jeffreys1983'/><ref name='jaynes2003'/><ref name='lindley1980'/><ref name="gelmanetal2014"/> More generally, the likelihood of an unknown quantity <math display="inline">X</math> given another unknown quantity <math display="inline">Y</math> is proportional to the ''probability of <math display="inline">Y</math> given <math display="inline">X</math>''.<ref name='good1950'/><ref name='jeffreys1983'/><ref name='jaynes2003'/><ref name='lindley1980'/><ref name='gelmanetal2014'/>

	====Likelihoodist interpretation====		====Likelihoodist interpretation====
	{{more footnotes needed\|date=April 2019}}		{{more footnotes needed\|section\|date=April 2019}}
	In frequentist statistics, the likelihood function is itself a [[statistic]] that summarizes a single sample from a population, whose calculated value depends on a choice of several parameters ''θ''<sub>1</sub> ... ''θ''<sub>p</sub>, where ''p'' is the count of parameters in some already-selected [[statistical model]]. The value of the likelihood serves as a figure of merit for the choice used for the parameters, and the parameter set with maximum likelihood is the best choice, given the data available.		In frequentist statistics, the likelihood function is itself a [[statistic]] that summarizes a single sample from a population, whose calculated value depends on a choice of several parameters ''θ''<sub>1</sub> ... ''θ''<sub>p</sub>, where ''p'' is the count of parameters in some already-selected [[statistical model]]. The value of the likelihood serves as a figure of merit for the choice used for the parameters, and the parameter set with maximum likelihood is the best choice, given the data available.