Maximum a posteriori

In statistics, the method of maximum a posteriori (MAP, or posterior mode) estimation can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data. It is closely related to Fisher's method of maximum likelihood (ML), but employs an augmented optimization objective which incorporates a prior distribution over the quantity one wants to estimate. MAP estimation can therefore be seen as a regularization of ML estimation.

Introduction

Assume that we want to estimate an unobserved population parameter heta on the basis of observations x. Let f be the sampling distribution of x, so that f(x| heta) is the probability of x when the underlying population parameter is heta. Then the function

: heta mapsto f(x | heta) !

is known as the likelihood function and the estimate

:hat{ heta}_{mathrm{ML(x) = argmax_{ heta} f(x | heta) !

as the maximum likelihood estimate of heta.

Now assume that a prior distribution g over heta exists. This allows us to treat heta as a random variable as in Bayesian statistics. Then the posterior distribution of heta is as follows:

: heta mapsto frac{f(x | heta) , g( heta)}{displaystyleint_{Theta} f(x | heta') , g( heta') , d heta'} !

where g is density function of heta, Theta is the domain of g. This is a straightforward application of Bayes' theorem.

The method of maximum a posteriori estimation then estimates heta as the mode of the posterior distribution of this random variable:

:hat{ heta}_{mathrm{MAP(x)= argmax_{ heta} frac{f(x | heta) , g( heta)} {displaystyleint_{Theta} f(x | heta') , g( heta') , d heta'}= argmax_{ heta} f(x | heta) , g( heta)!

The denominator of the posterior distribution does not depend on heta and therefore plays no role in the optimization. Observe that the MAP estimate of heta coincides with the ML estimate when the prior g is uniform (that is, a constant function). The MAP estimate is the Bayes estimator under the uniform loss function.

MAP estimates can be computed in several ways:
# Analytically, when the mode(s) of the posterior distribution can be given in closed form. This is the case when conjugate priors are used.
# Via numerical optimization such as the conjugate gradient method or Newton's method. This usually requires first or second derivatives, which have to be evaluated analytically or numerically.
# Via a modification of an expectation-maximization algorithm. This does not require derivatives of the posterior density.

While MAP estimation "is" a Bayes estimator (under the 0-1 loss function), it is not very representative of Bayesian methods in general. This is because MAP estimates are point estimates, whereas Bayesian methods are characterized by the use of distributions to summarize data and draw inferences: thus, Bayesian methods tend to report the posterior mean or median instead, together with posterior intervals. This is both because these estimators are optimal under squared-error and linear-error loss respectively - which are more representative of typical loss functions - and because the posterior distribution may not have a simple analytic form: in this case, the distribution can be simulated using Markov chain Monte Carlo techniques, while optimization to find its mode(s) may be difficult or impossible.

Example

Suppose that we are given a sequence (x_1, dots, x_n) of IID N(mu,sigma_v^2 ) random variables and an a priori distribution of mu is given by N(0,sigma_m^2 ). We wish to find the MAP estimate of mu.

The function to be maximized is then given by

:pi(mu) L(mu) = frac{1}{sqrt{2 pi} sigma_m} expleft(-frac{1}{2} left(frac{mu}{sigma_m} ight)^2 ight) prod_{j=1}^n frac{1}{sqrt{2 pi} sigma_v} expleft(-frac{1}{2} left(frac{x_j - mu}{sigma_v} ight)^2 ight),

which is equivalent to minimizing mu in the following

: sum_{j=1}^n left(frac{x_j - mu}{sigma_v} ight)^2 + left(frac{mu}{sigma_m} ight)^2.

Thus, we see that the MAP estimator for μ is given by

:hat{mu}_{MAP} = frac{sigma_m^2}{n sigma_m^2 + sigma_v^2 } sum_{j=1}^n x_j.

The case of sigma_m o infty is called a non-informative prior and leads to an ill-defined a priori probability distribution; in this case hat{mu}_{MAP} o hat{mu}_{ML}.

See also

* Maximum likelihood estimation, when no prior distribution is available.

References

* M. DeGroot, "Optimal Statistical Decisions", McGraw-Hill, (1970).
* Harold W. Sorenson, (1980) "Parameter Estimation: Principles and Problems", Marcel Dekker.


Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Maximum a posteriori — Die Maximum a posteriori Methode (= MAP) bezeichnet in der Statistik ein Schätzverfahren, das einen vorgegebenen Parameter durch den Modalwert der A posteriori Verteilung schätzt. Somit besteht eine gewisse Ähnlichkeit zur Maximum Likelihood… …   Deutsch Wikipedia

  • Maximum a posteriori — L estimateur du maximum a posteriori ou à postériori (MAP), tout comme la méthode du maximum de vraisemblance, est une méthode pouvant être utilisée afin d estimer un certain nombre de paramètres inconnus, comme par exemple les parametres d une… …   Wikipédia en Français

  • Maximum-A-Posteriori-Schätzung — Die Maximum A Posteriori Methode bezeichnet in der Statistik ein Schätzverfahren. Die Maximum A Posteriori Schätzung bestimmt ähnlich der Maximum Likelihood Methode den oder die plausibelsten Parameter einer Verteilung bei einer vorliegenden… …   Deutsch Wikipedia

  • Maximum a posteriori estimation — In Bayesian statistics, a maximum a posteriori probability (MAP) estimate is a mode of the posterior distribution. The MAP can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data. It is closely related to… …   Wikipedia

  • Maximum likelihood sequence estimation — (MLSE) is a mathematical algorithm to extract useful data out of a noisy data stream. Contents 1 Theory 2 Background 3 References 4 Further reading …   Wikipedia

  • Maximum De Vraisemblance — L estimation du maximum de vraisemblance est une méthode statistique courante utilisée pour inférer les paramètres de la distribution de probabilité d un échantillon donné. Cette méthode a été développée par le statisticien et généticien Ronald… …   Wikipédia en Français

  • Maximum Likelihood Sequence Estimation — (MLSE) is a mathematical algorithm to extract useful data out of a noisy data stream.ee also* Maximum likelihood * Maximum a posteriori (MAP) Estimation, which is more complex and requires a known distributionReferences*External links* http://www …   Wikipedia

  • Maximum-Likelihood — Die Maximum Likelihood Methode (von engl. maximale Wahrscheinlichkeit) bezeichnet in der Statistik ein parametrisches Schätzverfahren. Inhaltsverzeichnis 1 Motivation 2 Eine heuristische Herleitung 3 Definition 4 Beispiel …   Deutsch Wikipedia

  • Maximum-Likelihood-Algorithmus — Die Maximum Likelihood Methode (von engl. maximale Wahrscheinlichkeit) bezeichnet in der Statistik ein parametrisches Schätzverfahren. Inhaltsverzeichnis 1 Motivation 2 Eine heuristische Herleitung 3 Definition 4 Beispiel …   Deutsch Wikipedia

  • Maximum-Likelihood-Schätzer — Die Maximum Likelihood Methode (von engl. maximale Wahrscheinlichkeit) bezeichnet in der Statistik ein parametrisches Schätzverfahren. Inhaltsverzeichnis 1 Motivation 2 Eine heuristische Herleitung 3 Definition 4 Beispiel …   Deutsch Wikipedia


Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”

We are using cookies for the best presentation of our site. Continuing to use this site, you agree with this.