Fisher information

In <a href="/facts/Mathematical_statistics/DeCWSzsy">mathematical statistics</a>, the Fisher information is a way of measuring the amount of <a href="/facts/Information/3LdrzXtJ">information</a> that an observable <a href="/facts/Random_variable/TwTBXnLT">random variable</a> X carries about an unknown parameter θ of a distribution that models X. Formally, it is the <a href="/facts/Variance/ULBJKXD1">variance</a> of the <a href="/facts/Score_(statistics)/gnBI7IEy">score</a>, or the <a href="/facts/Expected_value/1XV0JKL8">expected value</a> of the <a href="/facts/Observed_information/S2gDCn8V">observed information</a>. 
The role of the Fisher information in the asymptotic theory of <a href="/facts/Maximum-likelihood_estimation/0Yq2dpQD">maximum-likelihood estimation</a> was emphasized and explored by the statistician <a href="/facts/Ronald_Fisher/I2FVIpy9">Sir Ronald Fisher</a> (following some initial results by <a href="/facts/Francis_Ysidro_Edgeworth/ETLYT91v">Francis Ysidro Edgeworth</a>). The Fisher information matrix is used to calculate the <a href="/facts/Covariance_matrices/kLhgjHC6">covariance matrices</a> associated with <a href="/facts/Maximum_likelihood/0Yq2dpQD">maximum-likelihood</a> <a href="/facts/Estimator/CbkjcKvN">estimates</a>. It can also be used in the formulation of test statistics, such as the <a href="/facts/Wald_test/bVs5k173">Wald test</a>.
In <a href="/facts/Bayesian_statistics/9w7b1Bw4">Bayesian statistics</a>, the Fisher information plays a role in the derivation of non-informative <a href="/facts/Prior_distribution/JQKAD4o0">prior distributions</a> according to <a href="/facts/Jeffreys_prior/6j4Wpu7t">Jeffreys' rule</a>. It also appears as the large-sample covariance of the <a href="/facts/Posterior_distribution/fojsMD0D">posterior distribution</a>, provided that the prior is sufficiently smooth (a result known as <a href="/facts/Bernstein%25E2%2580%2593von_Mises_theorem/X6U10a8t">Bernstein–von Mises theorem</a>, which was anticipated by <a href="/facts/Laplace/6ktp0txb">Laplace</a> for <a href="/facts/Exponential_families/1LkkqEIf">exponential families</a>). The same result is used when approximating the posterior with <a href="/facts/Laplace%2527s_approximation/wyCrWmpn">Laplace's approximation</a>, where the Fisher information appears as the covariance of the fitted Gaussian.
Statistical systems of a scientific nature (physical, biological, etc.) whose likelihood functions obey <a href="/facts/Shift-invariant_system/zaKY72py">shift invariance</a> have been shown to obey maximum Fisher information. The level of the maximum depends upon the nature of the system constraints.

Fisher information open-in-new

Fisher information