Logistic regression

In <a href="/facts/Statistics/DNPsKGYU">statistics</a>, a logistic model (or logit model) is a <a href="/facts/Statistical_model/jKkT5ftm">statistical model</a> that models the <a href="/facts/Logit/5674C2DL">log-odds</a> of an event as a <a href="/facts/Linear_function_(calculus)/sbyEbb6x">linear combination</a> of one or more <a href="/facts/Independent_variable/dINfzIF2">independent variables</a>. In <a href="/facts/Regression_analysis/n6z5Tf7K">regression analysis</a>, logistic regression (or logit regression) <a href="/facts/Estimation_theory/4nbgFDec">estimates</a> the parameters of a logistic model (the coefficients in the linear or non linear combinations). In binary logistic regression there is a single <a href="/facts/Binary_variable/PSFa5S2Q">binary</a> <a href="/facts/Dependent_variable/dINfzIF2">dependent variable</a>, coded by an <a href="/facts/Indicator_variable/LbT5Wcjm">indicator variable</a>, where the two values are labeled "0" and "1", while the <a href="/facts/Independent_variable/dINfzIF2">independent variables</a> can each be a binary variable (two classes, coded by an indicator variable) or a <a href="/facts/Continuous_variable/TWl5feEw">continuous variable</a> (any real value). The corresponding probability of the value labeled "1" can vary between 0 (certainly the value "0") and 1 (certainly the value "1"), hence the labeling; the function that converts log-odds to probability is the <a href="/facts/Logistic_function/IaQ254t7">logistic function</a>, hence the name. The <a href="/facts/Unit_of_measurement/pvTpWbmC">unit of measurement</a> for the log-odds scale is called a <a href="/facts/Logit/5674C2DL">logit</a>, from logistic unit, hence the alternative names. See § Background and § Definition for formal mathematics, and § Example for a worked example.
Binary variables are widely used in statistics to model the probability of a certain class or event taking place, such as the probability of a team winning, of a patient being healthy, etc. (see § Applications), and the logistic model has been the most commonly used model for <a href="/facts/Binary_regression/YERKnRcr">binary regression</a> since about 1970. Binary variables can be generalized to <a href="/facts/Categorical_variable/3UQNkLqs">categorical variables</a> when there are more than two possible values (e.g. whether an image is of a cat, dog, lion, etc.), and the binary logistic regression generalized to <a href="/facts/Multinomial_logistic_regression/aeF5lqDE">multinomial logistic regression</a>. If the multiple categories are <a href="/facts/Level_of_measurement/8GAxPxFv">ordered</a>, one can use the <a href="/facts/Ordinal_logistic_regression/tVm2X3Qd">ordinal logistic regression</a> (for example the proportional odds ordinal logistic model). See § Extensions for further extensions. The logistic regression model itself simply models probability of output in terms of input and does not perform <a href="/facts/Statistical_classification/jXXHRkXR">statistical classification</a> (it is not a classifier), though it can be used to make a classifier, for instance by choosing a cutoff value and classifying inputs with probability greater than the cutoff as one class, below the cutoff as the other; this is a common way to make a <a href="/facts/Binary_classifier/MeYYAMwp">binary classifier</a>.
Analogous linear models for binary variables with a different <a href="/facts/Sigmoid_function/lAXu3AwW">sigmoid function</a> instead of the logistic function (to convert the linear combination to a probability) can also be used, most notably the <a href="/facts/Probit_model/k7hCvtYj">probit model</a>; see § Alternatives. The defining characteristic of the logistic model is that increasing one of the independent variables multiplicatively scales the odds of the given outcome at a constant rate, with each independent variable having its own parameter; for a binary dependent variable this generalizes the <a href="/facts/Odds_ratio/7WcmubfN">odds ratio</a>. More abstractly, the logistic function is the <a href="/facts/Natural_parameter/1LkkqEIf">natural parameter</a> for the <a href="/facts/Bernoulli_distribution/ChCtYyvs">Bernoulli distribution</a>, and in this sense is the "simplest" way to convert a real number to a probability. In particular, it maximizes entropy (minimizes added information), and in this sense makes the fewest assumptions of the data being modeled; see § Maximum entropy.
The parameters of a logistic regression are most commonly estimated by <a href="/facts/Maximum-likelihood_estimation/0Yq2dpQD">maximum-likelihood estimation</a> (MLE). This does not have a closed-form expression, unlike <a href="/facts/Linear_least_squares_(mathematics)/dKctFUCV">linear least squares</a>; see § Model fitting. Logistic regression by MLE plays a similarly basic role for binary or categorical responses as linear regression by <a href="/facts/Ordinary_least_squares/q7H9k5vM">ordinary least squares</a> (OLS) plays for <a href="/facts/Scalar_(mathematics)/MkDbA4Eo">scalar</a> responses: it is a simple, well-analyzed baseline model; see § Comparison with linear regression for discussion. The logistic regression as a general statistical model was originally developed and popularized primarily by <a href="/facts/Joseph_Berkson/rFEq4JYE">Joseph Berkson</a>, beginning in Berkson (1944), where he coined "logit"; see § History.

Logistic regression open-in-new

Logistic regression