Errors-in-variables model

<h2 id="motivating-example">Motivating example</h2>
Consider a simple linear regression model of the form

y
          
            t
          
        
        =
        α
        +
        β
        
          x
          
            t
          
          
            ∗
          
        
        +
        
          ε
          
            t
          
        
        
        ,
        
        t
        =
        1
        ,
        …
        ,
        T
        ,
      
    
    {\displaystyle y_{t}=\alpha +\beta x_{t}^{*}+\varepsilon _{t}\,,\quad t=1,\ldots ,T,}

where 
 
 
 
 
 x
 
 t
 
 
 ∗
 
 
 
 
 {\displaystyle x_{t}^{*}}
 
 denotes the true but <a href="/facts/Latent_variable/ohg99BnW">unobserved regressor</a>. Instead, we observe this value with an error:

x
          
            t
          
        
        =
        
          x
          
            t
          
          
            ∗
          
        
        +
        
          η
          
            t
          
        
        
      
    
    {\displaystyle x_{t}=x_{t}^{*}+\eta _{t}\,}

where the measurement error 
 
 
 
 
 η
 
 t
 
 
 
 
 {\displaystyle \eta _{t}}
 
 is assumed to be independent of the true value 
 
 
 
 
 x
 
 t
 
 
 ∗
 
 
 
 
 {\displaystyle x_{t}^{*}}
 
.
A practical application is the standard school science experiment for <a href="/facts/Hooke%2527s_law/7hTUcKUm">Hooke's law</a>, in which one estimates the relationship between the weight added to a spring and the amount by which the spring stretches. 
If the 
 
 
 
 
 y
 
 t
 
 
 
 
 {\displaystyle y_{t}}
 
′s are simply regressed on the 
 
 
 
 
 x
 
 t
 
 
 
 
 {\displaystyle x_{t}}
 
′s (see <a href="/facts/Simple_linear_regression/SNINCEFU">simple linear regression</a>), then the estimator for the slope coefficient is

β
                ^
              
            
          
          
            x
          
        
        =
        
          
            
              
                
                  
                    1
                    T
                  
                
              
              
                ∑
                
                  t
                  =
                  1
                
                
                  T
                
              
              (
              
                x
                
                  t
                
              
              −
              
                
                  
                    x
                    ¯
                  
                
              
              )
              (
              
                y
                
                  t
                
              
              −
              
                
                  
                    y
                    ¯
                  
                
              
              )
            
            
              
                
                  
                    1
                    T
                  
                
              
              
                ∑
                
                  t
                  =
                  1
                
                
                  T
                
              
              (
              
                x
                
                  t
                
              
              −
              
                
                  
                    x
                    ¯
                  
                
              
              
                )
                
                  2
                
              
            
          
        
        
        ,
      
    
    {\displaystyle {\hat {\beta }}_{x}={\frac {{\tfrac {1}{T}}\sum _{t=1}^{T}(x_{t}-{\bar {x}})(y_{t}-{\bar {y}})}{{\tfrac {1}{T}}\sum _{t=1}^{T}(x_{t}-{\bar {x}})^{2}}}\,,}

which converges as the sample size 
 
 
 
 T
 
 
 {\displaystyle T}
 
 increases without bound:

β
                ^
              
            
          
          
            x
          
        
        
          →
          
            p
          
        
        
          
            
              Cov
              ⁡
              [
              
              
                x
                
                  t
                
              
              ,
              
                y
                
                  t
                
              
              
              ]
            
            
              Var
              ⁡
              [
              
              
                x
                
                  t
                
              
              
              ]
            
          
        
        =
        
          
            
              β
              
                σ
                
                  
                    x
                    
                      ∗
                    
                  
                
                
                  2
                
              
            
            
              
                σ
                
                  
                    x
                    
                      ∗
                    
                  
                
                
                  2
                
              
              +
              
                σ
                
                  η
                
                
                  2
                
              
            
          
        
        =
        
          
            β
            
              1
              +
              
                σ
                
                  η
                
                
                  2
                
              
              
                /
              
              
                σ
                
                  
                    x
                    
                      ∗
                    
                  
                
                
                  2
                
              
            
          
        
        
        .
      
    
    {\displaystyle {\hat {\beta }}_{x}\xrightarrow {p} {\frac {\operatorname {Cov} [\,x_{t},y_{t}\,]}{\operatorname {Var} [\,x_{t}\,]}}={\frac {\beta \sigma _{x^{*}}^{2}}{\sigma _{x^{*}}^{2}+\sigma _{\eta }^{2}}}={\frac {\beta }{1+\sigma _{\eta }^{2}/\sigma _{x^{*}}^{2}}}\,.}

This is in contrast to the "true" effect of 
 
 
 
 β
 
 
 {\displaystyle \beta }
 
, estimated using the 
 
 
 
 
 x
 
 t
 
 
 ∗
 
 
 
 
 {\displaystyle x_{t}^{*}}
 
,:

β
              ^
            
          
        
        =
        
          
            
              
                
                  
                    1
                    T
                  
                
              
              
                ∑
                
                  t
                  =
                  1
                
                
                  T
                
              
              (
              
                x
                
                  t
                
                
                  ∗
                
              
              −
              
                
                  
                    x
                    ¯
                  
                
              
              )
              (
              
                y
                
                  t
                
              
              −
              
                
                  
                    y
                    ¯
                  
                
              
              )
            
            
              
                
                  
                    1
                    T
                  
                
              
              
                ∑
                
                  t
                  =
                  1
                
                
                  T
                
              
              (
              
                x
                
                  t
                
                
                  ∗
                
              
              −
              
                
                  
                    x
                    ¯
                  
                
              
              
                )
                
                  2
                
              
            
          
        
        
        ,
      
    
    {\displaystyle {\hat {\beta }}={\frac {{\tfrac {1}{T}}\sum _{t=1}^{T}(x_{t}^{*}-{\bar {x}})(y_{t}-{\bar {y}})}{{\tfrac {1}{T}}\sum _{t=1}^{T}(x_{t}^{*}-{\bar {x}})^{2}}}\,,}

Variances are non-negative, so that in the limit the estimated 
 
 
 
 
 
 
 
 β
 ^
 
 
 
 
 x
 
 
 
 
 {\displaystyle {\hat {\beta }}_{x}}
 
 is smaller than 
 
 
 
 
 
 
 β
 ^
 
 
 
 
 
 {\displaystyle {\hat {\beta }}}
 
, an effect which statisticians call attenuation or <a href="/facts/Regression_dilution/yshVFGzE">regression dilution</a>.<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a> Thus the ‘naïve’ least squares estimator 
 
 
 
 
 
 
 
 β
 ^
 
 
 
 
 x
 
 
 
 
 {\displaystyle {\hat {\beta }}_{x}}
 
 is an <a href="/facts/Consistent_estimator/HAKDWYEM">inconsistent</a> estimator for 
 
 
 
 β
 
 
 {\displaystyle \beta }
 
. However, 
 
 
 
 
 
 
 
 β
 ^
 
 
 
 
 x
 
 
 
 
 {\displaystyle {\hat {\beta }}_{x}}
 
 is a <a href="/facts/Consistent_estimator/HAKDWYEM">consistent estimator</a> of the parameter required for a best linear predictor of 
 
 
 
 y
 
 
 {\displaystyle y}
 
 given the observed 
 
 
 
 
 x
 
 t
 
 
 
 
 {\displaystyle x_{t}}
 
: in some applications this may be what is required, rather than an estimate of the 'true' regression coefficient 
 
 
 
 β
 
 
 {\displaystyle \beta }
 
, although that would assume that the variance of the errors in the estimation and prediction is identical. This follows directly from the result quoted immediately above, and the fact that the regression coefficient relating the 
 
 
 
 
 y
 
 t
 
 
 
 
 {\displaystyle y_{t}}
 
′s to the actually observed 
 
 
 
 
 x
 
 t
 
 
 
 
 {\displaystyle x_{t}}
 
′s, in a simple linear regression, is given by

β
          
            x
          
        
        =
        
          
            
              Cov
              ⁡
              [
              
              
                x
                
                  t
                
              
              ,
              
                y
                
                  t
                
              
              
              ]
            
            
              Var
              ⁡
              [
              
              
                x
                
                  t
                
              
              
              ]
            
          
        
        .
      
    
    {\displaystyle \beta _{x}={\frac {\operatorname {Cov} [\,x_{t},y_{t}\,]}{\operatorname {Var} [\,x_{t}\,]}}.}

It is this coefficient, rather than 
 
 
 
 β
 
 
 {\displaystyle \beta }
 
, that would be required for constructing a predictor of 
 
 
 
 y
 
 
 {\displaystyle y}
 
 based on an observed 
 
 
 
 x
 
 
 {\displaystyle x}
 
 which is subject to noise.
It can be argued that almost all existing data sets contain errors of different nature and magnitude, so that attenuation bias is extremely frequent (although in multivariate regression the direction of bias is ambiguous<a class="footnote-ref" id="fnref:5" href="#fn:5">5</a>). <a href="/facts/Jerry_Hausman/V1q3hpwU">Jerry Hausman</a> sees this as an iron law of econometrics: "The magnitude of the estimate is usually smaller than expected."<a class="footnote-ref" id="fnref:6" href="#fn:6">6</a>

<h2 id="specification">Specification</h2>
Usually, measurement error models are described using the <a href="/facts/Latent_variable_model/FCcgtsmc">latent variables</a> approach. If 
 
 
 
 y
 
 
 {\displaystyle y}
 
 is the response variable and 
 
 
 
 x
 
 
 {\displaystyle x}
 
 are observed values of the regressors, then it is assumed there exist some latent variables 
 
 
 
 
 y
 
 ∗
 
 
 
 
 {\displaystyle y^{*}}
 
 and 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 which follow the model's "true" <a href="/facts/Function_(mathematics)/CdReEKCh">functional relationship</a> 
 
 
 
 g
 (
 ⋅
 )
 
 
 {\displaystyle g(\cdot )}
 
, and such that the observed quantities are their noisy observations:

{
            
              
                
                  
                    y
                    
                      ∗
                    
                  
                  =
                  g
                  (
                  
                    x
                    
                      ∗
                    
                  
                  
                  ,
                  w
                  
                  
                    |
                  
                  
                  θ
                  )
                  ,
                
              
              
                
                  y
                  =
                  
                    y
                    
                      ∗
                    
                  
                  +
                  ε
                  ,
                
              
              
                
                  x
                  =
                  
                    x
                    
                      ∗
                    
                  
                  +
                  η
                  ,
                
              
            
            
          
        
      
    
    {\displaystyle {\begin{cases}y^{*}=g(x^{*}\!,w\,|\,\theta ),\\y=y^{*}+\varepsilon ,\\x=x^{*}+\eta ,\end{cases}}}

where 
 
 
 
 θ
 
 
 {\displaystyle \theta }
 
 is the model's <a href="/facts/Parameter/zFvVFMv9">parameter</a> and 
 
 
 
 w
 
 
 {\displaystyle w}
 
 are those regressors which are assumed to be error-free (for example, when linear regression contains an intercept, the regressor which corresponds to the constant certainly has no "measurement errors"). Depending on the specification these error-free regressors may or may not be treated separately; in the latter case it is simply assumed that corresponding entries in the variance matrix of 
 
 
 
 η
 
 
 {\displaystyle \eta }
 
's are zero.
The variables 
 
 
 
 y
 
 
 {\displaystyle y}
 
, 
 
 
 
 x
 
 
 {\displaystyle x}
 
, 
 
 
 
 w
 
 
 {\displaystyle w}
 
 are all observed, meaning that the statistician possesses a <a href="/facts/Data_set/eDvegUTK">data set</a> of 
 
 
 
 n
 
 
 {\displaystyle n}
 
 <a href="/facts/Statistical_unit/oIW2uEBb">statistical units</a> 
 
 
 
 
 
 {
 
 
 y
 
 i
 
 
 ,
 
 x
 
 i
 
 
 ,
 
 w
 
 i
 
 
 
 }
 
 
 i
 =
 1
 ,
 …
 ,
 n
 
 
 
 
 {\displaystyle \left\{y_{i},x_{i},w_{i}\right\}_{i=1,\dots ,n}}
 
 which follow the <a href="/facts/Data_collection/rEGBdDzj">data generating process</a> described above; the latent variables 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
, 
 
 
 
 
 y
 
 ∗
 
 
 
 
 {\displaystyle y^{*}}
 
, 
 
 
 
 ε
 
 
 {\displaystyle \varepsilon }
 
, and 
 
 
 
 η
 
 
 {\displaystyle \eta }
 
 are not observed, however.
This specification does not encompass all the existing errors-in-variables models. For example, in some of them, function 
 
 
 
 g
 (
 ⋅
 )
 
 
 {\displaystyle g(\cdot )}
 
 may be <a href="/facts/Nonparametric_statistics/svTDJHTb">non-parametric</a> or semi-parametric. Other approaches model the relationship between 
 
 
 
 
 y
 
 ∗
 
 
 
 
 {\displaystyle y^{*}}
 
 and 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 as distributional instead of functional; that is, they assume that 
 
 
 
 
 y
 
 ∗
 
 
 
 
 {\displaystyle y^{*}}
 
 conditionally on 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 follows a certain (usually parametric) distribution.

<h3>Terminology and assumptions</h3>
<ul><li>The observed variable 
 
 
 
 x
 
 
 {\displaystyle x}
 
 may be called the manifest, indicator, or <a href="/facts/Proxy_(statistics)/W5V2eqaT">proxy variable</a>.</li>
<li>The unobserved variable 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 may be called the latent or true variable. It may be regarded either as an unknown constant (in which case the model is called a functional model), or as a random variable (correspondingly a structural model).<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a></li>
<li>The relationship between the measurement error 
 
 
 
 η
 
 
 {\displaystyle \eta }
 
 and the latent variable 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 can be modeled in different ways:
<ul><li>Classical errors: 
 
 
 
 η
 ⊥
 
 x
 
 ∗
 
 
 
 
 {\displaystyle \eta \perp x^{*}}
 
 the errors are <a href="/facts/Independence_(probability_theory)/NUzQtnUL">independent</a> of the latent variable. This is the most common assumption; it implies that the errors are introduced by the measuring device and their magnitude does not depend on the value being measured.</li>
<li>Mean-independence: 
 
 
 
 E
 ⁡
 [
 η
 
 |
 
 
 x
 
 ∗
 
 
 ]
 
 =
 
 0
 ,
 
 
 {\displaystyle \operatorname {E} [\eta |x^{*}]\,=\,0,}
 
 the errors are mean-zero for every value of the latent regressor. This is a less restrictive assumption than the classical one,<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a> as it allows for the presence of <a href="/facts/Heteroscedasticity/msBQWI7G">heteroscedasticity</a> or other effects in the measurement errors.</li>
<li><a href="/facts/Berkson_error_model/2I4K6haH">Berkson's errors</a>: 
 
 
 
 η
 
 ⊥
 
 x
 ,
 
 
 {\displaystyle \eta \,\perp \,x,}
 
 the errors are independent of the observed regressor x.<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a> This assumption has very limited applicability. One example is round-off errors: for example, if a person's age* is a <a href="/facts/Continuous_and_discrete_variables/TWl5feEw">continuous random variable</a>, whereas the observed age is truncated to the next smallest integer, then the truncation error is approximately independent of the observed age. Another possibility is with the fixed design experiment: for example, if a scientist decides to make a measurement at a certain predetermined moment of time 
 
 
 
 x
 
 
 {\displaystyle x}
 
, say at 
 
 
 
 x
 =
 10
 s
 
 
 {\displaystyle x=10s}
 
, then the real measurement may occur at some other value of 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 (for example due to her finite reaction time) and such measurement error will be generally independent of the "observed" value of the regressor.</li>
<li>Misclassification errors: special case used for the <a href="/facts/Dummy_variable_(statistics)/LbT5Wcjm">dummy regressors</a>. If 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 is an indicator of a certain event or condition (such as person is male/female, some medical treatment given/not, etc.), then the measurement error in such regressor will correspond to the incorrect classification similar to <a href="/facts/Type_I_and_type_II_errors/aazbF3fq">type I and type II errors</a> in statistical testing. In this case the error 
 
 
 
 η
 
 
 {\displaystyle \eta }
 
 may take only 3 possible values, and its distribution conditional on 
 
 
 
 
 x
 
 ∗
 
 
 
 
 {\displaystyle x^{*}}
 
 is modeled with two parameters: 
 
 
 
 α
 =
 Pr
 ⁡
 [
 η
 =
 −
 1
 
 |
 
 
 x
 
 ∗
 
 
 =
 1
 ]
 
 
 {\displaystyle \alpha =\operatorname {Pr} [\eta =-1|x^{*}=1]}
 
, and 
 
 
 
 β
 =
 Pr
 ⁡
 [
 η
 =
 1
 
 |
 
 
 x
 
 ∗
 
 
 =
 0
 ]
 
 
 {\displaystyle \beta =\operatorname {Pr} [\eta =1|x^{*}=0]}
 
. The necessary condition for identification is that 
 
 
 
 α
 +
 β
 <
 1
 
 
 {\displaystyle \alpha +\beta <1}
 
, that is misclassification should not happen "too often". (This idea can be generalized to discrete variables with more than two possible values.)</li></ul></li></ul>
<h2 id="linear-model">Linear model</h2>
Linear errors-in-variables models were studied first, probably because <a href="/facts/Linear_model/x1we3e1d">linear models</a> were so widely used and they are easier than non-linear ones. Unlike standard <a href="/facts/Ordinary_least_squares/q7H9k5vM">least squares</a> regression (OLS), extending errors in variables regression (EiV) from the simple to the multivariable case is not straightforward, unless one treats all variables in the same way i.e. assume equal reliability.<a class="footnote-ref" id="fnref:10" href="#fn:10">10</a>

<h3>Simple linear model</h3>
The simple linear errors-in-variables model was already presented in the "motivation" section:

{
            
              
                
                  
                    y
                    
                      t
                    
                  
                  =
                  α
                  +
                  β
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    ε
                    
                      t
                    
                  
                  ,
                
              
              
                
                  
                    x
                    
                      t
                    
                  
                  =
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    η
                    
                      t
                    
                  
                  ,
                
              
            
            
          
        
      
    
    {\displaystyle {\begin{cases}y_{t}=\alpha +\beta x_{t}^{*}+\varepsilon _{t},\\x_{t}=x_{t}^{*}+\eta _{t},\end{cases}}}

where all variables are <a href="/facts/Scalar_(mathematics)/MkDbA4Eo">scalar</a>. Here α and β are the parameters of interest, whereas σε and ση—standard deviations of the error terms—are the <a href="/facts/Nuisance_parameter/acJWVqxj">nuisance parameters</a>. The "true" regressor x* is treated as a random variable (structural model), independent of the measurement error η (classic assumption).
This model is <a href="/facts/Identifiable/YfE1ydRa">identifiable</a> in two cases: (1) either the latent regressor x* is not <a href="/facts/Normal_distribution/UapjjPyQ">normally distributed</a>, (2) or x* has normal distribution, but neither εt nor ηt are divisible by a normal distribution.<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a> That is, the parameters α, β can be consistently estimated from the data set 
 
 
 
 
 (
 
 x
 
 t
 
 
 ,
 
 
 y
 
 t
 
 
 
 )
 
 t
 =
 1
 
 
 T
 
 
 
 
 
 {\displaystyle \scriptstyle (x_{t},\,y_{t})_{t=1}^{T}}
 
 without any additional information, provided the latent regressor is not Gaussian.
Before this identifiability result was established, statisticians attempted to apply the <a href="/facts/Maximum_likelihood/0Yq2dpQD">maximum likelihood</a> technique by assuming that all variables are normal, and then concluded that the model is not identified. The suggested remedy was to assume that some of the parameters of the model are known or can be estimated from the outside source. Such estimation methods include<a class="footnote-ref" id="fnref:12" href="#fn:12">12</a>

<ul><li><a href="/facts/Deming_regression/j4bFDpub">Deming regression</a> — assumes that the ratio δ = σ²ε/σ²η is known. This could be appropriate for example when errors in y and x are both caused by measurements, and the accuracy of measuring devices or procedures are known. The case when δ = 1 is also known as the <a href="/facts/Orthogonal_regression/j4bFDpub">orthogonal regression</a>.</li>
<li>Regression with known <a href="/facts/Reliability_(statistics)/2QJrAeB5">reliability ratio</a> λ = σ²∗/ ( σ²η + σ²∗), where σ²∗ is the variance of the latent regressor. Such approach may be applicable for example when repeating measurements of the same unit are available, or when the reliability ratio has been known from the independent study. In this case the consistent estimate of slope is equal to the least-squares estimate divided by λ.</li>
<li>Regression with known σ²η may occur when the source of the errors in x's is known and their variance can be calculated. This could include rounding errors, or errors introduced by the measuring device. When σ²η is known we can compute the reliability ratio as λ = ( σ²x − σ²η) / σ²x and reduce the problem to the previous case.</li></ul>
Estimation methods that do not assume knowledge of some of the parameters of the model, include

<ul><li>Method of moments — the <a href="/facts/Generalized_method_of_moments/mNc3410V">GMM</a> estimator based on the third- (or higher-) order joint <a href="/facts/Cumulant/crLC9QLE">cumulants</a> of observable variables. The slope coefficient can be estimated from<a class="footnote-ref" id="fnref:13" href="#fn:13">13</a>

β
              ^
            
          
        
        =
        
          
            
              
                
                  
                    K
                    ^
                  
                
              
              (
              
                n
                
                  1
                
              
              ,
              
                n
                
                  2
                
              
              +
              1
              )
            
            
              
                
                  
                    K
                    ^
                  
                
              
              (
              
                n
                
                  1
                
              
              +
              1
              ,
              
                n
                
                  2
                
              
              )
            
          
        
        ,
        
        
          n
          
            1
          
        
        ,
        
          n
          
            2
          
        
        >
        0
        ,
      
    
    {\displaystyle {\hat {\beta }}={\frac {{\hat {K}}(n_{1},n_{2}+1)}{{\hat {K}}(n_{1}+1,n_{2})}},\quad n_{1},n_{2}>0,}

where (n1,n2) are such that K(n1+1,n2) — the joint <a href="/facts/Cumulant/crLC9QLE">cumulant</a> of (x,y) — is not zero. In the case when the third central moment of the latent regressor x* is non-zero, the formula reduces to

β
 ^
 
 
 
 =
 
 
 
 
 
 
 1
 T
 
 
 
 
 ∑
 
 t
 =
 1
 
 
 T
 
 
 (
 
 x
 
 t
 
 
 −
 
 
 
 x
 ¯
 
 
 
 )
 (
 
 y
 
 t
 
 
 −
 
 
 
 y
 ¯
 
 
 
 
 )
 
 2
 
 
 
 
 
 
 
 1
 T
 
 
 
 
 ∑
 
 t
 =
 1
 
 
 T
 
 
 (
 
 x
 
 t
 
 
 −
 
 
 
 x
 ¯
 
 
 
 
 )
 
 2
 
 
 (
 
 y
 
 t
 
 
 −
 
 
 
 y
 ¯
 
 
 
 )
 
 
 
  
 .
 
 
 {\displaystyle {\hat {\beta }}={\frac {{\tfrac {1}{T}}\sum _{t=1}^{T}(x_{t}-{\bar {x}})(y_{t}-{\bar {y}})^{2}}{{\tfrac {1}{T}}\sum _{t=1}^{T}(x_{t}-{\bar {x}})^{2}(y_{t}-{\bar {y}})}}\ .}
 
</li><li><a href="/facts/Instrumental_variables/cEQ1xwT6">Instrumental variables</a> — a regression which requires that certain additional data variables z, called instruments, were available. These variables should be uncorrelated with the errors in the equation for the dependent (outcome) variable (valid), and they should also be correlated (relevant) with the true regressors x*. If such variables can be found then the estimator takes form

β
 ^
 
 
 
 =
 
 
 
 
 
 
 1
 T
 
 
 
 
 ∑
 
 t
 =
 1
 
 
 T
 
 
 (
 
 z
 
 t
 
 
 −
 
 
 
 z
 ¯
 
 
 
 )
 (
 
 y
 
 t
 
 
 −
 
 
 
 y
 ¯
 
 
 
 )
 
 
 
 
 
 1
 T
 
 
 
 
 ∑
 
 t
 =
 1
 
 
 T
 
 
 (
 
 z
 
 t
 
 
 −
 
 
 
 z
 ¯
 
 
 
 )
 (
 
 x
 
 t
 
 
 −
 
 
 
 x
 ¯
 
 
 
 )
 
 
 
  
 .
 
 
 {\displaystyle {\hat {\beta }}={\frac {{\tfrac {1}{T}}\sum _{t=1}^{T}(z_{t}-{\bar {z}})(y_{t}-{\bar {y}})}{{\tfrac {1}{T}}\sum _{t=1}^{T}(z_{t}-{\bar {z}})(x_{t}-{\bar {x}})}}\ .}
 
</li><li>The geometric mean functional relationship. This treats both variables as having the same reliability. The resulting slope is the geometric mean of the ordinary least squares slope and the reverse least squares slope, i.e. the two red lines in the diagram.<a class="footnote-ref" id="fnref:14" href="#fn:14">14</a></li></ul>
<h3>Multivariable linear model</h3>
The multivariable model looks exactly like the simple linear model, only this time β, ηt, xt and x*t are k×1 vectors.

{
            
              
                
                  
                    y
                    
                      t
                    
                  
                  =
                  α
                  +
                  
                    β
                    ′
                  
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    ε
                    
                      t
                    
                  
                  ,
                
              
              
                
                  
                    x
                    
                      t
                    
                  
                  =
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    η
                    
                      t
                    
                  
                  .
                
              
            
            
          
        
      
    
    {\displaystyle {\begin{cases}y_{t}=\alpha +\beta 'x_{t}^{*}+\varepsilon _{t},\\x_{t}=x_{t}^{*}+\eta _{t}.\end{cases}}}

In the case when (εt,ηt) is jointly normal, the parameter β is not identified if and only if there is a non-singular k×k block matrix [a A], where a is a k×1 vector such that a′x* is distributed normally and independently of A′x*. In the case when εt, ηt1,..., ηtk are mutually independent, the parameter β is not identified if and only if in addition to the conditions above some of the errors can be written as the sum of two independent variables one of which is normal.<a class="footnote-ref" id="fnref:15" href="#fn:15">15</a>
Some of the estimation methods for multivariable linear models are

<ul><li><a href="/facts/Total_least_squares/1jRjpM8f">Total least squares</a> is an extension of <a href="/facts/Deming_regression/j4bFDpub">Deming regression</a> to the multivariable setting. When all the k+1 components of the vector (ε,η) have equal variances and are independent, this is equivalent to running the orthogonal regression of y on the vector x — that is, the regression which minimizes the sum of squared distances between points (yt,xt) and the k-dimensional hyperplane of "best fit".</li><li>The <a href="/facts/Generalized_method_of_moments/mNc3410V">method of moments</a> estimator<a class="footnote-ref" id="fnref:16" href="#fn:16">16</a> can be constructed based on the moment conditions E[zt·(yt − α − β'xt)] = 0, where the (5k+3)-dimensional vector of instruments zt is defined as

z
                  
                    t
                  
                
                =
                
                  
                    (
                    
                      1
                       
                      
                        z
                        
                          t
                          1
                        
                        ′
                      
                       
                      
                        z
                        
                          t
                          2
                        
                        ′
                      
                       
                      
                        z
                        
                          t
                          3
                        
                        ′
                      
                       
                      
                        z
                        
                          t
                          4
                        
                        ′
                      
                       
                      
                        z
                        
                          t
                          5
                        
                        ′
                      
                       
                      
                        z
                        
                          t
                          6
                        
                        ′
                      
                       
                      
                        z
                        
                          t
                          7
                        
                        ′
                      
                    
                    )
                  
                  ′
                
                ,
                
                
                  where
                
              
            
            
              
              
                
                  z
                  
                    t
                    1
                  
                
                =
                
                  x
                  
                    t
                  
                
                ∘
                
                  x
                  
                    t
                  
                
              
            
            
              
              
                
                  z
                  
                    t
                    2
                  
                
                =
                
                  x
                  
                    t
                  
                
                
                  y
                  
                    t
                  
                
              
            
            
              
              
                
                  z
                  
                    t
                    3
                  
                
                =
                
                  y
                  
                    t
                  
                  
                    2
                  
                
              
            
            
              
              
                
                  z
                  
                    t
                    4
                  
                
                =
                
                  x
                  
                    t
                  
                
                ∘
                
                  x
                  
                    t
                  
                
                ∘
                
                  x
                  
                    t
                  
                
                −
                3
                
                  
                    (
                  
                
                E
                ⁡
                [
                
                  x
                  
                    t
                  
                
                
                  x
                  
                    t
                  
                  ′
                
                ]
                ∘
                
                  I
                  
                    k
                  
                
                
                  
                    )
                  
                
                
                  x
                  
                    t
                  
                
              
            
            
              
              
                
                  z
                  
                    t
                    5
                  
                
                =
                
                  x
                  
                    t
                  
                
                ∘
                
                  x
                  
                    t
                  
                
                
                  y
                  
                    t
                  
                
                −
                2
                
                  
                    (
                  
                
                E
                ⁡
                [
                
                  y
                  
                    t
                  
                
                
                  x
                  
                    t
                  
                  ′
                
                ]
                ∘
                
                  I
                  
                    k
                  
                
                
                  
                    )
                  
                
                
                  x
                  
                    t
                  
                
                −
                
                  y
                  
                    t
                  
                
                
                  
                    (
                  
                
                E
                ⁡
                [
                
                  x
                  
                    t
                  
                
                
                  x
                  
                    t
                  
                  ′
                
                ]
                ∘
                
                  I
                  
                    k
                  
                
                
                  
                    )
                  
                
                
                  ι
                  
                    k
                  
                
              
            
            
              
              
                
                  z
                  
                    t
                    6
                  
                
                =
                
                  x
                  
                    t
                  
                
                
                  y
                  
                    t
                  
                  
                    2
                  
                
                −
                E
                ⁡
                [
                
                  y
                  
                    t
                  
                  
                    2
                  
                
                ]
                
                  x
                  
                    t
                  
                
                −
                2
                
                  y
                  
                    t
                  
                
                E
                ⁡
                [
                
                  x
                  
                    t
                  
                
                
                  y
                  
                    t
                  
                
                ]
              
            
            
              
              
                
                  z
                  
                    t
                    7
                  
                
                =
                
                  y
                  
                    t
                  
                  
                    3
                  
                
                −
                3
                
                  y
                  
                    t
                  
                
                E
                ⁡
                [
                
                  y
                  
                    t
                  
                  
                    2
                  
                
                ]
              
            
          
        
      
    
    {\displaystyle {\begin{aligned}&z_{t}=\left(1\ z_{t1}'\ z_{t2}'\ z_{t3}'\ z_{t4}'\ z_{t5}'\ z_{t6}'\ z_{t7}'\right)',\quad {\text{where}}\\&z_{t1}=x_{t}\circ x_{t}\\&z_{t2}=x_{t}y_{t}\\&z_{t3}=y_{t}^{2}\\&z_{t4}=x_{t}\circ x_{t}\circ x_{t}-3{\big (}\operatorname {E} [x_{t}x_{t}']\circ I_{k}{\big )}x_{t}\\&z_{t5}=x_{t}\circ x_{t}y_{t}-2{\big (}\operatorname {E} [y_{t}x_{t}']\circ I_{k}{\big )}x_{t}-y_{t}{\big (}\operatorname {E} [x_{t}x_{t}']\circ I_{k}{\big )}\iota _{k}\\&z_{t6}=x_{t}y_{t}^{2}-\operatorname {E} [y_{t}^{2}]x_{t}-2y_{t}\operatorname {E} [x_{t}y_{t}]\\&z_{t7}=y_{t}^{3}-3y_{t}\operatorname {E} [y_{t}^{2}]\end{aligned}}}

where 
 
 
 
 ∘
 
 
 {\displaystyle \circ }
 
 designates the <a href="/facts/Hadamard_product_(matrices)/NjAUd4o4">Hadamard product</a> of matrices, and variables xt, yt have been preliminarily de-meaned. The authors of the method suggest to use Fuller's modified IV estimator.<a class="footnote-ref" id="fnref:17" href="#fn:17">17</a>

This method can be extended to use moments higher than the third order, if necessary, and to accommodate variables measured without error.<a class="footnote-ref" id="fnref:18" href="#fn:18">18</a></li><li>The <a href="/facts/Instrumental_variables/cEQ1xwT6">instrumental variables</a> approach requires us to find additional data variables zt that serve as instruments for the mismeasured regressors xt. This method is the simplest from the implementation point of view; however, its disadvantage is that it requires collecting additional data, which may be costly or even impossible. When the instruments can be found, the estimator takes standard form

β
 ^
 
 
 
 =
 
 
 (
 
 
 
 X
 ′
 
 Z
 (
 
 Z
 ′
 
 Z
 
 )
 
 −
 1
 
 
 
 Z
 ′
 
 X
 
 
 
 )
 
 
 
 −
 1
 
 
 
 X
 ′
 
 Z
 (
 
 Z
 ′
 
 Z
 
 )
 
 −
 1
 
 
 
 Z
 ′
 
 y
 .
 
 
 {\displaystyle {\hat {\beta }}={\big (}X'Z(Z'Z)^{-1}Z'X{\big )}^{-1}X'Z(Z'Z)^{-1}Z'y.}
 
</li><li>The impartial fitting approach treats all variables in the same way by assuming equal reliability, and does not require any distinction between explanatory and response variables as the resulting equation can be rearranged. It is the simplest measurement error model, and is a generalization of the geometric mean functional relationship mentioned above for two variables. It only requires covariances to be computed, and so can be estimated using basic spreadsheet functions.<a class="footnote-ref" id="fnref:19" href="#fn:19">19</a></li></ul>

<h2 id="non-linear-models">Non-linear models</h2>
A generic non-linear measurement error model takes form

{
            
              
                
                  
                    y
                    
                      t
                    
                  
                  =
                  g
                  (
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  )
                  +
                  
                    ε
                    
                      t
                    
                  
                  ,
                
              
              
                
                  
                    x
                    
                      t
                    
                  
                  =
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    η
                    
                      t
                    
                  
                  .
                
              
            
            
          
        
      
    
    {\displaystyle {\begin{cases}y_{t}=g(x_{t}^{*})+\varepsilon _{t},\\x_{t}=x_{t}^{*}+\eta _{t}.\end{cases}}}

Here function g can be either parametric or non-parametric. When function g is parametric it will be written as g(x*, β).
For a general vector-valued regressor x* the conditions for model <a href="/facts/Identifiability/YfE1ydRa">identifiability</a> are not known. However, in the case of scalar x* the model is identified unless the function g is of the "log-exponential" form<a class="footnote-ref" id="fnref:20" href="#fn:20">20</a>

g
        (
        
          x
          
            ∗
          
        
        )
        =
        a
        +
        b
        ln
        ⁡
        
          
            (
          
        
        
          e
          
            c
            
              x
              
                ∗
              
            
          
        
        +
        d
        
          
            )
          
        
      
    
    {\displaystyle g(x^{*})=a+b\ln {\big (}e^{cx^{*}}+d{\big )}}

and the latent regressor x* has density

f
          
            
              x
              
                ∗
              
            
          
        
        (
        x
        )
        =
        
          
            {
            
              
                
                  A
                  
                    e
                    
                      −
                      B
                      
                        e
                        
                          C
                          x
                        
                      
                      +
                      C
                      D
                      x
                    
                  
                  (
                  
                    e
                    
                      C
                      x
                    
                  
                  +
                  E
                  
                    )
                    
                      −
                      F
                    
                  
                  ,
                
                
                  
                    if
                  
                   
                  d
                  >
                  0
                
              
              
                
                  A
                  
                    e
                    
                      −
                      B
                      
                        x
                        
                          2
                        
                      
                      +
                      C
                      x
                    
                  
                
                
                  
                    if
                  
                   
                  d
                  =
                  0
                
              
            
            
          
        
      
    
    {\displaystyle f_{x^{*}}(x)={\begin{cases}Ae^{-Be^{Cx}+CDx}(e^{Cx}+E)^{-F},&{\text{if}}\ d>0\\Ae^{-Bx^{2}+Cx}&{\text{if}}\ d=0\end{cases}}}

where constants A,B,C,D,E,F may depend on a,b,c,d.
Despite this optimistic result, as of now no methods exist for estimating non-linear errors-in-variables models without any extraneous information. However, there are several techniques which make use of some additional data: either the instrumental variables, or repeated observations.

<h3>Instrumental variables methods</h3>
<ul><li>Newey's simulated moments method<a class="footnote-ref" id="fnref:21" href="#fn:21">21</a> for parametric models – requires that there is an additional set of observed predictor variables zt, such that the true regressor can be expressed as

x
          
            t
          
          
            ∗
          
        
        =
        
          π
          
            0
          
          ′
        
        
          z
          
            t
          
        
        +
        
          σ
          
            0
          
        
        
          ζ
          
            t
          
        
        ,
      
    
    {\displaystyle x_{t}^{*}=\pi _{0}'z_{t}+\sigma _{0}\zeta _{t},}

where π0 and σ0 are (unknown) constant matrices, and ζt ⊥ zt. The coefficient π0 can be estimated using standard <a href="/facts/Ordinary_least_squares/q7H9k5vM">least squares</a> regression of x on z. The distribution of ζt is unknown; however, we can model it as belonging to a flexible parametric family – the <a href="/facts/Edgeworth_series/WJHXQ3JN">Edgeworth series</a>:

f
          
            ζ
          
        
        (
        v
        ;
        
        γ
        )
        =
        ϕ
        (
        v
        )
        
        
          
            ∑
            
              j
              =
              1
            
            
              J
            
          
          
          
            γ
            
              j
            
          
          
            v
            
              j
            
          
        
      
    
    {\displaystyle f_{\zeta }(v;\,\gamma )=\phi (v)\,\textstyle \sum _{j=1}^{J}\!\gamma _{j}v^{j}}

where ϕ is the <a href="/facts/Standard_normal/UapjjPyQ">standard normal</a> distribution.
Simulated moments can be computed using the <a href="/facts/Importance_sampling/jrbUnMXa">importance sampling</a> algorithm: first we generate several random variables {vts ~ ϕ, s = 1,…,S, t = 1,…,T} from the standard normal distribution, then we compute the moments at t-th observation as

m
          
            t
          
        
        (
        θ
        )
        =
        A
        (
        
          z
          
            t
          
        
        )
        
          
            1
            S
          
        
        
          ∑
          
            s
            =
            1
          
          
            S
          
        
        H
        (
        
          x
          
            t
          
        
        ,
        
          y
          
            t
          
        
        ,
        
          z
          
            t
          
        
        ,
        
          v
          
            t
            s
          
        
        ;
        θ
        )
        
          ∑
          
            j
            =
            1
          
          
            J
          
        
        
        
          γ
          
            j
          
        
        
          v
          
            t
            s
          
          
            j
          
        
        ,
      
    
    {\displaystyle m_{t}(\theta )=A(z_{t}){\frac {1}{S}}\sum _{s=1}^{S}H(x_{t},y_{t},z_{t},v_{ts};\theta )\sum _{j=1}^{J}\!\gamma _{j}v_{ts}^{j},}

where θ = (β, σ, γ), A is just some function of the instrumental variables z, and H is a two-component vector of moments

H
                  
                    1
                  
                
                (
                
                  x
                  
                    t
                  
                
                ,
                
                  y
                  
                    t
                  
                
                ,
                
                  z
                  
                    t
                  
                
                ,
                
                  v
                  
                    t
                    s
                  
                
                ;
                θ
                )
                =
                
                  y
                  
                    t
                  
                
                −
                g
                (
                
                  
                    
                      
                        π
                        ^
                      
                    
                  
                  ′
                
                
                  z
                  
                    t
                  
                
                +
                σ
                
                  v
                  
                    t
                    s
                  
                
                ,
                β
                )
                ,
              
            
            
              
              
                
                  H
                  
                    2
                  
                
                (
                
                  x
                  
                    t
                  
                
                ,
                
                  y
                  
                    t
                  
                
                ,
                
                  z
                  
                    t
                  
                
                ,
                
                  v
                  
                    t
                    s
                  
                
                ;
                θ
                )
                =
                
                  z
                  
                    t
                  
                
                
                  y
                  
                    t
                  
                
                −
                (
                
                  
                    
                      
                        π
                        ^
                      
                    
                  
                  ′
                
                
                  z
                  
                    t
                  
                
                +
                σ
                
                  v
                  
                    t
                    s
                  
                
                )
                g
                (
                
                  
                    
                      
                        π
                        ^
                      
                    
                  
                  ′
                
                
                  z
                  
                    t
                  
                
                +
                σ
                
                  v
                  
                    t
                    s
                  
                
                ,
                β
                )
              
            
          
        
      
    
    {\displaystyle {\begin{aligned}&H_{1}(x_{t},y_{t},z_{t},v_{ts};\theta )=y_{t}-g({\hat {\pi }}'z_{t}+\sigma v_{ts},\beta ),\\&H_{2}(x_{t},y_{t},z_{t},v_{ts};\theta )=z_{t}y_{t}-({\hat {\pi }}'z_{t}+\sigma v_{ts})g({\hat {\pi }}'z_{t}+\sigma v_{ts},\beta )\end{aligned}}}

With moment functions mt one can apply standard <a href="/facts/Generalized_method_of_moments/mNc3410V">GMM</a> technique to estimate the unknown parameter θ.</li></ul>
<h3>Repeated observations</h3>
In this approach two (or maybe more) repeated observations of the regressor x* are available. Both observations contain their own measurement errors; however, those errors are required to be independent:

{
            
              
                
                  
                    x
                    
                      1
                      t
                    
                  
                  =
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    η
                    
                      1
                      t
                    
                  
                  ,
                
              
              
                
                  
                    x
                    
                      2
                      t
                    
                  
                  =
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    η
                    
                      2
                      t
                    
                  
                  ,
                
              
            
            
          
        
      
    
    {\displaystyle {\begin{cases}x_{1t}=x_{t}^{*}+\eta _{1t},\\x_{2t}=x_{t}^{*}+\eta _{2t},\end{cases}}}

where x* ⊥ η1 ⊥ η2. Variables η1, η2 need not be identically distributed (although if they are efficiency of the estimator can be slightly improved). With only these two observations it is possible to consistently estimate the density function of x* using Kotlarski's <a href="/facts/Deconvolution/Q5ECQykB">deconvolution</a> technique.<a class="footnote-ref" id="fnref:22" href="#fn:22">22</a>

<ul><li>Li's conditional density method for parametric models.<a class="footnote-ref" id="fnref:23" href="#fn:23">23</a> The regression equation can be written in terms of the observable variables as

E
        ⁡
        [
        
        
          y
          
            t
          
        
        
          |
        
        
          x
          
            t
          
        
        
        ]
        =
        ∫
        g
        (
        
          x
          
            t
          
          
            ∗
          
        
        ,
        β
        )
        
          f
          
            
              x
              
                ∗
              
            
            
              |
            
            x
          
        
        (
        
          x
          
            t
          
          
            ∗
          
        
        
          |
        
        
          x
          
            t
          
        
        )
        d
        
          x
          
            t
          
          
            ∗
          
        
        ,
      
    
    {\displaystyle \operatorname {E} [\,y_{t}|x_{t}\,]=\int g(x_{t}^{*},\beta )f_{x^{*}|x}(x_{t}^{*}|x_{t})dx_{t}^{*},}

where it would be possible to compute the integral if we knew the conditional density function ƒx*|x. If this function could be known or estimated, then the problem turns into standard non-linear regression, which can be estimated for example using the <a href="/facts/NLLS/3K5Nmz0S">NLLS</a> method.
Assuming for simplicity that η1, η2 are identically distributed, this conditional density can be computed as

f
                ^
              
            
          
          
            
              x
              
                ∗
              
            
            
              |
            
            x
          
        
        (
        
          x
          
            ∗
          
        
        
          |
        
        x
        )
        =
        
          
            
              
                
                  
                    
                      f
                      ^
                    
                  
                
                
                  
                    x
                    
                      ∗
                    
                  
                
              
              (
              
                x
                
                  ∗
                
              
              )
            
            
              
                
                  
                    
                      f
                      ^
                    
                  
                
                
                  x
                
              
              (
              x
              )
            
          
        
        
          ∏
          
            j
            =
            1
          
          
            k
          
        
        
          
            
              
                f
                ^
              
            
          
          
            
              η
              
                j
              
            
          
        
        
          
            (
          
        
        
          x
          
            j
          
        
        −
        
          x
          
            j
          
          
            ∗
          
        
        
          
            )
          
        
        ,
      
    
    {\displaystyle {\hat {f}}_{x^{*}|x}(x^{*}|x)={\frac {{\hat {f}}_{x^{*}}(x^{*})}{{\hat {f}}_{x}(x)}}\prod _{j=1}^{k}{\hat {f}}_{\eta _{j}}{\big (}x_{j}-x_{j}^{*}{\big )},}

where with slight abuse of notation xj denotes the j-th component of a vector.
All densities in this formula can be estimated using inversion of the empirical <a href="/facts/Characteristic_function_(probability_theory)/rl3uAGKM">characteristic functions</a>. In particular,

φ
                        ^
                      
                    
                  
                  
                    
                      η
                      
                        j
                      
                    
                  
                
                (
                v
                )
                =
                
                  
                    
                      
                        
                          
                            
                              φ
                              ^
                            
                          
                        
                        
                          
                            x
                            
                              j
                            
                          
                        
                      
                      (
                      v
                      ,
                      0
                      )
                    
                    
                      
                        
                          
                            
                              φ
                              ^
                            
                          
                        
                        
                          
                            x
                            
                              j
                            
                            
                              ∗
                            
                          
                        
                      
                      (
                      v
                      )
                    
                  
                
                ,
                
                
                  where 
                
                
                  
                    
                      
                        φ
                        ^
                      
                    
                  
                  
                    
                      x
                      
                        j
                      
                    
                  
                
                (
                
                  v
                  
                    1
                  
                
                ,
                
                  v
                  
                    2
                  
                
                )
                =
                
                  
                    1
                    T
                  
                
                
                  ∑
                  
                    t
                    =
                    1
                  
                  
                    T
                  
                
                
                  e
                  
                    i
                    
                      v
                      
                        1
                      
                    
                    
                      x
                      
                        1
                        t
                        j
                      
                    
                    +
                    i
                    
                      v
                      
                        2
                      
                    
                    
                      x
                      
                        2
                        t
                        j
                      
                    
                  
                
                ,
              
            
            
              
                
                  
                    
                      
                        φ
                        ^
                      
                    
                  
                  
                    
                      x
                      
                        j
                      
                      
                        ∗
                      
                    
                  
                
                (
                v
                )
                =
                exp
                ⁡
                
                  ∫
                  
                    0
                  
                  
                    v
                  
                
                
                  
                    
                      ∂
                      
                        
                          
                            
                              φ
                              ^
                            
                          
                        
                        
                          
                            x
                            
                              j
                            
                          
                        
                      
                      (
                      0
                      ,
                      
                        v
                        
                          2
                        
                      
                      )
                      
                        /
                      
                      ∂
                      
                        v
                        
                          1
                        
                      
                    
                    
                      
                        
                          
                            
                              φ
                              ^
                            
                          
                        
                        
                          
                            x
                            
                              j
                            
                          
                        
                      
                      (
                      0
                      ,
                      
                        v
                        
                          2
                        
                      
                      )
                    
                  
                
                d
                
                  v
                  
                    2
                  
                
                ,
              
            
            
              
              
                
                  
                    
                      
                        φ
                        ^
                      
                    
                  
                  
                    x
                  
                
                (
                u
                )
                =
                
                  
                    1
                    
                      2
                      T
                    
                  
                
                
                  ∑
                  
                    t
                    =
                    1
                  
                  
                    T
                  
                
                
                  
                    (
                  
                
                
                  e
                  
                    i
                    
                      u
                      ′
                    
                    
                      x
                      
                        1
                        t
                      
                    
                  
                
                +
                
                  e
                  
                    i
                    
                      u
                      ′
                    
                    
                      x
                      
                        2
                        t
                      
                    
                  
                
                
                  
                    )
                  
                
                ,
                
                
                  
                    
                      
                        φ
                        ^
                      
                    
                  
                  
                    
                      x
                      
                        ∗
                      
                    
                  
                
                (
                u
                )
                =
                
                  
                    
                      
                        
                          
                            
                              φ
                              ^
                            
                          
                        
                        
                          x
                        
                      
                      (
                      u
                      )
                    
                    
                      
                        ∏
                        
                          j
                          =
                          1
                        
                        
                          k
                        
                      
                      
                        
                          
                            
                              φ
                              ^
                            
                          
                        
                        
                          
                            η
                            
                              j
                            
                          
                        
                      
                      (
                      
                        u
                        
                          j
                        
                      
                      )
                    
                  
                
                .
              
            
          
        
      
    
    {\displaystyle {\begin{aligned}&{\hat {\varphi }}_{\eta _{j}}(v)={\frac {{\hat {\varphi }}_{x_{j}}(v,0)}{{\hat {\varphi }}_{x_{j}^{*}}(v)}},\quad {\text{where }}{\hat {\varphi }}_{x_{j}}(v_{1},v_{2})={\frac {1}{T}}\sum _{t=1}^{T}e^{iv_{1}x_{1tj}+iv_{2}x_{2tj}},\\{\hat {\varphi }}_{x_{j}^{*}}(v)=\exp \int _{0}^{v}{\frac {\partial {\hat {\varphi }}_{x_{j}}(0,v_{2})/\partial v_{1}}{{\hat {\varphi }}_{x_{j}}(0,v_{2})}}dv_{2},\\&{\hat {\varphi }}_{x}(u)={\frac {1}{2T}}\sum _{t=1}^{T}{\Big (}e^{iu'x_{1t}}+e^{iu'x_{2t}}{\Big )},\quad {\hat {\varphi }}_{x^{*}}(u)={\frac {{\hat {\varphi }}_{x}(u)}{\prod _{j=1}^{k}{\hat {\varphi }}_{\eta _{j}}(u_{j})}}.\end{aligned}}}

To invert these characteristic function one has to apply the inverse Fourier transform, with a trimming parameter C needed to ensure the numerical stability. For example:

f
 ^
 
 
 
 
 x
 
 
 (
 x
 )
 =
 
 
 1
 
 (
 2
 π
 
 )
 
 k
 
 
 
 
 
 
 ∫
 
 −
 C
 
 
 C
 
 
 ⋯
 
 ∫
 
 −
 C
 
 
 C
 
 
 
 e
 
 −
 i
 
 u
 ′
 
 x
 
 
 
 
 
 
 φ
 ^
 
 
 
 
 x
 
 
 (
 u
 )
 d
 u
 .
 
 
 {\displaystyle {\hat {f}}_{x}(x)={\frac {1}{(2\pi )^{k}}}\int _{-C}^{C}\cdots \int _{-C}^{C}e^{-iu'x}{\hat {\varphi }}_{x}(u)du.}
 
</li><li>Schennach's estimator for a parametric linear-in-parameters nonlinear-in-variables model.<a class="footnote-ref" id="fnref:24" href="#fn:24">24</a> This is a model of the form

{
            
              
                
                  
                    y
                    
                      t
                    
                  
                  =
                  
                    
                      ∑
                      
                        j
                        =
                        1
                      
                      
                        k
                      
                    
                    
                      β
                      
                        j
                      
                    
                    
                      g
                      
                        j
                      
                    
                    (
                    
                      x
                      
                        t
                      
                      
                        ∗
                      
                    
                    )
                    +
                    
                      ∑
                      
                        j
                        =
                        1
                      
                      
                        ℓ
                      
                    
                    
                      β
                      
                        k
                        +
                        j
                      
                    
                    
                      w
                      
                        j
                        t
                      
                    
                    +
                    
                      ε
                      
                        t
                      
                    
                    ,
                  
                
              
              
                
                  
                    x
                    
                      1
                      t
                    
                  
                  =
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    η
                    
                      1
                      t
                    
                  
                  ,
                
              
              
                
                  
                    x
                    
                      2
                      t
                    
                  
                  =
                  
                    x
                    
                      t
                    
                    
                      ∗
                    
                  
                  +
                  
                    η
                    
                      2
                      t
                    
                  
                  ,
                
              
            
            
          
        
      
    
    {\displaystyle {\begin{cases}y_{t}=\textstyle \sum _{j=1}^{k}\beta _{j}g_{j}(x_{t}^{*})+\sum _{j=1}^{\ell }\beta _{k+j}w_{jt}+\varepsilon _{t},\\x_{1t}=x_{t}^{*}+\eta _{1t},\\x_{2t}=x_{t}^{*}+\eta _{2t},\end{cases}}}

where wt represents variables measured without errors. The regressor x* here is scalar (the method can be extended to the case of vector x* as well).
If not for the measurement errors, this would have been a standard <a href="/facts/Linear_model/x1we3e1d">linear model</a> with the estimator

β
              ^
            
          
        
        =
        
          
            (
          
        
        
          
            
              E
              ^
            
          
        
        [
        
        
          ξ
          
            t
          
        
        
          ξ
          
            t
          
          ′
        
        
        ]
        
          
            
              )
            
          
          
            −
            1
          
        
        
          
            
              E
              ^
            
          
        
        [
        
        
          ξ
          
            t
          
        
        
          y
          
            t
          
        
        
        ]
        ,
      
    
    {\displaystyle {\hat {\beta }}={\big (}{\hat {\operatorname {E} }}[\,\xi _{t}\xi _{t}'\,]{\big )}^{-1}{\hat {\operatorname {E} }}[\,\xi _{t}y_{t}\,],}

where

ξ
          
            t
          
          ′
        
        =
        (
        
          g
          
            1
          
        
        (
        
          x
          
            t
          
          
            ∗
          
        
        )
        ,
        ⋯
        ,
        
          g
          
            k
          
        
        (
        
          x
          
            t
          
          
            ∗
          
        
        )
        ,
        
          w
          
            1
            ,
            t
          
        
        ,
        ⋯
        ,
        
          w
          
            l
            ,
            t
          
        
        )
        .
      
    
    {\displaystyle \xi _{t}'=(g_{1}(x_{t}^{*}),\cdots ,g_{k}(x_{t}^{*}),w_{1,t},\cdots ,w_{l,t}).}

It turns out that all the expected values in this formula are estimable using the same deconvolution trick. In particular, for a generic observable wt (which could be 1, w1t, …, wℓ  t, or yt) and some function h (which could represent any gj or gigj) we have

E
        ⁡
        [
        
        
          w
          
            t
          
        
        h
        (
        
          x
          
            t
          
          
            ∗
          
        
        )
        
        ]
        =
        
          
            1
            
              2
              π
            
          
        
        
          ∫
          
            −
            ∞
          
          
            ∞
          
        
        
          φ
          
            h
          
        
        (
        −
        u
        )
        
          ψ
          
            w
          
        
        (
        u
        )
        d
        u
        ,
      
    
    {\displaystyle \operatorname {E} [\,w_{t}h(x_{t}^{*})\,]={\frac {1}{2\pi }}\int _{-\infty }^{\infty }\varphi _{h}(-u)\psi _{w}(u)du,}

where φh is the <a href="/facts/Fourier_transform/XXhKJtc8">Fourier transform</a> of h(x*), but using the same convention as for the <a href="/facts/Characteristic_function_(probability_theory)/rl3uAGKM">characteristic functions</a>,

φ
 
 h
 
 
 (
 u
 )
 =
 ∫
 
 e
 
 i
 u
 x
 
 
 h
 (
 x
 )
 d
 x
 
 
 {\displaystyle \varphi _{h}(u)=\int e^{iux}h(x)dx}
 
,
and

ψ
          
            w
          
        
        (
        u
        )
        =
        E
        ⁡
        [
        
        
          w
          
            t
          
        
        
          e
          
            i
            u
            
              x
              
                ∗
              
            
          
        
        
        ]
        =
        
          
            
              E
              ⁡
              [
              
                w
                
                  t
                
              
              
                e
                
                  i
                  u
                  
                    x
                    
                      1
                      t
                    
                  
                
              
              ]
            
            
              E
              ⁡
              [
              
                e
                
                  i
                  u
                  
                    x
                    
                      1
                      t
                    
                  
                
              
              ]
            
          
        
        exp
        ⁡
        
          ∫
          
            0
          
          
            u
          
        
        i
        
          
            
              E
              ⁡
              [
              
                x
                
                  2
                  t
                
              
              
                e
                
                  i
                  v
                  
                    x
                    
                      1
                      t
                    
                  
                
              
              ]
            
            
              E
              ⁡
              [
              
                e
                
                  i
                  v
                  
                    x
                    
                      1
                      t
                    
                  
                
              
              ]
            
          
        
        d
        v
      
    
    {\displaystyle \psi _{w}(u)=\operatorname {E} [\,w_{t}e^{iux^{*}}\,]={\frac {\operatorname {E} [w_{t}e^{iux_{1t}}]}{\operatorname {E} [e^{iux_{1t}}]}}\exp \int _{0}^{u}i{\frac {\operatorname {E} [x_{2t}e^{ivx_{1t}}]}{\operatorname {E} [e^{ivx_{1t}}]}}dv}

The resulting estimator 
 
 
 
 
 
 
 
 β
 ^
 
 
 
 
 
 
 {\displaystyle \scriptstyle {\hat {\beta }}}
 
 is consistent and asymptotically normal.</li><li>Schennach's estimator for a nonparametric model.<a class="footnote-ref" id="fnref:25" href="#fn:25">25</a> The standard <a href="/facts/Nadaraya%25E2%2580%2593Watson_estimator/7Txccjd4">Nadaraya–Watson estimator</a> for a nonparametric model takes form

g
              ^
            
          
        
        (
        x
        )
        =
        
          
            
              
                
                  
                    E
                    ^
                  
                
              
              [
              
              
                y
                
                  t
                
              
              
                K
                
                  h
                
              
              (
              
                x
                
                  t
                
                
                  ∗
                
              
              −
              x
              )
              
              ]
            
            
              
                
                  
                    E
                    ^
                  
                
              
              [
              
              
                K
                
                  h
                
              
              (
              
                x
                
                  t
                
                
                  ∗
                
              
              −
              x
              )
              
              ]
            
          
        
        ,
      
    
    {\displaystyle {\hat {g}}(x)={\frac {{\hat {\operatorname {E} }}[\,y_{t}K_{h}(x_{t}^{*}-x)\,]}{{\hat {\operatorname {E} }}[\,K_{h}(x_{t}^{*}-x)\,]}},}

for a suitable choice of the <a href="/facts/Kernel_(statistics)/otWLlBbA">kernel</a> K and the bandwidth h. Both expectations here can be estimated using the same technique as in the previous method.</li></ul>

<h2 id="further-reading">Further reading</h2>
<ul><li>Dougherty, Christopher (2011). <a href="https://books.google.com/books?id=UXucAQAAQBAJ&pg=PA300">"Stochastic Regressors and Measurement Errors"</a>. Introduction to Econometrics (Fourth ed.). Oxford University Press. pp. 300–330. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0-19-956708-9.</li>
<li><a href="/facts/Jan_Kmenta/RKLurjPH">Kmenta, Jan</a> (1986). <a href="https://books.google.com/books?id=Bxq7AAAAIAAJ&pg=PA346">"Estimation with Deficient Data"</a>. <a href="https://archive.org/details/elementsofeconom0003kmen/page/346">Elements of Econometrics</a> (Second ed.). New York: Macmillan. pp. <a href="https://archive.org/details/elementsofeconom0003kmen/page/346">346–391</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0-02-365070-3.</li>
<li><a href="/facts/Susanne_Schennach/2fY3Xm7M">Schennach, Susanne</a> (2013). "Measurement Error in Nonlinear Models – A Review". In Acemoglu, Daron; Arellano, Manuel; Dekel, Eddie (eds.). Advances in Economics and Econometrics. Cambridge University Press. pp. 296–337. <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1017%2FCBO9781139060035.009">10.1017/CBO9781139060035.009</a>. <a href="/facts/Hdl_(identifier)/rdebSxmC">hdl</a>:<a href="https://hdl.handle.net/10419%2F79526">10419/79526</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 9781107017214.</li></ul>
<h2 id="external-links">External links</h2>
<ul><li><a href="http://mathsdemo.cf.ac.uk/maths/resources/Gillard_Tech_Report.pdf">An Historical Overview of Linear Regression with Errors in both Variables</a>, J.W. Gillard 2006</li>
<li><a href="https://www.youtube.com/watch?v=brTmzHE9Gvw&index=11&list=PLD15D38DC7AA3B737&t=22m52s">Lecture on Econometrics (topic: Stochastic Regressors and Measurement Error)</a> on <a href="/facts/YouTube_video_(identifier)/db276jst">YouTube</a> by <a href="/facts/Mark_Thoma/ThV29HB0">Mark Thoma</a>.</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Griliches, Zvi; Ringstad, Vidar (1970). "Errors-in-the-variables bias in nonlinear contexts". Econometrica. 38 (2): 368–370. doi:10.2307/1913020. JSTOR 1913020. <a href="/wiki/Econometrica" target="_blank">/wiki/Econometrica</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Chesher, Andrew (1991). "The effect of measurement error". Biometrika. 78 (3): 451–462. doi:10.1093/biomet/78.3.451. JSTOR 2337015. <a href="/wiki/Biometrika" target="_blank">/wiki/Biometrika</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Carroll, Raymond J.; Ruppert, David; Stefanski, Leonard A.; Crainiceanu, Ciprian (2006). Measurement Error in Nonlinear Models: A Modern Perspective (Second ed.). CRC Press. ISBN 978-1-58488-633-4. <a href="978-1-58488-633-4" target="_blank">978-1-58488-633-4</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">Greene, William H. (2003). Econometric Analysis (5th ed.). New Jersey: Prentice Hall. Chapter 5.6.1. ISBN 978-0-13-066189-0. <a href="978-0-13-066189-0" target="_blank">978-0-13-066189-0</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">Wansbeek, T.; Meijer, E. (2000). "Measurement Error and Latent Variables". In Baltagi, B. H. (ed.). A Companion to Theoretical Econometrics. Blackwell. pp. 162–179. doi:10.1111/b.9781405106764.2003.00013.x (inactive 17 December 2024). ISBN 9781405106764.{{cite book}}: CS1 maint: DOI inactive as of December 2024 (link) <a href="9781405106764" target="_blank">9781405106764</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">Hausman, Jerry A. (2001). "Mismeasured variables in econometric analysis: problems from the right and problems from the left". Journal of Economic Perspectives. 15 (4): 57–67 [p. 58]. doi:10.1257/jep.15.4.57. JSTOR 2696516. <a href="https://doi.org/10.1257%2Fjep.15.4.57" target="_blank">https://doi.org/10.1257%2Fjep.15.4.57</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">Fuller, Wayne A. (1987). Measurement Error Models. John Wiley & Sons. p. 2. ISBN 978-0-471-86187-4. <a href="978-0-471-86187-4" target="_blank">978-0-471-86187-4</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">Hayashi, Fumio (2000). Econometrics. Princeton University Press. pp. 7–8. ISBN 978-1400823833. <a href="978-1400823833" target="_blank">978-1400823833</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">Koul, Hira; Song, Weixing (2008). "Regression model checking with Berkson measurement errors". Journal of Statistical Planning and Inference. 138 (6): 1615–1628. doi:10.1016/j.jspi.2007.05.048. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">Tofallis, C. (2023). Fitting an Equation to Data Impartially. Mathematics, 11(18), 3957. https://ssrn.com/abstract=4556739 https://doi.org/10.3390/math11183957 <a href="https://ssrn.com/abstract=4556739" target="_blank">https://ssrn.com/abstract=4556739</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">Reiersøl, Olav (1950). "Identifiability of a linear relation between variables which are subject to error". Econometrica. 18 (4): 375–389 [p. 383]. doi:10.2307/1907835. JSTOR 1907835. A somewhat more restrictive result was established earlier by Geary, R. C. (1942). "Inherent relations between random variables". Proceedings of the Royal Irish Academy. 47: 63–76. JSTOR 20488436. He showed that under the additional assumption that (ε, η) are jointly normal, the model is not identified if and only if x*s are normal. <a href="/wiki/Econometrica" target="_blank">/wiki/Econometrica</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">Fuller, Wayne A. (1987). "A Single Explanatory Variable". Measurement Error Models. John Wiley & Sons. pp. 1–99. ISBN 978-0-471-86187-4. <a href="978-0-471-86187-4" target="_blank">978-0-471-86187-4</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
<li id="fn:13">Pal, Manoranjan (1980). "Consistent moment estimators of regression coefficients in the presence of errors in variables". Journal of Econometrics. 14 (3): 349–364 (pp. 360–361). doi:10.1016/0304-4076(80)90032-9. <a href="/wiki/Journal_of_Econometrics" target="_blank">/wiki/Journal_of_Econometrics</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></li>
<li id="fn:14">Xu, Shaoji (2014-10-02). "A Property of Geometric Mean Regression". The American Statistician. 68 (4): 277–281. doi:10.1080/00031305.2014.962763. ISSN 0003-1305. <a href="https://dx.doi.org/10.1080/00031305.2014.962763" target="_blank">https://dx.doi.org/10.1080/00031305.2014.962763</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></li>
<li id="fn:15">Ben-Moshe, Dan (2020). "Identification of linear regressions with errors in all variables". Econometric Theory. 37 (4): 1–31. arXiv:1404.1473. doi:10.1017/S0266466620000250. S2CID 225653359. <a href="/wiki/Econometric_Theory" target="_blank">/wiki/Econometric_Theory</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></li>
<li id="fn:16">Dagenais, Marcel G.; Dagenais, Denyse L. (1997). "Higher moment estimators for linear regression models with errors in the variables". Journal of Econometrics. 76 (1–2): 193–221. CiteSeerX 10.1.1.669.8286. doi:10.1016/0304-4076(95)01789-5. In the earlier paper Pal (1980) considered a simpler case when all components in vector (ε, η) are independent and symmetrically distributed. <a href="/wiki/Journal_of_Econometrics" target="_blank">/wiki/Journal_of_Econometrics</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></li>
<li id="fn:17">Fuller, Wayne A. (1987). Measurement Error Models. John Wiley & Sons. p. 184. ISBN 978-0-471-86187-4. <a href="978-0-471-86187-4" target="_blank">978-0-471-86187-4</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></li>
<li id="fn:18">Erickson, Timothy; Whited, Toni M. (2002). "Two-step GMM estimation of the errors-in-variables model using high-order moments". Econometric Theory. 18 (3): 776–799. doi:10.1017/s0266466602183101. JSTOR 3533649. S2CID 14729228. <a href="/wiki/Econometric_Theory" target="_blank">/wiki/Econometric_Theory</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></li>
<li id="fn:19">Tofallis, C. (2023). Fitting an Equation to Data Impartially. Mathematics, 11(18), 3957. https://ssrn.com/abstract=4556739 https://doi.org/10.3390/math11183957 <a href="https://ssrn.com/abstract=4556739" target="_blank">https://ssrn.com/abstract=4556739</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></li>
<li id="fn:20">Schennach, S.; Hu, Y.; Lewbel, A. (2007). "Nonparametric identification of the classical errors-in-variables model without side information". Working Paper. <a href="/wiki/Susanne_Schennach" target="_blank">/wiki/Susanne_Schennach</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></li>
<li id="fn:21">Newey, Whitney K. (2001). "Flexible simulated moment estimation of nonlinear errors-in-variables model". Review of Economics and Statistics. 83 (4): 616–627. doi:10.1162/003465301753237704. hdl:1721.1/63613. JSTOR 3211757. S2CID 57566922. <a href="/wiki/Review_of_Economics_and_Statistics" target="_blank">/wiki/Review_of_Economics_and_Statistics</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></li>
<li id="fn:22">Li, Tong; Vuong, Quang (1998). "Nonparametric estimation of the measurement error model using multiple indicators". Journal of Multivariate Analysis. 65 (2): 139–165. doi:10.1006/jmva.1998.1741. <a href="https://doi.org/10.1006%2Fjmva.1998.1741" target="_blank">https://doi.org/10.1006%2Fjmva.1998.1741</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></li>
<li id="fn:23">Li, Tong (2002). "Robust and consistent estimation of nonlinear errors-in-variables models". Journal of Econometrics. 110 (1): 1–26. doi:10.1016/S0304-4076(02)00120-3. <a href="/wiki/Journal_of_Econometrics" target="_blank">/wiki/Journal_of_Econometrics</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></li>
<li id="fn:24">Schennach, Susanne M. (2004). "Estimation of nonlinear models with measurement error". Econometrica. 72 (1): 33–75. doi:10.1111/j.1468-0262.2004.00477.x. JSTOR 3598849. <a href="/wiki/Susanne_Schennach" target="_blank">/wiki/Susanne_Schennach</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></li>
<li id="fn:25">Schennach, Susanne M. (2004). "Nonparametric regression in the presence of measurement error". Econometric Theory. 20 (6): 1046–1093. doi:10.1017/S0266466604206028 (inactive 22 December 2024). S2CID 123036368.{{cite journal}}: CS1 maint: DOI inactive as of December 2024 (link) <a href="/wiki/Susanne_Schennach" target="_blank">/wiki/Susanne_Schennach</a> <a href="#fnref:25" class="footnote-back-ref">↩</a></li>
</ol>

Errors-in-variables model open-in-new

Errors-in-variables model