Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Two-step M-estimator
open-in-new
<h2 id="description">Description</h2> <p>The class of two-step M-estimators includes <a href="/facts/Heckman_correction/DwjBk5c6">Heckman's sample selection estimator</a>,<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a> weighted <a href="/facts/Non-linear_least_squares/3K5Nmz0S">non-linear least squares</a>, and <a href="/facts/Ordinary_least_squares/q7H9k5vM">ordinary least squares</a> with <a href="/facts/Generated_regressors/2ePQdvhs">generated regressors</a>.<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a> </p><p>To fix ideas, let { W i } i = 1 n ⊆ R d {\displaystyle \{W_{i}\}_{i=1}^{n}\subseteq R^{d}} be an <a href="/facts/I.i.d./othIRaWt">i.i.d.</a> sample. Θ {\displaystyle \Theta } and Γ {\displaystyle \Gamma } are subsets of Euclidean spaces R p {\displaystyle R^{p}} and R q {\displaystyle R^{q}} , respectively. Given a function m ( ; ; ; ) : R d × Θ × Γ → R {\displaystyle m(;;;):R^{d}\times \Theta \times \Gamma \rightarrow R} , two-step M-estimator θ ^ {\displaystyle {\hat {\theta }}} is defined as: </p> θ ^ := arg max θ ∈ Θ 1 n ∑ i m ( W i , θ , γ ^ ) {\displaystyle {\hat {\theta }}:=\arg \max _{\theta \in \Theta }{\frac {1}{n}}\sum _{i}m{\bigl (}W_{i},\theta ,{\hat {\gamma }}{\bigr )}} <p>where γ ^ {\displaystyle {\hat {\gamma }}} is an M-estimate of a <a href="/facts/Nuisance_parameter/acJWVqxj">nuisance parameter</a> that needs to be calculated in the first step. </p><p><a href="/facts/Consistent_estimator/HAKDWYEM">Consistency</a> of two-step M-estimators can be verified by checking consistency conditions for usual M-estimators, although some modification might be necessary. In practice, the important condition to check is the <a href="/facts/Identifiability/YfE1ydRa">identification condition</a>.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> If γ ^ → γ ∗ , {\displaystyle {\hat {\gamma }}\rightarrow \gamma ^{*},} where γ ∗ {\displaystyle \gamma ^{*}} is a non-random vector, then the identification condition is that E [ m ( W 1 , θ , γ ∗ ) ] {\displaystyle E[m(W_{1},\theta ,\gamma ^{*})]} has a unique maximizer over Θ {\displaystyle \Theta } . </p> <h3>Asymptotic distribution</h3> <p>Under regularity conditions, two-step M-estimators have <a href="/facts/Asymptotic_normality/dN7jVHqm">asymptotic normality</a>. An important point to note is that the <a href="/facts/Asymptotic_variance/dN7jVHqm">asymptotic variance</a> of a two-step M-estimator is generally not the same as that of the usual M-estimator in which the first step estimation is not necessary.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> This fact is intuitive because γ ^ {\displaystyle {\hat {\gamma }}} is a random object and its variability should influence the estimation of Θ {\displaystyle \Theta } . However, there exists a special case in which the asymptotic variance of two-step M-estimator takes the form as if there were no first-step estimation procedure. Such special case occurs if: </p> E ∂ ∂ θ ∂ γ m ( W 1 , θ 0 , γ ∗ ) = 0 {\displaystyle E{\frac {\partial }{\partial \theta \partial \gamma }}m(W_{1},\theta _{0},\gamma ^{*})=0} <p>where θ 0 {\displaystyle \theta _{0}} is the true value of θ {\displaystyle \theta } and γ ∗ {\displaystyle \gamma ^{*}} is the probability limit of γ ^ {\displaystyle {\hat {\gamma }}} .<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> To interpret this condition, first note that under regularity conditions, E ∂ ∂ θ m ( W 1 , θ 0 , γ ∗ ) = 0 {\displaystyle E{\frac {\partial }{\partial \theta }}m(W_{1},\theta _{0},\gamma ^{*})=0} since θ 0 {\displaystyle \theta _{0}} is the maximizer of E [ m ( W 1 , θ , γ ∗ ) ] {\displaystyle E[m(W_{1},\theta ,\gamma ^{*})]} . So the condition above implies that small perturbation in γ has no impact on the <a href="/facts/First-order_condition/5BmmVBAS">first-order condition</a>. Thus, in large sample, variability of γ ^ {\displaystyle {\hat {\gamma }}} does not affect the argmax of the objective function, which explains invariant property of asymptotic variance. Of course, this result is valid only as the sample size tends to infinity, so the finite-sample property could be quite different. </p> <h2 id="involving-mle">Involving MLE</h2> <p>When the first step is a <a href="/facts/Maximum_likelihood_estimator/0Yq2dpQD">maximum likelihood estimator</a>, under some assumptions, two-step M-estimator is more <a href="/facts/Asymptotic_efficiency/wTmgEqte">asymptotically efficient</a> (i.e. has smaller asymptotic variance) than M-estimator with known first-step parameter. <a href="/facts/Consistent_estimator/HAKDWYEM">Consistency</a> and <a href="/facts/Asymptotic_normality/dN7jVHqm">asymptotic normality</a> of the estimator follows from the general result on two-step M-estimators.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> </p><p>Let {Vi,Wi,Zi}ni=1 be a random sample and the second-step M-estimator θ ^ {\displaystyle {\widehat {\theta }}} is the following: </p> θ ^ := a r g max θ ∈ Θ ∑ i = 1 n m ( v i , w i , z i : θ , γ ^ ) {\displaystyle {\widehat {\theta }}:={\underset {\theta \in \Theta }{\operatorname {arg\max } }}\sum _{i=1}^{n}m(v_{i},w_{i},z_{i}:\theta ,{\widehat {\gamma }})} <p>where γ ^ {\displaystyle {\widehat {\gamma }}} is the parameter estimated by maximum likelihood in the first step. For the MLE, </p> γ ^ := a r g max γ ∈ Γ ∑ i = 1 n log f ( v i t : z i , γ ) {\displaystyle {\widehat {\gamma }}:={\underset {\gamma \in \Gamma }{\operatorname {arg\max } }}\sum _{i=1}^{n}\log f(v_{it}:z_{i},\gamma )} <p>where <i>f</i> is the conditional density of <i>V</i> given <i>Z</i>. Now, suppose that given <i>Z</i>, <i>V</i> is conditionally independent of <i>W</i>. This is called the <a href="/facts/Conditional_independence/udUXFaMw">conditional independence</a> assumption or selection on observables.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a><a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> Intuitively, this condition means that Z is a good predictor of V so that once conditioned on <i>Z, V</i> has no systematic dependence on <i>W</i>. Under the conditional independence assumption, the <a href="/facts/Asymptotic_variance/dN7jVHqm">asymptotic variance</a> of the two-step estimator is: </p> E [ ∇ θ s ( θ 0 , γ 0 ) ] − 1 E [ g ( θ 0 , γ 0 ) g ( θ 0 , γ 0 ) T ] E [ ∇ θ s ( θ 0 , γ 0 ) ] − 1 {\displaystyle \mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}\mathrm {E} [g(\theta _{0},\gamma _{0})g(\theta _{0},\gamma _{0})^{\mathrm {T} }]\mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}} <p>where </p> g ( θ , γ ) := s ( θ , γ ) − E [ ∇ γ s ( θ , γ ) ] E [ d ( γ ) d ( γ ) T ] − 1 d ( γ ) s ( θ , γ ) := ∇ θ m ( V , W , Z : θ , γ ) d ( γ ) := ∇ γ log f ( V : Z , γ ) {\displaystyle {\begin{aligned}g(\theta ,\gamma )&:=s(\theta ,\gamma )-\mathrm {E} [\nabla _{\gamma }s(\theta ,\gamma )]\mathrm {E} [d(\gamma )d(\gamma )^{\mathrm {T} }]^{-1}d(\gamma )\\s(\theta ,\gamma )&:=\nabla _{\theta }m(V,W,Z:\theta ,\gamma )\\d(\gamma )&:=\nabla _{\gamma }\log f(V:Z,\gamma )\end{aligned}}} <p>and ∇ represents partial derivative with respect to a row vector. In the case where <i>γ0</i> is known, the asymptotic variance is </p> E [ ∇ θ s ( θ 0 , γ 0 ) ] − 1 E [ s ( θ 0 , γ 0 ) s ( θ 0 , γ 0 ) T ] E [ ∇ θ s ( θ 0 , γ 0 ) ] − 1 {\displaystyle \mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}\mathrm {E} [s(\theta _{0},\gamma _{0})s(\theta _{0},\gamma _{0})^{\mathrm {T} }]\mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}} <p>and therefore, unless E [ ∇ γ s ( θ , γ ) ] = 0 {\displaystyle \mathrm {E} [\nabla _{\gamma }s(\theta ,\gamma )]=0} , the two-step M-estimator is more efficient than the usual M-estimator. This fact suggests that even when <i>γ0</i> is known a priori, there is an efficiency gain by estimating <i>γ</i> by MLE. An application of this result can be found, for example, in treatment effect estimation.<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> </p> <h2 id="examples">Examples</h2> <ul><li><a href="/facts/Generated_regressor/2ePQdvhs">Generated regressor</a></li> <li><a href="/facts/Heckman_correction/DwjBk5c6">Heckman correction</a></li> <li><a href="/facts/Generalized_least_squares/NI2GL3RK">Feasible generalized least squares</a></li> <li><a href="/facts/Generalized_method_of_moments/mNc3410V">Two-step feasible generalized method of moments</a></li></ul> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Adaptive_estimator/7kLhrpDb">Adaptive estimator</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Heckman, J.J., The Common Structure of Statistical Models of Truncation, Sample Selection, and Limited Dependent Variables and a Simple Estimator for Such Models, Annals of Economic and Social Measurement, 5,475-492. <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Wooldridge, J.M., Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, Mass. <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Wooldridge, J.M., Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, Mass. <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Newey, K.W. and D. McFadden, Large Sample Estimation and Hypothesis Testing, in R. Engel and D. McFadden, eds., Handbook of Econometrics, Vol.4, Amsterdam: North-Holland. <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Newey, K.W. and D. McFadden, Large Sample Estimation and Hypothesis Testing, in R. Engel and D. McFadden, eds., Handbook of Econometrics, Vol.4, Amsterdam: North-Holland. <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Wooldridge, J.M., Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, Mass. <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Wooldridge, J.M., Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, Mass. <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Heckman, J.J., and R. Robb, 1985, Alternative Methods for Evaluating the Impact of Interventions: An Overview, Journal of Econometrics, 30, 239-267. <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Wooldridge, J.M., Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, Mass. <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> </ol>