Berndt–Hall–Hall–Hausman algorithm

<h2 id="usage">Usage</h2>
<p>If a <a href="/facts/Nonlinear/4aYItALC">nonlinear</a> model is fitted to the <a href="/facts/Data/AUBFN97o">data</a> one often needs to estimate <a href="/facts/Coefficient/5Hwq2Wuq">coefficients</a> through <a href="/facts/Optimization_(mathematics)/oRn8Iv5I">optimization</a>. A number of optimisation algorithms have the following general structure. Suppose that the function to be optimized is <i>Q</i>(<i>β</i>). Then the algorithms are iterative, defining a sequence of approximations, <i>βk</i> given by
</p>

β
          
            k
            +
            1
          
        
        =
        
          β
          
            k
          
        
        −
        
          λ
          
            k
          
        
        
          A
          
            k
          
        
        
          
            
              ∂
              Q
            
            
              ∂
              β
            
          
        
        (
        
          β
          
            k
          
        
        )
        ,
      
    
    {\displaystyle \beta _{k+1}=\beta _{k}-\lambda _{k}A_{k}{\frac {\partial Q}{\partial \beta }}(\beta _{k}),}
  
,
<p>where 
  
    
      
        
          β
          
            k
          
        
      
    
    {\displaystyle \beta _{k}}
  
 is the parameter estimate at step k, and 
  
    
      
        
          λ
          
            k
          
        
      
    
    {\displaystyle \lambda _{k}}
  
 is a parameter (called step size) which partly determines the particular algorithm. For the BHHH algorithm <i>λk</i> is determined by calculations within a given iterative step, involving a line-search until a point <i>β</i><i>k</i>+1 is found satisfying certain criteria. In addition, for the BHHH algorithm, <i>Q</i> has the form
</p>

Q
        =
        
          ∑
          
            i
            =
            1
          
          
            N
          
        
        
          Q
          
            i
          
        
      
    
    {\displaystyle Q=\sum _{i=1}^{N}Q_{i}}

<p>and <i>A</i> is calculated using
</p>

A
          
            k
          
        
        =
        
          
            [
            
              
                ∑
                
                  i
                  =
                  1
                
                
                  N
                
              
              
                
                  
                    ∂
                    ln
                    ⁡
                    
                      Q
                      
                        i
                      
                    
                  
                  
                    ∂
                    β
                  
                
              
              (
              
                β
                
                  k
                
              
              )
              
                
                  
                    ∂
                    ln
                    ⁡
                    
                      Q
                      
                        i
                      
                    
                  
                  
                    ∂
                    β
                  
                
              
              (
              
                β
                
                  k
                
              
              
                )
                ′
              
            
            ]
          
          
            −
            1
          
        
        .
      
    
    {\displaystyle A_{k}=\left[\sum _{i=1}^{N}{\frac {\partial \ln Q_{i}}{\partial \beta }}(\beta _{k}){\frac {\partial \ln Q_{i}}{\partial \beta }}(\beta _{k})'\right]^{-1}.}

<p>In other cases, e.g. <a href="/facts/Newton%25E2%2580%2593Raphson/VlRhI2FL">Newton–Raphson</a>, 
  
    
      
        
          A
          
            k
          
        
      
    
    {\displaystyle A_{k}}
  
 can have other forms. The BHHH algorithm has the advantage that, if certain conditions apply, convergence of the iterative procedure is guaranteed.
</p>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Davidon%25E2%2580%2593Fletcher%25E2%2580%2593Powell_algorithm/5oYlloCx">Davidon–Fletcher–Powell (DFP) algorithm</a></li>
<li><a href="/facts/Broyden%25E2%2580%2593Fletcher%25E2%2580%2593Goldfarb%25E2%2580%2593Shanno_algorithm/aA6KXo6h">Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm</a></li></ul>

<h2 id="further-reading">Further reading</h2>
<ul><li>V. Martin, S. Hurn, and D. Harris, <i>Econometric Modelling with Time Series</i>, Chapter 3 'Numerical Estimation Methods'. Cambridge University Press, 2015.</li>
<li><a href="/facts/Takeshi_Amemiya/CVByAXGq">Amemiya, Takeshi</a> (1985). <a href="https://archive.org/details/advancedeconomet00amem/page/137"><i>Advanced Econometrics</i></a>. Cambridge: Harvard University Press. pp. <a href="https://archive.org/details/advancedeconomet00amem/page/137">137–138</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-674-00560-0.</li>
<li>Gill, P.; Murray, W.; Wright, M. (1981). <i>Practical Optimization</i>. London: Harcourt Brace.</li>
<li>Gourieroux, Christian; Monfort, Alain (1995). <a href="https://books.google.com/books?id=gqI-pAP2JZ8C&pg=PA452">"Gradient Methods and ML Estimation"</a>. <i>Statistics and Econometric Models</i>. New York: Cambridge University Press. pp. 452–458. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-521-40551-3.</li>
<li>Harvey, A. C. (1990). <i>The Econometric Analysis of Time Series</i> (Second ed.). Cambridge: MIT Press. pp. 137–138. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-262-08189-X.</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>Henningsen, A.; Toomet, O. (2011). "maxLik: A package for maximum likelihood estimation in R". Computational Statistics. 26 (3): 443–458 [p. 450]. doi:10.1007/s00180-010-0217-1. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Berndt, E.; Hall, B.; Hall, R.; Hausman, J. (1974). "Estimation and Inference in Nonlinear Structural Models" (PDF). Annals of Economic and Social Measurement. 3 (4): 653–665. <a href="https://www.nber.org/chapters/c10206.pdf" target="_blank">https://www.nber.org/chapters/c10206.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
</ol>

Berndt–Hall–Hall–Hausman algorithm open-in-new

Berndt–Hall–Hall–Hausman algorithm