Moment matrix

<h2 id="application-in-regression">Application in regression</h2>
<p>A multiple <a href="/facts/Linear_regression/5n998IhK">linear regression</a> model can be written as
</p>

y
        =
        
          β
          
            0
          
        
        +
        
          β
          
            1
          
        
        
          x
          
            1
          
        
        +
        
          β
          
            2
          
        
        
          x
          
            2
          
        
        +
        …
        
          β
          
            k
          
        
        
          x
          
            k
          
        
        +
        u
      
    
    {\displaystyle y=\beta _{0}+\beta _{1}x_{1}+\beta _{2}x_{2}+\dots \beta _{k}x_{k}+u}

<p>where 
  
    
      
        y
      
    
    {\displaystyle y}
  
 is the dependent variable, 
  
    
      
        
          x
          
            1
          
        
        ,
        
          x
          
            2
          
        
        …
        ,
        
          x
          
            k
          
        
      
    
    {\displaystyle x_{1},x_{2}\dots ,x_{k}}
  
 are the independent variables, 
  
    
      
        u
      
    
    {\displaystyle u}
  
 is the error, and 
  
    
      
        
          β
          
            0
          
        
        ,
        
          β
          
            1
          
        
        …
        ,
        
          β
          
            k
          
        
      
    
    {\displaystyle \beta _{0},\beta _{1}\dots ,\beta _{k}}
  
 are unknown coefficients to be estimated. Given observations 
  
    
      
        
          
            {
            
              
                y
                
                  i
                
              
              ,
              
                x
                
                  i
                  1
                
              
              ,
              
                x
                
                  i
                  2
                
              
              ,
              …
              ,
              
                x
                
                  i
                  k
                
              
            
            }
          
          
            i
            =
            1
          
          
            n
          
        
      
    
    {\displaystyle \left\{y_{i},x_{i1},x_{i2},\dots ,x_{ik}\right\}_{i=1}^{n}}
  
, we have a system of 
  
    
      
        n
      
    
    {\displaystyle n}
  
 linear equations that can be expressed in matrix notation.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a>
</p>

[
            
              
                
                  
                    y
                    
                      1
                    
                  
                
              
              
                
                  
                    y
                    
                      2
                    
                  
                
              
              
                
                  ⋮
                
              
              
                
                  
                    y
                    
                      n
                    
                  
                
              
            
            ]
          
        
        =
        
          
            [
            
              
                
                  1
                
                
                  
                    x
                    
                      11
                    
                  
                
                
                  
                    x
                    
                      12
                    
                  
                
                
                  …
                
                
                  
                    x
                    
                      1
                      k
                    
                  
                
              
              
                
                  1
                
                
                  
                    x
                    
                      21
                    
                  
                
                
                  
                    x
                    
                      22
                    
                  
                
                
                  …
                
                
                  
                    x
                    
                      2
                      k
                    
                  
                
              
              
                
                  ⋮
                
                
                  ⋮
                
                
                  ⋮
                
                
                  ⋱
                
                
                  ⋮
                
              
              
                
                  1
                
                
                  
                    x
                    
                      n
                      1
                    
                  
                
                
                  
                    x
                    
                      n
                      2
                    
                  
                
                
                  …
                
                
                  
                    x
                    
                      n
                      k
                    
                  
                
              
            
            ]
          
        
        
          
            [
            
              
                
                  
                    β
                    
                      0
                    
                  
                
              
              
                
                  
                    β
                    
                      1
                    
                  
                
              
              
                
                  ⋮
                
              
              
                
                  
                    β
                    
                      k
                    
                  
                
              
            
            ]
          
        
        +
        
          
            [
            
              
                
                  
                    u
                    
                      1
                    
                  
                
              
              
                
                  
                    u
                    
                      2
                    
                  
                
              
              
                
                  ⋮
                
              
              
                
                  
                    u
                    
                      n
                    
                  
                
              
            
            ]
          
        
      
    
    {\displaystyle {\begin{bmatrix}y_{1}\\y_{2}\\\vdots \\y_{n}\end{bmatrix}}={\begin{bmatrix}1&x_{11}&x_{12}&\dots &x_{1k}\\1&x_{21}&x_{22}&\dots &x_{2k}\\\vdots &\vdots &\vdots &\ddots &\vdots \\1&x_{n1}&x_{n2}&\dots &x_{nk}\\\end{bmatrix}}{\begin{bmatrix}\beta _{0}\\\beta _{1}\\\vdots \\\beta _{k}\end{bmatrix}}+{\begin{bmatrix}u_{1}\\u_{2}\\\vdots \\u_{n}\end{bmatrix}}}

y
        
        =
        
          X
        
        
          β
        
        +
        
          u
        
      
    
    {\displaystyle \mathbf {y} =\mathbf {X} {\boldsymbol {\beta }}+\mathbf {u} }

<p>where 
  
    
      
        
          y
        
      
    
    {\displaystyle \mathbf {y} }
  
 and 
  
    
      
        
          u
        
      
    
    {\displaystyle \mathbf {u} }
  
 are each a vector of dimension 
  
    
      
        n
        ×
        1
      
    
    {\displaystyle n\times 1}
  
, 
  
    
      
        
          X
        
      
    
    {\displaystyle \mathbf {X} }
  
 is the <a href="/facts/Design_matrix/yRTdMhf4">design matrix</a> of order 
  
    
      
        N
        ×
        (
        k
        +
        1
        )
      
    
    {\displaystyle N\times (k+1)}
  
, and 
  
    
      
        
          β
        
      
    
    {\displaystyle {\boldsymbol {\beta }}}
  
 is a vector of dimension 
  
    
      
        (
        k
        +
        1
        )
        ×
        1
      
    
    {\displaystyle (k+1)\times 1}
  
. Under the <a href="/facts/Gauss%E2%80%93Markov_theorem/j85SGhrw">Gauss–Markov assumptions</a>, the best linear unbiased estimator of 
  
    
      
        
          β
        
      
    
    {\displaystyle {\boldsymbol {\beta }}}
  
 is the linear <a href="/facts/Least_squares/50jIwbxC">least squares</a> estimator 
  
    
      
        
          b
        
        =
        
          
            (
            
              
                
                  X
                
                
                  
                    T
                  
                
              
              
                X
              
            
            )
          
          
            −
            1
          
        
        
          
            X
          
          
            
              T
            
          
        
        
          y
        
      
    
    {\displaystyle \mathbf {b} =\left(\mathbf {X} ^{\mathsf {T}}\mathbf {X} \right)^{-1}\mathbf {X} ^{\mathsf {T}}\mathbf {y} }
  
, involving the two moment matrices 
  
    
      
        
          
            X
          
          
            
              T
            
          
        
        
          X
        
      
    
    {\displaystyle \mathbf {X} ^{\mathsf {T}}\mathbf {X} }
  
 and 
  
    
      
        
          
            X
          
          
            
              T
            
          
        
        
          y
        
      
    
    {\displaystyle \mathbf {X} ^{\mathsf {T}}\mathbf {y} }
  
 defined as
</p>

X
          
          
            
              T
            
          
        
        
          X
        
        =
        
          
            [
            
              
                
                  n
                
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                  
                
                
                  ∑
                  
                    x
                    
                      i
                      2
                    
                  
                
                
                  …
                
                
                  ∑
                  
                    x
                    
                      i
                      k
                    
                  
                
              
              
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                  
                
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                    
                      2
                    
                  
                
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                  
                  
                    x
                    
                      i
                      2
                    
                  
                
                
                  …
                
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                  
                  
                    x
                    
                      i
                      k
                    
                  
                
              
              
                
                  ∑
                  
                    x
                    
                      i
                      2
                    
                  
                
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                  
                  
                    x
                    
                      i
                      2
                    
                  
                
                
                  ∑
                  
                    x
                    
                      i
                      2
                    
                    
                      2
                    
                  
                
                
                  …
                
                
                  ∑
                  
                    x
                    
                      i
                      2
                    
                  
                  
                    x
                    
                      i
                      k
                    
                  
                
              
              
                
                  ⋮
                
                
                  ⋮
                
                
                  ⋮
                
                
                  ⋱
                
                
                  ⋮
                
              
              
                
                  ∑
                  
                    x
                    
                      i
                      k
                    
                  
                
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                  
                  
                    x
                    
                      i
                      k
                    
                  
                
                
                  ∑
                  
                    x
                    
                      i
                      2
                    
                  
                  
                    x
                    
                      i
                      k
                    
                  
                
                
                  …
                
                
                  ∑
                  
                    x
                    
                      i
                      k
                    
                    
                      2
                    
                  
                
              
            
            ]
          
        
      
    
    {\displaystyle \mathbf {X} ^{\mathsf {T}}\mathbf {X} ={\begin{bmatrix}n&\sum x_{i1}&\sum x_{i2}&\dots &\sum x_{ik}\\\sum x_{i1}&\sum x_{i1}^{2}&\sum x_{i1}x_{i2}&\dots &\sum x_{i1}x_{ik}\\\sum x_{i2}&\sum x_{i1}x_{i2}&\sum x_{i2}^{2}&\dots &\sum x_{i2}x_{ik}\\\vdots &\vdots &\vdots &\ddots &\vdots \\\sum x_{ik}&\sum x_{i1}x_{ik}&\sum x_{i2}x_{ik}&\dots &\sum x_{ik}^{2}\end{bmatrix}}}

X
          
          
            
              T
            
          
        
        
          y
        
        =
        
          
            [
            
              
                
                  ∑
                  
                    y
                    
                      i
                    
                  
                
              
              
                
                  ∑
                  
                    x
                    
                      i
                      1
                    
                  
                  
                    y
                    
                      i
                    
                  
                
              
              
                
                  ⋮
                
              
              
                
                  ∑
                  
                    x
                    
                      i
                      k
                    
                  
                  
                    y
                    
                      i
                    
                  
                
              
            
            ]
          
        
      
    
    {\displaystyle \mathbf {X} ^{\mathsf {T}}\mathbf {y} ={\begin{bmatrix}\sum y_{i}\\\sum x_{i1}y_{i}\\\vdots \\\sum x_{ik}y_{i}\end{bmatrix}}}

<p>where 
  
    
      
        
          
            X
          
          
            
              T
            
          
        
        
          X
        
      
    
    {\displaystyle \mathbf {X} ^{\mathsf {T}}\mathbf {X} }
  
 is a square <a href="/facts/Normal_matrix/WVKxrEK9">normal matrix</a> of dimension 
  
    
      
        (
        k
        +
        1
        )
        ×
        (
        k
        +
        1
        )
      
    
    {\displaystyle (k+1)\times (k+1)}
  
, and 
  
    
      
        
          
            X
          
          
            
              T
            
          
        
        
          y
        
      
    
    {\displaystyle \mathbf {X} ^{\mathsf {T}}\mathbf {y} }
  
 is a vector of dimension 
  
    
      
        (
        k
        +
        1
        )
        ×
        1
      
    
    {\displaystyle (k+1)\times 1}
  
.
</p>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Design_matrix/yRTdMhf4">Design matrix</a></li>
<li><a href="/facts/Gramian_matrix/SGb2VGTU">Gramian matrix</a></li>
<li><a href="/facts/Projection_matrix/5bLbzjy9">Projection matrix</a></li></ul>

<h2 id="external-links">External links</h2>
<ul><li><a href="https://www.encyclopediaofmath.org/index.php?title=Moment_matrix">"Moment matrix"</a>, <i><a href="/facts/Encyclopedia_of_Mathematics/WC6mGtPm">Encyclopedia of Mathematics</a></i>, <a href="/facts/European_Mathematical_Society/B3h7b672">EMS Press</a>, 2001 [1994]</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>Lasserre, Jean-Bernard, 1953- (2010). Moments, positive polynomials and their applications. World Scientific (Firm). London: Imperial College Press. ISBN 978-1-84816-446-8. OCLC 624365972.{{cite book}}:  CS1 maint: multiple names: authors list (link) CS1 maint: numeric names: authors list (link) <a href="978-1-84816-446-8" target="_blank">978-1-84816-446-8</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Goldberger, Arthur S. (1964). "Classical Linear Regression". Econometric Theory. New York: John Wiley & Sons. pp. 156–212. ISBN 0-471-31101-4. <a href="0-471-31101-4" target="_blank">0-471-31101-4</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>Huang, David S. (1970). Regression and Econometric Methods. New York: John Wiley & Sons. pp. 52–65. ISBN 0-471-41754-8. <a href="0-471-41754-8" target="_blank">0-471-41754-8</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
</ol>

Moment matrix open-in-new

Moment matrix