Generalized linear array model

<h2 id="overview">Overview</h2>
The generalized linear array model or GLAM was introduced in 2006.<a class="footnote-ref" id="fnref:1" href="#fn:1">1</a> Such models provide a structure and a computational procedure for fitting <a href="/facts/Generalized_linear_model/Sf1htm5W">generalized linear models</a> or GLMs whose model matrix can be written as a Kronecker product and whose data can be written as an array. In a large GLM, the GLAM approach gives very substantial savings in both storage and computational time over the usual GLM algorithm.
Suppose that the data 
 
 
 
 
 Y
 
 
 
 {\displaystyle \mathbf {Y} }
 
 is arranged in a 
 
 
 
 d
 
 
 {\displaystyle d}
 
-dimensional array with size 
 
 
 
 
 n
 
 1
 
 
 ×
 
 n
 
 2
 
 
 ×
 ⋯
 ×
 
 n
 
 d
 
 
 
 
 {\displaystyle n_{1}\times n_{2}\times \dots \times n_{d}}
 
; thus, the corresponding data vector 
 
 
 
 
 y
 
 =
 vec
 ⁡
 (
 
 Y
 
 )
 
 
 {\displaystyle \mathbf {y} =\operatorname {vec} (\mathbf {Y} )}
 
 has size 
 
 
 
 
 n
 
 1
 
 
 
 n
 
 2
 
 
 
 n
 
 3
 
 
 ⋯
 
 n
 
 d
 
 
 
 
 {\displaystyle n_{1}n_{2}n_{3}\cdots n_{d}}
 
. Suppose also that the <a href="/facts/Design_matrix/yRTdMhf4">design matrix</a> is of the form

X
        
        =
        
          
            X
          
          
            d
          
        
        ⊗
        
          
            X
          
          
            d
            −
            1
          
        
        ⊗
        ⋯
        ⊗
        
          
            X
          
          
            1
          
        
        .
      
    
    {\displaystyle \mathbf {X} =\mathbf {X} _{d}\otimes \mathbf {X} _{d-1}\otimes \dots \otimes \mathbf {X} _{1}.}

The standard analysis of a GLM with data vector 
 
 
 
 
 y
 
 
 
 {\displaystyle \mathbf {y} }
 
 and design matrix 
 
 
 
 
 X
 
 
 
 {\displaystyle \mathbf {X} }
 
 proceeds by repeated evaluation of the scoring algorithm

X
          
          ′
        
        
          
            
              
                
                  W
                
                ~
              
            
          
          
            δ
          
        
        
          X
        
        
          
            
              θ
              ^
            
          
        
        =
        
          
            X
          
          ′
        
        
          
            
              
                
                  W
                
                ~
              
            
          
          
            δ
          
        
        
          
            
              θ
              ~
            
          
        
        ,
      
    
    {\displaystyle \mathbf {X} '{\tilde {\mathbf {W} }}_{\delta }\mathbf {X} {\hat {\boldsymbol {\theta }}}=\mathbf {X} '{\tilde {\mathbf {W} }}_{\delta }{\tilde {\boldsymbol {\theta }}},}

where 
 
 
 
 
 
 
 θ
 ~
 
 
 
 
 
 {\displaystyle {\tilde {\boldsymbol {\theta }}}}
 
 represents the approximate solution of 
 
 
 
 
 θ
 
 
 
 {\displaystyle {\boldsymbol {\theta }}}
 
, and 
 
 
 
 
 
 
 θ
 ^
 
 
 
 
 
 {\displaystyle {\hat {\boldsymbol {\theta }}}}
 
 is the improved value of it; 
 
 
 
 
 
 W
 
 
 δ
 
 
 
 
 {\displaystyle \mathbf {W} _{\delta }}
 
 is the diagonal weight matrix with elements

w
          
            i
            i
          
          
            −
            1
          
        
        =
        
          
            (
            
              
                
                  ∂
                  
                    η
                    
                      i
                    
                  
                
                
                  ∂
                  
                    μ
                    
                      i
                    
                  
                
              
            
            )
          
          
            2
          
        
        
          v
          a
          r
        
        (
        
          y
          
            i
          
        
        )
        ,
      
    
    {\displaystyle w_{ii}^{-1}=\left({\frac {\partial \eta _{i}}{\partial \mu _{i}}}\right)^{2}\mathrm {var} (y_{i}),}

and

z
        
        =
        
          η
        
        +
        
          
            W
          
          
            δ
          
          
            −
            1
          
        
        (
        
          y
        
        −
        
          μ
        
        )
      
    
    {\displaystyle \mathbf {z} ={\boldsymbol {\eta }}+\mathbf {W} _{\delta }^{-1}(\mathbf {y} -{\boldsymbol {\mu }})}

is the working variable.
Computationally, GLAM provides array algorithms to calculate the linear predictor,

η
        
        =
        
          X
        
        
          θ
        
      
    
    {\displaystyle {\boldsymbol {\eta }}=\mathbf {X} {\boldsymbol {\theta }}}

and the weighted inner product

X
          
          ′
        
        
          
            
              
                
                  W
                
                ~
              
            
          
          
            δ
          
        
        
          X
        
      
    
    {\displaystyle \mathbf {X} '{\tilde {\mathbf {W} }}_{\delta }\mathbf {X} }

without evaluation of the model matrix 
 
 
 
 
 X
 
 .
 
 
 {\displaystyle \mathbf {X} .}

<h3>Example</h3>
In 2 dimensions, let 
 
 
 
 
 X
 
 =
 
 
 X
 
 
 2
 
 
 ⊗
 
 
 X
 
 
 1
 
 
 
 
 {\displaystyle \mathbf {X} =\mathbf {X} _{2}\otimes \mathbf {X} _{1}}
 
, then the linear predictor is written 
 
 
 
 
 
 X
 
 
 1
 
 
 
 Θ
 
 
 
 X
 
 
 2
 
 ′
 
 
 
 {\displaystyle \mathbf {X} _{1}{\boldsymbol {\Theta }}\mathbf {X} _{2}'}
 
 where 
 
 
 
 
 Θ
 
 
 
 {\displaystyle {\boldsymbol {\Theta }}}
 
 is the matrix of coefficients; the weighted inner product is obtained from 
 
 
 
 G
 (
 
 
 X
 
 
 1
 
 
 
 )
 ′
 
 
 W
 
 G
 (
 
 
 X
 
 
 2
 
 
 )
 
 
 {\displaystyle G(\mathbf {X} _{1})'\mathbf {W} G(\mathbf {X} _{2})}
 
 and 
 
 
 
 
 W
 
 
 
 {\displaystyle \mathbf {W} }
 
 is the matrix of weights; here 
 
 
 
 G
 (
 
 M
 
 )
 
 
 {\displaystyle G(\mathbf {M} )}
 
 is the row tensor function of the 
 
 
 
 r
 ×
 c
 
 
 {\displaystyle r\times c}
 
 matrix 
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
 given by<a class="footnote-ref" id="fnref:2" href="#fn:2">2</a>

G
        (
        
          M
        
        )
        =
        (
        
          M
        
        ⊗
        
          
            1
          
          ′
        
        )
        ∘
        (
        
          
            1
          
          ′
        
        ⊗
        
          M
        
        )
      
    
    {\displaystyle G(\mathbf {M} )=(\mathbf {M} \otimes \mathbf {1} ')\circ (\mathbf {1} '\otimes \mathbf {M} )}

where 
 
 
 
 ∘
 
 
 {\displaystyle \circ }
 
 means element by element multiplication and 
 
 
 
 
 1
 
 
 
 {\displaystyle \mathbf {1} }
 
 is a vector of 1's of length 
 
 
 
 c
 
 
 {\displaystyle c}
 
.
On the other hand, the row tensor function 
 
 
 
 G
 (
 
 M
 
 )
 
 
 {\displaystyle G(\mathbf {M} )}
 
 of the 
 
 
 
 r
 ×
 c
 
 
 {\displaystyle r\times c}
 
 matrix 
 
 
 
 
 M
 
 
 
 {\displaystyle \mathbf {M} }
 
 is the example of <a href="/facts/Khatri%25E2%2580%2593Rao_product/7IDTvWlg">Face-splitting product</a> of matrices, which was proposed by <a href="/facts/Vadym_Slyusar/hNudNFU4">Vadym Slyusar</a> in 1996:<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a><a class="footnote-ref" id="fnref:4" href="#fn:4">4</a><a class="footnote-ref" id="fnref:5" href="#fn:5">5</a><a class="footnote-ref" id="fnref:6" href="#fn:6">6</a>

M
        
        ∙
        
          M
        
        =
        
          (
          
            
              M
            
            ⊗
            
              
                1
              
              
                
                  T
                
              
            
          
          )
        
        ∘
        
          (
          
            
              
                1
              
              
                
                  T
                
              
            
            ⊗
            
              M
            
          
          )
        
        ,
      
    
    {\displaystyle \mathbf {M} \bullet \mathbf {M} =\left(\mathbf {M} \otimes \mathbf {1} ^{\textsf {T}}\right)\circ \left(\mathbf {1} ^{\textsf {T}}\otimes \mathbf {M} \right),}

where 
 
 
 
 ∙
 
 
 {\displaystyle \bullet }
 
 means <a href="/facts/Khatri%25E2%2580%2593Rao_product/7IDTvWlg">Face-splitting product</a>.
These low storage high speed formulae extend to 
 
 
 
 d
 
 
 {\displaystyle d}
 
-dimensions.

<h2 id="applications">Applications</h2>
GLAM is designed to be used in 
 
 
 
 d
 
 
 {\displaystyle d}
 
-dimensional smoothing problems where the data are arranged in an array and the smoothing matrix is constructed as a Kronecker product of 
 
 
 
 d
 
 
 {\displaystyle d}
 
 one-dimensional smoothing matrices.

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Currie, I. D.; Durban, M.; Eilers, P. H. C. (2006). "Generalized linear array models with applications to multidimensional smoothing". Journal of the Royal Statistical Society. 68 (2): 259–280. doi:10.1111/j.1467-9868.2006.00543.x. S2CID 10261944. <a href="/wiki/Journal_of_the_Royal_Statistical_Society" target="_blank">/wiki/Journal_of_the_Royal_Statistical_Society</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Currie, I. D.; Durban, M.; Eilers, P. H. C. (2006). "Generalized linear array models with applications to multidimensional smoothing". Journal of the Royal Statistical Society. 68 (2): 259–280. doi:10.1111/j.1467-9868.2006.00543.x. S2CID 10261944. <a href="/wiki/Journal_of_the_Royal_Statistical_Society" target="_blank">/wiki/Journal_of_the_Royal_Statistical_Society</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Slyusar, V. I. (December 27, 1996). "End products in matrices in radar applications" (PDF). Radioelectronics and Communications Systems. 41 (3): 50–53. <a href="http://slyusar.kiev.ua/en/IZV_1998_3.pdf" target="_blank">http://slyusar.kiev.ua/en/IZV_1998_3.pdf</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">Slyusar, V. I. (1997-05-20). "Analytical model of the digital antenna array on a basis of face-splitting matrix products" (PDF). Proc. ICATT-97, Kyiv: 108–109. <a href="http://slyusar.kiev.ua/ICATT97.pdf" target="_blank">http://slyusar.kiev.ua/ICATT97.pdf</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">Slyusar, V. I. (1997-09-15). "New operations of matrices product for applications of radars" (PDF). Proc. Direct and Inverse Problems of Electromagnetic and Acoustic Wave Theory (DIPED-97), Lviv.: 73–74. <a href="http://slyusar.kiev.ua/DIPED_1997.pdf" target="_blank">http://slyusar.kiev.ua/DIPED_1997.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">Slyusar, V. I. (March 13, 1998). "A Family of Face Products of Matrices and its Properties" (PDF). Cybernetics and Systems Analysis C/C of Kibernetika I Sistemnyi Analiz. 1999. 35 (3): 379–384. doi:10.1007/BF02733426. S2CID 119661450. <a href="http://slyusar.kiev.ua/FACE.pdf" target="_blank">http://slyusar.kiev.ua/FACE.pdf</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
</ol>

Generalized linear array model open-in-new

Generalized linear array model