Hadamard product (matrices)

<h2 id="definition">Definition</h2>
For two matrices A and B of the same dimension m × n, the Hadamard product 
 
 
 
 A
 ⊙
 B
 
 
 {\displaystyle A\odot B}
 
 (sometimes 
 
 
 
 A
 ∘
 B
 
 
 {\displaystyle A\circ B}
 
<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a><a class="footnote-ref" id="fnref:5" href="#fn:5">5</a><a class="footnote-ref" id="fnref:6" href="#fn:6">6</a>[user-generated source]) is a matrix of the same dimension as the operands, with elements given by<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a>

(
        A
        ⊙
        B
        
          )
          
            i
            j
          
        
        =
        (
        A
        
          )
          
            i
            j
          
        
        (
        B
        
          )
          
            i
            j
          
        
        .
      
    
    {\displaystyle (A\odot B)_{ij}=(A)_{ij}(B)_{ij}.}

For matrices of different dimensions (m × n and p × q, where m ≠ p or n ≠ q), the Hadamard product is undefined.
An example of the Hadamard product for two arbitrary 2 × 3 matrices:

[
            
              
                
                  2
                
                
                  3
                
                
                  1
                
              
              
                
                  0
                
                
                  8
                
                
                  −
                  2
                
              
            
            ]
          
        
        ∘
        
          
            [
            
              
                
                  3
                
                
                  1
                
                
                  4
                
              
              
                
                  7
                
                
                  9
                
                
                  5
                
              
            
            ]
          
        
        =
        
          
            [
            
              
                
                  2
                  ×
                  3
                
                
                  3
                  ×
                  1
                
                
                  1
                  ×
                  4
                
              
              
                
                  0
                  ×
                  7
                
                
                  8
                  ×
                  9
                
                
                  −
                  2
                  ×
                  5
                
              
            
            ]
          
        
        =
        
          
            [
            
              
                
                  6
                
                
                  3
                
                
                  4
                
              
              
                
                  0
                
                
                  72
                
                
                  −
                  10
                
              
            
            ]
          
        
        .
      
    
    {\displaystyle {\begin{bmatrix}2&3&1\\0&8&-2\end{bmatrix}}\circ {\begin{bmatrix}3&1&4\\7&9&5\end{bmatrix}}={\begin{bmatrix}2\times 3&3\times 1&1\times 4\\0\times 7&8\times 9&-2\times 5\end{bmatrix}}={\begin{bmatrix}6&3&4\\0&72&-10\end{bmatrix}}.}

<h2 id="properties">Properties</h2>
<ul><li>The Hadamard product is <a href="/facts/Commutative/WaYxJLxd">commutative</a> (when working with a commutative ring), <a href="/facts/Associative/EQX8lV6r">associative</a> and <a href="/facts/Distributive_property/0LNBrVJl">distributive</a> over addition. That is, if A, B, and C are matrices of the same size, and k is a scalar: 
 
 
 
 
 
 
 
 A
 ⊙
 B
 
 
 
 =
 B
 ⊙
 A
 ,
 
 
 
 
 A
 ⊙
 (
 B
 ⊙
 C
 )
 
 
 
 =
 (
 A
 ⊙
 B
 )
 ⊙
 C
 ,
 
 
 
 
 A
 ⊙
 (
 B
 +
 C
 )
 
 
 
 =
 A
 ⊙
 B
 +
 A
 ⊙
 C
 ,
 
 
 
 
 (
 k
 A
 )
 ⊙
 B
 
 
 
 =
 A
 ⊙
 (
 k
 B
 )
 =
 k
 (
 A
 ⊙
 B
 )
 ,
 
 
 
 
 A
 ⊙
 0
 
 
 
 =
 0
 ⊙
 A
 =
 0.
 
 
 
 
 
 
 {\displaystyle {\begin{aligned}A\odot B&=B\odot A,\\A\odot (B\odot C)&=(A\odot B)\odot C,\\A\odot (B+C)&=A\odot B+A\odot C,\\(kA)\odot B&=A\odot (kB)=k(A\odot B),\\A\odot 0&=0\odot A=0.\end{aligned}}}
 
</li>
<li>The identity matrix under Hadamard multiplication of two m × n matrices is an <a href="/facts/Matrix_of_ones/wvacr9uD">m × n matrix where all elements are equal to 1</a>. This is different from the <a href="/facts/Identity_matrix/Glbcf6ol">identity matrix</a> under regular matrix multiplication, where only the elements of the main diagonal are equal to 1. Furthermore, a matrix has an inverse under Hadamard multiplication if and only if all of the elements are invertible, or equivalently over a field, if and only if none of the elements are equal to zero.<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a></li>
<li>For vectors x and y and corresponding diagonal matrices Dx and Dy with these vectors as their main diagonals, the following identity holds:<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a>: 479  
 
 
 
 
 
 x
 
 
 ∗
 
 
 (
 A
 ⊙
 B
 )
 
 y
 
 =
 tr
 ⁡
 
 (
 
 
 
 D
 
 
 
 x
 
 
 
 ∗
 
 
 A
 
 
 D
 
 
 
 y
 
 
 
 
 
 B
 
 
 
 T
 
 
 
 
 )
 
 ,
 
 
 {\displaystyle \mathbf {x} ^{*}(A\odot B)\mathbf {y} =\operatorname {tr} \left({D}_{\mathbf {x} }^{*}A{D}_{\mathbf {y} }{B}^{\mathsf {T}}\right),}
 
 where x* denotes the <a href="/facts/Conjugate_transpose/7eBWtqVG">conjugate transpose</a> of x. In particular, using vectors of ones, this shows that the sum of all elements in the Hadamard product is the <a href="/facts/Trace_(linear_algebra)/dchFFvHt">trace</a> of ABT where superscript T denotes the <a href="/facts/Matrix_transpose/8wmsagGS">matrix transpose</a>, that is, 
 
 
 
 tr
 ⁡
 
 (
 
 A
 
 B
 
 
 T
 
 
 
 
 )
 
 =
 
 
 1
 
 
 
 T
 
 
 
 
 (
 
 A
 ⊙
 B
 
 )
 
 
 1
 
 
 
 {\displaystyle \operatorname {tr} \left(AB^{\mathsf {T}}\right)=\mathbf {1} ^{\mathsf {T}}\left(A\odot B\right)\mathbf {1} }
 
. A related result for square A and B, is that the row-sums of their Hadamard product are the diagonal elements of ABT:<a class="footnote-ref" id="fnref:10" href="#fn:10">10</a> 
 
 
 
 
 ∑
 
 i
 
 
 (
 A
 ⊙
 B
 
 )
 
 i
 j
 
 
 =
 
 
 (
 
 
 B
 
 
 T
 
 
 
 A
 
 )
 
 
 j
 j
 
 
 =
 
 
 (
 
 A
 
 B
 
 
 T
 
 
 
 
 )
 
 
 i
 i
 
 
 .
 
 
 {\displaystyle \sum _{i}(A\odot B)_{ij}=\left(B^{\mathsf {T}}A\right)_{jj}=\left(AB^{\mathsf {T}}\right)_{ii}.}
 
 Similarly, 
 
 
 
 (
 
 y
 
 
 
 x
 
 
 ∗
 
 
 )
 ⊙
 A
 =
 
 D
 
 
 y
 
 
 
 A
 
 D
 
 
 x
 
 
 
 ∗
 
 
 .
 
 
 {\displaystyle (\mathbf {y} \mathbf {x} ^{*})\odot A=D_{\mathbf {y} }AD_{\mathbf {x} }^{*}.}
 
 Furthermore, a Hadamard matrix–vector product can be expressed as 
 
 
 
 (
 A
 ⊙
 B
 )
 
 y
 
 =
 diag
 ⁡
 (
 A
 
 D
 
 
 y
 
 
 
 
 B
 
 
 T
 
 
 
 )
 ,
 
 
 {\displaystyle (A\odot B)\mathbf {y} =\operatorname {diag} (AD_{\mathbf {y} }B^{\mathsf {T}}),}
 
 where 
 
 
 
 diag
 ⁡
 (
 M
 )
 
 
 {\displaystyle \operatorname {diag} (M)}
 
 is the vector formed from the diagonals of matrix M.</li>
<li>The Hadamard product is a principal <a href="/facts/Submatrix/qa8FI4ko">submatrix</a> of the <a href="/facts/Kronecker_product/gABGx6I6">Kronecker product</a>.<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a><a class="footnote-ref" id="fnref:12" href="#fn:12">12</a><a class="footnote-ref" id="fnref:13" href="#fn:13">13</a></li>
<li>The Hadamard product satisfies the rank inequality 
 
 
 
 rank
 ⁡
 (
 A
 ⊙
 B
 )
 ≤
 rank
 ⁡
 (
 A
 )
 rank
 ⁡
 (
 B
 )
 .
 
 
 {\displaystyle \operatorname {rank} (A\odot B)\leq \operatorname {rank} (A)\operatorname {rank} (B).}
 
</li>
<li>If A and B are <a href="/facts/Positive-definite_matrix/Hbr6AuPS">positive-definite matrices</a>, then the following inequality involving the Hadamard product holds:<a class="footnote-ref" id="fnref:14" href="#fn:14">14</a> 
 
 
 
 
 ∏
 
 i
 =
 k
 
 
 n
 
 
 
 λ
 
 i
 
 
 (
 A
 ⊙
 B
 )
 ≥
 
 ∏
 
 i
 =
 k
 
 
 n
 
 
 
 λ
 
 i
 
 
 (
 A
 B
 )
 ,
 
 k
 =
 1
 ,
 …
 ,
 n
 ,
 
 
 {\displaystyle \prod _{i=k}^{n}\lambda _{i}(A\odot B)\geq \prod _{i=k}^{n}\lambda _{i}(AB),\quad k=1,\ldots ,n,}
 
 where λi(A) is the ith largest <a href="/facts/Eigenvalue/8TjEoT8u">eigenvalue</a> of A.</li>
<li>If D and E are <a href="/facts/Diagonal_matrices/tjIAHrE6">diagonal matrices</a>, then<a class="footnote-ref" id="fnref:15" href="#fn:15">15</a> 
 
 
 
 
 
 
 
 D
 (
 A
 ⊙
 B
 )
 E
 
 
 
 =
 (
 D
 A
 E
 )
 ⊙
 B
 =
 (
 D
 A
 )
 ⊙
 (
 B
 E
 )
 
 
 
 
 
 
 =
 (
 A
 E
 )
 ⊙
 (
 D
 B
 )
 =
 A
 ⊙
 (
 D
 B
 E
 )
 .
 
 
 
 
 
 
 {\displaystyle {\begin{aligned}D(A\odot B)E&=(DAE)\odot B=(DA)\odot (BE)\\&=(AE)\odot (DB)=A\odot (DBE).\end{aligned}}}
 
</li>
<li>The Hadamard product of two vectors 
 
 
 
 
 a
 
 
 
 {\displaystyle \mathbf {a} }
 
 and 
 
 
 
 
 b
 
 
 
 {\displaystyle \mathbf {b} }
 
 is the same as matrix multiplication of the corresponding <a href="/facts/Diagonal_matrix/tjIAHrE6">diagonal matrix</a> of one vector by the other vector: 
 
 
 
 
 a
 
 ⊙
 
 b
 
 =
 
 D
 
 
 a
 
 
 
 
 b
 
 =
 
 D
 
 
 b
 
 
 
 
 a
 
 .
 
 
 {\displaystyle \mathbf {a} \odot \mathbf {b} =D_{\mathbf {a} }\mathbf {b} =D_{\mathbf {b} }\mathbf {a} .}
 
</li>
<li>The <a href="/facts/Diagonal_matrix/tjIAHrE6">
 
 
 
 diag
 
 
 {\displaystyle \operatorname {diag} }
 
 operator</a> transforming a vector to a diagonal matrix may be expressed using the Hadamard product as 
 
 
 
 diag
 ⁡
 (
 
 a
 
 )
 =
 (
 
 a
 
 
 
 1
 
 
 T
 
 
 )
 ⊙
 I
 ,
 
 
 {\displaystyle \operatorname {diag} (\mathbf {a} )=(\mathbf {a} \mathbf {1} ^{T})\odot I,}
 
 where 
 
 
 
 
 1
 
 
 
 {\displaystyle \mathbf {1} }
 
 is a constant vector with elements 
 
 
 
 1
 
 
 {\displaystyle 1}
 
, and 
 
 
 
 I
 
 
 {\displaystyle I}
 
 is the <a href="/facts/Identity_matrix/Glbcf6ol">identity matrix</a>.</li></ul>
<h2 id="the-mixed-product-property">The mixed-product property</h2>
The Hadamard product obeys certain relationships with other matrix product operators.

<ul><li>If 
 
 
 
 ⊗
 
 
 {\displaystyle \otimes }
 
 is the <a href="/facts/Kronecker_product/gABGx6I6">Kronecker product</a>, assuming 
 
 
 
 A
 
 
 {\displaystyle A}
 
 has the same dimensions as 
 
 
 
 C
 
 
 {\displaystyle C}
 
 and 
 
 
 
 B
 
 
 {\displaystyle B}
 
 as 
 
 
 
 D
 
 
 {\displaystyle D}
 
, then 
 
 
 
 (
 A
 ⊗
 B
 )
 ⊙
 (
 C
 ⊗
 D
 )
 =
 (
 A
 ⊙
 C
 )
 ⊗
 (
 B
 ⊙
 D
 )
 .
 
 
 {\displaystyle (A\otimes B)\odot (C\otimes D)=(A\odot C)\otimes (B\odot D).}
 
</li></ul>
<ul><li>If 
 
 
 
 ∙
 
 
 {\displaystyle \bullet }
 
 is the <a href="/facts/Khatri%25E2%2580%2593Rao_product/7IDTvWlg">face-splitting product</a>, then<a class="footnote-ref" id="fnref:16" href="#fn:16">16</a> 
 
 
 
 (
 A
 ∙
 B
 )
 ⊙
 (
 C
 ∙
 D
 )
 =
 (
 A
 ⊙
 C
 )
 ∙
 (
 B
 ⊙
 D
 )
 .
 
 
 {\displaystyle (A\bullet B)\odot (C\bullet D)=(A\odot C)\bullet (B\odot D).}
 
</li></ul>
<ul><li>If 
 
 
 
 ∗
 
 
 {\displaystyle \ast }
 
 is the column-wise <a href="/facts/Khatri%25E2%2580%2593Rao_product/7IDTvWlg">Khatri–Rao product</a>, then 
 
 
 
 (
 A
 ∙
 B
 )
 (
 C
 ∗
 D
 )
 =
 (
 A
 C
 )
 ⊙
 (
 B
 D
 )
 .
 
 
 {\displaystyle (A\bullet B)(C\ast D)=(AC)\odot (BD).}
 
</li></ul>
<h2 id="schur-product-theorem">Schur product theorem</h2>
Main article: <a href="/facts/Schur_product_theorem/nCcLcM4e">Schur product theorem</a>
The Hadamard product of two <a href="/facts/Positive-definite_matrix/Hbr6AuPS">positive-semidefinite matrices</a> is positive-semidefinite.<a class="footnote-ref" id="fnref:17" href="#fn:17">17</a><a class="footnote-ref" id="fnref:18" href="#fn:18">18</a> This is known as the Schur product theorem,<a class="footnote-ref" id="fnref:19" href="#fn:19">19</a> after Russian mathematician <a href="/facts/Issai_Schur/rqn0wQaR">Issai Schur</a>. For two positive-semidefinite matrices A and B, it is also known that the <a href="/facts/Determinant/eGYqJ2TF">determinant</a> of their Hadamard product is greater than or equal to the product of their respective determinants:<a class="footnote-ref" id="fnref:20" href="#fn:20">20</a>
 
 
 
 det
 (
 
 A
 
 ⊙
 
 B
 
 )
 ≥
 det
 (
 
 A
 
 )
 det
 (
 
 B
 
 )
 .
 
 
 {\displaystyle \det({A}\odot {B})\geq \det({A})\det({B}).}

<h2 id="analogous-operations">Analogous operations</h2>
Other Hadamard operations are also seen in the mathematical literature,<a class="footnote-ref" id="fnref:21" href="#fn:21">21</a> namely the Hadamard root and Hadamard power (which are in effect the same thing because of fractional indices), defined for a matrix such that:
For

B
                
              
              
                
                =
                
                  
                    A
                  
                  
                    ∘
                    2
                  
                
              
            
            
              
                
                  B
                  
                    i
                    j
                  
                
              
              
                
                =
                
                  
                    
                      A
                      
                        i
                        j
                      
                    
                  
                  
                    2
                  
                
              
            
          
        
      
    
    {\displaystyle {\begin{aligned}{B}&={A}^{\circ 2}\\B_{ij}&={A_{ij}}^{2}\end{aligned}}}

and for

B
                
              
              
                
                =
                
                  
                    A
                  
                  
                    ∘
                    
                      
                        1
                        2
                      
                    
                  
                
              
            
            
              
                
                  B
                  
                    i
                    j
                  
                
              
              
                
                =
                
                  
                    
                      A
                      
                        i
                        j
                      
                    
                  
                  
                    
                      1
                      2
                    
                  
                
              
            
          
        
      
    
    {\displaystyle {\begin{aligned}{B}&={A}^{\circ {\frac {1}{2}}}\\B_{ij}&={A_{ij}}^{\frac {1}{2}}\end{aligned}}}

The Hadamard inverse reads:<a class="footnote-ref" id="fnref:22" href="#fn:22">22</a>

B
                
              
              
                
                =
                
                  
                    A
                  
                  
                    ∘
                    −
                    1
                  
                
              
            
            
              
                
                  B
                  
                    i
                    j
                  
                
              
              
                
                =
                
                  
                    
                      A
                      
                        i
                        j
                      
                    
                  
                  
                    −
                    1
                  
                
              
            
          
        
      
    
    {\displaystyle {\begin{aligned}{B}&={A}^{\circ -1}\\B_{ij}&={A_{ij}}^{-1}\end{aligned}}}

A Hadamard division is defined as:<a class="footnote-ref" id="fnref:23" href="#fn:23">23</a><a class="footnote-ref" id="fnref:24" href="#fn:24">24</a>

 
 
 
 
 
 
 
 
 C
 
 
 
 
 =
 
 A
 
 ⊘
 
 B
 
 
 
 
 
 
 C
 
 i
 j
 
 
 
 
 
 =
 
 
 
 A
 
 i
 j
 
 
 
 B
 
 i
 j
 
 
 
 
 
 
 
 
 
 
 {\displaystyle {\begin{aligned}{C}&={A}\oslash {B}\\C_{ij}&={\frac {A_{ij}}{B_{ij}}}\end{aligned}}}

<h2 id="in-programming-languages">In programming languages</h2>
Most scientific or numerical <a href="/facts/Programming_languages/cWTYbgWa">programming languages</a> include the Hadamard product, under various names.
In <a href="/facts/MATLAB/qPjLISCk">MATLAB</a>, the Hadamard product is expressed as "dot multiply": a .* b, or the function call: times(a, b).<a class="footnote-ref" id="fnref:25" href="#fn:25">25</a> It also has analogous dot operators which include, for example, the operators a .^ b and a ./ b.<a class="footnote-ref" id="fnref:26" href="#fn:26">26</a> Because of this mechanism, it is possible to reserve * and ^ for matrix multiplication and matrix exponentials, respectively.
The programming language <a href="/facts/Julia_(programming_language)/AoB0PJ9C">Julia</a> has similar syntax as MATLAB, where Hadamard multiplication is called broadcast multiplication and also denoted with a .* b, and other operators are analogously defined element-wise, for example Hadamard powers use a .^ b.<a class="footnote-ref" id="fnref:27" href="#fn:27">27</a> But unlike MATLAB, in Julia this "dot" syntax is generalized with a generic broadcasting operator . which can apply any function element-wise. This includes both binary operators (such as the aforementioned multiplication and exponentiation, as well as any other binary operator such as the Kronecker product), and also unary operators such as ! and √. Thus, any function in <a href="/facts/Prefix_notation/dUzpf2Ys">prefix notation</a> f can be applied as f.(x).<a class="footnote-ref" id="fnref:28" href="#fn:28">28</a> 
<a href="/facts/Python_(programming_language)/YbuGqofa">Python</a> does not have built-in array support, leading to inconsistent/conflicting notations. The <a href="/facts/NumPy/T6FnhWWD">NumPy</a> numerical library interprets a*b or a.multiply(b) as the Hadamard product, and uses a@b or a.matmul(b) for the matrix product. With the <a href="/facts/SymPy/vWSsBCG0">SymPy</a> symbolic library, multiplication of array objects as either a*b or a@b will produce the matrix product. The Hadamard product can be obtained with the method call a.multiply_elementwise(b).<a class="footnote-ref" id="fnref:29" href="#fn:29">29</a> Some Python packages include support for Hadamard powers using methods like np.power(a, b), or the <a href="/facts/Pandas_(software)/VKkTK8RL">Pandas</a> method a.pow(b).
In C++, the <a href="/facts/Eigen_(C%252B%252B_library)/js8e67k3">Eigen</a> library provides a cwiseProduct member function for the Matrix class (a.cwiseProduct(b)), while the <a href="/facts/Armadillo_(C%252B%252B_library)/ANyYTA2o">Armadillo</a> library uses the operator % to make compact expressions (a % b; a * b is a matrix product).
In <a href="/facts/GAUSS_(software)/hrlzrTMG">GAUSS</a>, and <a href="/facts/HP_Prime/czJje3Ne">HP Prime</a>, the operation is known as array multiplication.
In <a href="/facts/Fortran/m1KqjcMU">Fortran</a>, <a href="/facts/R_(programming_language)/LSrkr8K8">R</a>, <a href="/facts/APL_(programming_language)/E2sb98gC">APL</a>, <a href="/facts/J_(programming_language)/EdIVMBj7">J</a> and <a href="/facts/Wolfram_Language/MDxl50Yq">Wolfram Language</a> (<a href="/facts/Mathematica/dRwoGFG2">Mathematica</a>), the multiplication operator * or × apply the Hadamard product, whereas the matrix product is written using matmul, %*%, +.×, +/ .* and ., respectively.
The R package <a href="https://cran.r-project.org/web/packages/matrixcalc/matrixcalc.pdf">matrixcalc</a> introduces the function hadamard.prod() for Hadamard Product of numeric matrices or vectors.<a class="footnote-ref" id="fnref:30" href="#fn:30">30</a>

<h2 id="applications">Applications</h2>
The Hadamard product appears in <a href="/facts/Lossy_compression/jHW7DDv3">lossy compression</a> algorithms such as <a href="/facts/JPEG/HQ8QS3uB">JPEG</a>. The decoding step involves an entry-for-entry product, in other words the Hadamard product.
In <a href="/facts/Image_processing/zenalNCf">image processing</a>, the Hadamard operator can be used for enhancing, suppressing or masking image regions. One matrix represents the original image, the other acts as weight or masking matrix.
It is used in the <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a> literature, for example, to describe the architecture of recurrent neural networks as <a href="/facts/Gated_recurrent_unit/afweFlCM">GRUs</a> or <a href="/facts/Long_short-term_memory/NBBVEoft">LSTMs</a>.<a class="footnote-ref" id="fnref:31" href="#fn:31">31</a>
It is also used to study the statistical properties of random vectors and matrices.
<a class="footnote-ref" id="fnref:32" href="#fn:32">32</a><a class="footnote-ref" id="fnref:33" href="#fn:33">33</a>

<h2 id="the-penetrating-face-product">The penetrating face product</h2>

According to the definition of <a href="/facts/Vadym_Slyusar/hNudNFU4">V. Slyusar</a> the penetrating face product of the p×g matrix 
 
 
 
 
 A
 
 
 
 {\displaystyle {A}}
 
 and n-dimensional matrix 
 
 
 
 
 B
 
 
 
 {\displaystyle {B}}
 
 (n > 1) with p×g blocks (
 
 
 
 
 B
 
 =
 [
 
 B
 
 n
 
 
 ]
 
 
 {\displaystyle {B}=[B_{n}]}
 
) is a matrix of size 
 
 
 
 
 B
 
 
 
 {\displaystyle {B}}
 
 of the form:<a class="footnote-ref" id="fnref:34" href="#fn:34">34</a>

A
        
        [
        ∘
        ]
        
          B
        
        =
        
          [
          
            
              
                
                  
                    A
                  
                  ∘
                  
                    
                      B
                    
                    
                      1
                    
                  
                
                
                  
                    A
                  
                  ∘
                  
                    
                      B
                    
                    
                      2
                    
                  
                
                
                  ⋯
                
                
                  
                    A
                  
                  ∘
                  
                    
                      B
                    
                    
                      n
                    
                  
                
              
            
          
          ]
        
        .
      
    
    {\displaystyle {A}[\circ ]{B}=\left[{\begin{array}{c | c | c | c }{A}\circ {B}_{1}&{A}\circ {B}_{2}&\cdots &{A}\circ {B}_{n}\end{array}}\right].}

<h3>Example</h3>
If

A
        
        =
        
          
            [
            
              
                
                  1
                
                
                  2
                
                
                  3
                
              
              
                
                  4
                
                
                  5
                
                
                  6
                
              
              
                
                  7
                
                
                  8
                
                
                  9
                
              
            
            ]
          
        
        ,
        
        
          B
        
        =
        
          [
          
            
              
                
                  
                    
                      B
                    
                    
                      1
                    
                  
                
                
                  
                    
                      B
                    
                    
                      2
                    
                  
                
                
                  
                    
                      B
                    
                    
                      3
                    
                  
                
              
            
          
          ]
        
        =
        
          [
          
            
              
                
                  1
                
                
                  4
                
                
                  7
                
                
                  2
                
                
                  8
                
                
                  14
                
                
                  3
                
                
                  12
                
                
                  21
                
              
              
                
                  8
                
                
                  20
                
                
                  5
                
                
                  10
                
                
                  25
                
                
                  40
                
                
                  12
                
                
                  30
                
                
                  6
                
              
              
                
                  2
                
                
                  8
                
                
                  3
                
                
                  2
                
                
                  4
                
                
                  2
                
                
                  7
                
                
                  3
                
                
                  9
                
              
            
          
          ]
        
      
    
    {\displaystyle {A}={\begin{bmatrix}1&2&3\\4&5&6\\7&8&9\end{bmatrix}},\quad {B}=\left[{\begin{array}{c | c | c }{B}_{1}&{B}_{2}&{B}_{3}\end{array}}\right]=\left[{\begin{array}{c c c | c c c | c c c }1&4&7&2&8&14&3&12&21\\8&20&5&10&25&40&12&30&6\\2&8&3&2&4&2&7&3&9\end{array}}\right]}

then

 
 
 
 
 A
 
 [
 ∘
 ]
 
 B
 
 =
 
 [
 
 
 
 
 1
 
 
 8
 
 
 21
 
 
 2
 
 
 16
 
 
 42
 
 
 3
 
 
 24
 
 
 63
 
 
 
 
 32
 
 
 100
 
 
 30
 
 
 40
 
 
 125
 
 
 240
 
 
 48
 
 
 150
 
 
 36
 
 
 
 
 14
 
 
 64
 
 
 27
 
 
 14
 
 
 32
 
 
 18
 
 
 49
 
 
 24
 
 
 81
 
 
 
 
 ]
 
 .
 
 
 {\displaystyle {A}[\circ ]{B}=\left[{\begin{array}{c c c | c c c | c c c }1&8&21&2&16&42&3&24&63\\32&100&30&40&125&240&48&150&36\\14&64&27&14&32&18&49&24&81\end{array}}\right].}

<h3>Main properties</h3>

A
 
 [
 ∘
 ]
 
 B
 
 =
 
 B
 
 [
 ∘
 ]
 
 A
 
 ;
 
 
 {\displaystyle {A}[\circ ]{B}={B}[\circ ]{A};}
 
<a class="footnote-ref" id="fnref:35" href="#fn:35">35</a>

M
        
        ∙
        
          M
        
        =
        
          M
        
        [
        ∘
        ]
        
          (
          
            
              M
            
            ⊗
            
              
                1
              
              
                
                  T
                
              
            
          
          )
        
        ,
      
    
    {\displaystyle {M}\bullet {M}={M}[\circ ]\left({M}\otimes \mathbf {1} ^{\textsf {T}}\right),}

where 
 
 
 
 ∙
 
 
 {\displaystyle \bullet }
 
 denotes the <a href="/facts/Face-splitting_product/7IDTvWlg">face-splitting product</a> of matrices,

c
 
 ∙
 
 M
 
 =
 
 c
 
 [
 ∘
 ]
 
 M
 
 ,
 
 
 {\displaystyle \mathbf {c} \bullet {M}=\mathbf {c} [\circ ]{M},}
 
 where 
 
 
 
 
 c
 
 
 
 {\displaystyle \mathbf {c} }
 
 is a vector.
<h3>Applications</h3>
The penetrating face product is used in the <a href="/facts/Tensor/pNjFo5tV">tensor</a>-matrix theory of <a href="/facts/Digital_antenna_array/567FpdPN">digital antenna arrays</a>.<a class="footnote-ref" id="fnref:36" href="#fn:36">36</a> This operation can also be used in <a href="/facts/Artificial_neural_network/6V1jMlkx">artificial neural network</a> models, specifically convolutional layers.<a class="footnote-ref" id="fnref:37" href="#fn:37">37</a>

<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Frobenius_inner_product/E8DB5N9E">Frobenius inner product</a></li>
<li><a href="/facts/Pointwise_product/781oq7Xu">Pointwise product</a></li>
<li><a href="/facts/Kronecker_product/gABGx6I6">Kronecker product</a></li>
<li><a href="/facts/Khatri%25E2%2580%2593Rao_product/7IDTvWlg">Khatri–Rao product</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Horn, Roger A.; Johnson, Charles R. (2012). Matrix analysis. Cambridge University Press. <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Davis, Chandler (1962). "The norm of the Schur product operation". Numerische Mathematik. 4 (1): 343–44. doi:10.1007/bf01386329. S2CID 121027182. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Million, Elizabeth (April 12, 2007). "The Hadamard Product" (PDF). buzzard.ups.edu. Retrieved September 6, 2020. <a href="http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf" target="_blank">http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">"Hadamard product - Machine Learning Glossary". machinelearning.wtf. <a href="https://machinelearning.wtf/terms/hadamard-product/" target="_blank">https://machinelearning.wtf/terms/hadamard-product/</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">"linear algebra - What does a dot in a circle mean?". Mathematics Stack Exchange. <a href="https://math.stackexchange.com/q/815315" target="_blank">https://math.stackexchange.com/q/815315</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">"Element-wise (or pointwise) operations notation?". Mathematics Stack Exchange. <a href="https://math.stackexchange.com/a/601545/688715" target="_blank">https://math.stackexchange.com/a/601545/688715</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">Million, Elizabeth (April 12, 2007). "The Hadamard Product" (PDF). buzzard.ups.edu. Retrieved September 6, 2020. <a href="http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf" target="_blank">http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">Million, Elizabeth. "The Hadamard Product" (PDF). Retrieved 2 January 2012. <a href="http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf" target="_blank">http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">Horn, Roger A.; Johnson, Charles R. (2012). Matrix analysis. Cambridge University Press. <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">Styan, George P. H. (1973), "Hadamard Products and Multivariate Statistical Analysis", Linear Algebra and Its Applications, 6: 217–240, doi:10.1016/0024-3795(73)90023-2, hdl:10338.dmlcz/102190 <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">Liu, Shuangzhe; Trenkler, Götz (2008). "Hadamard, Khatri-Rao, Kronecker and other matrix products". International Journal of Information and Systems Sciences. 4 (1): 160–177. <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">Liu, Shuangzhe; Leiva, Víctor; Zhuang, Dan; Ma, Tiefeng; Figueroa-Zúñiga, Jorge I. (2022). "Matrix differential calculus with applications in the multivariate linear model and its diagnostics". Journal of Multivariate Analysis. 188: 104849. doi:10.1016/j.jmva.2021.104849. S2CID 239598156. <a href="https://doi.org/10.1016%2Fj.jmva.2021.104849" target="_blank">https://doi.org/10.1016%2Fj.jmva.2021.104849</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
<li id="fn:13">Liu, Shuangzhe; Trenkler, Götz; Kollo, Tõnu; von Rosen, Dietrich; Baksalary, Oskar Maria (2023). "Professor Heinz Neudecker and matrix differential calculus". Statistical Papers. 65 (4): 2605–2639. doi:10.1007/s00362-023-01499-w. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></li>
<li id="fn:14">Hiai, Fumio; Lin, Minghua (February 2017). "On an eigenvalue inequality involving the Hadamard product". Linear Algebra and Its Applications. 515: 313–320. doi:10.1016/j.laa.2016.11.017. <a href="https://doi.org/10.1016%2Fj.laa.2016.11.017" target="_blank">https://doi.org/10.1016%2Fj.laa.2016.11.017</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></li>
<li id="fn:15">"Project" (PDF). buzzard.ups.edu. 2007. Retrieved 2019-12-18. <a href="http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf" target="_blank">http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></li>
<li id="fn:16">Slyusar, V. I. (1998). "End products in matrices in radar applications" (PDF). Radioelectronics and Communications Systems. 41 (3): 50–53. <a href="http://slyusar.kiev.ua/en/IZV_1998_3.pdf" target="_blank">http://slyusar.kiev.ua/en/IZV_1998_3.pdf</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></li>
<li id="fn:17">Million, Elizabeth (April 12, 2007). "The Hadamard Product" (PDF). buzzard.ups.edu. Retrieved September 6, 2020. <a href="http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf" target="_blank">http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></li>
<li id="fn:18">Styan, George P. H. (1973), "Hadamard Products and Multivariate Statistical Analysis", Linear Algebra and Its Applications, 6: 217–240, doi:10.1016/0024-3795(73)90023-2, hdl:10338.dmlcz/102190 <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></li>
<li id="fn:19">Million, Elizabeth. "The Hadamard Product" (PDF). Retrieved 2 January 2012. <a href="http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf" target="_blank">http://buzzard.ups.edu/courses/2007spring/projects/million-paper.pdf</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></li>
<li id="fn:20">Styan, George P. H. (1973), "Hadamard Products and Multivariate Statistical Analysis", Linear Algebra and Its Applications, 6: 217–240, doi:10.1016/0024-3795(73)90023-2, hdl:10338.dmlcz/102190 <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></li>
<li id="fn:21">Reams, Robert (1999). "Hadamard inverses, square roots and products of almost semidefinite matrices". Linear Algebra and Its Applications. 288: 35–43. doi:10.1016/S0024-3795(98)10162-3. <a href="https://doi.org/10.1016%2FS0024-3795%2898%2910162-3" target="_blank">https://doi.org/10.1016%2FS0024-3795%2898%2910162-3</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></li>
<li id="fn:22">Reams, Robert (1999). "Hadamard inverses, square roots and products of almost semidefinite matrices". Linear Algebra and Its Applications. 288: 35–43. doi:10.1016/S0024-3795(98)10162-3. <a href="https://doi.org/10.1016%2FS0024-3795%2898%2910162-3" target="_blank">https://doi.org/10.1016%2FS0024-3795%2898%2910162-3</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></li>
<li id="fn:23">Wetzstein, Gordon; Lanman, Douglas; Hirsch, Matthew; Raskar, Ramesh. "Supplementary Material: Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting" (PDF). MIT Media Lab. <a href="http://web.media.mit.edu/~gordonw/TensorDisplays/TensorDisplays-Supplement.pdf" target="_blank">http://web.media.mit.edu/~gordonw/TensorDisplays/TensorDisplays-Supplement.pdf</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></li>
<li id="fn:24">Cyganek, Boguslaw (2013). Object Detection and Recognition in Digital Images: Theory and Practice. John Wiley & Sons. p. 109. ISBN 9781118618363. <a href="9781118618363" target="_blank">9781118618363</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></li>
<li id="fn:25">"MATLAB times function". <a href="https://www.mathworks.com/help/matlab/ref/times.html" target="_blank">https://www.mathworks.com/help/matlab/ref/times.html</a> <a href="#fnref:25" class="footnote-back-ref">↩</a></li>
<li id="fn:26">"Array vs. Matrix Operations". <a href="https://www.mathworks.com/help/matlab/matlab_prog/array-vs-matrix-operations.html" target="_blank">https://www.mathworks.com/help/matlab/matlab_prog/array-vs-matrix-operations.html</a> <a href="#fnref:26" class="footnote-back-ref">↩</a></li>
<li id="fn:27">"Vectorized "dot" operators". Retrieved 31 January 2024. <a href="https://docs.julialang.org/en/v1/manual/mathematical-operations/#man-dot-operators" target="_blank">https://docs.julialang.org/en/v1/manual/mathematical-operations/#man-dot-operators</a> <a href="#fnref:27" class="footnote-back-ref">↩</a></li>
<li id="fn:28">"Dot Syntax for Vectorizing Functions". Retrieved 31 January 2024. <a href="https://docs.julialang.org/en/v1/manual/functions/#man-vectorized" target="_blank">https://docs.julialang.org/en/v1/manual/functions/#man-vectorized</a> <a href="#fnref:28" class="footnote-back-ref">↩</a></li>
<li id="fn:29">"Common Matrices — SymPy 1.9 documentation". <a href="https://docs.sympy.org/latest/modules/matrices/common.html?highlight=multiply_elementwise#sympy.matrices.common.MatrixCommon.multiply" target="_blank">https://docs.sympy.org/latest/modules/matrices/common.html?highlight=multiply_elementwise#sympy.matrices.common.MatrixCommon.multiply</a> <a href="#fnref:29" class="footnote-back-ref">↩</a></li>
<li id="fn:30">"Matrix multiplication". An Introduction to R. The R Project for Statistical Computing. 16 May 2013. Retrieved 24 August 2013. <a href="https://cran.r-project.org/doc/manuals/r-release/R-intro.html#Multiplication" target="_blank">https://cran.r-project.org/doc/manuals/r-release/R-intro.html#Multiplication</a> <a href="#fnref:30" class="footnote-back-ref">↩</a></li>
<li id="fn:31">Sak, Haşim; Senior, Andrew; Beaufays, Françoise (2014-02-05). "Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition". arXiv:1402.1128 [cs.NE]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:31" class="footnote-back-ref">↩</a></li>
<li id="fn:32">Neudecker, Heinz; Liu, Shuangzhe; Polasek, Wolfgang (1995). "The Hadamard product and some of its applications in statistics". Statistics. 26 (4): 365–373. doi:10.1080/02331889508802503. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:32" class="footnote-back-ref">↩</a></li>
<li id="fn:33">Neudecker, Heinz; Liu, Shuangzhe (2001). "Some statistical properties of Hadamard products of random matrices". Statistical Papers. 42 (4): 475–487. doi:10.1007/s003620100074. S2CID 121385730. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:33" class="footnote-back-ref">↩</a></li>
<li id="fn:34">Slyusar, V. I. (March 13, 1998). "A Family of Face Products of Matrices and its properties" (PDF). Cybernetics and Systems Analysis C/C of Kibernetika I Sistemnyi Analiz. 1999. 35 (3): 379–384. doi:10.1007/BF02733426. S2CID 119661450. <a href="http://slyusar.kiev.ua/FACE.pdf" target="_blank">http://slyusar.kiev.ua/FACE.pdf</a> <a href="#fnref:34" class="footnote-back-ref">↩</a></li>
<li id="fn:35">Slyusar, V. I. (March 13, 1998). "A Family of Face Products of Matrices and its properties" (PDF). Cybernetics and Systems Analysis C/C of Kibernetika I Sistemnyi Analiz. 1999. 35 (3): 379–384. doi:10.1007/BF02733426. S2CID 119661450. <a href="http://slyusar.kiev.ua/FACE.pdf" target="_blank">http://slyusar.kiev.ua/FACE.pdf</a> <a href="#fnref:35" class="footnote-back-ref">↩</a></li>
<li id="fn:36">Slyusar, V. I. (March 13, 1998). "A Family of Face Products of Matrices and its properties" (PDF). Cybernetics and Systems Analysis C/C of Kibernetika I Sistemnyi Analiz. 1999. 35 (3): 379–384. doi:10.1007/BF02733426. S2CID 119661450. <a href="http://slyusar.kiev.ua/FACE.pdf" target="_blank">http://slyusar.kiev.ua/FACE.pdf</a> <a href="#fnref:36" class="footnote-back-ref">↩</a></li>
<li id="fn:37">Ha D., Dai A.M., Le Q.V. (2017). "HyperNetworks". The International Conference on Learning Representations (ICLR) 2017. – Toulon, 2017.: Page 6. arXiv:1609.09106.{{cite journal}}: CS1 maint: multiple names: authors list (link) <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:37" class="footnote-back-ref">↩</a></li>
</ol>

Hadamard product (matrices) open-in-new

Hadamard product (matrices)