For finite-dimensional real vectors in R n {\displaystyle \mathbb {R} ^{n}} with the usual Euclidean dot product, the Gram matrix is G = V ⊤ V {\displaystyle G=V^{\top }V} , where V {\displaystyle V} is a matrix whose columns are the vectors v k {\displaystyle v_{k}} and V ⊤ {\displaystyle V^{\top }} is its transpose whose rows are the vectors v k ⊤ {\displaystyle v_{k}^{\top }} . For complex vectors in C n {\displaystyle \mathbb {C} ^{n}} , G = V † V {\displaystyle G=V^{\dagger }V} , where V † {\displaystyle V^{\dagger }} is the conjugate transpose of V {\displaystyle V} .
Given square-integrable functions { ℓ i ( ⋅ ) , i = 1 , … , n } {\displaystyle \{\ell _{i}(\cdot ),\,i=1,\dots ,n\}} on the interval [ t 0 , t f ] {\displaystyle \left[t_{0},t_{f}\right]} , the Gram matrix G = [ G i j ] {\displaystyle G=\left[G_{ij}\right]} is:
where ℓ i ∗ ( τ ) {\displaystyle \ell _{i}^{*}(\tau )} is the complex conjugate of ℓ i ( τ ) {\displaystyle \ell _{i}(\tau )} .
For any bilinear form B {\displaystyle B} on a finite-dimensional vector space over any field we can define a Gram matrix G {\displaystyle G} attached to a set of vectors v 1 , … , v n {\displaystyle v_{1},\dots ,v_{n}} by G i j = B ( v i , v j ) {\displaystyle G_{ij}=B\left(v_{i},v_{j}\right)} . The matrix will be symmetric if the bilinear form B {\displaystyle B} is symmetric.
The Gram matrix is symmetric in the case the inner product is real-valued; it is Hermitian in the general, complex case by definition of an inner product.
The Gram matrix is positive semidefinite, and every positive semidefinite matrix is the Gramian matrix for some set of vectors. The fact that the Gramian matrix is positive-semidefinite can be seen from the following simple derivation:
The first equality follows from the definition of matrix multiplication, the second and third from the bi-linearity of the inner-product, and the last from the positive definiteness of the inner product. Note that this also shows that the Gramian matrix is positive definite if and only if the vectors v i {\displaystyle v_{i}} are linearly independent (that is, ∑ i x i v i ≠ 0 {\textstyle \sum _{i}x_{i}v_{i}\neq 0} for all x {\displaystyle x} ).3
See also: Positive definite matrix § Decomposition
Given any positive semidefinite matrix M {\displaystyle M} , one can decompose it as:
where B † {\displaystyle B^{\dagger }} is the conjugate transpose of B {\displaystyle B} (or M = B T B {\displaystyle M=B^{\textsf {T}}B} in the real case).
Here B {\displaystyle B} is a k × n {\displaystyle k\times n} matrix, where k {\displaystyle k} is the rank of M {\displaystyle M} . Various ways to obtain such a decomposition include computing the Cholesky decomposition or taking the non-negative square root of M {\displaystyle M} .
The columns b ( 1 ) , … , b ( n ) {\displaystyle b^{(1)},\dots ,b^{(n)}} of B {\displaystyle B} can be seen as n vectors in C k {\displaystyle \mathbb {C} ^{k}} (or k-dimensional Euclidean space R k {\displaystyle \mathbb {R} ^{k}} , in the real case). Then
where the dot product a ⋅ b = ∑ ℓ = 1 k a ℓ ∗ b ℓ {\textstyle a\cdot b=\sum _{\ell =1}^{k}a_{\ell }^{*}b_{\ell }} is the usual inner product on C k {\displaystyle \mathbb {C} ^{k}} .
Thus a Hermitian matrix M {\displaystyle M} is positive semidefinite if and only if it is the Gram matrix of some vectors b ( 1 ) , … , b ( n ) {\displaystyle b^{(1)},\dots ,b^{(n)}} . Such vectors are called a vector realization of M {\displaystyle M} . The infinite-dimensional analog of this statement is Mercer's theorem.
If M {\displaystyle M} is the Gram matrix of vectors v 1 , … , v n {\displaystyle v_{1},\dots ,v_{n}} in R k {\displaystyle \mathbb {R} ^{k}} then applying any rotation or reflection of R k {\displaystyle \mathbb {R} ^{k}} (any orthogonal transformation, that is, any Euclidean isometry preserving 0) to the sequence of vectors results in the same Gram matrix. That is, for any k × k {\displaystyle k\times k} orthogonal matrix Q {\displaystyle Q} , the Gram matrix of Q v 1 , … , Q v n {\displaystyle Qv_{1},\dots ,Qv_{n}} is also M {\displaystyle M} .
This is the only way in which two real vector realizations of M {\displaystyle M} can differ: the vectors v 1 , … , v n {\displaystyle v_{1},\dots ,v_{n}} are unique up to orthogonal transformations. In other words, the dot products v i ⋅ v j {\displaystyle v_{i}\cdot v_{j}} and w i ⋅ w j {\displaystyle w_{i}\cdot w_{j}} are equal if and only if some rigid transformation of R k {\displaystyle \mathbb {R} ^{k}} transforms the vectors v 1 , … , v n {\displaystyle v_{1},\dots ,v_{n}} to w 1 , … , w n {\displaystyle w_{1},\dots ,w_{n}} and 0 to 0.
The same holds in the complex case, with unitary transformations in place of orthogonal ones. That is, if the Gram matrix of vectors v 1 , … , v n {\displaystyle v_{1},\dots ,v_{n}} is equal to the Gram matrix of vectors w 1 , … , w n {\displaystyle w_{1},\dots ,w_{n}} in C k {\displaystyle \mathbb {C} ^{k}} then there is a unitary k × k {\displaystyle k\times k} matrix U {\displaystyle U} (meaning U † U = I {\displaystyle U^{\dagger }U=I} ) such that v i = U w i {\displaystyle v_{i}=Uw_{i}} for i = 1 , … , n {\displaystyle i=1,\dots ,n} .4
The Gram determinant or Gramian is the determinant of the Gram matrix: | G ( v 1 , … , v n ) | = | ⟨ v 1 , v 1 ⟩ ⟨ v 1 , v 2 ⟩ … ⟨ v 1 , v n ⟩ ⟨ v 2 , v 1 ⟩ ⟨ v 2 , v 2 ⟩ … ⟨ v 2 , v n ⟩ ⋮ ⋮ ⋱ ⋮ ⟨ v n , v 1 ⟩ ⟨ v n , v 2 ⟩ … ⟨ v n , v n ⟩ | . {\displaystyle {\bigl |}G(v_{1},\dots ,v_{n}){\bigr |}={\begin{vmatrix}\langle v_{1},v_{1}\rangle &\langle v_{1},v_{2}\rangle &\dots &\langle v_{1},v_{n}\rangle \\\langle v_{2},v_{1}\rangle &\langle v_{2},v_{2}\rangle &\dots &\langle v_{2},v_{n}\rangle \\\vdots &\vdots &\ddots &\vdots \\\langle v_{n},v_{1}\rangle &\langle v_{n},v_{2}\rangle &\dots &\langle v_{n},v_{n}\rangle \end{vmatrix}}.}
If v 1 , … , v n {\displaystyle v_{1},\dots ,v_{n}} are vectors in R m {\displaystyle \mathbb {R} ^{m}} then it is the square of the n-dimensional volume of the parallelotope formed by the vectors. In particular, the vectors are linearly independent if and only if the parallelotope has nonzero n-dimensional volume, if and only if Gram determinant is nonzero, if and only if the Gram matrix is nonsingular. When n > m the determinant and volume are zero. When n = m, this reduces to the standard theorem that the absolute value of the determinant of n n-dimensional vectors is the n-dimensional volume. The Gram determinant is also useful for computing the volume of the simplex formed by the vectors; its volume is Volume(parallelotope) / n!.
The Gram determinant can also be expressed in terms of the exterior product of vectors by
The Gram determinant therefore supplies an inner product for the space ⋀ n ( V ) {\displaystyle {\textstyle \bigwedge }^{\!n}(V)} . If an orthonormal basis ei, i = 1, 2, ..., n on V {\displaystyle V} is given, the vectors
will constitute an orthonormal basis of n-dimensional volumes on the space ⋀ n ( V ) {\displaystyle {\textstyle \bigwedge }^{\!n}(V)} . Then the Gram determinant | G ( v 1 , … , v n ) | {\displaystyle {\bigl |}G(v_{1},\dots ,v_{n}){\bigr |}} amounts to an n-dimensional Pythagorean Theorem for the volume of the parallelotope formed by the vectors v 1 ∧ ⋯ ∧ v n {\displaystyle v_{1}\wedge \cdots \wedge v_{n}} in terms of its projections onto the basis volumes e i 1 ∧ ⋯ ∧ e i n {\displaystyle e_{i_{1}}\wedge \cdots \wedge e_{i_{n}}} .
When the vectors v 1 , … , v n ∈ R m {\displaystyle v_{1},\ldots ,v_{n}\in \mathbb {R} ^{m}} are defined from the positions of points p 1 , … , p n {\displaystyle p_{1},\ldots ,p_{n}} relative to some reference point p n + 1 {\displaystyle p_{n+1}} ,
then the Gram determinant can be written as the difference of two Gram determinants,
where each ( p j , 1 ) {\displaystyle (p_{j},1)} is the corresponding point p j {\displaystyle p_{j}} supplemented with the coordinate value of 1 for an ( m + 1 ) {\displaystyle (m+1)} -st dimension. Note that in the common case that n = m, the second term on the right-hand side will be zero.
Given a set of linearly independent vectors { v i } {\displaystyle \{v_{i}\}} with Gram matrix G {\displaystyle G} defined by G i j := ⟨ v i , v j ⟩ {\displaystyle G_{ij}:=\langle v_{i},v_{j}\rangle } , one can construct an orthonormal basis
In matrix notation, U = V G − 1 / 2 {\displaystyle U=VG^{-1/2}} , where U {\displaystyle U} has orthonormal basis vectors { u i } {\displaystyle \{u_{i}\}} and the matrix V {\displaystyle V} is composed of the given column vectors { v i } {\displaystyle \{v_{i}\}} .
The matrix G − 1 / 2 {\displaystyle G^{-1/2}} is guaranteed to exist. Indeed, G {\displaystyle G} is Hermitian, and so can be decomposed as G = U D U † {\displaystyle G=UDU^{\dagger }} with U {\displaystyle U} a unitary matrix and D {\displaystyle D} a real diagonal matrix. Additionally, the v i {\displaystyle v_{i}} are linearly independent if and only if G {\displaystyle G} is positive definite, which implies that the diagonal entries of D {\displaystyle D} are positive. G − 1 / 2 {\displaystyle G^{-1/2}} is therefore uniquely defined by G − 1 / 2 := U D − 1 / 2 U † {\displaystyle G^{-1/2}:=UD^{-1/2}U^{\dagger }} . One can check that these new vectors are orthonormal:
where we used ( G − 1 / 2 ) † = G − 1 / 2 {\displaystyle {\bigl (}G^{-1/2}{\bigr )}^{\dagger }=G^{-1/2}} .
Horn & Johnson 2013, p. 441, p.441, Theorem 7.2.10 - Horn, Roger A.; Johnson, Charles R. (2013). Matrix Analysis (2nd ed.). Cambridge University Press. ISBN 978-0-521-54823-6. ↩
Lanckriet, G. R. G.; Cristianini, N.; Bartlett, P.; Ghaoui, L. E.; Jordan, M. I. (2004). "Learning the kernel matrix with semidefinite programming". Journal of Machine Learning Research. 5: 27–72 [p. 29]. https://dl.acm.org/citation.cfm?id=894170 ↩
Horn & Johnson (2013), p. 452, Theorem 7.3.11 - Horn, Roger A.; Johnson, Charles R. (2013). Matrix Analysis (2nd ed.). Cambridge University Press. ISBN 978-0-521-54823-6. ↩