MDS algorithms fall into a taxonomy, depending on the meaning of the input matrix:
It is also known as Principal Coordinates Analysis (PCoA), Torgerson Scaling or Torgerson–Gower scaling. It takes an input matrix giving dissimilarities between pairs of items and outputs a coordinate matrix whose configuration minimizes a loss function called strain,4 which is given by Strain D ( x 1 , x 2 , . . . , x n ) = ( ∑ i , j ( b i j − x i T x j ) 2 ∑ i , j b i j 2 ) 1 / 2 , {\displaystyle {\text{Strain}}_{D}(x_{1},x_{2},...,x_{n})={\Biggl (}{\frac {\sum _{i,j}{\bigl (}b_{ij}-x_{i}^{T}x_{j}{\bigr )}^{2}}{\sum _{i,j}b_{ij}^{2}}}{\Biggr )}^{1/2},} where x i {\displaystyle x_{i}} denote vectors in N-dimensional space, x i T x j {\displaystyle x_{i}^{T}x_{j}} denotes the scalar product between x i {\displaystyle x_{i}} and x j {\displaystyle x_{j}} , and b i j {\displaystyle b_{ij}} are the elements of the matrix B {\displaystyle B} defined on step 2 of the following algorithm, which are computed from the distances.
It is a superset of classical MDS that generalizes the optimization procedure to a variety of loss functions and input matrices of known distances with weights and so on. A useful loss function in this context is called stress, which is often minimized using a procedure called stress majorization. Metric MDS minimizes the cost function called “stress” which is a residual sum of squares:
Stress D ( x 1 , x 2 , . . . , x n ) = ∑ i ≠ j = 1 , . . . , n ( d i j − ‖ x i − x j ‖ ) 2 . {\displaystyle {\text{Stress}}_{D}(x_{1},x_{2},...,x_{n})={\sqrt {\sum _{i\neq j=1,...,n}{\bigl (}d_{ij}-\|x_{i}-x_{j}\|{\bigr )}^{2}}}.}
Metric scaling uses a power transformation with a user-controlled exponent p {\textstyle p} : d i j p {\textstyle d_{ij}^{p}} and − d i j 2 p {\textstyle -d_{ij}^{2p}} for distance. In classical scaling p = 1. {\textstyle p=1.} Non-metric scaling is defined by the use of isotonic regression to nonparametrically estimate a transformation of the dissimilarities.
In contrast to metric MDS, non-metric MDS finds both a non-parametric monotonic relationship between the dissimilarities in the item-item matrix and the Euclidean distances between items, and the location of each item in the low-dimensional space.
Let d i j {\displaystyle d_{ij}} be the dissimilarity between points i , j {\displaystyle i,j} . Let d ^ i j = ‖ x i − x j ‖ {\displaystyle {\hat {d}}_{ij}=\|x_{i}-x_{j}\|} be the Euclidean distance between embedded points x i , x j {\displaystyle x_{i},x_{j}} .
Now, for each choice of the embedded points x i {\displaystyle x_{i}} and is a monotonically increasing function f {\displaystyle f} , define the "stress" function:
S ( x 1 , . . . , x n ; f ) = ∑ i < j ( f ( d i j ) − d ^ i j ) 2 ∑ i < j d ^ i j 2 . {\displaystyle S(x_{1},...,x_{n};f)={\sqrt {\frac {\sum _{i<j}{\bigl (}f(d_{ij})-{\hat {d}}_{ij}{\bigr )}^{2}}{\sum _{i<j}{\hat {d}}_{ij}^{2}}}}.}
The factor of ∑ i < j d ^ i j 2 {\displaystyle \sum _{i<j}{\hat {d}}_{ij}^{2}} in the denominator is necessary to prevent a "collapse". Suppose we define instead S = ∑ i < j ( f ( d i j ) − d ^ i j ) 2 {\displaystyle S={\sqrt {\sum _{i<j}{\bigl (}f(d_{ij})-{\hat {d}}_{ij})^{2}}}} , then it can be trivially minimized by setting f = 0 {\displaystyle f=0} , then collapse every point to the same point.
A few variants of this cost function exist. MDS programs automatically minimize stress in order to obtain the MDS solution.
The core of a non-metric MDS algorithm is a twofold optimization process. First the optimal monotonic transformation of the proximities has to be found. Secondly, the points of a configuration have to be optimally arranged, so that their distances match the scaled proximities as closely as possible.
NMDS needs to optimize two objectives simultaneously. This is usually done iteratively:
Louis Guttman's smallest space analysis (SSA) is an example of a non-metric MDS procedure.
Main article: Generalized multidimensional scaling
An extension of metric multidimensional scaling, in which the target space is an arbitrary smooth non-Euclidean space. In cases where the dissimilarities are distances on a surface and the target space is another surface, GMDS allows finding the minimum-distortion embedding of one surface into another.6
An extension of MDS, known as Super MDS, incorporates both distance and angle information for improved source localization. Unlike traditional MDS, which uses only distance measurements, Super MDS processes both distance and angle-of-arrival (AOA) data algebraically (without iteration) to achieve better accuracy.7
The method proceeds in the following steps:
This concise approach reduces the need for multiple anchors and enhances localization precision by leveraging angle constraints.
The data to be analyzed is a collection of M {\displaystyle M} objects (colors, faces, stocks, . . .) on which a distance function is defined,
These distances are the entries of the dissimilarity matrix
The goal of MDS is, given D {\displaystyle D} , to find M {\displaystyle M} vectors x 1 , … , x M ∈ R N {\displaystyle x_{1},\ldots ,x_{M}\in \mathbb {R} ^{N}} such that
where ‖ ⋅ ‖ {\displaystyle \|\cdot \|} is a vector norm. In classical MDS, this norm is the Euclidean distance, but, in a broader sense, it may be a metric or arbitrary distance function.8 For example, when dealing with mixed-type data that contain numerical as well as categorical descriptors, Gower's distance is a common alternative.
In other words, MDS attempts to find a mapping from the M {\displaystyle M} objects into R N {\displaystyle \mathbb {R} ^{N}} such that distances are preserved. If the dimension N {\displaystyle N} is chosen to be 2 or 3, we may plot the vectors x i {\displaystyle x_{i}} to obtain a visualization of the similarities between the M {\displaystyle M} objects. Note that the vectors x i {\displaystyle x_{i}} are not unique: With the Euclidean distance, they may be arbitrarily translated, rotated, and reflected, since these transformations do not change the pairwise distances ‖ x i − x j ‖ {\displaystyle \|x_{i}-x_{j}\|} .
(Note: The symbol R {\displaystyle \mathbb {R} } indicates the set of real numbers, and the notation R N {\displaystyle \mathbb {R} ^{N}} refers to the Cartesian product of N {\displaystyle N} copies of R {\displaystyle \mathbb {R} } , which is an N {\displaystyle N} -dimensional vector space over the field of the real numbers.)
There are various approaches to determining the vectors x i {\displaystyle x_{i}} . Usually, MDS is formulated as an optimization problem, where ( x 1 , … , x M ) {\displaystyle (x_{1},\ldots ,x_{M})} is found as a minimizer of some cost function, for example,
A solution may then be found by numerical optimization techniques. For some particularly chosen cost functions, minimizers can be stated analytically in terms of matrix eigendecompositions.9
There are several steps in conducting MDS research:
Mead, A (1992). "Review of the Development of Multidimensional Scaling Methods". Journal of the Royal Statistical Society. Series D (The Statistician). 41 (1): 27–39. doi:10.2307/2348634. JSTOR 2348634. Abstract. Multidimensional scaling methods are now a common statistical tool in psychophysics and sensory analysis. The development of these methods is charted, from the original research of Torgerson (metric scaling), Shepard and Kruskal (non-metric scaling) through individual differences scaling and the maximum likelihood methods proposed by Ramsay. /wiki/Doi_(identifier) ↩
Borg, I.; Groenen, P. (2005). Modern Multidimensional Scaling: theory and applications (2nd ed.). New York: Springer-Verlag. pp. 207–212. ISBN 978-0-387-94845-4. 978-0-387-94845-4 ↩
Genest, Christian; Nešlehová, Johanna G.; Ramsay, James O. (2014). "A Conversation with James O. Ramsay". International Statistical Review / Revue Internationale de Statistique. 82 (2): 161–183. JSTOR 43299752. Retrieved 30 June 2021. https://www.jstor.org/stable/43299752 ↩
Wickelmaier, Florian. "An introduction to MDS." Sound Quality Research Unit, Aalborg University, Denmark (2003): 46 ↩
Bronstein AM, Bronstein MM, Kimmel R (January 2006). "Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching". Proc. Natl. Acad. Sci. U.S.A. 103 (5): 1168–72. Bibcode:2006PNAS..103.1168B. doi:10.1073/pnas.0508601103. PMC 1360551. PMID 16432211. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1360551 ↩
de Abreu, G. T. F.; Destino, G. (2007). Super MDS: Source Location from Distance and Angle Information. 2007 IEEE Wireless Communications and Networking Conference. Hong Kong, China. pp. 4430–4434. doi:10.1109/WCNC.2007.807. /wiki/Doi_(identifier) ↩
Kruskal, J. B., and Wish, M. (1978), Multidimensional Scaling, Sage University Paper series on Quantitative Application in the Social Sciences, 07-011. Beverly Hills and London: Sage Publications. /wiki/Joseph_Kruskal ↩
Kruskal, J. B. (1964). "Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis". Psychometrika. 29 (1): 1–27. doi:10.1007/BF02289565. S2CID 48165675. /wiki/Joseph_Kruskal ↩
Leeuw, Jan de; Mair, Patrick (2009). "Multidimensional Scaling Using Majorization: SMACOF in R". Journal of Statistical Software. 31 (3). doi:10.18637/jss.v031.i03. ISSN 1548-7660. http://www.jstatsoft.org/v31/i03/ ↩