Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Stochastic transitivity
open-in-new
<h2 id="a-toy-example">A toy example</h2> <p>The marble game - Assume two kids, Billy and Gabriela, collect marbles. Billy collects blue marbles and Gabriela green marbles. When they get together they play a game where they mix all their marbles in a bag and sample one randomly. If the sampled marble is green, then Gabriela wins and if it is blue then Billy wins. If B {\displaystyle B} is the number of blue marbles and G {\displaystyle G} is the number of green marbles in the bag, then the probability P ( Billy ≿ Gabriela ) {\displaystyle \mathbb {P} ({\text{Billy}}\succsim {\text{Gabriela}})} of Billy winning against Gabriela is </p><p> P ( Billy ≿ Gabriela ) = B B + G = e ln ( B ) e ln ( B ) + e ln ( G ) = 1 1 + e ln ( G ) − ln ( B ) {\displaystyle \mathbb {P} ({\text{Billy}}\succsim {\text{Gabriela}})={\frac {B}{B+G}}={\frac {e^{\ln(B)}}{e^{\ln(B)}+e^{\ln(G)}}}={\frac {1}{1+e^{\ln(G)-\ln(B)}}}} . </p><p>In this example, the marble game satisfies linear stochastic transitivity, where the comparison function F : R → [ 0 , 1 ] {\displaystyle F:\mathbb {R} \to [0,1]} is given by F ( x ) = 1 1 + e − x {\displaystyle F(x)={\frac {1}{1+e^{-x}}}} and the merit function μ : A → R {\displaystyle \mu :{\mathcal {A}}\to \mathbb {R} } is given by μ ( M ) = ln ( M ) {\displaystyle \mu (M)=\ln(M)} , where M {\displaystyle M} is the number of marbles of the player. This game happens to be an example of a <a href="/facts/Bradley%E2%80%93Terry_model/0qdoXn49">Bradley–Terry model</a>.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> </p> <h2 id="applications">Applications</h2> <ul><li><i>Ranking and Rating</i> - Stochastic transitivity models have been used as the basis of several ranking and rating methods. Examples include the <a href="/facts/Elo_rating_system/elxHsGzk">Elo-Rating system</a> used in chess, go, and other classical sports as well as Microsoft's <a href="/facts/TrueSkill/3I1Q1Irt">TrueSkill</a> used for the Xbox gaming platform.</li> <li><i>Models of Psychology and Rationality</i> - <a href="/facts/Thurstonian_model/ymSL5hIB">Thurstonian models</a><a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> (see Case 5 in <a href="/facts/Law_of_comparative_judgment/dFPoDkWP">law of comparative judgement</a>), Fechnerian models<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> and also <a href="/facts/Luce%27s_choice_axiom/oF9T9JjN">Luce's choice axiom</a><a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> are theories that have foundations on the mathematics of stochastic transitivity. Also, models of <a href="/facts/Rational_choice_theory/NgEDuLBx">rational choice theory</a> are based on the assumption of transitivity of <a href="/facts/Preference/hWCEmfuX">preferences</a> (see <a href="/facts/Von_Neumann%E2%80%93Morgenstern_utility_theorem/jKfsUBQ0">Von Neumann's utility</a> and <a href="/facts/Debreu_theorems/at0YJI45">Debreu's Theorems</a>), these preferences, however, are often revealed with noise in a stochastic manner.<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a><a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a><a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a></li> <li><i>Machine Learning and Artificial Intelligence (see <a href="/facts/Learning_to_rank/4Js3B8LG">Learn to Rank</a>)</i> - While Elo and TrueSkill rely on specific LST models, machine learning models have been developed to rank without prior knowledge of the underlying stochastic transitivity model or under weaker than usual assumptions on the stochastic transitivity.<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a><a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a><a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> Learning from paired comparisons is also of interest since it allows for AI agents to learn the underlying preferences of other agents.</li> <li><i>Game Theory</i> - Fairness of random knockout tournaments is strongly dependent on the underlying stochastic transitivity model.<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a><a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a><a class="footnote-ref" id="fnref:20" href="#fn:20"><sup>20</sup></a> Social choice theory also has foundations that depend on stochastic transitivity models.<a class="footnote-ref" id="fnref:21" href="#fn:21"><sup>21</sup></a></li></ul> <h2 id="connections-between-models">Connections between models</h2> <p>Positive Results: </p> <ol><li>Every model that satisfies Linear Stochastic Transitivity must also satisfy Strong Stochastic Transitivity, which in turn must satisfy Weak Stochastic Transitivity. This is represented as: LST ⟹ {\displaystyle \implies } SST ⟹ {\displaystyle \implies } WST ;</li> <li>Since the Bradley-Terry models and Thurstone's Case V model<a class="footnote-ref" id="fnref:22" href="#fn:22"><sup>22</sup></a> are LST models, they also satisfy SST and WST;</li> <li>Due to the convenience of more structured models[<i>clarify</i>], a few authors<a class="footnote-ref" id="fnref:23" href="#fn:23"><sup>23</sup></a><a class="footnote-ref" id="fnref:24" href="#fn:24"><sup>24</sup></a><a class="footnote-ref" id="fnref:25" href="#fn:25"><sup>25</sup></a><a class="footnote-ref" id="fnref:26" href="#fn:26"><sup>26</sup></a><a class="footnote-ref" id="fnref:27" href="#fn:27"><sup>27</sup></a><a class="footnote-ref" id="fnref:28" href="#fn:28"><sup>28</sup></a> have identified axiomatic justifications[<i>clarify</i>] of linear stochastic transitivity (and other models), most notably <a href="/facts/G%C3%A9rard_Debreu/h1lKfo20">Gérard Debreu</a> showed that:<a class="footnote-ref" id="fnref:29" href="#fn:29"><sup>29</sup></a> Quadruple Condition[<i>clarify</i>] + Continuity[<i>clarify</i>] ⟹ {\displaystyle \implies } LST (see also <a href="/facts/Debreu_theorems/at0YJI45">Debreu Theorems</a>);</li> <li>Two LST models given by <a href="/facts/Invertible_function/Tg5nnXlu">invertible</a> comparison functions F ( x ) {\displaystyle F(x)} and G ( x ) {\displaystyle G(x)} are equivalent[<i>clarify</i>] if and only if F ( x ) = G ( κ x ) {\displaystyle F(x)=G(\kappa x)} for some κ ≥ 0. {\displaystyle \kappa \geq 0.} <a class="footnote-ref" id="fnref:30" href="#fn:30"><sup>30</sup></a></li></ol> <p>Negative Results: </p> <ol><li>Stochastic transitivity models are empirically unverifiable[<i>clarify</i>],<a class="footnote-ref" id="fnref:31" href="#fn:31"><sup>31</sup></a> however, they may be falsifiable;</li> <li>Distinguishing[<i>clarify</i>] between LST comparison functions F ( x ) {\displaystyle F(x)} and G ( x ) {\displaystyle G(x)} can be impossible even if an infinite amount of data is provided over a finite number of points[<i>clarify</i>];<a class="footnote-ref" id="fnref:32" href="#fn:32"><sup>32</sup></a></li> <li>The estimation problem[<i>clarify</i>] for WST, SST and LST models are in general <a href="/facts/NP-hardness/mQeNPv4R">NP-Hard</a>,<a class="footnote-ref" id="fnref:33" href="#fn:33"><sup>33</sup></a> however, near optimal polynomially computable estimation procedures are known for SST and LST models.<a class="footnote-ref" id="fnref:34" href="#fn:34"><sup>34</sup></a><a class="footnote-ref" id="fnref:35" href="#fn:35"><sup>35</sup></a><a class="footnote-ref" id="fnref:36" href="#fn:36"><sup>36</sup></a></li></ol> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Nontransitive_game/uTlbBG7D">Nontransitive game</a></li> <li><a href="/facts/Decision_theory/TbXcQX0v">Decision theory</a></li> <li><a href="/facts/Utilitarianism/CVw8Dmmt">Utilitarianism</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Fishburn, Peter C. (November 1973). "Binary choice probabilities: on the varieties of stochastic transitivity". Journal of Mathematical Psychology. 10 (4): 327–352. doi:10.1016/0022-2496(73)90021-7. ISSN 0022-2496. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Clark, Stephen A. (March 1990). "A concept of stochastic transitivity for the random utility model". Journal of Mathematical Psychology. 34 (1): 95–108. doi:10.1016/0022-2496(90)90015-2. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Ryan, Matthew (2017-01-21). "Uncertainty and binary stochastic choice". Economic Theory. 65 (3): 629–662. doi:10.1007/s00199-017-1033-4. ISSN 0938-2259. S2CID 125420775. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Oliveira, I.F.D.; Zehavi, S.; Davidov, O. (August 2018). "Stochastic transitivity: Axioms and models". Journal of Mathematical Psychology. 85: 25–35. doi:10.1016/j.jmp.2018.06.002. ISSN 0022-2496. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Donald Davidson and Jacob Marschak (Jul 1958). Experimental tests of a stochastic decision theory (PDF) (Technical Report). Stanford University. <a href="https://web.stanford.edu/group/csli-suppes/techreports/IMSSS_17.pdf" target="_blank">https://web.stanford.edu/group/csli-suppes/techreports/IMSSS_17.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Michel Regenwetter and Jason Dana and Clintin P. Davis-Stober (2011). "Transitivity of Preferences" (PDF). Psychological Review. 118 (1): 42–56. doi:10.1037/a0021150. PMID 21244185. <a href="https://www.chapman.edu/research/institutes-and-centers/economic-science-institute/_files/ifree-papers-and-photos/michel-regenwetter1.pdf" target="_blank">https://www.chapman.edu/research/institutes-and-centers/economic-science-institute/_files/ifree-papers-and-photos/michel-regenwetter1.pdf</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Donald Davidson and Jacob Marschak (Jul 1958). Experimental tests of a stochastic decision theory (PDF) (Technical Report). Stanford University. <a href="https://web.stanford.edu/group/csli-suppes/techreports/IMSSS_17.pdf" target="_blank">https://web.stanford.edu/group/csli-suppes/techreports/IMSSS_17.pdf</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Bradley, Ralph Allan; Terry, Milton E. (December 1952). "Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons". Biometrika. 39 (3/4): 324. doi:10.2307/2334029. JSTOR 2334029. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Thurstone, L. L. (1994). "A law of comparative judgment". Psychological Review. 101 (2): 266–270. doi:10.1037/0033-295X.101.2.266. ISSN 0033-295X. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>Ryan, Matthew (2017-01-21). "Uncertainty and binary stochastic choice". Economic Theory. 65 (3): 629–662. doi:10.1007/s00199-017-1033-4. ISSN 0938-2259. S2CID 125420775. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>Luce, R. Duncan (Robert Duncan) (2005). Individual choice behavior : a theoretical analysis. Mineola, N.Y.: Dover Publications. ISBN 0486441369. OCLC 874031603. <a href="0486441369" target="_blank">0486441369</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Debreu, Gerard (July 1958). "Stochastic Choice and Cardinal Utility" (PDF). Econometrica. 26 (3): 440–444. doi:10.2307/1907622. ISSN 0012-9682. JSTOR 1907622. <a href="http://cowles.yale.edu/sites/default/files/files/pub/d00/d0039.pdf" target="_blank">http://cowles.yale.edu/sites/default/files/files/pub/d00/d0039.pdf</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>Regenwetter, Michel; Dana, Jason; Davis-Stober, Clintin P. (2011). "Transitivity of preferences". Psychological Review. 118 (1): 42–56. doi:10.1037/a0021150. ISSN 1939-1471. PMID 21244185. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Cavagnaro, Daniel R.; Davis-Stober, Clintin P. (2014). "Transitive in our preferences, but transitive in different ways: An analysis of choice variability". Decision. 1 (2): 102–122. doi:10.1037/dec0000011. ISSN 2325-9973. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>Shah, Nihar B.; Balakrishnan, Sivaraman; Guntuboyina, Adityanand; Wainwright, Martin J. (February 2017). "Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues". IEEE Transactions on Information Theory. 63 (2): 934–959. arXiv:1510.05610. doi:10.1109/tit.2016.2634418. ISSN 0018-9448. <a href="https://doi.org/10.1109%2Ftit.2016.2634418" target="_blank">https://doi.org/10.1109%2Ftit.2016.2634418</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>Chatterjee, Sabyasachi; Mukherjee, Sumit (June 2019). "Estimation in Tournaments and Graphs Under Monotonicity Constraints". IEEE Transactions on Information Theory. 65 (6): 3525–3539. arXiv:1603.04556. doi:10.1109/tit.2019.2893911. ISSN 0018-9448. S2CID 54740089. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>Oliveira, Ivo F.D.; Ailon, Nir; Davidov, Ori (2018). "A New and Flexible Approach to the Analysis of Paired Comparison Data". Journal of Machine Learning Research. 19: 1–29. <a href="http://www.jmlr.org/papers/v19/17-179.html" target="_blank">http://www.jmlr.org/papers/v19/17-179.html</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> <li id="fn:18"><p>Israel, Robert B. (December 1981). "Stronger Players Need not Win More Knockout Tournaments". Journal of the American Statistical Association. 76 (376): 950–951. doi:10.2307/2287594. ISSN 0162-1459. JSTOR 2287594. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li> <li id="fn:19"><p>Chen, Robert; Hwang, F. K. (December 1988). "Stronger players win more balanced knockout tournaments". Graphs and Combinatorics. 4 (1): 95–99. doi:10.1007/bf01864157. ISSN 0911-0119. S2CID 44602228. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li> <li id="fn:20"><p>Adler, Ilan; Cao, Yang; Karp, Richard; Peköz, Erol A.; Ross, Sheldon M. (December 2017). "Random Knockout Tournaments". Operations Research. 65 (6): 1589–1596. arXiv:1612.04448. doi:10.1287/opre.2017.1657. ISSN 0030-364X. S2CID 1041539. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></p></li> <li id="fn:21"><p>Sen, Amartya (January 1977). "Social Choice Theory: A Re-Examination". Econometrica. 45 (1): 53–89. doi:10.2307/1913287. ISSN 0012-9682. JSTOR 1913287. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></p></li> <li id="fn:22"><p>Thurstone, L. L. (1994). "A law of comparative judgment". Psychological Review. 101 (2): 266–270. doi:10.1037/0033-295X.101.2.266. ISSN 0033-295X. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></p></li> <li id="fn:23"><p>Fishburn, Peter C. (November 1973). "Binary choice probabilities: on the varieties of stochastic transitivity". Journal of Mathematical Psychology. 10 (4): 327–352. doi:10.1016/0022-2496(73)90021-7. ISSN 0022-2496. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></p></li> <li id="fn:24"><p>Clark, Stephen A. (March 1990). "A concept of stochastic transitivity for the random utility model". Journal of Mathematical Psychology. 34 (1): 95–108. doi:10.1016/0022-2496(90)90015-2. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></p></li> <li id="fn:25"><p>Ryan, Matthew (2017-01-21). "Uncertainty and binary stochastic choice". Economic Theory. 65 (3): 629–662. doi:10.1007/s00199-017-1033-4. ISSN 0938-2259. S2CID 125420775. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:25" class="footnote-back-ref">↩</a></p></li> <li id="fn:26"><p>Oliveira, I.F.D.; Zehavi, S.; Davidov, O. (August 2018). "Stochastic transitivity: Axioms and models". Journal of Mathematical Psychology. 85: 25–35. doi:10.1016/j.jmp.2018.06.002. ISSN 0022-2496. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:26" class="footnote-back-ref">↩</a></p></li> <li id="fn:27"><p>Blavatskyy, Pavlo R. (2007). Stochastic utility theorem. Inst. for Empirical Research in Economics. OCLC 255736997. <a href="/wiki/OCLC_(identifier)" target="_blank">/wiki/OCLC_(identifier)</a> <a href="#fnref:27" class="footnote-back-ref">↩</a></p></li> <li id="fn:28"><p>Dagsvik, John K. (October 2015). "Stochastic models for risky choices: A comparison of different axiomatizations". Journal of Mathematical Economics. 60: 81–88. doi:10.1016/j.jmateco.2015.06.013. ISSN 0304-4068. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:28" class="footnote-back-ref">↩</a></p></li> <li id="fn:29"><p>Debreu, Gerard (July 1958). "Stochastic Choice and Cardinal Utility" (PDF). Econometrica. 26 (3): 440–444. doi:10.2307/1907622. ISSN 0012-9682. JSTOR 1907622. <a href="http://cowles.yale.edu/sites/default/files/files/pub/d00/d0039.pdf" target="_blank">http://cowles.yale.edu/sites/default/files/files/pub/d00/d0039.pdf</a> <a href="#fnref:29" class="footnote-back-ref">↩</a></p></li> <li id="fn:30"><p>Yellott, John I. (April 1977). "The relationship between Luce's Choice Axiom, Thurstone's Theory of Comparative Judgment, and the double exponential distribution". Journal of Mathematical Psychology. 15 (2): 109–144. doi:10.1016/0022-2496(77)90026-8. ISSN 0022-2496. <a href="https://escholarship.org/uc/item/7z91732x" target="_blank">https://escholarship.org/uc/item/7z91732x</a> <a href="#fnref:30" class="footnote-back-ref">↩</a></p></li> <li id="fn:31"><p>Oliveira, I.F.D.; Zehavi, S.; Davidov, O. (August 2018). "Stochastic transitivity: Axioms and models". Journal of Mathematical Psychology. 85: 25–35. doi:10.1016/j.jmp.2018.06.002. ISSN 0022-2496. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:31" class="footnote-back-ref">↩</a></p></li> <li id="fn:32"><p>Rockwell, Christina; Yellott, John I. (February 1979). "A note on equivalent Thurstone models". Journal of Mathematical Psychology. 19 (1): 65–71. doi:10.1016/0022-2496(79)90006-3. ISSN 0022-2496. <a href="http://www.escholarship.org/uc/item/3c86p1kc" target="_blank">http://www.escholarship.org/uc/item/3c86p1kc</a> <a href="#fnref:32" class="footnote-back-ref">↩</a></p></li> <li id="fn:33"><p>deCani, John S. (December 1969). "Maximum Likelihood Paired Comparison Ranking by Linear Programming". Biometrika. 56 (3): 537–545. doi:10.2307/2334661. ISSN 0006-3444. JSTOR 2334661. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:33" class="footnote-back-ref">↩</a></p></li> <li id="fn:34"><p>Shah, Nihar B.; Balakrishnan, Sivaraman; Guntuboyina, Adityanand; Wainwright, Martin J. (February 2017). "Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues". IEEE Transactions on Information Theory. 63 (2): 934–959. arXiv:1510.05610. doi:10.1109/tit.2016.2634418. ISSN 0018-9448. <a href="https://doi.org/10.1109%2Ftit.2016.2634418" target="_blank">https://doi.org/10.1109%2Ftit.2016.2634418</a> <a href="#fnref:34" class="footnote-back-ref">↩</a></p></li> <li id="fn:35"><p>Chatterjee, Sabyasachi; Mukherjee, Sumit (June 2019). "Estimation in Tournaments and Graphs Under Monotonicity Constraints". IEEE Transactions on Information Theory. 65 (6): 3525–3539. arXiv:1603.04556. doi:10.1109/tit.2019.2893911. ISSN 0018-9448. S2CID 54740089. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:35" class="footnote-back-ref">↩</a></p></li> <li id="fn:36"><p>Oliveira, Ivo F.D.; Ailon, Nir; Davidov, Ori (2018). "A New and Flexible Approach to the Analysis of Paired Comparison Data". Journal of Machine Learning Research. 19: 1–29. <a href="http://www.jmlr.org/papers/v19/17-179.html" target="_blank">http://www.jmlr.org/papers/v19/17-179.html</a> <a href="#fnref:36" class="footnote-back-ref">↩</a></p></li> </ol>