Rank reversals in decision-making

In <a href="/facts/Decision-making/NbJYprSw">decision-making</a>, a rank reversal is a change in the rank ordering of the preferability of alternative possible decisions when, for example, the method of choosing changes or the set of other available alternatives changes. The issue of rank reversals lies at the heart of many debates in decision-making and <a href="/facts/Multi-criteria_decision-making/IXQLJgBx">multi-criteria decision-making</a>, in particular.
Unlike most other computational procedures, it is hard to tell if a particular decision-making method has derived the correct answer or not. Such methods analyze a set of alternatives described in terms of some criteria. They determine which alternative is the best one, or they provide relative weights of how the alternatives perform, or just how the alternatives should be ranked when all the criteria are considered simultaneously. This is exactly where the challenge with decision making exists. Often it is hard, if not practically impossible, to determine whether a correct answer has been reached or not. With other computational methods, for instance with a job scheduling method, one can examine a set of different answers and then categorize the answers according to some metric of performance (for instance, a project's completion time). But this may not be possible to do with the answers derived by most decision making methods. After all, determining the best decision making method leads to a <a href="/facts/Decision_making_paradox/hE83GFM0">decision making paradox</a>.
Thus the following question emerges: How can one evaluate decision-making methods? This is a very difficult issue and may not be answered in a globally accepted manner.
A critical part in answering this fundamental question is played by what is known as rank reversals.

Rank reversals in decision-making open-in-new

Rank reversals in decision-making