Rank Reversal Analysis

This method is a robustness check for when you already hold a ranking computed by an MCDM method (TOPSIS, SAW, PROBID, say) and want to test how resistant that ranking is to small changes in the alternative set. It does not produce a ranking; it checks whether an existing ranking is shaken. Its output is a ratio showing how many of the perturbations tried changed the order among the original alternatives: the rank reversal rate (RRR). The concept was treated systematically in Triantaphyllou's (2000) book comparing MCDM methods, which showed that methods such as TOPSIS and AHP sometimes change the order among the original alternatives when a new one is added.

The implicit assumption behind most ranking methods is "independence from irrelevant alternatives": the outcome of comparing A and B should not be affected by whether a third alternative, C, is added to the mix. But this assumption does not always hold for methods whose normalisation step looks at the entire alternative set, dividing by the maximum, or building the ideal point from the alternatives, say. When a new C is added, the normalisation constants (column sum, column maximum, ideal point) change, and this can change A's and B's scores too; rarely, it also reverses the order between A and B.

Rank Reversal Analysis's philosophical stance is to treat this not as an "error" but as a "property to be questioned": every method carries this sensitivity to a different degree, and a decision-maker needs to know it when choosing which method to use. The method itself makes no choice; it only gives a numerical answer to the question "how stable is this base method, on this dataset, under this type of perturbation."

The method proceeds through four steps.

First, the base ranking. A chosen base MCDM method (TOPSIS, say) is applied to the original alternative set, producing a base ranking and base scores.

Second, perturbing the alternative set. One or more scenarios to be tested are defined: adding a new hypothetical alternative, adding a near-copy of an existing alternative, or removing the best or worst alternative. The base method is rerun for each scenario, producing a new ranking.

Third, detecting reversal. For every perturbation, each pairwise comparison among the original alternatives (whether A comes before B) is checked to see whether it changed between the base ranking and the new ranking.

Fourth, the aggregate rate. Across all perturbations and all pairwise comparisons, the number that changed order is counted and divided by the total number of possible comparisons, giving a single rank reversal rate (RRR). This rate lies between 0 and 1; 0 means no reversal occurred, and 1 means every perturbation produced complete reversal.

The formulas behind each step, the intermediate tables and the citation formats are given on the DecisionMind method page; this card carries no formulas.

The rank reversal rate shows how stable the chosen base method is on this dataset and under these types of perturbation; it cannot be generalised to a different dataset or a different base method. A rate close to zero does not mean "this ranking is always correct," only "it was not disturbed under the perturbations tried"; the ranking can still be fragile under a perturbation type that was never tested, a weight change or a data error, say.

A high rate does not declare the base method "wrong"; it shows that the method needs careful use in this kind of decision setting. Which pairwise comparison reversed is more informative than the rate itself; if a reversal occurs frequently between two lower-ranked alternatives while the top-ranked alternative never changes, the most critical part of the decision, first place, is robust.

Thus instead of writing:

"This ranking is safe against rank reversal"

the report should read:

"This ranking did not reverse under the perturbation types tried; a different perturbation type, a weight change, say, should be tested separately"

The method works with crisp data: the same decision table, weights and direction information the base MCDM method requires. You need a choice of base ranking method (RANK-REVERSAL is not itself a ranking method; it tests an existing one); perturbation scenarios to be tested (a hypothetical alternative to add, a near-copy, or an alternative to remove); and enough computational budget for repeated runs. RANK-REVERSAL itself does not ask for weights or direction information; these belong to the base method being tested. A minimum of two alternatives and two criteria is required; the computational burden grows as the number of perturbations grows, because every perturbation requires a full rerun of the base method.

This method is suitable when you want to show, before presenting an MCDM result to a decision-maker, how resistant that result is to small changes in the alternative set. It is especially useful in processes where a new candidate is expected to be added later, a new supplier proposal or a new candidate project, say, to know this sensitivity before explaining the decision.

It should not be used where the alternative set is fixed and will not change, or where the base method already carries no normalisation-driven sensitivity, some elimination-based methods built on fixed reference points, say; in such cases the extra computational burden may not be necessary. The method also measures only sensitivity to the alternative set, not to the weights; how robust the weights are requires a separate sensitivity analysis.

Alternative set may change later, robustness must be shown before the decision → Rank Reversal Analysis

The real question is the robustness of the weights → weight sensitivity analysis (weight perturbation)

Alternative set is fixed and final → additional analysis may not be needed

Weights are needed rather than a ranking → AHP, BWM, SWARA (subjective) · Entropy, CRITIC (objective)

This method's greatest strength is that it makes it possible to present an MCDM result not as a single static table but as a result tested against changes in the alternative set. The method can work alongside any base MCDM method; it can be applied to TOPSIS, SAW, PROBID or any other ranking method. The literature accumulated since Triantaphyllou's (2000) work offers concrete evidence on which method families are more exposed to this sensitivity (Wang and Luo, 2009; García-Cascales and Lamata, 2012; Aires and Ferreira, 2018).

The method's most conspicuous limitation is its computational burden: every perturbation scenario requires a full rerun of the base method from scratch, and this burden grows if many scenarios are to be tried. Second, the result holds only for the perturbation types tried; an untested scenario, a different near-copy, a different removal, say, can give a different reversal rate. Third, a low reversal rate can give the decision-maker a false sense of confidence; the method shows robustness only for the scenarios tried, it offers no general guarantee. Fourth, which perturbation scenarios count as "reasonable" or "likely" is a subjective choice, and this choice can differ from analysis to analysis.

The most common mistake is generalising a low rank reversal rate into "this method is immune to rank reversal"; the rate holds only for the perturbations tried. A second mistake is trying only one perturbation scenario, a single new alternative, say, and generalising the result; different scenario types, a near-copy, removing the worst alternative, can give different results. A third mistake is using RANK-REVERSAL itself like a ranking method and searching for "the most robust alternative"; the method does not rank alternatives, it measures the robustness of an existing ranking. A fourth mistake is declaring the base method "wrong" once a reversal is detected and switching to another method without justification; which method is actually less sensitive must also be tested separately.

The governing principle is this:

The rank reversal rate is a summary of how stable your chosen base method is under the perturbation scenarios you tried; it carries no guarantee for an untested scenario, and the report must show this plainly.