Weight Sensitivity Analysis

Any weighted decision method (TOPSIS, SAW, VIKOR, whichever) produces a ranking. But that ranking arises from the weights given to the criteria, and weights are usually not exact measurements but estimates set by expert opinion or negotiation. Weight sensitivity analysis interrogates this ranking: if one of the weights changes a little, does the winner change or not? Its output is not a new ranking but a handful of numbers showing how fragile the winner is. It reports how much one criterion's weight must be shifted to overturn the winner (the critical weight shift), in how many of a set of random weight trials the winner stayed the same (the stability ratio), and how similar the rankings are to one another (the average Spearman correlation). Triantaphyllou and Sánchez first defined a sensitivity inquiry of this kind systematically for TOPSIS, SAW and WPM in 1997.

The idea behind this method is simple: a decision outcome is only as reliable as its inputs. If the weights are contested or estimated, the winner produced with those weights is contested too. Weight sensitivity analysis does not answer "which alternative is best"; it answers "how robust is this answer." It does this from two different angles. The first is perturbing the criteria one at a time (treating one criterion as changing while behaving as though the others stay fixed); this shows which single criterion threatens the ranking most. The second is shuffling all the weights at once, at random (Monte Carlo); this captures the more realistic situation in which criteria change together. The philosophical consequence is this: the method produces neither elimination nor preference, it performs an audit. It attaches a label of "how robust is this" to the back of a ranking.

The analysis proceeds through five steps, all of which rest on repeatedly rerunning the chosen host method (TOPSIS, SAW, VIKOR or the like).

First, the base run. The host method is run once with the decision-maker's initial weights. The ranking and scores from this run are stored as the reference point, the base ranking.

Second, one-at-a-time perturbation (OAT). Each criterion is taken in turn. That criterion's weight is moved up and down in small steps (for example, five per cent at a time); the remaining weight is redistributed among the other criteria while preserving their own proportions. Total weight thus always remains at one. The host method is rerun for every step, and that scenario's ranking is recorded.

Third, comparison. Every scenario's ranking is compared with the base ranking. Whether the winner has stayed the same, and how similar the rankings are, is extracted at this step.

Fourth, three summary numbers. All the scenarios are brought together to calculate three numbers: the proportion of scenarios in which the winner did not change (the stability ratio), the Spearman correlation showing how closely each scenario's ranking resembles the base ranking, and the critical weight shift showing how far one criterion must be moved to overturn the winner.

Fifth, the Monte Carlo extension. One-at-a-time perturbation cannot see situations in which criteria change together. The method therefore generates a large number of random weight sets summing to one (drawn from a Dirichlet distribution), reruns the host method for each set, and reports the average Spearman correlation and the average stability ratio.

The formulas behind each step and the intermediate tables are given on the DecisionMind method page; this card carries no formulas.

The three numbers answer three different questions. The critical weight shift shows at what weight the winner will be overturned; a large value (say, 0.30) means the winner will not easily change, while a small value (0.05) means the winner could be overturned in any ordinary disagreement. The stability ratio says in what percentage of the scenarios tried the winner stayed the same; close to a hundred per cent, the winner is robust. The average Spearman correlation compares not just the winner but the whole ranking; close to one, the entire order has stayed almost unchanged, close to zero, the weights are scrambling the order from top to bottom. None of them says "these weights are correct"; all of them answer the question "what happens to the answer produced with these weights if the weights shift a little." Thus:

"The analysis proved that A1 is the best alternative"

should read instead:

"A1 is the best alternative under the given weights; this result holds up to a shift of this much in the weights"

Weight sensitivity analysis works with crisp data and needs no decision table of its own. What it needs is an already-completed weighted ranking analysis: a decision table, criterion directions, initial weights, and a host method result produced with those weights. In addition, two settings are required: the size of the one-at-a-time perturbation step (for example, five per cent) and the number of Monte Carlo trials (a few thousand is usually enough). DecisionMind holds no extension of this method; the same logic is complemented by neighbouring sensitivity tools that work through boundary values or criterion removal rather than weight (Morris, Sobol, critical criterion removal).

If the weights were set by expert opinion, by vote, or through negotiation, and you want to report how robust the winner is, this method is exactly the right tool. It also gives a concrete answer, where several stakeholders propose different weights, to "which proposal changes the ranking."

There are also cases where it should not be used. If the weights are fixed by law, standard or contract and changing them is not actually in question, sensitivity analysis creates an unnecessary impression of uncertainty. If a host ranking analysis has not yet been carried out, this method has nothing to test; ranking comes first, sensitivity second. If the number of criteria is held fixed and only the accuracy of a single input score, one cell, is in question, a measurement-uncertainty analysis is more suitable than this method.

A completed weighted ranking exists, and how robust the winner is needs to be known → Weight Sensitivity Analysis

Interaction and non-linearity between weights are also of interest → Morris screening, then Sobol

How similar the rankings from two different analyses are is being asked → Spearman rank correlation

Not the weights themselves but the consistency of a pairwise comparison matrix is in question → GCI, HCI

The method's greatest strength is that it can be attached to any weighted decision method (TOPSIS, SAW, VIKOR, WPM); it does not alter the host method, it only reruns it repeatedly. The critical weight shift is easily conveyed as a single number and gives the decision-maker a concrete threshold: "the result changes if it moves this much." Offering one-at-a-time perturbation together with the Monte Carlo extension answers, in a single report, both "what if one criterion changes" and "what if they all change together." Its computational load is low; if the host method is fast, thousands of scenarios can be tried in a short time.

Its limitations inherit the limitations of weighted methods themselves. One-at-a-time perturbation does not show interaction between criteria; an overturn that emerges when two criteria change together may not appear in an OAT scan (Saltelli, Tarantola, Campolongo and Ratto, 2002). The critical weight shift shows only the nearest overturn point; it does not show the second or third nearest. The Monte Carlo extension requires a large number of repetitions; if the host method is slow, for example a large ELECTRE or PROMETHEE calculation, thousands of repetitions can take time. Finally, the method does not decide which weight is "correct"; it only shows how fragile the existing weights are.

The most common mistake is reading the stability ratio or the critical weight shift as "the method found the correct result"; these numbers measure fragility, not correctness. A second mistake is doing only the one-at-a-time perturbation and skipping the Monte Carlo step; this misses overturns arising from criteria changing together. A third mistake is choosing a very small perturbation step (say, one per cent) and concluding "the result is very robust"; the smaller the step, the fewer overturns are caught, which is measurement insensitivity, not robustness. A fourth mistake is, having found the critical weight shift, reporting in the end only "robust" or "fragile" without saying which criterion causes this fragility.

The governing principle is this:

Weight sensitivity analysis does not validate a ranking; it says over what range of weights that ranking holds, and the report must state this range explicitly.