Spearman Rank Correlation (Spearman Rank Correlation)

In a decision process the same alternatives can be ranked more than once: by two different methods (TOPSIS and VIKOR, say), by two different weight sets, or at two different points in time. Spearman rank correlation compares two such rankings and compresses their similarity into a single number between minus one and plus one. Plus one shows that the two rankings are identical; zero shows no systematic relationship between them; minus one shows that one is the exact reverse of the other. The method looks only at the alternatives' rank positions, not at their underlying scores, so it can even compare rankings produced on entirely different scales. Spearman introduced it in 1904 to study the relationship between two measurements in psychology, and it has since become the standard tool for measuring agreement between two rankings in decision analysis.

The idea behind this method is to compress the question "how much do two rankings say the same thing" into a single number. When a decision process tries several methods or weight sets, agreement on the same winner and the same order reassures the decision-maker; when the rankings differ, the decision cannot be made without knowing the size of that difference. Spearman correlation renders that difference measurable. Its philosophical consequence is this: the method does not decide which ranking is "correct," it only reports how far the two rankings overlap. It is a measure of agreement, not a tool for elimination or preference.

The method proceeds through three steps.

First, the two rankings. The two rankings to be compared are placed side by side; each alternative's position (first, second, third) in each ranking is recorded.

Second, the rank differences. For every alternative, the difference between its two rank positions is taken, squared, and summed. An alternative holding the same position in both rankings contributes zero to this sum; the further apart the positions, the larger the total grows.

Third, compression into a single number. The summed squared difference is scaled according to the number of alternatives and subtracted from one. When every alternative sits in the same position in both rankings, the sum is zero and the result is one; when the rankings are exact reverses of each other, the result is minus one.

The formula behind each step is given on the DecisionMind method page; this card carries no formulas.

The result is a single number between minus one and one, and it shows only the agreement between the two rankings; it says nothing about which ranking is correct. A value close to one (0.80 or above, say) points to strong agreement: two methods or two weight sets have produced almost the same order. A value close to zero shows no systematic relationship between the two rankings. Negative values indicate a reversed relationship, where an alternative near the top of one ranking sits near the bottom of the other. The value itself is not a percentage; 0.50 does not mean "fifty per cent alike," only a moderate degree of agreement. With small numbers of alternatives (three or four, say) this figure can shift substantially from just a couple of alternatives swapping places, so a correlation computed on few alternatives is not, on its own, enough for a firm conclusion. For this reason:

"The two methods gave the same result"

should be written instead as:

"The Spearman correlation between the two methods' rankings is such-and-such; this corresponds to such-and-such a degree of similarity between perfect agreement and no relationship"

Spearman rank correlation works with crisp data; all it needs is two separate rankings produced over the same set of alternatives, whether these come from two different methods, two different weight sets, or two different points in time. It does not need the alternatives' own scores, only their rank positions. The two rankings being compared must cover the same set of alternatives; if an alternative appears in one ranking but not the other, the rankings must first be redone over the alternatives they share. DecisionMind holds no extension of this method; when more than two rankings must be compared at once, neighbouring tools such as Kendall's W coefficient of concordance fill that gap.

Spearman correlation is exactly the right tool when the question is how alike the rankings produced by several methods, weight sets or points in time are, over the same set of alternatives. It is also used to compare every scenario of a sensitivity analysis (a weight change, a Monte Carlo run) against the baseline ranking.

The cases where it should not be used are as follows. If the only question is whether the winner (the top-ranked alternative) stays the same, Spearman correlation, which accounts for the whole ranking, carries more information than needed; a simple "is the winner the same" comparison suffices. If the two lists being compared are raw scores rather than rankings (both being numerical measurements), a direct correlation between the scores, such as Pearson's, is more informative than Spearman's. If the question is how far more than two rankings agree all at once, Spearman is limited to pairwise comparison; a separate tool is needed for agreement across several rankings.

General similarity between two rankings is being measured → Spearman Rank Correlation

Only whether the winner changes is in question → Weight Sensitivity Analysis (stability ratio)

Agreement across more than two rankings at once is in question → Kendall's W coefficient of concordance

The consistency of a pairwise comparison matrix is in question, not the similarity of rankings → GCI, HCI

The method's greatest strength is its simplicity: a single number summarises the agreement between two rankings and is easy to explain to anyone. It does not look at the alternatives' own scoring scale, only at rank position, so it can directly compare the rankings of two very different methods, one producing scores between 0 and 1 and another using a different unit entirely. Its computational burden is very light, it can be paired with any ranking method, and it can be recalculated for every scenario inside a sensitivity analysis without difficulty.

Its limitations stem from small sample size and information loss. Calculated on few alternatives (three or four, say), the figure can jump substantially from just a couple of alternatives swapping places; a single correlation value should not be over-interpreted in small samples. The method uses only rank information and ignores the size of the score gap between alternatives; a small gap between first and second looks the same in the ranking as a large one. It assumes the relationship between the two rankings is monotonic; it does not capture a complex, irregular type of relationship. Finally, a high correlation does not mean the two methods are "correct": two methods can also produce a high correlation because they share the same bias.

The most common mistake is reading the correlation value as a percentage: 0.50 means a moderate degree of agreement, not "fifty per cent the same." A second mistake is presenting a correlation computed on a small number of alternatives (three, say) as firm evidence, the way it would be treated in a large sample. A third is interpreting a high correlation between two methods' rankings as "both reached the correct result"; the correlation only shows that the two rankings resemble each other, not that either reflects the truth. A fourth mistake is directly comparing two rankings produced over different sets of alternatives (one with four, the other with five); the correlation is meaningful only over the same set of alternatives.

The governing principle is this:

Spearman correlation measures how much two rankings say the same thing; it does not say which one is correct, and that judgement must come from the data and the method themselves.