Kendall's Tau (Kendall's Rank Correlation Coefficient)

Kendall's tau is not a ranking method; it does not rank alternatives. If you hold two separate rankings produced for the same alternatives, two orders coming from two different weight sets, two orders given by two different methods, or two orders given by two experts, tau summarises how alike the two are in a single figure. Its output is a coefficient between minus one and one: one means the two rankings are identical, minus one means one is the exact reverse of the other, and zero means there is no systematic relationship between them.

The method was defined by Maurice Kendall in 1938 and is statistics's most frequently used alternative to Spearman's rank correlation. In multi-criteria decision analysis it is used to compare the rankings produced by two different methods or two different weight scenarios.

The idea behind Kendall's tau is to evaluate two rankings not as a whole but through pairwise comparisons. Any two alternatives are taken; if the one ahead in the first ranking is also ahead in the second, the pair is counted as "concordant." If the order is reversed, the pair is "discordant." Every possible pair is examined this way, one by one, and the net excess of concordant pairs over discordant pairs is expressed as a proportion of the total number of pairs.

The consequence of this philosophy is that tau looks not at the rankings themselves but at their relative order. The alternatives' scores may differ greatly between two rankings; tau does not see this, it looks only at the question of who is ahead of whom. This makes tau independent of units of measurement and score distributions, but it also means it says nothing about the size of the score difference.

The method rests on a single comparative logic.

First, extracting the pairs. Every possible pairwise combination that can be drawn from the alternatives being compared is listed. With n alternatives, the number of such pairs grows rapidly as the number of alternatives rises.

Second, classifying each pair. Both rankings are examined for every pair. If the same alternative is ahead in both rankings, the pair is concordant; if the order is reversed, the pair is discordant. Alternatives tied at the same rank in either ranking are excluded from this count and treated separately.

Third, the net difference. The number of discordant pairs is subtracted from the number of concordant pairs. This is the raw measure of how far the two rankings move in the same direction.

Fourth, scaling. The net difference is divided by the total number of pairs and scaled to between minus one and one. The result is the Kendall tau coefficient.

The formulas behind each step and the intermediate tables are given on the DecisionMind method page; this card carries no formulas.

The tau coefficient shows how far two rankings move in the same direction at the level of pairs. If tau is one, the two rankings are identical; if it is minus one, one is the exact reverse of the other; if it is zero, concordant and discordant pairs balance each other out. Tau is not a percentage such as "what share of the rankings is the same"; it shows the net rate of agreement between pairs, not how many alternatives exactly coincide between the two rankings.

The magnitude of tau also depends on the number of alternatives. In comparisons with very few alternatives, three or four, say, tau can take only a handful of discrete values, because the number of possible pairs is itself small; in that case even a small change in order shifts tau considerably. As the number of alternatives grows, tau moves on a finer scale.

Thus instead of writing:

"Tau came out at 0.33, so the rankings are 33 per cent the same"

the report should read:

"Tau came out at 0.33; this is the net agreement rate among pairs of alternatives, and it shows that the two rankings partly coincide but are not identical"

Kendall's tau works with crisp data; its input is not a numerical decision table but two rankings produced for the same alternatives. DecisionMind carries no separate extension of this method; if the agreement among three or more rankings is to be measured, Kendall's W is used instead.

You need two separate rankings for the same set of alternatives, or two score lists that can be converted into rankings. If tied alternatives exist, this must be handled separately, because the classic tau does not count ties. A minimum of two alternatives is required; for a meaningful interpretation, three to twelve alternatives work comfortably, and with more alternatives tau can draw finer distinctions.

If you want to summarise, in a single figure, how closely the rankings produced by two different weight scenarios, two different methods, or two different experts coincide, Kendall's tau is exactly the right tool. Because it is less sensitive to small changes in order than Spearman's rank correlation, it is preferred where extreme values affect the ranking.

There are also cases where it should not be used. If the joint agreement of more than two rankings is to be measured, tau is locked into pairwise comparison, and Kendall's W should be used instead. If it is the magnitude of the underlying score differences, rather than the rankings themselves, that needs comparing, tau does not carry this information; the scores themselves should be compared. If the alternative set itself can vary, with some alternatives differing each time, and the effect of that variability is of interest, Bootstrap Resampling is appropriate, not tau.

Want to summarise the agreement of two rankings in a single figure → Kendall's tau

Want to measure the agreement of three or more rankings → Kendall's W

Score differences, not rankings, are to be compared → the scores themselves, not tau

Robustness to randomness in the alternative set is of interest → Bootstrap Resampling

Many tied alternatives exist → a tie-corrected form such as tau-b is required

Kendall's tau's most important strength is that it works purely from rank information, without making any distributional assumption. It is less affected by extreme values than Spearman's rank correlation, because it looks not at how extreme an alternative's score is but only at who is ahead of whom (Noether, 1981). Its interpretation is intuitive: "the net agreement rate among pairs of alternatives" is easy to explain to a decision-maker. Its calculation is simple, and anyone holding two rankings can apply it.

Its limitations arise from this same simplicity. First, tau compares only two rankings; measuring the joint agreement of three or more requires Kendall's W. Second, in comparisons with few alternatives, tau can take only a handful of discrete values, so a small change causes a large jump in tau. Third, tied alternatives cause problems in the classic tau formula; corrected forms (tau-b, tau-c) are needed, and this card describes only the basic tau. Fourth, tau does not see the size of the score differences; whether the gap between two alternatives is very small or very large, tau stays the same as long as the order is the same (García-Cascales and Lamata, 2012, make a similar warning for ranking methods).

The most common mistake is reading tau as a percentage. The sentence "tau is 0.33, so the rankings are 33 per cent the same" is wrong; the correct statement is that tau is the net agreement rate among pairs of alternatives.

A second mistake is comparing more than two rankings pairwise with tau and trying to average the results; Kendall's W should be used directly in that case. A third mistake is ignoring tied alternatives and applying the classic tau formula regardless; ties must be handled with a corrected form. A fourth mistake is interpreting a tau value from a comparison with very few alternatives, three, say, with the same precision as a tau value from a comparison with many alternatives; with few alternatives tau can take only a handful of discrete values.

The governing principle is this:

Kendall's tau measures how far two rankings move in the same direction at the level of pairs of alternatives; it compares only their order, not the size of the score difference, and the report must state this distinction plainly.