Average Ranking

A decision process can produce not one but several rank lists: several experts may have ranked the same alternatives according to their own judgement, or several different sub-approaches of a multi-criteria decision method (a ratio system, a reference point, a full multiplicative approach, say) may each have produced their own ranking. Average ranking reduces these several rankings to a single one: it takes the arithmetic mean of each alternative's rank numbers across all the lists, and the alternative with the lower average rank comes out ahead in the final ranking. Its output is not a score or a weight but a combined ranking. It is one of the simplest and most widely used solutions to the rank-aggregation problem.

The idea behind average ranking is this: every rank list carries partial and noisy information about the alternatives; rather than trusting a single list, taking the "middle ground" of all the lists gives a more reliable result. If an alternative comes out very well in one list and poorly in another, the average sits somewhere between these two extremes, placing the alternative neither too far ahead nor too far behind. This makes average ranking a compensatory aggregation rule: a poor rank in one list is offset by a good rank in another. The philosophical consequence is that average ranking does not hide the disagreement between lists but reconciles it; it does not try to decide which list is "correct," it trusts all of them equally.

The method proceeds in a single step.

The single step, averaging the ranks. You have K rank lists, and in each list every alternative has been given a rank number (1 = best). For every alternative, its rank numbers across the K lists are summed and divided by K. The resulting figure is that alternative's average rank. Alternatives are then ordered from the smallest to the largest average rank; the alternative with the smallest average rank finishes first in the final ranking.

The formula for this step and the intermediate tables are given on the DecisionMind method page; this card carries no formulas.

The average-rank value shows roughly where an alternative sits, on average, across these K lists; it is neither a percentage nor a probability. A value close to 1 does not mean "first in every list," it means the average across the lists is close to first; an alternative that is very good in one list and poor in another can also end up with a middling average rank, and this does not mean the alternative is genuinely "middling," only that the lists disagree. If two alternatives' average ranks are very close, that is, the gap is smaller than a single rank step in one list, the order between them can switch places with a small change in a single list.

Thus instead of writing:

"The alternative with the lowest average rank is unquestionably the best"

the report should read:

"According to the average of these K lists, this is the alternative that comes out ahead; if the lists disagree substantially, this ranking can shift with a change in a single list"

Average ranking works with crisp data, and its input is a rank matrix: alternatives in rows, rank lists (experts, criteria or sub-methods) in columns, and every cell holding an integer rank from 1 to m (the number of alternatives), not a raw score. DecisionMind currently carries no fuzzy or other data-type extension of this method; it stands alone in its base form. Every rank list must be checked to ensure it ranks all the alternatives, with no incomplete ranking. A minimum of two alternatives and two rank lists is required; three to twelve rank lists work comfortably. The method neither produces nor requires weights; its input is the rank numbers directly.

Average ranking is suitable when you hold several rank lists for the same alternatives (several expert opinions, several evaluation rounds, or rankings produced by several sub-methods) and you want to reduce these to a single ranking with a fast, explainable rule. If what you hold is raw scores or criterion values rather than ranks, the method cannot be applied directly; each list must first be converted into a ranking. Where there is strong, systematic disagreement between the rank lists, that is, some lists run diametrically opposed to others, looking only at the average hides this disagreement; the disagreement itself needs to be reported.

Several rank lists exist, a fast and explainable aggregation is needed → Average Ranking

The same rank lists, but a vote-counting logic is wanted for the aggregation → Borda Count

A definite superiority between rank lists is sought, elimination rather than compromise is wanted → Dominance Theory

What you hold is raw scores, not ranks → a ranking step first, then average ranking

Average ranking's greatest strength is its simplicity and explainability: its calculation is a single sum-and-divide operation, easy to explain to a decision-maker. It works the same way no matter how many rank lists there are, and it needs no additional parameter or threshold. Where there is no major disagreement between the rank lists, it produces a reliable and stable aggregation.

Its limitations also stem from its simplicity. Average ranking does not guarantee the "best" combined ranking that minimises the total disagreement between the rank lists; it is only an easy-to-compute, widely used ad hoc rule (Cook, 2006). An extreme rank in one list, an expert unfairly placing an alternative last, say, strongly affects that alternative's average when the number of lists is small; this effect weakens as the number of lists grows. The method does not separately report the strength of agreement between the lists; when two alternatives have very close averages, it is not visible whether this closeness comes from genuine agreement between the lists or from a coincidental balancing-out (Dwork et al., 2001).

The most common mistake is reading the average-rank value as a score and comparing it across lists; an average rank is meaningful only within this particular set of K lists. A second mistake is computing the average without first completing a rank list that is incomplete, an expert who has not ranked some alternatives, say; an incomplete ranking artificially distorts the average of the remaining alternatives. A third mistake is declaring a single winner, without reporting the closeness, in situations where average ranks come out very close to one another; small differences can reverse with a change in a single list.

The governing principle is this:

Average ranking summarises the agreement between the lists; where the lists are close to one another it gives a reliable summary, and where the lists are far apart the summary ranking should turn a hidden compromise into a visible disagreement rather than concealing it.