ROC (Rank Order Centroid)

You have criteria and an expert who can order them by importance, but who does not want to, or cannot, give a number for "exactly how many times more important is this criterion than that one." ROC is built precisely for this situation. It asks the expert for nothing but an ordering: which criterion is most important, which is second, which is last. From this ordering, it computes a weight vector directly with a closed formula; it asks no further questions and requires no iterative process. Barron derived this formula in 1992 as the centroid, the centre point, of the rank-constrained weight set — the region formed by every weight combination consistent with the rank. The output is a weight vector that sums to 1. ROC does not rank; it distributes weight across criteria.

The idea behind ROC is this: infinitely many weight combinations are consistent with a given ordering; assigning 0.90 to the most important criterion, 0.09 to the second and 0.01 to the last is consistent with the ordering, and so is assigning 0.40, 0.35 and 0.25. ROC takes the average, the centroid, of this infinite set. So when the only information you have is the ordering, it gives the most "typical" weight set consistent with that ordering — neither one that runs to the extremes nor one that clusters in the middle.

This idea has a consequence. Because ROC uses no information beyond the ordering, it needs no subjective judgement at all; the only thing the expert supplies is the ordering. But precisely for this reason, it never sees the true magnitude of the importance gap between criteria. Even when two criteria are almost equally important, if one is placed first and the other second, ROC still puts a clear weight gap between them, because the formula knows only the order, not the true distance between them.

The method proceeds through a single step.

First, computing the rank centroid. Among n criteria, the weight of the criterion at rank r is found by averaging the reciprocals of all ranks from that rank onward (including itself). The most important criterion (rank 1) receives the highest weight, because the calculation draws on the reciprocal of every rank; the last criterion (rank n) takes only the average of its own reciprocal and carries the lowest weight. More than one criterion may not share the same rank; ROC requires a strict ordering.

At the end of this single step, a weight vector that sums to 1 is obtained. The formula for this step, the intermediate values and the citation format are given on the DecisionMind method page; this card carries no formulas.

A weight shows the average of the weights consistent with a criterion's position in the ordering; it does not measure the criterion's true magnitude of importance. The most important criterion's weight always comes out highest, but the size of this weight depends only on how many criteria there are and where the criterion sits in the order; it does not depend on whether the true importance gap between two criteria is small or large.

This also shows a limit of ROC weights. Even if the expert regards the first and second criteria as almost equally important, because the ordering forces a choice, ROC places a fixed gap between them, one determined by the formula. This gap can be pronounced in small alternative sets. The report should therefore state that the weights come from order alone and may not reflect the true importance ratio.

Thus instead of writing:

"The ROC analysis showed this criterion to be three times more important than the other"

the report should read:

"These weights are derived solely from the ordering the expert supplied; they carry no claim about the true importance ratio between the criteria"

ROC works with crisp data: a single rank number for each criterion, with no ties. DecisionMind holds no extension of this method; the base rank-centroid formula is used on its own.

You need the list of criteria to be weighted and a strict order (ties are not permitted) in which the expert has ranked these criteria from most to least important. The method produces weights, it does not ask for weights from outside; it needs no alternative data. A minimum of two criteria is required; three to twelve criteria work comfortably. As the number of criteria grows, the weight gap between the most and least important criterion widens, because the formula sums the reciprocals of all the ranks.

ROC is a suitable choice if the expert can confidently order the criteria but does not want to give pairwise comparisons or direct numerical judgements, and a fast, uncontroversial weight set is needed. It stands out under time pressure or when the expert is willing to give only a small number of judgements.

There are two situations where it should not be used. If the expert also knows the importance ratio between criteria, not just the order, and this information should not be left unused, ROC falls short; direct rating or BWM should be preferred instead. If two criteria are almost equally important yet an ordering is still forced, the weight gap ROC produces can be misleading; equal importance should either be stated explicitly or a different method chosen.

Only order information is available, a fast and simple weight set is needed → ROC

Ratio information beyond order is also available → SWING, direct rating

A consistency check with few comparisons is also wanted → BWM

Card ordering and gap information can also be supplied → REVISED-SIMOS

No expert, weights should come from the data → Entropy, CRITIC (objective)

ROC's most important advantage is its simplicity: a single closed formula, a single input type, the ordering, with no iterative computation or consistency check required. The judgement asked of the expert is among the easiest kinds; ordering criteria carries far less cognitive load than giving numerical ratios. In their comparative study, Barron and Barrett (1996) showed that ROC weights preserve choice quality better than other simple formulas derived from the same ordering information.

Its limitations also stem from this simplicity. First, ROC uses only ordinal information; whether the true importance ratio between two criteria is large or small, the same formula produces the result. Second, ROC's advantage in simulations has not always been confirmed in comparisons with real-world data; Roberts and Goodwin (2002) compared how closely different rank-based approaches approximate true weights and showed that no single formula is best in every case. Third, as the number of criteria grows, the weight gap between the top-ranked and bottom-ranked criterion widens, which also increases the result's dependence on the precision of the ordering. Fourth, criteria of equal importance must still be forced into an order; ROC does not permit ties.

The most common mistake is reading a ROC weight as an "importance ratio"; the weight comes from order alone and does not carry the true magnitude of difference between criteria.

A second mistake is forcing two criteria that are actually close in importance into different ranks and then taking the resulting pronounced gap at face value. A third mistake is reducing existing ratio information (for example, "this criterion is twice as important as that one") down to a mere ordering instead of using it; this discards part of the information already available. A fourth mistake is directly comparing ROC weights computed with different numbers of criteria; the magnitude of the weights also depends on how many criteria there are.

The governing principle is this:

ROC weights are the average of the weight set consistent solely with the ordering; they carry no claim about the true importance ratio between criteria, and if you have ratio information you should move to a method that uses it.