SMAA (Stochastic Multiobjective Acceptability Analysis)

SMAA is a ranking approach developed for situations where you hold a decision table but do not know the criterion weights (or some of the measurement values) with certainty. Most ranking methods ask for a single set of weights and produce a single result. SMAA does the opposite: it asks "whatever the weights turn out to be," and, by randomly sampling thousands of possible weight combinations, counts in how many samples each alternative comes out first. Its output is an "acceptability index" for every alternative: this shows the probability that the alternative comes out first even with no specific information about the weights. Lahdelma, Hokkanen and Salminen proposed it in 1998 for environmental and public-policy decisions where no consensus can be reached on weights.

SMAA's underlying idea is this: in real decisions, there is usually no firm consensus on the importance of the criteria, and different stakeholders argue for different weights. Classical methods ignore this uncertainty and declare a single "winner" with a single set of weights. SMAA instead places the uncertainty at the centre of the calculation: it asks "does this alternative win under every defensible set of weights, or only within a narrow weight range?" If an alternative comes out first across a large share of the weights, this means "whichever weight anyone argues for, this alternative is likely to come out ahead"; if it wins only within a narrow weight range, the choice depends on the preference of whoever argues for that narrow range.

This idea carries a consequence: SMAA is a consensus-seeking method, not one that tries to find a single "correct" weight. Where stakeholders disagree over weights, or where the measurement values themselves are uncertain (given as an estimate range, say), SMAA does not ignore this uncertainty but builds it directly into the model.

The method proceeds through six steps.

First, defining the uncertainty. The decision table's measurement values can be entered as exact numbers or with a margin of uncertainty ("±ten per cent," say). If there is no prior preference for the weights, the "no prior information" assumption is used, giving equal probability to every weight combination.

Second, drawing thousands of samples. SMAA draws a large number of random samples (typically tens of thousands), consistent with the defined uncertainty, from both the measurement values and the weight combinations.

Third, calculating a simple total score for every sample. In each sample, every alternative's weighted total score is calculated using that sample's weights and measurement values.

Fourth, counting the rank probabilities. In each sample, the alternatives are ranked by this score. After thousands of samples, it is counted in how many samples each alternative comes first, in how many it comes second, and so on; the rate at which it comes first is reported as the "acceptability index."

Fifth, finding the central weight. The average is taken of the weights across all the samples in which a given alternative comes first; this answers the question "what does the typical set of weights that makes this alternative come first look like?"

Sixth, calculating the confidence factor and interpreting the result. The calculation is redone with the central weight to check whether this alternative genuinely comes out first; this rate is a confidence factor. Alternatives with both a high acceptability index and a high confidence factor are recommended.

The formulas behind each step, the intermediate tables and the citation formats are given on the DecisionMind method page; this card carries no formulas.

The acceptability index shows the probability that an alternative comes out first across every defensible weight combination. It is not a measure of "certainty": a value such as 0.74 does not mean "this is the correct alternative with 74 per cent probability," but "with no specific information about the weights, this alternative comes first in 74 per cent of the randomly tried weights." The indices of two alternatives need not sum to 1, because a third or fourth alternative may also come first under some weights. A small acceptability index (0.01, say) does not mean that alternative is bad; it only means it stands out within a very narrow weight range.

Thus instead of writing:

"SMAA identified the best alternative"

the report should read:

"With no prior information about the weights, this alternative came first in this proportion of the randomly tried weights; this rate shows how robust the alternative is against weight uncertainty"

SMAA works with probabilistic (stochastic) data: measurement values can be given either as exact numbers or as an uncertainty distribution (a central value with a margin, say); weights, too, are treated not as exact numbers but as a probability distribution (usually with the "no prior information" assumption of equal probability across all combinations). DecisionMind currently has no separate extension of SMAA; its close relative SMAA-2 is offered as a distinct method that considers not just the first rank but every rank in the ranking.

You need alternatives in rows, criteria in columns, a measurement value in every cell (optionally with a margin of uncertainty), and direction information for every criterion. SMAA asks for no exact external set of weights; if there is no information about weights, this is fed directly into the model as "every weight equally likely," and if there is partial information about the weights (for instance, "the first criterion is at least as important as the second"), this too can be added to the model. As the number of samples grows (into the tens of thousands), the results become more stable; working with too few samples makes the result unreliable.

SMAA is suitable when stakeholders cannot agree on the weights, or when the measurement values themselves are uncertain (resting on an estimate, a projection or expert opinion), and the decision-maker wants an answer to the question "which alternative stays strong despite this uncertainty?" Its typical territory includes public policy, environmental decision-making, and multi-stakeholder group decisions.

It should not be used where a clear consensus on weights already exists, or where the decision-maker prefers to work with a single exact set of weights; in that case SMAA's probabilistic output brings needless complexity, and single-weight methods such as TOPSIS or SAW give a more direct answer. Likewise, in simple decisions with very few alternatives and criteria, SMAA's Monte Carlo machinery is a disproportionate effort.

No consensus on weights, uncertainty should be taken into account → SMAA

Weights already clear, a single exact result is wanted → TOPSIS, SAW, VIKOR

Not just the first rank but the probability of the whole ranking is wanted → SMAA-2

Uncertainty lies not in the weights but in the criterion values, a single exact method is preferred → fuzzy or grey extensions

SMAA's greatest strength is that it displays weight uncertainty openly rather than hiding it; it gives the decision-maker an answer to the question "is this alternative strong under every condition, or only under one particular weight preference?" Where stakeholders cannot agree on weights, it shifts the debate from "which weight is correct" to "which alternative is strong across a wide range of weights," and this generally makes consensus easier. It can handle both measurement uncertainty and weight uncertainty within the same framework.

Its limitations stem from the method's probabilistic nature. First, the result depends on the number of samples; with too few samples (under a thousand), acceptability indices can come out unstable, and tens of thousands of samples are needed for a reliable result (Tervonen and Lahdelma, 2007). Second, the "no prior information" assumption (equal probability for every weight) may not always be realistic; where stakeholders in fact argue for weights close to one another, this assumption can paint an unrealistically wide picture of uncertainty. Third, the output is a probability distribution rather than a classical "single winner" ranking; this is a result that is harder to explain to a decision-maker unused to it. Fourth, in its basic form, which uses an additive score function, SMAA carries the same compensatory logic as TOPSIS or SAW; a weakness on one criterion can be papered over by another (Tervonen and Figueira, 2008).

The most common mistake is reading the acceptability index as a certainty rather than a probability; 0.74 does not mean "74 per cent correct" but "comes first in 74 per cent of cases under weight uncertainty." A second mistake is keeping the number of samples (K) too low and assuming the result is stable; the analysis should be rerun with different random seeds to check whether the result changes. A third is using the "no prior information" assumption without questioning it; if stakeholders in fact hold partial information about the weights (a consensus that one criterion outweighs another, say), this information should be added to the model. A fourth is eliminating an alternative outright because of a low acceptability index; a low index can still mean it stays strong within a narrow weight range, and the central weight and confidence factor should also be examined.

The governing principle is this:

SMAA's acceptability index shows how robust an alternative remains under weight uncertainty; it does not declare a definitive winner as though a single "correct" set of weights existed.