Monte Carlo Simulation (Uncertainty Propagation in a Decision Model)

Monte Carlo simulation is not, on its own, a ranking method; it is a building block used to test how reliable a ranking method's result (TOPSIS, SAW, VIKOR and the like) is under uncertainty in the inputs. In DecisionMind, this building block sits inside whichever "host" ranking method you choose. It takes its name from Metropolis and Ulam's (1949) paper introducing the computational technique based on random-number generation. Its use for propagating weight and performance uncertainty in multi-criteria decision analysis was developed in the later literature (Butler, Jia and Dyer, 1997). Its output is not a single order but a probability distribution showing how often, and in which rank, each alternative comes out.

Most ranking methods assume that the weights and the table's values are known precisely. In reality, weights are usually the average of expert opinion, and a slightly different group of experts could have given slightly different weights. The question Monte Carlo simulation asks is this: "If the weights (or the table's values) had come out slightly different, would the order still be the same?" The method answers this question thousands of times, each time drawing the weights (or performance values) randomly from a chosen probability distribution and re-running the host method for each draw.

This idea has a consequence: Monte Carlo simulation does not make a choice, it measures how robust a choice is. If an alternative comes first in most of thousands of trials, its lead is resilient to small uncertainties in the weights. If an alternative comes first only within a narrow band of weights, its lead is fragile, and the report must say so plainly.

The method proceeds through five steps.

First, defining the uncertainty and finding the baseline order. The uncertainty in the weights or performance values is first defined through a probability distribution, for instance a Dirichlet distribution for weights or a normal distribution for performance values. The host method is also run once with the given nominal weights, giving a baseline order.

Second, sampling. Thousands of random weight sets (or performance tables) are drawn from the chosen distribution. The Dirichlet distribution used for weights ensures that the sampled weights always sum to 1; the higher the distribution's "concentration" parameter, the closer the samples stay to the nominal weights.

Third, repeated computation. For every sample, the host method (TOPSIS, say) is run afresh, and that sample's order is recorded.

Fourth, the rank acceptability index. After thousands of trials, a probability table (RAI) is obtained for every alternative, showing how many times it came first, how many times second, how many times last, each divided by the total number of trials.

Fifth, expected rank. Each alternative's rank probabilities in the RAI table are weighted and averaged to obtain an "expected rank"; the alternative with the smallest expected rank, meaning the one that comes out on top most often on average, is the most robust alternative. The probability of coming first (the top-1 probability) is also reported separately as a further robustness indicator.

The formulas behind each step are given on the DecisionMind method page; this card carries no formulas.

Monte Carlo simulation's output is not a single score but a probability table. The statement "A2 comes first with 92 per cent probability" does not mean A2 is the best alternative in the real world with 92 per cent probability; it means only that, among the weight sets sampled within the defined uncertainty range, the host method placed A2 first in 92 per cent of them. This figure depends both on the chosen uncertainty distribution, how widely the weights were allowed to vary, and on the host method itself; TOPSIS and SAW can produce different robustness tables under the same uncertainty.

An alternative that never comes first but also never comes last may show that it is a "middle-ground" compromise alternative; this is information that does not appear in the nominal (single-weight) result. For this reason, Monte Carlo simulation should answer not only "what is the ranking" but also "how reliable is this ranking."

Thus instead of writing:

"Monte Carlo simulation confirmed that A2 is the best alternative"

the report should read:

"Under reasonable uncertainty in the weights, A2 came first in this share of the trials; this share depends on the host method and the assumed uncertainty range"

Monte Carlo simulation works on a crisp decision table and weight set; it does not take this table as certain, but adds an uncertainty distribution on top of it. DecisionMind holds no separate extension of this building block for another data type. You need the full decision table and nominal weights required by a host ranking method, plus a choice of how uncertain the weights (or performance values) are to be treated, for instance the concentration parameter of a Dirichlet distribution, and how many trials to run, typically a few thousand. If the number of trials is too low, below a few hundred, the probability estimates stay unstable.

At the time this card was written, this building block in the DecisionMind engine still awaits an explicit choice of which host method and which uncertainty model it will run with; the DecisionMind team is completing this connection. The figures shown in this card have been calculated in Python by this card's author, following the method's definition in the literature.

Monte Carlo simulation is a suitable complementary step if your weights come from expert opinion, you know that a different group of experts might give slightly different weights, and you want to show how resilient the order is to that difference. It is also used in high-stakes decisions, public investment or security investment, say, to answer whether "the order depends on a single set of weights."

If your weights are clear and uncontested, or your computing resources cannot support a simulation with several thousand repetitions, Monte Carlo simulation adds an unnecessary layer. If the uncertainty distribution itself, how wide, in what shape, is chosen without justification, the robustness result stays tied to that unjustified choice.

Weights are contested, robustness of the order matters → Monte Carlo simulation (together with a host method)

Weights are clear, a single order suffices → the host method alone

Not a ranking but weights are needed → AHP, BWM, SWARA (subjective) · Entropy, CRITIC (objective)

Not weight but the effect of removing/adding a criterion needs testing → CRITERIA-REMOVAL, CROSS-VALIDATION

Monte Carlo simulation's fundamental advantage is that it exposes the hidden fragility of a ranking that rests on a single weight set; a small gap between two alternatives may look "decisive" at a glance, yet the simulation shows how sensitive this gap is to weight uncertainty. It can be used alongside almost any ranking method (TOPSIS, SAW, VIKOR); it does not change the method itself, it adds a robustness layer on top. Because the result is a probability table, it allows a more honest form of words, "very likely first but not guaranteed," in place of "definitely first."

Its limitations largely follow from its assumptions. First, the result depends on the chosen uncertainty distribution; a narrow distribution gives a false sense of confidence, while an excessively wide one can bring every alternative close together and render the robustness analysis meaningless. Second, the computational burden multiplies a thousandfold over the host method; if the number of trials is kept too low, the probability estimates stay unstable. Third, it models only weight or performance uncertainty; it does not address the uncertainty in the host method's own assumptions, for instance TOPSIS's full-compensation assumption. Fourth, the robustness literature is still developing alongside closely related methods such as stochastic multicriteria acceptability analysis (SMAA), and there is no consensus on a single "most correct" uncertainty distribution (Tervonen and Figueira, 2008).

The most common mistake is saying "Monte Carlo simulation confirmed this" without stating the uncertainty distribution's parameters, how many trials, how wide a range; these parameters are a chosen decision and must be stated clearly in the report.

A second mistake is presenting the top-1 probability as a definite real-world probability; the value only shows this alternative set's relative behaviour under the chosen uncertainty model. A third mistake is keeping the number of trials very low, for instance under a hundred, and assuming the results are stable. A fourth mistake is applying the robustness analysis to the weights alone and never assessing the measurement uncertainty in the performance table.

The governing principle is this:

A Monte Carlo simulation result is a summary of the uncertainty distribution and the number of trials you chose; if the distribution is not realistic, the robustness claim is not realistic either, and the report must show this.