Hesitant MARCOS (Li, Geng and Yuan, 2023)

This example is not drawn from the literature. It is a small table built to make the steps of the HF-MARCOS engine traceable by hand. Two alternatives are evaluated on two criteria. The first criterion is "higher is better," the second "lower is better." There is a single expert's evaluation, so the sets are taken directly as input.

The method first takes the mean of every set and converts it into a score: for A1, 0.6 on criterion 1 and 0.6 on criterion 2; for A2, 0.4 on criterion 1 and 0.4 on criterion 2. It then builds the ideal and anti-ideal rows from these scores, ratios them, multiplies by the weights, and combines the two utility ratios to find the final utility degree.

The result reads as follows. A1 is exactly the ideal itself on the first criterion (0.6, because this is the highest score between the two alternatives). On the second criterion, A1's score (0.6) sits closer to the anti-ideal, because this criterion is "lower is better" and A2's score (0.4) is lower. Because the first criterion's weight (0.6) is higher than the second's, A1 comes out ahead.

The hesitation here is this. What happens if the scoring rule takes the set's most pessimistic value (its smallest element) instead of the mean? A1's score on criterion 1 becomes 0.5, while its criterion 2 score stays unchanged at 0.6; A2's criterion 1 score stays at 0.4, while its criterion 2 score falls to 0.3. In this case the ranking reverses: A2 comes first at 0.690, and A1 second at 0.627. The choice of scoring rule directly changes the ranking in this small example.

In the report: "With the mean scoring rule, A1 has the highest final utility degree (0.684). When the most pessimistic scoring rule is used, the ranking reverses; the scoring rule used must therefore be stated in the report."

Source: DecisionMind's HF-MARCOS manifest, validation example. The steps follow the definition of Li, Geng and Yuan (2023). The final utility degrees and the scoring-rule sensitivity were independently recomputed by this card's author with the same algorithm.

A metropolitan fire service will select one of three supplier bids for a new fire-engine fleet. There are four criteria: unit vehicle cost, water-pump capacity, response readiness time and maintenance/spare-parts availability score. Cost and readiness time are "lower is better." The service knows each supplier's performance from field reports on reference vehicles in different provincial fire brigades. Every supplier-criterion pair is therefore given not a single score but a set that varies by province. The weights come from the service's technical board.

The method first reduces the sets to scores, builds the ideal and anti-ideal rows, and ratios and combines them with the weights. Suppose the supplier with the highest pump capacity is also the most expensive. It still comes out first, because the weight on pump capacity is higher than on cost.

The service's hesitation is this. The breadth of the readiness-time set shows that this supplier's vehicles perform very variably from province to province. If the most pessimistic value is taken instead of the mean as the scoring rule, the score of the supplier with the most variable readiness time worsens, and the ranking can change. The service should bear this risk in mind and state in the report why it chose the scoring rule it used.

In the report: "With the high weight given to pump capacity, the supplier with the highest capacity reaches the highest final utility degree. The breadth of the readiness-time set reflects variability by province, and this supplier's ranking may change depending on the choice of scoring rule."

In the illustrative example, saying "the default was used" without stating the scoring rule is wrong. Because the ranking reverses between the mean and the most pessimistic rule, the rule used must be written into the report. The second mistake is, with more than one expert, scoring each expert's set separately before merging the sets, and then averaging; this skips DecisionMind's grouping step and can give a different result. The third mistake is reading A1's final utility degree of 0.684 as "68 per cent suitable." This value only compares these two alternatives on this set's own ideal-anti-ideal axis.