Cross-Validation - k-fold hold-out validation of MCDM decision consistency

Overview

Cross-Validation - k-fold hold-out validation of MCDM decision consistency

Output

robustness score, higher is better

Data

Crisp, complete numeric matrix

Size

2+ alternatives, 3-12 criteria works best

Used for

Sensitivity analysis, uncertainty quantification

How it works

1. 1

   Validate inputs for cross-validation.

   $$\text{CV error}=\frac{1}{k}\sum _{f=1}^k{\text{err}}_f;{\text{err}}_f=\frac{1}{|{\text{fold}}_f|}\sum _{i\in {\text{fold}}_f}|{\hat{r}}_i-r_i|$$

   Stone 1974, (pending PDF page verification)

Look elsewhere when

Assumptions to verify

• A base ranking method is selected
• Computational budget for repeated runs

Edge cases and pitfalls

• Applying CROSS-VALIDATION without verifying this assumption.

  Requirement: A base ranking method is selected

• Applying CROSS-VALIDATION without verifying this assumption.

  Requirement: Computational budget for repeated runs

• Using CROSS-VALIDATION when: Quick analysis needed → defer robustness check.

  An alternative method is recommended in this situation.

How to cite