Performance-based resource allocation – a cautionary tale

Multilateral development institutions often use mathematical formulas based on a list of performance criteria to inform the allocation of development assistance or environmental funding. This is called performance-based allocation (PBA) and it is assumed to be more objective and transparent than other approaches. However, the way the formulas are designed can have material and often unintended consequences on the allocation of funds.

This paper highlights a particular problem called the ‘cardinal fallacy’. It relates to using ordinal rankings (best, good, bad, worst) as if they had cardinal meaning (i.e. can be manipulated mathematically, say added up).

The authors demonstrate this with the formula that the World Bank uses to allocate concessional funding by the International Development Association (IDA). A seemingly innocuous change in the arbitrarily chosen numerical scale for the ordinal performance rankings, say from the current 1, 2, 3, 4, 5, 6 to 0, 1, 2, 3, 4, 5 – makes a material difference to the aid allocation to individual countries. In IDA’s fiscal year 2017, this change would have seen a reallocation of funding in the order of US$750 million. Countries like Afghanistan, Haiti and Yemen could see their IDA allocation change by more than a third in such a scenario.

Key points for decision-makers

• Multilateral development institutions now disburse more than US$20 billion annually, mostly using performance-based allocation (PBA).
• PBA frameworks are popular among funding agencies because of their perceived objectivity. Measurable criteria are thought to ensure that funds are directed to the most deserving recipients.
• However, the authors of this paper raise a methodological concern that casts doubt over the objectivity and robustness of PBA frameworks. The problem arises when frameworks fail to distinguish between cardinal performance measurement, where differences between performance levels possess a meaningful quantitative interpretation; and ordinal performance ranking, where they do not.
• Country performance assessments typically involve a set of broad categories, which are evaluated using ordinal criteria. Crucially, this process treats ordinal performance levels as if they were cardinal in nature, thus committing what the authors call the ‘cardinal fallacy’.
• The authors show that committing the ‘cardinal fallacy’ of treating ordinal structures as if they were cardinal in nature can have profound real-world implications in performance-based resource allocation frameworks.