A magical use of evidence: the World Bank’s State of Social Safety Nets 2018 report

With great fanfare, the World Bank recently released its 2018 The State of Social Safety Nets report, writes Stephen Kidd.

The big question was: could the Bank manage to put together a strong, evidence-based paper without straying into advocacy? More specifically, would the Bank continue its practice of using the report to advocate for poverty-targeting and, if so, would it do so on the basis of solid evidence, or fantasy?

Alas, a quick skim of the report shows the Bank up to its normal tricks, making bold – but incorrect – claims about the wonders of poverty-targeting employing a standard of ‘evidence’ that probably wouldn’t even be acceptable in a primary school.

Take, for example, page 83, where the Bank makes the following bold statement: “Dorfman (2015) shows that despite the challenges of implementing an old-age social pension system targeted exclusively at the elderly poor, the policy has been found to reduce national poverty at almost twice the rate of a universal approach.”

Now, that’s a pretty amazing claim given that we know that: a) targeted schemes have very high errors (see here and here); and, b) transfer values are, in general, higher in universal schemes than in similar schemes targeted at the poor (see here and here). So, we would hope that the Bank has included in the report pretty amazing evidence with which to dazzle and confound us.

As we might have suspected, this is not the case. A quick read of Dorfman’s paper shows that the evidence is beyond weak. Rather than examining real old age pensions from around the world, Dorfman merely recycles an old paper by Kakwani and Subbarao (2005) which undertook simulations to compare fictitious universal and targeted social pensions. Furthermore, Kakwani and Subbarao’s analysis was severely flawed in that they used the patently silly assumption of perfect targeting for their poverty-targeted options, a strategy commonly used by the World Bank as part of its advocacy.

In fact, the Bank offered no empirical evidence at all in support of its claim that poverty-targeted schemes are more effective in reducing national poverty than universal schemes. And, of course, they couldn’t, because they are claiming the impossible. Once the limited budgets, high exclusion errors and lower transfer values associated with poverty targeted social pensions are taken into account, a universal scheme will always have a significantly higher impact on poverty than one targeted at those living in poverty.

However, I don’t want to fall into the same trap as the Bank in not offering empirical evidence. So, Figure 1 compares the coverage of Georgia’s universal old age pension and India’s poverty-targeted programme. As the graphs show, almost no older person living in poverty is excluded from Georgia’s universal scheme while India’s poverty targeted scheme excludes 73 per cent of its target population (all those to the left of the red line). Furthermore, the transfer value of Georgia’s pensions is 28 per cent of GDP per capita compared to 2.2 per cent for India’s [1].

In fact, as Figure 2 below shows, Georgia’s pension is much more effective in increasing consumption than India’s. Among those in the poorest decile, in Georgia the pension is the equivalent of 86 per cent of the consumption of households with older people, while in India the equivalent figure is a mere 7 per cent. And, at the 50^th percentile – where many older people would still be struggling financially in the absence of a pension – Georgia’s pension still makes a meaningful difference to people’s lives, since it is the equivalent of around 50 per cent of household consumption, while the support offered by India’s pension is minimal.

The reality is that universal schemes are more effective in reaching people living in poverty than poverty-targeted schemes and with significantly larger impacts on poverty. The inability of the World Bank to recognise this simple fact undermines the credibility of The State of Social Safety Nets report. We can accept – and respect – the World Bank’s ideologically-driven support for poor relief and low investment in social protection (alongside lower taxes), even if it condemns countries to a second-hand Lada type of social protection. But, what we can’t accept is the Bank’s continuous distorting of evidence to pursue its ideological agenda, one which has confused many policymakers worldwide.

So, as the World Bank’s Spring Meetings are held this week, let’s call on the Bank’s Board – and independent evaluation group – to take a fresh look at the performance and use of evidence by the Social Protection and Labor Group and ensure that, in future, only the highest standards are permitted.

Notes

[1] Transfer values are from Pension Watch (available here). The reference years are 2015 and 2014 for Georgia and India respectively.

[2] Own calculations based on nationally representative household surveys. The surveys are: the Georgia – Welfare Monitoring Survey 2013 and the India Human Development Survey-II (IHDS-II), 2011-12.