We probably all know the story of the scorpion that was given a lift across a river by a frog. Halfway across, the scorpion stung the frog, condemning them both to death by drowning. As she was dying, the frog asked the scorpion why he’d done it and he replied: “it’s in my nature.” I was reminded of this story by the latest World Bank attack on a universal social protection programme, this time in Kenya. It seems that, whenever World Bank staff see a universal programme, they can’t help but attack it, no matter how successful the scheme. Anyway, here’s the story from last week…
In 2017, Kenya took a large step forward in the design of its national social protection system when it decided to implement a universal old age pension, accessed by everyone on reaching 70 years of age. The introduction of the pension was hailed as a great achievement: indeed, it was the first national universal pension in the East Africa region. It is regarded by the Government of Kenya as the platform for building an inclusive lifecycle social protection system, as outlined in its forthcoming Social Protection Investment Plan (which includes ambitious plans for universal child and disability benefits). The scheme replaced a poverty-targeted pension aimed at those aged 65 years and over who were living in extreme poverty, but which, due to its high targeting errors, excluded the vast majority of its intended recipients. The targeted pension had used a mix of community-based targeting and a proxy means test to identify recipients and analysis by Development Pathways had found that its targeting was little better than random selection.* The move to a universal pension was regarded as a means of ensuring income security for all Kenyans in old age, in particular those living in extreme poverty.
Yet, on 31st July 2019, Kenya’s Standard newspaper published the following controversial headline: “Cash for Elderly going to rich, says World Bank.” The article claimed that, according to the World Bank, for every shilling spent on the universal pension, “73 cents go to the well-off, a situation that has consigned many elderly people to poverty.” The information was taken from the World Bank’s recently released Kenya Social Protection and Jobs Programs Public Expenditure Review. In its report, the World Bank claimed that the universal pension was only reaching 43 per cent of over-70s and that coverage was lowest in the poorest counties. For example, it included Figure B-6 which claimed to show low coverage across most of Kenya’s counties, in particular in some of the poorest counties (such as Turkana, Wajir, Mandera and Marsabit). If this were true, it would have been a shockingly poor performance for a universal pension.
However, the problem with the World Bank’s findings is that they were derived from very poor quality analysis and were riddled with errors. This blog will outline some of the main errors and set the record straight.
Claim non-poor main beneficiaries non-sensical
The first very simple point is that, even if it were true that the very poorest older people were excluded from the scheme, the claim that the non-poor were the main beneficiaries is non-sensical. According to Kenya’s 2015/16 national household survey (KHIBS), around 80 per cent of over-70s were living in households with per capita consumption below US$2.65 per day, which sounds to me like poverty. In fact, only one per cent were living in households with per capita consumption above US$10 per day (yet even at this level of income, it is pushing it to call people ‘rich’). Further, the World Bank failed to recognise that many of the older people living in better-off households have no independent source of income, meaning that they are absolutely in need of an old age pension, if they are to regain their autonomy.
However, the World Bank’s core claim that coverage of the universal pension is just 43 per cent is plain wrong. In reality, coverage is probably between 83 and 100 per cent, depending on the data source used. The lack of precision in the result is due to the fact that the exact number of over-70s in Kenya is not known. The last national census was undertaken in 2009 so the current number of over-70s can only be estimated. The United Nations Department of Economic and Social Affairs, for example, estimates that there are 678,000 over-70s while the national statistics office estimates 977,000. Given that there were – in July 2019 – 849,900 older people registered for the pension of whom around 808,000 were over-70 years, the likely coverage is, therefore, probably in the range given above, and definitely well above the World Bank estimate of 43 per cent. For a new programme, this is a pretty good result.
So, how did the World Bank’s staff make such a mistake?
Their first error appears to have been their estimate of the number of recipients of the universal pension. It seems that the World Bank assumed that there were only 566,000 people registered, which is well below the actual figure of 808,000. It was a simple mistake: the 566,000 were the number of new recipients registered after the programme became universal. The World Bank forgot that there were already around 277,000 older people aged 70+ years on the poverty targeted programme in 2017, who they also needed to include in their analysis. In fact, in Figure B-1 in their report, they actually show more than 800,000 older people registered – so they had the correct figure – but somehow managed to use the wrong figure in their analysis.
The second error made by the World Bank was to use data from the 2015/16 KIHBS household survey to calculate the number of older people nationally and in each county. While the KIHBS data provides an estimate of the number of over-70s in Kenya, it is a very small dataset, especially when compared to the national census. So, it is unlikely to be that accurate. Indeed, while the UN and the national statistics office estimate numbers between 670,000 and 980,000, the KIHBS data gives a population of more than 1.1 million. Clearly, the World Bank had a choice on which population figure to use and, for some reason, decided to use the higher figure, even though it was the least reliable. When they combined this with the artificially low number of recipients, the result was a coverage figure well below reality.
Furthermore, the KIHBS data is not representative at county level and even less representative among over-70s. Indeed, the number of over-70s in the dataset in each county ranges from 10 individuals in Marsabit to 93 in Kitui and Murang’a. As a result of these small numbers, when undertaking analysis at county level – as in Figure B-6 above – the margins of error are very large. So, for example:
- In Nairobi, while the KIHBS data predicted a population of 25,615 over-70s, in reality we can only say, with 95 per cent certainty, that the confidence interval of between 5,824 and 45,407 includes the actual population;
- In Kirinyanga, while the KIHBS data predicted a population of 27,110 over-70s, the 95 per cent confidence interval is between 15,825 and 38,396; and,
- In Murang’a, while the KIHBS data predicted a population of 65,349 over-70s, the confidence interval is between 44,688 and 86,010.
Simple mistakes made on pension performance
Yet, these margins of error were not acknowledged by the World Bank. Instead, they created an image of precision that was entirely unwarranted. We have re-done the analysis using the correct number of recipients and the results can be seen in Figure 1. We show that, in the vast majority of counties, the results were within the margins of error. Consequently, even when the KIHBS data is used, the World Bank conclusion that there is systematic under-coverage of the pension across counties is without foundation. Of course, if we were to use more realistic estimates of the number of people aged 70+ years, the results across counties would look even better.
The World Bank made further simple mistakes in their analysis, which they also used to make the universal pension look as if it were performing poorly. For example, they undertook simulations to estimate the impacts of the pension, finding that the poverty rate among those aged above 65 years of age has fallen by 5 percentage points, in other words, by 15 per cent. In reality, though, as Figure 2 shows, the potential impacts among recipients are much larger, although the scale of the impact depends on assumptions we make on the coverage. So, using coverage estimates of either 60, 80 or 100 per cent, the impact on the poverty rate in recipient households would be between 32 and 41 per cent while the impacts on the poverty gap would be between 41 and 50 per cent.
The benefits from the universal pension are likely to be very progressive. Figure 3 shows the increase in per capita consumption across recipient households, from the poorest to the richest decile, again according to different coverage estimates. Among the poorest households, the increases are very high – between 90 and 120 per cent – while they are still very impressive across those in the middle of the welfare distribution (at around 20 per cent).
Bizarrely, the World Bank also misunderstood the results of its own simulations. They claimed that the impacts were only on the pension recipients themselves and that other household members did not benefit. This allowed them to claim that only 4 per cent of the population – i.e. only older people – benefitted from the pension. Yet, in reality, since they were using a household survey, it was not possible for them to differentiate older people from other household members. In fact, their simulations would have assumed that the cash from the pension is distributed across all household members (and we know from qualitative research that older people often share a high proportion of their pension with others). As a result, they would show impacts on everyone in the household and those benefiting would comprise 9 per cent of the national population, more than double the number indicated by the World Bank. In fact, the simulated impacts of the pension are pretty impressive and much greater than those claimed by the World Bank.
So, why did the World Bank staff make these false claims? Of course, anyone can make a mistake and, clearly, their analyst made a number of rather large ones. Yet, despite a team of 11 working on the report – supported by an additional 7 staff, 3 peer reviewers and 7 commentators – the mistakes weren’t spotted by anyone. It’s an incredible case of myopia!
False and very public accusations
My best guess on what really happened is that, given the World Bank’s antipathy to universal programmes, they were so happy to see results that showed Kenya’s pension in a bad light that their critical senses were shut down. Was this malign intent or just a default reaction? We can’t know. However, an indication that the World Bank may not have had the purest of motives can be seen in Figure 4 in which they compared Kenya’s social security system with other countries and claimed that it was not in line with international practice. Yet, if you look at the comparison countries they are almost all from the OECD, with social security systems that have evolved, in many cases, over more than a century (OECD countries invest, on average, 12 per cent of GDP in social security while Kenya only invests 0.4 per cent of GDP, although it is growing rapidly, largely due to the universal pension). Condemning Kenya’s investment in social security spending by comparing it with countries that have mature systems and a far more expensive array of lifecycle social security programmes that have been built over decades, gives rise to a misleading and short-sighted conclusion. A much fairer comparison would have been with other low and middle-income countries. Yet, this wasn’t done, probably because it would have shown that Kenya has a system that is very much the norm for countries in the early stages of growing their social security systems.
In fact, a recent study has shown how World Bank staff function according to a ‘confirmation bias driven by ideological predisposition, despite having an explicit mission to promote evidence-informed and impartial decision making.’ The Kenya public expenditure review seems to be just another example of this: World Bank staff saw what they wanted to see, rather than what was really there.
So, what needs to happen now? First of all, the World Bank should publicly apologise to the Government of Kenya while releasing a statement to the press withdrawing its false claims and explaining their errors. They then need to fully re-examine its public expenditure review and correct all mistakes (and there almost certainly are many more). Finally, they need to re-write it in an objective manner, focusing on the evidence and detoxifying it of its ideological bias.
It does seem that the World Bank is taking some notice. Following an immediate complaint from the Government of Kenya, the report was taken down from the World Bank’s website. But, these attacks on successful universal programmes – and, indeed, on sovereign countries – continue on a regular basis (including in Thailand, Namibia, Mongolia and across the world). They need to stop.
In conclusion, what we have is, ultimately, an example of fake news backed up by fake analysis. It is completely untrue that Kenya’s pension has very significant exclusion errors; and, the claim that 73 per cent of transfers are going to the rich is ludicrous. Any objective examination of the evidence would show that the pension is improving the lives of the vast majority of over-70s in Kenya – alongside their children and grandchildren – and is a fantastic example for other African countries to follow. Undoubtedly, the programme has some teething problems, but the correct response should be to support the Government of Kenya to further strengthen the scheme rather than unjustly criticising it using false statistics. It is the World Bank that has to change, not Kenya’s pension.
*The analysis will be forthcoming in a soon-to-be-published Social Protection Sector Review.