As crises spread across the globe combined with increasing pressure on funding, the question of how best to target aid has entered the humanitarian world in a big way. However, while targeting has long been debated in social protection – and continues to be highly contested – humanitarian actors appear to still need to wake up to the issues at stake in targeting humanitarian aid.
Unfortunately, some problematic practices from the world of social protection schemes have already made their way into the field of humanitarian aid, where they have been uncritically adopted.
This is particularly the case regarding the use of the proxy means test (PMT), which selects recipients through a mathematical formula, based on a survey of household assets. The PMT methodology is now widely used in both humanitarian aid and social protection programmes, despite well-known issues.
Research has shown the PMT methodology to be highly inaccurate, with high exclusion errors. Selection is relatively arbitrary, and perceived as such by communities, and the methodology is static so cannot respond to changes in wellbeing. In addition, the lack of transparency often leads to social conflict and undermines accountability.
There are already good examples of the PMT failing in a humanitarian context. In a voucher programme in Burkina Faso, an evaluation showed that the PMT used was no more effective than a random distribution in identifying the most food insecure households. Similarly, an assessment by the World Bank of the Vulnerability Assessment Framework used by UNHCR for targeting in Jordan showed exclusion errors of 58 per cent. Evaluations of PMTs used in social protection programmes around the world has shown similarly high exclusion errors.
The high exclusion errors of all PMTs is particularly problematic in refugee settings, where most people are in need of assistance. In Lebanon, the fear that households may be excluded from assistance by the PMT led to an emergency verification exercise involving household visits to re-assess wrongly excluded households. In the end, the appeals process had to re-incorporate 23 per cent of cases who appealed after being excluded from assistance.
Many complaints mechanisms are unable to effectively handle the very large numbers of complaints generated by PMTs. For example, in Jordan, over 50,000 cases – amounting to around 170,000 people – appealed the proxy means-test targeting and, in the end, 33,000 people were approved for re-inclusion.
Another downside of the PMT approach is that it is very costly and time-consuming to collect the necessary detailed data about households. Typical data used include characteristics and equipment of the house, household assets, coping mechanisms, reported expenditure as well as social and demographic characteristics of the head of household.
New guidelines for targeting that have been developed by WFP and UNHCR highlight the importance of adopting approaches that ensures transparent eligibility criteria. However, when using a PMT there is no way for beneficiaries to know the eligibility criteria. In fact, the targeting formula is often kept secret in order to prevent beneficiaries from ‘gaming the system’.
Qualitative research into the perceptions of refugees of the PMT targeting mechanism used in Lebanon showed that refugees do not understand how beneficiaries were selected. As one person said: “It is luck! The computer picks names and assistance is given to those names”. Even the field staff administering the assistance in Lebanon would sometimes tell people that the programme uses random selection, or would not know what the selection criteria are, adding to the frustration of refugees: “The agency visits, they record 5,000 families on their computer, and they select only 10 tents within this tented settlement. When you ask them why, they say it was the computer’s selection”. This is very similar to the responses found in social protection programmes using PMTs.
The lack of transparency creates mistrust and conflicts between beneficiaries and non-beneficiaries and animosity towards agency staff. In Turkey, an assessment of the e-voucher programme implemented by the Danish Refugee Council (DRC) faced difficulties because of the “widespread, strong negative reaction among households who were assessed by DRC but not selected for monthly assistance – culminating in protests outside DRC offices and harassment of staff”.
The refugees in Turkey reported inclusion errors, for example observing that people with jobs were receiving assistance, and attributed this to luck, negligence from agencies, corruption and deception (e.g. refugees hiding assets). Exclusion errors were also an important concern among refugees, in particular exclusion of large households with no sources of income, and female-headed households.
Because of the high levels of exclusion errors, a report on refugee vulnerability and targeting in Kakuma refugee settlement in Kenya, concluded that proxy means testing would not be in line with a ‘do no harm’ principle.
Targeting according to vulnerability also constitutes a departure from established UNHCR principles and procedures of targeting of humanitarian aid based on protection criteria and there has, therefore, reportedly been some concern in UNHCR about the use of the PMT in humanitarian settings. Nonetheless, the agency has moved ahead with developing a PMT, with assistance from the World Bank, and there seems to have been no public debate about the use of the methodology so far.
Humanitarian agencies need to wake up to the issues involved in targeting and, in particular, question whether it is acceptable to leave selection of beneficiaries to an opaque algorithm, in violation of principles of ‘do no harm,’ transparency and accountability.
Sources on the use of the PMT in targeting humanitarian aid
Bailey, Sarah and Barbelet, Veronique (2014). Towards a resilience-based response to the Syrian refugee crisis: A critical review of vulnerability criteria and frameworks. Overseas Development Institute and UNDP, May 2014.
Gilligan, D.O.; J. Hoddinott; A.R. Quisumbing and M. Sharma (2005). Assessing the Effectiveness of Community-Based Targeting of Emergency Food Aid in Bangladesh, Ethiopia, and Malawi. Linking Research and Action – Strengthening Food Assistance and Food Policy Research, IFPRI and WFP.
Guyatt, Helen; Flavia Della Rosa and Jenny Spencer (2016). Refugees Vulnerability Study, Kakuma, Kenya. UNHCR, WFP and Kimetrica.
Jacobsen, Karen and Armstrong, Paula (2016). Cash Transfer Programming for Syrian Refugees: Lessons learned on vulnerability, targeting and protection from the Danish Refugee Council’s e-voucher intervention in Southern Turkey. UNHCR, DRC, Feinstein International Center and Tufts University .
Lebanon Cash Consortium (2017). Lessons learned from large scale cash-programming in Lebanon 2014-2017. Lebanon Cash Consortium.
Lebanon Cash Consortium (2016). Community Consultation on Targeting. Lebanon Cash Consortium.
Sharp, Kay (2015). Review of Targeting of Cash and Food Assistance for Syrian Refugees in Lebanon, Jordan and Egypt. UNHCR and WFP, July 2015.
World Bank and UNHCR. Improving Targeting and Welfare of the Syrian Refugees.
Sources on the use of the PMT in social protection programmes
Kidd, Stephen; Gelders, Bjorn and Bailey-Athias, Diloá (2017). Exclusion by design: An assessment of the effectiveness of the proxy means test poverty targeting mechanism. International Labour Office.
Brown, Caitlin; Ravallion, Martin; van de Walle, Dominique (2016). A Poor Means Test? : Econometric Targeting in Africa. World Bank.
Cichon, Michael (2018) Proxy means testing: failing both the economics test, and the rights test? Development Pathways.
Freeland, Nicholas (2014). Do targeting techniques tend to be incompatible with the human rights standards of transparency and access to information? UNRISD.
Freeland, Nicholas (2017). Poxy Means Testing: It’s official! Development Pathways
Freeland, Nicholas (2018). The social protection flaw – or how not to win fiscal space for entitlements. Development Pathways.