The Problem with PbR

As ideas continue to be sketched on the backs of envelopes as to the future of probation and how the MoJ extricates itself from the omnishambles created by Grayling, here's an interesting blog post about the failings of PbR on the NCVO website:-

PAYING FOR DATA PRODUCTION: THE PROBLEM WITH PAYMENT BY RESULTS

It is important that questions are raised about the effectiveness of payment by results (PbR), as this policy has avoided scrutiny for much of its lifetime.

Unfortunately, the recent treatment of the issue in the Economist only scraped the surface of the damage that PbR has done to the delivery of effective services in the UK. While it cast PbR as an innovative funding mechanism between the state and private outsourcing companies, the voluntary sector has been extensively drawn into the tangled web of PbR experiments at local and national level.

In particular, the article fails to engage with the core problem of PbR – that it inevitably creates perverse incentives. The perverse incentives are not an unfortunate by-product of PbR programmes, nor are they a technical problem that can be fixed with better metrics or measurement regimes.

Misguided incentives

PbR inevitably creates perverse incentives for two reasons. Firstly, the necessarily simple payment metrics upon which all PbR programmes depend (3rd Commandment of PbR – ‘thy metrics shall be simple’) are simplified, top-down targets which force providers to ignore the complex and messy lives of the real people with whom they work. They force providers to pursue goals that have been decided in advance, with no ability to respond to changing circumstances or people’s individual needs.

What happens when getting a job with unsocial hours or arduous travel interferes with family routines and the caring responsibilities of a Work Programme client? It doesn’t matter, because that’s the target that must be pursued. It doesn’t matter that being forced to adapt to the employment goal will end up costing the state more in terms of social care, that’s not the target.

Secondly, the article fails to address the most significant problem of PbR – that real life outcomes are produced by a huge range of factors and interventions working together (in technical terms – outcomes are emergent properties of complex systems).

To hold one organisation or programme accountable for producing a result distorts the complex systems by which outcomes are created. Making organisations accountable for results which are beyond their control means that organisations learn instead to manage what they can control – the production of data.

This is exactly what the evidence says happens in PbR programmes, and why PbR programmes often produce good-looking data. It’s because the organisations are concentrating on data production at the expense of actually doing the complex reality of the job that clients need.

What the evidence tells us

All this is made abundantly clear in the conclusion of the National Audit Office report from 2015. Whilst this report is mentioned in the Economist piece, inexplicably, the article does not refer to the report’s conclusion: that PbR should only be used where outcomes can directly be attributed to interventions.

In other words, PbR cannot be used in any complex, real-life situations, where outcomes are the product of complex, inter-related factors most of which are beyond the control or influence of the programmes at hand.

Organisations funded through PbR don’t necessarily produce bad results. Some professionals and some organisations, driven to do a good job by their values, manage to push back against the distorting drivers that PbR imposes on them. However, these people and organisations manage to do the right thing in spite of PbR, not because of it. Again, this is clearly seen in the evidence.

The real motivations

We already have strong evidence from behavioural and occupational psychology about what motivates people to improve, innovate and achieve in their day-to-day work. The findings are consistent across all sectors and types of activity.

Pay matters, in the sense of assuring everyone a decent reliable income to live on, but additional ‘performance-related’ financial rewards don’t incentivise heightened performance – not even in the finance sector!

It’s having a sense of purpose, autonomy and increasing mastery of their skills that makes people perform to their very best. Instead of offering autonomy and flexibility for the creativity of their skills, PbR renders the service practitioner a mere cog in a pre-programmed finance-driven machine by expecting them to repeat a set model of intervention over and over again, as reliably and consistently as possible, to produce the best ‘measured’ result for a maximum turnover of clients.

This industrial model of social support is in fact a recipe for creating even worse performance than the same service might achieve without PbR funding drivers – and the disempowerment of practitioners may be one possible explanation of the significantly worse practice performance in Rikers Island SIB in New York, and in the initial findings from PbR pilots in drug treatment here.

PbR is a destructive, costly farce. It pays organisations to produce data, rather than focussing on the real job in hand. And in so doing, it undermines good practice in social interventions and wastes huge amounts of taxpayer resources.

Dr Toby Lowe is senior research associate at Newcastle University Business School.
Kathy Evans is chief executive at Children England.