Following the publication of results from the Peterborough PbR pilot, there's been much discussion as to what they mean and whether Chris Grayling is correct in saying it had been a success. It's a complicated issue, but the following articles seem to unravel the truth from the spin. This first article is on the Civil Society website:-
Can the Peterborough social impact bond be judged a success?
Last week the first social impact bond, at Peterborough prison, announced initial results. David Ainsworth examines the implications.
A couple of months ago I wrote a really very long assessment of the pros and cons of social impact bonds, and I wasn’t necessarily planning on writing another one. But given that we now have the results from the first tranche of prisoners from the pilot SIB, it seems sensible to look at whether it’s been a success.
What is the Peterborough SIB?
The basic model of a SIB is that the charity agrees with a public funder that it will try to solve a problem – in this case, reducing the number of crimes committed by short-stay offenders leaving Peterborough prison. If the charity succeeds in reducing the problem, the public body – in this case the Ministry of Justice – will make a payment. Otherwise, it pays out nothing.
But the charity doesn’t take the risk. That’s taken by social investors, who front the cash to deliver the service. In this case, 17 charitable foundations jointly put up the money, around £5m. They will make a profit if the intervention works. If the intervention fails, they will lose out.
Where the intervention works or not, with a SIB the delivery organisation is financially secure. In this case, the delivery organisation is the One Service, a special purpose vehicle set up by Social Finance, the organisation which created the idea of the SIB, which works with a number of charities.
The SIB was intended to work with three tranches of 1,000 offenders, but will be halted early after two. This is because of a new national programme, Transforming Rehabilitation, which also uses payment by results, but based on different metrics.
What do we know now?
Well the headline findings of the recent results show that the SIB reduced reoffending by 8.4 per cent (from an eye-watering 155 reconvictions per 100 prisoners to 142 reconvictions), which isn’t enough to trigger a payment to the investors. That needed a 10 per cent reduction. But Social Finance is pretty confident it will generate greater reductions next time. Even if it stayed at the same level, that would be enough for investors would get paid next time because only a 7.5 per cent reduction is needed over all tranches to trigger a payment.
Of course the other news is that they’re stopping the SIB two thirds of the way through, which isn’t a great advert for the model. But I’m fairly convinced that it isn’t practical to run the SIB alongside TR, so while it’s unfortunate that it’s stopping, it’s also probably inevitable. (Whether TR is a good idea is a very different question. My guess would be that it’s based on a good principle which is going to be badly applied, and it won’t work. But that’s another story.)
The fact that investors haven’t made any money yet looks like the key fact to take from the results, but I’m not sure it’s that important. It doesn’t tell you whether the SIB delivered useful outcomes, just whether they hit a pretty arbitrary target.The pricing structure for the first SIB was inevitably pretty finger-in-the-air, because no one quite knew what could be achieved. In order to tempt government into putting up the money, the first SIB made some stretching promises. And even then the Big Lottery Fund had to promise to put up much of the cash to pay for it.
As far as I can see, future SIBs have been priced in a way that’s more generous to the investor, and indeed, several of them have already paid out.
So perhaps this SIB was priced ambitiously and government will have to adjust its pay scales. But there’s plenty of room between the money government saves and the price it paid this time. Government can clearly make SIBs work if it’s prepared to pay the right price.
Does the model work?
The question is not whether this particular SIB made a profit for investors – although I daresay it will – but what it tells us about the viability of the model as a whole.I’ve said before that a SIB is actually two separate things – a service delivery model which gives charities unusual freedom and a funding model intended to move risk away from them. Can we now say that both of these have worked?
One thing to say is that it’s quite hard to measure whether this SIB’s succeeded. Numbers of convictions actually slosh about quite a lot naturally, it appears. This could be down to changes in the law or the economy, to changes in regional culture and demographics, or factors in the local criminal justice system such as the individual judges and magistrates.
Social Finance, the inventors of the SIB and the people running the Peterborough project, believe that Peterborough prison seems to have its own kind of reoffending microclimate which they don’t quite understand.
These changes can swamp the difference a charity intervention makes. All of this makes government quite reluctant to pay out without a high burden of proof. Still, the stats show this SIB has reduced reoffending, compared to a control group. How reliable that is, with reference to the above, is not clear. The statisticians say there’s only a 90 per cent confidence interval. I’m not a statistician, so I don’t know exactly what that means, but I think we can probably say the delivery model, at least, is effective.
Of course this was the first SIB and as a result it’s probably quite atypical. This is the one where they made all the mistakes, but it’s also the one that got all the time and effort. It was designed to a very high specification. The outcomes to be measured were well-calibrated, and the public body sponsoring the programme put a lot of effort in. It had a lot of legal support.
All of this can’t be replicated every time. Some of the other SIBs in existence are not such high-quality work. I suspect it’s hard to get the right metric to measure success. I suspect it’s hard to stop the commissioner trying to direct the service. And I suspect it’s really hard to actually help the people you’re supposed to help. I think in some of the other SIBs, at least one of these things has not happened.Still, this SIB offers evidence that getting good people working on the same project for a long period of time, with freedom to pick the best interventions, is an effective tactic. You’d hope this doesn’t come as too much of a shock, but it seems to have surprised some people.
Funding models
So good delivery model, but I think everyone in the charity sector already liked long term delivery with secure funding before we had SIBs. The more contentious question is whether the SIB is also a good funding mechanism.To be honest, there are problems. First, we don’t know if there are enough people willing to become social investors to make it work. Also it’s expensive to set up, and it needs a lot of expertise to make it work. Also, there are ethical concerns. It distorts behaviour, particularly if your metrics aren’t good, and can lead to creaming off the easy to help and parking the hard cases. Also, like it or not, it marketises disadvantaged people. I suspect if I was a disadvantaged person, I might put up with being marketised in exchange for some help, but I also find it pretty easy to understand why some people find it distasteful.
On the other hand, there are big pluses. First, it moves risk away from charities towards professional risk takers. At least it does if done right. PBR has had mixed results so far moving the risk to the right place. Just look at the Work Programme. Also, it has the potential to bring in new types of capital. It’s already got some cash out of investments in the stock market and into charity, although the amount remains tiny compared to the world of investment, and it’s mostly from charity investors.Most importantly, though, outcomes funding incentivises effective commissioning.Now we know that the SIB’s holistic style of intervention works – long-term, connected to all stakeholders, focused on a broad outcome. In an earlier piece, I suggested maybe we should keep the delivery model and just grant fund it. This met with a lot of support, but I don’t think government is going to do it. Outcomes finance – whether a SIB specifically or something similar – seems to trigger a willingness in government to then take a hands-off approach to bodies they fund. We’d like them to take the same approach with contracts or grants, but they won’t. I don’t know that public bodies are completely wrong, either.This article by Julian Corner is on the NewStart website:-
After all, they do need to protect their money, and it takes a lot of trust just to say to someone – ‘here’s some money, get on with it, we trust you’, which is effectively what they’d do if they grant-funded something like the Peterborough project. “Poor implementation always trumps a good model,” says Steve Goldberg, an American social investment consultant. Meaning that it doesn’t matter how good your intervention is, if your people are rubbish.
But if you only pay if the outcome is achieved, there’s no trust necessary. So long as you’ve designed your metric right – and that’s a big if – you either get what you want and save money, or you don’t get what you want and save nothing. Also, you pay when the saving’s achieved, not when you agree the deal. And in government, that’s important. So whether the sector likes outcomes finance or not, it might be the price it has to pay to get a good delivery model.
Transforming Rehabilitation is being dressed in sheep’s clothing
Initial results from the world’s first Social Impact Bond (SIB) were announced last week. The Peterborough SIB’s One Service achieved an 8.4% reduction in the reoffending of prisoners sentenced to less than 12 months. This result exceeded the 7.5% minimum required over the life of the project for investors (of which LankellyChase is one) to get their investment back, but fell short of the 10% stretch target needed to trigger an outcome payment this time around.
Inevitably a great deal of the commentary that followed the results – including whether 8.4% was a success or failure – ranged along the battle lines drawn by the wider Payment by Results (PbR) debate. The landscape has changed radically since the SIB was first dreamt up, so that it gets cited as evidence for or against reforms that its creators could not have anticipated. The uses and abuses of its name have been legion and nearly all opportunistic. All of which obscures any attempt to judge the merits of the SIB on its own terms.
Background to the SIB
Some of the commentators who have been quick to dismiss the SIB would do well to return to the original problem it was trying to solve – and perhaps they should answer for how they would have solved it. The absence of support for prisoners sentenced to less than twelve months has been a nonsense and a disgrace for decades. Reviews have come and gone recommending change, such as the Halliday review Making Punishments Work (2001) and the Social Exclusion Unit report Reducing Reoffending by Ex-prisoners (2002). It has been abundantly clear that the social and financial costs of releasing prisoners into destitution are incredibly high, and yet no one has been able to work out where the additional resource should come from to supervise so many additional people.
It is worth recalling that, until recently, every additional responsibility on public services has been issued with a substantial price tag. The idea that the Probation Service might reconfigure existing resources was complete anathema and fiercely resisted. As such, the Government’s main change lever was to pour in extra cash. This is why we became obsessed with piloting ‘invest to save’ approaches. The hope was that if we could prove that supporting the resettlement of short term prisoners reduced their reoffending, and if we could also prove that this saved the Exchequer money, perhaps the Treasury would find a way of expanding the public money available to resettlement services.
Many pilots have come, gone and been forgotten. Quite ambitious regional schemes funded by Europe were set up only to be dismantled. None of these (in marked contrast to the SIB) caused the slightest stir among policy makers or other opinion formers. Such pilots have become almost piecemeal stopgaps rather than serious attempts to model change. A great deal of money has already been spent in this vein to little or no effect.
What did all these failed initiatives have in common? All had at least one of the following limitations: short timescales, poor resourcing, tiny and/or partial cohorts, poor or no through-the-gate support. Even when they got these elements right, they didn’t have anything like the rigour around data to nail the cash saving argument sufficiently. And all were characterised by predetermined inputs and outputs: numbers found a hostel on release; numbers attending a job centre interview, numbers referred to drug treatment, numbers starting or completing a cognitive behavioural programme.
The input/output model has acted as a particularly dead hand. As so often with social policy, the outcome (reduced reoffending) could not be contractually controlled and so it was reduced to proxy measures which appeared to correlate closely to it and could be counted. What this inevitably drove was (a) a whole lot of box ticking whereby providers were incentivised to chase numbers of inputs and outputs but not people and (b) a whole lot creaming and cropping, with the easiest to help being helped first. Classic stuff, and especially problematic for pilots trying to test the effectiveness of services for people with multiple needs.
You might wonder why the proven capabilities of the Probation Service, who after all were supervising all other offenders, weren’t trusted with the task. The expense of extending a public service aside, the levels of reoffending of released prisoners under Probation supervision have never been its strongest suit. The review of Probation’s resettlement practice and prioritisation in HM Inspectorates of Prisons and Probation’s Through the Gate (2001) wasn’t exactly a ringing endorsement. And those voluntary sector organisations who had been attempting to fill the gap for years were chomping at the bit to take the challenge on themselves and, candidly, were clear they could do it better.
What the SIB offered
Tens of thousands of victims later, with all the misery and cost that this implies, Social Finance entered the fray proposing a Social Impact Bond mechanism. What immediately recommended the SIB was that it offered a method of unsticking the seemingly intractable resourcing problem. The tax payer would only foot the bill if the cost was offset by savings generated through improved outcomes. Whether or not you believe that a SIB mechanism could underpin an entire resettlement system, at the very least the financial circle was squared so that an ambitious pilot could be afforded. And critically, the Government was tied into validating and responding to its results.
Several other things recommended the SIB: (i) the timescale (6 years) and scale (3,000 prisoners) allowed the service to be tested thoroughly; (ii) the cohort was the entire short term prisoner population of HMP Peterborough matched to a control, so cherry picking was avoided; (iii) the scheme was conceived as a fully resourced prison to community service; and (iv) the outcome payment mechanism meant that the initiative didn’t have to play the input/output game, and was free to focus on and follow the person, moving resources between services depending on emerging evidence of need. Significantly, the financial mechanism drove an entirely new level of rigour around data which would allow a clear assessment this time of whether the service was working and whether it was generating cashable savings.
In other words, its design overcame all of the limitations that had dogged so many previous pilots. As a result, the workers have been freed up to contend with and learn from the real challenges of the lives of short term prisoners. They have been able to build relationships of trust and have moved well beyond previous models by engaging people with lived experience to support the service and offer mentoring. They have been able to make mistakes, learn and change the service, and have had a clear incentive to do so. All the things that you hope for from a public service but which we frequently find so hard to get right, especially when the service landscape is highly complex. I have visited the service and thought it was the best realisation of a short term prisoner resettlement service that I have seen, with strong values and hard won expertise running through it.
All of which speaks to the fact that how you fund a service fundamentally determines the quality and effectiveness of the results that it can achieve. I write this in the full knowledge that many PbR mechanisms lead to all manner of perverse consequences and gaming behaviours. In the SIB, however, Social Finance seem to have found a way of shaping and protecting a space in which front line workers can focus on doing the right thing, not just through gut instinct or individual judgement, but through ongoing iterative data gathering and learning.
Conclusion
What conclusions can we draw? Social Finance, to their credit, have been very cautious. We only have results from the first cohort, hence it is still far too early to pronounce on its cost benefit and replicability. 8.4% is certainly an encouraging reduction, especially as engagement levels are continuing to grow since the first cohort; but even when later results come through there will still be a lot of cost data that will need to be crunched. There is also a lot more analysis needed on the relationship between the financial model of the SIB and the practice model of the One Service. Is the SIB mechanism the only model that could incentivise the same behaviours and performance on the ground? We just don’t know.
No such caution has been evident in the Government’s response. The SIB was being repeatedly cited by Justice Ministers as evidence for their Transforming Rehabilitation (TR) reforms long before its initial results were available. In which case, they were never actually interested in what the SIB might teach them. And this is seen in the way that TR appears to be rewarding results, with a rigidity that will make it very hard for the Community Rehabilitation Companies (CRCs) to emulate the crucial flexibility of the SIB. What they are rolling out is not what is being piloted.
The full resourcing of resettlement which the SIB made possible through the outcome mechanism is also going to be exceedingly tough to replicate under TR. Short term prisoners have simply been added to the responsibilities of the CRCs without any new resourcing. Following further public spending cuts, the CRCs’ budgets will be significantly lower than the former Probation Trusts and yet they will have far higher caseloads. Once short term prisoners are subject to statutory license on release, we face the prospect of overstretched staff reduced once again to the bureaucratic enforcement of sentences. In other words, reconvictions triggered by breached licenses are set to sky rocket.
Within this much tighter financial context it was vital that incentive structures forced CRCs to make short term prisoners a priority. News emanating from contractual negotiations, however, suggests that penalties and rewards are stacked towards managing the public safety risks posed by more serious offenders. This is understandable, but TR was justified on the basis of reducing high reoffending rates. Given that the SIB was dreamt up to address the existing system bias towards public safety, then this is simply a reversion to type.
TR is being dressed in the sheep’s clothing of the SIB, but risks systematically disabling most of what the SIB has been trying to achieve. For those of us who have been trying for years to find a workable way of resettling short term prisoners, this is quite a blow. Whatever your views on the PbR mechanism that underpinned the SIB, here was a scheme that was finally grappling with the core questions of what it takes to do the job properly. It is deeply frustrating, to put it mildly, to see the Ministry of Justice welcoming the Peterborough SIB results when in fact they should be explaining why they have chosen to ignore the opportunity that the SIB represented.
It is not too late in the TR process to get some of this right. Peterborough type schemes could be stimulated elsewhere in the system through enhanced incentive structures. If not, and if MoJ drops this ball completely, it is going to be very interesting to see how they welcome the next set of results in 2016 when TR is well under way.This article was signposted from a comment on the blog:-
NOTE ON PETERBOROUGH SOCIAL IMPACT BOND PILOT AND TRANSFORMING REHABILITATION PROGRAMME
A) INTRODUCTION
Peterborough Social Impact Bond Pilot
The Peterborough SIB was set up by Jack Straw MP as Home Secretary in 2010 and is an outcomes based model. Social investors provided £5m to fund interventions to reduce reoffending among three cohorts of 1000 short-sentenced male prisoners leaving Peterborough prison. Investors will receive a return if reoffending is reduced by 10% per cohort compared to a national control group. The service was due to last until 2017 – a 7 year period.
The “Phase 2 Report from the Payment by Results Social Impact Bond Pilot at HMP Peterborough” by Emma Disley and Jennifer Rubin RAND Europe May 2014 shows a generous funding model:
The Peterborough Social Impact Bond will now continue in its current form until June 2015 when the delivery of support for the second cohort is due to end. Results and related outcomes payments are expected in the summer of 2014 (first cohort) and in 2016 (second cohort).
Transforming Rehabilitation Programme
Under the Government's plans 35 Probation Trusts will be replaced by 21 'Community Rehabilitation Companies' (CRCs). CRCs will be responsible for the supervision of most individuals on community sentences and on release from prison.
The contracts to run the CRCs are currently being put out to tender by the Ministry of Justice. The successful bidders are expected to be large private companies mostly, awarded relatively long contracts of between seven and ten years in duration.
Alongside the CRC structure, a new National Probation Service (NPS), responsible for the supervision of so-called 'high risk' individuals, as well as some technical functions such as risk assessment and pre-sentence reports, will be established. This means that in place of a single probation service, operating locally, there will be two probation services (the new National Probation Service and the contracted provider) in every locality delivering similar services side by side and sometimes via one another.
The National Association of Probation Officers, UNISON and the GMB Unions all oppose the £800mn Transforming Rehabilitation programme to reduce Regional Probation Trusts from 35 to 21 and to invite tenders for supervision of more than 200,000 offenders each year. ACEVO and NCVO have received £150,000 as part of £500,000 Voluntary Sector support funding to assist "Voluntary Community Sector groups successfully bid as possible mutuals for rehabilitation contracts to manage and reform offenders”. It is very difficult not to interpret this as ACEVO and NCVO encouraging Voluntary and Community groups’ bidding to take union members' jobs and ultimately reduce their terms and conditions
The 21 Ministry of Justice Contract Package Areas have “Indicative Values” ranging from up to £13mn in Norfolk and Suffolk to £72mn for London. Alongside this, the Ministry of Justice published “30 Potential Bidding Entities” that have now passed the PreQualification Questionnaire stage. Many are also Work Programme Prime Contractors.
B) HOUSE OF COMMONS JUSTICE COMMITTEE
The Committee’s Twelfth Report “Crime Reduction Policies: a Co-ordinated Approach? Interim Report on the Government's Transforming Rehabilitation Programme” of January 14 2014 was highly sceptical about the programme:
Data on the results from the Peterborough SIB is incomplete and represents only a rough comparison. The “Phase 2 Report from the Payment by Results Social Impact Bond Pilot at HMP Peterborough” Emma Disley and Jennifer Rubin RAND Europe. May 2014:
D) SECURITY OF FUNDING
The “Phase 2 Report from the Payment by Results Social Impact Bond Pilot at HMP Peterborough” Emma Disley and Jennifer Rubin RAND Europe. May 2014 is comprehensive about the relative security and consistency of funding provided to those delivering the Peterborough service:
Because the Phase 2 Peterborough Evaluation is highly selective and makes no references to Transforming Rehabilitation, it may be concluded that in comparison the national programme is seriously under resourced and takes too many risks.
LESLIE HUCKFIELD RESEARCH SUNDAY 15 JUNE 2014
Finally, a couple of comments:-
*******
Whilst the main effort is certainly to address the conditions the inspector found, there is some evidence in there of our predictions for the future - the focus on those offenders (less than 12 month sentences) at the expense of all others to gain payment from the PbR. So the "success" described last week by MoJ of the pilot was at the expense of good offender management and public protection.
With Catch 22 bidding in South Yorkshire CPA do we see a successful provider or an organisation focused on commercial gain at the expense of safety and the wider resettlement approach? You decide....
(p12) “Social Finance raised £5mn from 17 investors (Social Finance, 2011). The majority of investors were charities or foundations, including the Barrow Cadbury Charitable Trust, Esmée Fairbairn Foundation, Friends Provident Foundation, The Henry Smith Charity, Johansson Family Foundation, Lankelly Chase Foundation, The Monument Trust, Panahpur Charitable Trust, Paul Hamlyn Foundation and the Tudor Trust (Social Finance, 2010”).The Phase 2 Report shows that the Peterborough Pilot is generously staffed:
(p10) “The SIB is used to fund the One Service. This is a voluntary scheme which offers offenders support to address a range of needs and challenges to reduce reoffending. The Service is delivered by a mix of paid caseworkers and volunteers (from St Giles Trust), paid staff and volunteers (from Sova), paid specialist practitioners (from Ormiston Children and Families Trust), specialist trainers (from John Laing Training), recovery workers (from Mind) and gym volunteers (from YMCA). Staff contact offenders in prison to help them prepare for release and offer support for up to 12 months in the community following release”Staffing includes 6 full time caseworkers and 6 volunteers provided by St Giles Trust, 50 volunteers recruited, trained and managed by Sova and other specialist practitioner from other organisations. The Big Lottery Fund and the Ministry of Justice are providing the outcomes payments. Though the amount of Lottery “topping up” support for these payments is not known, under Big Lottery’s £40mn “Commissioning Better Outcomes” programme – supported by the Cabinet Office £20mn Social Outcomes Fund - this can be up to 20%. This means that Big Lottery is providing some underwriting for payments under the programme – which may also be the case at Peterborough.
The Peterborough Social Impact Bond will now continue in its current form until June 2015 when the delivery of support for the second cohort is due to end. Results and related outcomes payments are expected in the summer of 2014 (first cohort) and in 2016 (second cohort).
Transforming Rehabilitation Programme
Under the Government's plans 35 Probation Trusts will be replaced by 21 'Community Rehabilitation Companies' (CRCs). CRCs will be responsible for the supervision of most individuals on community sentences and on release from prison.
The contracts to run the CRCs are currently being put out to tender by the Ministry of Justice. The successful bidders are expected to be large private companies mostly, awarded relatively long contracts of between seven and ten years in duration.
Alongside the CRC structure, a new National Probation Service (NPS), responsible for the supervision of so-called 'high risk' individuals, as well as some technical functions such as risk assessment and pre-sentence reports, will be established. This means that in place of a single probation service, operating locally, there will be two probation services (the new National Probation Service and the contracted provider) in every locality delivering similar services side by side and sometimes via one another.
The National Association of Probation Officers, UNISON and the GMB Unions all oppose the £800mn Transforming Rehabilitation programme to reduce Regional Probation Trusts from 35 to 21 and to invite tenders for supervision of more than 200,000 offenders each year. ACEVO and NCVO have received £150,000 as part of £500,000 Voluntary Sector support funding to assist "Voluntary Community Sector groups successfully bid as possible mutuals for rehabilitation contracts to manage and reform offenders”. It is very difficult not to interpret this as ACEVO and NCVO encouraging Voluntary and Community groups’ bidding to take union members' jobs and ultimately reduce their terms and conditions
The 21 Ministry of Justice Contract Package Areas have “Indicative Values” ranging from up to £13mn in Norfolk and Suffolk to £72mn for London. Alongside this, the Ministry of Justice published “30 Potential Bidding Entities” that have now passed the PreQualification Questionnaire stage. Many are also Work Programme Prime Contractors.
B) HOUSE OF COMMONS JUSTICE COMMITTEE
The Committee’s Twelfth Report “Crime Reduction Policies: a Co-ordinated Approach? Interim Report on the Government's Transforming Rehabilitation Programme” of January 14 2014 was highly sceptical about the programme:
“'an unavoidable consequence of the way this programme is designed is that one element of the competition will be about how cheaply providers can deliver the residual service to enable the maximum resource to be unlocked to “reinvest” in rehabilitative provision for short-sentenced prisoners and others in prison and after their release.”
“Ian Lawrence, General Secretary of Napo, a trade union representing probation staff, characterised the reforms as "...a recipe for disaster. They pose a massive risk to public safety, and are untried, untested and in our view ideologically flawed" and "destroying what works to put in place an experiment." The Probation Chiefs Association similarly argued that "[the] high performing Probation Trust structure would be replaced by a system that is untested and where there is no evidence that it will perform better or deliver efficiencies as intentioned. Richard Monkhouse of the Magistrates' Association observed "We have no evidence that anything [that is proposed] will work". Other witnesses highlighted in particular the shortage of evidence on the application of payment by results to the criminal justice arena and on what drives reoffending rates.'C) RESULTS FROM PETERBOROUGH SIB AND DONCASTER PBR PILOT
Data on the results from the Peterborough SIB is incomplete and represents only a rough comparison. The “Phase 2 Report from the Payment by Results Social Impact Bond Pilot at HMP Peterborough” Emma Disley and Jennifer Rubin RAND Europe. May 2014:
(p2)”These data show that the frequency of reconviction events among offenders released from Peterborough has declined by 11% over the period of the pilot while the equivalent national figures have risen by 10%. The bulletin presents the Ministry of Justice’s best assessment of change at this point in time. The statistical bulletin notes that care should be taken when interpreting the figures because they are not based on the methodology which will be used to assess the success of the Peterborough pilot. For the Peterborough pilot, success will be determined based on 2 comparison with a control group of comparable offenders from across the country, which is not available for these interim results”.Alongside this, a “Process Evaluation of the HMP Doncaster Payment by Results Pilot: Phase 2 Findings” by Evelyn Hichens and Simon Pearce, GVA April 2014 is less encouraging about a scheme which is more similar to that proposed under Transforming Rehabilitation. The Doncaster Payment by Results pilot is being delivered by “The Alliance” – a partnership of Serco and Catch 22. This operation of this pilot is similar to those to be funded under Transforming Rehabilitation.
(p2) “The 12 month reconviction rate for cohort 1 offenders released from Doncaster Prison between October 2011 and June 2012 is 52.6%, based on 1,014 releases. This is a reduction of 6.95% compared to those released between October 2008 and June 2009, and a reduction of 4.4% compared to those released between October 2009 and June 2010.”“The 6 month reconviction rate for cohort 2 offenders released from Doncaster Prison between October 2012 and March 2013 is 43.1%, based on 580 releases. This is a reduction of 2.5% compared to those released between October 2008 and March 2009, and a rise of 1.9% compared to those released between October 2009 and March 2010”.
D) SECURITY OF FUNDING
The “Phase 2 Report from the Payment by Results Social Impact Bond Pilot at HMP Peterborough” Emma Disley and Jennifer Rubin RAND Europe. May 2014 is comprehensive about the relative security and consistency of funding provided to those delivering the Peterborough service:
“(p5) “SIB funding was perceived to be more flexible than traditional sources. Interviewees reported that decisions about spending SIB funds could be made quickly and with fewer restrictions than traditional funding for similar interventions. There are, for example, fewer procurement restrictions and no requirements for SIB funds to be spent within a specified time frame (such as a financial year).
(p8) “Service providers are not normally paid by results and do not bear risk in a SIB. Under a SIB, several providers can deliver services which aim to improve outcomes.”
(p40) “Up-front payment: All providers are paid up-front for service delivery. Staff costs and other overheads are fixed under the contract. Importantly, none of the providers bear financial risk dependent on outcomes. An interviewee from one provider reported that this had allowed them to invest resources into the project”
(p12) “If a 10% reduction in reconviction events is not detected for any of the three cohorts, at the end of the entire SIB period, the three cohorts will be evaluated together. If a reduction in conviction events of 7.5% or more is detected across all 3,000 offenders, when measured against a matched comparison group, investors will be paid an agreed fixed sum per reduced reconviction event”In comparison, Transforming Rehabilitation requires those who bid for a contract to take the risk and provide their own cash up front – which may lead to cutting corners, or an incentive to “cherry pick”. The Peterborough Phase 2 Evaluation described difficulties for cherry picking under this scheme. To prevent cherry picking and parking, prisoners meeting the Cohort Criteria are included in the pilot, whether or not they engage with SIB funded services:
(p4)”No evidence was identified of ‘cherry picking’ and ‘parking’. A risk identified in payment by results programmes is that providers are incentivised to maximise the chance of achieving an outcome payment by prioritising service users who are easiest to help (cherry picking).Transforming Rehabilitation will operate on a similar basis to the Work Programme, with problems of cash flow, risks of ‘cherry picking’ and smaller providers’ being used as “bid candy”. As an indication of funding complexity, the consultancy Rocket Science has produced a “Transforming Rehabilitation Contract Cruncher” to enable calculations for how much they might receive:
“Rocket Science and ACEVO, with funding from Cabinet Office, have produced a free to use tool, the TR Contract Cruncher, designed to help potential Transforming Rehabilitation tier two and tier three subcontractors assess offers from Tier one bidders, and plan for the future, by assessing cashflow projections and risk, and comparing different payment and performance offers from potential Tier one providers.
“The tool will help you to work out unit costs, assess and compare different performance and payment offers, alongside provisional client volumes. This will enable you to consider annual expenditure and income, annual profit and loss and cumulative cashflow”.E) CONCLUSION
Because the Phase 2 Peterborough Evaluation is highly selective and makes no references to Transforming Rehabilitation, it may be concluded that in comparison the national programme is seriously under resourced and takes too many risks.
LESLIE HUCKFIELD RESEARCH SUNDAY 15 JUNE 2014
Finally, a couple of comments:-
"Still, this SIB offers evidence that getting good people working on the same project for a long period of time, with freedom to pick the best interventions, is an effective tactic. You’d hope this doesn’t come as too much of a shock, but it seems to have surprised some people".Bloody hell this is how the Probation Service worked when I started, before national Standards. We are professional we can do it all without politicians and without much of the management we have.
*******
"Most resettlement work was undertaken by a partner organisation called Catch 22 and was linked very closely to a through-the-gate payment by results (PBR) pilot. Work was predicated on a good and up-to-date needs analysis but focused on prisoners sentenced to less than 12 months. Resettlement workers provided a very good service to this group. Outcomes across most of the resettlement pathways were delivered reasonably well but there were significant short-comings in respect of offender management amongst the longer term or higher risk prisoners. While offender assessment systems were up-to-date, risk management was weak. Offender supervision was minimal and reactive. The poor quality and lack of thoroughness of public protection work was a significant concern."An interesting statement from Nick Hardwick HMIP about HMP Doncaster.
Whilst the main effort is certainly to address the conditions the inspector found, there is some evidence in there of our predictions for the future - the focus on those offenders (less than 12 month sentences) at the expense of all others to gain payment from the PbR. So the "success" described last week by MoJ of the pilot was at the expense of good offender management and public protection.
With Catch 22 bidding in South Yorkshire CPA do we see a successful provider or an organisation focused on commercial gain at the expense of safety and the wider resettlement approach? You decide....
Completely agree with comment about what Probation used to do...we were done for when there was an edict saying everyone HAD to be on some sort of programme in order to increase prog completions...why? What happened to needs assessments. Again it was a problem created by policy wonks who didn't know anything about working with our client group, failing to recognise that they have very diverse needs and Think First really doesn't hit the spot for many of them. That level of external interference was the beginning of the end and the failure of senior managers to resist has certainly contributed to where we are now.
ReplyDeleteI will have to read through the blog above in more detail later today, thanks very much Jim for laying it out for us, I hope that others give it as much time ( outside the probation service)
As we all already knew, PbR does not work. The is a reason probation has existed for the past 107 years, and that's because we do a good job. Offender supervision, resettlement and community rehabilitation will not be better delivered by anyone other than probation and so the government should take heed of that old saying 'if it ain't broke don't fix it'.
ReplyDeleteit is so obvious that post-sale (if it gets that far - I still have hope) there are going to be CRCs that bidders will walk away from due to difficulties of one sort or another and what then? What is plan b? Or will the government bankroll them at all costs?
ReplyDeleteWhilst I know there is nobody foolish enough to put their job at risk there are many who are strictly sticking to their job role and have no intention at all of contributing to, for example, 'brain storming' meetings to make CRC work - private companies really are taking on a 'hot potato' and without a wiling workforce they're backing a loser. The fallout from this is going to be immense and Chris Grayling should seriously think about implementing a 'damage limitation' exercise because at the moment he really is cutting his nose to spite his face.
Chris Grayling was asked the same question in Parliament during the debates last November, I think and he made it clear, that the NPS would pick up the pieces, if CRC's failed, or walked away. It's in Hansard!
DeleteThe whole thing reads as 'having established the fastest route fron London to Edinburgh, we can now set off for Paris in the knowledge that we know the best way to get there'.
ReplyDeleteFront Page of Newcastle Journal today - great work by local Napo branch who have been persistent with their PR & it seems to be paying off - I do hope someone involved is a nominee for the National campaigning committee, that still has vacancies (or did a few days ago).
ReplyDeletehttp://www.thejournal.co.uk/news/north-east-news/probation-workers-fear-visiting-hmp-7645560
"The expense of extending a public service aside, the levels of reoffending of released prisoners under Probation supervision have never been its strongest suit."
ReplyDeleteSays Julian Corner is on the NewStart website: - quoted above.
I utterly refute that as far as discretionary parolees, including Lifers are concerned - especially during the period of supervision.
What went wrong was automatic conditional release was introduced, presumably primarily to reduce the costs of custody overall - it totally changed the relationship between probation workers and clients.
Previously - at least before national standards Probation did work voluntarily - albeit sometimes in piecemeal ways with released prisoners not on licence - but sadly resources for this work were very limited and so obviously, gradually more energy was directed to getting the ticks in the boxes that Ministers seemed to want - whether those ticks - for such things as speed of breaching after missed appointments actually had a better impact on reoffending rates than say, giving time to voluntary clients after release who really wanted to engage.
Maybe some statistician can tell me whether this sort of thing was ever monitored thoroughly enough to be meaningful - accepting conviction statistics are inevitably unreliable as an indicator of progress of individual clients for a host of reasons.
This is the only information I have about this second reported suicide for a probation worker: -
ReplyDeleteFrom the "Know the Danger" - Facebook page: -
" Know The Danger
4 hrs ·
Hi KTD, I would just like to tell you about another death in the probation service due to reforms made by Chris Grayling...!!! A probation officer has taken her life... This is two confirmed probation service staff that have taken their own lives due to work.. One lady was a dear friend... Please please POA fight for your members..!!! "
https://www.facebook.com/Knowthedangeruk/posts/622617384521696
It is mainly a prison related Facebook Account and is already attracting many comments.
do we know who the poor woman was or where she was from?
DeleteQUOTE from my last comment with added capitalisation and marks.
Delete"This is the **ONLY** information I have about this second reported suicide for a probation worker: -
Anon at 17.40 might consider enquiring further via the Facebook link I provided - sorry I did not make myself clear originally!
Here is the Facebook link again - where at least one other person has commented Thus: -
"I am a probation officer n can honestly say that I am the most stressed I've been since joking the service 12 years ago. Government have no idea" (I presume the word 'joking' is a typo)
https://www.facebook.com/Knowthedangeruk/posts/622617384521696
Another Poster in that thread has written thus : -
"How awful. I work for the service and that's the fourth one I've heard about in the last 8 months. Tragic stuff."
AT 18.05 there are 30 comments so far.
It does not surprise me that with the all the increasing stress that TR is having on staff that they are taking the ultimate step to deal with the enormous pressures coming down on them. It is so sad that those in charge are working frantically at making a name for themselves at any cost. Workers literally do not seem to have a clue as to what to do next. Someone needs to start paying clear attention as to the impact of fucking TR and it's effect on both our mental and physical health. I hate Graying, and the worst is that he doesn't even seem to care, arsehole, this description can also apply to the managers who are colluding with him.
ReplyDeleteBut there is NO crisis, Chris 'Comical Ali' Grayling has made this crystal clear.
ReplyDeleteHow many times must he repeat this before people believe him?
This is a blog in two halves. The febrile debate about, and opposition to, the relentless Grayling plans to privatise get ever more complex and forensic on one hand, and the raw grief and desolation of a magnificent band of people and their great heritage on the other. I for one am going. Going going gone. But if I EVER encounter That Man on the street, I will spit.
ReplyDeleteI hope you can spit hard.
DeleteIn the same time period as the Peterborough Pilot was going on,;Northumberland Probation Officers and Probation Service Officers contributed toward reducing reoffending in our area by 16%. This was against a target of 7 point something. 16%. Now we can't even keep our offices open.
ReplyDeleteThe Peterborough article mentions a 90% confidence interval. I'm not going to explain what a confidence interval is here because there are plenty of good articles on the internet for those who want to find out more. The convention amongst statisticians is to set it at 95%, and there needs to be a good reason for doing it any other way. Moving it from 95% to 90% amounts to widening the goalposts but couching it in terms only someone trained in statistics would understand.
ReplyDeleteBefore an organisation commits to a major policy change or significant expenditure on the basis of test results, the test needs to have its results evaluated statistically. It's important to get some idea about whether the test results would just have occurred anyway by chance, or whether the experiment actually affected the outcome. Without such analysis, the organisation could end up investing heavily to back a course of action that isn't as effective as they think. And this why it is important to set a confidence interval at the correct level. Lower it (eg to 90% instead of 95%) and you run a greater risk of getting results that simply justify the course of action you'd already decided on before you started the experiment.
It would be interesting to know what the outcome would be if those Peterborough results were re-analysed with the conventional 95% confidence interval. It would be very easy to do. The data doesn't change, you simply adopt a stricter criteria for screening out results that may have occurred by chance.
Thank you. I just took a trip into a world of statistics where they spoke a language I didn't understand, but it was fascinating. Someone called Jeff Sauro wrote this:
Delete"We are 95% confident in the method of computing confidence intervals, not in any given interval. Any given interval we compute from sample data may or may not contain the population average. Even though 95% of samples will contain the interval we don't know if the one sample we took is one of the 5% that don't contain the average."
Given the very purpose & nature of statistical analysis then observations like the following surely undermine everything:
"Social Finance, the inventors of the SIB and the people running the Peterborough project, believe that Peterborough prison seems to have its own kind of reoffending microclimate which they don’t quite understand."
It seems the "outcomes" of this SIB experiment are as flimsy, unrealistic and without basis in fact as the nonsense spouted by Grayling & Spurr etc. who seem to live in their own worlds, with their own microclimates and their own barmy notions.
I know of one staff member who took their life due to the stress of being on ill health capability just prior to TR, I wonder how many others have been lost to the service?
ReplyDeleteWe have judged the effectiveness of something based on giving one group of individuals this treatment against giving another group of individuals, albeit with similar characteristics, no treatment.
ReplyDeleteI'm not versed in any of this but is that line of thinking not fundamentally screwed from the start?
Bizarre way to judge the effectiveness of anything. Like SOTP.