"Today's probation stats 13 CRCs have reduced binary reoffending - get PbR payment 2 CRCs have increased & have money deducted."
We can't put off talking about them any longer, but what the hell do they mean? It's all devilishly complicated and I notice Russell turned to a statistical expert Jack Cattell to try and make sense of it:-
PbR figures for first cohort
Last week (27 October 2017) saw the release of the Transforming Rehabilitation (TR) Payment by Results figures for the October to December 2015 cohort. The overall result was encouraging, and defy the view that TR’s radical changes to probation, and the ensuing problems, would result in increased reoffending – though it is very important that I point out that this is just the first set of results of many and overall judgement should be reserved for at least a year. The reoffending rate for all CRCs was 45.6% compared to a 2011 baseline rate of 47.5%. I had to make some (conservative) assumptions to estimate the baseline rate but I think it is also safe to say that the difference was statistically significant, suggesting reoffending rates have reduced under TR. Please see the the note at the end of this blog to understand better how I completed the analysis.
CRC Performance
The chart below describes each CRC’s reoffending rate in relation to the baseline 2011 rate. The grey line represents the range of reoffending rates that would indicate no change from 2011 (the baseline confidence interval). If the CRC’s rate is outside this range, we are confident in statistically terms to state that the CRC’s performance was either better or worse than the reoffending rate achieved in 2011. The green bars represent the reoffending rates of CRCs that outperformed 2011, the orange bars represent those that performed the same as 2011 and the red bars present those that performed worse than 2011.
Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).
Thirteen of the CRCs beat the baseline rate. The best performing CRC was Cumbria and Lancashire, which beat the baseline rate by 8.2% (49.9% to 41.7%). [Editor’s note: Cumbria was the subject of a recent, very positive Probation Inspector’s report.] The nest best was Hampshire and the Isle of Wight which beat the baseline by 5.4% and the third best was Northumbria with a better rate by 4.3%. Two of the CRCs performed worse than the 2011 baseline. Warwickshire and West Mercia recorded a reoffending rate 3% worse than the baseline rate, and South Yorkshire’s rate was 2.8% worse. With most CRCs, however, outperforming the reoffending rate form 2011, the figures are a promising set of results.
Comparing CRC performance
Now that the baseline rates have been published, we can better understand how well each area was performing in 2011 and whether a CRC is now being asked to better good or bad performance achieved in that year. The chart below describes the difference between the actual baseline rate and the 2011 baseline’s OGRS score (in other words their expected rate of reoffending). A negative result in the chart means the area performed better in 2011 than the OGRS score expected.
2011 Baseline Performance
Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).
The charts highlights that six of the CRCs are being asked to beat better than expected performance in 2011 (in other words to be better than good). Whereas other CRCs, notably London and Wales, are being asked to outperform potentially poor performance in 2011. It it interesting that South Yorkshire and, Warwickshire & West Mercia – the two areas that recorded poor performance for TR – are being asked to beat good performance from 2011.
Merseyside and Cheshire & Greater Manchester, however, are equally being asked to beat good performance from 2011 and were able to do so for the October to December 2015 cohort. The OGRS score does not allow for area effects, which will exist and could explain the differences between the OGRS score and the baseline rate. It not possible now to conclude whether payment by results will be easier in some areas than others, but, going forward, I will monitor the impact of whether a CRC is being asked to perform better than good or poor performance from 2011 on their ability to achieve payment by results bonuses.
Editor’s Conclusion
I am very grateful for Jack Cattell’s technical skills in analysing the first cohort of offenders supervised under the government’s new private/public probation service created by the Transforming Rehabilitation (TR) initiative. Jack will be following the performance of the new private Community Rehabilitation Companies via the publication of reoffending rates every three months.
Many will be surprised at the apparent good performance of the CRCs (and it is important to remember that reoffending performance is linked to contract remuneration via the TR payment by results contracts).
However, I would like readers to bear a key fact in mind: These figures relate to the first three months of the CRCs operation when, by general consent (confirmed by HMI Probation reports) there was considerable chaos and confusion, as would be expected in moving from public to private ownership. Most CRCs have taken the best part of two years to implement fully their new models of operation.
There are two consequences to this fact:
On Nov 28th, Jack will be talking more about TR’s PbR results and how rigorous analysis can help probation services and delivery chains to identify how best to reduce reoffending. See here for more details.
Tue 28 November 201714:00 – 16:30 GMT
Transforming Rehabilitation: Learning from the PbR results
Transforming Rehabilitation challenged Community Rehabilitation Companies (CRCs) to reduce reoffending significantly and October will see the publication of the reoffending rates of the first Transforming Rehabilitation cohort.
These results are an important test of the government’s TR agenda and the success of the new CRCs. Some CRCs will be happy with their results, while others will need to improve. The published results also provide opportunities to commission new services for offenders.
Besides the implications for policy and practice, the results also present an opportunity for your organisation to learn how best to reduce reoffending with the tools at your disposable. You might want to increase a CRC’s performance, identify “what works?” and roll that out across all your CRCs, or prove that your specific service will make the difference.
Come join our experts at our TR PbR event, to discuss what needs to be learnt, what is within your control, and how embedding impact analytics in your CRC or service delivery organisation will help you to reduce reoffending.
Speakers:
Professor Darrick Jolliffe, University of Greenwich
“How to measure the quality of relationships between offenders and probation officers.”
Darrick is Professor of Criminology at the University of Greenwich. Darrick is interested in the broad areas of developmental life-course criminology, programme evaluation, prison research and psychology, individual differences and offending. He has developed tools to measure the satisfaction of offenders and the quality of their relationships with probation officers, and has estimated the association between relationship quality and reoffending. Darrick will also chair a discussion after the presentations.
Dr Sam King, University of Leicester
“What is high quality offender management?”
Sam is an expert on offender management and rehabilitation, and has contributed significantly to developments in desistence theory. His talk will discuss the latest evidence on what makes high quality offender management and what can be implemented to reduce reoffending. Sam has published widely on probation work and desistence theory, and has helped CRCs to implement innovative offender management tools to measure offender motivation.
Jack Cattell, Get the Data
“Predict your PbR results and continuously learn how to improve them before they happen.”
Jack is an expert on the prediction of reoffending rates, and the analysis of how a probation service can audit and improve its reoffending rate. He has worked for the Ministry of Justice, former probation trusts and Sodexo Justice Service’s six CRCs on research and analysis projects. He heads up GtD’s thought leadership on the use of predictive analysis to improve adult and youth offender management.
The Purpose and Benefits of Community Resolutions
PbR figures for first cohort
Last week (27 October 2017) saw the release of the Transforming Rehabilitation (TR) Payment by Results figures for the October to December 2015 cohort. The overall result was encouraging, and defy the view that TR’s radical changes to probation, and the ensuing problems, would result in increased reoffending – though it is very important that I point out that this is just the first set of results of many and overall judgement should be reserved for at least a year. The reoffending rate for all CRCs was 45.6% compared to a 2011 baseline rate of 47.5%. I had to make some (conservative) assumptions to estimate the baseline rate but I think it is also safe to say that the difference was statistically significant, suggesting reoffending rates have reduced under TR. Please see the the note at the end of this blog to understand better how I completed the analysis.
CRC Performance
The chart below describes each CRC’s reoffending rate in relation to the baseline 2011 rate. The grey line represents the range of reoffending rates that would indicate no change from 2011 (the baseline confidence interval). If the CRC’s rate is outside this range, we are confident in statistically terms to state that the CRC’s performance was either better or worse than the reoffending rate achieved in 2011. The green bars represent the reoffending rates of CRCs that outperformed 2011, the orange bars represent those that performed the same as 2011 and the red bars present those that performed worse than 2011.
Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).
Thirteen of the CRCs beat the baseline rate. The best performing CRC was Cumbria and Lancashire, which beat the baseline rate by 8.2% (49.9% to 41.7%). [Editor’s note: Cumbria was the subject of a recent, very positive Probation Inspector’s report.] The nest best was Hampshire and the Isle of Wight which beat the baseline by 5.4% and the third best was Northumbria with a better rate by 4.3%. Two of the CRCs performed worse than the 2011 baseline. Warwickshire and West Mercia recorded a reoffending rate 3% worse than the baseline rate, and South Yorkshire’s rate was 2.8% worse. With most CRCs, however, outperforming the reoffending rate form 2011, the figures are a promising set of results.
Comparing CRC performance
Now that the baseline rates have been published, we can better understand how well each area was performing in 2011 and whether a CRC is now being asked to better good or bad performance achieved in that year. The chart below describes the difference between the actual baseline rate and the 2011 baseline’s OGRS score (in other words their expected rate of reoffending). A negative result in the chart means the area performed better in 2011 than the OGRS score expected.
2011 Baseline Performance
Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).
The charts highlights that six of the CRCs are being asked to beat better than expected performance in 2011 (in other words to be better than good). Whereas other CRCs, notably London and Wales, are being asked to outperform potentially poor performance in 2011. It it interesting that South Yorkshire and, Warwickshire & West Mercia – the two areas that recorded poor performance for TR – are being asked to beat good performance from 2011.
Merseyside and Cheshire & Greater Manchester, however, are equally being asked to beat good performance from 2011 and were able to do so for the October to December 2015 cohort. The OGRS score does not allow for area effects, which will exist and could explain the differences between the OGRS score and the baseline rate. It not possible now to conclude whether payment by results will be easier in some areas than others, but, going forward, I will monitor the impact of whether a CRC is being asked to perform better than good or poor performance from 2011 on their ability to achieve payment by results bonuses.
Editor’s Conclusion
I am very grateful for Jack Cattell’s technical skills in analysing the first cohort of offenders supervised under the government’s new private/public probation service created by the Transforming Rehabilitation (TR) initiative. Jack will be following the performance of the new private Community Rehabilitation Companies via the publication of reoffending rates every three months.
Many will be surprised at the apparent good performance of the CRCs (and it is important to remember that reoffending performance is linked to contract remuneration via the TR payment by results contracts).
However, I would like readers to bear a key fact in mind: These figures relate to the first three months of the CRCs operation when, by general consent (confirmed by HMI Probation reports) there was considerable chaos and confusion, as would be expected in moving from public to private ownership. Most CRCs have taken the best part of two years to implement fully their new models of operation.
There are two consequences to this fact:
- It is inappropriate to judge CRCs (either way) on this first set of figures and, in truth, we may have to wait another two years to get a reliable indicator of reoffending performance.
- It forces us to consider whether reoffending rates are a reliable way of measuring probation performance or whether other factors (such as policing) have too great an influence.
Jack Cattell
--oo00oo--
It's a free event designed, I suspect, for the serious enthusiast:-
Tue 28 November 201714:00 – 16:30 GMT
Transforming Rehabilitation: Learning from the PbR results
Transforming Rehabilitation challenged Community Rehabilitation Companies (CRCs) to reduce reoffending significantly and October will see the publication of the reoffending rates of the first Transforming Rehabilitation cohort.
These results are an important test of the government’s TR agenda and the success of the new CRCs. Some CRCs will be happy with their results, while others will need to improve. The published results also provide opportunities to commission new services for offenders.
Besides the implications for policy and practice, the results also present an opportunity for your organisation to learn how best to reduce reoffending with the tools at your disposable. You might want to increase a CRC’s performance, identify “what works?” and roll that out across all your CRCs, or prove that your specific service will make the difference.
Come join our experts at our TR PbR event, to discuss what needs to be learnt, what is within your control, and how embedding impact analytics in your CRC or service delivery organisation will help you to reduce reoffending.
Speakers:
Professor Darrick Jolliffe, University of Greenwich
“How to measure the quality of relationships between offenders and probation officers.”
Darrick is Professor of Criminology at the University of Greenwich. Darrick is interested in the broad areas of developmental life-course criminology, programme evaluation, prison research and psychology, individual differences and offending. He has developed tools to measure the satisfaction of offenders and the quality of their relationships with probation officers, and has estimated the association between relationship quality and reoffending. Darrick will also chair a discussion after the presentations.
Dr Sam King, University of Leicester
“What is high quality offender management?”
Sam is an expert on offender management and rehabilitation, and has contributed significantly to developments in desistence theory. His talk will discuss the latest evidence on what makes high quality offender management and what can be implemented to reduce reoffending. Sam has published widely on probation work and desistence theory, and has helped CRCs to implement innovative offender management tools to measure offender motivation.
Jack Cattell, Get the Data
“Predict your PbR results and continuously learn how to improve them before they happen.”
Jack is an expert on the prediction of reoffending rates, and the analysis of how a probation service can audit and improve its reoffending rate. He has worked for the Ministry of Justice, former probation trusts and Sodexo Justice Service’s six CRCs on research and analysis projects. He heads up GtD’s thought leadership on the use of predictive analysis to improve adult and youth offender management.
--oo00oo--
I was struck by the following tweet from Sally Lewis, former probation CEO:-
"As has always been the problem it's actually reconviction that is measured, not reoffending. Flawed measure. Police activity has huge impact."A view endorsed by Russell Webster:-
"Agreed (although reoffending obviously unmeasurable). I'm increasingly convinced about impact of policing."Another contributor put it rather more graphically thus:-
"Have to understand police performance alongside this. Run a CRC in an area where police aren't catching offenders and £££ to be made."In trying to make sense of all this, factors such as contained in this policy document by West Midlands Police become increasingly relevant:-
The Purpose and Benefits of Community Resolutions
1.1. It is a long-established principle of British law that justice does not always demand a court prosecution. In many cases, the public interest is best served by using alternatives to prosecution, especially where such alternatives can provide swift redress for victims; and where we can intervene at an early juncture to change offending behaviour before it becomes entrenched. Peelian Principles emphasise the role of the police in preventing crime: Community Resolutions offer an opportunity to reduce the likelihood of future offending, and therefore, crime.
1.2. Many of the crimes the police investigate are not committed by prolific, dangerous offenders. Rather, they represent momentary lapses in judgement by otherwise lawabiding citizens. In such situations, a formal sanction would often be a disproportionate response: the decision to give someone a criminal record is one that should never be taken lightly, and neither should the police underestimate how daunting the judicial system can be for victims and offenders alike.
1.3. Community Resolutions are designed to deal with such situations. The use of a Community Resolution offers victims an informal, flexible response to the matter they have reported. At the same time, they offer offenders a ‘second chance’ - an opportunity to make amends for their mistakes without suffering consequences which can dramatically alter future life chances.
1.4. Due to the lack of a criminal record, it is not appropriate to issue multiple Community Resolutions. However, the updated policy makes clear that an offender may still receive a Community Resolution in some circumstances where they have already received one previous disposal. If however, a Community Resolution is to be issued to an offender who has already received a previous disposal; more will be expected from the suspect. Specifically, it is expected that the suspect will be required to participate in activities which will directly address the underlying cause of their offending behaviour.
1.5. The distinction between offenders who have no previous offending history and those with one previous disposal reflects a desire to intervene in offending behaviour at the earliest possible juncture, but without limiting the discretion of officers where it appears that the offence in question genuinely appears to be an isolated incident.
1.6. Community Resolutions also offer the opportunity to deal with matters in the quickest, simplest fashion. In doing so, they help to ensure that the officers who serve the public are free to do more with their time. However, the need for efficiency must always be balanced against - and should never outweigh – the overriding goals of protecting and serving our communities.
1.7. A Community Resolution is not the same as Restorative Justice, but they bear many similarities. Community Resolutions are founded on a commitment to empower victims by allowing them to have a say in how their matter is dealt with. They provide an opportunity for offenders to better understand the impact they have had on their victims, and to make amends for the harm they have caused.
1.8. Fundamental to the success of every Community Resolution is the professionalism of the officer who administers it. Community Resolutions allow officers to exercise the greatest discretion, and with this discretion comes a responsibility to ensure that we can maintain the faith and confidence of the public.
--oo00oo--
Footnote
A glance at the archives reveals that I've only dared raise the subject of statistics once before Lies, Damned Lies and Figures published on 6th October 2012 and viewed just 76 times and with no comments......
--oo00oo--
Footnote
A glance at the archives reveals that I've only dared raise the subject of statistics once before Lies, Damned Lies and Figures published on 6th October 2012 and viewed just 76 times and with no comments......
