NAO : Problems with TR

Every now and then regular readers are irritated by bald statements on this blog saying things to the effect 'TR is a success' . I normally delete them. Well, here we have the key findings of today's National Audit Office report and despite some brave attempts at positive spin, I think it's pretty clear as to how TR is doing:-

Key findings 

The performance of the reformed system 

6 Services have been sustained throughout a period of major change, with users reporting that services had stayed the same or improved since the reforms. Based on survey data from service users across four CRCs, overall 77% of service users considered they had not noticed any change in the overall service they personally received. However, users also provided views on specific services they received. User dissatisfaction was highest in obtaining help with housing; having to repeat information to different people; the level of support that supervisors provided to offenders; and help with finding employment. Such aspects are in part influenced by wider factors outside the control of probation bodies (paragraphs 3.14 to 3.18). 

7 The performance of CRCs and the NPS remains unclear given limitations around data quality and availability. Until data on reoffending are compiled in late 2017, data on performance are limited to information on service levels for the completion of probation activities. The Ministry allowed eight months until September 2015 before performance of CRCs would be open to contractual penalties.

• As at December 2015, NOMS has no data for three of 24 CRC service levels and assurance metrics, and insufficiently robust data in another two. Nationally, CRC performance is at or above target levels in seven of the remaining 19 measures, including positive completion of court orders, seen by the Ministry as a leading indicator for future reoffending. However, performance varies significantly across CRCs and the contracts require that CRCs achieve all targets by February 2017. NOMS is currently only applying service credits for poor performance against one level, due to data availability and quality for others. To date some £78,000 in service credits have been applied, at two CRCs. 

• The NPS has similar issues including currently no data for five of 25 NPS service levels and insufficiently robust data in another two. Performance is at or above targets in seven of the remaining 18 measures. However service level agreements require that NPS achieves all targets by April 2017. In the important measure of positive completion of court orders, NPS performance is lower than the equivalent performance by CRCs (70% versus 80% in December 2015) (paragraphs 2.2 to 2.8). 

8 NOMS has established robust and thorough contract management and assurance arrangements but has no plan for moving to a more risk-based approach as delivery under the contracts matures. NOMS has applied lessons from previous failures and has invested heavily in robust CRC contract management, which accounts for 2.1% of contract spend. However, many staff in CRCs were concerned about the extent and trajectory of contract management and operational assurance activity. NPS has much more limited contract management capability, albeit for much lower-value contracts. It is currently trying to identify all the contracts it holds, establish precisely what goods and services it is paying for and revise its approach to commercial activity (paragraphs 2.9 to 2.19). 

Meeting current operational challenges 

9 The reforms have established new organisations with different incentives, creating unsurprising frictions between CRC and NPS staff at working level, which will take time to work through. Close cooperation is essential to handle the transfer of offenders between CRCs and the NPS when their risks change or when they breach the terms of their probation. Many junior staff we spoke to in CRCs considered their NPS contacts were often unduly critical and dismissive, while many junior staff in the NPS thought that their CRC contacts were often not providing them with necessary information and had become too focused on their commercial interests as opposed to the best interests of offenders. We saw efforts by local CRC and NPS managers to address such differences and build trust, but at this early stage the organisations have more to do to ensure that they work together more effectively to improve case management (paragraphs 3.1 to 3.4).

10 Concerns over probation workloads are not new, although staff in both the CRCs and the NPS considered that high workloads have reduced the supervision and training that they receive and the service they provide. CRCs are reducing their workforce in advance of transformation while the NPS is increasing staff. There is no single ‘right’ number for workload, which depends on case risk and complexity. In the four CRCs we visited, only three provided individual caseload data and these were presented as an average, which masks any variation within and across CRCs. While the average caseload was between 34 and 42 cases, we met staff handling significantly higher caseloads, which they considered prevented them providing an adequate service. The NPS has been operating above recommended capacity in two of its seven regions, although ongoing recruitment of some 650 trainee probation officers should help address shortfalls in the medium term (paragraphs 3.8 and 3.9). 

11 The various ICT systems used in probation casework create severe inefficiencies. New tools used by the NPS for assessing and allocating offenders are cumbersome and require repeated data re-entry. Staff also attributed several hours per person per week of lost working time to nDelius, the main probation case management system adopted before the Transforming Rehabilitation reforms. The NPS expects to continue using these systems for the foreseeable future. All CRCs planned to replace existing ICT systems once they could develop new case management and assessment systems, but NOMS was delayed in developing and implementing the interface (the Gateway) required for CRCs to share data on offenders. The interface was originally planned for delivery in June 2015, but was delayed due to other priorities and increased scope. At the time of finalising this report the Ministry advised us that the Gateway had been developed and was awaiting joint testing with CRCs’ systems. As CRCs consider such links as essential to their transformation plans they have estimated consequent costs, which are subject to ongoing negotiations (paragraphs 3.10 to 3.13). 

Ensuring that transformation is achieved 

12 The Ministry did well to sustain competition and conclude deals for all 21 CRCs within the cost limits and timescales set by ministers, but the procurement has left some difficult issues to manage. The Ministry attracted interest from many providers new to probation and as a result secured affordable bids for an expanded range of services at all 21 CRCs by February 2015. Selected bidders offered cost savings sufficient to fund the expansion of supervision to short sentence offenders, and to fund an estimated £259 million of success payments over ten years for reducing offending. Offers were received from a total of 19 bidders, down from 30 originally invited to negotiate, as the Ministry maintained its position on key contract terms. This reduction in bidders resulted in only one compliant bid for five of the 21 CRCs, although these all met the qualitative and financial thresholds required by the department. Completing the procurement in a challenging timetable, combined with uncertainties arising from the concurrent changes in the probation system, limited bidders’ understanding of their exposure to business risk (paragraphs 1.5 to 1.10).

13 CRC business volumes are much lower than the Ministry modelled during the procurement, which, if translated into reduced income, would affect the ability of CRCs to transform their businesses. The volume reductions vary greatly, from 6% to 36%. The Ministry attributes the volume reductions to fewer cases going through the justice system, including fewer than expected low- and medium-risk cases for CRCs, and the declining use of certain sentences, which was accentuated by new deadlines for allocating cases. Income shortfalls, which are under commercial negotiation, would affect CRCs’ capacity to bring in new ways of rehabilitating offenders, introduce new ICT systems, implement estates strategies and reform corporate support services. They also increase the risk of underperformance or default. The Ministry has contractual powers to help it mitigate some of these risks, although having to replace a failing provider would be challenging and disruptive. Its insights into CRC finances and funding challenges are still developing (paragraphs 4.1 to 4.10). 

14 CRCs are paid primarily for completing specified activities with offenders rather than for reducing reoffending, which also risks hindering innovative practice. This was a realistic choice, reflecting the limited appetite of providers to accept a higher element of payment by results. But given the limited weight of payment by results, it is critical that these fees for activities (‘fees for service’) better incentivise CRCs to adopt innovative approaches to reduce reoffending, and not just established practice (paragraphs 1.14 to 1.19). 

15 The NPS has higher than predicted caseloads and faces a difficult further period of change if it is to play a fully effective role in the transformed and national probation service. Its front-line managers face increasing pressure, including dealing with higher than expected workloads, now of high-risk offenders, while assimilating a heavy influx of trainees, who will take time to become fully effective professionals. At the same time, probation managers are acquiring new responsibilities for managing support services, such as human resources and office management; a key source of dissatisfaction among staff we interviewed. The NPS’s new change programme, announced in November 2015, is attempting to tackle regional variations in probation practices but has not focused specifically on support services (paragraphs 3.6 and 3.7). 

16 Arrangements to resettle offenders ‘Through the Gate’ are still in their early stages. CRCs delivering resettlement services in prisons have been focused on commencing services and meeting contractual measures based on completing processes, rather than on service quality, which we understand varies significantly across prisons. To date, it is unclear what new processes CRCs will introduce into resettlement services and the impact these will have on providers’ overall payment by results (paragraphs 4.15 and 4.16).

17 The Ministry has more work to do to sustain the supply chain of mainly voluntary sector bodies now working to CRCs and the NPS. Although the Ministry put extensive effort into attracting voluntary sector bidders, these largely lost out to private sector contractors when bidding to lead CRCs, due to their more limited resources and appetite for risk. The voluntary bodies still have a major role as suppliers to CRCs, although recent surveys of the sector indicate increased uncertainty and instability in funding of their work with offenders. Similarly, the Ministry has identified gaps in provision, which it and CRCs will need to address (paragraph 4.11). 

Conclusion on value for money 

18 The Ministry has successfully restructured the probation landscape, avoiding major disruptions in service during a difficult transition period. But this is only the beginning. If the Ministry is to stabilise, and improve, the performance of CRCs and the NPS it needs to continue to address operational problems, such as underlying capacity issues, weaknesses in ICT systems and performance data, and improve working relationships between NPS and CRC staff – some of which are unsurprising given the scale of reforms. 

19 Ultimately, the success of the Transforming Rehabilitation reforms will depend on the extent to which they create the conditions and incentives to reduce reoffending. While NOMS’ oversight of CRCs is robust, significantly lower levels of business than the Ministry projected will affect some CRCs’ ability to deliver the level of innovation they proposed in their bids. Furthermore, the NPS is not yet operating as a truly national, sustainable service. Achieving value for money from the new probation system will require resolving these fundamental issues, and ensuring the right incentives for all participants in the system. 

Recommendation to the Infrastructure and Projects Authority 

a The Authority should ensure that its guidance to departments outsourcing complex transformed services considers how to mitigate or reduce risk and uncertainty from concurrent changes, including through different phasing. High uncertainty over future business can reduce competition during procurement and cause later problems. Key issues affecting Transforming Rehabilitation during and since procurement are due to outsourcing immature CRC businesses within a changing new probation system.

Recommendations to the Ministry of Justice 

Sustaining the new market 

b NOMS should combine its ongoing analysis of the CRC supply chain with feedback from voluntary organisations to identify and address gaps in provision in consultation with CRCs. 

Achieving business transformation 

c NOMS needs deeper understanding of the financial and service viability of CRCs. It should focus its analysis on CRCs’ financial capacity to sustain their full transformation and service delivery plans. 

d The NPS should expand its change programme. The programme should be expanded to include corporate support services and establish an operational assurance function to assess the quality of work and regional compliance with the new ways of working. 

• NPS risks being left behind by CRCs’ investment in new offender management systems; it needs to replace its own unfit and inefficient systems, learning from CRCs’ progress in making replacements. 

Contract management 

e NOMS should map out the trajectory of its investment in contract management and how that will impact its CRC contract assurance functions. NOMS should also give CRCs a stronger incentive to improve the rigour of their own performance and reporting systems by offering reduced contract management oversight to proven robust systems. 

• The management of NPS’s CRC contracts should be delivered by the existing teams in NOMS who already manage CRCs. 

Managing and incentivising performance 

f The Ministry should, as a matter of urgency, ensure data are available to support the contract and performance management of CRCs and the NPS. Performance against all service levels should be based on at least monthly data. 

g The Ministry should regularly review the composition of the fee for services to ensure that it incorporates and incentivises innovative approaches to reducing reoffending.