Some TR Insights

I never quite know where this blog is going to take me next. Saturday's off-piste foray into economics opened up some absolutely revelatory stuff for me which I intend to revisit at some point. But most surprisingly, in the process of my researches, it threw up a significant character in the TR saga whom thus far had not appeared on our radar - one Stephen Muers. 

Seeing as he held the key post of Director, Sentencing and Rehabilitation, Ministry of Justice at the time, I'm somewhat bemused that his name hadn't surfaced before. Anyway, I've always wondered what key people at the MoJ actually felt about the whole TR project and especially their reflections on the subsequent omnishambles. 

Well, such is the wonder of the internet that I've been able to track down three articles, the first of which was written when Mr Muers was still in post and published by the Civil Service Quarterly blog on 15 October 2014. The whole article can be viewed here. It's a bit esoteric for my liking, but I've pulled out what I feel is the most relevant and telling for us:-   

Is your policy a dodo?

Speeding up the “evolution” of policy could help promote the survival of the fittest in policy solutions, argues Stephen Muers, Director, Sentencing and Rehabilitation, Ministry of Justice.

Talk to any worker in the public sector and there is a good chance they will say they face never-ending change. But an analysis of Queen’s Speeches showed that many of the same issues keep coming up. Sometimes it is only when looking across many decades that very large shifts in policy become apparent, such as changes to the way services are provided by the state.

The tendency is for constant small-scale change but overall stability in the short term, coupled with large, unpredictable shifts in the long-term. This can also be seen in another environment: the natural world. Can we learn lessons from Darwin’s theory of evolution to improve how we develop public policy?

--//--
Evolving the policy process

If there is value in this comparison between the natural world and the policy one, what does that mean for our policy making practice? Organisms that reproduce fast and rapidly eliminate unhelpful mutations are best at succeeding in changing environments. The continual rapid changes to the influenza virus are what make it such a formidable opponent.

To improve policy development, we need to make it quick to get a new policy into implementation but equally quick to kill it off if not successful. Small experiments that can be scaled up and down rapidly are the best way to replicate this behaviour. A good example of experimentation is the series of pilots as part of the Ministry of Justice’s Transforming Rehabilitation programme, which looked at new payment methods for reducing re-offending. Some demonstrated results worth scaling up because they showed a significant fall in reconviction rates against a comparable group. Others demonstrated that ideas were not worth pursuing: offering additional money to providers of employment support, under the Work Programme, would not significantly prevent reoffending because there were too few prison leavers joining the programme.

A larger version of the experimental approach can be seen in the “What Works” movement. For example, the Education Endowment Foundation is involving more than two-thousand schools in over seventy controlled trials that test alternative ways of increasing the educational performance of disadvantaged students. Results are published and head teachers can use them to decide where to invest Pupil Premium funding.

In the private sector, competition, entrepreneurship and take-overs force the pace of change. However, many public service markets feature long-term contracts and high barriers to entry. Such an environment suits incumbents, and does not encourage experimentation. Despite this, it’s possible to create a broad and flexible supply chain. A key part of the Transforming Rehabilitation programme, for instance, is to make sure that smaller voluntary sector organisations are equipped to participate, and that there is a diverse and competitive market.

Another approach would be to lower the barriers to introducing small, innovative policies. There are clear processes through which any policy has to go, but could more approval processes be subject to thresholds, enabling small experiments to be launched more quickly? Any failures could be nipped in the bud by applying strict time limits to these projects. Communicating that some policies are pilots and are expected to fail could help manage the expectations of stakeholders. By presenting a series of experiments under an overall ‘brand’, success is tied to the brand rather than individual components. The Innovation Fund led by NESTA in partnership with the Cabinet Office works along these lines. Government departments could put teams together specifically to launch and then kill off experiments without developing an institutional interest.

And while the legislative process is (for good reasons) time-consuming and pushes government towards specifying detailed plans early on, there is scope for speeding things up. The Welfare Reform Act 2012 created a power to allow small experiments in benefit rules. This opens the door to more rapid experimentation, and is an approach that could be used more widely. Above all, policy makers need to view the public policy environment as a complex and shifting ecosystem and let this guide their approach.

The technology industry uses ‘Agile’ techniques, characterised by rapid rollout of rough prototypes and constant feedback from users. The Government Digital Service and others are already using Agile techniques in a government context. Invariably, boundaries and roles will shift as policies develop. So incentives for the system as a whole need to work however individual parts develop. In education, for example, there are clear incentives set by the funding and inspection regime, which push the overall system in the desired direction. This is regardless of the different models of schools and their relationships with the Department for Education or their local authorities. However, this approach must be supported by transparency about performance.

Thinking about what makes organisms succeed in the battle of evolution and natural selection suggests ways to improve policy making. If we speed up the “evolution” of policy and recognise how public sector organisations compete with one another, we could promote the “survival of the fittest” in policy solutions. There will be small variations in policy and implementation, but by embracing such differences and learning from them, we can deliver continuous improvement. Such an approach requires a focus on the empirical evidence of what actually works, and an acceptance that we do not know what is best when we start the policy process. But it offers the prospect of accelerating evolution and improvement across public services.

Stephen Muers

--oo00oo--

It's interesting to note that Mr Muers subsequently decided to 'move on' from what looked like a promising civil service career:-   

Stephen joined Big Society Capital after series of senior roles in government. Most recently he was Director, Criminal Justice Policy at the Ministry of Justice, but has also worked in the Cabinet Office, the Department for Energy and Climate Change and the Homes and Communities Agency among others. He has been a non-executive director of an NHS trust, a Visiting Fellow at the Institute of Policy Research, University of Bath, and has published several papers on policy making and public services. He is also a Trustee of the Friends Provident Foundation.

--oo00oo--

Big Society Capital improves the lives of people in the UK by connecting social investment to charities and social enterprises. We know that investment can help charities and social enterprises achieve more. We believe the greatest chance to improve lives comes when investors and enterprises are both motivated by social mission.

We engage with investors, fund managers, charities and social enterprises to make it easier to use social investment. With our co-investors, we have made over £1bn of new capital available to organisations with a social mission, through investments into fund managers and social banks. We have a special focus on: providing homes for people in need; supporting communities to improve lives; and early action to prevent problems.

--oo00oo--

Having become the CEO of Big Society Capital, here we have an article published on the University of Bath IPR blog website on 12th August 2016. Again, I've selected what I feel are the most relevant sections for our situation, but even the sharp-eyed will search in vain for any mention of TR, but surely it is being alluded to?

Culture comes first: putting culture and values at the centre of public policy

--//--
How to promote value systems

The most obvious way to create a culture that supports positive change and learning is for government to send messages through its words and actions that this is desirable. This would include launching and welcoming experiments in policy and practice and being open to learning from failure, to bottom-up innovation and to constant iteration rather than over-specifying plans from the start.

Such an approach can, however, seem rather intangible. The obvious way to give it harder edges is to use incentives to reward service improvement and thereby encourage innovation that delivers such improvement across the system. This is the philosophy behind recent school reform; the changes gave schools autonomy to innovate, and strong incentives to do so, by introducing a rigorous performance regime.

However, performance incentives alone are unlikely to engender the culture of wholesale creative experimentation discussed above, for several reasons:

• There is a long-standing body of literature which argues that it is hard to design incentives which can’t be gamed and that do not lead to distorting behaviour. There is always a risk that hard performance measures produce great innovation in the management of the measures themselves, rather than genuinely improving services across the board. Classic examples include hospitals meeting the four-hour A&E waiting time target by creating other queues elsewhere in the system, and schools focusing their efforts on pupils around the borderline of exam targets at the expense of the less able, who were never likely to make it.

• Problems that require collaboration between lots of agencies, with the costs and benefits potentially falling asymmetrically, are difficult to address through performance incentives. While it is theoretically possible to design an outcomes framework that pulls all agencies together behind a common goal (the UK model of joint PSA targets, for example) it is hard to do so in a way that isn’t highly complex and bureaucratic. Simple measures of their own performance will tend to have more traction with service managers.

• Measuring and rewarding performance as a way of promoting innovation has the downside of pushing innovation towards current problems that we know how to measure. The most valuable feature of a dispersed system is, in fact, its ability to react quickly to a changing situation, beginning the task of innovating to respond to new challenges before the central authority has even clocked their existence. The makers of Blackberry phones were proud of the way they encouraged employees to innovate and improve keypad mobile phones – but missed the real innovation of moving to touch-screens and disposing of keypads altogether.

Service outputs, or even outcomes, are not the only objective of public services in a democratic state. Such services also need to operate in a way that is recognised as fair and legitimate, and which promotes trust between citizens and institutions. While trust and legitimacy is, in some cases, measurable, it is much harder to target with performance incentives.

--//--

Stephen Meurs

--oo00oo--

Finally, here we have another article published on the same academic website this year on 23rd June 2017. Again, I have only selected what I feel is relevant and any reference to TR is very conspicuously absent, but I get the distinct feeling some inferences can be drawn:-

Accountable for what?

Accountability is fundamental to democracy. Holding decision-makers to account for what they do and the impact they have can be seen both as a good in itself and a way of aligning their choices with the interests of the public at large. So effective democracy needs effective accountability, defined here as a system that holds decision-makers to account for things they control in a way that is meaningful and legitimate in the eyes of the public, and which is likely to promote desired outcomes. In turn, therefore, accountability needs to be based on an understanding of what different decision-makers can and should be doing, to fit with public expectations and to promote effective outcomes. Without such an understanding there is a risk that the accountability framework creates the wrong incentives and promotes neither legitimacy nor the right decisions.

What should decision-makers be doing?

In a previous piece I argued that policy outcomes are heavily influenced by culture and value systems, that governments are part of the prevailing culture and, crucially, that they can also affect it. Therefore a critical role of decision-makers is to embody and shape a culture that supports the outcomes they (and in a democracy those who elected them) desire.

One of the most important ways in which culture and values shape policy outcomes is through the individual choices and decisions made every day by the people responsible for implementation: teachers, social workers, employment advisers, police officers and so on. They interpret and act on policy according to their values and the values embodied by the organisations they work in. As I argued in another previous piece these front-line decisions create constant mutation and evolution in what policy means on the ground. As with evolution in the natural world, the resulting pattern is one of periods of stability interspersed with large and often unpredictable shifts. Decision-makers need to recognise this unpredictable dynamic of front-line evolution, and use their position to shape a culture that supports positive experimentation and learning.

If this understanding of how policy works is correct, then central decision-makers should be focusing on shaping a culture that promotes desired outcomes, and that supports front-line decision-makers in a process of learning that leads towards those same outcomes.

--////--
And what are they not held to account for?

Therefore there is a fit between what it makes sense for leaders to be accountable for, given how we know policy actually feeds through into practice, and what voters use as the basis for deciding how to cast their ballots. So what is the problem?

There would be no accountability problem if political leaders did indeed devote their efforts to affecting values, culture and the overall properties of the system within which they sit: voters are good at holding them to account for that. The problem arises in that in practice they do a lot of other things. Political leaders devote considerable time and effort to designing and implementing detailed policy changes. Democratic accountability for such changes is weak. This is for two reasons.

First, a policy change can be completely disastrous in its own terms without seriously impinging on the welfare of individual voters. Very large sums of money lost to the public purse are hardly noticeable to individuals, especially if such losses are from future potential value rather than current income. A good example would be under-valuation when privatising an asset: no voter feels an immediate loss even if they are in fact worse off because their share of a valuable asset has been given to someone else. But this is true of any large policy failure that wastes money, makes a large service incrementally worse or damages the environment in lasting but not immediately apparent ways.

Second, even if voters are aware of a policy (usually not the case) and are affected by it, it is unlikely to change how they vote. In fact there is evidence that the causation often runs the other way: how someone is inclined to vote affects their understanding of what a policy has achieved. Whether or not someone is aware of a policy and what they believe its effects to be are influenced by their political starting point. This is down to confirmation bias: we interpret information in line with our starting positions. A recent piece of research showed that people’s ability to interpret statistics correctly is dramatically worse when the same statistics are used to describe a divisive political topic (immigration) rather than a neutral one (effectiveness of a skin cream). When political control changes after an election, partisan perceptions of other events changes dramatically. To use another contemporary US example, there was an 82% net positive swing among Republican voters in perceptions of how the US economy was doing in six months Trump’s election as president, at a time when objective economic indicators were fairly stable. So if we believe a political leader is acting in line with our values, that shapes how we see policy and we will tend to register information that implies they are being successful. This dynamic gives politicians considerable leeway to implement policy that is damaging as long as a majority believe they have the right values.

In an attempt to remedy this weakness, we have created a structure of accountability intended to expose policy failure and thereby create incentives for politicians to do the right thing. In the UK this includes the National Audit Office and Select Committee scrutiny. The media also performs an important function in exposing policy failure. Part of the thesis behind such structures is that the appearance of competence is vital to politicians and so methods that expose the opposite will create a strong drive towards successful policymaking.

These structures are, however, unlikely to succeed. Media coverage of a damning NAO report on a multi-billion pound policy will still go unnoticed by the vast majority of the population. And there is no evidence (or even a very plausible theoretical case) for arguing that such a report is likely to contribute to any significant change in party perception and voting behaviour. Confirmation bias is important here too: even if a hypothetical damning audit of a policy became widespread news, people would interpret that news according to their existing cultural framing of the political situation and use it to confirm their existing biases about what policies are or aren’t desirable. It is much more psychologically plausible to believe that a report is biased or wrong than to change your view of a policy and political leader with whom you identify in a deep-seated cultural sense.

Audit and scrutiny take a long time, because of the understandable desire for thoroughness and rigour. This creates two further difficulties. The first is that, because ministers and civil servants – especially in the UK system – move around frequently and fast, by the time a major project or policy is evaluated, those needing to explain it are probably not those responsible for implementation. Again, this means that the incentive to make good policy created by this part of the accountability system is limited.

The second is a more fundamental point. I argued previously that the evolutionary nature of policy means that rapid feedback is critical. It is important to know immediately whether a deliberate or inadvertent change has started to make a difference, allowing the front-line policy implementers to adjust accordingly. The clearer and faster the feedback, the more likely it is that people will learn and iterate towards improvement. In technology projects it has become the norm to use “agile” techniques: build something small, test it with users, learn fast and make repeated small changes. Such techniques have evolved as a way of coping with uncertainty about how people will behave in the face of change and the fact that requirements and goals shift as we find out more about what the front-line users actually want. Such uncertainty and changing goals are a strong feature of policy implementation and so these techniques, and the rapid feedback on which they depend, could have major benefits. However the norm in the public policy field is one of long-term detailed studies that aim to assess, retrospectively, the impact of a whole programme against its stated objectives once enough time has passed to measure progress against them.

--//--

Stephen Meurs