Policy-based evidence-making…no, hang on…


Musings on evidence-based policy making, political principles and individual disposition

Anyone who has done an humanities degree at university will be familiar with the pattern of getting a low 2:1 score for an essay. It’s the sort you leave till the last minute, cram all you can then breathe a sign of relief that you didn’t fail the assignment. You pick your argument, search the internet for evidence bases that back up your argument (plus one or two that don’t – for balance and all that), and then say: ‘Done it!’

Political policy-making can be a bit like that. Beecroft for example. Get one of your chums to do an ‘independent report’ and cite as your evidence base. Let’s not forget the infamous dodgy dossier either in the run up to the Iraq War in 2002/03. Until fairly recently it has been relatively straight forward to drive ‘principled-based policies’ through despite the lack of empirical evidence.

During my final couple of years in the civil service, there was a push towards more evidence-based policy-making. There have been (& continue to be) a greater number of academics being seconded into Whitehall and a greater number of seminars where policy makers and academics are brought into the same place. The better organised ones are also live-streamed and live-blogged/tweeted.

Feedback loops

I’ve mentioned these before. The one thing that has not been as clear as I’d like it to be in policy world is just how these feedback loops function. It’s all very well bringing in academics to do evaluations, and push for improvements to existing practices – including getting more robust evidence sets.

One vital area for me is ensuring civil servants have the skills to understand and scrutinise the evidence that they gather and/or is put in front of them. One of the team leaders in my last civil service team was a statistician by training, although in a generalist role. That meant all of the data that I was playing with and analysing received more than a decent amount of scrutiny – intelligent questions which fed into decisions on whether things could go up to ministers or whether more research was necessary.

There is also a further onus on ministers to understand what is being put in front of them. What are the skills sets required of ministers? Being able to interpret data and statistics is surely one of them – in particular knowing what top level questions to ask. Who was the study carried out by? What was the sample size/number of observations? Are the results statistically significant? What similar studies have been done and what were the results? …For example.

Finally – and this is where evidence bases begin to clash with political principle and disposition: To what extent are ministers and other decision-makers prepared to be influenced by evidence bases? 

If a minister is going to go ahead come what may, it may feel like ‘what is the point?’. In the recent meetings and events that I have been to, the question above in bold has been the dragon in the room. It’s one thing putting in place and ensuring that comprehensive evidence has been well-analysed and good recommendations are put to ministers, but what if the minister concerned chooses to ignore all of it? There might be sound party political reasons to do so. If it relates to a flagship policy that has the personal stamp of a minister on it, then any retreat from such a policy could cost the minister his or her job. Is it not more politically expedient to keep going until the next reshuffle then allow a successor to do the retreat later on? That may not be good policy, but it might be sound politics – especially if the government of the day is unstable.

University policy institutes – the thinktank busters

Puffles and I were kindly invited to a reception launching Cambridge University’s new Masters in Public Policy programme at the Institute for Government recently. David Willetts, the Universities’ Minister made the keynote speech and acknowledged the tension between evidence bases and the political/philosophical/intellectual framework that a political party works within. It is political parties with the democratic mandates.

The bit that he didn’t touch upon was the role of political parties and how they use evidence to shape their policies. Willetts’ challenge to academics working in public policy roles was not just to do studies of the distant and very recent past, but to significantly increase their output in terms of looking forward. He said that there’s too little analysis and discussion within academia on what the forward-looking public policy implications of their work actually are. That said, the rise of the university-based public policy institute could well change that. This automatically puts such institutes in direct competition with think tanks – in particular the partisan ones.

Transparency has long been an issue for think-tank watchers. It’s actually relatively straight forward to set up a think tank. Interestingly, one of the most clear examples of this comes not from the right, but from the left of politics – the Centre for Labour and Social Studies – CLASS. An easy on the eye professional-looking website with all the social media links and content links. It has a wide advisory panel containing the great and the good of centre-left and moderate left-wing politics. Yet it only lists three members of staff. The Tax Payers Alliance at the other end of the spectrum has less than 20 people on its staff, yet has a significant media profile.

Competition vs collaboration and co-operation

I’m interested in seeing how relationships between the university public policy institutes develop over the next few years. While there is competition between think tanks to get their ideas adopted by political parties, they are now going to face growing levels of scrutiny from academia as the latter look to see how their research is applied in a public policy environment. Especially so as the Government has given them their blessing to do so. Oxford, Cambridge, Manchester and Bristol are just a few names that bounce off the top of my head as having set up growing public policy units and programmes. To what extent will academics within these institutes look to compete, and to what extent will they look to co-operate? Will this vary depending on policies and overall political and economic climate? Will there be any room for co-operation between think tanks and university public policy institutes?

How will political parties and ministers respond?

Think tanks are well-known as being originators and incubators of new policies that are later adopted by political parties. Will we see university public policy institutes making similar pitches to political parties, and if so, which ones? Will this lead to a tendency to lean towards one political party or another? Or will that change dependent on economic conditions or who is in power?

The push for transparency and open policy

The Coalition has made it clear it wants to go down the route of outsourcing some policy functions and having competitive policy making teams. I still have issues with outsourcing of policy functions, but like the idea of opening up policy-making processes to those interested and to those who have something positive to contribute. (Even if it means: “This policy is unlikely to work, here’s why”).

But the dragon in the room remains: To what extent are politicians and ministers prepared to be influenced by the feedback that comes about from opening up policy-making? The same question applies to university public policy institutes as it does think tanks. Both Labour and Plaid Cymru are already experimenting with crowd-sourcing policy ideas – Labour’s here (though note you do have to sign up, so watch that personal information!) and Plaid Cymru’s here. In Labour’s case, will the party leadership be prepared to relinquish that much control over policy formulation given the centralisation of past decades? What do grassroots Labour activists think?


3 thoughts on “Policy-based evidence-making…no, hang on…

  1. Not a *hope* that the Labour system will do anything. They might choose one policy from there, but that’s it. Labour policy is made up by the top people on the hoof as it is, and there is no reason for that to change.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s