evidence based or intelligence driven policy?
May 8, 2009OK, so I may be the only one to care about whether we talk about evidence based or intelligence driven policy, and I’m very aware I stand to be accused of campaigning for replacing one piece of jargon with another. but I keep on wanting to return to this theme in various pieces of work at the moment.
here’s a short paper I did for Kent to accompany a workshop which the Social Innovation Lab for Kent ran with a mix of policy people from across the council. Basically I argue that human and social factors need to count for as much as data and trends when it comes to taking account of ‘evidence’ in policy work. Sounds so simple and yet the barriers to embedding this kind of approach are many. Culturally, the public sector still prefers rational analysis to emotions and experiences. Skills-wise, very few councils have the research know-how or methods at their fingertips to do this kind of work well. Organisationally, research and policy functions are rarely co-located…
Sophie – what’s your evidence for those assertions?
<>
X Alfred
Sophia
The SILK paper which you refer to in your post ends with a quote by Louise Casey. I was at the event where, glass in hand, she made the threat to deck members of the No 10 policy unit. It remains the most memorable police dinner I have been to. I still have my suspicions as to who recorded Louise’s after dinner speech and gave the tape to the press but, rather like in American Beauty, where by the end of the film everyone has a motive to murder the character played by Kevin Spacey, so just about all the assembled Home Office researchers and ACPO research leads might have thought ill of Louise that evening.
I exclude myself from the suspects though – I thought what Louise said was really challenging. Her subsequent work has not eschewed evidence by any means. Engaging Communities in Fighting Crime (aka the Casey report) is supported by IPSOS-MORI, GfK and other surveys of the views of the public on their engagement with policing and crime issues, as well as focus groups, public meetings etc. I think Louise’s point, which is the one you make, is about the nature of evidence and research. There is inevitably a conflict between long-term, large sample surveying with peer review to follow at the end and the demands of political cycles which require early demonstration of action. There is indeed a positivist aspect to a lot of evidence based policy making – based on a belief that the facts are accessible, as long as the research design is right and the sample size is big enough, and that the policy solution will be derived relatively straightforwardly from their interpretation.
I wonder if a possible driver for this is the conflation in the official mind of evaluation of what works – which calls for a systems understanding of interventions if it is to have real value – and the accountability agenda. Typically, a project is expected to identify some projected benefits before it starts and then to gather evidence, preferably quantifiable and financial, to demonstrate to the auditors that those benefits have been realised. This kind of reporting on delivery of projects is necessary and desirable, up to a point, to show the taxpayer what he or she got for their money. But it isn’t the same as gathering evidence for future policy making and we need to distinguish between the two.
I think that the research community in government is getting more realistic about things like delivery timescales for research. It is less obvious that there is very much non-positivistic methodology being developed in any systematic way. I urge my colleagues to get more ethnomethodological when I can. But we shouldn’t lose sight of the fact that we also still need a lot more well-founded evaluations of the efficacy of specific interventions. The randomised control trial has its place and there have been too few rather than too many in parts of the public sector. We wouldn’t want to undergo medical procedures if we didn’t think they had been properly tested before going into general use. Should the same apply to educational innovation or changes in policing methods, for example, where the recipient of the service may have little choice but the impact of change can be profound? The challenge is in arriving at consensus about what is the right approach, and then about the authority and significance of the results.
I think I have ended on a call for more research…