Can we get better at generating impacts from research? How can evaluation help?

By Dr Laura Meagher, Technology Development Group and Visiting Fellow, Innogen and
Dr Catherine Lyall, Innogen

Like all governments that invest in university research, the UK government hopes to see returns, not only in terms of academic excellence, but also in the form of economic benefits and other societal impacts including health, culture, education, justice and well-being1. This is particularly apparent as we await further details from the pilot exercise being conducted by HEFCE as part of the consultation on the Research Excellence Framework2 which proposes to assess, for the first time, demonstrable economic and social impacts deriving from academic research.

We have conducted several evaluations of such non-academic impacts. In the process we have learned about the elusive, subtle, diffuse and long-term nature of impacts which, even where identifiable, can be virtually impossible to attribute to individual research projects. Identification of tangible impacts is especially challenging when many social science research projects potentially contribute, not to the commercialisation of a product, but to policymaking and professional practice.

But these evaluation challenges can also be seen as opportunities – chances to deconstruct and improve understanding of those processes which can mean that research has a greater impact beyond academia. By identifying what characterises these knowledge exchange processes we can help the wider research community to adopt them more readily in the future in ways that complement (rather than conflict with) the generation of excellent research.

An array of types of impacts is currently recognised(3) including:

• Instrumental use (“direct impact of research on policy and practice decisions”)
• Conceptual use (“where research changes ways of thinking, alerting policy makers and practitioners to an issue or playing a more general ‘consciousness-raising role’)
• Capacity-building (this can refer to education, training or even development of collaborative abilities)

We suggest that there are also indicators of demand as well as process-oriented effects that can be identified as proxy indicators of relative likelihood that long-term impacts are in train:

• positive changes in institutional cultures and individual attitudes toward knowledge exchange
• enduring connectivity, when researchers and prospective users stay in contact even after a funded project ends.

We have sought to understand these processes in diverse evaluations(4) that recognise the inevitable heterogeneity that exists among researchers, research problems and users. We have found that(5):

• impacts can take a long time to manifest themselves, and can go through various stages or iterative cycles of development (albeit not in neat linear fashion)
• recognition or incentives for the generation of non-academic impacts can make a difference
• supporting ongoing interactions and dialogue between researchers and users can foster two-way knowledge flows leading to longer-term uptake or utilisation of research findings
• funding makes a difference – infrastructure and dedicated staff playing a “research brokerage” role can expedite the knowledge exchange process while removing some of the burden from a busy researcher
• skilled, targeted communication can increase the perceived accessibility and relevance of research
• knowledge intermediaries (in the media, a funding body, professional organisation, or professionally qualified individuals such as advisory board members) can play key roles
• availability of flexible “bridge” funding (such as the ESRC’s Follow-on fund) might help prolong dialogue and help users develop and incorporate research ideas.

But there is much more to learn. There is a reservoir of tacit knowledge and both academic and non-academic participants in the knowledge exchange process have many useful lessons to share. We need to build on this learning and answer some deeper questions in order to probe the dynamism of knowledge exchange processes and hopefully capture more research impacts in the longer term.


Notes

1. See, for example, ESRC (2009), ESRC Strategic Plan 2009-2014. Delivering impact through social science.
2. HEFCE Research Excellence Framework Impact Pilot Exercise
3. See, for example, Nutley, S., Walter, I. and Davies, H. (2007), Using Evidence. How Research can Inform Public Services, (Bristol: Policy Press), p.36.
4. These include a Scottish scheme funding centres in emerging research areas, a UK-wide programme offering catalytic seed grants, a set of ESRC responsive-mode grants in one discipline, an assessment of ESRC research brokerage mechanisms and an ESRC/EPSRC/DTI programme in an emerging area combining physical and social sciences. For details see Innogen Working Paper 77
5. Meagher, L.M., Lyall, C. and Nutley, S. (2008), “Flows of knowledge, expertise and influence: a method for assessing policy and practice impacts from social science research” Research Evaluation Vol. 17(3): 163-173.