Developing a shared approach to evaluation in the West

How can we meet the call for more appropriate and robust evaluation? How do we generate more rigorous and relevant evidence? These were some of the questions explored by more than 30 people with an interest in evaluation, including commissioners, researchers and the public, at a workshop on Wednesday 16 December 2015 organised by the West of England Evaluation Strategy Group.

The interactive workshop explored the principles of pragmatic evaluation, and was led by Dr Peter Brindle, Leader for Commissioning Evidence Informed Care at the West of England Academic Health Science Network (WEAHSN) and Evaluation Lead at the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care West (NIHR CLAHRC West). It was jointly organised by CLAHRC West, WEAHSN, Avon Primary Care Research Collaborative (APCRC) and Bristol Health Partners.

Peter set the scene of the workshop as a chance to learn, listen and discuss the opportunities and challenges for evaluation. He was clear that this was the beginning of a collaborative effort to create a shared common vision, strategy and, especially, an action plan for evaluation in the West of England with the potential for collaboration with similar initiatives across the country. He went on to provide his own reflections on why Evaluation is important, the barriers to evaluation and some of solutions going forward, explaining that we need to: “Choose the right tool for the job whether it is quality improvement, evaluation or research”.

Evaluation should cut across each stage of the service change / innovation / research pipeline #weevaluate15 pic.twitter.com/I1INYS2Hc7

— NIHR CLAHRC West (@CLAHRC_West) December 16, 2015

The key note speaker for the day was Professor Jo Rycroft-Malone, Director of the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) Programme. Jo described some of the challenges, opportunities and implications of pragmatic evaluation for the Programme. These included the implication for research methods, the need to agree what is ‘good enough’ evaluation, how to bridge the gap between those knowing and those doing, and what the rules of engagement are.

#weevaluate15 how to close the Know-Do gap. Better collaborative processes from @jorycroftmalone @WEAHSN @AvonNHSRD @CLAHRC_West

— Peter Brindle (@petbri) December 16, 2015

Lunch was followed by round table discussions on the following topics:

1. How do we best involve patients, carers and the public in evaluation?
2. How do we ensure that evaluation is of value to health and social care?
3. How do we build a culture of evaluation?
4. How do we create an ethical approach to evaluation?
5. How do we make the trade-off between rigor and relevance?

These table sessions inspired a broad range of discussions and reflections as well as a chance to share good practice and innovative ideas and solutions. These sessions were fed back to the room for further discussion. Themes emerging included the need for:

• genuine and early co-production in evaluation
• good early evaluation planning with all the right stakeholders and information
• development of capacity, skills and understanding of evaluation
• both rigorous and relevant evaluations

There was also interest in the Researcher-in-Residence model being driven by Martin Marshall at University College London, Peter Brindle through his work at the Avon Primary Care Research Collaborative and Lesley Wye at the University of Bristol. This was acknowledged as an excellent way of bridging the ‘know-do’ gap and supporting putting evidence into practice.

The final step involved participants making a commitment to action as a result of the workshop discussions, whether something they personally intended to do or that they would take back to their organisation. A large number of actions were pledged and these will be pulled together and followed up in the coming weeks. Examples include:

I am going to “lobby for equal status of evaluation and research”

I am going to “ensure evaluation is embedded in transformation at the outset”

I am going to develop a “key guidance sheet”

Throughout the event there was a positive and energetic atmosphere in the room, and post event feedback found that the event met attendees’ expectations fully (n=16, 89%) or at least partially (n=2, 11%). Comments included:

“An excellent event, really well put together”

“I think it has laid some great foundations”

Dr Peter Brindle said:

“I feel that the event has helped to create an energetic and supportive community of action in the West of England, to overcome the barriers to integrating evaluation into service change and innovation.”

The organisers would like to say a big thank you to all those who participated for their excellent contributions and commitment to actions to help drive this agenda forward – Thank you!

The full workshop and evaluation report is being pulled together and will be available in the coming weeks. This report will be used to inform the future vision, strategy and action plan for evaluation across the West of England. For more information or to get involved contact Jo Bangoura, Evaluation and Commissioning Lead on [email protected].

great to be part of principles of pragmatic evaluation event in Bristol today #WEevaluate

— Jo Rycroft-Malone (@jorycroftmalone) December 16, 2015