Evaluation. It’s something we often think of as aimed at a particular individual, or team. It's not something many roles usually have to have too much involvement in, especially in the development stages.
That’s something we wanted to do differently in the Get Yourself Active team at Disability Rights UK (DR UK). Our team has the knowledge and experience of working on the programme to develop an evaluation framework, but what could others bring to the process? Not only wider DR UK staff members, but people outside of our organisation too.
One thing we were keen to do from the beginning is to co-design the framework to ensure we are working collaboratively to shape what it looks like from the beginning. We value the insight from those we work with and want to make sure we capture this as part of our work.
Tim Bidey, an Impact Measurement Adviser who supported us with this work, said this - ‘The insights from Disabled people and representatives were fundamental in shaping the final product. Their contributions ranged from helping the DR UK team think more strategically and consider the potential influence of the evaluation on the disability sector more broadly, to providing more nuanced insights that will influence how the evaluation data is interpreted. For instance, highlighting the potential for attitudinal data to be skewed during a Paralympic year’
Developing our evaluation framework was essential in order to demonstrate and share the important work we’re doing, helping us to guide how we capture information about the progress and impact of the programme. This ensures we are able to demonstrate and share the work we’re doing and the difference that DR UK is making in the ‘bigger picture’. The framework outlines in a transparent, consistent, and systematic way the overarching questions DR UK aims to answer through our evaluation. It details the specific changes we intend to measure, the methods we will use to measure those changes, and how we will use the collected data.
In order to co-design the framework with members of DR UK from different parts of the organisation, as well as other external organisations and individuals, we adopted a two-phase approach that was designed to balance meaningful participation in a complex process with a manageable amount of information and a reasonable time commitment from the people involved.
The first phase involved equal participation from all stakeholders through a series of facilitated meetings that focused on determining the scope and focus of the evaluation. This approach ensured that everyone shared equal power and responsibility over the fundamental building blocks of the framework.
The second phase shifted to supporting the DR UK team to lead the development of the framework’s evaluation indicators and data collection methods, as these more technical tasks would have required an unreasonable amount of time for people to develop the required knowledge of the tasks and DRUK’s organisational capacity. Instead, Disabled people were supported to stay at the heart of the process though through reviewing, discussing, and signing off on draft and final versions of the indicators, methods and framework.
Bringing people into the process wasn’t always easy, especially with something as complex and time consuming as developing an evaluation framework.
‘Ensuring genuine and meaningful co-creation was challenging. One of the main early obstacles was overcoming the initial ‘information gap’ between the DRUK team and other participants. The Get Yourself Active programme is ambitious and complex, and the DRUK team entered the process with a clear understanding of its intricacies. As a result, it was crucial to spend time supporting people to understand the overall programme in the first instance, and then shortlist which programme outcomes might be evaluated to make detailed explanation and discussion more manageable’. – Tim Bidey
Another challenge was ensuring that everyone felt they could contribute meaningfully to the evaluation framework design process – which can be perceived as something reserved for ‘experts.’ We addressed this by using plain English whenever possible (for example, ‘changes’ rather than ‘outcomes’), offering mini-evaluation training sessions around more technical elements, and providing additional support as required.
We have now developed a robust framework that is going to be invaluable in helping us to capture information on our programme and measure the change we are contributing to. Working in this way allowed us to ‘think outside the box’, because outside perspectives often led us to think about aspects that we may not have considered by just working as an internal team.
“The approach DRUK have taken in developing their evaluation framework is really refreshing. They have adopted a really open and collaborative approach with multiple partners which has allowed us to share our thoughts and have some great discussions with others on a topic that is beneficial to us all. Being part of these discussions has also helped to reflect on our evaluation journey at Sense, and has supported the connectivity of us as System Partners" Alissa Ayling, Head of Sense Active
‘I’ve enjoyed being part of the work as it’s allowed me the time and space to think about evaluation in more depth than I’ve had before. I benefitted from the prioritisation discussions as with resource and capacity being tight it’s crucial to prioritise.’ – a member of the wider DR UK organisation who was involved in the process.
Our top tips for co-designing your own evaluation framework
- Break everything down into manageable segments or stages.
- Make sure information is easy to digest and that everyone can understand.
- Offer training for people who have less knowledge around evaluation.
- Be prepared to take steps back or for things to change direction, it’s all part of the process!