These guidance notes are aimed at Disabled People's Organisations (DPOs) and are divided into different sections for ease of use. They provide an explanation of the key principles with additional hyperlinks for those people wanting in-depth information and real-life examples.
Resource contents
Monitoring and Evaluation Spotlight
Difference between ‘Monitoring’ and Evaluation’
The Logic Model – A way of designing projects and evaluating impact
How can we use the information that we have gathered through evaluation?
Telling the story of your work
Monitoring and Evaluation Spotlight
What is monitoring?
- Monitoring helps us keep track of our activity, and depending on what we are monitoring, it helps us understand our work in numbers and helps us check our progress against our plans for the work.
- Monitoring helps us to understand things like - How many people we are working with, how many times they have used our services, and how many different types of contacts we have had with them. Telephone calls/e-mails / in-person meetings/group activities/visits etc.
What is evaluation?
- Evaluation helps us understand the difference our work is making. Evaluation goes one step further than monitoring because rather than focusing on what we are doing, it looks more at what our work is achieving.
- Evaluation helps us to understand things like: Are we providing a good service? Do the people we work with experience our values?
Why is monitoring and evaluation important for a DPO?
- Monitoring and evaluation are important in every organisation but crucial for values-led organisations.
- It helps us produce evidence that we can share about our progress and helps us understand and demonstrate our work's impact to our staff, commissioners and funders.
It can help to generate helpful, evidence-based learning that we can use to improve how we do things in our organisation.
Introduction
We can sometimes think of evaluation as something that we do for people outside our organisation, such as for funders or commissioners, but monitoring and evaluation can be an excellent tool for us as leaders and staff, too. In this section on monitoring and evaluation, we introduce you to:
- The difference between monitoring and evaluation
- Key definitions
- Different tools that you can use for monitoring and evaluation
- How to use evidence to ‘tell the story’ of your organisation/work.
Before we go on, we’d like to inspire you with this quote from a DPO Leader:
“Ten years ago, 100% of our income as a DPO came from the council. Most of our contracts were monitored numerically –for example, by how many people we worked with. We knew that we made a big difference, but we didn’t need to talk about the kind of difference we made or how we achieved it. We just knew that we did.
When we lost all of our council contracts and had to start trying to bring in money from new places, we realised we didn’t have evidence to show what we could achieve (outcomes, impact) or how we did it (USP - Unique Selling Point). So we went on a considerable journey, and in a few years, we have funded programmes from 8 different funders and have a turnover of 1 million pounds. Getting our research and evaluation right was one of the most important things we did to achieve this. It means we get to do exciting work, serve more disabled people than ever, and advocate for the impact that a majority disabled staff team can make in our community” - DPO Leader.
Difference between ‘Monitoring’ and 'Evaluation’
Monitoring helps us to keep track of our activity, and depending on what we are monitoring, it helps us to understand things like:
- How many people are we working with?
- How many times have we seen them?
- What period of time are we working with them over?
- How many different types of contacts have we had with them? Telephone calls/e-mails / in-person meetings/group activities/visits etc.
It helps us to track our activity.
Evaluation helps us understand the difference we are making through our work. Evaluation helps us to understand things like:
- Are we providing a good service?
- What difference does our work make?
- Do the people we work with experience our values?
- What is the value of our project/activity/service/organisation?
A key aspect of evaluation is using information to learn and improve what we do or how we do things.
More information: Sport England’s Evaluation Framework
It is easier to think about evaluation as a cycle, where the focus is on generating useful insights. These insights can help you think about the difference you want to make and improve your project accordingly. The diagram below helps us think about the process this ‘cycle’ takes.
Alt text: The Evaluation Learning Cycle Diagram: In the centre, at the top of the diagram is a box with the title ‘What are the issues people face’, from this box is an arrow pointing down to a circular cycle with five boxes. The first box says ‘what difference do you want to make’, the second box says ‘what will you do to make this difference?’ The third says ‘how will you track your progress’ the fourth says ‘what have you learned’ and the fifth says ‘how will you use your findings’ which then leads back into the first box ‘What difference do you want to make’.
The Logic Model – A way of designing projects and evaluating impact
The logic model is increasingly used to plan organisational/monitor and evaluate activity. It helps us to think from the start, about why we are doing something, what we will do, how we define success, how we measure this success, and what we hope the longer-term benefits of this work will be.
We may be more familiar with SMART (Specific, Measurable, Achievable, Realistic and Timely) goals, but the logic model system is a far more sophisticated way to organise and plan around our strategic objectives. This is because it includes ways of linking purpose, activity, measurement and impact.
Once you have completed a logic model for all organisational objectives, you should create a project plan to implement each objective. Each organisation will have its own project plan templates, but an example logic model template from Sport England can be found below. This is an interactive document to which you can add content online and then save/download it to your device.
Example: Sport England’s logic model template
More information: Inclusion Barnet’s Management Handbook
How can we use the information that we have gathered through evaluation?
Evidence-led improvement
Building on our strengths
- Define what we are doing well.
- Plan how we can do more of this.
- Communicate what we are doing to build our profile.
- Share this knowledge with other people.
What could we be doing better?
- What needs improvement?
- What is our plan for addressing this?
- What extra resources do we need to do this work?
- What is our plan for making the above happen?
Telling the story of your work
Describing how change happens for the people you support
As someone working to create change for Deaf and Disabled people, you may already be used to telling the ‘story’ of your work – describing to the people who use your services, partners, and funders how your work makes a difference.
There are many ways to tell this story, from simple to complex. However, try to show the logical connections between your work (your outputs), the changes you create in the short to medium term (your outcomes), and any longer-term changes you might contribute to (your impact).
These story-building processes aim to clarify your strategic thinking and support an ongoing conversation about organisational purpose. It is often a great opportunity to draw in different voices and engage the people who use your services to understand how they see the purpose of your work and what they value most.
These stories also provide a solid foundation for evaluation by generating a testable theory about how the work being delivered creates change and clearly defined, measurable outputs and outcomes.
More information: Merton CIL’s Hate Crime Evaluation Handbook
Evaluation - Key Definitions:
Problem/Rationale: What are the issues that the people you want to work with are facing?
Objective: What is/are your goals for your project/service?
Activities: What is delivered to the people you are working with?
Input: What resources do you need to run your project/service?
Outputs: For monitoring – outputs are simply the number of activities you will deliver / the number of people you will deliver them to.
Indicators: What will you measure to understand if you have achieved your outcomes?
Outcomes: What changes are you trying to achieve in short and medium terms?
Impact: The long-term changes your work has achieved – the difference that your work makes over time.
More information: Merton CIL’s Hate Crime Evaluation Handbook
More information: Inclusion Barnet’s Management Handbook
Written in partnership between:
Disability Rights UK is the UK's leading organisation led by, run by, and working for Disabled people. They work with Disabled People's Organisations and Government across the UK to influence regional and national change for better rights, benefits, quality of life and economic opportunities for Disabled people.
Aman Ahluwalia-Hinrichs, founder of Birthing Ourselves, has 16 years of experience in social justice and community leadership development – this includes many years of having a leadership role in and working with DDPOs. Aman’s vision for Birthing Ourselves is to build a world where all leaders interested in social justice can access the tools, skills, training and relationships they need to achieve significant, transformational change. To find out more, visit wwww.birthingourselves.co.uk or e-mail Aman at aman@birthingourselves.co.uk
Inclusion Barnet is a north London based DPO that promotes independent living so that Disabled people can have choice and control over the way they live their lives. They provide information advice, advocacy and brokerage service for Disabled people.