Design your Change Strategy
- Matt Kepple
Measure every aspect of your impact
Every project on Makerble has a Change Strategy. (this is similar to a logic model or theory of change)
It's common to create a project for each intervention, campaign, programme or service you manage or fund.
If you do the same work in different geographic locations, it is helpful to create a project for each location.
A Change Strategy lets you plan and measure the change that happens as a result of your project. (You might already think of this as outcomes monitoring, evaluation or impact measurement.) The progress towards each Change will be displayed on the Progress Panel of that project.
7 Steps to design your Change Strategy
Who benefits from your intervention?
Which outcomes do your impact recipients experience?
Who can confirm whether the outcomes have been achieved?
Which questions tell you the extent to which each outcome has been achieved?
When will those questions be asked and observations made?
Where will the answers and observations be stored?
How, When and for Whom do you want to report on the results?
1. Who benefits from your intervention?
The people who attend your intervention are the obvious group of impact recipients. You might already refer to them as Beneficiaries, Clients, Members, Participants, Patients, Service Users, etc.
There might be people or groups who indirectly benefit from your intervention as well.
Here are six additional groups who might indirectly benefit from your intervention:
Friends and Family members of participants
Professionals who support them
Local Community Members whose lives are affected by the behaviour of the participants
Society at large
Government
The Environment
On Makerble you can
Create a Contact Profile for each individual impact recipient.
Alternatively, only record the number of each type of impact recipient
2. Which outcomes do your impact recipients experience?
Don’t underestimate the insight you and your team have. Based on what you have heard anecdotally, what differences does your programme make to the lives of your participants and the other impact recipients?
Think about those differences in terms of
What difference does your programme make to how people think?
What difference does your programme make to what people or organisations do; and how they behave?
What difference does your programme make to what people or organisations have; be it financial wealth or mental health?
You will find these terms less ambiguous than thinking about ‘short-term outcomes', ‘medium-term outcomes’ and ‘long-term outcomes’ which can get misinterpreted due to the length of a 'term’ being subjective.
There are other ways to discover your outcomes beyond looking anecdotally at the difference you make. Book a call to talk to us about Outcome Discovery.
3. Who can confirm whether the outcomes have been achieved?
Your staff and volunteers are a good place to start. They will observe the difference that is being achieved for your participants
Your participants themselves will have a good perspective
There are additional people, organisations and digital platforms that can provide a perspective on whether an outcome has been achieved. Examples:
People: each of your Impact Recipients could potentially give their opinion on whether outcomes have been achieved for themselves or for the participants
Organisations: organisations you partner with or which also work with that participant are able to provide a perspective on the outcomes that have been achieved for that participant
Apps and Digital Services: if your programme improves people’s fitness, rather than asking your participants if they feel fitter, you could connect to their Fitbit accounts to see how many steps they have been taking on average each day.
On Makerble you can add Perspective Providers to each contact.
A Perspective Provider is someone who can provide a perspective on a contact’s progress towards change.
You can collect Perspective Providers' perspectives using 360° Surveys.
4. Which questions tell you the extent to which each outcome has been achieved?
Your questions can take various formats in addition to simply asking open questions. For example, indicators relating to gaining employment:
Yes or No indicators: e.g. “Did the participant find employment, yes or no?”
Scale indicators: e.g. “How much do you agree with this statement: “This programme has enabled me find employment; Strongly Disagree, Disagree, No Comment, Agree, Strongly Agree?”
Numerical indicators: e.g. “How many jobs did you apply for this week?”
When asking questions, you can apply these principles to ensure they stand up to the scrutiny of commissioners, funders and the media:
Before and After: rather than asking someone at the end of your programme if they felt that your programme contributed to them finding employment; ask someone at the start of the programme how fulfilled they were in their last job and then ask them at the end of the programme how fulfilled they are in the job you have helped them find. This will accurately show the difference you have made.
Questions should be Fair; Not Leading: Rather than asking: “Has this programme been life changing for you, yes or no?” ask someone to rate their experience of your programme on a scale.
Observable Not Subjective: while asking people how they feel about something is useful, when you are asking people how they feel about themselves, what you are actually asking is their level of self-awareness. Often times, programmes ask people how confident they feel about something and people usually say their confidence is high. Then they attend the programme and realise that they are underperforming. When they are assessed at the end of the programme they are wiser and more humble and their answers show that they are less confident than when they started. This looks like a negative outcome. Which is why it is better to ask questions about specific things people do rather than asking them about their own perception of what they do.
On Makerble you can create a Progress Tracker for each type of question:
Yes or No questions: use an Achievement Tracker
Scales with Levels: use a Multiple Choice Tracker
Numbers with Units: use a Numerical Tracker
You can add those questions to a survey to capture responses to one or more questions at once. You can then:
5. When will those questions be asked and observations made?
Programme Start: Where it is appropriate, assess your participants as early in their engagement with your programme as possible. This will give you baseline data from which you can see the difference caused by your programme.
During the Programme: The people on your team who work with your participants can record their meeting notes and observations about the participant on an ongoing basis. You can also assess how your participants are progressing towards each outcome. This will give you a truer sense of the journey that your participants go on as some will progress through the outcomes faster than others. This will give you the data to demonstrate that.
Programme End: This is the most common moment to ask questions and typically these are of the participants in the form of a survey.
6 Months After the Programme: Checking in with your participants 6 months later is a good way of proving the lasting impact of your programme.
Several times After the End of the Programme: This provides even more proof of the difference that your programme has made. The stories and data about these people for whom you do longitudinal studies, i.e. question them over a long period of time, will be valuable and is persuasive to commissioners and funders.
Makerble automatically timestamps each survey response so that you can how people’s responses to questions vary over time (Distance Travelled). However you can also:
Use Waves to categorise survey responses e.g. as being “Baseline”, “Midline” and “Endpoint”
6. Where will the answers and observations be stored?
In your head
On paper
In spreadsheets
In online survey systems
In one place: a database. This is best for reporting, for understanding cohorts of participants over time and for complying with GDPR.
On Makerble your survey responses are stored within a Survey Campaign. A survey campaign exists for each Survey used on each Project (intervention).
7. How, When and to Whom do you want to report on the results?
How do you want to report on the results?
Manually created reports: each time you want a report, you manually create one
Automatically generated reports: you instruct your database to create a specific report for you at a specific time each week, month, quarter, year, etc.
Interactive reporting dashboards: rather than producing reports, you give the people who want the reports access to a dashboard where they can explore your results for themselves
On Makerble you can view survey results in several places
Considerations
Your approach can improve over time
There is so much that you could do when it comes to measuring change.
It’s understandable to feel overwhelmed!
The main decision is the decision to start measuring change, even if it’s in a low effort way to begin with.
As your organisation grows, you can always improve your approach over time.
Ways to measure change
Whichever approach you take to measuring change, Makerble has tools to help.
Method | What it means |
|---|---|
Case Studies | Stories and anecdotes which give individual examples of the difference your intervention makes |
Outputs | The number of activities you do (Activity Trackers) and the number of attendees you have (Attendance Trackers) |
Outcomes | Statements for each of the improvements that your intervention makes. I.e.
|
Indicators | Questions that tell you the extent to which an improvement (outcome) is happening, rather than simply saying that it happened. I.e. |
Distance Travelled | The difference between a person’s initial response to a question (indicator) e.g. asked before the intervention and their final response to that same question e.g. asked after the intervention. The difference between the start and the end is called the Distance Travelled. |
Triangulation | Combining different people’s perspectives on the extent to which progress has been made. E.g. asking fellow pupils, parents and teachers whether a pupil’s attitude towards learning has improved This adds more confidence to your results as it shows that you are not just relying on a single source of information but that instead you are able to back up what you are saying about progress because you have several 'witnesses' of the progress, which you can include on Makerble by using perspective providers. |
Comparisons (Impact) | Compare what happens to people who experience your intervention with what happens to similar people who do not experience your intervention. The people who don’t experience your intervention are called a “control group”. The difference between what happens to the people in these two groups is called your “Impact”. This comparison will prove that your intervention is needed because it arms you with evidence to say that if your intervention was not there, “this is what would have happened”. This approach is sometimes used in medical trials and scenarios where it is necessary to prove that a particular intervention genuinely does make a difference compared to what would have happened anyway. |
Help is at hand
At Makerble our Strategists are on hand to walk you through the 7 steps for your intervention(s).
We work alongside you to create a measurement plan that meets the needs of your work.
We help you balance the effort involved with the upside of the improved trust in your impact.