Describe the difference your work makes Build your framework for

Loading...

Describe the difference your work makes Build your framework for evaluation

1

Acknowledgements This guide has been developed by Eve Blair for Charities Evaluation Services and funded by Trust for London.

Design and production by Julia Essame

Published by Charities Evaluation Services

Charities Evaluation Services (CES) is a company limited by guarantee. Registered in England and Wales number 2510318. Registered charity number 803602. Registered charity in Scotland (number SCO40937). Registered office address: 4 Coldbath Square, London, EC1R 5HL, UK.

Copyright

Unless otherwise indicated, no part of this publication may be stored in a retrievable system or reproduced in any form without prior written permission from CES. CES will give sympathetic consideration to requests from small organisations for permission to reproduce this publication in whole or in part but terms upon which such reproduction may be permitted will remain at CES’ discretion. © Charities Evaluation Services 2013 ISBN 978-0-9568090-2-5

1

Contents Section One: Using the CES Planning Triangle to build a narrative of how your work creates change  Introduction  Beginning your Planning Triangle – defining your overall aim  Defining your specific aims  Completing your triangle – adding in your activities  Reviewing your completed triangle  What next? 

Section Two: Developing a practical, streamlined evaluation framework  About the evaluation framework  Before you begin  Describing your outputs  Setting output indicators  Describing your outcomes  Setting outcome indicators  Defining your impact  Completing your evaluation framework 

5 5 6 10 2 15 16

17 17 18 18 19 21 23 27 28

Glossary 

29

Resources 

31

2

Introduction About this guide

Third sector organisations are under growing pressure to be able to describe and evidence the difference their work makes, both in the short to medium term (their outcomes) and in the longer term (their impact). For organisations looking to become more effective at what they do, and to compete in an ever more competitive funding environment, outcome and impact evaluation – the process of gathering and reviewing evidence of the difference their work makes – has never been more vital. However, many remain uncertain about where to start, while others find themselves either drowning in data or operating in an information vacuum that makes evaluation impossible. This guide is designed to help organisations plan for evaluation more effectively using a monitoring and evaluation framework. The foundations of the framework are built through using the CES Planning Triangle – a simple tool that helps you to reflect on, and clarify, the connections between the work you deliver and the changes that you create. The guide is broken down into two sections.

Part one: using the CES Planning Triangle to articulate how your work makes a difference

Identifying and agreeing your outcomes and impact is the first step towards evidencing them. The first part of this guide will help you to use the CES Planning Triangle to articulate how your work makes a difference in the short, medium and longer term. Using the Planning Triangle will help you to: • • •

plan for a new project, or clarify the purpose and aims of an existing project demonstrate how different activities or services contribute to the overall aim of your organisation or project communicate the purpose of your work more effectively, both internally to staff and volunteers, and externally to funders, partners, and service users.

Part two: developing a practical, streamlined framework for evaluation

To understand, evidence, and learn from your outcomes, you need the right information to hand. The second part of this guide will show you how to break down the elements of your planning triangle into a practical, streamlined framework for evaluation, setting out exactly what information will be collected, why, when, how and by whom. Developing an evaluation framework will help you to: • • • • •

clarify the outputs, outcomes, and impact of your work decide which pieces of key information you need to gather as evidence (your indicators) make sure that you have all the right information you need to evaluate avoid wasting time collecting unhelpful, unnecessary or duplicated information make better decisions about how, when and why to collect the information you need.

How to use the guide

This guide sets out the key steps to creating a narrative of your work, and developing an evaluation framework using the CES Planning Triangle. Each step is illustrated with worked examples for frontline, infrastructure, and campaigning organisations. Completed triangles and 3

frameworks for each organisational type can be found in the resources section of this guide, along with templates for creating your own Planning Triangle and framework. Each step comes with a checklist, to help you move through the process with confidence.

Before you begin

Agreeing and articulating your organisation’s purpose and outcomes, and deciding what information you would need to gather as evidence, is best done as a group discussion rather than by one person working on their own. Colleagues, trustees, partners and service users will all have something to add to the process, and involving them will help to ensure that you build an accurate narrative and a sensible, robust framework for evaluation.

4

Section One: Using the CES Planning Triangle to build a narrative of how your work creates change Introduction

Most organisations are driven by a clear sense of shared purpose and mission, but this is not always captured on paper. This can lead to problems when it comes to communicating the value of your work, and developing a plan for evaluation. The CES Planning Triangle is a simple, visual tool for making clear the connections between the work that you do, the changes that your work creates, and how these changes contribute to your overall mission and purpose. The tool is useful for planning new projects and thinking about the best way to respond to an identified need. It is also helpful for clarifying and articulating the changes that you want to achieve with an existing project or area of work. The Planning Triangle asks you to begin the story of your work by thinking about the change that you want to create, rather than the services, goods or activities that you want to provide. This can often be helpful if you are used to talking about your work in terms of ‘what you do’ rather than in terms of what changes as a result, and if you are struggling to define your outcomes or impact. A properly completed Planning Triangle can turn even very complex interventions into a powerful, easily understandable story of change. The triangle breaks this story down into three levels.

The Planning Triangle

Overall aim

Specific aims

Activities

The broader, longer-term change you want to contribute to

The changes you want to create

The work you do

© CES Planning Triangle Note: The term objective is often used in the third level of the triangle, rather than the term ‘activities’.

Overall aim

Your overall aim sets out the ‘big change’ that your organisation wants to be a part of – the purpose or mission of your work. Usually, your overall aim describes the longer-term, broader 5

difference that you want to contribute to, rather than a change you plan to achieve singlehandedly. This overall aim sits at the apex of the Planning Triangle; all of the more immediate changes that you want to create through your work, and all of the actual work being delivered, sit beneath it and support it.

Specific aims

Your specific aims describe the areas of change that you hope to create in the short- to mediumterm as a result of your work. These are the changes that have to happen in order to support progress towards the big, longer-term change described in your overall aim.

Activities

Finally, your activities set out the areas of work that you aim to provide, in order to achieve your specific aims. It is possible to read a completed triangle from the bottom to the top, or vice versa: Either: In order to contribute to our overall aim, we will have to achieve our specific aims; in order to achieve our specific aims, we will have to provide these activities. Or: We provide these activities, so that we can achieve our specific aims – and we want to achieve our specific aims, so that we can contribute to our overall aim. Whichever way you read your completed triangle, its usefulness lies in the fact that it sets out a series of simple, logical connections that can help you, and other people, to understand how you plan to make a difference through your work. We are using three examples of voluntary sector projects throughout the workbook to illustrate how to develop a planning triangle and framework. These are: • • •

an employment project for people leaving prison a campaign to promote sustainable living a project to build capacity in women’s sector organisations.

Three completed triangles and frameworks are included in the Appendix of this publication.

Beginning your Planning Triangle – defining your overall aim

Firstly, decide the scope of your Planning Triangle exercise. Do you want to capture the story of the organisation’s work as a whole, or of a single, specific project? Bear in mind that if you are new to the process, it is much easier to begin by focusing on a specific area of work.

What is an overall aim?

The Planning Triangle is designed to help you focus on the changes that you want to create with your work. As such, it is completed from the top down, beginning with your overall aim. Everything else in the triangle supports the overall aim, so it is important to spend time making sure that this part of your triangle is clear and realistic. There are a number of key steps to developing a robust overall aim. These are outlined below.

6

How to define your overall aim 1. Who is your target group? First, think carefully about who or what you want to create change for over the longer term. Who are the people or groups that will ultimately benefit from your work? For example:

Who is our target group? People leaving prison Local communities The women’s sector Bear in mind that these groups may differ from the people experiencing more immediate changes as a result of your work. For example, if you are working with parents to improve their parenting skills, the ultimate beneficiary of your work will be their children. Equally, if you are lobbying policymakers about the living conditions of refugees, the end beneficiary you have in mind is the refugee community, not the policymakers themselves. 2. Where are your target group based? Clarify your target group by adding in where they are – for example, do you want to support all people leaving prison, or will you work over a particular region? Are you aiming to contribute to change for the women’s sector across the whole of England and Wales, or in a specific city? This is the first step to making sure that your overall aim is set at a realistic level.

Who is our target group?

Where are they based?

People leaving prison Local communities The women’s sector

In Northern Ireland In Devon In London

3. What issue does your target group face? Think of the ‘big-ticket’ issue or problem faced by your target group. This will be the need that drives your work and the area where you want to create change in the longer term. This doesn’t have to be an issue that your organisation can solve quickly, or singlehandedly. It should be a long-term problem where your organisation’s work will be part of the solution, rather than a problem that your organisation can solve on its own. For example:

Who is our target group?

7

Where are they based?

What is the big issue they face?

People leaving prison

In Northern Ireland

Local communities

In Devon

The women’s sector

In London

They don’t have enough access to employment. They are not living sustainable ’green’ lifestyles. It is not operating as effectively as it could be.

4. What is the positive change you want to see? Having clearly set out the problem that your target group faces, simply turn this negative statement into a positive statement of change to spell out your overall aim. You should be left with a clear statement that sets out how you want to make a difference in the longer-term, for whom, and where. For example: Our overall aim is: • • •

To improve access to employment for people leaving prison in Northern Ireland To increase sustainable living among local communities in Devon To improve the effectiveness of the women’s sector in London

Before you move on... Since your overall aim dictates everything else that goes into your Planning Triangle, it’s worth checking it carefully before moving on to the next stage.

Does your overall aim describe a change?

Note that all of the examples above use the language of change – increase, reduce, maintain, improve, decrease. If you find that your overall aim is describing what you’re doing – providing, offering, running, organising – you need to step back and consider the change you want to create, rather than the services you want to offer.

Is there anything missing?

In order to be accurate, realistic and easily understood, your overall aim will need to include (a) what changes, (b) for whom and (c) where. Other example overall aims: • • • • • •

To increase the number of young people in Lewisham remaining drug free after treatment To reduce the number of people who die by suicide in the UK To improve the quality of life for older people living with macular degeneration in Wales To maintain biodiversity in Northumberland To reduce homelessness among English ex-servicemen and women To decrease child poverty in the UK

8

Is it realistic?

Remember, your overall aim sets out the area of change you want to contribute to – not a change that you’re claiming you can bring about single-handedly. However, you should still think critically about the scale of your overall aim before adding it to your triangle. It is important to make sure that your overall aim is set at the right level for your organisation, its resources, remit, and capabilities. For example, if your organisation is small and focused in its work, the aim should be similarly focused.

Is it clearly expressed?

Watch out for woolly overall aims using language around improving access, enabling people, or empowering groups. If you have an overall aim around improving opportunities, or accessing rights, spaces, services or activities, think carefully – why does access matter to people? If you have an overall aim that talks about enabling or empowering, make sure that it is clear what people are being ‘enabled’ or ‘empowered’ to do. For example: A group is working with young people on a local estate to increase access to positive activities, because without anything positive to do, young people become involved in antisocial behaviour. Is their overall aim ‘to increase local young people’s access to positive activities’? Or, is it actually ‘to reduce antisocial behaviour on the Northwick estate’? A new project is working with pregnant women held in immigration detention centres, linking women with community midwives and care professionals. Without this care, the health of mothers and babies might be at risk. Is their overall aim ‘to enable pregnant women in immigration detention to access prenatal care’? Or, is it ‘to improve the health of pregnant women and their babies in immigration detention’? Be cautious of recycling existing material into an overall aim. Often, the change that goes in the top of the triangle as your overall aim will already be captured somewhere as your ‘mission’. This might need clarifying and reviewing before you begin completing your triangle. Has your overall aim changed over time? Is it clear enough to be understood by funders, service users, and the general public?

Does it contain one big idea?

Your overall aim should be written as clearly and briefly as possible. It should contain one big idea, rather than a number of smaller ideas stuck together. The examples in the right hand column show how to make your aim clear and simple.

Too many single ideas

One clear statement

To improve access to employment, drug treatment, and social activities for homeless single men in Glasgow

To improve integration into the local community for homeless single men in Glasgow

9

To help older people in Southwark manage their money better and reduce debt

To improve the financial wellbeing of older people in Southwark

Once you are happy with your overall aim, you are ready to move on to the next stage of completing your triangle – defining your specific aims.

Defining your specific aims What are specific aims?

Your specific aims set out the changes that your work creates in the more short to medium term. These are the changes that you will have to achieve in order to contribute to your overall aim. They are the changes that sit beneath your overall aim, and support progress towards it.

How to define your specific aims

There are a number of key steps to defining your specific aims; these are set out below. 1. What are the barriers to your overall aim? To define your specific aims, think carefully about the barriers that stand in the way of your overall aim being achieved. This table shows examples of barriers that might prevent overall aims from being achieved.

Overall aim

Barriers

To improve access to employment for people leaving prison in Northern Ireland

• • •

To increase sustainable living among local communities in Devon

• • •

Ex-offenders lack the skills and work experience employers are looking for. Ex-offenders lack confidence around applying for jobs. Employers are unwilling to engage someone with a previous conviction.

Local government does not invest enough in sustainable transport options. Sustainability initiatives are being carried out by individuals, but not by communities. Local people do not understand how to live more sustainably.

10

To improve the effectiveness of the women’s sector in London



• •

Organisations cannot survive and grow without access to longer-term grant funding. Organisations do not work in partnership often enough. Skills, expertise and resources are not shared across organisations.

2. How do you plan to address these barriers? Next, simply turn the barriers into positive statements of change. These are your specific aims – the changes that your work seeks to produce in the short to medium term, and which will support progress towards your overall aim. The table below shows examples in the right hand column of how identified barriers have been used to develop specific aims. Barriers

• • •









• •

11

Specific aims

Ex-offenders lack the skills and work experience employers are looking for. Ex-offenders lack confidence around applying for jobs. Employers are unwilling to engage someone with a previous conviction.



Local government does not invest enough in sustainable transport options. Sustainability initiatives are being carried out by individuals, but not by communities. Local people do not understand how to live more sustainably.



Organisations cannot survive and grow without access to longer-term grant funding. Organisations do not work in partnership often enough. Skills, expertise and resources are not shared across organisations.



• •

• •

• •

To improve the skills and work experience of ex-offenders To increase ex-offenders’ confidence in applying for jobs To improve employers’ perceptions of candidates with a previous conviction

To increase local government investment in sustainable transport options To increase the number of community sustainability initiatives To improve the awareness of local people around sustainable living

To increase access to sustainable funding To increase partnership working between organisations To increase skill-sharing and resource sharing among organisations

Before you move on... Have you used the language of change?

Your specific aims spell out changes that you want to create. Make sure you are using the language of change, for example: • • • • •

To increase To improve To enhance To reduce To prevent

Do not fall into the trap of describing the work you want to deliver – this is the next stage of the Planning Triangle. If you find yourself using words like ‘deliver’, ‘organise’, ‘run’, ‘provide’, then you will need to step back and think carefully about the changes you want to achieve, not the work you want to provide.

Are your specific aims clear and concise?

You should aim for around three to five specific aims and these should be expressed as clearly and as briefly as possible. Don’t worry if your specific aims seem difficult to measure – at this stage, you are working to describe the changes that your work produces. The second section of this guide sets out how to turn this description into a plan for measurement.

Is there a clear, logical link between your specific aims and overall aim?

There should be a strong, logical link between your specific aims, and your overall aim. This is an important part of building a clear narrative of how your organisation makes a difference in the short to medium term, and the longer term. Look carefully at your specific aims – do they link logically to your overall aim? Does the story make sense? Are there any key pieces missing, or are there specific aims which do not seem to fit, and which do not contribute towards your overall aim? If the link between your overall aim and specific aims still seems unconvincing or unclear, you may have to spend some time working together to tweak and amend them until you are happy that they make sense, and provide an accurate account of what your work sets out to do. Having completed your specific aims, you are now ready to complete your triangle by adding in your activities.

Completing your triangle – adding in your activities What are activities?

Your activities sit beneath your specific aims in the Planning Triangle. They spell out how you plan to achieve your specific aims, completing the story of your work.

12

How to define your activities 1. What needs to be done? Think carefully about your specific aims. What sort of work would you have to do in order to bring those changes about? Which goods, services, and products would you have to offer? The examples below show the link between the change you want to make and your planned activities.

If you wanted

Your activities might be



To improve the skills and work experience of ex-offenders



To provide one-to-one support around writing CVs and job searches



To increase ex-offenders’ confidence in applying for jobs



To signpost ex-offenders on to other skills training services To run workshops around disclosing criminal convictions To facilitate work experience schemes for ex-offenders with local employers To run basic skills courses in prisons

• • •



To improve employers’ perceptions of candidates with a previous conviction



To run awareness-raising events for local employers



To increase local government investment in sustainable transport options



To lobby local government around investing in sustainable transport



To increase the number of community sustainability initiatives



To provide a ‘Transition Streets’ programme for local communities



To improve the awareness of local people around sustainable living



To run a website providing information on sustainable living To run a campaign around sustainable living To put on a sustainable living roadshow across Devon

• •

13



To increase access to sustainable funding



To run funding clinics providing help with applications



To increase partnership working between organisations



To organise ‘meet the funder’ events locally To facilitate partnership working between local organisations

To increase skill-sharing and resource sharing among organisations







To provide networking and skill sharing events

Before you move on... Do your activities describe what you plan to do, and not what changes as a result?

This is the part of the Planning Triangle that describes the work you do, and not what changes as a result. Your activities set out the areas of work you plan to deliver; they should use the language of delivery, for example: • • • • • •

To provide To run To offer To deliver To organise To facilitate

Make sure you are using the language of delivery, and not describing what changes as a result.

Are your activities clear and concise?

You do not need to write in a different activity for each specific aim – it may be that one activity contributes to several of the specific aims. If you only plan to offer one activity (for example, if you are a counselling service running one-to-one sessions, and nothing else) this is fine. Try to keep the number of activities to under six, all concisely expressed. This will make your Planning Triangle easier to read and understand.

Is there a logical link between your activities and the changes you want to create?

Once you have described your activities, look carefully at the relationship between your activities and your specific aims. Given the activities you plan to provide, do your specific aims still seem realistic and achievable? If you are completing the triangle for an existing project, you may find that you have identified ‘orphan’ activities – an area of ongoing work that does not link to any of your specific aims. Often this is the basis of a separate discussion. If you are delivering work that does not support your purpose and mission, do you want this area of activity to continue?

Have any internal processes crept into your activities? 14

Your activities set out the goods, services and products that you will need to provide in order to achieve your specific aims. The Planning Triangle does not include you organisation’s internal processes. These are all of the processes that you would need to get right in order to run an effective service, for example: • • • • • • • •

fundraising research evaluation staff training health and safety quality assurance recruitment marketing.

Because the Planning Triangle focuses outwards on the change that you want to create through your work, it assumes that all of these internal processes are in place and working well. Before completing your triangle, review your activities and take out any internal processes, so that your triangle only focuses on the work you deliver, and the changes you want to create. Once you are happy with your activities, you can move on to reviewing your completed Planning Triangle.

Reviewing your completed triangle Reviewing the story

Before moving on to developing your evaluation framework, pause and review your completed Planning Triangle. Check that the story told by your triangle is: Clear Your aims and activities should be clearly and simply expressed. Make sure that they are worded concisely, and that you do not have too many specific aims (no more than five) or activities (no more than six). Logical Consider how each level of the triangle fits together. Would it be easy for someone unfamiliar with your organisation and its context to follow the story of how your work will create change? Does each part of the story link together logically? Think of your completed triangle as an arrow, with everything pointing towards the overall aim at its apex. Is there anything missing, or is there anything in there that doesn’t fit in? Compare your overall aim and your specific aims to make sure that your overall aim spells out the biggest, longest-term change in your triangle. If you have a specific aim that is larger or more ambitious than your overall aim, you may need to swap these over, or reconsider what should go in the apex of your triangle. Realistic Review your completed triangle with a critical eye. Is the story it tells realistic and achievable? Examine the activities you plan to provide – can these realistically create the kind of changes you 15

have set out in your aims? Recognisable Would the story told by your triangle be recognisable to your colleagues, Trustees, partner agencies and service users? Developing a Planning Triangle as a group can often be a useful way of building a realistic and accurate description of your work. Evidence based If you are planning an entirely new project, you may have research to support your story of how change happens, or the type of activities that would deliver the best results against your specific aims. If you don’t have this sort of evidence, bring in your experience of running similar projects – how does the story set out in your completed triangle compare to these previous experiences? Clear Your completed triangle should be useful for describing the value of your work both internally and to the outside world. Make sure that it is free from jargon and acronyms!

What next?

Once you have used the Planning Triangle to describe your work and the areas of change that you want to bring about, you will be ready to begin thinking about what evidence you might need in order to evidence the difference your work makes. This can be done by developing an evaluation framework. The key steps to this process are outlined in section two of this workbook. You may also decide to develop your Planning Triangle into a more fully developed ‘theory of change’. A theory of change creates a more detailed and in-depth narrative about how, when and why change happens as a result of your work. It also asks you to think critically about the assumptions that underpin your story about how change happens as a result of the activities you provide. CES’ publication Making Connections: Using a Theory of Change to Develop Planning and Evaluation contains helpful, practical guidance on developing a theory of change for your organisation.

16

Section Two: Developing a practical, streamlined evaluation framework The first section of this workbook describes how to use the CES Planning Triangle to set out how your work makes a difference in simple but powerful terms. This second section sets out the key steps to breaking down your completed Planning Triangle into a framework for measurement, helping you to clarify which pieces of information you might need in order to evidence your story of change.

About the evaluation framework

Evaluation is the process of using information to make judgements on how an organisation, project or programme is doing. The quality of your evaluation depends on the quality of the data you have to hand, and this means that your monitoring – the way you systematically collect and record information – is absolutely key. Developing a sensible, robust evaluation framework will help to make sure that your monitoring collects everything you need for your evaluation and nothing more. In turn, this avoids gaps, unhelpful duplication, and needless frustration further down the line. This section of the workbook will take you through the key steps involved in developing an evaluation framework: • •

Firstly, by breaking down your activities and specific aims into a list of measurable outputs and outcomes Secondly, by setting output and outcome indicators – well-defined, easily measurable pieces of information which show how well you are doing.

Breaking down your Planning Triangle

Overall aim

The broader, longer-

Impact

term change you want to contribute to

Specific aims The changes you want to create

Activities

The work you do

Outcomes

Outputs

Indicators

Indicators

© CES Planning Triangle Understanding exactly what information you need allows you to build a practical plan for data collection, including deciding how and when to go about gathering data, and who should be responsible for collecting it. You will need to think about these questions in order to complete your evaluation framework; however, this workbook does not cover data collection in detail. For more information about choosing and applying data collection tools, see Information Collection 17

Methods: Choosing tools for assessing impact. (Further resources on evaluation are included in the ‘Resources’ section of this workbook.)

Before you begin

It is best to work in a group when you define your outcomes, decide on indicators, and set priorities for collection. It is good practice to include people who will be collecting the data when you develop your evaluation framework. You may also wish to include beneficiaries, volunteers, Trustees and partner organisations.

Describing your outputs What are outputs?

Before you can decide which pieces of information you will need to collect in order to report effectively on the work you do, you will need to develop a complete list of your outputs – the specific goods, services and products being delivered. You can build this list by revisiting your activities, which describe a general area of work, and writing down all of the specific outputs that fall within these areas.

How to describe your outputs

In some cases, building this list of outputs will be very straightforward. We can see this in the examples in the following table.

Activity – description of an area of work

Outputs – specific goods, services or products being delivered

To run workshops around disclosing criminal convictions To provide 1-2-1 support around writing CVs and job searches To run awareness raising events for local employers To signpost ex-offenders on to other skills training services

Workshops 1-2-1 support session Events Signposting

In other cases, activities may be ‘hiding’ more than one output, and you will need to pull these out to build a complete list. For example:

Activity

Outputs

To run a poster, leaflet and online campaign around sustainable living

• • • • • •

To lobby local government around investing in sustainable transport

Posters Leaflets Online resources Meetings Policy responses Campaign events

18

Before you move on...

Have any internal processes crept in?

Remember, the Planning Triangle describes the work you deliver, and the changes that happen as a result. It does not focus on your internal processes – the processes that you need to have in place to run your organisation effectively, such as promotional activities, recruitment, staff training, evaluation, or quality assurance. Review your list of outputs; if any internal processes have crept in, they will need to be removed.

Have you defined your priorities for measurement?

There may well be some areas where you are more interested in gathering information than others. For example, you may be keen to collect information about how many campaign events you delivered, and how well these were attended, but less interested in gathering detailed information about the number and type of leaflets that were distributed. Before moving on to the next stage of building your framework, reflect on which outputs are priorities for measurement.

Setting output indicators What are output indicators?

Output indicators are the specific pieces of information that you will need to collect in order to keep track and report back on the work you have delivered. Generally, your output indicators will give you information about: • •

the goods, services, or products you delivered who you delivered it to.

You can also set satisfaction or quality indicators for your outputs, which will tell you if the groups you worked with were satisfied with what you delivered and whether or not they thought your work was of a good quality. The table on the next page provides example output indicators.

19

Output Meet The Funder events

Output indicators Information about the events

• •

Information about who attended

• • • •

Information about satisfaction/ quality





umber of events N Location of events (postcode) Number of attendees Profile of attendees (age, gender, postcode) Type of organisation they work for % of attendees who had been to a previous Meet The Funder events Number of attendees who would recommend the event to a friend How attendees rated the quality of the presentations

How to set output indicators

Review your list of outputs and consider which output indicators you would need in order to keep track of the goods, services or products you delivered, the groups you served, and the quality of your work.

Before you move on... Do your output indicators tell you about the work you deliver?

Your output indicators should provide you with information about the work you deliver and not what changes as a result. Make sure that you have not jumped ahead to measuring change. (this is covered in the section on page 23, ‘Setting outcome indicators’). For example, if we wanted to set output indicators about the 1-2-1 job search and CV-building support sessions run in prisons, we might decide to gather the following information: • • •

number of sessions delivered number of people supported profile of people supported (age, gender, length of sentence, whether or not they have worked previously).

However, we would not look to collect information about: • •

number of people who feel more confident about using the internet to look for jobs number of people completing an up-to-date CV.

These indicators tell us about what changed for people as a result of attending the sessions, and not about the sessions themselves. They are outcome indicators; while outcome indicators 20

make up an important part of your evaluation framework, they do not belong in this section, which focuses only on the work delivered.

Will your output indicators help you to track how accessible your services are?

Many funders and local authority commissioners require a specific set of information about the people you provide your services to, including age, gender, sexual orientation, ethnicity and religious belief. More importantly, you will need this information to understand whether or not your service is accessible to everyone you intended to reach.

Have you defined priorities for measurement?

It is possible to develop a very detailed list of output indicators, very quickly! However, it is sensible to set realistic priorities for measurement at this point and to try and streamline your data collection as much as possible by limiting the number of output indicators you set.  

Describing your outcomes What are outcomes?

The specific aims set out in your completed Planning Triangle describe the areas where you want to make a difference through your work in the short to medium term. However, they will often contain a number of different ideas or changes within them, and this can present a problem when it comes to measurement. To get ready for measurement, you will have to turn this more general description of change into something more precise. This involves breaking down your specific aims into a list of measurable outcomes – the separate, individual changes that come about as a result of your work.

How to define your outcomes

In some cases, breaking down your specific aims into a list of measurable outcomes will be very straightforward. For example:

Specific aims (areas of change you want to achieve through your work) • • •

Outcomes (specific, measurable, single changes)

To increase access to sustainable funding • To increase partnership working between • organisations To increase skills sharing and resource • sharing among organisations •

Increased access to sustainable funding Increased partnership working between organisations Increased skills sharing among organisations Increased resource sharing among organisations

Sometimes, however, your specific aims will ‘hide’ a number of separate changes, and you will have to pull these out to create a full list of outcomes. For example:

21

Specific aims (areas of change you want to achieve through your work) • To improve the awareness of local people around sustainable living

Outcomes (specific, measurable, single changes) • Increased awareness of local recycling facilities • Increased awareness of energy efficiency measures in the home • Increased awareness of reducing domestic water use

It will be difficult to measure outcomes which are unclear or muddled, so defining your outcomes properly is a critical step.

Before you move on... Do your outcomes all describe single changes?

Composite outcomes – outcomes containing more than one change – are difficult to measure. This can include outcomes where two ideas are stuck together, for example: ‘improved self-esteem and self-confidence’. Separating these changes out into single outcomes – ‘improved self-esteem’ and ‘improved self-confidence’ is important for the next stage of the framework, when we come to define outcome indicators: the specific pieces of information we will need to collect in order to track our progress in terms of creating change. Outcomes which describe big, complex changes such as ‘improved wellbeing’ are also difficult to measure. ‘Wellbeing’ is a complex idea. Are we referring to physical wellbeing, emotional wellbeing, or mental wellbeing? Do we mean loneliness, happiness, physical fitness or mental health? Until your outcomes are properly defined, they will be impossible to measure.

Have you included any important negative outcomes?

Our hope is that all the outcomes from our work will be positive ones. However, in order to gather a full picture of the difference your work makes, it is good practice to include any important possible negative outcomes in your evaluation framework. This will help you to check that your work is not having any undesirable effects. For example: Researchers conducting an evaluation of a project challenging the behaviour of male perpetrators of domestic violence decided to gather evidence on the following positive outcomes for men: • • •

improved understanding of healthy relationships better understanding of the impact of abusive behaviour improved strategies for managing conflict.

However, they were also aware that challenging abusive behaviour could potentially increase the risk of abuse, and so they also contacted the men’s partners to collect information on the following negative outcomes: •

increased physical violence 22

• •

increased emotional abuse increased controlling behaviour.

Do your outcomes all describe changes?

Make sure that your outcomes describe the changes that you want to create, and not the work that you are delivering – this is already covered in the outputs section of your framework. Be cautious of outcomes around accessing the services you provide. If you are running skills workshops in prisons, for example, there is no real value in measuring the outcome ‘increased access to skills workshops’, since this doesn’t tell you anything about what changed for people as a result. If you find that you have outcomes around accessing your own services, consider what changes as a result of coming to the workshop, attending the events, or visiting your website.

Are your outcomes clear?

Make sure that your outcomes spell out the change as clearly as possible. Avoid woolly outcomes such as ‘service users feel more empowered’. What does this empowerment mean to service users? What changes for them as a result? Do they feel better able to cope with their situation? Do they change their behaviour? The easier your outcomes are to understand, the easier they will be to measure.

Have you defined priorities for measurement?

Having broken down all of your specific aims into measurable outcomes, you may find that you have a very long list! It may well be that you are more interested in evidencing some outcomes than others. These priority outcomes may be those that: • • • •

are pivotal to your work represent the ‘end result’ of a chain of smaller changes unlock other outcomes are helpful for attracting funding.

It is unlikely that you will be able to spend time measuring everything that changes, so before you go any further, consider which outcomes are priorities for measurement.

Do you need to review or re-word your specific aims?

The process of developing your outcomes asks you to think carefully about all of the individual changes that you want to bring about through your work. At this stage, you may decide to review or re-word your specific aims; this is not unusual.  

Setting outcome indicators What are outcome indicators?

Outcome indicators are the specific, measurable pieces of information that you will need to collect in order to keep track of the difference that your work is making. They tell you how much change has occurred, and whether or not you are achieving your outcomes. They are the backbone of your monitoring system.

How to set outcome indicators Set more than one indicator for each outcome In order to get a clear picture of change, it is good practice to collect information on at least two indicators for each outcome. Review your list of priority outcomes, and think carefully about what 23

information you would need to collect as evidence of change. For the case example where work is being done in prisons to increase the employability of exoffenders, we might choose to measure the following indicators:

Outcome – what is the change?

Outcome indicators – what would we measure?

Improved basic maths skills

• •

Improved literacy skills Increased work experience

• • • •

Accuracy of times tables (2-12) Ability to do basic calculations relating to budgeting, shopping and credit Number of students passing adult Level 2 literacy at C grade or above Student reading age Total hours of work experience Type of work experience tasks completed

Tailor your indicators to your context Good outcome indicators reflect your specific context. Think carefully about the outcomes you want to achieve: what are the signs that you have been successful? How can you tell when you have made progress? These signs will generally make strong outcome indicators.

There are banks of indicators available online, but these will only help you to collect good quality information if the indicators are suitable for the type of groups you are supporting and the type of changes that you want to create. Combine different perspectives It can be useful to set outcome indicators that capture different perspectives on the same change. For example, to assess whether or not the ex-offenders felt more confident in interviews, we could ask them directly about how anxious they felt about going for an interview. Since the project offers mock interviews to help people practice feeling confident, we could also ask the mock interviewer about what they thought of the ex-offender’s performance – how confident did they appear?

Outcome – what is the change?

Outcome indicator – what would we measure?

Increased confidence in job interviews

• •

Level of anxiety around attending interviews (ex-offender’s perspective) Observed level of confidence in mock interviews (interviewer’s perspective)

Combine subjective and objective indicators Combining subjective and objective outcome indicators can also give you a more robust picture of whether or not the desired change has happened. For example, if we wanted to know if people had increased their skills around applying for grants, we could ask people directly about how confident they would feel about making an application for a larger grant – this gives us a subjective view on whether or not the outcome has been 24

achieved. However, it would also be interesting to know what percentage of their applications were successful; this gives us a more objective measure of the degree to which their skills have improved, although other factors may also be an element in the success of an application.

Outcome – what is the change?

Outcome indicator – what would we measure?

Increased ability to raise funds

• • •

Level of confidence in writing applications for grants of over £30,000 % of successful applications Size and type of successful applications

Test knowledge and awareness carefully Simply asking people if they think they know more than before will not necessarily give you meaningful or accurate information; it is better to set more targeted indicators. For example, the Greener Devon campaign plans to increase awareness of local recycling facilities. Rather than asking people about their ‘level of awareness’, the framework sets out some more specific outcome indicators.

Outcome – what is the change?

Outcome indicator – what would we measure? Increased awareness of local recycling facilities • Whether or not people know where to recycle: - glass - paper and card - batteries - plastic - unwanted clothes Count and describe Some outcome indicators collect numbers (quantitative data) and some collect descriptions or narrative (qualitative data). Including a combination of both quantitative and qualitative outcome indicators in your framework can give you a fuller picture of what has changed. For example, the Women Together organisation wants to understand whether or not its work has led to increased partnership working. To do this, they need not only to count how often organisations are working together, but also to describe the type of projects and initiatives that organisations are cooperating on. Together, these indicators will help them to evidence the extent and depth of partnership work.

Outcome – what is the change? Increased partnership working between organisations

25

Outcome indicator – what would we measure? • How often groups work together • Type of work they cooperate on • Duration of partnership activities

Evidencing intangible outcomes Some of the most important outcomes from your work may be ‘soft’ outcomes – internal changes to the way people feel or think that are impossible to observe directly. For example: • • • •

improved self-esteem improved confidence increased happiness increased job satisfaction.

To set outcome indicators for these sorts of changes, think carefully about how you would be able to tell if the outcome had been successfully achieved. When people are making progress, do they behave differently? For example, when self-confidence goes up, it might be possible to observe changes in the way people talk, dress, or behave with others. Our self-confidence indicators might be: • •

readiness to make new social contacts how comfortable they are to speak in a group or to make a presentation.

By measuring these more easily observable behavioural changes, we can get closer to measuring the outcome itself. (This is sometimes known as setting ‘proxy indicators’.) For example, the New Starts project hopes to increase people’s motivation around looking for work. In this context, it is assumed that if you are more motivated in your job search, you spend more time job searching, and that you will feel more hopeful and positive about the future.

Outcome – what is the change? Increased motivation

Outcome indicator – what would we measure? • Amount of time each week spent on jobsearch activities • How positive ex-offenders feel about the future

Before you move on...

Have you used neutral language to describe your indicators?

Outcome indicators are always expressed as a neutral measure rather than as a change. For example, ‘increased partnership working’ is an outcome. ‘How often organisations work in partnership’ is an outcome indicator – the information you are going to track in order to understand whether or not the outcome has been achieved.

26

Other ways of expressing indicators include: • • • • •

Level of... Number of... Type of... How often... Percentage of...

(You can also refer to completed frameworks included in the Appendix to this workbook for more examples of wording outcome indicators.)

Have you established priorities for measurement?

You will need to measure at least two indicators for each outcome in order to collect good evidence of change. However, after drafting your outcome indicators, you may find that you are left with a long list of information to collect and track. Think carefully about your priorities for measurement. Which outcomes are most crucial to evidence? If you have set a large number of indicators for each outcome, which are the most interesting or revealing?

Have you incorporated what you already collect?

If you are already collecting information, think carefully about whether this could be used in your framework. For example, if you are using quizzes and tests to check that students are improving their understanding, the score on these could be a useful indicator for increased knowledge; or, if you are using validated scales and tools, such as the CORE counselling tool, you can use the score as an indicator of overall mental health.

Will your indicators give you accurate information?

Consider your list of outcome indicators critically. Will they give you accurate information about the outcomes you are interested in? If you collected evidence against these indicators, would it be easy for someone external to pick holes in your data, or to discount it? Your outcome indicators will be the foundation for your questionnaire questions, your focus group topics, and your interview schedules so, before you begin gathering data, it is important to make sure that they are measuring what you intended to measure.  

Defining your impact What is impact?

Your impact is the longer-term difference that you want to contribute to through your work. Your overall aim should translate directly into a statement of your desired impact, since it describes the ‘big change’ that your organisation wants to be a part of – the purpose or mission of your work.

Evidencing impact

Longer-term change can be difficult to evidence. Firstly, it takes time to measure change over time. Secondly, a number of different actors and factors will be involved in creating significant, longerterm changes. For example, the impact for the prison leavers project would be: •

Improved access to employment for people leaving prison in Northern Ireland.

It is easy to see how other factors, such as the work done with prisoners by similar organisations, 27

the job market in Northern Ireland, and changes to government policy could all factor in whether or not this longer-term change comes about. This makes untangling the specific role of the individual project more problematic. In order to track impact in a meaningful way, you will need to be able to demonstrate the role that your work had to play in creating longer-term change. This includes situating your work in its context, and understanding the role that other organisations and wider factors had in contributing to impact (attribution). It also includes understanding how much change would have happened anyway even if your organisation did not exist (sometimes referred to as the counterfactual, or deadweight). Getting to grips with these issues often requires a more sophisticated approach to evaluation and a more detailed research design. Generally, evidencing your organisation’s longerterm impact will require more time, skill and resources than simply evidencing your outcomes. However, you may decide to set indicators for keeping track of your impact, if you feel that you can measure longer-term change in a meaningful way. In this case, review your impact statement and think carefully about which indicators you would need to collect information against. For example, suitable impact indicators for tracking improved access to employment for people leaving prison might be: • • •

Percentage of ex-offenders entering the workforce after leaving prison How soon ex-offenders entered work after leaving prison The type of work ex-offenders take on after leaving prison

Completing your evaluation framework Finishing the framework

Having set your output and outcome indicators, you are now clear about which pieces of information you will need to gather. To complete your evaluation framework, you will need to decide: • • • •

how to collect the information when to collect the information who will be responsible for collecting the information, and how it will be used.

The first three questions are crucial to building a clear action plan for data collection. The fourth is also important to help you double-check that you are not setting out to collect information that may end up not being used. This workbook does not cover planning for data collection. Information and guidance about choosing and using different data collection tools can be found in Information Collection Methods: Choosing tools for assessing impact. Once your framework is complete, don’t forget to update it after each cycle of evaluation! This will help to keep it streamlined and up to date.

28

Glossary Aims describe the changes you plan to achieve in your user or target group. Counterfactual an estimate of what the outcomes of those receiving an intervention would have been without the intervention

Deadweight is a measure of the amount of outcome that would have happened even if the activity had not taken place.

Effective is having the results or effect you want; producing the intended results. Evaluation involves using monitoring and other information to make judgements on how your organisation, project or programme is doing.

Inputs are the resources you put into an organisation to carry out an activity. Inputs may be human, material, financial or expressed as time.

Indicators are well-defined information which can be observed or measured. They can be used to assess whether outputs, outcomes and impacts are being achieved.

Intermediate outcomes are outcomes achieved in the short term, but linking to longer-term outcomes.

Impact is the long-term, broad and sustained change resulting from your activities and services,

and may relate to a wider population than your target group. It is likely to relate to your overall aim, although there may be unexpected or negative impacts.

Mission statement is a short statement of the overall aim or purpose of the organisation, usually concentrating on the different it wants to make and defining the values that it will work by.

Monitoring is the routine, systematic collection of information for the purpose of checking your project’s progress against your project plans.

Objectives describe the planned areas of activities by which you are going to achieve your aims. Outcomes are all the changes, benefits, learning or other effects that happen as a result of your activities. Outcomes can be end or intermediate (also called interim outcomes). Intermediate outcomes are steps along the way to end outcomes. They are often smaller changes that happen before the final, desired outcome can be reached.

Outputs are the goods, services and products provided by an organisation. Proxy indicators are the things you measure or assess that will indicate changes that cannot be measured directly.

Qualitative data is data that is primarily descriptive. Quantitative data is data that is expressed in numbers. 29

Target group is the main group or groups you are working with and the people your organisation or project is aiming to benefit.

Trustees are the people responsible for controlling the management and administration of an organisation; another name for management committee members.

30

Resources CES has produced a wide range of guides on monitoring and evaluation. You might find the following particularly helpful:

Guide

Description

The CES Resource Guide: Evaluating outcomes and impact (2013)

A guide to over 130 resources on monitoring and evaluating outcomes and impact, including approaches, methods and creative tools, sub-sector specific guidance, and links to useful websites containing further resources. Free to download: www.ces-vol.org.uk/ informationcollectionmethods

Assessing Change: Developing and using outcomes monitoring tools (2010)

Practical handbook to help you identify and develop ways to collect information on the outcomes of your work. Free to download: www.ces-vol.org.uk/assessingchange

Information Collection Methods: Choosing tools for assessing impact (2013)

This practical guide will help you think through how to collect the right data to demonstrate the outcomes and impact of your work. Free to download www.ces-vol.org.uk/ informationcollectionmethods

First Steps in Monitoring and Evaluation (2002)

A basic guide for organisations who are looking at monitoring and evaluation for the first time. Free to download: www.ces-vol.org.uk/firststepsme

31

Practical Monitoring and Evaluation: A guide for voluntary organisations, 3rd edition (2009)

Comprehensive guide to monitoring and evaluation in the voluntary sector, now in its third edition. Available from: www.ces-vol.org.uk/practical

Your Project and its Outcomes (2007)

A guide focusing on how to identify outcomes and their indicators and how to assess their achievement. It uses case examples throughout to demonstrate different stages of the process and different types of organisation. Free to download: www.ces-vol.org.uk/freedownloads

Keeping on Track: A guide to setting and using indicators (2008)

A guide to help third sector organisations improve their skills in setting output, outcome and process indicators. Free to download: www.ces-vol.org.uk/ontrack

Using ICT to Improve Your Monitoring and Evaluation (2008)

This work book covers the steps and issues you need to consider in developing appropriate computer systems that will help you monitor and evaluate your work. Free to download: www.ces-vol.org.uk/freedownloads

Making Connections: Using a theory of change to develop planning and evaluation (2011)

Next Steps: Monitoring and Evaluation on a Shoestring (2011)

This guide provides an introduction to the theory of change approach to planning, monitoring and evaluation, explaining why and how to use it. Free to download: A practical guide to help organisations www.ces-vol.org.uk/theoryofchange develop their monitoring and evaluation approach and improve their effectiveness on a limited budget. Free to download: www.ces-vol.org.uk/shoestring

32

Discussion papers

CES’ series of papers on evaluation covers seven important topics, including outcomes monitoring and assessing impact. Available from: www.ces-vol.org.uk/discussionpapers

33

Appendix Example one

The New Starts employment project

To improve access to employment for people leaving prison in Northern Ireland

To improve the basic skills and work experience of ex-offenders

To provide To run basic 1-2-1 support skills courses around writing in prisons CVs and job searches

To increase ex-offenders’ job search and application skills

To improve employers’ perceptions of candidates with a previous conviction

To run To run To facilitate workshops awareness work around raising events experience disclosing for local schemes for criminal employers ex-offenders convictions with local employers

© CES Planning Triangle

To signpost ex-offenders on to other skills training services

I

Activity – the area of work we plan to provide

Output – the goods, service or product being delivered

Indicator – what will we measure?

To provide 1-2-1 1-2-1 support • support around writing sessions CVs and job searches • To run basic skills Maths skills courses courses in prisons Literacy skills courses • •

To run workshops around disclosing criminal convictions

II

Workshops



How will we collect When will we it? collect it?

Number of Workers’ log sessions delivered Location of Workers’ log sessions (which prison) Intake form Number of individual people supported Profile of people supported: - age - gender - ethnicity - sexuality - disability - previous work history

Every time a session is delivered

Level of satisfaction

At the end of the support offered

End of work questionnaire

When a new student starts

Who will collect it? Support workers

Example one continued. To run awarenessEvents raising events for local employers

To facilitate work experience schemes for ex-offenders with local employers

Work placements

• •

Number of events Events calendar Location of events

Ongoing



Number of employers attending

Event register

At each event



Profile of employers attending: - job role - type of organisation Number of work placements created Number of work placements filled % of placements successfully completed Type of placement: - job role - type of organisation

Registration form

At each event

Placement tracking spreadsheet

Ongoing

• • • •

Events officer

Employer engagement officer

III

Example one continued. To facilitate work experience schemes for ex-offenders with local employers

Work placements



Profile of people on placement: - age - gender - ethnicity - sexuality - disability - previous work history

Placement application At application stage form



% of employers satisfied with their individual placement % of employees satisfied with their individual placement

End of placement form



IV

At the end of the placement

Employer engagement officer

Example one continued.

Outcome – a Indicator – what will we measure? single, measurable change

How will we collect When will we it? collect it?

Who will collect it?

Improved basic maths • skills •

Maths quiz

At the start and end of the course

Support officer

Level 2 exam

At the end of the course

Support officer

Reading age matrix

At the start and end of the course

Intake/ end of placement form

At the start and end of the placement

End of session notes

End of the workshop

Observation/ judgement Intake form/ end of course questionnaire

End of the workshop

Improved literacy skills

• •

Increased work experience



Improved CV



Increased job search skills

Increased confidence in job interviews

Accuracy of times tables (2-12) Ability to do basic calculations relating to budgeting, shopping and credit Number of students passing adult Level 2 literacy at C grade or above Student reading age Total hours of relevant work experience - Type of work experience tasks undertaken Number of ex-offenders who have updated their CV

Workshop facilitator



Quality of CV presentation



Number of online job portals exoffenders use to search for jobs



Suitability of posts applied for

Observation/ judgement

At the end of the course



Level of anxiety around attending interview (self-reported)

Intake form/ end of course questionnaire

At the start and end of the course

Course facilitator



Level of confidence in mock interviews

Observation/ judgement

At the end of the course

Volunteer mock interviewer

At the start and end of the course

Course facilitator

V

Increased motivation

• •

Reduced stigma • related to ex-offenders •

VI

Amount of time per week spent searching for jobs How positive ex-offenders feel about the future Reported willingness to employ a suitably skilled ex-offender

Intake form/ end of course questionnaire

At the start and end of the course

Course facilitator

Registration form Feedback form

Employer engagement officer

Type of tasks employers would trust to ex-offenders (for example, cashing up, welcoming visitors, working with children)

Registration form Feedback form

At the start of the event, then feedback form sent one week later

Example two

Greener Devon campaign

To increase sustainable living among local communities in Devon

To increase local government investment in sustainable transport

To lobby local government around investing in sustainable transport

To increase the number of community sustainability initiatives

To provide a ‘greener together’ programme for local communities

To improve the awareness of local people around sustainable living

To run a poster, leaflet and online campaign around sustainable living

© CES Planning Triangle

To put on a sustainable living roadshow across Devon

VII

Activity – the area of work we plan to provide

Output – the good, service or product being delivered

To lobby local Meetings government around investing in sustainable transport Policy responses

When will we Who will collect it? collect it?

• • •

Ongoing

Policy and campaigning team

Communities team

Number of meetings Focus of meetings Profile of attendees (role, political party/ organisation) Number of responses written Topic of response Number of events Profile of attendees (role, political party/ organisation, age, gender, sexual orientation, ethnicity, disability, postcode) Number of meetings Number of individual community groups supported

Outlook diary

Set-up log

Ongoing



Profile of group (total number, postcode, average age, new group or existing group)

Start-up form

At registration



Level of satisfaction of group leader with set-up phase

Satisfaction quiz

At end of setup

Greener Together packs



Number of packs disseminated

Admin log

Ongoing



Satisfaction quiz

At end of setup

Celebration events

• •

Level of satisfaction of group leader with resources provided in pack Number of events Number of visitors to celebration event Postcode of visitors

Events log

Ongoing

Sign-up sheet

At registration

Campaign events

To provide a ‘greener together’ programme for local communities

Indicator – what will we measure? How will we collect it?

Set up meetings

• • • •

• •

• VIII

Publication log Events log

Administrator

Example two continued. To run a poster, leaflet and online campaign around sustainable living

Posters



Number of posters

Leaflets



Number of leaflets

Online resources

• •

Number of hits Number of downloads

Web analytics

• • •

Number of events Number of visitors Postcode of visitors

To put on a sustainable Roadshow events living roadshow across Devon

Resources log

Ongoing

Administrator

Events log

Ongoing

Sign-up sheet

At event

Communities team

IX

Example two continued. Specific aim – the area of change we want to create To increase local government investment in sustainable transport options To increase the number of community sustainability initiatives To improve the awareness of local people around sustainable living

X

Outcome – a single, measurable change

Indicator – what will we measure?

Increased investment • in sustainable transport options • Increased number of community sustainability initiatives Increased awareness of local recycling facilities

• • • •

Increased awareness of energy efficiency measures in the home



Increased awareness of reducing domestic water use



Total amount invested in sustainable transport options % of overall transport budget dedicated to sustainable transport Number of community initiatives Type of initiative (gardening, energy efficiency, car share, water use, recycling) Duration of initiative (in months) Whether or not people know where to recycle: - glass - paper and card - batteries - plastic - unwanted clothes Whether or not people know about: - home insulation scheme - double glazing discount card - local renewable energy initiatives Whether or not people know about: - water metering - grey water collection - devices to reduce toilet flush

How will we collect it?

When will we collect it?

Who will collect it?

Public data

On release of new budget

Policy and campaigns team

Group leader interviews

Six months after set-up phase

Communities team

Online quiz

Ongoing

Policy and campaigns team

Online quiz

Ongoing

Policy and campaigns team

Online quiz

Ongoing

Policy and campaigns team

Example three

The Women Together membership organisation

To improve the effectiveness of the women’s sector in London

To increase access to sustainable funding

To run funding clinics providing help with applications

To increase partnership working between organisations

To organise ‘meet the funder’ events locally

To increase skills sharing and resource sharing among organisations

To provide networking and skill sharing events

© CES Planning Triangle

To facilitate partnership working between local organisations

XI

Activity – the area of work we plan to provide

Output – the good, service or product being delivered

Indicator – what will we measure?

How will we collect it?

To run funding clinics providing help with applications

Funding clinics

• • •

Facilitator’s log Ongoing

• To organise ‘Meet the Funder’ events locally

Meet the Funder events • •

To provide networking and skills sharing events

Networking events

• •

Skills sharing events • •



XII

Number of clinics Number of attendees Profile of attendees (age, gender, postcode, ethnicity, organisation) Type and size of bids reviewed Number of events Location of events (postcode) Number of attendees Profile of attendees (age, gender, postcode, ethnicity, organisation) % of attendees who had been to a previous event Number of attendees who would recommend the event to a friend Satisfaction with quality of presentations

Event application form

When will we Who will collect it? collect it? Administrator

When people sign up

Facilitator’s log Ongoing

Administrator

Event application form

Event facilitator

When people sign up

Event feedback Immediately form after the event

Example three continued.

Activity – the area of work we plan to provide

Output – the good, service or product being delivered

To facilitate partnership Meet the Provider working between local annual forum organisations Borough meetings

Indicator – what will we measure?

How will we collect it?

When will we Who will collect it? collect it?

• •

Events log

Ongoing

Forum application form

When people sign up

• •

Number of forums Number of borough meetings Number of attendees Profile of attendees (age, gender, postcode, ethnicity, organisation)

Administrator

XIII

Example three continued. Specific aim – the area of change we want to create To increase access to sustainable funding

Outcome – a single, measurable change

Indicator – what will we measure?

How will we collect it?

When will we collect it?

Who will collect it?

Increased access to sustainable funding



Follow-up online survey

Four months after the event

Information officer

Increased ability to raise funds





• • To increase partnership Increased partnership working between working organisations To increase skills sharing and resource sharing among organisations

• •

• Increased skills sharing • • Increased resource sharing

• •

XIV

Total amount of funding % of grants which are 3 years or longer Level of confidence in writing applications for grants of over £30,000 % of successful applications Size and type of successful applications How often groups work together Type of work they cooperate on Length of partnership How often organisations share skills with each other Type of skills shared How often organisations share resources Type of resources being shared (buildings, volunteers, etc)

Charities Evaluation Services 4 Coldbath Square London EC1R 5HL 020 7713 5722 www.ces-vol.org.uk [email protected] Charity number in England and Wales 803602 Charity number in Scotland SCO40937 Company limited by guarantee Registered in England and Wales number 2510318

Loading...

Describe the difference your work makes Build your framework for

Describe the difference your work makes Build your framework for evaluation 1 Acknowledgements This guide has been developed by Eve Blair for Chari...

402KB Sizes 7 Downloads 17 Views

Recommend Documents

Eiosh | Build Your Future
Affiliation. British safety council Uk · OSHAcademy (USA) · NASP (USA) · IOSH (UK) · KPK Trade Testing Board · KPK Board

Essay describe your mother - Drawyourcomics
Reference philosophy essays the tell tale heart essay analysis advertisements intellectual virtues an essay in regulativ

SMART CHOICE FOR YOUR WORK
DOCUMENT SCANNER. Solusi lengkap dalam pemindaian / scanning yang ... Materi Presentasi, Surat perintah kerja hingga del

MAKE A DIFFERENCE: MUDRAS FOR YOUR HEALTH
Apr 22, 2011 - "பிரசாதம்”, “தோல்வி என்பது இடைவேளை”, பிரமிடுகள் தேசத்தில் ஞானத் தேடல், ஆழ்மனதின் அற்புத சக்திகள், சங்கீத

Diane Gilson - Build Your Numbers
Diane Gilson's passion is designing, customizing, and upgrading accounting systems for small to mid-sized construction a

Describe room your house essay - creohub
Describe room your house essay. 0. 0. 0. Our Services. Disadvantages of reality television essays extended essay cover s

Describe your computer experience? - Bayt.com Specialties
Oct 28, 2017 - I have used many different computer programs, both computer and web-based. I'm comfortable both with the

Amazon.com: Manage Your Day-to-Day: Build Your Routine, Find
Manage Your Day-to-Day: Build Your Routine, Find Your Focus, and Sharpen Your Creative Mind (The 99U Book Series) - Kind

Weird Sounds Your Body Makes - Men's Health
20 Apr 2015 - Involuntary sounds could be your body's way of warning you something's wrong. Learn how to interpret (and

smart choice for your work - printers - doczz
03 UKM (SMB) Proposal penawaran, materi presentasi, laporan keuangan, dokumen usaha, surat perjanjian kerjasama, sales o