Results and Impact

Community and voluntary organisations are usually required to monitor their work, evaluate the results of the work and assess what the impact of the work on people’s lives.

Monitoring is simply the process of checking what has been done or spent in line with plans and budgets through the systematic collection of information for analysis.  Organisations monitor their activities and income and expenditure to ensure that they are operating in line with their plans and budget to ensure they are delivering their organisational purposes and funders’ requirements.

“Monitoring is the routine collection and recording of information about a piece of work or an organisation, to keep track of day to day activities and operations.  Its purpose is to provide regular feedback on how things are going and to help the organisation make decisions.”   The Evaluation Trust 2008

Evaluation is an assessment or judgement made by comparing what actually happened with what was intended.  Assessments of judgements made are based on the analysis of information.

“The assessing and judging of the value of a piece of work an organisation or a service.  Its main purpose is to help an organisation reflect on what it is trying to achieve, assessing how far it is successful and identify required changes.”   The Evaluation Trust 2008

Benefits of evaluating your activities:

• Identifies our priorities (may have different focus from e.g. funders)
• Can assist progress towards the achievement of organisational goals
• To assess suitability and quality of methods and approaches used
• Identifies what work was actually done (outputs)
• Identifies the impact of the work on the lives of beneficiaries (outcomes)
• Can help highlight suitability/clarity of goals
• Can identify difficulties
• Leads to ways to improve and develop work
• Gives a voice to all stakeholders in our work

The four-step process of evaluation

STEP 1:  Planning

• decide what area(s)/activities to evaluate
• decide how to measure it – define successful outcomes
• plan what sort of information you need and
• decide on methods of collecting the information (where it is, when is it available, who can access it)

STEP 2:  Monitoring

• collecting and recording the information

STEP 3:  Evaluating

• analysing data and
• interpreting results

STEP 4:  Making improvements

• identifying and prioritising improvements
• planning and implementing action and
• monitoring and evaluating work

Some organisations develop a ‘Theory of Change’ or Logic Model in which they articulate:

• The final desirable outcomes of the organisation’s work i.e. how they hope the lives of the beneficiaries of the organisation have changed for the better
• Intermediate outcomes that need to be achieved if the final outcomes are to be obtained (e.g. gaining a qualification or writing a CV may be intermediate outcomes on the way to gaining employment)
• The outputs (the number of clients, sessions, etc.)
• The processes that the organisation will carry out to achieve the outcomes
• The inputs (money, people, resources, etc.) needed to do the work

With the encouragement of the Northern Ireland Executive, many community and voluntary organisations have adopted an approach known as Outcomes-Based Accountability developed by Mark Friedman, which suggests that organisations should ask itself three fundamental questions:

• How much did it do (outputs)
• How well did it do it (quality indicators; service-user satisfaction; etc.)
• How people’s lives were changed for the better (impact/outcomes)

The most important, and most challenging to measure, is the last one.  In order to measure it, organisations need to agree what they think the desired outcomes are i.e. how they want people’s lives to improve.  These might be outcomes in terms of education or training; health; employment; sense of control of their own lives; etc.  The organisation then needs to find a robust way of measuring the change.  This might be a questionnaire before they receive a service from the organisation and then the same questionnaire afterwards.  The difference the organisation has made can then be measured.  If the organisation does not currently have a system to answer these three questions robustly, it may need to draw up a ‘data development strategy’ so it has appropriate mechanisms for measuring outputs, satisfaction, and, most importantly, the impact on people’s lives.

The Northern Ireland Executive has a Programme for Government which contains outcomes which the government considers important and a set of indicators by which they can measure the changes at population level.  Individual organisations are unlikely to be able to influence outcomes at the population level, but they can in terms of their own service-users.  It creates a powerful case to government if an organisation can show that its work contributes towards the outcomes and indicators in the Programme for Government.

More on Monitoring and Evaluation

More on Monitoring Your Strategic Plan