How will you determine if your program is working? This section will provide you with the tools to monitor and evaluate your activities.
The processes of monitoring and evaluation use carefully planned and well-thought-out methods to measure the success of an activity in meeting its goals. They are an important part of the project management process, because they provide:
Key terms used when talking about monitoring and evaluation are shown in Box 1.
Box 1: Monitoring and evaluation key terms
Monitoring and evaluation are related processes, but each has a different focus:
Table 1 provides examples of different outputs and outcomes for the TIS program. Monitoring outputs relies on describing and counting project activities and the number of people who come to events. In the past TIS reporting focused only on what is included in the outputs column – ‘what we do and who we reach.’ Now, however, you are being asked to think more about ‘what difference do our activities make?’ This is a question about your activity outcomes.
Table 1: Examples of outputs and outcomes for TIS
Smoke-free workshops delivered to 100 workers in 10 community organisations
85 workers have increased knowledge of benefits of smoke-free workplace and increased commitment to being smoke-free at work
8 Organisations are smoke-free
Social media campaign (call to action – take the ‘we can be smokefree’ pledge).
45 families took the pledge
Follow up survey showed that this has potential to reach over 500 individuals living in these homes
You can find out more about the ‘how, who and what’ of TIS activities evaluation in Box 2.
Box 2: The how, who and what of evaluation
How to plan an evaluation?
It’s a good idea to work out an evaluation plan for your activities, so you can keep track of the evaluation. In your plan consider the following points:
Who will do the evaluation?
Work out who will be responsible for organising and writing up the evaluation.
Who should participate in an evaluation?
Good evaluations involve those who are interested and affected by your activities. Involving people from the earliest stages of the activity’s development to the final evaluation can encourage local communities to set up, control and own an activity. Putting together a working group (a group appointed to study or report on a particular question) of community members also helps the evaluation process. A working group brings the values and shared interests of the community into the process. Often they are also the ones who are best placed to talk about the needs of their community.
What should you evaluate?
What aspect of the activities do you want to evaluate? For example, do you want to know if the activities’ objectives were met? Or do you want to know what people liked about the activities? There are lots of evaluation questions you could use. Examples of the sorts of questions you might use to answer different questions are provided in the
.
What sort of resources will you need for your evaluation?
Be realistic about what resources are available to undertake the evaluation. This includes funding, time, staff, salaries, material, equipment and operating costs.
Source: material adapted from Kruger K, McMillan N, Russ P and Smallwood H (2007) Talkin’ up good air: Australian Indigenous tobacco control resource kit. Melbourne: Centre for Excellence in Indigenous Tobacco Control
[14810]
Or for a more in-depth general discussion of behaviour change evaluation and definitions of some other terms used in monitoring and evaluation, you might find the Evaluation toolbox useful.
Monitoring and evaluation should be built into your activities from the start. It is also a requirement of TIS program funding to measure your outcomes every six months for Indigenous Australians’ Health Programme (IAHP) grant agreement progress reporting. The
developed by HealthInfoNet on behalf of the Australian Government (2012), describes a simple six step process to building evaluation into an activities plan:
Examples of ways to present your data using charts and dashboards for good visual impacts can be found in this monitoring and evaluation resource or on the Resources to monitor and evaluate your program page here.
Figure 1: Keeping your activities on track
Figure 1 shows how monitoring and evaluation forms a central part of your activities. You can use the information that you collect to make sure activities are on track and that you are achieving the outcomes you intended, adapting and improving your activities as you go. This process of using monitoring and evaluation data to improve your activities as you go is known as continual quality improvement (CQI). For support in undertaking CQI, you can download Reach, implementation, satisfaction and effectiveness (RISE): a tool for supporting CQI for TIS. More tips for how to use monitoring and evaluation data are provided in Box 3.
Box 3: Using monitoring and evaluation data
Activities do not always run as planned or as expected, and are a learning experience for all those involved. Monitoring your activities and evaluating outcomes will help you keep track of progress, identify important changes and make it easier to see what you have achieved. Answering the following questions may help keep your activities on track:
Source: adapted from Australian Indigenous HealthInfoNet (2012) [2]
[1] This material is used with permission from the Commonwealth of Australia. Talkin’ up good air contains materials that were contributed by Quit Victoria (Anti-Cancer Council of Victoria), Apunipima Cape York Health Council, National Heart Foundation of Australia (NSW Division), Council of Social Services of New South Wales and Queensland Health and which remain their property.
[2] Australian Indigenous HealthInfoNet (2012) Healthy, Deadly and Strong: Healthy Lifestyle Worker Toolkit. Perth, WA: Australian Indigenous HealthInfoNet
