The name of this program is the COPE (creating opportunities for parent empowerment) NICU (neonatal intensive care unit) Program

Evaluation Practice Workbook

Introduction

This set of worksheets provides guidance for conducting an evaluation using the Participatory Model for Evaluation in the book Evaluating Public and Community Health Programs (Harris, 2010). The individual worksheets may be used as a set to conduct an evaluation as part of a course that provides hands on real life experience for teaching and learning evaluation, or in conducting an evaluation in a community based organization or agency. The worksheets draw from many of the chapters from the book and provide a systematic approach to evaluation. This workbook is designed to help readers think through and apply components that are critical to conducting an evaluation.

Step 1: Designing the Evaluation

1. Describing the Program

2. Determining the Evaluation Questions and the Evaluation Design

Describing the Program

Background

• What are the social, political, economic and organizational factors that influenced the creation and development of the initiative?

• What community assessment data provided a rationale for this initiative from the perspective of the stakeholders or the initiative’s available documentation?

• What epidemiological data supports the development of such an initiative from the perspective of the literature? Conduct a literature review that focuses on the health problem and preferably for the priority population of the initiative. For example: If the outcome is to lower the rates of heart disease, conduct a literature review to understand the incidence, prevalence and distribution of disease nationally and locally.

• What factors support the development of the initiative? What is the organization’s mission, organizational structure, available resources?

• Who are the stakeholders? How will they participate in the initiative? What roles will they play?

Note: Using a table similar to the one below, list all the stakeholders and the roles they will play during the course of the evaluation.

Program Characteristics

• What is the population(s) being served or benefiting from the initiative by age, gender, location of residence, socio-economic/cultural group, language and other relevant characteristics?

• What are the characteristics of the program or policy that is being evaluated?

• What are the goals? What improved health condition does the initiative hope to accomplish?

• What are the objectives? What risk and/or protective factors does the initiative address?

Note: Using a table similar to the one below, make a list of all the objectives of the program listing the behavioral objectives on the left and the outcome objectives that support the behavioral objective being met on the right. An example has been included.

• What is the theoretical basis for the initiative? Which social/behavioral/organizational theory was used by the organization to design this project? What is the Theory of Change of the intervention?

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

• What intervention activities are used by the organization to achieve the outcome and behavioral objectives (e.g. materials, workshops, seminars, outreach, health fairs, counseling, case management, convening, e-mail alerts etc). What are the corresponding activity objectives?

Note: Draw a table similar to the one below with the Behavioral or Outcome Objective on the left and the Activity Objectives that are directly related to achieving them on the right.

• What are the intervention activities that will be carried out?

• What resources are required to conduct the activities that are required to achieve the objectives?

Note: In the next table list the activities that are contained in the Activity Objective on the left and identify the resources that are required to carry out each activity on the right. There may be some overlap.

• What are the activities of the intervention? What are the results of the activities that are conducted? For example, if a workshop is conducted and 40 people are trained, then the output is 40 people trained. If the activity was to send e-mail alerts to all the members of the coalition, then the output will be the number of e-mail alerts that were actually sent.

Note: In the table below list the intervention activities in the left column and the corresponding outputs that the organization expects from the initiative

• What is the initiative’s Theory of Change?

• Construct a Logic Model for the initiative that reflects the goal the organization is trying to reach, the outcomes that are associated with the goal, the activities, outputs and resources. This should be a summary and a pictorial representation of the information that is contained earlier in this workbook.

Note: The logic model contains the following components from left to right: Resources, Activities, Outputs, Outcomes and the Goal

Determine the Evaluation Questions and the Evaluation Design

• What do the stakeholders and the evaluation team want to know about the initiative?

  • What are the evaluation questions?

Note: Use a table similar to the one below to list the evaluation questions in two groups, process evaluation and outcome evaluation questions. Remember that process evaluation questions ask about the implementation of the initiative and the outcome evaluation questions ask about the effects of the initiative.

• • What are the activity (process) and outcome objectives that are associated with this intervention that must be included in this evaluation? List them in the table below. Do any of the objectives support the evaluation questions listed above? If yes, list them. Convert the objectives into evaluation questions.