Discussion 1: Writing an evaluation report
This week, you will review a program evaluation and present a report on the findings.
Different stakeholders will have different interests in the evaluation and may seek answers to different questions.Below are some examples that might apply to the interests of different stakeholders, but you may think of others, too:
Interested community members will want to know about the benefits, costs, and anticipated long-term impacts of the program. If the benefits were significant, they may want to know how about plans for continuing or expanding the program.Funders will want to know what activities/services/interventions were provided, what cost-cutting measures were used (if known), whether any obstacles arose to prevent the delivery of promised services, and what modifications to the original plan were needed to achieve the goals of the program.
Your task this week is to select an research study or program report from the learning resources or from another peer-reviewedsource if you have a program for a special population in which you are interested and prepare an report that summarizes your evaluation of the study or program.
Program descriptionWhy was the program or study developed? What problem or condition did it intend to illuminate or mitigate?What were the goals of the program or study?Who were the intended recipients of the program?How many intended recipients participated?If the intended recipients did not participate, who did?What prevented the intended participants from getting involved?What activities were provided in the program and by whom?Briefly describe the activitiesBriefly list qualifications or training of the providersWere these the intended activities? If not, describe what forced the leaders to modify or revise the program activities (resources not available, opposition from parents, etc.)Choose either a process or a product evaluation to report. Name the type of evaluation you are describing. Process evaluations, for continuing programsreview how activities are being conductedare the activities conducted well?are improvements or revisions needed?Identify the data collection methods used (interviews, surveys, focus groups, questionnaires, etc.)Who collected the responses from recipients or other stakeholders?What was the response rate from the recipients or other stakeholders who were asked about the program?Findings. A report of the results in an evaluation report includes an interpretation of how the findings pertain to the goals of the project. For process evaluations: how much were recipients proceeding toward the goals of the program? What was most useful in helping them to reach these goals? If the goals were not being met, what recommendations did the study authors make, if any?For product evaluations: how effective was the program in aiding recipients to achieve the goals of the program?If goals were not met to the degree intended, what recommendations or limitations did the study authors report that made it difficult for recipients to succeed?DisseminationWhat method would you use to disseminate the results of your evaluation?Consider your category of stakeholder. Agency staff may prefer a conference presentation, funders prefer white papers (detailed reports as you have been writing for this class), while interested community members may prefer a range of venues from meetings to mailings as well as conference presentations, PowerPoint presentations at a workshop, and program summaries.Of course, this all depends on the audience.You choose!Explain what type of dissemination method you would choose and provide a rationale.
Only two students may select the same study. As soon as you have made your choice, enter it into the discussion forum. You do not have to enter a post – just identify your selection.
Tailor the following components of your evaluation to the perspective of the stakeholder you listed in #3
OR ii. Product or outcome evaluation
Writers for the State Government of Victoria Department of Sustainability and Environment (2010) show the contrast between reporting basic information and reporting evaluation results and advise that evaluators should be careful “[not to] confuse presenting data with presenting … evaluation findings” as shown below.
This is NOT evaluation
This is evaluation
36 people attended the workshop
The workshops did not attract as many participants as planned. Only 36 people attended compared to the original target of 60. The participant demographics was also not representative of the intended audience. A likely reason for this is that the workshop clashed with a number of other community events that drew potential participants away.
86% of the participants have stated they are taking shorter showers
The project delivery model was successful in driving more sustainable behaviours. This is supported by 86% of respondents stating that they had reduced their showering time from their participation in the project. A key factor in changing behaviour was the shower timer that was provided to participants to prompt them to get out.
For further reading:
If you would like to see how some organizations prepare their evaluation reports, there are some good examples on the internet, including the ones that follow.
State Government of Victoria Department of Sustainability and Environment (2010). Presenting your evaluating findings. Retrieved from
http://evaluationtoolbox.net.au/index.php?option=com_content&view=article&id=46&Itemid=6
**** Highly recommended!
Wisconsin Coalition Against Sexual Assault (n.d.).Stage 3: Writing an evaluation report. Retrieved from
https://s3.us-east-2.amazonaws.com/wcasa/old-website-resources/05_EvalGuide_STAGE3.pdf
Wisconsin Coalition Against Sexual Assault (2013, March 10).Prevention Education Webinar Series 2013 (1 of 5): Introduction to Evaluation. [Video file]. Retrieved from
Writing Evaluation Report of a Project (n.d.). Retrieved from
http://www.mnestudies.com/evaluation/writing-evaluation-report-project
Developing an effective evaluation report: Setting the course for effective program evaluation. (2013). Atlanta, Georgia: Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, Division of Nutrition, Physical Activity and Obesity. Retrieved from
https://www.cdc.gov/eval/materials/Developing-An-Effective-Evaluation-Report_TAG508.pdf
(Pay special attention to the first 8 pages).