Select one of the scenarios describing a program in Chapter 8 of Dudley (2014; pp. 167-210) and review the challenge that was addressed with a process evaluation. Challenges are identified in italics at the top of each description. Skim through the list of already posted discussions, and ensure that you are the first or second student to use the scenario for your post. To ensure a wide range of examples, no more than two students may use the same scenario.
Enter your choice of scenario in the discussion forum as soon as you select it. You may return to complete the discussion later.
2. Determine the purpose for the evaluation. If one is not provided, infer it from the description.
3. Review the Process Evaluation Planner in Getting to Outcomes (GTO) (Rand Corporation, 2019) to see how an actual process evaluation is conducted (https://www.rand.org/pubs/tools/TL259/step-07.html). Scroll half-way down the page; it is located under Tools Used in This Step. The goal for this discussion is to apply that approach.
4. Find four questions in Table 2 that you can answer about your selected program, either from information provided or from information you can reasonably infer about it. Delete the rest of the questions so your table does not get too large
5. Post your responses to the questions in Table 2 (below) if provided in the description in Dudley (2014) or if you can reasonably infer them from the context of the scenario.
6. Write a brief summary of the findings from your evaluation of the program.
Format for your discussion post. (Due by the end of day on Wednesday). Please label your response as “Original Post”.
- Complete Table 2 below, which is a process evaluation table. Include responses to your four selected questions as follows.
- Name the scenario you intend to use for the discussion from Chapter 8 of Dudley (2014) at the top of the table. Include the page number where the scenario is found. (1 sentence)
- Identify the purpose of your selected process evaluation. Most purposes are described in the scenarios in the textbook, but if you have to infer it from the description, choose one of the purposes identified in the lists at the top of this discussion post. Place the purpose in the second line of the table. (1 sentence)
- Select four questions (not all of them!) listed in the first column of Table 2 that you can reasonably answer about your program. If the scenario describes the question but not the responses the evaluators obtained, provide a plausible response from your interpretation of the scenario and indicate that it is your suggested answer.
- Identify the types of individuals who provided the information or data for the evaluation in the scenario. If no respondents are described, suggest several who could reasonably provide the best answer to your selected question (one informant per question). List the informants for the process evaluation by their roles (staff, leaders, participants, etc.) and insert them in the table below. If the evaluation has not been conducted, list at least two types of informants who might provide useful information for the evaluation and their roles, and indicate that these roles are suggestions for the evaluation.
- In the third column, provide an answer to the questions you selected from the description of the scenario in Dudley (2014). If no answers to the questions are given in the scenario you selected, provide a plausible answer that can be inferred from the scenario.
Table 2: Process Evaluation
| Selected scenario from Dudley (2014) and page number: |
| Purpose of the evaluation (if none is provided, select one from the lists at the top of the discussion) |
| Process Evaluation Questions (Select 4; delete the rest) |
Name the role of the informants who provided an answer to the selected question OR suggest an informant who could provide valid answers for the evaluation. (brief word or phrase) |
In column 3, enter the data collection method used to obtain responses to your selected questions and the responses that were obtained. If not provided in Dudley (2014), list the data collection methods you could use and the types of responses you might expect, based on the scenario description, and indicate that you are estimating the response. (1-3 sentences.) |
| What interventions were planned and what were the desired outcomes of the interventions? |
|
|
| What were the characteristics of program participants? Were these people in the group that the program was designed to serve? |
|
|
| How much did the program participants use the program compared to the amount that the planners intended? Were the locations and times adequate? |
|
|
| Did the program describe efforts made toward cultural responsiveness and inclusiveness of staff, program activities, and/or recruitment of participants? |
|
|
| How closely did the program follow the Logic Model? Were all program components delivered as planned? If not, why not? |
|
|
| What problems or barriers arose to prevent delivery of program components as planned, if any? Was it possible to work around the obstacles? |
|
|
| How effective were the program activities in addressing the need for which the program was designed? Which were most/least effective? Were there any unintended outcomes (positive or negative? |
|
|
| What was the quality of the program components that were delivered? How well did staff meet expectations for performance? |
|
|
| What was the staff’s (including volunteers) perception of the quality of the program services, adequacy of resources, and their treatment by program administrators, participants, and stakeholders? |
|
|
| How satisfied were the participants with the program as delivered? |
|
|
| Should the program be continued or repeated? If so, what changes are recommended for the tasks and methods used in the program, if any? |
|
|
Adapted from Rand Corporation (2019). Getting to outcomes; Step 07. Process evaluation. Retrieved from https://www.rand.org/pubs/tools/TL259/step-07.html
2. Summarize the information obtained in the evaluation. Include the following information (4-10 sentences MAXIMUM):
2a. What strengths and weaknesses of the program were identified? (1-3 sentences).
2b. From the description given in Dudley (2014) does it appear that your selected program was implemented as planned? If not, provide the explanation for diverting from the program plan, if any. If the scenario lacks a description about the program being implemented as planned, which would be a red flag for most auditors, note this lack of information and explain why a description of program fidelity is important in a process evaluation. (2-4 sentences)
2c.Include any information regarding client perceptions of the program. (1-3 sentences).
3. Explain how the method of obtaining the answers to the questions (review of records, interview and questionnaires with open and/or closed questions, focus groups, phone calls, etc.) might affect the results obtained. (See the data collection methods handout used in previous weeks for ideas.) What methods could you use to ensure that the findings and responses of informants were valid? (Maximum: 2 sentences)
Recent Comments