Program evaluation

Instruction: I need to respond to the student’s response

Provide an alternative suggestion of how program evaluation can be used.
Provide support or an alternative perspective on the elements of program evaluation.

Student reaponse: A description of your experience and interests in program evaluation.

As a prior clinical supervisor of a drug and alcohol agency, program evaluations were incredibly important for multiple different reasons. One of these reasons included feedback from clients who were involved in the agency. It was pertinent to understand what they felt that they needed more or to assist with their recovery process as well as understand what the agency was doing right or wrong, in their opinion. Another component of program evaluation that was considered was for the non-profits ability to continue to provide counseling services as identified by the state of New York. Because we were considered to be part of the Office of Alcoholism and Substance Abuse Services (OASAS), an evaluation of our agency was conducted annually. This evaluation did not consist of opinion, rather, of factual information that was evaluated by OASAS. This component of the program evaluation looked at one of the key parts of the evaluation, were we doing what we said we were doing? This answer was found through chart audits, note reading as well as a policy and procedure manual.

Next, explain what type of program evaluation is most consistent with your interests and experience (a necessity for the program, program improvement, program continuation/dissemination, or accountability) and why.

In the context of my previous employment, there are two identifiable types of program evaluation. The first type of program evaluation type would be program improvement. Program improvement can be noted through the use of survey data that was collected from the currently active clients within our program (this was approximately 500 individuals at any one time). The data was collected through a survey and returned to a quality control team to assess if the client’s social determinants of health were adequately being addressed. This survey was completed through a Likert scale as well as with open-ended questioning. Program improvement was most notable through the addition of certain services as well as minimal items that were requested. For example, one item that was a consistent request within the survey (which was completed quarterly) was for coffee. After coffee was provided to the clients, engagement increased, and we were able to see a positive change in client’s both attending and staying in treatment longer.

The second type of program evaluation that can be noted is the one completed by OASAS which looks for program continuation. Program continuation can look at both the need for the program as well as if the program is meeting all of the identified objectives. In this instance, OASAS looked at both of these items. Was there a continued need for the agency based on the geographic location and current substance abuse problem as well as was the agency meeting the required objectives as identified by the state. This program evaluation provided for continuation licensure of services that could span from 1 year to 3 years. If the agency was awarded a 3-year license, the state would not come back to evaluate for continuation. However, if the agency only received a 1- or 2-year license, the evaluators would come back to assess for accountability of the agency as well as to assure that the agency had made the required changes to meet the objectives identified.

Also, explain how you might use this type of program evaluation in your future professional role.

Assessment of program improvement and the way that we utilized this survey was only slightly successful. Although there were some changes made based on the desires and identified needs of the clients, other items were not able to be addressed. Some of these items include the length of treatment and length of a counseling session. These items were beyond the control of the agency and fell on the shoulders of insurance. I would utilize survey data as a goal to improve an agency in the future, however, I would tailor it to the items that were in control of the agency. By allowing for other items to be listed, the program will not be able to improve to the standards of the participants which can cause anger or dissatisfaction.

As a current adjunct professor, class evaluations take place at the end of every semester. While I do not have access to these evaluations until after the semester is over and grades have been submitted, I utilize the information provided in these surveys to make attempts to improve on my teaching methods as well as assignments that are needed. I will especially rely on this evaluation this semester due to the changes that had to be made within the university to account for COVID regulations.