Assignment Three

Program Evaluation Assessment
Of a
Non-Profit Weight-Loss Group
In
Rural Saskatchewan
Janelle R. Christensen
University of Saskatchewan

As a member of a non-profit weight-loss group in rural Saskatchewan, it has been brought to this program evaluator’s attention that the current leader of this program has some concerns regarding the efficacy of her work with respect to assisting all members in their weight-loss journey. In order to determine the overall scope and design of an evaluation plan, one must first decide whether an evaluation needs to be conducted. In order to make this decision, an evaluation assessment, or a needs assessment must be carried out. The purpose of this assignment is to carefully examine the components of a specific weight-loss group in rural Saskatchewan in order to best decide whether or not an evaluation should occur.

Nature of the Program

This program “is a non-profit, non-commercial weight-loss support organization” (About #### Club Inc.) based in the United States. Its objective is to “encourage healthy lifestyles through weight management support groups and to sponsor obesity research” (About #### Club Inc.). This non-profit weight-loss group meets weekly during the evening. Like all other chapters of this group, “a private weigh-in is followed by a program on a wide variety of topics pertaining to the weight-loss journey and healthy lifestyle (About #### Club Inc.). The goal of the meetings is to provide “members with positive reinforcement and motivation in adhering to their food and exercise plans” (About #### Club Inc.). There are many resources provided to members, such as *** ***** ** **** that contains information about the exchange system for meal planning and motivational tips and practical advice and the companion workbook. There are some strategies used in order to attempt to motivate members, such as contests in which members compete with each other and the winner(s) are awarded non-monetary prizes. Following numerous consultations with the major stakeholders of this group, specifically the group members, including the Leader and the Co-Leader who are also members, it was determined that there were many perceived objectives of this group that were not being met. In addition, it was also questioned as to whether or not the mandate of the overall weight-loss group was being met. Specifically, it was questioned whether or not the program was doing what it has set out to do. Therefore, a needs assessment will be carried out in order to determine if a formal evaluation of this program needs to occur.

Needs Assessment

Step One

In order to conduct a needs assessment of this group, this internal evaluator has chosen to follow some steps outlined by Rouda and Kusy (1995). In particular, it is suggested to do an initial “gap analysis” (Rouda and Kusy, 1995). Specifically, the current skills, abilities, and knowledge of the major stakeholders can be assessed, and compared to the desired skill levels and knowledge base of all stakeholders. It will be these differences, or the “gap” between the current and the desired that will determine the nature and direction of this evaluator’s future evaluation. Therefore, in respect to the specific weight-loss group, the current skills and abilities of the members can be assessed by inquiring about their daily practices and efforts to eat healthier and become more physically active. These abilities have been assessed by asking participants to indicate their knowledge of various issues related to weight loss and changes in eating habits, exercise levels, and stress management. In order to obtain some evidence of these abilities, a Needs Assessment Questionnaire, as found in Appendix A, was designed by this internal evaluator and was administered to the members during a regular meeting. Members worked in small groups and discussed the questions and provided responses to them. Prior to administering this survey, the internal evaluator provided instructions to the group. In addition, members were assured of anonymity and confidentiality as they were asked not to identify themselves in any way on the questionnaire, including omitting their names from the page, and that the information they provided on the questionnaires would only be seen by the internal evaluator. It was initially desired to ask members to verbally respond to several questions in the context of our regular meeting. The questions were: (a) Do you leave each weekly meeting with the tools that you think you need to be successful, (b) In what ways are these tools provided , (c) In what way are members encouraged to succeed, (d) What occurs during each meeting that helps each person to become motivated to succeed, (e) Are members encouraged to use the various resources that are available to them on the website, (f) Are members made aware of these specific resources during the meetings, and (g) Are any of these resources integrated into the ‘program’ component of the meetings? However, due to time constraints during the meeting time, these questions were not asked.

The desired skill levels of all major stakeholders were also addressed by questions raised in the group sessions. For example, the group’s Leader desires to know what she can do to be of assistance to all members. In addition, members were also asked what information they would like to have brought forward that they believe would help to enhance their knowledge base that would assist them upon their weight-loss journey. A variety of responses were provided, thus warranting a need for further investigation. Admittedly, one cannot ignore the fact that weight loss and improvements to one’s own overall health and well-being are individual efforts. However, the perceived role of the group is to provide support and encouragement through the various activities so that each individual is inspired to set out on their own weight-loss journey each week. Therefore, it must be determined if this role is being fulfilled, and if so, if it is being done so in the most effective way possible.

Step Two

The next step in a needs analysis, as identified by Rouda and Kusy (1995), is to identify priorities based upon the issues that were raised in the first step. The authors identified numerous ways of doing so, such as conducting a cost-benefit analysis. Specifically, this evaluator can assess the ways in which the cost of the problem compares to the cost of implementing a solution. In this case, the cost of implementing a solution is much less than the cost of the problem. It must be noted that the word “cost” means much more than dollars and cents. Specifically, the implementation of a solution may mean that the group’s leader will engage in activities that will enhance her knowledge base of ways in which she can best foster motivation amongst the members, as well as provide tools that will educate members with new strategies for weight-loss and different options for becoming more physically active. The cost of the problem extends far beyond this particular small group. Furthermore, the cost to society in general of being overweight or obese is multi-dimensional, and is immeasurable. In particular, if the Leader does not have the knowledge base and the ability to provide members with information pertaining to healthy life-style choices, and to assist members when they feel helpless and/or have exhausted all options pertaining to healthy food choices and means of becoming physically active, then some individuals may become discouraged, leave the group because they do not receive the support they need, and the group process may eventually disintegrate.

However, one another approach of identifying priorities that the authors did not address, is to simply visually examine the responses provided by the group members on the Needs Assessment Questionnaire, thematically categorize them, and prioritize those general themes. The content of the program evaluation, specifically the survey or focus group questions, may be based upon these general themes.

Step Three

In order to determine some actual causes of performance problems and some possible opportunities for change (Rouda & Kusy, 1995), the internal evaluator looked at the responses of the final two questions on the Needs Assessment Questionnaire. Prior to administering the survey, it was anticipated that the questions posed to the group members would determine whether they believed that the leader and co-leader are effectively fulfilling the duties of their roles and meeting the goals and objectives of the organization. Interestingly, it was the leader who first expressed concern about this issue. Thus, one area of interest of the future evaluation will be to determine if the Leader and/or Co-leader are fulfilling the duties of their respective roles and what do members believe are some specific areas of improvement.

Step Four

The final step in Rouda and Kusy’s (1995) model is to identify possible solutions and growth opportunities. There were very several suggestions for improvement for change that were brought forth. It appears that members felt very comfortable sharing with one another in their small groups, and providing written responses to questions on the questionnaire. It is anticipated that the verbal responses to the additional questions, had they been posed, would have been much more parsimonious than the written responses on the questionnaire. One possible reason for this is that members would not feel comfortable sharing this information with the group when the Leader and Co-Leader are present. However, one way for the Leader and Co-Leader to enhance their knowledge base is to attend general meetings and conferences, and to engage in as many professional development opportunities as possible. If a program evaluation is conducted, it will address whether this is a practice that the Leader and Co-Leader engage in and the degree to which they deem that it is effective.

Conclusion

In September, 2009, the Leader of the non-profit weight-loss group in rural Saskatchewan approached this evaluator, who is also a member of the group, and expressed concerns about the efficacy of her leadership and the degree to which the meetings were beneficial to members. At that same time, this internal evaluator was cognizant of the fact that an upcoming assignment for her graduate class involved conducting an evaluation assessment of a program, and discussed the possibility of performing such an assessment and a possible evaluation plan. The Leader was delighted at this possibility, as she has many concerns, such as whether the practices that have been implemented to assist and encourage individuals as they learn to make healthier food choices, get more physical activity, and learn how to achieve a healthier lifestyle and an improved quality of life are actually meeting the objectives of the program? Following the evaluation assessment that occurred within the context of a regular meeting of this non-profit weight-loss group in rural Saskatchewan, it has been determined that a formal evaluation of this program must be conducted. Therefore, a formal evaluation plan will be designed within the next six weeks.


References

About #### Club Inc. (n.d.). Retrieved 09 29, 2009, from http://www.####.org/About ####.aspx
Rouda, R. a. (1995). Needs Assessment--THe First Step. Retrieved October 12, 2009, from http://alumnus.calech.edu/~rouda/T2_NA.html

List of Appendixes
Appendix A: Needs Assessment Questionnaire

Appendix A: Needs Assessment Questionnaire
******####—$$$$$$$ !!!!!

1. What are the three aspects of the group that you like the most?



2. What are the three aspects of the group that you dislike the most?


3. What are some things that you know NOW about weight loss and healthy lifestyle habits that you did not know when you joined ####? (example: food choices, types of physical activity, stress management techniques) Please be as specific as you can.





4. What are some things about weight loss and healthy lifestyle habits that you would like to know? Please be as specific as you can. (Examples: see above!)





5. What activities would you like to see incorporated into our meetings that you feel would help you achieve your weight-loss goals? Examples: discussion about specific topics (foods, behaviour changes, types of physical activity); cooking demonstrations; exercise demonstrations?