Professionals in the parks and recreation field know that program evaluation is imperative to demonstrate outcomes and ensure continuous quality improvement. In addition, professionals know that research is a necessary part of the parks and recreation profession. Research and program evaluation can assist in ways such as improving service delivery, securing funding, validating the importance and value of recreation to a community, and/or increasing the body of existing knowledge of parks and recreation and support professional practice, just to name a few. With increasing pressure to show outcomes to sustain programs and services, research can assist organizations to continue to demonstrate on-going improvement. Action research, in particular, can provide a great tool for quality program evaluation and a practitioner-friendly model for research. The article will briefly review foundational aspects of action research, including a systematic process to conduct it. A case study will serve as an example of the process. What is Action Research? Action research is a disciplined inquiry and process that supports the attainment of an organization’s goals while building and creating community (Calhoun, 1994). Creating community is important as it conveys that individuals, whether they are administrators, supervisors, line staff or participants and family members, are a valued and integral part of the organization. Action research focuses on improving the quality of an organization, and provides reliable and valid information to support such improvements. Although it has been traditionally conducted in educational environments, action research is a powerful tool for improving recreation programs and services while at the same time improving the health of the organization (Calhoun, 1994). A healthy organization has the strength and vitality to weather difficult times and can serve to motivate members of the organization to achieve their best. Essentially, action research is establishing priorities and taking action based on the systematic collection of data. What is Collaborative Action Research? Collaborative action research is a specific type of action research that brings together a group of individuals to conduct systematic inquiry. The research group or team is comprised of individuals who have mutual interests and work together to study those interests. For example, team membership may include, but not be limited to, participants, parents, program leaders and supervisors. Parks and recreation professionals are experienced in working together in teams, and, as such, collaborative action research can be ideal in supporting program evaluation (da Gama & Hironaka-Juteau, 2002). In addition, a unique feature of collaborative action research is that it lends itself to including university students in the process. With this comes the potential for future professionals to gain hands-on experience in conducting research while at the same time supporting the research team’s quest to study a particular topic of interest. Sagor (1993) states, “only two guidelines must be followed when conducting collaborative action research: (1) the phenomena chosen for study must concern the teaching/learning process, and (2) those phenomena must also be within the practitioner’s scope of influence” (p.9). As such, unlike other forms of research, the members of the research team and the organization are the primary audience for the results of collaborative action research. However, when action researchers take the time to share the results from their studies outside of the research team through presentations at professional conferences or publications other professionals and programs can benefit. A Five-Step Process to Conduct Collaborative Action Research A five-step process of collaborative action research is offered by Sagor (1993), which is just one starting point for using action research as part of program evaluation. Sagor’s process includes the following steps:
Problem Formulation Needs or issues that are of the highest professional concern to the research team are identified by the research team. Characteristics of the issues are identified, as well as additional information to aid with a more comprehensive understanding. For example timing, such as when it first appears, may be a characteristic of an issue or need. Data Collection For each research question that surfaced during the problem formulation phase, action researchers are expected to collect three sets of data. Triangulation assists with improving the quality of the findings and allows for adequate data collection. Some examples of data sources are written surveys, interviews, observations, attendance records, and program plans. A plan to collect the data is created to aid in the collection process. This plan would reflect the data to be collected and the respective timeline for the collection to occur. Data Analysis Team members examine the patterns that the data reveal and work to draw conclusions. It is important that this is done systematically and objectively to assure accurate and useful information. Although the main audience of the research is the research team itself, it is imperative that the data are analyzed with impartiality. Reporting Results As stated earlier, the primary audience for the results from collaborative action research is the research team; however, the results do not need to be limited to this audience. Sharing the results via professional publications or conferences so that others may benefit should be considered. Additionally, results may be a key in securing funding or community support. Action Planning Since the purpose of collaborative action research is the improvement of professional practice, developing an action plan based on the reported results is essential. Together, team members examine the results, determine the priorities, and establish action steps for the attainment of the organization’s goals to create community. Case Study The following is a case study that is intended to serve as an example of the five-step process of collaborative action research previously outlined. The case is a joint project conducted by Break the Barriers, a non-profit organization in Fresno, California and students in the Recreation Administration and Leisure Studies Program’s Programming and Evaluation in Recreation, Parks and Tourism class at California State University, Fresno. Break the Barriers is an all inclusive environment teaching people with and without disabilities various sports, arts, recreation, aquatics, health, and fitness programs. This process would allow for the 37 students enrolled in this course to gain hands-on experience with program evaluation while at the same time providing a service to Break the Barriers. The involvement by students in the process is a distinctive feature of collaborative action research. Problem Formulation The purpose of the study was to work collaboratively on an action-oriented research project to investigate a professional recreation service delivery concern in an existing setting. The focus of this project evolved from dialogue about a problem/need/concern identified by Break the Barriers that could benefit from the “people power” that a class of college students could provide. This study focused on answering the following question: “Are individuals participating in inclusive group recreation programs receiving the level of instruction and support needed to maximize their potential?” The overarching goal of the study was to assess the effectiveness that Break the Barriers has in maximizing the potential of participants. An orientation tour and meeting for the RLS 139 Evaluation Team occurred at the Break the Barriers facility prior to the beginning of the data collection process. Data Collection Quantitative and qualitative methods were employed to study the research question. The following data collection tools were developed by the RLS 139 Evaluation Team and approved for use by Break the Barriers: (1) Parent/Guardian Survey, (2) Participant Observation Form, and (3) Staff Inquiry Form. Target programs were selected based on interest and availability of the RLS 139 Evaluation Team members. Target programs were observed twice, and at either time parent/guardian surveys were administered and staff inquiries made. A five point Likert scale format was utilized, along with open ended questions on each of the three data collection forms. Data was gathered on the following programs: Dance, Gymnastics, Marital Arts, Power Camp, Sign Language, and Swimming. Data Analysis Measures of central tendencies and thematic analysis were used to analyze the data. Mean scores were identified for each Likert scale item. Qualitative data were coded and analyzed, and themes were identified from the three data sources. Overall, results of the study indicated satisfaction with the class sessions and that the program is effective in maximizing the potential of participants at Break the Barriers. Of those parent/guardians surveyed (n=154), the majority (89%) indicated satisfaction with the class sessions. Comments and suggestions were categorized and revealed that participants actively participated, demonstrated improvement in skill, and were comfortable with the class instructors. Suggestions for enhancement were also revealed. Active participation during class sessions was noted by 90% of the parent/guardians surveyed. Participants who did not initially engage in the activity were provided with encouragement. Staff was aware of individual student needs and was flexible in their instruction, adapting or changing to those needs. Narrative comments highlighted a preference for smaller class sizes, which would allow for more one-on-one instruction time. This was the most consistent suggestion for improvement. As a result of the parent/guardian surveys, Likert scale ratings suggested that participants demonstrated improvement in skills as a result of attending classes. Eighty-eight percent (88%) indicated that the class gives their student self-confidence. Positive changes also included increased physical skill. Participant observation data revealed that the majority of students demonstrated improvement in skill during the class session. Staff evaluated the progression of their respective classes through formal levels, testing, and assessment, building upon the students’ existing skills. Staff consistently stated goals of the respective classes as improving self-esteem, confidence and having fun. Students appeared comfortable with the instructors during the majority of the observations (97%). High fives, smiles and laughter were plentiful from both students and instructors. Respondents’ narrative comments suggested additional comfort with the facility environment. When asked about why a parent/guardian chose to enroll their student in a class at Break the Barriers, word of mouth and diversity of the students who attend Break the Barriers were reoccurring themes with the parent/guardian survey. Reporting Results Results and recommendations were presented to select audiences in three formats by members of the RLS 139 Evaluation Team. Two in-service presentations were conducted, one to Break the Barriers department heads and one to Break the Barriers department teachers. Four separate poster presentations were developed and presented at the Central California Therapeutic Recreation Symposium. Posters were created for Dance, Gymnastics and Swimming as these were the largest target programs, while the fourth poster summarized findings from all the target programs. Finally, a presentation was made to the Break the Barriers Board of Directors. The content of each presentation included five key pieces of information:
Each hour long in-service included an experiential activity and a post in-service evaluation form that was distributed to in-service participants to provide feedback on the presentation. Each member of the RLS 139 Evaluation Team was involved in one of the four presentation groups based on interest and availability. Care was taken in assuring representation from each target group in each of the presentation groups. Action Planning Based on the findings and recommendations by the RLS 139 Evaluation Team, preliminary action planning occurred. One area for potential enhancement included exploring the development of a volunteer program to increase use of volunteers to assist with the redirection of students who may become distracted. In addition, action planning included incorporating the use of an informal game or activity to engage students while waiting for their turn. For example, an oversized dice game where students could engage in a movement or behavior based on the number rolled and the specific activity identified (e.g. do five jumping jacks, say one thing you like about this class) would allow for focused attention during turn taking. Conclusion This article presented a useful tool for program evaluation that can have meaningful application for park and recreation professionals. Collaborative action research is an ideal way for agencies to systematically investigate a programmatic area in need of improvement or enhancement. A case study example was provided which outlined the process of collaborative action research. This case study describes the collaboration between university students and a non-profit agency and is anticipated to serve to motivate others to explore this as a viable tool for program evaluation. These authors concluded that collaborative action research is an effective way to bring together professionals and future professionals, to engage in a mutually beneficial process designed to ultimately improve the health of a park and recreation organization. By including various stakeholders as well as incorporating college students pursing formal education in the park and recreation industry, the benefits are mutual. For the organization, the benefits of being part of the research team can be multi-fold. When writing for grants, the collaborative action research process can assist in identifying the evaluation component. Teaming allows for additional support and “outside” eyes to offer external perspectives which ultimately help the program to improve. During the process, staff is motivated to internally explore, “What can I do differently to have an even better outcome?” Brainstorming and striving to create better programs in the future becomes part of the organization’s culture. When board members hear directly from the research team, they begin to more fully understand and appreciate the day-to-day impact of the organization’s programs and services on the community. And even after the research project is completed, the mission of the organization stays with team members well into the future. Students involved in the collaborative action research process also benefit. The vision of becoming a recreation professional becomes clearer to the students during the process. Less seasoned eyes become open to the importance of evaluating recreation programs as they realize the positive effects. With explicit directions on what to evaluate, students begin to observe behaviors and approaches that they would not have considered if simply watching a program. When students notice the dramatic improvement in a program as a result of small changes directly implemented as a result of their effort, the importance of evaluation grows. Finally, when students are active participants in the process, they see significant improvement in their own confidence, and grow in their understanding of the importance of inclusion in all settings. Ultimately, they gain valuable information in preparing them for their future careers. At a time when resources are diminishing and there is increasing need to demonstrate outcomes, collaborative action research is an ideal tool in program evaluation where benefits are plentiful. It is the authors’ hope that other professionals will explore opportunities to apply this tool as the work to justify the existence of programs through regular program evaluation. References Calhoun, E.F. (1994). How to use action research in the self-renewing school. Alexandria, VA: Association for Supervision and Curriculum Development. da Gama, G. & Hironaka-Juteau, J.H. (2002, Summer). Action research: A new tool in program evaluation. California Parks & Recreation Magazine, 58, 3. Sagor, R. (1993). How to conduct collaborative action research. Alexandria, VA: Association for Supervision and Curriculum Development. |
