Aboriginal community-centered injury surveillance: a community-based participatory process evaluation.

While injuries are a leading health concern for Aboriginal populations, injury rates and types vary substantially across bands. The uniqueness of Aboriginal communities highlights the importance of collecting community-level injury surveillance data to assist with identifying local injury patterns, setting priorities for action and evaluating programs. Secwepemc First Nations communities in British Columbia, Canada, implemented the Injury Surveillance Project using the Aboriginal Community-Centered Injury Surveillance System. This paper presents findings from a community-based participatory process evaluation of the Injury Surveillance Project. Qualitative data collection methods were informed by OCAP (Ownership, Control, Access, and Possession) principles and included focus groups, interviews and document review. Results focused on lessons learned through the planning, implementation and management of the Injury Surveillance Project identifying lessons related to: project leadership and staff, training, project funding, initial project outcomes, and community readiness. Key findings included the central importance of a community-based and paced approach guided by OCAP principles, the key role of leadership and project champions, and the strongly collaborative relationships between the project communities. Findings may assist with successful implementation of community-based health surveillance in other settings and with other health issues and illustrate another path to self-determination for Aboriginal communities. The evaluation methods represent an example of a collaborative community-driven approach guided by OCAP principles necessary for work with Aboriginal communities.

Canada includes a diverse Aboriginal population. British Columbia (BC) is the Canadian province with the largest number of First Nations bands, numbering approximately 200 unique bands (British Columbia Provincial Health Officer 2009). In this paper, we use the term First Nations as it was defined in the Canadian Constitution Act 1982, Section 35 to refer exclusively to Indian communities or individuals, and the term Aboriginal as inclusive of Indians, Inuit and Métis people (Waldram et al. 2006).

Historic colonial patterns of interaction and oppressive policies that outlawed indigenous cultures have left a difficult legacy, including effects on health and well-being that many are working to counteract. One impact has been on the rate of injuries. Numerous reports and research confirm both intentional (e.g., suicides, assaults) and unintentional (e.g., motor vehicle crashes, falls) injuries as a leading cause of death and hospitalization for Aboriginal populations, with mortality rates greatly exceeding those found in non-Aboriginal populations and differences in injury mechanisms and risk factors (British Columbia Provincial Health Officer 2009; Health Canada 2009; McDonald 2006; Tjepkema 2005). For example, in BC in 2006, the age-standardized mortality rate due to motor vehicle crashes was 1.9 per 10,000 standard population amongst Status Indians compared to 0.7 for other residents. The suicide rate was 1.7 for Status Indians compared to 0.7 for other residents. Potential mechanisms for the increased injury rates may include both disparities in accessibility to prevention mechanisms, programs and health care facilities as well as incompatibility between the cultural characteristics and needs of the Aboriginal populations and the policies, programs and services that are provided (Beals et al. 2009; Hernandez et al. 2009).

Any health promotion activities within Aboriginal communities must respect the integrity of the community and traditional sources of knowledge (Cochran et al. 2008), as well as ensure that ownership of the work and information is preserved within the community. In Canada, this requires collaborative community-driven approaches underpinned by OCAP principles (First Nations Centre 2007). OCAP principles, which refer to Ownership, Control, Access, and Possession, were developed by First Nations and sanctioned by the First Nations Information Governance Committee (First Nations Centre 2007). They advocate for the control by Aboriginal populations of data and health information concerning Aboriginal populations, and support self-determination of Aboriginal people in relation to research processes. The collaborative OCAP-based approach works to empower Aboriginal people by providing control and self-determination over the research process (First Nations Centre 2007).

There are a number of evaluation methods that are based on principles of collaboration, participation and empowerment of disempowered groups and are well suited to OCAP principles and the needs and priorities identified by First Nations in Canada. In a seminal review of participatory evaluation techniques, Cousins and Whitmore (1998) describe three key elements of participatory evaluation, including control of the evaluation process, stakeholder selection for participation in the evaluation, and depth of participation of stakeholders. Each of these elements can be considered with respect to the level of control over decisions regarding the evaluation process and conduct that the different parties have. This can range from control resting entirely with the researcher, to resting entirely with the stakeholders. Participatory approaches seek to maximize control for stakeholders.

Fetterman (1994, 2001) describes empowerment evaluation as designed to use the evaluation process and findings towards improvement and self-determination. Later writings outline ten principles for empowerment evaluation: 1) improvement of individuals, organizations and communities; 2) community ownership, including power over and responsibility for the evaluation; 3) inclusion of and collaboration with stakeholders; 4) democratic participation of stakeholders, ensuring process is transparent and demystified; 5) social justice in use of the evaluation to facilitate attainment of resources and improve inequalities; 6) community knowledge is reflected in the evaluation tools; 7) scientific evidence is valued and appreciated; 8) capacity building occurs in organizations’ abilities to use data and sustain evaluation efforts; 9) organizational learning and change results; and 10) accountability for the evaluation is shared by all involved with the evaluation (Wandersman et al. 2005). Adherence to these ten principles is sought to maximize empowerment and change.

The principles that these community-based participatory approaches espouse formed the basis of the evaluation described in this paper. We present a Canadian evaluative case study of a community-centered injury surveillance system in a First Nations community.

Importance of Community-Based Injury Surveillance

The disproportionate burden that injuries represent for Aboriginal populations has already been described above. However, available national and provincial statistics do little to illuminate the diversity in injury patterns or risk factors in regional settings (Bell et al. 2011). In BC, overall injury prevalence and rates of specific types of injuries within First Nations populations vary by geographic region. Even a crude cut parsing of the province into four geographic units reveals a great deal of variability with rates varying from 11.9 to 22.1 deaths per 10,000 standard population (British Columbia Vital Statistics Agency 2000). Rates of injury due to specific causes, such as suicides, have also been found to vary widely across communities (Chandler and Lalonde 1998). A recent report indicated no youth suicides between 1992–2006 for many BC bands, with others reporting rates exceeding 700 per 100,000 population (British Columbia Provincial Health Officer 2009).

The vast variability in injury rates highlights the importance of obtaining community-level data to facilitate communities’ understanding of their injury profile and hazards. Not only do First Nations bands vary substantially in the risk and protective factors present in their communities, but they also may reside in different geographical contexts, such as mountainous or aquatic terrains, that can impact the rates and types of injuries. As a result of these varying contexts, collecting and analyzing community-level data are key to understanding injury patterns and risk factors to assist in efforts to reduce injury rates (Bell et al. 2011; Mullany et al. 2009).

Few published studies describe injury data collection initiatives in Aboriginal communities internationally, and none to our knowledge in Canada. Two studies in the United States and Australia (Helitzer et al. 2009; Shannon et al. 2001) report on the design and evaluation of general injury prevention community development projects in Aboriginal communities. These projects collected baseline injury data in the community, and designed and evaluated intervention activities. The US study showed significant improvements in attitudes towards the targeted safety concerns, as well as improvements in knowledge, skills and confidence among stakeholders and increased community capacity to conduct and evaluate safety projects (Helitzer et al. 2009). The Australian study found a statistically significant drop in average monthly injuries at a community medical clinic from 96 to 65 injuries following the introduction of community-generated interventions (Shannon et al. 2001).

One study addressing the establishment of community-based injury surveillance systems in Aboriginal communities describes a surveillance system developed to gather data from both community and clinical settings on suicidal behavior among Apache youth in Arizona (Mullany et al. 2009). Surveillance activities assisted with identification of unique risk factors, guided targeted programs such as adaptation of an evidence-based emergency department intervention for suicidal youth. Evidence from these international studies suggests the potential for positive results from community-based injury surveillance in Aboriginal communities.

A group of community health leaders (Health Directors) representing the Secwepemc Nation in BC recognized the burden that injuries represented for their communities and the absence of community-level data allowing for local injury prevention planning. They identified the need to establish a community-based injury surveillance system. The Secwepemc Nation includes 17 bands diverse in size and geography, with populations ranging from less than 100 residents to over 1000 residents. The communities also vary considerably with respect to distances to town centers and access to health service facilities. In 2005, several Secwepemc First Nations communities officially launched the Injury Surveillance Project (ISP) and initiated data collection using the Aboriginal Community-Centered Injury Surveillance System (ACCISS).

Aboriginal Community-Centered Injury Surveillance System (ACCISS)

ACCISS was designed to be implemented and owned by Aboriginal communities for the purposes of self-determining action on injury (Auer and Andersson 2001). Development of ACCISS was sponsored by the Canadian government, and focused on capacity building and facilitating community administration of the four key injury surveillance activities: collecting, analyzing, interpreting and using data. Basic tenets associated with the system are that it is community-based and community-paced to ensure that injury surveillance activities are undertaken at a community-determined pace and based on individual community needs, readiness and capacity. ACCISS was developed to function in the diverse circumstances of Aboriginal communities, such that it can be implemented in communities with varied population sizes, geographic characteristics, resources, and health service delivery models. Prior to commencing data collection, each community undertakes an injury mapping process to identify how injury cases can be systematically identified, where key sources of injury data exist and methods by which injury data can best be collected, ensuring that data collection methods are tailored to reflect the unique structures and services of individual communities (Brussoni et al. 2009). For example, a remote community may consider collecting data in the local health center, school, child care facility and worksite, whereas a community located within or near an urban center may also consider partnering with local hospitals and tertiary care facilities to obtain data on community members.

Three tools support the injury data management component of ACCISS: an injury surveillance form, a data entry database, and a data analysis program. The electronic component uses Epi Info TM (Centers for Disease Control and Prevention 2008) as its software platform to drive data entry and analysis functions. The ACCISS 2.0 user manual (Health Canada & First Nations and Inuit Health Branch, 2007) provides an additional resource.

Injury Surveillance Project (ISP)

Secwepemc Nation Health Directors sought to build capacity to manage heath data and to improve the health, safety and well-being of the people, thus providing impetus for undertaking ISP. The project was implemented in three phases: 1) pre-implementation activities setting the groundwork for undertaking surveillance; 2) implementation activities included data collection, analysis and reporting followed by using and sharing data for injury prevention activities; and 3) overall project maintenance and monitoring activities.

Health Directors assumed leadership and coordination of injury surveillance in their respective communities. Figure 1 illustrates the organizational diagram of personnel and their roles in the ISP and the evaluation described herein. At the project level, the Health Director’s Project Team was formed to aid with project coordination and administration, and consisted of a subset of Health Directors from participating communities. Communities also had access to a Project Support Team that provided technical assistance and training.

Fig. 1

Personnel and roles for ISP and implementation evaluation. White circles indicate personnel involved in ISP, as well as evaluation participants. Gray circles indicate personnel involved in evaluation. The dotted line indicates that members of the Project Support Team were the same as internal evaluators

ISP personnel within each community included data collectors, and data entry and report generators. Data collectors completed the injury surveillance form and included people already responsible for identifying or treating injured community members, as well as a wide range of staff and supervisors. Data entry and report generators coordinated and supported the data collection network within their community; entered, cleaned, and analyzed data; and produced summary reports.

In 2007, the Health Directors Project Team initiated an evaluation of the implementation of ISP focusing on the pre-implementation and implementation phases, which included the establishment of data collection, analysis and reporting for the first 3 years of the project. They established a collaboration with the evaluation team, which consisted of external evaluators (authors), and internal evaluators, the latter having been part of the Project Support Team that worked with the project communities in establishing and maintaining ACCISS. The evaluation participants were the Health Directors, and community staff involved in implementation and ongoing management of ACCISS, as well as key stakeholders identified by the Health Directors Project Team.

This paper presents the methods and findings from the process evaluation of ISP. The evaluation was community driven, guided by the Health Directors and underpinned by OCAP principles. Our aim in this paper is to identify lessons learned regarding implementation of an injury surveillance system that may benefit other communities considering implementation of health surveillance.

Methods

Evaluation Approach

Figure 2 illustrates the collaborative evaluation process undertaken for this evaluation. Initial steps involved collaborative planning by the Health Directors Project Team and the evaluation team through which evaluation objectives were developed that guided data collection and analysis strategies. The objectives focused on in this paper include those addressing: identification and description of project implementation facilitating factors and challenges; project learnings and promising practices; and project outcomes achieved to date. At the time of the evaluation, ten project communities were actively participating in injury surveillance, had collected at minimum 22 months of injury data and were producing community-specific injury reports. The evaluation was sanctioned as a project activity by Band Council Resolutions and community-based protocols. Ethics approval was obtained from the university research ethics board.

Fig. 2

Collaborative evaluation process, including tasks, data collection, analysis procedure and timelines

Consistent with Cousins and Whitmore’s (1998) conceptualization, a collaborative participatory community-based approach was used to develop and carry out evaluation activities. This approach used evaluation questions driven by the community, methods and measures that were community-sensitive and reporting that was community-focused (Mathison 2005). This approach was critical for ensuring that analyses and findings were culturally sensitive and accurate (Davis and Reid 1999; Mullany et al. 2009). Also guiding the evaluation process were OCAP principles (First Nations Centre 2007), which were relevant for ensuring appropriate and sensitive evaluation methodology, and ownership of the evaluation process, protocols and products resting with the Secwepemc Nation. The Health Directors Project Team decided that a qualitative approach for data collection and analysis was the most culturally aligned methodology for maximizing cultural sensitivity of data collection strategies, for understanding these issues from the perspective of community members and other stakeholders, and for developing an in-depth understanding of the contextual issues impacting implementation of the surveillance system. Primary responsibility for data collection was assumed by the external evaluators while data interpretation and reporting of findings was undertaken by all team members.

Evaluation Process and Activities

As illustrated in Fig. 2, the evaluation planning process occurred through a series of face-to-face meetings between evaluators and the Health Directors Project Team. This planning process included defining the focus and scope of the evaluation, refining evaluation objectives, and developing evaluation protocols and data collection instruments. A logic model was developed to outline the overall project, delineate the focus of the evaluation and define evaluation objectives.

Data Collection

Data collection was undertaken in two locations within the Secwepemc Nation to facilitate the participation of all project communities and accommodate natural northern and southern geographical groupings. Data were collected via multiple sources including document review, focus group discussions, and individual interviews. The methods and focus of data collection activities are described in Table 1.

Table 1

Data collection activities for ISP process evaluation

Data collection method	Sources (n)	Data collection focus
Document review	Annual reports, contracts, correspondence, manual, presentations, meeting agendas	Identifying stakeholders, clarifying context, processes and activities
Focus groups (total = 5)	All focus groups (total n = 32a)	Capacity building, training, lessons learned, injury surveillance tools, activities and processes
(1)	Health Directors Project Team (n = 8)	Community readiness, sustainability, use of injury data
(2)	Data collectors (n = 12; 9)	Collecting injury data
(2)	Data entry/report generators (n = 3; 4)	Project readiness, use of injury data
Interviews	All interviews (n = 10)	Community readiness, project challenges and successes, project outcomes, sustainability
	Health Directors Project Team (n = 3)	Value and utility of the project and surveillance system
	Federal, provincial and regional government representatives (n = 4)	Interest of organization in surveillance system, working with Aboriginal communities, government-related challenges, influence on government priorities
	Non-participating community (n = 1)	Community factors affecting readiness
	Project Support Team (n = 2)	Implementation logistics, shifting roles, value and utility

aParticipants with overlapping roles attended all applicable focus groups

Document Review

The document review aided in identifying key ISP stakeholders; increasing understanding of operations and processes; gathering and synthesizing historical context; and verifying information about project timelines, activities and outcomes. Materials examined included annual reports, contracts between bands and government agencies, e-mail correspondence, ACCISS user manual, project presentation slides, meeting agendas, and other relevant documents.

Focus Groups

Five focus groups were conducted: one with the Health Director’s Project Team, and two each with data collectors and data entry/report generators. In situations where participants filled multiple roles (e.g. data collector and data enterer), they participated in more than one focus group. Each focus group also included three members of the evaluation team—a facilitator, a note taker and a flip chart recorder. All project communities were represented at the focus groups, which included a total of 32 participants.

All focus groups covered topics of capacity building, training received, and lessons learned. In addition, the Health Directors Project Team Focus Group included discussion of: community readiness to take on ISP; issues related to the injury surveillance tools, activities and processes; sustainability; and the use of injury data. The Data Collectors Focus Groups also explored: participants’ perspectives on the process of collecting injury data; surveillance tools; and data collection activities and processes. The Data Entry/Report Generators Focus Group discussed use of the injury surveillance tools and associated tasks; project readiness; and the use of injury data.

Individual Interviews

Ten in-person and telephone interviews with key stakeholders were conducted to provide in-depth information from a range of perspectives. The selection of interview respondents was informed by discussion with the Health Directors Project Team who identified individuals able to provide perspectives on topics relevant to the evaluation objectives. This included individuals within the communities, the Project Support Team, and regional and federal government representatives. All interview respondents were asked core questions related to community readiness, project challenges and facilitators, project outcomes and sustainability. Additional interview questions were developed for the different project-related roles that interviewees held to gain insight into their unique perspectives.

Consistent with ensuring that the methodology was culturally acceptable to the project communities, handwritten notes (rather than audio recordings) were taken at all interviews and focus groups. Each data collection event included an interviewer/facilitator and at least one other note taker. Notes were transcribed and then imported into NVivo 7™ software for analysis.

Data Analysis

Figure 2 shows the four steps involved in analysis of the evaluation data. Analysis activities were guided by methods outlined by Miles and Huberman (1994) and involved the external evaluators, internal evaluators, and Health Directors Project Team in data analysis and interpretation. The analysis process was structured to support consensual analysis whereby evaluation team members and Health Directors were involved in data analysis and interpretation (Hill et al. 2005). This served to minimize bias, improve validity and support OCAP principles.

Open coding resulted in the identification of 77 codes (e.g., developing leadership skills; increasing awareness of injury issues in the community). Through a consensus process, this list was condensed to 12 categories (e.g., capacity building outcomes, use of data for prevention activities). Data fractured during initial coding were reassembled via development of preliminary themes to provide coherence and illuminate relationships (e.g., leadership and champions, culture of prevention). Themes were refined initially via discussion with internal evaluators and subsequently via consultation with the Health Directors Project Team. This process resulted in four main areas of focus that were outlined in the final evaluation report, with up to six themes for each (total themes = 18). This paper describes a selection of these findings with broader relevance to other communities and contexts.

Results

We identified lessons learned regarding planning, implementation, management and early outcomes of ISP across five main thematic areas: 1) Project leadership and staff; 2) training; 3) project funding; 4) initial project outcomes; and 5) community readiness for implementation.

Project Leadership and Staff Lessons Learned

Strong leadership for ISP and community leaders’ support held central importance throughout the project: from deciding to undertake ISP; to identifying resources for implementation; through to implementing and administering project activities; and working to ensure sustainability.

Evaluation participants identified that project champions, who often were community Health Directors, were essential. Champions were seen as contributing through recognizing the need for local injury surveillance and advocating for ISP to government organizations, to staff working in the field and to general community members. Furthermore, they provided education to staff and potential data collectors about the purpose and importance of the project, particularly since staff were often concerned about how ISP might add to existing workloads. Continuity in Health Director leadership, particularly in early stages of planning and implementation, facilitated assuming the role of champion and overseeing project implementation activities.

Champions also played a key role in advocating for ISP among community leaders. Formal support of community leadership through Band Council Resolutions or Board Motions was critical to initiation of ISP. This support was obtained in project communities despite potential challenges in convincing leaders as to the purpose and benefit of collecting injury data that may not be available for use for a year or more.

Health Directors representing the project communities developed collaborative, solution-oriented approaches that contributed to their ability to successfully negotiate challenges that arose throughout the course of the project. Unexpectedly, working as a team created “positive peer pressure” to continue; maintained a mutual project path; and ensured a stable course of action in moving the project forward. As ISP progressed, a smaller group of Health Directors established the Project Management Team, which held primary responsibility for project administration. Several benefits seen to be associated with having this team included facilitating ongoing decision-making, fostering project stability and reducing workload for any one Health Director. Access to external expertise via the Project Support Team was also seen as a crucial component for project implementation through provision of ongoing methods expertise, training, mentoring and support.

Project staff attitudes played a role in project implementation, particularly since staff needed to manage surveillance tasks in addition to existing workloads. Staff expressed concerns around competing workload priorities and commented on how it was sometimes difficult to incorporate data collection as part of routine activity. Participants noted that ideally more than one staff member in each community should be trained in injury surveillance so as to minimize disruptions from staffing change. Despite challenges identified by project staff, they expressed enthusiasm and recognized the value of ISP. Project staff members were interested in the capacity building associated with developing injury surveillance skill sets that also had the potential to transfer to other areas of work, and were prepared to undertake the extra work and activities involved.

The community-based and community-paced nature of ISP enabled individual communities to drive implementation of injury surveillance activities based on their community-specific needs, schedules and capacities, rather than conforming to external deadlines. Health Directors strongly emphasized this as an important factor influencing their decision to become involved, since it allowed them to consider their community’s readiness to undertake surveillance.

Training Lessons Learned

Training for project staff was delivered using three main formats: 1) group training sessions focusing on injury surveillance and prevention theory, as well as hands-on surveillance skills training; 2) on-site training relating to community specific issues and individual training needs; and 3) ad hoc training of data collectors emphasising the correct completion of injury surveillance forms. The first two formats were provided by the Project Support Team and modelled on adult learning approaches to accommodate differing abilities, learning styles and life experiences. The third training format was provided on an as-needed basis by project staff to new data collectors.

Evaluation participants reported that the diverse educational backgrounds and skill levels of project staff posed the greatest challenge to formal group training sessions. However, the mixed skill level was also seen as an opportunity for more experienced staff to mentor those with less experience. The ad hoc training provided to data collectors by project staff used varied informal approaches (e.g., staff meetings). This training served to meet immediate needs and circumstances; however some evaluation participants raised concerns regarding the less consistent delivery and lower levels of detail provided in the ad hoc training approach.

Project Funding Lessons Learned

Evaluation participants representing government organizations expressed their recognition of the substantial burden that injuries represent for Aboriginal populations, highlighting the lack of correspondence between official government priorities for funding and the main sources of impact on health in Aboriginal communities. Furthermore, lack of government priority and resources for injury prevention meant that obtaining resources and funding for implementation of injury surveillance and for action on injury priorities identified through data collection represented an ongoing challenge. However, evaluation participants considered the Health Directors’ determination to implement the project, develop innovative solutions for identifying project funding and community resources as strong influencers on project success and sustainability

Initial Project Outcomes Lessons Learned

Project staff reported extensive and often unexpected capacity building across a variety of areas including: 1) the ability to establish a surveillance system for other health issues; 2) expanded understanding of all health data; 3) development and enhancement of skills in leadership, project management, and communication; and 4) development of OCAP-based management policies that could be applied to the general management of community-based health data. They perceived that their skill development had a significant impact across their work, and resulted in ongoing commitment and championing for ISP as a worthwhile activity to have undertaken and to continue involvement with into the future.

There were early indications that community-specific injury data were already leading to injury prevention efforts in several communities. For example, one community that identified falls occurring among Elders in home bathrooms as an issue implemented a bath mat distribution program. Another community established an ice shoe loaner program to address falls resulting from icy outdoor surfaces. Evaluation participants noted that some of their injury prevention efforts already appeared to be reducing injuries.

Community Readiness Lessons Learned

Based on the findings outlined in the sections above, the evaluation team worked with the participating Secwepemc First Nations communities’ Health Directors to identify factors of community readiness for injury surveillance that were seen to be relevant for communities contemplating implementation of surveillance. The Health Directors highlighted three factors as particularly important for project implementation success and longer-term sustainability:

Awareness and knowledge that injuries are a problem as a basic precursor to promoting the preventability of injuries and the central function that surveillance plays for injury prevention efforts.

Having shared vision and values amongst the project team regarding project implementation to keep the project focused and cohesive in its capacity building and problem solving approaches.

Leadership stability within the project team to ensure continued championing of the project and a stable course for project activities.

Discussion

The disproportionate burden that injuries represent in Aboriginal populations makes injury prevention a priority. Addressing this health issue has numerous challenges, including the impact of colonialism that resulted in the loss of Aboriginal culture and tradition, the geographic location that may limit access to services, and the lack of local injury data to assist with injury prevention planning (Auer and Andersson 2001). While it is important to recognize the risk factors, negative conditions and challenges associated with health issues in Aboriginal populations, many Aboriginal communities have worked to overcome impacts associated with colonialism. Highlighting their successes is important for illustrating another path to self-determination, as a tool for knowledge transfer and as inspiration for other communities and health practitioners. The Secwepemc First Nations participating communities’ ISP is one such example of an innovative collaborative initiative with the potential to provide new insights to other communities and practitioners working on injury issues and to policy makers with interest in this area. The lessons learned from this project can serve to assist with successful implementation of injury and other types of health surveillance in community settings. Below are highlighted methodological and community-related factors that emerged as key findings in the process evaluation of ISP that may be applicable to other settings and health issues.

Importance of Community-Based Participatory Approaches to Injury Surveillance in Aboriginal Communities

This evaluation clearly demonstrated how the implementation of injury surveillance in the Secwepemc Nation benefited from use of a community-based and community-paced approach sensitive to the needs and challenges specific to communities involved. A key aspect of this approach was the use of ACCISS, which was designed as a tool to be adapted to community circumstances and provide data owned and held at the community level.

The OCAP principles that guided surveillance data collection, analysis and management fostered a sense of community control over the information and helped mitigate issues of distrust. The Secwepemc Nation experience using OCAP principles as a central guiding framework helped to support the communities’ self-determination and highlights their potential to assist with implementing injury prevention data collection efforts in a culturally relevant and sensitive way.

Project and Community-Related Factors Impacting Injury Surveillance

Through this process evaluation, it was found that project leadership and staff contributed in important ways to the ability of the ten Secwepemc communities to persevere in implementing injury surveillance despite a lack of dedicated funding. These lessons included the importance of champions at several project levels, stable leadership, collaboration within and between community structures, and training responsive to community needs.

While published research on health promotion or injury prevention in Aboriginal communities is limited, what is available in the literature supports the findings of this process evaluation. For example, Helitzer et al. (2009) noted the importance of perseverance in maintaining stakeholders’ interest and enthusiasm for mobilizing community action in an agricultural safety program among members of the Navajo Nation. Broader research on health promotion in community settings also highlights the role of perseverance, in addition to the need to coordinate efforts in mobilizing community action (Butterfoss 2006). Finally, the importance of positive relationships among key stakeholders is seen as a central attribute of sustainable innovations (Johnson et al. 2004). Similar to the previous research, a significant component of our findings related to the collaborative relationship between Health Directors and project staff across communities and the role they played as ongoing champions for the project, persevering despite competing pressures.

The knowledge gained through this evaluation regarding community-related factors that emerged as important may assist Aboriginal and non-Aboriginal communities to assess their own readiness to undertake a similar initiative. Secwepemc Nation Health Directors recognized potentially detrimental effects to communities when efforts to implement injury surveillance did not succeed due to implementation challenges. These concerns are supported by the research literature that suggests that implementing a strategy in a community that is not ready for it can lead to negative results ranging from project delays to failures (Nilsen 2004). Negative impacts can be mitigated through the development of improved understanding of factors that signal a community’s likelihood for success. It is our hope that the learnings from this project and the factors for community readiness identified above will assist communities with determining their own preparedness to establish injury surveillance. These factors may also be applicable to other health promotion issues, such as prevention of infectious diseases. Further research is needed on these and other factors of community readiness for health promotion activities to determine their value in predicting community success for project implementation.

Capacity Building and Injury Prevention Planning

This evaluation focussed on the process of planning and implementation. However, at the time of the evaluation several promising project outcomes were identified, such as capacity building in the community and initial efforts to use local data for planning prevention strategies. The wide-ranging capacity building that participants reported as enhancing their ability to undertake surveillance, as well as a multitude of other skills may provide additional incentive for communities considering undertaking similar projects. This close connection between surveillance activities and prevention planning efforts in each community are a major strength, particularly since the lack of such connection has received criticism in the literature in that surveillance activities are often removed from the population of interest with little linkage between the data and preventive activities (Auer and Andersson 2001; Johnston 2009; Pless 2008).

A main strength of this evaluation lies in the incorporation of a community-based participatory approach reflecting OCAP principles. This collaborative approach for the design, data collection and interpretation of findings ensured input from communities at all stages of the process and added to the credibility, relevance and cultural sensitivity of the results for project communities. It is consistent with Cousins and Whitmore’s (1998) definition of participatory evaluation in that primary control for decisions regarding the evaluation process, stakeholder involvement, and the depth of participation of stakeholders rested with the Health Directors Project Team. Likewise, the evaluation adhered to many of the principles of empowerment evaluation outlined by Wandersman et al. (2005), such as community ownership, inclusion, democratic participation, community knowledge, evidence based, and accountability. Other principles may emerge with additional time. For example, the results of the evaluation provide support for the value of ACCISS and justification for allocation of resources to community-based injury surveillance (social justice principle); however, it is too soon to tell whether it will influence key decision-makers to identify resources for this activity.

Study rigor of the evaluation results was enhanced through methods that included data and investigator triangulation. In this evaluation, we used multiple data sources including interview, focus group and document review data. Investigator triangulation was ensured through a five-member evaluation team supporting interdisciplinary, internal and external perspectives. In addition, Health Directors provided guidance regarding the cultural sensitivity and appropriateness of methods, as well as interpretation of findings. Ongoing consultation with project communities supported the consensual evaluation process.

Study Limitations

Focus groups conducted with the data entry/report generators were smaller than what is considered an optimal number (Krueger and Casey 2009); however, they were consistent with the community context within which this evaluation was conducted, that includes small community sizes with commensurate level of staffing All individuals involved in these roles participated in the focus groups, and the discussions were extensive. The Secwepemc Nation Health Director Project Team requested that the recording of evaluation data from focus groups and interviews rely on written notes and flip charts, rather than audio recording. While this method did not allow for transcribing of exact wording, accuracy was maximized by including at least two note-takers at each data collection session.

Conclusion

Injuries continue to pose a major burden to the health of Aboriginal people living in Canada. This process evaluation utilized a collaborative, participatory, community-based approach to better understand the challenges and facilitators associated with implementation of an injury surveillance system in a group of First Nations communities in Canada, taking steps to address the injury burden among community members. This evaluation outlines factors that may inform communities considering similar activities in relation to readiness to undertake injury surveillance. It also supports the importance of ensuring that policy and programming efforts within Aboriginal communities are community-based and consistent with OCAP principles.