Methods for development of structure, process and outcome indicators for prioritized spinal cord injury rehabilitation Domains: SCI-High Project.

Background: High-quality rehabilitation care following spinal cord injury or disease (SCI/D) is critical for optimizing neurorecovery and long-term health outcomes. This manuscript describes the methods used for developing, refining, and implementing a framework of structure, process, and outcome indicators that reflect high-quality rehabilitation among adults with SCI/D in Canada.
Methods: This quality improvement initiative was comprised of the following processes: (1) prioritization of care Domains by key stakeholders (scientists, clinicians, therapists, patients and stakeholder organizations); (2) assembly of 11 Domain-specific Working Groups including 69 content experts; (3) conduct of literature searches, guideline and best practice reviews, and outcome synthesis by the Project Team; (4) refinement of Domain aim and construct definitions; (5) conduct of cause and effect analysis using Driver diagrams; (6) selection and development of structure, process and outcome indicators; (7) piloting and feasibility analysis of indicators and associated evaluation tools; and, (8) dissemination of the proposed indicators. Result: The Project Team established aims, constructs and related structure, process, and outcome indicators to facilitate uniform measurement and benchmarking across 11 Domains of rehabilitation, at admission and for 18 months thereafter, among adult Canadians by 2020.
Conclusion: These processes led to the selection of a feasible set of indicators that once implemented should ensure that adults with SCI/D receive timely, safe, and effective rehabilitation services. These indicators can be used to assess health system performance, monitor the quality of care within and across rehabilitation settings, and evaluate the rehabilitation outcomes of the population to ultimately enhance healthcare quality and equity.

Introduction

Quality of care can be defined as “the degree to which health care services for individuals and populations increase the likelihood of desired outcomes, and are consistent with current professional knowledge”[1] (1161). The Institute of Medicine has defined six key dimensions of a high-quality health care system that are included in the mnemonic, “STEEEP,” which includes: Safety, Timeliness, Efficiency, Equitable Access, Effectiveness and Patient-Centered.[2] There is extensive evidence that measurement of indicators by audit and feedback,[3-5] can improve quality of care. Measuring high-quality care is not easy, but can be achieved through the development and implementation of appropriate evidence-based quality of care indicators. Best practice recommendations grounded upon evidence-based indicators of quality care are successfully implemented in stroke,[6,7] cancer,[8] diabetes,[9] and cardiac care.[10] To date among neurological populations, significant efforts have been made to develop indicators for measuring and reporting the quality of stroke rehabilitation care[11,12] and are an explicit example of how the use of indicators can optimize neurorecovery and post-discharge patient outcomes. However, this has not been done in the field of spinal cord injury rehabilitation.

Spinal cord injury or disease (SCI/D) onset results in complex changes in an individual’s physical health, functional abilities, emotional well-being, and financial independence.[13] To add insult to injury, individuals with SCI/D often experience barriers to accessing the optimal tertiary rehabilitation care they require. The heterogeneity of SCI/D care in part results from individuals having diverse motor, sensory and autonomic impairments, and the individual’s social and emotional adjustment. In addition, care is delivered across acute, rehabilitation, and community settings, which make measuring quality extremely challenging. Recent research highlights the many inefficiencies in the way care is provided across the continuum from SCI onset to community integration.[14,15] Hagglund et al.[16] have noted that lack of transportation, physical and architectural barriers, and slow health care reform, further confound an individual’s community reintegration and care if they are living in rural regions. Current outcome tools fail to capture many Domains relevant to SCI/D rehabilitation care.[17] The lack of unanimity regarding a comprehensive standard set of quality of care measures is a specific and substantive barrier to advancing the quality of SCI/D rehabilitation care in Canada.

In the last five years, Accreditation Canada, a nationwide not-for-profit organization dedicated to working with patients, policy-makers and the public to improve the quality of health and social services, has developed specific SCI rehabilitation standards.[18] With the later advent of the Health Services Organization (HSO),[19] and the need for hospitals to demonstrate how they use data to inform SCI/D rehabilitation during the Accreditation process, the development of consensus-derived indicators could be a powerful tool for advancing rehabilitation care if the indicators are embedded in the Accreditation process.[20]

Indicators can represent key features of the quality improvement process and can be categorized as structure, process, or outcome. Structure indicators are measures of the properties or characteristics of the health care setting that are judged to be necessary resources to ensure the quality of care occurs.[21] For example, the attributes of the material resources such as facilities, equipment, and financial and human resources, as well as organizational resources are considered structure indicators. Process indicators examine what is actually done in providing and receiving elements of care judged critical to achieving the desired outcomes. Process indicators are key elements of ensuring quality.[7] Outcome indicators are widely used to evaluate the effects of the care provided to a given individual or population on their long-term health and function.[7,8] Typical outcome measures include morbidity, health status, health-related quality of life, and patient/family/provider satisfaction.[22] An ideal outcome indicator should capture the effect of care on the well-being of individual patients and populations. Outcome indicators are most familiar to SCI/D rehabilitation clinicians and researchers and have been studied extensively in the SCI/D literature.[23] The highest priority elements of SCI/D rehabilitation identified by consumers,[24] health care professionals, researchers[25] and health system leaders that should be measured have not been identified, until recently.[26] Routine measurement of indicators can: (1) determine the effects of health care on desired outcomes; (2) assess how well health care processes adhere to scientific evidence or professional consensus; and, (3) ensure the outcomes are consistent with patient preferences.[10]

The Spinal Cord Injury Rehabilitation Care High Performance Indicators (SCI-High Project) is a bold endeavor which aims to select, implement and evaluate indicators of quality rehabilitation care in Canada in the first 18 months after SCI/D rehabilitation admission, by 2020. This manuscript describes the project leadership structure and steps taken to develop a comprehensive framework of structure, process, and outcome indicators for 11 prioritized Domains of SCI/D rehabilitation. This manuscript is the second in a series of 14 SCI-High Project manuscripts. A detailed description of the process for domain prioritization[26] and the development of Domain-specific indicators[27-33] are described in related manuscripts in this issue.

Methods

Theoretical underpinnings of the rehabilitation framework and Domain identification

The SCI-High Project is a quality improvement initiative to advance the quality of rehabilitation care, that intuitively followed from a prior scoping review and environmental scan (E-Scan) of SCI/D rehabilitation services in Canada conducted between 2009 and 2012.[34,35] The E-Scan involved the collaborative efforts of 17 tertiary SCI/D rehabilitation programs and 224 SCI/D expert clinicians, scientists and policymakers. The theoretical underpinning of the E-scan was based on a hybrid International Classification of Functioning, Disability, and Health model[36] that was developed a priori to E-scan conduct.[35] The 37 Domains represented within the SCI Rehabilitation Framework[37] were used to guide the E-scan data collection, collation, and reporting. Application of the SCI Rehabilitation Framework is customized to the individual and their personal rehabilitation goals within specific rehabilitation Domains, which are typically derived from the individual’s impairments, health beliefs, and life situation.

The E-Scan contained 17 Domain-specific national report cards summarizing the current state of knowledge generation, clinical application, and policy, which highlighted the gaps between knowledge generation and clinical application in SCI/D rehabilitation.[34] The SCI-High Project Leaders chose to use these E-scan Domain-specific national report cards to identify Domains of rehabilitation with an opportunity to substantially advance care by 2020 through reduction of the gap between knowledge generation and clinical application. Recognizing that systematic development of quality indicators requires substantial time and resource, the SCI-High Project Leaders planned to develop quality indicators for ten Domains of SCI/D rehabilitation, necessitating a process to rank the Domains and narrow our focus from 17 Domains with the aforementioned “gaps” contained in rehabilitation framework, to a smaller number of prioritized Domains.

Rehabilitation Domain prioritization

The “Hanlon Method for Prioritizing Health Problems” developed by J.J. Hanlon,[38] is a well-respected technique to objectively rank health priorities based on defined priority criteria and feasibility factors. Detailed results of the SCI-High Team’s utilization of a modified Hanlon method[26] for stakeholder ranking of SCI/D rehabilitation Domains based on priority and feasibility scores are described in a related manuscript. Using the modified Hanlon method and UCLA/RAND consensus methods,[16,23] the following were the top 11 prioritized SCI/D rehabilitation Domains included: Cardiometabolic Health; Community Participation and Employment; Emotional Well-Being; Reaching, Grasping, and Manipulation; Self-Management; Sexual Health; Tissue Integrity, Urinary Tract Infection (UTI); Urohealth; Walking; and, Wheeled Mobility. Figure 1 displays a modification of the E-Scan SCI Rehabilitation Framework highlighting the prioritized Domains and their distribution within the International Classification of Functioning, Disability and Health framework. Among the 11 prioritized Domains: seven relate to body structure and function, one to activity, and three to participation.

Figure 1

Framework of rehabilitation goals.

Project Team structure and responsibilities

The SCI-High Project Team consists of two senior scientists, one scientist, one advanced practice leader, one project manager, two postdoctoral fellows, and one research analyst. The Team members have extensive experience in SCI/D rehabilitation care and evaluating healthcare quality. The SCI-High Project Leaders’ role (CC, MB, FF) was to guide project progression serving as an administrative, logistical, communication and evaluation hub during the indicator development, implementation, and evaluation processes. The Project Team’s activities were supported by the External Advisory Committee, Minimum Data Set Committee, the Toronto Rehab Spinal Cord Injury Rehabilitation Program (including program leaders, patients, and staff) and the Domain-specific Working Groups (Fig. 2). The project manager was responsible for communication within and among the Committees and Working Groups. The Toronto Rehabilitation Institute’s Lyndhurst Centre (TRI-LC) is home to the Spinal Cord Rehabilitation Program. This site houses Canada’s largest freestanding SCI/D rehabilitation facility, with 60 in-patient beds to serve 300 inpatients per year, and provide support for 20,000 outpatient tertiary clinic visits per year.

Figure 2

Overview of SCI-High Project Team structure which illustrates how the Project Team interacts with and coordinates activities across the Domain-specific Working Groups, External Advisory Committee, Toronto Rehab Spinal Cord Program, and the Data Set Committee.

External Advisory Committee

The External Advisory Committee members were responsible for participating in the ranking of rehabilitation Domains. The External Advisory Committee was comprised of 17 experts including people with lived experience, clinical experts in rehabilitation, scientists with expertise in processing and interpreting administrative data, and representatives from key stakeholder organizations including: Accreditation Canada (www.accreditation.ca), Canadian Spinal Research Organization (www.csro.com), SCI Canada (https://sci-can.ca), Ontario Neurotrauma Foundation (www.onf.org), and the Rick Hansen Institute (www.rickhanseninstitute.org).

Domain-specific Working Groups

Eleven Domain-specific Working Groups of content experts from across the country were formed to develop Domain-specific structure, process, and outcome indicators, aligned with the Domain Construct and Aim. There were 6–9 experts in each Working Group and the SCI-High Project Team members. In total, the Working Groups were comprised of 69 experts [14 physicians, 3 nurses, 5 occupational therapists, 10 physiotherapists, 2 social workers, 3 psychologists and 29 scientists (neuro-rehabilitation, kinesiology, psychology, etc.)], 3 individuals with lived experience, 4 administrative leaders and 2 patient and family educators. Some members contributed more than one area of expertise (e.g. clinician-administrators, advance practice leaders), and participated in more than one Domain-specific Working Group. Meetings were done via a series of 6–11 teleconferences. Indicator selection was based on scientific evidence, practical/clinical considerations, and the context for their decision making stipulated by the SCI-High Project Team. To ensure feasibility, the SCI-High Project Team stipulated that the outcome indicators in their entirety for all Domains should be completed within 60 min, ideally using patient self-report measures at the 18-month time point.

The SCI-High Project Team generated and synthesized documents and resources pre and post each teleconference. Outside of the teleconferences, Working Group members reviewed the prepared materials, shared resources and practice standards with one another, or conducted independent evaluations of the proposed indicators. The SCI-High Leaders actively discussed issues of concern with the Working Groups, clinicians and administrative leaders at the TRI-LC, and with others within the team’s infrastructure.

Toronto Rehab program leaders and staff

From the project outset, the leaders and staff members of the TRI-LC Spinal Cord Rehabilitation Program agreed to provide substantive in-kind support to enable rapid cycle evaluation of potential indicators, provide advice about the feasibility of implementation of proposed indicators and ensure clinical validity of the developed indicators (Fig. 3). As such, seven members of the TRI-LC leadership team chose to participate in the SCI-High Project in a variety of manners including as: members of the External Advisory Committee, Working Groups, local site teams that completed quality improvement Plan-Do-Study-Act[39] cycle evaluation of potential indicators and the November 2017 report out meeting. In addition, routine indicator pilot updates were provided to the TRI-LC Spinal Cord Rehabilitation Program’s Quality Council Committee, and at daily inpatient huddles and weekly outpatient huddles during active piloting (i.e. head-to-toe skin checks on inpatient unit or depression screening in outpatient setting).

Figure 3

Relationship between national Working Groups and local teams that piloted indicators and provided feedback iteratively between the two groups.

During the pilot processes, groups of local staff typically by profession were invited to participate in the indicator vetting and piloting processes (i.e. occupational therapists inform the reaching and grasping indicator, physical therapists discuss limitations and challenges with the Standing and Walking Assessment Tool - SWAT)[40] and provide advice about implementation barriers and facilitators of the proposed indicators. Local staff at TRI-LC were actively engaged in leading or participating in collaborative activities that helped to bootstrap the efforts of the Working Groups including but not limited to: the Knowledge Mobilization Network,[41] Rick Hansen Spinal Cord Injury Registry (RHSCIR 2.0),[42] Ontario Neurotrauma Foundation-funded Urology Summit,[43] the Electrical Stimulation Therapy Collaborative,[44] the SCI Solutions Alliance (alliance.sciontario.org/sci-solutions-alliance), a systematic review of Depression Screening tools,[45] and dissemination of the Standing and Walking Assessment Tool.[40] Leaders from TRI-LC and the SCI-High project leaders met with designates from HSO and Accreditation Canada to examine the overlap and synergies between the proposed indicators and current SCI Accreditation Standards to inform the activities of the Minimum Data Set Committee.

Minimum Data Set Committee

Early in SCI-High project implementation, the need for a core minimum dataset was recognized. The minimum data set will assist with determining whether an individual with SCI/D received the right treatment at the correct time. To help achieve this goal, a Minimum Data Set Committee was established, which was comprised of experts in SCI/D rehabilitation, health services and several data custodians including the RHSCIR team.[46] This Committee’s role was to select data elements from available patient administrative databases, registries, and clinical workload tools deemed critical for indicator inclusion to allow for national data collection. Key data elements considered were the individual’s age, sex/gender, postal code, relationship, and employment status, admission and discharge dates, impairment (International Standards for Neurological Classification of Spinal Cord Injury)[47] and Spinal Cord Independence Measure (SCIM)[48] disability scores which are sensitive to functional change among impairment subgroups. These variables are essential for identifying the numerator and denominator for indicator calculations and reporting.

A priori, the Project Leaders determined that data for all patients with SCI/D was to be collected from the time of rehabilitation admission to 18 months thereafter. The rationale for this decision was that: (1) the current case-finding strategies for patients with non-traumatic SCI[49] are not well established and an international classification of diseases codes for case finding was proposed for the first time in 2018;[50,51] (2) the Project Team wanted to allow for data collection beyond one year post-injury due to the current lack of community datasets[52] describing patient outcomes following rehab discharge; (3) recognizing that the median 78 day length-of-stay in Canada, and current length-of-stay targets[53,54] may limit the time for outcome indicator data collection; and (4) recognizing the high rates of divorce rates in the first 3 years after injury,[55] and the high rates of depression onset 6 months post discharge[56] and the role for poor coping skills to impede self-management[57] in the first few months following discharge from the inpatient rehabilitation setting. Figure 4 is a conceptual diagram illustrating how elements from the Canadian Institute for Health Information National Rehabilitation Reporting System (www.cihi.ca/en/national-rehabilitation-reporting-system-metadata), RHSCIR 3.0 (www.rickhanseninstitute.org) and local site health records are planned to be combined with indicator data to enable reporting in the fiscal year 2019–2020. The Project Leaders intent is to incrementally move toward transparent annual reporting of indicators and development of benchmarks of quality care.

Figure 4

The Minimum Data Set Committee will be comprised of data from the National Rehab Reporting System, RHSCIR 3.0 minimal data set, and site-specific medical records. This data will be merged with the Domain-specific indicators to form the SCI-High Project data set to evaluate the quality of SCI/D rehabilitation care.

Process for development and selection of Domain-specific indicators

The approach to the Working Group (see section 3.b) development of Domain-specific structure, process, and outcome indicators followed a modified, but substantially similar, approach to that described by Mainz (2003).[21] This included the following planning and development phases: (a) selecting the Domain to investigate (see Domain prioritization); (b) forming the national and local Working Groups (see section 3.b); (c) refining the Domain construct originally derived from the E-scan;[20] (d) identifying an aim linked to the Domain construct; (e) reviewing a summary of existing evidence and practice; (f) developing and interpreting a Driver diagram; (g) selecting indicators; and (h) pilot testing and refinement of the Domain-specific structure, process and outcome indicators. Although similar across groups, the processes as outlined above did not occur in a linear fashion, nor in the same order, across Working Groups as shown in Fig. 5.

Figure 5

Common processes summarizing the development of structure, process and outcome indicators for each of the 11 Domains of care.

Throughout these processes, a facilitated discussion occurred amongst the SCI-High Project Team (see section 3.a) and the Domain-specific Working Groups (see section “Domain-specific Working Groups”) in order to best utilize the relevant expertise of members, while ensuring the broader goals of the SCI-High Project were aligned across the 11 Domain Working Groups.

Working Group tools

Construct definitions and aim

The Domain-specific Working Groups (see section “Domain-specific Working Groups”) began their activities with review and discussion of the E-scan derived Domain constructs prior to creating a precise Construct definition. Developing and/or refining the Construct definition was a simple task for some groups, and very complex for others groups depending on the nature of the construct or the evolving scientific literature pertaining to a Construct (i.e. Cardiometabolic Health, Sexual Health, Community Participation), or the absence of a prior Construct. The UTI Working Group had a challenging task, as UTI was originally a component of the Urohealth construct and was added as a new Domain at the request of the study sponsor. Many groups defined a concept that was a broad idea and then defined an Aim that was the target for incremental changes in care by 2020. For example, Emotional Well-Being was broadly defined as “Emotional well-being is a state of mind in which the individual realizes his or her own abilities, is able to cope with the stresses of life, and can interact and participate in the community … ”; while the Aim was to initiate routine depression and anxiety screening as their near term goal.

Systematic searches and fishbone (Ishikawa) or Driver diagrams

Systematic searches were conducted by Project Team members (MO and MA) of Medline, Embase, and CINAHL, to identify factors influencing the desired rehabilitation outcomes, and to summarize the available clinimetric properties of published Domain-specific outcome assessment tools. The search results were synthesized and tabulated to facilitate comparison between outcomes, content, and face validity review. In addition, review of available published systematic reviews from SCIRE (scireproject.com) and current clinical practice guidelines relevant to the Canadian context were independently conducted for each Domain. These literature reviews were reviewed and refined by the Working Groups; thereby, providing an overview of the strength of evidence for each Domain.

Identify factors relevant to outcomes and display in Driver diagram

The Driver diagram (Ishikawa or fishbone diagram) is one of the seven basic tools of quality control. In this diagram the ‘fish head’ represents the main problem and potential causes of the problem usually derived from brainstorming sessions or literature search, are indicated in the ‘fish bones’.[58] The cause and effect or Driver diagrams are a graphic illustration which convey the relationships between the Domain and factors that influence the outcome. These diagrams can be used as a brainstorming tool to identify components of care or care processes that contribute to the outcome of interest.[59] All Working Groups used the standard categories in this cause and effect framework that included: Equipment, Process, Personnel, Materials, and Environment, although modifications to the factors were allowed to facilitate the development of a relevant picture for each Domain (see Fig. 6). The driver diagram and literature syntheses were used to inform the Working Groups selection of the most important and feasible drivers for the Domain-specific outcome.

Figure 6

Typical structure for Driver (Ishikawa) diagram. UEMS: upper-extremity motor score; LEMS: lower-extremity motor score; NLI: neurological level of injury; AIS: ASIA impairment scale; HR: heart rate; BP: blood pressure; * This part of the fishbone is common to all SCI-High project's fishbone diagrams (11 Domains).

Guided by the Domain-specific Driver diagrams and using the RAND/UCLA Appropriateness Method,[60] each Working Group was instructed to prioritize the key elements of care that should be measured. The Working Groups used several criteria for selecting the arms of the Driver diagram, including, the strength of the evidence for the factors’ importance, and the extent to which it was within the control of clinicians, At the outset, the Project Team planned to develop and implement a collection of one structure, process and outcome indicator for each Domain. Anticipating that development of a total of 33 indicators across 11 domains would be a substantial burden for clinical programs to implement and wanting to ensure indicator feasibility Working Groups were instructed to develop/select indicators requiring 6–10 min to administer at any time point admission, discharge or 18 months after admission.

Select structure indicator

Structure indicators are measures of the properties or characteristics of the health care setting that are judged to be necessary resources to ensure the quality of care.[61] For example, the attributes of the material resources such as facilities, equipment, financial and human resources, as well as organizational resources. Each Working Group was asked to select one structure indicator that could be administered annually. These indicators could be administered by accreditors or self-reported by programs to determine whether there were sufficient resources to provide high-quality care in each Domain.

Select process indicator

Process indicators examine what is actually done in providing and receiving elements of care judged critical to result in the desired outcomes. These are often overlooked as key elements of ensuring quality.[1] To select the process indicator, Working Groups were advised to consider measures that could be administered during rehabilitation typically for each patient and should be embedded in day-to-day care. The SCI-High Team stipulated that only one or maximum two process indicators could be selected for each Domain. Some measurements were dichotomous where the answers are “yes” or “no” such as “Completion of the daily head to toe skin check” as a process indicator for the Skin Integrity Domain, and some were more quantitative such as number of minutes of wheelchair skills training.

Select outcome indicator

Outcomes are widely used to evaluate the effects of the health care provided to a given individual or population on long-term health and function.[1,14] These indicators are most familiar to SCI/D clinicians and researchers as they have been extensively studied in the SCI/D literature. Typical outcome measures include morbidity, health status, health-related quality of life, and patient/ family/ provider satisfaction.[62] An ideal outcome indicator captures the effect of care on the well-being of patients and populations. To ensure feasibility, the SCI-High Project Team stipulated that the outcome indicators for all Domains should be completed within 60 min, ideally using self-report measures, by the individual patient at the 18-month timeframe. An intermediary outcome could be administered at rehabilitation discharge. For both outcome indicators, the experts were strongly encouraged to select published measures with established reliability and validity. The majority of the Domains will require indicator collection at rehabilitation admission, rehabilitation discharge, and 18-months post rehabilitation admission.

Pilot testing

Preliminary testing of the defined indicators and measurement tools helped to determine relative bias, precision, and reliability. Preliminary testing used quality improvement iterative cycles (i.e. Plan-Do-Study-Act cycles, or audit and feedback) that included using small samples of SCI/D individuals or staff (n = 5–10) and rehabilitation centers (n = 1–2), considering eligibility for each indicator.

Finalize definitions and technical aspects of all indicators

The results from these Plan-Do-Study-Act cycles of preliminary testing informed refinement of the constructs and indicators, for different environments to fulfill the SCI-High Project needs not previously identified by the Working Groups. Many operative definitions, diagnostic and treatment thresholds, and interpretations of tools were determined prior to SCI-High tool inclusion.

Stakeholder engagement

Stakeholders participated in the aforementioned prioritization of rehabilitation Domains, and during a formal National Report-Out meeting in November 2017, where stakeholders were presented potential indicators for the prioritized Domains and then asked to rate their importance and feasibility using an audience response system. Following the meeting, administrative leaders from 15 tertiary SCI/D rehabilitation centers participated in telephone interviews to confirm their rankings of Domain importance and feasibility from their organizational perspective. A summary report and executive summary from the National Report-Out meeting is available on the project website (www.sci-high.ca). The feedback from this consultation process was used to refine the indicators prior to publication of the Domain-specific manuscripts, and inform the strategic planning for the order of Domain deployment provincially and nationally.

Results

The aforementioned Working Group processes led to the development of 11 rehabilitation Domain Constructs and Working Group Aims (Table 1) and 52 indicators derived from the rehabilitation framework (see Fig. 1). The Cardiometabolic Health, Emotional Well-Being, Tissue Integrity, Sexual Health, UTI, Walking and/or Wheeled Mobility are detailed in separate manuscripts in this issue. These six Domains have been selected for provincial deployment at five sites in Ontario in 2019–2020, and the Cardiometabolic Health indicators will be piloted in Ontario and Quebec in this same time period. The relevant discussions, definitions, literature reviews, and final decisions about outcomes and indicators are outlined in subsequent manuscripts.

Table 1

SCI-High Domains, construct definitions, and aims.

Domain	Construct and aim
Cardiometabolic Health	Construct: The cardiovascular system is responsible for the transport of oxygen-rich blood and energy supply throughout the body and is controlled on a beat-by-beat basis by the autonomic nervous system. After SCI/D, disruption of autonomic control, changes in metabolic profile, and inactivity combine to alter the functioning of the cardiovascular system at rest, and especially during exercise. Optimal cardiovascular health can be maintained or achieved through establishing appropriate health behaviors (i.e. physical activity and diet) and health interventions (i.e. treated total cholesterol and blood glucose levels) to mitigate dysautonomia, cardiometabolic risk and reduce cardiovascular morbidity and mortality.
	Aim: To promote cardiometabolic health after SCI/D by initiating prevention strategies to mitigate cardiovascular disease risk through implementation of SCI/D-specific aerobic exercise and dyslipidemia recommendations.
Community Participation	Construct: Community participation is a broad construct defined by the World Health Organization as involvement in life situations. Within the International Classification of Functioning, Disability and Health, a life situation encompasses several areas, including an individuals’ ability to move around their home and community, bathe and dress themselves, engage in relationships with others, participate in social activities and civic life, in addition to employment, education, recreation and leisure activities.
	Aim: The aim of rehabilitation service delivery is to ensure individuals living with SCI/D are healthy, able and empowered to participate fully in the life situations they deem important.
Employment	Construct: Employment is a critical social and economic determinant to health and quality of life of Canadians with SCI/D, and can be an important outcome. Employment is defined as the performance of activities that enable involvement in vocational roles and is related to the generation of income or other benefits. Characteristics of the individual and their SCI/D, as well as their work, family, sociocultural, and political context can determine their ability to participate in employment.
	Aim: To assess the individual’s readiness and participation in employment following SCI/D and while transitioning into the community.
EmotionalWell-Being	Construct: Emotional well-being is a state of mind in which the individual realizes his or her own abilities, is able to cope with the stresses of life, and can interact and participate in the community. Enhanced emotional well-being is associated with improved mood states, self-esteem, meaningful roles and relationships, resilience, physical and mental health, economic stability and longevity.
	Aim: The aim is to improve the screening and management of depression and anxiety early post-injury or disease in order to maximize long-term rehabilitation outcomes.
Reaching, Grasping & Manipulation	Construct: Reaching, grasping and manipulation are the important components of upper limb function that allow individuals to use the sensorimotor integrity of their arm and hand to perform activities that meet their personal needs, and to explore and participate in their external environment in meaningful ways.
	Aim: To implement standardized testing of arm and hand function among individuals with tetraplegia in order to optimize neuro-recovery and functional ability.
Self-Management	Construct: Self-management relates to the tasks and skills that an individual must undertake to live well with a SCI/D. These tasks and skills include having or gaining the confidence and problem-solving abilities to deal with medical management, role management, and emotional management.
	Aim: The goal of self-management education is to empower the individual to manage their health and daily activities toward successful community integration.
Sexual Health	Construct: The World Health Organization (WHO) defines sexual health as a state of physical, emotional, mental and social well-being in relation to sexuality; it is not merely the absence of disease, dysfunction or infirmity. Sexual health requires a positive and respectful approach to sexuality and sexual relationships, as well as the possibility of having pleasurable and safe sexual experiences, free of coercion, discrimination and violence. For sexual health to be attained and maintained, the sexual rights of all persons must be respected, protected and fulfilled.[77] In this context, sexuality encompasses: sexual activity, gender identity, gender roles, sexual orientation, eroticism, pleasure, intimacy, contraception and reproduction. Sexual health rehabilitation requires a positive and respectful approach to sexuality, self-esteem, sexual relationships, and reproductive wishes, as well as the potential to have consensual, pleasurable and safe sexual experiences.
	Aim: To create a permissive environment among regulated healthcare professionals to enable open discussion and individual sexual health inquiry.
Tissue Integrity	Construct: Maintaining tissue integrity after SCI/D involves the prevention and management of pressure injury in areas of the body where sensation is diminished or absent.
	Aim: To reduce the overall incidence and severity of pressure injuries among individuals with SCI/D throughout their lifetime.
Urinary Tract Infection	Construct: Urinary Tract Infection refers to significant bacteriuria among individuals with SCI/D and Neurogenic Lower Urinary Tract Dysfunction with symptoms or signs of infection.
	Aim: To reduce inappropriate antibiotic prescription for Urinary Tract Infection in order to reduce the rising incidence of antibiotic resistance and the associated complications among individuals with chronic SCI/D living in the community
Urohealth	Construct: The goals of urohealth after SCI/D are: to achieve continence with socially acceptable, regular and timely bladder emptying; to avoid urinary stasis, high filling and voiding pressures; to prevent hydronephrosis and renal impairment; to reduce urinary frequency and urgency; and to prevent and treat complications such as urinary tract infections, stones, strictures and autonomic dysreflexia.
	Aim: To improve continence rates and advance the quality of life for persons with SCI/D through implementation of appropriate and timely urohealth care.
Walking	Construct: Walking is the ability to move forward over ground using voluntary lower limb movement while controlling one's balance in an upright posture. Safe and efficient walking allows individuals to move purposefully from place to place to explore and participate in their external environments, with or without the assistance of others and/or assistive technologies – in other words, to be independent moving about their home and community, and in their life activities.
	Aim: To maximize the recovery of walking for individuals following SCI/D.
Wheeled Mobility	Construct: Wheeled mobility refers to the skilled use of any personal device with wheels including power wheelchairs, and manual wheelchairs (with arm or foot propulsion), by individuals with physical impairments such as SCI/D, to allow full participation in daily life.
	Aim: To maximize community wheelchair mobility through the implementation of routine standardized wheelchair mobility assessments.

Discussion

To our knowledge, this is the first concerted effort to identify priority indicators of quality care for SCI/D rehabilitation that could be incorporated into usual clinical care. The process benefits from a unique engagement of clinicians, consumers, health care leaders and decision-makers throughout multiple steps in the process. The modified Hanlon methodology provided a rigorous and novel approach to identify and rank key rehabilitation Domains for indicator development from the SCI rehabilitation framework.

Each step in the process led to important developments. The Construct definition process assured Working Group members were clear on the core concept for each Domain. The identification of Aim, and review of related best practices, and existing measures ensured that available relevant outcomes, which were psychometrically sound, and had been tested in the SCI/D population, were reviewed for inclusion. Driver diagram development ensured that all relevant factors that influence outcomes were displayed in an easily comprehensible form and allowed for group consideration of near term rehabilitation priorities. The selection of a structure indicator ensured that the right characteristics and resources are present in the environment to provide high-quality interventions. The process indicators measured key processes that ultimately are critical for the outcomes. The outcome measures selected were chosen because of their feasibility to be administered at rehabilitation discharge and at 18 months after rehabilitation admission to look at the explanatory relationship between the structure and processes of care and how they result in the expected rehabilitation outcomes.

A key strength of the process was the prioritization of Domains for evaluation. Further narrowing of the set of indicators challenged the expert Working Groups to identify key drivers of outcomes and select related measures based on the Driver diagrams. There has been a number of prior initiatives to measure the outcomes of inpatient care.[63] Furthermore, given the dearth of evidence-based indicators of quality care, particularly in outpatient settings (i.e. lack of community/long-term follow-up),[52] there is a case to be made for the development, implementation, and evaluation of indicators to measure the quality of SCI/D rehabilitation care. An approach to advancing this initiative is to first determine indicators during the early post-discharge stage (i.e. initial 18 months). Currently, the trend of decreasing length-of-stay in tertiary SCI/D rehabilitation centers in North America[64] has led to individuals with SCI/D entering the community after a limited time for adjustment to the physical and psychological changes post-injury. Certainly, discharge from inpatient rehabilitation is a crucial transition in the care path for individuals with SCI/D, as well as their informal networks (e.g. family members/caregivers) as there is initial fear and anxiety in attempting to undertake and manage clinical, logistic and organizational issues.[65-67] Low-incidence conditions such as SCI do not create enough of a market to support development of specialized services in remote areas.[68] Factors threatening sustainability of health services in general include: populations of insufficient in size to support traditional models of care, difficulties recruiting and retaining an adequate health workforce, geographic isolation of individuals living in remote areas, and reliance on periodic visiting services due to insufficient access to rehabilitation centers. Furthermore, most family disintegration and divorce occurs during the early stages of injury when care demands are the highest[69-71] due to the challenges in adjusting to new physical functions, creating new normalcy, and maintaining relationships. It is apparent that, at the initial stage of injury (18 months post-discharge), both explicit and implicit needs and challenges arise from individuals with SCI/D, and there is a need to enhance care delivery across the continuum.[52]

Indicators development is an iterative process

Beyond the initial implementation of indicators, it must be clearly stated that these quality indicators are intended to be iterative in design, and will continually evolve with audit and feedback informing alterations in health service delivery and best practice over time. A benchmark of care is setting a threshold or expectation for quality care of a specific type, magnitude or timing thereby allowing the reviewer to compare their facility’s care to quality benchmarks (peer and national) to identify areas where they are performing well, and those that merit immediate attention and action. As the selected indicators provide relevant evaluative information about the health system, the SCI-High Project Team will conduct ongoing appraisal of the information provided and, as appropriate, refine the indicators and related benchmarks in order to enhance their overall effectiveness. The SCI-High Project Team intends to conduct the appraisal of the indicators at regular, pre-determined time points to harness the capabilities of indicators to create a rapidly responding framework that will create and reinforce a learning health system for individuals with SCI/D living in the community.

Moving from outcomes to health surveillance with indicators

Clinical research falls into the broader field of outcomes research, which sets out to develop knowledge that is useful in guiding health care decisions and optimizing care through preventative, diagnostic, prognostic, and therapeutic interventions. Outcomes research has been a focus of the rehabilitation community and can be categorized into three types: discovery science, application studies (i.e. development, use and evaluation of interventions), and surveillance.[72] To date, most of the focus of outcomes research has been on discovery science and application studies. As a result, health care decision-makers have neglected the benefits of health surveillance. Functioning as a learning health system goes beyond data collection and report generation, and involves a continuous monitoring of distributions or trends in indicator outcomes by thoroughly collecting, consolidating and evaluating all relevant information as proposed in the SCI-High Project.[73] The outcomes of interest in health surveillance are: health indicators; “a measure designed to summarize information about a given priority topic in population health or health system performance.”[64] The goal of health indicators is to “provide comparable and actionable information across different geographic, organizational or administrative boundaries and/or can track progress over time.”[64] This is differentiated from metrics collected in most application studies because indicators are comparable through adjustment or standardization; by placing the information into context.[64] The current impact of outcomes research on the quality of rehabilitation care for the SCI/D population is unknown. However, with few resources dedicated to health surveillance following rehabilitation discharge, there is no doubt that the benefits of this type of initiative are not being maximized. Thus, increased resources invested into the development of a more sophisticated rehabilitation surveillance system through the use of rehabilitation indicators has the potential to rapidly augment the quality of care and wellbeing of the SCI/D population.

Some limitations are worthy of discussion prior to generalizing the indicators. First, since there are numerous indicators for the 11 prioritized Domains, the Working Group attempted to choose the most feasible indicators; therefore, it is possible that the selected indicators may not be optimal for health care assessment. Second, aspects of care assessed by indicators must relate to enough patients to ensure data comparison is feasible. However, as SCI/D is a rare event,[74] feasibility testing and implication of these indicators for comparison would be difficult, due to small sample sizes and heterogeneity in impairment within the SCI/D population. Finally, we used a limited group of experts (a minimum of 12 and a maximum 17 members) for each Domain; thus, it is possible that a different group of experts might disagree on the Construct definition and selected indicators. To address this problem, we plan to measure the reliability and validity of the indicators for quality improvement over time.

In terms of next steps, these indicators alone are meaningless without appropriate interpretation and suitable actions taken to improve priority setting, policy formulation, and care processes. A sustained effort to enhance care processes and implement the indicators in a variety of sites will be required. The widespread implementation and collection of the indicators will allow the development of achievable benchmarks of care as similarly developed for stroke care.[51] Implementation tools for the indicators will also be critical. Some Domain-specific Working Groups developed flow diagrams for clinical implementation, for example, screening of depression and anxiety, or dyslipidemia. We also partnered with consumer advocacy groups: the Ontario SCI network and the Ontario SCI Alliance. These two groups will facilitate the creation of site-specific methodologies for indicator implementation, creation, and adoption of best clinical practices, and the approximation between the SCI-High goals and consumer’s priorities. We expect that the appropriate measurement of indicators will help to understand gaps in SCI/D rehabilitation care and identify strengths and weaknesses of each rehabilitation program. In this context, strengths can be shared between different sites with the long-term aim of increasing equity in healthcare. Therefore, timely and transparent reporting of indicators is key to success. There is also an opportunity to link the indicators with the HSO and Accreditation Canada for the development of a distinction program, creating an environment where indicator data collection becomes a routine part of day-to-day practices.

HSO and Accreditation Canada

The HSO is a group that is focused on constructing standards and assessment methodologies (e.g. instruments, benchmarking, training, and coaching) to improve health care quality.[69] Standards are developed using the highest quality evidence available and with the collaboration of policymakers, clinicians, health care providers, patients, patient family members, and community leaders.[68] These standards are then used by Accreditation Canada to form a basis for evaluating whether health and social service organizations meet a level of excellence in delivering services to individuals in need of care. Through their accreditation program, Accreditation Canada provides confirmation that an organization has achieved or exceeded the standards set by HSO.[67] The goal of providing accreditation is to promote an organizational culture of improvement that will result in improved safety and efficiency and outcomes of care.[70] Accreditation Canada has an established Trauma Distinction and Stroke Distinction program, for trauma and stroke organizations. Organizations with an “Accredited” status can achieve Distinction if they meet trauma/stroke specific standards, and meet the threshold for specific performance indicators, have care protocols in place, and are involved with a project or initiative that has enhanced the quality of services delivered.[75] The performance indicators of the Trauma Distinction Program are the foundation for the development of a revised Distinction program that will be more relevant to the SCI population.[65] The Rick Hansen Institute will facilitate this planned future work of HSO and Accreditation Canada through the Access to Care and Timing Workshop. The development of an SCI-specific Distinction Program is part of a national strategy to improve the care of Canadians with SCI/D.[65] The SCI-High Project is well positioned to inform a future distinction program based on the planned development of benchmarks for the prioritized Domains of SCI/D rehabilitation care.

Looking to the future

The indicators resulting from the described processes, are intended for implementation in a Canadian “universal payer” health system, and may not be appropriate in other SCI/D settings. The current changing demographics of the adult SCI/D population (rising incidence of non-traumatic injury) and the associated increased service demands of individuals with incomplete injuries and a good prognosis for neurological recovery, are likely to challenge and potentially overwhelm rehabilitation resources in our resource-constrained health system, with requests to provide (1) an adequate volume of therapy within appropriate timeframes, and (2) services for those aging with secondary health conditions. The economic resource requirements for the developed indicators has yet to be developed. The developed indicators are intended to capture changes in processes of care and the health outcomes of patients’ with SCI/D, as they transition to community living. Although the incidence of SCI is relatively low: in 2010 only 3,675 new cases occurred in Canada (estimated prevalence of 85,556), with 40% of this population residing in the province of Ontario.[74] However, the mean five-year (2005–2010) direct health system costs of incident SCI cases in Ontario was $213,800 CAD, and the direct lifetime costs for SCI onset at age 25, range from $2.1 to 5.4 million USD depending on injury severity.[76] Thus, SCI/D is a complex, relatively rare condition, with substantial adverse personal, health system and economic impacts for which implementation of quality indicators are urgently needed to advance accountability, transparency and rehabilitation care for stakeholders through implementation of audit and feedback relative to evolving health system structure, process and outcome benchmarks in the next decade.

Conclusion

The results from this process can be used for different purposes, including assessing the quality of care provided to individuals and populations with SCI/D within and across health care settings and monitoring the effects of changes on patients’ health status. Using these indicators for evaluating rehabilitation care can ensure continuous quality improvement (i.e. benchmarking) and ultimately lead to evidence-based advances in SCI/D rehabilitation practice and improvements in healthcare equity.