An International Standard Set of Patient-Centered Outcome Measures After Stroke.

BACKGROUND AND
PURPOSE: Value-based health care aims to bring together patients and health systems to maximize the ratio of quality over cost. To enable assessment of healthcare value in stroke management, an international standard set of patient-centered stroke outcome measures was defined for use in a variety of healthcare settings.
METHODS: A modified Delphi process was implemented with an international expert panel representing patients, advocates, and clinical specialists in stroke outcomes, stroke registers, global health, epidemiology, and rehabilitation to reach consensus on the preferred outcome measures, included populations, and baseline risk adjustment variables.
RESULTS: Patients presenting to a hospital with ischemic stroke or intracerebral hemorrhage were selected as the target population for these recommendations, with the inclusion of transient ischemic attacks optional. Outcome categories recommended for assessment were survival and disease control, acute complications, and patient-reported outcomes. Patient-reported outcomes proposed for assessment at 90 days were pain, mood, feeding, selfcare, mobility, communication, cognitive functioning, social participation, ability to return to usual activities, and health-related quality of life, with mobility, feeding, selfcare, and communication also collected at discharge. One instrument was able to collect most patient-reported subdomains (9/16, 56%). Minimum data collection for risk adjustment included patient demographics, premorbid functioning, stroke type and severity, vascular and systemic risk factors, and specific treatment/care-related factors.
CONCLUSIONS: A consensus stroke measure Standard Set was developed as a simple, pragmatic method to increase the value of stroke care. The set should be validated in practice when used for monitoring and comparisons across different care settings.

The global stroke epidemic continues to increase, with a disproportionate burden present and increasing among low-income countries.[1] There is an urgent need for better strategies to deliver efficient and effective care, while reducing disparities between countries, because of the societal burden posed by stroke. A proposed strategy for improving quality of care involves measuring the value-based health care given to patients.[2] In this framework, value is defined as the total benefit gained by a patient relative to the cost of obtaining that benefit (ie, health outcomes divided by the cost to achieve those outcomes).[3] Defining condition-specific measurable outcomes that are meaningful to patients is critical to this equation. Outcomes can be broken into the broad categories of survival, disease control, complications of treatment, and long-term quality of life. The importance of each can vary from patient to patient.[4] Despite existing efforts in the area of patient-reported outcome measures (PROMs) to quantify stroke outcomes accurately using validated instruments, there is significant variability across instruments and domains, and no agreement about which critical measures should be routinely captured.[5-8] To define a set of global standards for measuring outcomes that matter most to stroke patients, an international expert panel was assembled representing patients, advocates, and clinician experts in stroke outcomes, registers, global health, epidemiology, and rehabilitation.

Methods

Assembling the Expert Panel

The primary aim of this expert consensus group was to define the Stroke Standard Set, a minimum set of outcomes and risk adjustment variables that are highest priority to collect for all patients hospitalized with stroke and designed to be able to be measured in any country within an existing register or as a free-standing set. The working group was created and coordinated by the International Consortium for Health Outcomes Measurement (ICHOM, http://www.ICHOM.org), a nonprofit organization focused on the development of standard sets of outcomes and related risk factors for individual medical conditions. An executive leadership team was composed of a volunteer senior stroke outcomes expert (L.H.S.), a salaried ICHOM project manager (S.S.), and a neurology trainee ICHOM research fellow (J.S.). The executive team identified and invited international expert members to participate with the aim of establishing a geographically diverse group covering a broad range of stroke specialties. Members represented stroke patients, specialties from all phases of stroke care, and major international professional societies, stroke registers, and centers (Table I in the online-only Data Supplement).

Process

The working group defined the Stroke Standard Set by executing a structured consensus-driven modified Delphi method by the way of frequent iterative teleconferences, videoconferences, and online surveys to develop proposals based on evidence and expert opinion.[9] The group convened 6 times from June 2014 to January 2015. Each teleconference or videoconference was followed by formal web surveys to gather feedback and make decisions on the proposals discussed during the conference calls. In decision-making, the group used a two-thirds majority vote for determining which variables or measures should be included in the Standard Set. All voting results were reviewed with the group at each call, and there was unanimous agreement on the final selected set, detailed data elements, and abstraction instructions provided.

Core Principles

In reviewing candidate measures, the panel’s decision to include or exclude elements from the Standard Set was governed by a set of guiding principles, which were unanimously adopted by the panel at the onset of the process. These included emphasizing or prioritizing (1) pragmatism over idealism; (2) completeness in data collection over breadth of areas surveyed; (3) measures that can also be collected through retrospective abstraction; (4) instruments that are perpetually freely available and ideally with a digital platform; (5) instruments made of modular subunits that permit recombination of elements; and (6) measures robust to comparison in both low- and high-income countries and with available cost utility values to calculate measures of cost-effectiveness.

Results

Condition Scope

The Standard Set was developed for evaluation of adult patients (age ≥18 years) presenting to a hospital with ischemic stroke (IS) or intracerebral hemorrhage (ICH). This scope of IS and ICH covers >90% of the global burden of incident stroke with high diagnostic reliability based on epidemiological studies performed worldwide at varied proportions between countries.[10,11] Inclusion of both IS and ICH is needed to create a global model for stroke and allows a greater focus on uniformly capturing stroke severity (an essential predictor of outcome).

Subarachnoid hemorrhage (SAH) was excluded from case entry because of the substantially different course of treatment and outcomes in patients with SAH. Although SAH is more likely to be distinguished from IS or ICH based on clinical presentation and age, differentiating between ISH and ICH in settings where imaging technologies are not available would require inferential (rather than definite) classification based on proxy variables. Given heterogeneity in SAH case ascertainment, as well as the markedly different nature of SAH management, and impact on outcomes, not including SAH in the initial Standard Set was favored. Future working groups will define outcomes and relevant risk factors for SAH and childhood stroke.

Inclusion of transient ischemic attack (TIA) or of patients with IS or ICH, who are evaluated but not hospitalized, is recommended to be optional. The recommendation for optional inclusion of TIA or nonhospitalized patients reflects a suggested balance between avoiding the exclusion of an important patient group and maximizing accuracy and reliability of data collection, given the challenges of variation in imaging and pathogenic workup that produces low inter-rater reliability in TIA case ascertainment most pronounced in resource limited settings. In lieu of this recommendation, the Standard Set includes a report of symptom duration to probabilistically assist in distinguishing case subtypes.

Treatments

The treatment approaches collected in the Standard Set are restricted to thrombolysis, endovascular thrombectomy, and hemicraniectomy, the only procedures for which evidence convincingly shows a large impact on mortality and disability. Given the wide variation in definition and function of stroke units worldwide, admission to a stroke unit is not collected in the Standard Set. It is expected and encouraged that many sites will continue to collect existing additional process and performance measures, but this minimum data set was constructed to be pragmatic and feasible worldwide.

Outcome Domains and Measures

The full Delphi voting results of the outcome domains were collected and shared with all group members (Table II in the online-only Data Supplement).

Survival and Disease Control

We selected overall survival and recurrence of disease as core measures of treatment effectiveness and disease prevention for the Standard Set. Effectiveness of smoking cessation was also selected given the attributable risk of future vascular events associated with smoking and its disproportionate impact in low-income countries. Survival is measured as all-cause mortality, calculated from clinical and administrative data sources, and collected at 7 days from index hospital admission for stroke or at discharge, whichever is first, and then again at 90 to 120 days and 1 year after admission for the index event (Figure). The group recommended tracking survival annually ≤5 years largely via administrative data sources. All-cause mortality is preferred over cardiovascular mortality because of poor reliability in the classification of cause of death. It is acknowledged that it may be challenging to collect this information (eg, 19% of countries do not have a system in place for reporting cause-specific mortality)[12]; however, it is key to understanding the long-term outcomes of stroke patients whose care usually transitions back to general primary care after the first year post stroke.[13]

Timeline of data collection.

Recurrence of disease (stroke and TIA) is defined as a self or proxy report of new stroke symptoms assessed at 90 days. Although some health systems are capable of accurately tracking new diagnoses of stroke, there is still variability in the accessibility of clinical data and the reliability of diagnoses. Thus, there may be a higher degree of consistency and reliability through a standardized self or proxy report of recurrent stroke or TIA.

Acute Complications

Because the treatments for stroke and their sequelae are heterogeneous and exert their effects via long-term disability measurable through patient-reported functional outcomes, the group narrowed the focus of measuring treatment-related complications to symptomatic ICH after thrombolysis. This information would be abstracted from clinical data at discharge or at 7 days after index hospital admission. Data on complications after other treatments, such as carotid surgery, are not included in the Standard Set.

Patient-Reported Health Status

Stroke survivors may have symptoms that are not typically captured by more conventional PROMs or clinician-reported outcomes. Therefore, individual subdomains of fatigue, depression, and anxiety were collected rather than a single composite domain. This resulted in elements and subdomains for cognitive and psychiatric functioning, motor functioning, nonmotor functioning, social functioning, general health status, and health-related quality of life (Table).

Data on mobility, feeding, toileting, dressing, and ability to communicate serve as a minimum data set at discharge because of its practical availability from clinical data or provider reporting, and its wide applicability across different regions in the world where the use of postacute care facilities (eg, rehabilitation, chronic nursing care, or home care) is highly variable. The full assessment of all PROMs occurs as part of a single focused assessment at 90 to 120 days after index admission.

PROMs have gradually been integrated with more traditional process-oriented metrics of stroke care.[14,15] To find the optimal balance between precision and pragmatism, the group sought to select a single well-validated instrument that would be able to address the multidimensional outcome domains in stroke with minimal floor or ceiling effects common to many instruments, while maximizing the likelihood of reliable longitudinal data collection. Many options and their various permutations were considered including promising tools still in development, such as Neuro-QOL.[16] In the interest of identifying a standardized tool that can be used at any center in any country, a core PROMs instrument was recommended: the Patient Reported Outcomes Measurement Information System 10-Question Short Form (PROMIS SF v1.1 Global Health, equivalent to the PROMIS global items and domain item banks).[17,18] The PROMIS-10 is available through the National Institutes of Health (NIH) Assessment Center as analog (printed paper algorithms) or a digital platform (web-enabled or offline). There are no associated fees for its use, and it is available in multiple languages. The PROMIS-10 scores can be converted to other scores of established instruments for comparison: the Short Form 36-Question Health Survey, modified Rankin Scale (mRS), Barthel Index, and the widely used EuroQOL-5 Dimension Questionnaire that also allows for calculation of quality-adjusted life-years.[17,19-21]

The PROMIS-10 covers the majority of the outcome domains considered most important by the expert panel. Although single PROMIS items are being developed for the remaining subdomains (ie, feeding, communication, and selfcare and grooming), the group chose to use well-established single items from existing stroke registers for these domains (Table). It is anticipated that there may be some degree of overlap between elements, which will be revised in future refinements after feasibility and early user acceptance testing. It was decided to include overlap with established data collection efforts as well by including the mRS at the 90-day assessment. The mRS is commonly used and can be obtained with the simplified mRS Questionnaire (smRSq), which is validated, easy to administer, translatable into multiple languages, and can be performed by raters with diverse professional experiences and skill levels with substantial reliability.[22]

Risk Adjustment Variables

The minimum set of risk adjustment variables that can be obtained in many countries included stroke type and severity (stroke type, stroke severity, and duration of symptoms); general patient demographics (age, sex, race/ethnicity, prestroke functional status, whether living alone, and living location); vascular and systemic risk factors (prior stroke, prior TIA, prior myocardial infarction, coronary artery disease, atrial fibrillation, diabetes mellitus, hypertension, hyperlipidemia, smoking status, and alcohol use); and treatment/care-related factors (diagnostic evaluation, length of stay, comfort care, rehabilitation, and discharge destination; Table III in the online-only Data Supplement). The risk factors selected reflect an emphasis on covariates that are most strongly associated with disease outcomes and are feasible for collection. The group favored a single set of measures and variables for both IS and ICH rather than a different set of variables for each disease for ease of use and for appropriateness in care settings, where IS and ICH cannot be so easily distinguished because of the lack of neuroimaging. Most of these risk adjustment variables can be obtained by patient or proxy report with guidance from supporting clinical or administrative data (eg, history of diabetes mellitus, hypertension, or coronary artery disease). The Charlson Comorbidity Index can be estimated from administrative data for sites that have this capacity.[23,24] Of note, the categories and specificity of race/ethnicity differ by country, as do the means to capture them, and their collection and reporting may also be constrained by regulation. Stroke subtype is obtained from clinical information and confirmed by imaging, with the understanding that certain resource limited settings may not have access to specialty care or diagnostic imaging studies. The NIH Stroke Scale (NIHSS) was considered the best option for assessment of stroke severity given its wide acceptance in global clinical practice.[13] The group decided to focus on 3 single patient–reported items (mobility, toileting, and dressing) to obtain a readily available and pragmatic measure of prestroke functional status because of the challenges inherent in the retrospective use of more in-depth instruments. The risk adjustment variables are collected at the time of hospital presentation for the index event. Despite the minimum set of risk adjustment variables recommended, it is expected and encouraged that many sites will use the opportunity to begin or continue collecting additional data elements and measures, such as the use of anticoagulants before admission. It is expected that with implementation, additional candidate variables will be identified and included in future versions.

Data Collection

To compare data easily between providers, institutions, and populations requires data collection methods that minimize variation. For example, using the same sources of data can improve consistency regardless of the inherent differences that underlie the details of data collection. The recommended sources outlined in Table include patient- or proxy-reported sources, clinical data abstraction or provider report, and administrative data (eg, death registers or claims data). An international standard set also allows for opportunities to pilot inexpensive forms of digital data collection, such as automated smartphone surveys or remote follow-up visits using telehealth as opposed to the traditional and expensive method of in-person or dedicated phone-based assessments.[25]

For every measure, the group also recommended that the data source and the response rate of patient-reported measures be tracked. Although the Standard Set is amenable to be included as part of an existing register, the group recommended that data could be tracked locally by centers and providers who do not have access to a national register or central data repository because of regulatory or technical challenges.

Discussion

Strengths of the Standard Set

An established set of standard data collection items creates an opportunity to increase patient value by improving the reliability and consistency of data about the quality of stroke care.[26] Our aim acknowledges the challenges and certain uncertainty that confront patients during the acute and subacute stroke periods.[27] Although randomized clinical trials are the mainstay of comparing outcomes between treatments, registers built into routine clinical practice are an incontrovertible complement in determining the degree to which a treatment or service is safe, effective, patient-centered, timely, efficient, and equitable as outlined in the Institute of Medicine’s report on quality in healthcare.[28] To ensure high rates of data completeness, which is critical for valid comparisons across settings, the Standard Set favors simplicity and pragmatism of data collection over complexity and highly detailed data specification. Functional outcome scales commonly used in clinical trials, such as the Barthel Index, Short Form-36, and Stroke Specific Quality of Life were not selected because many of their relevant domains are covered adequately by the simplified, open-access, and versatile PROMIS-10 platform.

Implementation

A complete data specification and collection manual is available on the ICHOM website (http://www.ichom.org/medical-conditions/stroke), which describes each domain and subdomain, their definitions and corresponding measures, and their potential data sources. In the next phase of work, ICHOM and affiliated organizations will begin pilot collection, develop a standardized collection platform, and compare data quality and outcomes from the Standard Set in various settings.

Limitations

The utility of the Standard Set in practice is undetermined, and it is derived from expert consensus rather than high levels of evidence. To create a worldwide standard that could be applied in both high- and low-income countries, many elements familiar to registers in high-income countries are absent. There is limited ability to account for variability, including availability of mechanical thrombolysis, detailed ascertainment of stroke cause, and other treatment variations. Many items have been validated in single registers; however, the value of the Standard Set will only be discernable if it is implemented and field tested with rigorous evaluation criteria. This includes feasibility testing with proxy responses for patients that are unable to respond for themselves, and determination of data elements that can better address the challenges in capturing information about cognitive impairment from self-assessments. If the Standard Set becomes adopted as an acceptable measure set, then there may be a prospective role for involving other larger scale stakeholders. The group also recognizes that to support greater transparency in quality of care for stroke patients, continued investments in register infrastructure and health information technology are required to overcome the financial and logistical barriers in collecting and tracking this type of outcome data. Given the growing ubiquity of smartphones globally, data collection through simple, web-based, and mobile phone strategies is a potential opportunity to increase feasibility.

There exist wide variations in stroke outcomes based on institutional and provider differences, suggesting substantial room for improvement in the implementation and development of global stroke services.[29] The global adoption of the Standard Set will assist healthcare providers and policy makers in their efforts to improve stroke systems of care and improve equitable access to care. This in turn creates powerful opportunities to inform and learn from each other through meaningful comparison of outcomes and facilitates the broader implementation of essential stroke services.

Conclusions

The stroke outcomes working group has defined a minimum recommended set of consensus patient-centered outcomes for collection in all adults with new stroke that can be implemented in a variety of healthcare settings. The use of the Standard Set will help inform healthcare providers in the delivery of effective, equitable, patient-centered, value-based stroke care worldwide.

Sources of Funding

This work was developed by the Schwamm Marriott Clinical Care Research Fellowship Program and the International Consortium for Health Outcomes Measurement, a nonprofit organization that received sponsorship from the American Heart Association/American Stroke Association, USA, in an unrestricted grant to produce these recommendations. Neither International Consortium for Health Outcomes Measurement nor the funders of the Stroke Standard Set had any editorial control over the submitted publication.

Disclosures

Dr Schwamm reports serving as a consultant to YaleCORE, a Centers for Medicare and Medicaid Services measure vendor in defining improved risk-adjusted stroke performance measures for public reporting; to the Massachusetts Department of Public Health, as a Stroke systems consultant for the Paul Coverdell Acute Stroke Registry Grant; and as a volunteer for the AHA, Chair of the Healthcare Accreditation Science Committee and the AHA Get With The Guidelines–Stroke program’s Stroke Clinical Workgroup. For additional coauthor registry affiliations, refer to Table I in the online-only Data Supplement. The other authors report no conflicts. The findings and conclusions in this report do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Table.

Summary of Outcomes and Measures Included in the ICHOM Standard Set for Stroke