Health-weighted Composite Quality Metrics Offer Promise to Improve Health Outcomes in a Learning Health System.

Health system leaders sometimes adopt quality metrics without robust supporting evidence of improvements in quality and/or quantity of life, which may impair rather than facilitate improved health outcomes. In brief, there is now no easy way to measure how much "health" is conferred by a health system. However, we argue that this goal is achievable. Health-weighted composite quality metrics have the potential to measure "health" by synthesizing individual evidence-based quality metrics into a summary measure, utilizing relative weightings that reflect the relative amount of health benefit conferred by each constituent quality metric. Previously, it has been challenging to create health-weighted composite quality metrics because of methodological and data limitations. However, advances in health information technology and mathematical modeling of disease progression promise to help mitigate these challenges by making patient-level data (eg, from the electronic health record and mobile health (mHealth) more accessible and more actionable for use. Accordingly, it may now be possible to use health information technology to calculate and track a health-weighted composite quality metric for each patient that reflects the health benefit conferred to that patient by the health system. These health-weighted composite quality metrics can be employed for a multitude of important aims that improve health outcomes, including quality evaluation, population health maximization, health disparity attenuation, panel management, resource allocation, and personalization of care. We describe the necessary attributes, the possible uses, and the likely limitations and challenges of health-weighted composite quality metrics using patient-level health data.

Introduction

Quality metrics are often viewed as necessary tools to ensure that health systems realize their potential for delivering health benefit, yet they measure this goal indirectly rather than directly. In brief, there is no easy way to measure how much “health” is conferred by a health system.1 When health systems focus on quality metrics that have substantial clinical benefit and robust supporting evidence, such as prescribing aspirin in the secondary prevention of cardiovascular events, substantial health care benefit may ensue.2 However, health systems sometimes adopt quality metrics without robust supporting evidence or clear links to substantial improvements in quality and/or quantity of life, such as use of spirometry testing in the assessment and diagnosis of chronic obstructive pulmonary disease.3 Even learning health systems have limited bandwidth and resources for improving quality, and therefore any particular quality improvement effort has an opportunity cost: greater attention towards a quality metric with less health benefit may inadvertently divert attention away from a quality metric with greater health benefit, thereby preventing that health system from delivering the most health with its available resources.

Previously, it has been challenging to create a health-weighted composite quality metric (eg, the total magnitude of health improvement attributable to a health system’s activities) because it requires sophisticated analytic methods and extensive patient-level data. However, advances in health information technology (eg, adoption of electronic health records) and validation of new mathematical models of disease may mitigate these challenges in the future.6 It may now be possible for learning health systems to use health information technology more innovatively: to calculate and track a health-weighted composite quality metric for each patient that reflects the attributable health benefit conferred to that patient by the health system. These health-weighted composite metrics could then be used as tools by learning health systems to improve outcomes and care, enhance population health and to mitigate health disparities.7

Health-weighted composite quality metrics can strive to measure “health” by synthesizing individual evidence-based quality metrics into a summary measure, utilizing relative weightings that reflect the relative amount of health benefit conferred by each constituent quality metric and, ideally, taking into account each patient’s individualized risk factors and medical history. Mathematical models of multifactorial risk are increasingly being used in the clinical setting. For example, the recently published model of Taksler and Braithwaite4 measures how much additional health would be conferred to a particular patient by improvements in preventive care, and can be employed at the patient, clinician, or health system level.

Eddy (2012), who developed the widely-recognized Archimedes model, has also proposed a framework for using any valid mathematical model to derive a “Global Outcomes Score” to measure a state of risk for adverse clinical outcomes.5 While existing models may differ in scope or methodology, they are generally motivated by the shared goal of more comprehensive and meaningful health status assessment. To help clarify how these ambitious goals might be pursued to answer key questions raised by a variety of decision makers, including health system leaders, clinicians, patients, or others, we first will describe the necessary attributes of health-weighted composite quality metrics; second, their possible uses; and third, their likely limitations and challenges.

Desirable attributes of health-weighted composite quality metrics

Because health services may improve health across a wide range of disease categories (eg, infectious diseases, cancer, cardiovascular disease, or injury prevention), it is important for a health-weighted composite quality metric to be generic (that is, reflecting performance across a range of disease domains) rather than disease-specific (reflecting performance in one particular disease domain). Indeed, health-weighted composite quality metrics should represent a broad spectrum of diseases that comprise, in aggregate, the bulk of preventable morbidity and mortality burden of a population. Furthermore, health-weighted composite health metrics can facilitate personalized healthcare by considering variations in the relative importance of clinical guidelines that may occur from patient to patient because of varying risk factors, medical histories, and other relevant heterogeneous characteristics, as in the model of Taksler and Braithwaite.4

In addition, because health benefits may involve improvement in morbidity as well as mortality, it is desirable—although not essential—for a health-weighted composite quality metric to account for quality as well as quantity of life. Finally, to ensure a sufficiently tight link between the health-weighted composite quality metric and the health of the population to which it is applied, the metric should be validated against clinical and epidemiological measures from target populations (eg, disease incidence, disease-specific mortality, all-cause mortality, or quality of life), should be able to consider long-term as well as short-term disease reduction strategies, and should predict outcomes that matter to patients. Many different types of metrics satisfy these criteria for health-weighted composite quality metrics and are therefore plausible choices (Table 1).

Table 1:

Examples of alternative health-weighted composite quality metrics

Construct	Metric	Lay explanation	Example
Life expectancy or health-adjusted life expectancy (for example, quality-adjusted life expectancy or disability-adjusted life expectancy)	Life-expectancy or health-adjusted life expectancy	How long you are expected to live, on average (this is not a guarantee, of course)	15 more years
	Biological age (the chronological age that would typically correspond to the patient’s life expectancy or health-adjusted life expectancy given his/her sex)	Your age that reflects the condition of your body and how well you take care of it	67 years-old
	Health compared to others (percentile compared to life expectancy or health-adjusted life expectancy of others of same age and sex)	Your health “grade” compared to others your age	You are healthier than 80% of men your age
	Fraction of personal best health (life expectancy with current preventive guideline compliance minus life expectancy without preventive guideline compliance, divided by the difference between life expectancy with versus without complete preventive guideline compliance. Health-adjusted life expectancy can be substituted for life expectancy in the above formula.	How much of what you can do to improve your health that you are actually doing	You are getting 60% of the health benefit you would be getting if you were doing everything possible to improve your health

It is important for a health-weighted composite quality metric to be generic rather than disease specific, to represent a broad spectrum of diseases that comprise the bulk of preventable morbidity and mortality burden of a population, and to have face validity for patients, consumers, and health system leaders

Uses of health-weighted composite quality metrics

Health-weighted composite quality metrics can be employed by learning health systems for a multitude of important aims that are united by the goal of conferring the most “health,” including quality evaluation, panel management, health disparity attenuation, personalization of care, and resource allocation. The following short case examples are provided to illustrate the potential utility of health-weighted composite quality metrics for a range of decision makers.

Comparing different health plans

Suppose that a region is served by three different health plans, A, B, and C, each of which are sufficiently integrated and have sufficient HIT capabilities to employ a particular health-weighted composite quality metric (Figure 1). This metric could be used by employers, payers, consumers and the health plans themselves to compare and contrast (1) the quantity of health improvement realized by enrollees in each health plan, (2) the quantity of additional health improvement that is theoretically possible yet is not being realized by enrollees in each health plan, and (3) the extent to which the amount of health improvement realized by enrollees has improved compared to the previous year (Figure 1). In this hypothetical example, Health plan A serves a lower socioeconomic status population, with higher prevalences of smoking, obesity, diabetes, and hypertension. Health plans B and C serve higher socioeconomic status populations, with lower prevalences of these risk factors. Because patients in Health plan A have so many modifiable risk factors for diseases, they have large potential health benefit from their health plan (shown by the height of the bar in Figure 1). However, because many patients do not comply with evidence-based clinical guidelines, they are only realizing 40% of the health benefit that would be possible if they received all services recommended by applicable evidence-based clinical guidelines. Nonetheless, even while realizing only 40% of possible benefit, they are receiving greater quantities of health benefit than are enrollees of Health plans B and C. In addition, the magnitude of benefit received by enrollees in Health plan A has improved substantially over the last year, unlike enrollees in Health plan C. Based on these insights, Health plan A could be recognized by employers, payers, and consumers for being a high-performing health system, for conferring more health benefit than any other health plan in the region, and for making substantial improvements in the amount of health benefit conferred, while also being nudged constructively because of the large additional gains that would be possible from continued improvements. In the macro, within a region, employers, payers, and purchasers participating in a review of their data, particularly if willing to change their benefits structure as a result, could be considered active participants in a learning heath system.

Figure 1:

Benefits from compliance with evidence-based clinical guidelines

Suppose that a region is served by three different health plans, A, B, and C, each of which employ a health-weighted composite quality metric (such as High-quality life-years shown in Figure 1, but any metric in Table 1 could be used). These results could be used by employers, payers, consumers and the health plans themselves to compare and contrast (1) the quantity of health improvement realized by enrollees in each health plan, (2) the quantity of additional health improvement that is theoretically possible yet is not being realized by enrollees in each health plan, and (3) the extent to which the quantity of health improvement realized by enrollees has improved compared to the previous year.

Estimating population health impact

Health plans A, B, and C serve a municipality with 5 million persons. Health plan A takes care of 2 million lives, whereas Health plans B and C each take care of 1 million lives. The health-weighted composite quality metric indicates that people in Health plan A could potentially increase their health by 0.05 high-quality life-years per person per year if their adherence with evidence-based preventive guidelines were perfect. Here, “high quality” is defined as time spent in near-perfect health, as indicated by a high score on a preference-weighted health-related quality of life measure such as the EQ-5D. Public health authorities could note that if these currently unrealized gains were to accrue, the magnitude of population health improvement would endow the municipality with an additional 75,000 high-quality life-years. Alternatively, the health-weighted composite quality metric indicates that people in Health plan B and Health plan C could potentially increase their health by 0.02 high-quality life-years per person per year if their adherence with evidence-based preventive guidelines were perfect. Noting that the potential population health improvement would be nearly 4 times as great by focusing on Health plan A than focusing on either of the other health plans, public health authorities may decide to encourage new initiatives that synergize with the learning health system approaches undertaken by Health plan A, such as launching community-based preventive care initiatives. Indeed, these estimates could be compared or aligned with estimates for high-quality health years gained by non-medical interventions.

Attenuating health disparities

Because a health-weighted composite quality metric could be calculated individually for each patient in each health plan, it would be possible to stratify health plan results by subgroups of patients that have been historically impacted by health disparities. For example, stratifying health-weighted composite quality metrics for African Americans versus non-African Americans could reveal dramatic disparities in health benefit across all three health plans (Figure 2). However, in this example we see that Health plan A has greater disparities in health benefit between African Americans and non-African Americans compared to Health plans B and C. Even if we wonder whether these disparities are simply a manifestation of the low socioeconomic status of patients in Health plan A, and therefore we decide to further stratify results by socioeconomic status, we may find that disparities in health benefit between African Americans and non-African Americans persist. Based on these results and its commitment to a being a learning health system, Health plan A may decide to take more active measures to mitigate these health disparities, such as increased use of culturally-targeted prevention programs (eg, health coaches) for African Americans. These results may also be used to guide deployment of complementary social services by community partners.

Figure 2:

Benefits to African-Americans from compliance with evidence-based clinical guidelines

Stratifying a health-weighted composite quality metric for African Americans versus non-African Americans reveals that there are dramatic disparities in health benefit realized by African Americans across all three health plans. However, we see that Health plan A has greater disparities in health benefit between African Americans and non-African Americans compared to Health plans B and C. (High-quality life-years shown here, but any metric in Table 1 could be used).

Improving panel management

If statistical power is sufficient, we can choose to stratify health-weighted composite quality metrics by different provider units with the health plan (eg, facilities, firms, providers), thereby harnessing health-weighted composite quality metrics as a panel management tool (Figure 3). For example, if we stratify a metric for different clinics in Health plan A, we may see that one of the clinics (Clinic 5) is realizing much smaller health benefits than the other clinics. Based on these results, the leadership of Health plan A learns that it could improve the health of its population by directing a proportionately greater amount of quality improvement resources towards Clinic 5. Indeed, based on the stratified health-weighted composite quality metric results, the leadership of Health plan A could decide to make Clinic 5 the centerpiece of a new quality improvement effort devoted to incentivization, care integration and coordination, and to incorporate real-time updates of performance data.

Figure 3:

Benefits to different facilities within a single health plan from compliance with evidence-based guidelines

Stratifying a health-based composite quality metric based on different facilities in Health plan A shows that one of the clinics (Clinic 5) is realizing much smaller health benefits than the other clinics. Based on these results, the leadership of Health plan A infers that it could improve the health of its population by directing a proportionately greater amount of quality improvement resources towards Clinic 5. (High-quality life-years shown here, but any metric in Table 1 could be used).

Improving resource allocation

In order to improve how they allocate resources towards prevention and management of particular diseases, the leadership of Health plan A decides to stratify health-weighted composite quality metric data for Clinic 5 so that the unrealized health benefit attributable to each disease category can be quantified...

For example, health system leaders may find that more than half of unrealized health benefit arises from reducible risk factors for cardiovascular disease, whereas relatively small amounts of the unrealized health benefit arises from reducible risk factors for cancer or infectious diseases (Figure 4). Based on these results, Health plan A continues on its path of learning and decides to further target its incentives, health coaching resources, care integration, outreach programs, and community-based programs towards cardiovascular risk reduction.

Figure 4:

Benefits to treating certain diseases from compliance with evidence-based guidelines

The leadership of Health plan A decides to stratify health-weighted composite quality metric data for Clinic 5 in order to quantify how much unrealized health benefit is attributable to each disease category. They find that more than half of unrealized health benefit arises from reducible risk factors for cardiovascular disease, whereas relatively small amounts of the unrealized health benefit arises from reducible risk factors for cancer or infectious diseases. (High-quality life-years shown here, but any metric in Table 1 could be used).

Personalizing care

Health-weighted composite quality metrics can be provided to individual patients, giving them personalized information about how they could additionally improve their health based on their individual medical history and risk factor profile. This information could be incorporated at the point of care or at other times when patients are particularly receptive to health information (eg, web-based applications) or are particularly likely to be experiencing a “teachable moment” (eg, ED or inpatient discharge). Indeed, the personalized prevention model developed by Taksler and Braithwaite is currently being piloted for this purpose at Bellevue Hospital.4 Each patient could receive personalized graphical displays of how much additional health benefit they could accrue from more complete adherence with various evidence-based clinical guidelines that are applicable to them because of age, risk factors, etc (Figure 5). For example, a person in Clinic 5 of Health plan A may discover that, based on her particular risk factors, she would greatly benefit by controlling her hypertension, and that the magnitude of this benefit dwarfs the benefit she would sustain by complying with alternative recommendations that may be more difficult for her (for example, losing weight). Consequently, patients might be more likely to set and achieve health goals that are maximally aligned with the health benefit they could achieve. These health goals could be preference-congruent, and could be formulated in accord with established principles of shared decision making. 8–11, 12–14 Based on this and similar insights, the patient and her clinician implement self-monitoring and a follow-up plan that can be used to help other patients progress in controlling their hypertension.

Figure 5:

Benefits to individual patients from compliance with evidence-based guidelines

Health-weighted composite quality metric information can provide individual patients with personalized information about how they could additionally improve their health based on their individual medical history and risk factor profile. This information could be incorporated at the point of care or at other times when patients are particularly receptive to health information. (High-quality life-years shown here, but any metric in Table 1 could be used).

Workflow

A pilot study employing the preventive measure of Taksler and Braithwaite is underway in the primary care clinic at Bellevue Hospital Center, a major hub of the New York City’s public hospital system.4 This currently involves a subset of patients meeting with a Nurse Practitioner to populate a mortality model, using patient-reported data and the electronic medical record, to prioritize and guide preventive counseling. Future work will explore the feasibility of incorporating a broader range of data and more streamlined workflow, such that a health-weighted composite quality metric could be calculated for each patient at each visit. If successful, this type of metric could be used by the health system as a learning tool, to improve panel management and to assess whether care and outcomes may be improved.

Limitations

Health-weighted composite quality metrics have pitfalls as well as promises, with multiple technical and conceptual challenges. Complete data required for composite quality metric calculation may not be readily available in current EHR. However, health-weighted composite quality measures can be employed in systems without sophisticated EHRs by work-around measures such as using a non-EHR-based computer system with suitable privacy and security standards to run the mathematical model that produces the health-weighted quality metric, and by using staff to enter the necessary information at patient “check in.”

Not all aspects of high quality care are manifested in evidence-based quality metrics though, regardless of how thoughtfully they are constructed and regardless of how exhaustively they are validated. In order for HIT to be useful for yielding health-weighted composite quality metrics, it must overcome its historical roots of being developed primarily for use in billing tools rather than in clinical decision support tools. Finally, no health-weighted composite quality metric will be generalizable across all patients in a complicated and diverse panel, as some patients will have unusual or high-burden comorbidity profiles (eg, high Charlson score or large number of Elixhauser conditions), dominant comorbidities (eg,, metastatic cancer or severe CHF), or unusual preferences (eg, seeking to avoid procedures that may require blood transfusion regardless of lifesaving potential).15, 16 For example, while the model of Taksler and Braithwaite reflects the amount of health gain that would be conferred by evidence-based preventive care, some patients may have preferences that would make the measure unsuitable for decision making (for example, preferring to forgo a small loss in life expectancy in order to avoid the inconvenience and discomfort of a screening colonoscopy). However, it is important to note that concerns about generalizability limitations are an important yet universal problem faced by any application of evidence-based medicine to decision making, including current efforts to apply individual quality metrics.

Conclusion

Learning health systems have unprecedented opportunities to harness current and future HIT investment to improve health outcomes by new data-driven approaches, including health-weighted composite quality measures.4, 5 Using a health-weighted composite quality metric to help guide decision making could realign incentives towards paying for health, rather than paying for particular processes that have widely varying links of health improvement, particularly as health systems transition towards ACOs or other structures that aggregate accountability for health performance. We invite further dialogue and collaborations in this effort, for example a discussion of whether a health-based quality metric should be incorporated into future iterations of ACA-mandated health exchanges. Finally, we hope researchers and stakeholders will benefit from this submission by “keeping their eye on the ball” and maximizing what really matters: health.