Looking Under the Streetlight? A Framework for Differentiating Performance Measures by Level of Care in a Value-Based Payment Environment.

The majority of quality measures used to assess providers and hospitals are based on easily obtained data, focused on a few dimensions of quality, and developed mainly for primary/community care and population health. While this approach supports efforts focused on addressing the triple aim of health care, many current quality report cards and assessments do not reflect the breadth or complexity of many referral center practices.In this article, the authors highlight the differences between population health efforts and referral care and address issues related to value measurement and performance assessment. They discuss why measures may need to differ across the three levels of care (primary/community care, secondary care, complex care) and illustrate the need for further risk adjustment to eliminate referral bias.With continued movement toward value-based purchasing, performance measures and reimbursement schemes need to reflect the increased level of intensity required to provide complex care. The authors propose a framework to operationalize value measurement and payment for specialty care, and they make specific recommendations to improve performance measurement for complex patients. Implementing such a framework to differentiate performance measures by level of care involves coordinated efforts to change both policy and operational platforms. An essential component of this framework is a new model that defines the characteristics of patients who require complex care and standardizes metrics that incorporate those definitions.

Since the inception of the Joint Commission’s ORYX initiative in 1997,[1] which integrated the use of performance measures into the hospital accreditation process, our health care system has been inundated with performance measures. Yet, Institute of Medicine (IOM) reports continue to identify problems; these findings have spurred improvements in both system performance and health care quality.[2,3] Still, in 2006, only 54.9% of patients received the recommended care,[4] which led to the implementation of additional performance measures, primarily assessing processes of care.[5-8] As a result, many regulatory agencies, payers, and providers encounter duplicative and custom measures that are difficult to obtain and replicate and may not be linked to better patient outcomes.

Using reporting and assessment measures that are based on easily obtainable data is like “looking under the streetlight”[9]; that is, it is assessing health care quality and value by using only easy-to-find data, presumably in an attempt to reduce the burden of data collection on providers and payers. For the same reason of ease of use, health care claims and other administrative data are being used as performance measures, which has led to performance metrics being impacted by coding and billing practice issues. Many current quality report cards and assessments, then, do not reflect the breadth or complexity of many referral center practices, which are those institutions that provide advanced diagnosis and treatment specialists and facilities (usually academic centers and multispecialty group practices), after referral from primary or secondary care practices.

The National Quality Strategy, mandated by the Patient Protection and Affordable Care Act, established three aims for our health care system—better care, healthy people/healthy communities, and affordable care.[10] Population health efforts, however, tend to focus on addressing high-frequency or common problems that are applicable to primary/community care along with developing performance measures to assess that care. At the opposite end of the illness spectrum is complex care. Complex care treats diseases that require subspecialty care or illnesses that are esoteric and/or difficult to diagnose and treat. This clinical complexity makes it difficult to assess the quality of the care provided using performance measures designed for common conditions. Measures that track the outcomes of episodes of care for specific individual conditions do not generally account for the impact of comorbidities on patient outcomes. With the dynamic shift to assess not only the quality but also the cost and value of care, as demonstrated by the Centers for Medicare and Medicaid Services (CMS) Accountable Care Organization and Value Based Purchasing (VBP) programs, performance measures may need to differ across levels of care and incorporate additional risk adjustment to eliminate referral bias. Referral or selection bias represents the reality that patients with more complex problems (even with the same underlying condition) usually represent a higher proportion of the patients at referral centers than at community practices. Risk adjustment is a method to adjust payment or measurement to account for some of these differences in patient mix across providers.

In this article, we highlight the differences between population health efforts and referral care and address issues related to value measurement and performance assessment in each of these domains.

Levels of Care

Health care services can be categorized into three tiers in a pyramid structure—primary/community care, secondary care, and complex care. This pyramid is dynamic, with most of the population accessing community care for acute illnesses, wellness support, and preventive services. Secondary care is needed when acute or chronic illnesses or traumas require a specialist consultation or concentrated resources for a specific disease or body system, such as a particular treatment, procedure, or alternative therapy (e.g., primary joint replacement, general surgery, cancer treatment). This level of care includes “focused factories.”[11] At the top of the pyramid is complex care. Complex care, which is generally provided at tertiary and quaternary referral centers by multiple specialists, focuses on diagnostic mystery, treatment of rare or unusual diseases, previous failed therapies, and advanced diseases with no treatment options at other centers. It follows a “solution shop” model.[12] While the focus of complex care is generally highly technical and specialized in nature, there is a misconception that only patients with severe, complex, or uncommon health problems require this type of care. Patients with multiple comorbidities also may require specialized, integrated care. Although the conditions themselves may be straightforward, managing them simultaneously may require all the knowledge, skill, and resources of an integrated care team or subspecialist.[13]

Table 1 summarizes the definitions, goals of care, and types of providers at each level of this pyramid.

Table 1

Characteristics of the Three Levels of Medical Care

Defining and Measuring Complexity

While managing a complex patient generally requires greater clinician effort and entails factors other than medical decision making, no standard definition for the term “complex patient” exists. A Veterans Affairs working group defined health care complexity as requiring challenging clinical decision making and care processes that are not routine or standard.[14] Others have used proxies like the number of unique diagnoses[15]; domains that characterize complexity (e.g., chronic conditions, functional status, health care use)[16]; and factors such as medical decision making, care coordination, and socioeconomic status[17] to identify complex care.

Identifying performance measures that accurately capture medical complexity remains a challenge. Many current quality reporting and improvement initiatives do not adequately distinguish the differences between the delivery of community care and that of complex care; hence, risk profiles do not capture the complexity of the patient.[18] In addition, standard quality measures are often developed in study populations that exclude complex patients.[19] Yet, in the era of pay-for-performance, measures that account for clinical complexity are needed.

In a review, a technical panel of experts found that VBP programs generally focus on a narrow set of measures and “estimated that a small fraction (less than 20 percent) of all care that is delivered by providers is addressed by performance measures in VBP programs.”[20] A 2014 analysis by Kaiser Health News found that about half of academic medical centers (143 of 292) have been financially penalized for high rates of hospital-acquired conditions. Further analysis revealed that penalties were levied against 32% of the hospitals with the sickest patients, while only 12% of the hospitals with the least complex patients received penalties. Sicker patients and more complex cases are likely to have more complications, and complex procedures performed at academic medical centers, such as organ transplants and invasive cancer surgeries, are more likely to result in adverse events.[21]

Many of the current measures that compare providers treating complex patients versus those not treating these patients are victims of Simpson’s paradox.[22,23] Simpson’s paradox (also known as the Yule–Simpson effect) explains how overall performance can show that one provider who actually performs poorer than a second provider with both common and complex patients can appear to perform better overall because of the relative proportions of common and complex patients treated by each provider.[24,25] Using an indirect standardization (or regression adjustment) approach to account for case mix, severity, and comorbidity adjustments does not actually remove the differences when directly comparing provider profiles.[26]

Differences in patient panels then may lead to community or single-specialty hospitals dominating quality rating lists, like the recently released Medicare Overall Hospital Quality Star Ratings. A review by Kaiser Health News indicated that 102 hospitals received the top rating of five stars; yet, few of those hospitals are considered to be the nation’s best by private ratings sources such as U.S. News & World Report or as the most elite within the medical profession. Five stars were awarded to relatively unknown hospitals and to at least 40 hospitals that specialize in just a few types of surgery, such as knee replacements.[27]

Examples of the Effects of Referral Bias and Complex Care on Quality Measures

If complex patients were randomly distributed across the health care centers being measured, we would expect high variability in performance measures. The overall picture, though, would still be accurate. However, if these complex patients were concentrated at a few referral centers, as they are now, they could have a great impact on the performance measures of those centers, resulting in unadjusted referral bias.

A popular performance measure among patients with diabetes is a composite measure of blood glucose (HbA1c < 8), cholesterol (LDL < 100), and blood pressure (SBP/DBP < 140/80), as well as smoking cessation advice and daily aspirin prescription. This measure appears to be a very good indication of diabetes control in a primary care population but not in patients who are referred to endocrinologists.[28] Increasing complexity is associated with poorer performance on diabetes metrics.[29] For example, the subset of patients requiring specialty care usually are those with advanced or difficult-to-control disease, multiple medications and comorbidities, or unique situations, such as patients requiring chronic steroids or members of the transplant population. In these cases, patient-specific goals that differ from standard goals may be in the patient’s best interest. Additional risk adjustment then is needed to capture the greater management efforts or more intense interventions needed to obtain results similar to those in primary care.

Potential referral bias is not restricted to ambulatory care measures. Among surgical patients, the focus on select complications, including infections and the Agency for Healthcare Research and Quality patient safety indicators (PSIs), has increased.[30] PSI-15 (accidental puncture/laceration) is one example where referral bias could potentially affect both the perception of the quality of care provided by colorectal surgeons and their reimbursement. It currently makes up almost 50% of the PSI-90 score, which is a composite of eight complication measures. PSI-90 accounts for 35% of a hospital’s overall quality score, which could lead to penalties for hospital-acquired conditions, and for 30% of a hospital’s overall score in the VBP program.[31] Patients with complex abdominal surgeries or with prior abdominal surgeries are more vulnerable to PSI-15 injuries, and these surgeries tend to be concentrated at a few referral centers. While this issue appears to be a simple definition problem, coding variability, institutional coding policies, and the lack of an opportunity to indicate prior surgery in administrative databases influence the appropriate identification of an accidental puncture/laceration and limit potential risk adjustment.

Another example of potential referral bias, affecting colon surgery, occurs in the measurement of colon surgical site infection rates, which are evaluated as part of the VBP program. The Centers for Disease Control and Prevention National Healthcare Safety Network changed its surgical hierarchy for classifying procedure types in 2013.[32] Prior to the change, patients who had colon procedures and either small bowel or rectal procedures were counted in the latter two categories, which have higher expected infection rates. After the change, these patients were counted only in the publicly reported colon surgery category; however, the expected infection rates and the standardized infection ratio were still based on historical data captured using the original categories. For most institutions, there was little impact. However, for at least one referral center with an active practice in treating inflammatory bowel disease, the 10% increase in colon surgery cases with additional small bowel or rectal procedures tripled the number of observed infections, with no corresponding increase in expected infection rates, leading to unadjusted referral bias.

Performance measures for nonsurgical hospital patients are similarly affected by referral bias. In 2014, CMS began publicly reporting 30-day mortality and 30-day readmission rates for ischemic stroke patients, despite concerns that the measures were not adequately risk-adjusted. No valid measure of initial stroke severity was incorporated into these calculations, even though research indicates that initial stroke severity, as indexed by the National Institutes of Health Stroke Scale, is a dominant predictor of mortality in ischemic strokes.[33]

Operationalizing Value Measurement and Payment for Specialty Care

Most health care needs are relatively straightforward and involve issues that are appropriately directed to primary/community care. Specialty care is needed when the issues become more complex and exceed the ability or capacity of primary care. A patient encounter for specialty care can take three distinct forms: (a) episodic consultation (e.g., standard advice and treatment); (b) diagnosis and treatment as part of a well-defined episode (i.e., “focused factory”); or (c) diagnosis and/or treatment involving uncertainty of approach, uncertainty of time frame, or patient complexity (i.e., “solution shop”) (see Figure 1). Patients’ health care needs are dynamic; they move back and forth between the levels of care based on their medical needs. The ultimate goal of specialty care is treatment and the return of the patient to the appropriate level of care or, if necessary, continuing specialty care to enable the patient to maintain optimum health. This movement between the levels of care contributes to the challenge of obtaining accurate performance measures. As patients move between the levels of care, the expected intensity of care changes, as do the resources needed to provide care; both performance measures and reimbursement schemes need to reflect this change in intensity.

Figure 1

Framework for differentiating quality measures by level of care (primary/community care, secondary care, complex care), including definitions, expectations, and reimbursement models.

Most existing quality and performance measures appropriately focus on primary care and population health; however, these metrics may not be relevant and/or adequately risk-adjusted to reflect the breadth and intensity of the complex care provided at many referral centers. This “one size fits all” approach of using the same quality and performance measures for all levels of care needs to be revamped to provide relevant information about the value of complex care provided.

Recommendations for Differentiating Between Levels of Care in Performance Measurement

While several publications have called for changes to the way quality is measured,[34-38] a workable solution has yet to be identified. CMS has taken steps to incorporate value measures into payment models for hospitals, physicians, home health care, and bundled care. The announcement by the Department of Health and Human Services that they were going to increase the proportion of traditional Medicare payments tied to quality and value to 85% by 2016 and 90% by 2019, coupled with the move to the Medicare Shared Savings Program and the advent of alternative payment models like the Bundled Payments for Care Improvement initiative, increases the need to accommodate the complex care offered by referral centers in value measurement.[39] To provide a level playing field for referral centers, we propose the following policy changes, operational actions, and new model development. Table 2 summarizes the implementation strategy, implementation partners, and potential barriers for each of these recommendations.

Table 2

Recommendations for Differentiating Performance Measures by Level of Care

Policy changes

Match performance measures to levels of care.

To be useful to patients and consumers and to adequately assess performance, different measures should be applied to different levels of care and reported in a way that compares care at like institutions. In addition, in Crossing the Quality Chasm: A New Health System for the 21,[3] the IOM identified six domains that are pivotal to improving health care system performance—care should be safe, effective, patient-centered, timely, efficient, and equitable. We recommend that performance measures align with the level of the care and encompass these domains. Examples of metrics that meet these criteria are listed in Table 3.

Table 3

Performance Measures Across the Institute of Medicine Quality Domains by Level of Care

Move away from global or composite comparisons of providers and institutions.

In many cases, provider groups and health care centers have “institutes” or other entities in which specialty resources are focused on a specific diagnostic group of conditions (e.g., a cancer center). Most health care costs are also related to a small group of high-cost, high-prevalence conditions. Focusing on these conditions and reporting performance measures for specific conditions and procedures (e.g., cancer care, elective major surgeries, congestive heart failure) will allow for the differentiation of value to emerge, will provide a more valid and reliable basis for evaluating complex care, and should be more useful for consumers. The development of clinical interinstitutional registries (e.g., cardiac surgery, neonatology, transplant) has followed this model, and the recent release of seven core measure sets by CMS and America’s Health Insurance Plans is another step in the right direction.[40]

Align quality/outcome measures with appropriate reimbursement models.

Each level of care has different goals and expectations and requires a matching reimbursement model (see Figure 1) with appropriate value measures. For example, bringing down a diabetic patient’s HbA1c from 11.0 to 10.0 (still elevated) may have more “value” than lowering it from 8.5 to 7.5 (the measure goal).

Operational actions

Develop operational definitions to categorize level of care.

An agreed-upon operational definition for complex care using existing data sources does not exist. In addition, although not all specialty patients are complex, complex patients are more concentrated among specialty practices than primary care practices. To address this patient distribution bias, following our framework for measuring performance (see Figure 1), standard specialty care (secondary care) could be incorporated into primary/community care payments (global cost value-based payment), and well-defined, high-risk episodes could be addressed with risk-adjusted bundled services. Care that involves a higher degree of medical uncertainty, however, will require a more flexible approach that provides incentives that support high-value providers and allows patients to access and receive needed care (outlier reimbursement model).

Identify referral centers of excellence that consistently deliver high-quality care.

To identify these centers, we must first define the scope of services they provide and performance measures for standard care, using evidence-based medicine and practices. Then, we must identify what constitutes an outlier episode of care that falls outside the standard scope of services, requiring a more complex level of care and reimbursement model. Finally, we must establish, codify, and develop performance measures (e.g., mortality rate, complication rate, patient-reported outcomes) to identify referral centers of excellence that provide complex care. Birkmeyer and colleagues[41] showed that referral centers that have higher volumes of complex surgeries also have improved outcomes.

New model development: Identify the characteristics of patients who require complex care and standardize performance measures that incorporate those definitions

Referral centers must identify the characteristics that distinguish their patient populations that require complex care and thus fall outside the standard care process. Once these populations have been identified, referral centers could analyze these groups to determine commonalities that tend to trigger complex care. Metrics targeting reductions in these triggers or that include risk adjustments that account for them then could be developed. For example, a recent study of adult cardiac surgery patients demonstrated a change in the cost curve at the 75th percentile, indicating a change in the complexity of care. More complex care was required for patients in the top 25th percentile.[42]

Conclusion

While quality measurement in health care has been a catalyst for performance improvement and payment reform, current metrics focus on primary care and population health. Complex care is not well represented in these efforts. Including referral centers, which provide highly specialized complex care and episodic procedural treatments, in population-focused measurement will not provide an accurate representation of the quality of the care provided. Without a change in measurement domains to account for the increased risk in caring for more complex patients, physicians and hospitals will likely reassess their willingness to take on such complex cases for fear of damaging their reputations and reducing their reimbursements.

In this article, we provided examples that highlight the measurement problems that referral centers face when they are evaluated using primary care/population health measures as well as recommendations to address these shortcomings. Our proposed approach to quality measurement and suggested reimbursement schemes will require a shift from the current norm of using population-based quality measures to assess care at all levels. This shift is necessary to continue improving health care quality and value. Referral centers, like those in academic medicine, should take a lead role in furthering this approach.