An Environmental Scan of Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease Currently Used in Canada.

BACKGROUND: Quality indicators can be used to identify gaps in care and drive frontline improvement activities. These efforts are important to prevent adverse events in the increasing number of ambulatory patients with advanced kidney disease in Canada, but it is unclear what indicators exist and the components of health care quality they measure.
OBJECTIVE: We sought to identify, categorize, and evaluate quality indicators currently in use across Canada for ambulatory patients with advanced kidney disease.
DESIGN: Environmental scan of quality indicators currently being collected by various organizations.
SETTING: We assembled a 16-member group from across Canada with expertise in nephrology and quality improvement. PATIENTS: Our scan included indicators relevant to patients with chronic kidney disease in ambulatory care clinics. MEASUREMENTS: We categorized the identified quality indicators using the Institute of Medicine and Donabedian frameworks.
METHODS: A 4-member panel used a modified Delphi process to evaluate the indicators found during the environmental scan using the American College of Physicians/Agency for Healthcare Research and Quality criteria. The ratings were then shared with the full panel for further comments and approval.
RESULTS: The environmental scan found 28 quality indicators across 7 provinces, with 8 (29%) rated as "necessary" to distinguish high-quality from poor-quality care. Of these 8 indicators, 3 were measured by more than 1 province (% of patients on a statin, number of patients receiving a preemptive transplant, and estimated glomerular filtration rate at dialysis start); no indicator was used by more than 2 provinces. None of the indicators rated as necessary measured timely or equitable care, nor did we identify any measures that assessed the setting in which care occurs (ie, structure measures). LIMITATIONS: Our list cannot be considered as an exhaustive list of available quality indicators at hand in Canada. Our work focused on quality indicators for nephrology providers and programs, and not indicators that can be applied across primary and specialty providers. We also focused on indicator constructs and not the detailed definitions or their application. Last, our panel does not represent the views of other important stakeholders.
CONCLUSIONS: Our environmental scan provides a snapshot of the scope of quality indicators for ambulatory patients with advanced kidney disease in Canada. This catalog should inform indicator selection and the development of new indicators based on the identified gaps, as well as motivate increased pan-Canadian collaboration on quality measurement and improvement. TRIAL REGISTRATION: Not applicable as this article is not a systematic review, nor does it report results of a health intervention on human participants.

Introduction

Chronic kidney disease (CKD) is an important global health concern, and it is associated with increased mortality, morbidity, and health care costs.[1] In Canada, approximately 12% of adults and 30% of older adults (≥65 years) are living with CKD (an estimated 2.9 million Canadians),[2,3] Most of these patients are managed in primary care settings,[4] whereas patients with advanced CKD are often referred to multidisciplinary kidney clinics,[5,6] In Canada, patients receive team-based care from nephrologists, nurses, dieticians, pharmacists, and/or social workers to slow CKD progression, reduce complications, and plan for patient-centered end-stage kidney disease management. These efforts may be further aided by quality indicators, which allow health care providers to assess their performance and identify opportunities for quality improvement.

Quality indicators are predefined quantitative or qualitative benchmarks that measure different domains of health care quality (ie, safe, effective, efficient, timely, patient-centered, and equitable care) and can be further classified as structure (the setting in which care occurs), process (how care is delivered), or outcome (how care affects patients) measures.[7-9] These indicators can be used to set internal quality of care targets, compare performance with other centers, or implement payment-for-performance programs.[10] Furthermore, quality indicators can help motivate local quality improvement activities, which a recent meta-analysis demonstrated may improve surrogate outcomes and reduce dialysis incidence in patients with CKD.[11]

Despite these potential benefits, it is unclear what quality indicators are measured in ambulatory patients with advanced CKD across Canada. Our objectives were to describe the scope of existing indicators and the domains of health care quality covered, as well as evaluate indicators using the American College of Physicians/Agency for Healthcare Research and Quality criteria.[12] This knowledge should help programs select indicators and promote national collaboration to fill any performance measurement gaps that are identified.

Methods

Indicator Identification and Categorization

Initially, we collected publicly available quality indicators from provincial and local nephrology programs. For indicators not publicly available, we used a snowball sampling methodology, where we relied on study participants to recruit and recommend future participants. We then contacted nephrology programs across the country (including British Columbia, Alberta, Saskatchewan, Manitoba, Quebec, Ontario, and the Atlantic Provinces) to query current provincial and local practices on performance measurement. This sampling took place from January 31, 2019, to March 29, 2019. Program representatives contacted included provincial data leads, division heads, and content experts, who provided data or identified regional leaders for assistance. We asked nephrology programs “which quality indicators are currently collected at the provincial and local levels to evaluate ambulatory kidney care practices?” The remainder of the discussion was free form. We stopped the environmental scan once we achieved representation from all the aforementioned provinces.

Once a list of indicators was collected, we consolidated similar indicators into a single measure. We categorized each indicator according to the Institute of Medicine (IOM; safe, effective, efficient, timely, patient-centered, equitable) and Donabedian (structure, process, outcome) frameworks of health care quality.[8] We also collected balancing indicators to look for inadvertent harms that may occur in the health care system (eg, hospitalizations for drug-related hyperkalemia).[13]

Indicator Evaluation

We rated each indicator using a modified version of the American College of Physicians/Agency for Healthcare Research and Quality performance measure review criteria. These include the following dimensions (Supplemental Table 1):[14-16]

Importance: The metric will lead to measurable and meaningful improvement, or a clear performance gap exists.

Evidence-based: The metric evidence is robust and of high quality.

Measure specifications: The metric can be clearly defined (ie, numerator and denominator) and reliably captured.

Feasibility and applicability: The metric may have an influence on health care providers and/or the health care system, and data collection and improvement activities are achievable and acceptable.

We rated each dimension using a 9-point scale, where 1-3 indicates “does not meet criteria,” 4-6 “meets some criteria,” and 7-9 “meets criteria.” Based on the ratings, each indicator then received a final global rating (instead of an average score) based on its overall ability to distinguish good quality from poor quality.[15] For the global rating, we considered quality indicators as “necessary” if the median rating was 7, 8, or 9 and there was no disagreement between members. We considered indicators as “unnecessary” if the median rating was 1, 2, or 3 and there was no disagreement by any member. We considered all other indicators as “supplemental.”

Modified Delphi Process

We used a modified version of the Delphi consensus technique to evaluate quality indicators based on the RAND method.[17] This format is an established technique for developing and evaluating quality indicators in health care.[16,18-24] The Delphi panel consisted of a 16-member volunteer national quality indicator committee with nephrology representatives from 7 of 10 provinces and most possessing advanced training in quality improvement. Four of the members formed an ambulatory kidney care subcommittee.

First, we made the identified ambulatory kidney quality indicators available to each member of the subcommittee. Each member then individually reviewed the indicators and rated them according to the above criteria. Next, the ambulatory kidney subcommittee met by teleconference to compare their initial ratings and provide feedback as needed. Any disagreements prompted further discussion until consensus was achieved. These consensus ratings were then shared with the entire 16-member committee, with further review of any ratings that differed by ≥3 points. The full 16-member committee prior to publication approved the final ratings. Formal research ethics board review was not required by Queen’s University based on the Tri-Council Policy Statement for ethical human research, as the focus of the study involved quality indicators and not human participants.

Results

We identified 28 ambulatory kidney quality indicators across 7 provinces in Canada (Table 1), of which 4 provinces collected province-wide standardized metrics. The IOM domains covered included safe (n = 5, 18%), effective (n = 9, 32%), efficient (n = 5, 18%), timely (n = 2, 7%), patient-centered (n = 6, 21%), and equitable (n = 1, 4%) care. Donabedian categories covered included process (n = 10, 36%), outcome (n = 14, 50%), and balancing (n = 4, 14%). We did not identify any structure indicators.

Table 1.

Environmental Scan of Current Canadian Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease.

Institute of Medicine domains of quality	Structure	Donabedian framework of health care quality
		Process	Outcome	Balancing
Safe		– % of patients with comorbidity assessment (1)– eGFR at the time of nephrology clinic registration (1)– Time spent in nephrology clinic prior to dialysis initiation (1)	– % of hemodialysis starts that are outpatient starts (1)	– % of patients with rapid decline in kidney function over 12 mo (1)
Effective		– Albuminuria screening in diabetes (1)	– Achievement of anemia targets (2)– Achievement of iron targets (1)– Achievement of calcium, phosphate, PTH targets (1)– % of patients on ACEi or ARB (1)– % of patients on statin (2)– % of patients with blood pressure <140/90 mm Hg (1)– eGFR <15 mL/min/1.73 m2 with hemodialysis as choice and preemptive fistula in place (1)– % of hemodialysis starts with functional permanent access (1)
Efficient		– % patients with chronic kidney disease followed according to standardized clinical pathway for access creation (1)	– Movement out of nephrology clinic with reasons (ie, death, transplant, dialysis conservative care) (1)– Patients receiving preemptive transplant (2)– eGFR at dialysis start (2)	– Referrals to nephrology that do not meet standardized criteria (1)
Timely				– Time from clinic referral to nephrologist visit (1)– No. of days between home dialysis referral and assessment (2)
Patient-centered		– Goals of care documented (1)– eGFR <20 mL/min/1.73 m2 and modality decision documented (1)– eGFR <15 mL/min/1.73 m2 and assessed for symptoms– every 6 mo (1)– Modality decision documented at dialysis initiation (1)	– % initiated on chosen modality (1)– Patient-reported experience measures for shared decision-making (1)
Equitable		– % of nephrology patients eligible for multidisciplinary care who are followed in a multidisciplinary clinic (1)

Note. The number of provinces currently using the listed indicator is indicated in the adjacent parentheses. The denominator is 7 provinces (territories excluded and Atlantic provinces combined). ACEi = angiotensin-converting enzyme inhibitors; ARB = angiotensin receptor blocker; eGFR = glomerular filtration rate; PTH = parathyroid hormone.

We found significant heterogeneity in the quality-of-care indicators across provinces, with 5 indicators being collected in more than 1 province (achievement of anemia targets, % of patients on a statin, estimated glomerular filtration rate [eGFR] at dialysis start, number of patients receiving a preemptive kidney transplant, and number of days between home dialysis referral and assessment). More than 2 provinces measured no quality-of-care indicator.

With respect to overall ability to distinguish good-quality from poor-quality care (ie, necessary versus unnecessary for improvement), we rated 8 (29%) indicators as “necessary,” 14 (50%) as “supplemental,” and 6 (21%) as “unnecessary” (Table 2). The 8 “necessary” indicators focused on safe (n = 1, 13%), effective (n = 4, 33%), efficient (n = 2, 17%), and patient-centered care (n = 1, 13%) and consisted of 2 process and 6 outcome measures. None of the necessary rated indicators measured timely or equitable care. Of these 8 “necessary” indicators, 3 were measured by more than 1 province. These included % of patients on statin, eGFR at dialysis start, and number of patients receiving preemptive kidney transplant.

Table 2.

Quality Indicators Rated by the American College of Physicians/Agency for Healthcare Research and Quality Performance Measure Criteria Using a Modified Delphi Technique.

Indicator type	Targets important improvements	Strong level of evidence	Performance gap exists	Precisely defined and specified	Feasible to collect	Usable for quality improvement	Global rating	Comments
Process
% of nephrology patients eligible for multidisciplinary care who are followed in a multidisciplinary clinic	3	1	7	5	2	6	3	Intervention difficult to measure and would require outreach to primary care to improve
% of patients with comorbidity assessment	5	2	5	7	1	3	3
eGFR at the time of nephrology clinic registration	5	7	8	7	6	6	6	Improvements would require outreach to primary care
Time spent in nephrology clinic prior to dialysis initiation	7	7	6	9	7	7	7	Depends on referral pattern and acute “crash starts”; needs to be case-mix adjusted
Albuminuria screening in diabetes	7	8	6	7	7	7	7	More useful for primary care; need to specify measurement method (urine ACR vs dipstick) and frequency
Goals of care documented	8	3	8	8	1	6	4	Has important implications for decision-making; hard to measure quality of the process rather than checkbox completion
eGFR <20 mL/min/1.73 m2 and modality decision documented	5	3	8	9	5	6	5	Documentation does not mean high-quality modality education given; decisions at this eGFR may not be appropriate for all patients
eGFR <15 mL/min/1.73 m2 and assessed for symptoms every 6 mo	8	4	8	9	4	5	5	Need resources to measure and way for system to respond to symptoms found
% patients with chronic kidney disease followed according to standardized clinical pathway for access creation	5	3	6	2	2	3	3	May lead to unnecessary referrals and investigations; lots of resources used for potential futile access
Modality decision documented at dialysis initiation	8	6	3	9	5	4	5
Outcome
Achievement of anemia targets	7	5	6	9	5	7	6	Resource intense if not done by electronic medical record
Achievement of iron targets	5	4	6	9	5	5	4
Achievement of calcium, phosphate, PTH targets	3	4	7	8	5	6	3
% of patients on ACEi or ARB	8	8	6	4	4	7	8
% of patients on statin	7	8	8	6	5	7	7
% of patients with blood pressure <140/90 mm Hg	9	9	8	8	5	8	7
eGFR <15 mL/min/1.73 m2 with HD as choice and preemptive fistula in place	7	5	7	7	7	5	5	Fistula first has strong evidence, but eGFR target does not; not patient-centered and likely resource-intensive
% of HD starts with functional permanent access	6	6	7	9	9	5	5
Movement out of nephrology clinic with reasons (ie, death, transplant, dialysis, conservative care)	4	2	3	8	5	4	2
Patients receiving preemptive transplant	8	8	8	8	9	9	8
eGFR at dialysis start	7	7	7	6	4	7	7	Categorical measure (eg, % starting above 9.5 mL/min/1.73 m2) may be more useful than mean eGFR
% of HD starts that are outpatient starts	6	5	8	8	8	5	4	Not patient-centered and may have unintended consequences (eg, earlier dialysis initiation)
% initiated on chosen modality	7	4	8	2	4	5	5	Need resources to increase home dialysis uptake to maximize utility
Patient-reported experience measures for shared decision-making	8	7	8	3	3	6	7	Need accompanying pathways and strategies to respond to the issues identified by patients
Balancing
Referrals to nephrology that do not meet standardized criteria	4	3	5	3	1	4	4
Time from clinic referral to nephrologist visit	6	6	7	4	3	6	5
% of patients with rapid decline in kidney function over 12 mo	7	5	5	8	3	1	3	May not be modifiable
No. of days between home dialysis referral and assessment	5	5	9	9	5	7	5

Note. Each domain was rated on a 9-point scale, where 1-3 indicated “does not meet criteria,” 4-6 “meets some criteria,” and 7-9 “meets criteria.” After considering and rating each of these domains, the panelists then rated the overall measure (1-3 = unnecessary, 4-6 = supplemental, 7-9 = necessary). ACEi = angiotensin-converting enzyme inhibitors; ARB = angiotensin receptor blocker; eGFR = glomerular filtration rate; PTH = parathyroid hormone; HD = hemodialysis.

Five common themes emerged during the rating process. First, the strength of evidence for most indicators was limited, with 9 (32%) indicators receiving ratings of 7-9. Second, feasibility of data collection was a major concern among committee members due to differing provincial infrastructures and electronic medical record (EMR) capabilities. Overall, 6 (21%) indicators received ratings of 7-9 for “feasible to collect.” Third, the majority of indicators were precisely defined and specified, with notable exceptions including necessary rated indicators related to the use of angiotensin-converting enzyme inhibitors (ACEi) or angiotensin receptor blockers (ARBs), statins, and patient-reported experience measures (PREMs). Fourth, there was unclear attribution of some metrics to nephrologists (ie, pathways for access creation, referrals not meeting standardized criteria) or lack of delineation from primary care (ie, albuminuria screening, goals of care discussions). Finally, the environmental scan did not identify any indicators related to kidney transplant awareness, education, or referral thresholds.

Discussion

Our environmental scan of the current ambulatory care quality indicators for patients with advanced kidney disease in Canada found 28 different indicators. Using the American College of Physicians/Agency for Healthcare Research and Quality criteria, our panel with expertise in nephrology and quality improvement rated 8 of the indicators as “necessary” to distinguish good-quality care from poor-quality care. We also revealed wide variation in indicator choices across Canada, with 5 indicators being identified as being collected by more than 1 province. Furthermore, very few indicators focused on the IOM domains of timely or equitable care, and there were no structure indicators. This work provides nephrology programs with examples of indicators from which to select, and we hope it motivates more pan-Canadian collaboration on quality measurement to avoid “re-inventing the wheel” when new indicators are developed.

Most published ambulatory care quality indicators for patients with CKD are aimed at primary care providers and not nephrologists.[25-28] As a result, many indicators focus on CKD detection (ie, screening in high-risk patients), monitoring (ie, frequency of serum creatinine and proteinuria testing), and blood pressure/cardiovascular disease management, rather than CKD complications and kidney replacement therapy (KRT) planning. The American Society of Nephrology (ASN) Quality Committee recently evaluated measures from kidney and quality metric organizations,[16] identifying 20 ambulatory CKD indicators (17 unique indicators after removing duplicates). Overall, they rated 8 of 17 (47%) highly based on the American College of Physicians/Agency for Healthcare Research and Quality criteria, which was slightly higher than our rating (29%) due to their inclusion of vaccinations (influenza and pneumococcal) and smoking cessation as CKD indicators. Of their 5 remaining necessary rated indicators, our group similarly rated blood pressure control and use of ACEi/ARB as “necessary” metrics to distinguish good-quality from poor-quality care. These indicators are commonly cited in the primary care literature, along with our necessary rated indicators of albuminuria screening in diabetes and use of statins.[25-28] This overlap between primary care and advanced CKD indicators may help motivate both groups to share limited resources in the pursuit of joint quality improvement initiatives. Also, both primary care and nephrologists should jointly develop and define indicators to ensure resources are deployed and used in the most efficient manner possible.

Our results also differed from the primary care literature and the ASN study in several important ways, mainly related to KRT planning metrics. It is not surprising that the CKD primary care literature lacks KRT metrics, but the only KRT indicators identified by the ASN study as necessary rated metrics were catheter rate at hemodialysis start and advance care planning.[16] We rated hemodialysis starts with permanent access (similar to catheter rate) as a “supplemental” indicator due to concerns about resource intensity and lack of patient-centeredness.[29] We did not rate advance care planning as a necessary indicator despite its importance, because as a binary measure it may not adequately capture the decision-making process with sufficient granularity, promoting checkbox completion as an unintended consequence.[30] However, these metrics still provide value by ensuring a standard is met for patient care, support staff adherence to care pathways, and may improve team-based models of care. Unlike the ASN study, more than 50% of the Canadian indicators focused on CKD complications (eg, anemia, uremic symptoms) or KRT planning (eg, modality decisions, eGFR at dialysis start). This disparity could be related to health care system differences, but the low provincial overlap suggests that quality-of-care measurement for advanced CKD is likely in its early stages, presenting an opportunity for provinces to collaborate on indicator selection that will help align quality improvement activities and promote shared learning.

Our environmental scan contributes to these goals by categorizing existing indicators according to the IOM and Donabedian frameworks. In this way, we highlight an absence of measures rated as necessary for timely and equitable care, as well as a complete lack of structural indicators. Large variation has been noted in ambulatory kidney clinic staffing ratios and consistency, which likely contributes to differences in team member roles, clinic processes, and potentially outcomes.[5,6] While additional research is needed to determine the optimal clinic structure, some minimum standards may be helpful to ensure key processes (eg, home dialysis and transplant education) can be completed. In addition to further development of timely and equitable indicators, we identified 1 metric each for safe and patient-centered care. Potential indicators could focus on medication safety and patient-reported outcome measures (PROMs)/PREMs,[31,32] with work needed so that the indicators are attributable to nephrology providers and usable by frontline staff for improvement efforts.

As an initial step in this process, we have provided an example of a balanced quality indicator scorecard for the ambulatory care of patients with advanced kidney disease that covers all of the IOM and Donabedian domains (Table 3). This incorporates several of the indicators rated as necessary, also found in the primary care literature and ASN study (albuminuria screening in diabetes, blood pressure achievement, and use of statins), many of which require specification of the numerator, denominator, and performance targets,[16,25-28] Use of ACEi/ARBs is similarly rated as necessary by all groups and could replace blood pressure achievement depending on local data infrastructure and EMR capabilities. The eGFR at dialysis start metric is one of the few necessary rated indicators used by multiple provinces and supported by randomized trial evidence.[33] We also included a patient-reported measure, although work is needed to standardize its measurement, reporting, and staff responses. Other newly proposed indicators are intended to fill gaps in the clinical care of patients with advanced CKD (eg, rates of home dialysis and preemptive transplantation) and/or the IOM/Donabedian domains. For example, time to third next available appointment is a common measure of timely health care access.[34] These new indicator suggestions are meant to stimulate further discussions among administrators, frontline providers, and patients so that balanced scorecards may be developed that are feasible and usable for quality improvement in multiple different local and provincial settings.

Table 3.

First Step Towards Development of a Balanced Quality Indicator Scorecard for Ambulatory Patients With Advanced Kidney Disease.

Institute of Medicine domains of quality	Donabedian framework of health care quality
	Structure	Process	Outcome	Balancing
Safe				– Hospitalizations for kidney-related adverse drug events
Effective		– Albuminuria screening in diabetes (need to specify measurement method and frequency)	– % of patients on statin (need to specify numerator, denominator, and minimum acceptable target)– % of patients with blood pressure <140/90 mm Hg (target may need to vary based on comorbidities)
Efficient	– Nurse to patient ratios		– eGFR at dialysis start (specify an upper eGFR threshold and report % below the target)
Timely		– Time to third next available appointment
Patient-centered			– Patient-reported outcome and experience measures (need infrastructure to measure, report, and act on the results in a timely manner)
Equitable	– Presence of dedicated conservative care pathway		– Rates of home dialysis and preemptive transplantation, stratified by population (eg, rural, indigenous, socioeconomic status)

Note. Several indicators rated as necessary from the environmental scan have been populated (in regular font), with examples of future potential indicators (in bold) and additional work needed to complete the scorecard (in italics).

The major strength our work is that we used established IOM and Donabedian frameworks to organize our indicators and the American College of Physicians/Agency for Healthcare Research and Quality criteria for indicator evaluation. To ensure our work was useful for frontline health care providers, our panel consisted of members from most provinces in Canada who had advanced training in quality improvement and experience with quality improvement activities in ambulatory settings.

There are some limitations to consider. First, the goal was not to survey all ambulatory centers that care for patients with advanced kidney disease across Canada, but rather to obtain an initial impression of which quality indicators exist in different provinces. Our snowball sampling methodology should not be considered exhaustive. Second, we targeted quality indicators that measure the care provided by nephrology providers and programs for ambulatory patients with advanced kidney disease. We did not intend to capture indicators that apply to the quality of care delivered to patients with kidney disease in the primary care setting, although some indicators may be relevant to that patient population. Next, our work focused broadly on the indicator constructs and not the details of inclusion/exclusion criteria or risk adjustment, which must be determined prior to application.[14] Last, our panel consisted of 15 physicians and 1 nurse practitioner, and so does not represent the views of other important stakeholders that will be needed when new indicators are developed and selected, such as patients, pharmacists, and administrators.

Conclusion

The 28 different quality indicators that our panel identified and evaluated across Canada illustrate the variation that exists and highlights current gaps in the measurement of the quality of care provided to ambulatory patients with advanced kidney disease. We rated 8 indicators as “necessary” to delineate good from poor quality of care, and 3 of these indicators were collected by more than 1 province. This work demonstrates that development of quality indicators for ambulatory patients with advanced kidney disease in Canada is in its early stages, and the small number of indicators rated as necessary presents an opportunity for pan-Canadian collaboration, as several new indicators are needed particularly in the domains of timely and equitable care. Our environmental scan should be considered a preliminary step to help guide this process, with the goal of creating indicators that address patient priorities and are useful for frontline quality improvement efforts.

Click here for additional data file.

Supplemental material, sj-pdf-1-cjk-10.1177_2054358121991096 for An Environmental Scan of Ambulatory Care Quality Indicators for Patients With Advanced Kidney Disease Currently Used in Canada by Jay Hingwala, Amber O. Molnar, Priyanka Mysore and Samuel A. Silver in Canadian Journal of Kidney Health and Disease