Use of Electronic Administrative Databases to Measure Quality Indicators of Breast Cancer Care: Experience of Five Regional Oncology Networks in Italy.

PURPOSE: Assuring quality of care, while maintaining sustainability, in complex conditions such as breast cancer (BC) is an important challenge for health systems. Here, we describe a methodology to define a set of quality indicators, assess their computability from administrative data, and apply them to a large cohort of BC cases.
MATERIALS AND METHODS: Clinical professionals from the Italian Regional Oncology Networks identified 46 clinically relevant indicators of BC care; 22 were potentially computable using administrative data. Incident cases of BC diagnosed in 2016 in five Italian regions were identified using administrative databases from regional repositories. Each indicator was calculated through record linkage of anonymized individual data.
RESULTS: A total of 15,342 incident BC cases were identified. Nine indicators were actually computable from administrative data (two structure and seven process indicators). Although most indicators were consistent with guidelines, for one indicator (blood tumor markers in the year after surgery, 44.2% to 64.5%; benchmark ≤ 20%), deviation was evident throughout the five regions, highlighting systematic overlooking of clinical recommendations. Two indicators (radiotherapy within 4 months after surgery if no adjuvant chemotherapy; 42% to 83.8%; benchmark ≥ 90%; and mammography 6 to 18 months after surgery, 55.1% to 72.6%; benchmark ≥ 90%) showed great regional variability and were lower than expected, possibly as result of an underestimation in indicator calculation by administrative data.
CONCLUSION: Despite highlighting some limitations in the use of administrative data to measure health care performance, this study shows that evaluating the quality of BC care at a population level is possible and potentially useful for guiding quality improvement interventions.

INTRODUCTION

In many countries, cancer treatment represents an increasing burden on the health care budget. Major determinants of health care costs include the increase in cancer incidence, primarily caused by the aging of the population, and the use of high-cost drugs and technologies. The combination of these factors challenges the sustainability of health systems. Consequently, interest in evaluating and optimizing the standard of care for patients with cancer has grown in recent years.[1,2]

Breast cancer (BC) is the most frequent malignancy in women in developed countries.[3] Moreover, because of its relatively good prognosis as compared with other neoplasms, BC presents an extremely high prevalence in developed countries. In 2018, almost 800,000 women in Italy had a previous diagnosis of BC.[4]

BC is a clinically complex condition, requiring a coordinated multidisciplinary approach.[5] In this context, evaluation of adherence to guidelines and quality of care is relevant to provide all patients with adequate treatment while maintaining sustainability in public health systems. In fact, waste expenditures, accounting for approximately 20% to 30% of global health costs,[6] derive from inappropriate interventions (ie, actions that are not recommended by national and international guidelines) without adding significant therapeutic advantages for patients. Avoiding loss of resources is imperative to maintain system sustainability; in addition, waste interventions can ultimately damage patients and affect the quality of care. In such a challenging context, policy makers and health care providers are striving to create reproducible and comparable performance measurement systems to develop quality indicators as the essential first step toward quality improvement.

Measuring health care performance, defined as the health outcome achieved at the population level per amount of expenditure, is a debated issue.[7] This is extremely challenging in the oncology field because of (1) the long interval between the delivery of initial care and the outcome, (2) the multidisciplinary nature of care,[5] and (3) the rapidly changing standards of care.

These considerations suggest the use of structure and process indicators, in addition to outcome indicators such as long-term survival, to evaluate the compliance of delivered care to standard of care. Several sets of quality indicators have been proposed by different local institutions and international associations.[8,9] However, most of these have been evaluated using ad hoc data collected on a voluntary basis.[10] This represents a significant limitation to the use of these indicators on a larger scale because it might require a significant effort for data collection.

Administrative routine data are less informative than ad hoc databases. However, indicators based on the former source are not affected by selection biases and easily cover large populations. Research using administrative data at a population level has been conducted previously (eg, in the Scottish Cancer Taskforce study).[11]

With the increased use of electronic databases and management systems, reliable data from routinely collected information on outpatient delivered care, drug prescriptions, hospital records, and causes of death registers have become available and can be used to calculate quality indicators encompassing the entire process of BC care at a population level.[9,12] However, the pathway of care of a patient with BC might differ slightly according to the local health care organization, and the administrative data collected might vary as well. Therefore, the measurability of quality indicators of BC care using administrative data should be assessed on a local basis.

In Italy, a national cancer network does not exist. However, several Italian regions have formally established regional cancer networks that include local health institutions involved in cancer care, to improve its efficacy, efficiency, and equity. Each regional network has established multidisciplinary groups in charge of defining the pathways of care related to the diagnosis and treatment of several malignancies (percorsi diagnostico terapeutico assistenziali [PDTA]). Multidisciplinary groups devoted to defining the pathways of care for patients with BC include surgeons, oncologists, pathologists, radiotherapists, radiologists, pharmacists, physiatrists, general practitioners, palliative care practitioners, psychologists, and epidemiologists. The PDTAs are designed to implement the national and international guidelines in the practical setting of each regional/local organization, taking into account the actual features and resources of the regional health system. They also define a number of indicators derived from the guidelines and deemed relevant by the multidisciplinary group, which also indicates appropriate benchmarks. The PDTAs and related indicators are formally adopted by decree at the appropriate institutional level, which coincides with the regional government in almost all cases.

In this article, we describe the methodology used to select a set of indicators and show their application in the cohort of incident cases of BC diagnosed in five Italian regions in the year 2016. Our rationale was as follows: first, to define key performance indicators for quality in BC care; second, to verify the possibility of calculating BC care indicators from routinely collected information; and third, to measure key performance indicators for patients diagnosed with BC in five Italian regions.

MATERIALS AND METHODS

Identification of the Set of Indicators

The project, sponsored by the Periplo Association and approved by the central competent institutional review board in February 2019, was implemented throughout the certified Italian Cancer Networks. The Cancer Networks of the participating regions (Campania, Emilia-Romagna, Lazio, Liguria, Piemonte, Toscana, Umbria, and Veneto) were involved in the selection of performance indicators, and for five regions (Liguria, Piemonte, Toscana, Umbria, and Veneto) exploitable data for the evaluation of BC care indicators were available.

The regional Oncology Networks established an Indicators’ Working Group, composed of oncologists and epidemiologists, that compared the pathways of care defined in the regional/local PDTAs and identified common points in these pathways. On this basis, they then compared the indicators identified by regional PDTAs, selecting only those indicators assessing features that were common to all the regional/local PDTAs, thus identifying 46 indicators (nine structure indicators, 29 process indicators, and eight outcome indicators; Table 1). The respective benchmarks were already defined by the regional/local PDTA documents and were incorporated by the Indicators’ Working Group. These indicators were further validated by a Web-based survey conducted in October and November 2016, in which the directors of the 65 Italian Breast Units were asked to judge the relevance of each indicator and relative benchmark. All the indicators identified by the Indicators’ Working Group were valued relevant or very relevant by > 80% of the 39 Breast Unit directors who replied to the survey.

TABLE 1.

Indicators of Quality of BC Care

The methodologists participating in the group considered 22 of these indicators to be potentially computable on the basis of what could be retrieved from administrative databases (Table 1).

Calculation of Each Indicator at the Patient Level

To calculate the indicators, each Cancer Network used available computerized sources of health information from 2011 to 2018. These included hospital discharge forms (HDR, 2011 to 2017), outpatients’ records of diagnostic and therapeutic procedures (2013 to 2018), prescriptions of drugs reimbursed by the National Health Service both in hospital and in the outpatient setting (2013 to 2017), the regional health registry (updated to January 31, 2017), and the regional mortality registry.

All data used came from regional repositories, which collect data from all National Health Service providers in the region, and not from single institutional repositories, to minimize the risk of missing data (eg, procedures performed at institutions different from the one where the patient was originally diagnosed).

Regional repositories share the same classifications for most of the items concerned and exhibit similar procedures for data collection, selection, and validation. A unique personal code was used to match data at the individual level. Data were anonymized, so that they could not be traced back to the single patient or the single administrative form.

For structure indicators, surgical interventions of quadrantectomy or mastectomy (International Classification of Diseases [9th revision, clinical modification] procedure codes 85.2x, 85.33, 85.34, 85.35, 85.36, 85.4.x) with a diagnosis of invasive BC (International Classification of Diseases [9th revision, clinical modification] diagnosis codes 174, 198.81, 233.0)[13] were identified from all HDRs (ordinary hospitalization or day surgery) from Italian health care institutions finalized between January 1, 2016, and December 31, 2016. At the hospital level, the mean yearly volume of breast surgery interventions over the study time interval was calculated.

For process indicators, cases were counted on a patient, and not on an intervention basis, and only new incident cases, which did not have a previous diagnosis of BC in the 5 years preceding the surgical intervention, were considered. The incidence period was January 1 to December 31, 2016, and HDR data from 2011 to 2015 were used to exclude prevalent cases.

The correct calculation of some indicators required the exclusion of the cases with presurgical treatment, individuated by outpatients’ records of diagnostic and therapeutic procedures and drug prescriptions from 2013 to 2015. Some indicators required assessing therapeutic and diagnostic procedures in the following 18 months; therefore, data from the previously mentioned sources from 2017 to 2018 were also used.

More precise information regarding patient selection for each single indicator is presented in the Data Supplement (online only). Information was available only for patients residing in the region of interest. At the patient level, data on the age at diagnosis and sex were collected.

Statistics

Each indicator was calculated as the proportion of patients who received the involved procedure, in the defined time window, among those eligible. Precise definition of data source and calculation for each indicator is reported in the Data Supplement. Each indicator was calculated separately for the five regions involved in the study. The management software available at each Oncology Network (mostly SAS 9.4) was used to perform the calculations.

RESULTS

Participants

From January 1, 2016, to December 31, 2016, 16,927 surgical hospitalizations for interventions for newly diagnosed invasive and/or in situ BC were recorded in the registers of the five regions (Table 2), corresponding to 15,342 different patients newly diagnosed with invasive and/or in situ BC. The study cohort was composed mainly of women (99.1%), and almost one half were 50 to 69 years of age (45%). Seventy-eight percent of patients received surgery at a hospital with a mean yearly volume of ≥ 150 breast surgical interventions (data not shown).

TABLE 2.

Surgical Hospitalizations for Newly Diagnosed Invasive and In Situ BC Interventions in 2016

Selected Indicators and Computable Indicators

Of the 22 indicators selected by clinicians and methodologists, only nine were finally computable from administrative data (Table 3).

TABLE 3.

Definition of Indicators and Their Raw Calculation

Final Set of Indicators and Adherence to Guidelines

Of the final set of nine computable indicators, two were indicators of structure: the proportion of facilities with > 150 interventions for BC/year and the proportion of facilities with a formally established multidisciplinary Breast Unit. Seven were indicators of process. In the setting of primary disease treatment, we observed a lower than expected (between 65.4% and 75.1%, benchmark ≥ 80%) proportion of patients who started adjuvant systemic treatment (chemotherapy or endocrine therapy) within 60 days after surgery.

The proportion of patients evaluated with magnetic resonance imaging before surgery (between 10.4% and 24.6%; benchmark ≤ 20%) was consistent overall with BC care guidelines, despite some regions presenting borderline values. The proportion of patients who received radiotherapy within 4 months after surgery (if not receiving adjuvant chemotherapy; between 42% and 83.8%; benchmark ≥ 90%), as calculated using administrative data, was highly variable across regions, and in some cases was significantly lower than expected. In the postsurgical setting, an unexpectedly low percentage of patients (between 55.1% and 72.6%; benchmark ≥ 90%) seemed to have undergone mammography 6 to 18 months after surgery.

Use of a follow-up bone scan in the first year after surgery (5.5% to 8.8%; benchmark ≤ 10%) was consistently low across the five regions, as expected; on the contrary, blood tumor markers were tested extensively in the first year after surgery (44.2% to 64.5%; benchmark ≤ 20% proposed to account for stage IV patients), and this was observed consistently throughout the five regions.

Use of chemotherapy in the last 30 days before death was limited (approximately 6%), consistent with guidelines that discourage its use (benchmark < 10%). However, this indicator was only evaluable in a limited number of regions.

DISCUSSION

This study presents a set of indicators calculated from data obtained from multiple administrative databases gathered in a real-world setting across five Italian regions, covering an overall population of almost 15,500,000 people (in 2016). Our data confirms the findings of previous studies demonstrating that administrative data are suitable for measuring performance in health care, even in large population settings.[9,10,12,14,15]

These indicators offer several opportunities: they are simple indicators that can be measured repeatedly over time and can be implemented in different regions, as well shown in our study. This allows reproducibility and comparability to be maintained, while rapidly monitoring the diagnostic and therapeutic pathway of an oncology cohort at a population level. Moreover, these indicators afford the opportunity of observing the diagnostic and therapeutic pathway of a large number of patients with BC and its timing (eg, data regarding the timing of adjuvant systemic treatment and radiotherapy were available for > 10,000 and 5,000 patients, respectively). Therefore, these indicators provide substantial information on the general functioning of the health care system. Considering the capillarity of administrative data, the analysis reported here might be extended and performed at a national level to allow the assessment of discrepancies in BC care among different institutions. Moreover, a similar methodology might be applied to other malignancies.

This is of extreme importance, to allow stakeholders to obtain prompt feedback on adherence to guidelines and to monitor the impact of quality improvement initiatives. In our application, a clear example of this is shown by the observation of the extensive use of blood tumor markers in BC follow-up. National, international, and regional guidelines (the Italian Association of Medical Oncology; the European Society of Medical Oncology; PDTAs of single Italian regions) discourage the use of blood tumor markers in this setting; however, the proportion of patients undergoing this evaluation in the first year after BC diagnosis seems to be significantly higher than the 20% benchmark, which was established to account for stage IV patients and other specific conditions where markers might be indicated. In fact, clinical practice guidelines were systematically overlooked, and this was observed consistently throughout the five regions. Whether this is caused by a rejection of the guideline itself or by other factors is currently unknown, but this observation is a starting point to understand how to improve clinical practice. Indeed, the extensive use of blood tumor markers that was observed not only carries a burden in terms of economic costs for the National Health System (patients might have repeated the evaluation more than once and other examinations might have been triggered by the results), but also could cause potential damage to patients in terms of anxiety generated by false positive results and false reassurance produced by false negative results. Recognizing these wastages may be the first step of a process of reallocation of resources to higher-value procedures for patients, such as implementation of new treatments.

However, some limitations are also evident. In fact, not all indicators judged to be important to monitor the care process could be calculated, highlighting several issues related to the use of administrative data. In fact, some indicators were only available for some regions. For example, Breast Units are formally established only in some Italian regions, and the related indicator (I2) was evaluable only in these regions. Moreover, several indicators were not assessable because of the unavailability of data from the pathology laboratories. In other cases, such as in the proportion of patients with a previous histologic evaluation among patients undergoing surgery, data were computable but unreliable. In fact, administrative data can in some cases be incomplete, because detailed information regarding examinations performed before, during, or after hospitalization and related to the hospitalization, or second-level examinations performed as a follow-up to screening, are not included in the administrative information flow of outpatients’ records. Moreover, examinations paid for directly by patients are not reported by the management systems of the National Health System. This, combined with the inability to access data from pathology archives, made it impossible to calculate some of indicators, such as the percentage of patients undergoing surgery with a histologically confirmed diagnosis. In addition, indicators relating to treatments provided during the last month of life are computable only in the few regions where a Mortality Registry with systematic coding of the cause of death exists. The implementation of more recent and complex electronic patients’ records and the possibility of accessing some of the previously unavailable files (such as pathology archives) will certainly ensure data completion to a greater extent in the future.

Furthermore, the use of administrative data could also potentially lead to an underestimation of some indicators. In particular, because in some regions the date of radiotherapy recorded in the administrative flows often refers to the end of the therapy rather than its start, and because the therapy lasts approximately 5 weeks, a marked underestimation may occur, and this could account for the low value observed specifically in one region. The unexpectedly low rate of patients undergoing mammography 6 to 18 months after surgery (between 55.1% and 72.6%; benchmark ≥ 90%) could also be an underestimation. In fact, a consistent number of patients with BC might prefer to pay out-of-pocket money to undergo mammography where and when they prefer, and these re-evaluations might not be recorded in the National Health Service information flows.

The fact that only one half of the indicators originally designed to be extracted from administrative data were actually evaluable could also represent a drawback to the use of indicators to optimize quality of care, because this might focus stakeholders’ attention on calculable indicators and processes and distract attention from noncalculable indicators, despite their clinical relevance and impact on patient outcomes.

Moreover, benchmarking might also be critical. In this study, benchmark values for each indicator were initially defined by regional/local multidisciplinary groups and confirmed by the Indicators’ Working Group. However, there might be a lack of consensus on benchmarks for some indicators, and benchmarking should be used with caution when the reasons for exception to a guideline cannot be directly assessed. Indeed, administrative data do not allow the identification of subgroups of patients not receiving treatment because of medical contraindications or patient refusal.

In conclusion, despite highlighting some limitations in the use of administrative data to measure health care performance, this study shows that evaluating the quality of BC care at a population level is possible, with the goal of increasing the appropriateness of and adherence to guidelines and improving the quality of the care delivered, while controlling waste expenditures. Projects such as this are needed to trigger the implementation of administrative data collection, to include the information necessary to monitor processes and diseases and to increase the data availability from existing databases.