Validity of the Danish National Registry of Patients for chemotherapy reporting among colorectal cancer patients is high.

BACKGROUND: The Danish National Registry of Patients (DNRP) is a potentially valuable resource for monitoring national trends in the use of chemotherapy and evaluating the benefits and harms of alternative treatments among colorectal cancer (CRC) patients in Denmark. However, the validity of chemotherapy reporting in the DNRP is unknown. In this study, we evaluated the validity of the DNRP for identifying the receipt of chemotherapy and specific treatments, and the timing and number of treatments among CRC patients, using medical records and pharmacy data as the reference standard.
METHODS: We selected a random sample of CRC patients with lymph node involvement who were diagnosed at Aarhus University Hospital (n = 25) or Aalborg University Hospital (n = 25) from 2009 to 2010. Administration dates, specific treatments, and number of treatment courses were abstracted for the 180 days post diagnosis from the DNRP, medical records, and pharmacy production databases. The prevalence of chemotherapy, timing of first administration, and number of courses were described. DNRP data were compared with the reference standard for each hospital, and the kappa, sensitivity, specificity, positive and negative predictive values, and 95% confidence intervals were calculated for the receipt of any chemotherapy and specific treatments.
RESULTS: The prevalence of chemotherapy was 72% and 68% among CRC patients treated in Aarhus and Aalborg, respectively, with >90% of patients without distant metastasis receiving treatment within 90 days from diagnosis. Patients received on average 4.6 and 4.7 treatment courses in Aarhus and Aalborg, respectively. Kappa, sensitivity, and specificity of chemotherapy reporting in the DNRP was high (≥0.88), but the sensitivity of individual chemotherapies varied by hospital.
CONCLUSION: The validity of chemotherapy reporting in the DNRP was high, although some variation by hospital exists. The DNRP represents a population-based nationwide resource that can be used to provide timely and accurate evaluations of chemotherapy use among CRC patients in Denmark.

Introduction

In 2011, colorectal cancer (CRC) was the third most commonly diagnosed cancer among both men and women in Denmark.1 Treatment for CRC may include a combination of surgical resection, radiation therapy, and/or chemotherapy. When a CRC has spread to the lymph nodes, chemotherapy is typically indicated.2 Over the past ten years, randomized controlled trials have documented the efficacy of a number of new chemotherapeutic regimens for the treatment of CRC that have led to substantial improvements in survival and quality of life.3,4 Population-based medical registries, such as those developed in Denmark, may be a useful resource for monitoring the use and dissemination of new therapies into clinical practice, and provide a unique setting to evaluate their benefits and harms among cancer patients who are frequently underrepresented in randomized controlled trials including older adults and patients with multiple comorbidities.5

Until 2004, the Danish Cancer Registry routinely recorded whether patients diagnosed with cancer received various cancer-directed treatments (including chemotherapy) within 4 months from the date of diagnosis. However, detailed information regarding the use of specific regimens and dates of administration was not available and limited the utility of the data for research purposes. After 2004, the Danish Cancer Registry completely ceased to collect information regarding chemotherapy. As a result, alternative data sources for monitoring and evaluating the use of chemotherapy in the Danish population are indicated. The aim of this study was to evaluate the validity of the Danish National Registry of Patients (DNRP), a hospital-based medical registry, to identify the receipt of chemotherapy and specific treatments, as well as the timing and number of treatment courses among CRC patients using information from patient medical records and pharmacy production data as the reference standard.

Materials and methods

Data sources and validation sample

The Danish national health care system provides the entire Danish population with unrestricted access to tax-supported public health services.6 Through the use of a unique ten-digit civil registration number, medical registry and administrative data can be obtained and linked.7 Our study relied upon linkage of the Danish Cancer Registry, the DNRP, patient medical records, and two hospital pharmacy production databases located within Aarhus University Hospital and Aalborg University Hospital. These two large hospitals were selected because they provided a sufficient number of new CRC cases each year, and we obtained approval to access patient medical records and pharmacy production data from the regions where these hospitals are located for the conduct of validation studies.

We identified all patients with an incident CRC diagnosed between January 1, 2009 and December 31, 2010 from the Danish Cancer Registry and included those who had lymph node involvement using the tumor, node, metastasis (TNM) staging system (ie, N ≥ N1).8 We further restricted our sample to individuals who also had a diagnosis of CRC in the DNRP within 14 days of the diagnosis date reported by the Danish Cancer Registry, because this registry does not identify the hospital of diagnosis.9 Using this algorithm, we identified all patients who were diagnosed at Aarhus University Hospital (2009–2010) and Aalborg University Hospital (2010 available only). Due to resource constraints associated with the review of patient information, we selected a random sample of 25 patients from each hospital (Figure 1) to include in our analysis. Because cancer care is centralized at the regional level in Denmark, it is reasonable to assume that if a patient was diagnosed with cancer at a hospital in Aarhus or Aalborg, then that patient would receive their chemotherapy at the same hospital. Aarhus is a university town and as a result the background population is relatively young compared with Aalborg. Aarhus University Hospital performs surveillance examinations for patients with inflammatory bowel disease and inherited conditions, such as Lynch syndrome, while diagnostic work-up of symptomatic patients is performed at other hospitals in the region. As such, the patient population diagnosed with CRC at Aarhus University Hospital tends to be slightly younger than the population diagnosed at Aalborg University Hospital. Surgery and chemotherapy for CRC were also provided outside of university centers in the county hospitals in Viborg, Herning, Horsens (colectomy only), and Hjørring during the study period; however, patients diagnosed with CRC at these hospitals were not included in the present study.

Figure 1

Study flow chart of patients selected for inclusion in the validation study.

Notes: aRegional lymph node involvement was identified using the tumor, node, metastasis (TNM) staging system (N ≥ N1); busing an administrative algorithm requiring a hospital diagnosis of colorectal cancer (International Classification of Diseases (ICD)-10 codes: C18-20) ± 14 days from the diagnosis date, as reported by the Danish Cancer Registry.

Reporting of chemotherapy from DNRP, medical records, and pharmacy production system

For each CRC patient, we obtained chemotherapy information for the 180 days following the diagnosis date as reported by the Danish Cancer Registry (the “study period”) using the DNRP, the patient medical record, and the hospital pharmacy production system. We selected this study period because most patients should initiate their first course of adjuvant or postoperative chemotherapy during this time period (if indicated), and it would be unlikely to observe treatment for recurrence or progression during this window. We were particularly interested in initial treatments containing: 5- fluorouracil (5-FU) or capecitabine only, oxaliplatin in addition to 5-FU (FOLFOX) or capecitabine (XELOX) without bevacizumab, and bevacizumab in addition to FOLFOX, XELOX, irinotecan + 5-FU (FOLFIRI), or 5-FU. These three groups are mutually exclusive.

In Denmark, oncology care is delivered in hospitals, and treatments and procedures are coded in the DNRP using Danish treatment codes. We retrieved all patient records with treatment codes indicating chemotherapy administration or the use of specific agents and regimens for the treatment of CRC during the study period (Supplementary Table 1). Electronic patient medical records were reviewed by a physician (TD), and chemotherapy data including the receipt of any chemotherapy and specific treatments, the dates of first administration of each treatment, and the number of treatment courses received during the study period were abstracted. Finally, we obtained data from the Aarhus University Hospital Pharmacy and Aalborg University Hospital Pharmacy production systems, which record orders placed by oncologists at the hospital for production in the hospital pharmacy. Data regarding the specific agents and regimens and dates of production were obtained for each patient during the study period.

Comparison of chemotherapy reporting in DNRP, medical records, and pharmacy production system

During the majority of the study period, the pharmacy production systems did not record oral chemotherapy agents, because they were disbursed to the hospital in bulk orders and not on a per-patient basis. Because the patient medical record captures oral chemotherapy use, we combined these two data sources to serve as the reference standard for all of the validation analyses. For the description of the number of treatment courses administered during the study period, we relied solely on the pharmacy production data (excluding those treated with oral chemotherapy alone), because reporting of production dates was most straightforward.

We reported measures of validity for chemotherapy reporting in the DNRP compared with the reference standard. Specifically, we estimated the sensitivity, specificity, positive and negative predictive values, and their corresponding 95% confidence intervals. We also reported overall agreement comparing the two data sources using Cohen’s kappa and corresponding 95% confidence intervals. We evaluated reporting of the receipt of any chemotherapy and specific treatments. The time from cancer diagnosis to treatment was described for each data source among patients without distant metastasis in the sample (≤30 days, 31–60 days, 61–90 days, and 91+ days). In addition, the number of months with any reported treatment was used as a proxy for the number of treatment courses completed,10 and we described this distribution within the study sample using the pharmacy production data and the DNRP (excluding those treated with oral chemotherapy only). All analyses were performed using SAS version 9.2 (SAS Institute Inc, Cary, NC, USA). This study was approved by the Danish Data Protection Agency (2006-53-1346 and 2006-53-1396).

Results

We identified 8,251 individuals with an incident CRC diagnosed between January 1, 2009 and December 31, 2010 in Denmark. Among the 3,104 patients who had regional lymph node involvement, 3,030 could be matched to a hospital where they were diagnosed. There were 114 patients diagnosed at Aarhus University Hospital between 2009 and 2010, and 101 patients diagnosed at Aalborg University Hospital in 2010. Our study sample of 50 patients was randomly selected from this group (Figure 1). The median age at diagnosis was slightly higher for patients diagnosed in Aalborg compared with those diagnosed in Aarhus (70 years versus 68 years) and lower for those receiving chemotherapy compared with those who did not (Table 1). There were more rectal cancers included in the Aalborg sample compared with the Aarhus sample (56% versus 40%). The majority of CRCs in the overall sample had advanced stage (T3–T4, 90%), had four or more lymph nodes involved (N2, 62%), but did not have distant metastases (M0, 66%). According to the reference standard, the prevalence of chemotherapy was 72% and 68% in Aarhus and Aalborg, respectively.

Table 1

Characteristics of 50 randomly selected patients diagnosed with colorectal cancer in Denmark by hospital, 2009–2010

Characteristic	Aarhus University Hospital						Aalborg University Hospital
	Total sample		Chemotherapy received (reference standard)				Total sample		Chemotherapy received (reference standard)
	n	Col %	No	Row %	Yes	Row %	n	Col %	No	Row %	Yes	Row %
	n = 25		n = 7		n = 18		n = 25		n = 8		n = 17
Age, mean (SD)	68 (12)		74 (13)		64 (9)		70 (14)		79 (8)		65 (14)
Female	12	48	4	33	8	67	8	32	2	25	6	75
Colon	15	60	3	20	12	80	11	44	4	36	7	64
Rectum	10	40	4	40	6	60	14	56	4	29	10	71
Tumor invasion
T1	1	4	0	0	1	100	0	0	0	–	0	–
T2	1	4	0	0	1	100	0	0	0	–	0	–
T3	12	48	3	25	9	75	16	64	7	44	9	56
T4	9	36	3	33	6	67	8	32	0	0	8	100
NOS	2	8	1	50	1	50	1	4	1	100	0	0
Nodal involvement
N1	11	44	4	36	7	64	8	32	3	38	5	63
N2	14	56	3	21	11	79	17	68	5	29	12	71
Metastases
M0	15	60	4	27	11	73	18	72	7	39	11	61
M1	9	36	2	22	7	78	6	24	0	0	6	100
NOS	1	4	1	100	0	0	1	4	1	100	0	0
Chemotherapy type
No chemotherapy	7	28					8	32
5-FUa	2	8					8	32
Oxaliplatin-basedb	12	48					3	12
Bevacizumabc	4	16					6	24
Treatment months, mean (SD)d	–		–		4.6 (1)		–		–		4.7 (1)

Notes:

Includes administration of 5-FU or capecitabine alone;

includes administration of oxaliplatin in combination with 5-FU (FOLFOX) or capecitabine (XELOX), but not with bevacizumab;

includes administration of bevacizumab in combination with FOLFOX, XELOX, irinotecan, and 5-FU (FOLFIRI), or 5-FU;

based on pharmacy production data only, excluding patients reporting treatment with capecitabine only in the DNRP.

Abbreviations: Col %, column percentage; DNRP, Danish National Registry of Patients; Row %, row percentage; 5-FU, 5-fluorouracil; NOS, not otherwise specified; SD, standard deviation.

The comparisons of any chemotherapy and specific treatments reported by the reference standard and the DNRP for each hospital are shown in Table 2. Overall, the measures of validity and agreement for identifying the receipt of any chemotherapy were high. For Aarhus University Hospital, the sensitivity, specificity, positive predictive value, negative predictive value, and kappa were 94%, 100%, 100%, 87%, and 90%, respectively, and for Aalborg University Hospital were 100%, 88%, 94%, 100%, and 90%, respectively.

Table 2

Measures of agreement and validity comparing reporting of chemotherapy in the Danish National Registry of Patients with the reference standard by hospital

Treatment	Source of reporting				Sensitivity 95% CI	Specificity 95% CI	PPV 95% CI	NPV 95% CI	Kappa 95% CI
	Ref YDNRP Y	Ref NDNRP N	Ref YDNRP N	Ref NDNRP Y
Aarhus
Chemotherapy	17	7	1	0	0.94(0.84–1.00)	1.00	1.00	0.87(0.65–1.00)	0.90 (0.72– 1.00)
5-FUa	0	23	2	0	0.00	1.00	–	0.92(0.81–1.00)	–
Oxaliplatinb	3	13	9	0	0.25 (0.05–0.50)	1.00	1.00	0.59 (0.39–0.80)	0.26 (0.00–0.52)
Bevacizumabc	3	21	1	0	0.75(0.33–1.00)	1.00	1.00	0.95(0.87–1.00)	0.83 (0.52–1.00)
Aalborg
Chemotherapy	17	7	0	1	1.00	0.88(0.65–1.00)	0.94(0.84–1.00)	1.00	0.90(0.72–1.00)
5-FUa	8	16	0	1	1.00	0.94(0.83–1.00)	0.89(0.68–1.00)	1.00	0.91 (0.74–1.00)
Oxaliplatinb	3	22	0	0	1.00	1.00	1.00	1.00	1.00
Bevacizumabc	6	19	0	0	1.00	1.00	1.00	1.00	1.00

Notes:

Includes administration of 5-FU or capecitabine without oxaliplatin or bevacizumab;

Includes administration of oxaliplatin in combination with 5-FU (FOLFOX) or capecitabine (XELOX), but without bevacizumab

Includes administration of bevacizumab in combination with FOLFOX, XELOX, irinotecan and 5-FU (FOLFIRI), or 5-FU.

Abbreviations: CI, confidence interval; Ref, reference standard; Y, yes; N, no; DNRP, Danish National Registry of Patients; PPV, positive predictive value; NPV, negative predictive value; 5-FU, 5-fluorouracil.

The measures of validity and agreement for specific treatment reporting varied by treatment and were lower for patients diagnosed at Aarhus University Hospital compared with those diagnosed at Aalborg University Hospital (Table 2). For patients diagnosed at Aarhus, the validity of bevacizumab reporting in the DNRP was high, with a sensitivity, specificity, positive predictive value, and negative predictive value of 75%, 100%, 100%, and 95%, respectively. Reporting of all specific treatments from the DNRP among patients diagnosed at Aalborg resulted in perfect sensitivity of 100%, specificity ≥94%, and kappa ≥91%.

Consistent with current treatment guidelines,2 adjuvant chemotherapy was initiated within 90 days from the diagnosis date in >90% of the non-metastatic CRC patients in both hospitals, regardless of the data source used for reporting (Figure 2). After excluding patients receiving oral chemotherapies exclusively, 73% and 43% of colon cancer patients and 33% and 80% of rectal cancer patients treated in Aarhus and Aalborg, respectively, completed five or more courses of treatment according to the pharmacy production data. Overall, the DNRP reported a similar distribution of the number of treatment courses administered by CRC site as compared with the pharmacy production data (Figure 3).

Figure 2

Proportion of patients with non-metastatic disease according to time between diagnosis and first administration of chemotherapy, by data source and hospital. Aarhus University Hospital is shown in the left panel, while Aalborg University Hospital is shown in the right panel. Date of diagnosis was defined according to the Danish Cancer Registry.

Abbreviation: DNRP, Danish National Registry of Patients.

Figure 3

Proportion of patients completing a range of chemotherapy courses within 180 days from cancer diagnosis by hospital and cancer site, excluding patients treated with oral chemotherapy only. The number of completed courses is estimated as number of months with at least one record of administration of chemotherapy. Treatment data from the DNRP are shown in black and the pharmacy production data are shown in gray.

Abbreviation: DNRP, Danish National Registry of Patients.

Discussion

We found that the DNRP accurately recorded the receipt of chemotherapy administered within 180 days from diagnosis of colorectal cancer, as well as the date of first chemotherapy administration and number of treatment courses when compared with the treatment information obtained from the patient medical record and pharmacy production systems. However, the ability of the DNRP to record the use of specific chemotherapy treatments varied by hospital, where the sensitivity of individual chemotherapy agent reporting was higher among patients treated in Aalborg compared with Aarhus.

This is the first study to assess the validity of the DNRP for identifying chemotherapy treatment in CRC patients and as such provides a framework for future studies seeking to determine the accuracy of treatment coding for other cancer sites. Prior studies in the United States have shown similar results when examining the validity of administrative data, such as Medicare claims, to identify the receipt of chemotherapy among older CRC patients.11,12 These data have become widely used to describe trends in the utilization and comparative effectiveness and safety of a variety of treatments over time in the United States, but are limited to patients diagnosed with cancer at age 65 years and older.13–17

A major strength of the current study is our ability to identify and link population-based patient information across multiple data sources by using the Danish CPR number. As a result, we were able to obtain data from complementary sources including patient medical records and pharmacy production data to construct a robust reference standard. The DNRP is not restricted to specific patient populations (eg, the elderly), as is often the case in the United States, and thus can provide more generalizable inferences regarding chemotherapy use and effectiveness for CRC patients of all ages.

Our study has some limitations. We were only able to validate chemotherapy administered in two large hospitals in Denmark for the time period from 2009 to 2010, which may impact the generalizability of our findings to other hospitals and time periods (ie, 2004–2008). However, oncology care in Denmark is considered to be relatively uniform due to the nationalized structure of health care and the National Cancer Plan I and II,18,19 and as such, treatment protocols and standards should be similar across hospital settings. We observed that recording of particular chemotherapy treatments varied according to hospital, and as such analyses seeking to identify the use of these treatments will be limited to specific hospitals. In addition, our analysis did not directly examine the reliability of chemotherapy reporting in the DNRP. However, analysis of the number of treatment courses indirectly provides evidence that the DNRP reliably captures chemotherapy treatment over the study period, given that the distribution in the two data sources is similar. Finally, we relied upon data regarding nodal involvement from the Danish Cancer Registry, which is known to be incomplete.20 However, this is unlikely to impact the generalizability of our findings because treatment reporting in the DNRP is unlikely to be related to the completeness of TNM reporting in the Danish Cancer Registry.

In conclusion, we assessed the validity of the DNRP for identifying the receipt of any chemotherapy and specific treatments. Generally, the DNRP accurately identified the receipt of any chemotherapy. However, the sensitivity of specific treatment reporting depended upon individual hospital coding practices. The DNRP, in combination with data from the Danish Cancer Registry and other medical registries and clinical quality databases, provides investigators with a unique opportunity to conduct high quality health services and cancer outcomes research.

Treatment codes used to identify chemotherapy for the treatment of colorectal cancer in the DNRP

Type of chemotherapy	DNRP treatment code
Any chemotherapy for colorectal cancer	BWHA1, BWHA2, BOHJ19B
5-FU or capecitabine	BWHA110, BWHA123
Oxaliplatin alone	BWHA108
Oxaliplatin + 5-FU (FOLFOX)	BWHA231
Oxaliplatin + capecitabine (XELOX)	BWHA222
Irinotecan alone	BWHA212
Irinotecan + 5-FU (FOLFIRI)	BWHA212
Bevacizumab alone	BOHJ19B

Abbreviations: 5-FU, 5-fluorouracil; DNRP, Danish National Registry of Patients.