Validity of the National Death Index to ascertain the date and cause of death in men having undergone prostatectomy for prostate cancer.

BACKGROUND: The National Death Index (NDI) is a centralized database containing information from death certificates that are frequently referenced by health and medical investigators to ascertain vital statistics. Yet, it commonly includes misclassified causes of death. Since the NDI is frequently relied upon in studies that evaluate outcomes following radical prostatectomy (RP) for prostate cancer (PC), we evaluated its validity by referencing mortality data from the Shared Equal Access Regional Cancer Hospital (SEARCH) database which is a prospectively managed database of 5009 Veterans who underwent a RP at eight Veterans Affairs medical centers between 1982 and 2016.
METHODS: We compared vital status, cause of death and date of death from the SEARCH database with the NDI.
RESULTS: A total of 1312 men in SEARCH were deceased, yet the NDI reported 17% (219) of those men as still alive. Among the 1093 men who had concordant vital status in both SEARCH and NDI, the date of death was an exact match within one day, a week, or 31 days in 94%, 97%, 99%, and 100%, respectively. Of those men coded as dying from prostate cancer in the SEARCH database (n = 105), 12% were coded as having died from non-PC causes in the NDI. Meanwhile, among patients coded by the NDI as having died of PC (n = 139), 34% were coded in SEARCH as having died of non-PC causes.
CONCLUSIONS: These findings demonstrate that the NDI provides accurate dates of death, but frequently misclassifies whether a death was due to prostate cancer. Studies that rely upon death certificates, as captured in the NDI, may be unreliable to report prostate cancer-specific mortality rates after prostatectomy.

Retrospective studies that analyze mortality outcomes frequently rely on the National Death Index (NDI) to ascertain vital status, as well as date and cause of death.[1-3] It is managed by the US National Center for Health Statistics, and since 1979 has been used in more than 1,000 health research publications.[4] Information on the cause of death in this database relies upon death certificates that frequently have classification errors, and are almost always completed by individuals who did not have a formal clinical relationship with the decedent. Yet, investigations that evaluate prostate cancer-specific mortality (PCSM) rates commonly reference data sources that rely upon death certificates.[5-9] This is even though a review by medical examiners has found that up to 60% of death certificates contain errors in the underlying cause-of-death code.[10] This high error rate has been attributed to the characteristics of death certifiers,[11] and has led to concerns about the accuracy of cancer-specific mortality statistics that have been found to be erroneous in the literature.[12,13] Herein, we investigate the impact of this known misclassification of cause of death on studies evaluating PCSM following prostatectomy.

The Shared Equal Access Regional Cancer Hospital (SEARCH) database is a retrospective, IRB-approved database that maintains records on 5,009 Veterans who underwent a radical prostatectomy (RP) at eight Veterans Affairs Medical Centers between 1982 and 2016. It is routinely updated with manual reviews of individual patient electronic medical records available through the Veterans Health Administration (VHA) that includes information on labs, pathology, radiology, and clinical notes, and is coded by personnel who are trained by the SEARCH team. Data are abstracted according to detailed guidelines that were developed by urologic oncologists, and 10% of cases are routinely audited to ensure the validity and consistency of data collection. Cause of death in the SEARCH database is coded as “prostate cancer” whenever there was progressive metastatic disease following hormonal therapy without another obvious cause of death, “other” whenever a death related to prostate cancer could be ruled out, and “unknown or other” whenever information was incomplete. Whenever in doubt, cases are reviewed by a urologic oncologist who specializes in prostate cancer (SJF, WJA).

We cross-referenced data from the SEARCH database coded between 1989–2011 with the NDI for vital status, cause of death, and date of death. Social security numbers were used to reconcile this information, and the analysis relied on ICD-9 codes to match the cause-of-death. A total of 1,327 and 1,093 patients were coded as deceased in SEARCH and NDI, respectively. All deceased patients in the NDI were coded as dead in SEARCH, though 17% (219) of deaths in SEARCH were listed as alive in the NDI (see Table 1). This rate of incongruence for vital status decreased to 4% after 1998 (see Figure 1). For patients with concordant death status, the dates were an exact match, within one day, within a week, or within 31 days for 94%, 97%, 99%, and 100%, respectively.

Table 1.

Cross tabulation of death status in SEARCH vs. NDI.

	NDI DEAD	NDI ALIVE	Total
SEARCH DEAD	1093	219	1312
SEARCH ALIVE	0	3604	3604
Total	1093	3823	4,916

Figure 1.

Histogram showing the number of death statuses that agree between SEARCH and NDI and deaths missed by NDI between 1988 and 2011.

NDI and SEARCH were in concordance on 941 deaths from other causes and 92 PC deaths (see Table 2). However, 13 patients were misclassified by SEARCH as having died of prostate cancer and 47 were misclassified by NDI as dying from non-PC causes. Of the 104 patients coded in SEARCH as dead from PC, the NDI coded 77% accurately, with 11% as non-PC, and 12% as alive. Of the 139 patients coded in the NDI as dead from PC, SEARCH confirmed 66% to be accurate, with 34% as non-PC, and 0% as alive. Dropping patients with an unknown cause of death in either source, the positive predictive value for PC death in NDI was 72% and negative predictive value was 98%.

Table 2.

Cross tabulation of cause of death in SEARCH vs. NDI.

	NDI PC Death	NDI non-PC Death	Total
SEARCH PC Death	92	13	105
SEARCH non-PC Death	47	941	988
Total	139	954	1,093

Previous evaluations have shown variable rates of agreement between the cause of death coded in death certificates versus available hospital records. Uncertainties during completion of death certificates and transcription errors contribute to challenges in coding cause of death. Yet, challenges are present even in prospective clinical trials when complete clinical records are available. For example, the randomized PIVOT trial utilized an Endpoints Committee comprising an internist and two urologists to assign the cause of death. Each committee member was presented with “death summary packets” that included a death certificate; a brief report of the site investigator’s conclusions regarding cause of death; study PSA values; study bone scan results; and relevant medical records, including discharge summaries, radiology reports, and pathology reports, particularly when another type of cancer was suspected. Even with this amount of detailed clinical information, the committee members agreed in only 56% of cases before discussing whether the death was “definitely” or “probably” due to prostate cancer or treatment. Agreement increased to 86% when the “definitely” and “probably” assignments were collapsed (a strategy that was defined a priori), maintaining concerns about the reliability of the PCSM endpoint even in this phase III trial. As this experience indicates, ascertaining the of cause of death, even in a prospective clinical trial, can be challenging and is an inexact science. As such, there is the possibility of misclassification in SEARCH as well, given clinical records may be missing for decedents who received their care outside the VA at the end of life. There may be scenarios where the individuals completing death certificates had access to more meaningful clinical records just before death, though our results included an analysis that excluded patients in SEARCH that had an “unknown” cause of death.

Using cause-specific survival, as opposed to overall survival, to evaluate treatment efficacy has the advantage of minimizing biases introduced by competing causes of death. However, deaths from prostate cancer are frequently misclassified, especially in databases such as the NDI which rely on death certificates, and thus studies using a PCSM endpoint should be interpreted with caution.