Comparison of Expert Adjudicated Coronary Heart Disease and Cardiovascular Disease Mortality With the National Death Index: Results From the REasons for Geographic And Racial Differences in Stroke (REGARDS) Study.

BACKGROUND: The National Death Index (NDI) is widely used to detect coronary heart disease (CHD) and cardiovascular disease (CVD) deaths, but its reliability has not been examined recently. METHODS AND
RESULTS: We compared CHD and CVD deaths detected by NDI with expert adjudication of 4010 deaths that occurred between 2003 and 2013 among participants in the REGARDS (REasons for Geographic And Racial Differences in Stroke) cohort of black and white adults in the United States. NDI derived CHD mortality had sensitivity 53.6%, specificity 90.3%, positive predictive value 54.2%, and negative predictive value 90.1%. NDI-derived CVD mortality had sensitivity 73.4%, specificity 84.5%, positive predictive value 70.6%, and negative predictive value 86.2%. Among NDI-derived CHD and CVD deaths, older age (odds ratios, 1.06 and 1.04 per 1-year increase) was associated with a higher probability of disagreement with the adjudicated cause of death, whereas among REGARDS adjudicated CHD and CVD deaths a history of CHD or CVD was associated with a lower probability of disagreement with the NDI-derived causes of death (odds ratios, 0.59 and 0.67, respectively).
CONCLUSIONS: The modest accuracy and differential performance of NDI-derived cause of death may impact CHD and CVD mortality statistics.

Introduction

The National Death Index (NDI) collects information on deaths occurring in the United States based on state death certificate data.1 The NDI has been an important tool to detect all‐cause, coronary heart disease (CHD), and cardiovascular disease (CVD) deaths in research study participants.2, 3 In addition, death certificates are used to calculate national vital statistics, including trends and disparities in CHD and CVD mortality over time. The NDI is a national source of mortality data, and its rules for selecting underlying cause of death act as a means of standardizing classification, which contributes to comparability and uniformity in mortality statistics.1

NDI is accurate in ascertaining vital status.4, 5, 6 Through automated processing and analysis of conditions listed on the death certificate, the NDI Plus system identifies an underlying cause of death, defined by the World Health Organization as “(a) the disease or injury which initiated the train of events leading directly to death, or (b) the circumstances of the accident or violence which produced fatal injury.”7 Causes of death are further converted into International Classification of Diseases, Tenth Revision codes.1 Sathiakumar et al, while assessing the accuracy of NDI Plus cause of death, reported a 4% discrepancy rate in determining all‐cause, but not CHD or CVD, mortality between the NDI and trained study nosologists who examined death certificates.8 However, death certificates may not be accurate.9, 10 The reliability of the cause of death as documented on death certificates, which propagate into NDI data, has been shown to be variable with poor reliability and error rates ranging between 16% and 40%.9, 10

The NDI's algorithmically determined underlying cause of death may not agree with the judgment of expert adjudicators using all available data, including reports of next of kin, about the circumstances surrounding the death and medical records, which may not be available to the physician filling out the death certificate.

Disagreement between the NDI and adjudicated causes of death may be more likely for some individuals than others, which may have impact on studies of disparities and studies of risk factors for CHD and CVD death. In particular, people with known chronic health conditions and those who die in hospital may have more accurate information on death certificates because of contact with the healthcare system, whereas older adults may have CHD or CVD listed as the underlying cause of death by default. A previous study in France compared official mortality statistics and cohort study classifications for causes of death and found differences for CVDs, with older age being an important predictor of lower accuracy.11 We are not aware of comparable US studies.

Therefore, to examine the accuracy of contemporary NDI determination of CHD and CVD as the main underlying causes of death in a US cohort, we compared the expert adjudication‐derived main underlying causes of death as the gold standard with the NDI annotated main underlying causes of death with deaths that occurred between 2003 and 2013 in the REGARDS (REasons for Geographic And Racial Differences in Stroke) study population. We also examined characteristics that correlate with discordance between NDI and expert adjudication.

Methods

Study Population

The REGARDS study is a population‐based longitudinal study that recruited 30 239 black and white participants aged ≥45 years, residing in the 48 continental United States between 2003 and 2007.12 Participants were randomly selected with the goal of recruiting half (50%) of the study population from the stroke buckle (coastal North and South Carolina and some parts of Georgia) and stroke belt (remaining areas of North Carolina, South Carolina, and Georgia, Tennessee, Mississippi, Alabama, Louisiana, and Arkansas) regions and the other half (50%) from the rest of the continental United States.12 Blacks were oversampled by design. Baseline data were collected by telephone interviews followed by in‐home visits. The in‐home visit, conducted 3 to 4 weeks after the telephone interview, included a brief physical exam with collection of height and weight, blood pressure measurements, blood and urine samples, and an ECG. The REGARDS study was approved by institutional review boards at participating centers, and all participants provided written informed consent.

Every 6 months, participants or proxies were interviewed by telephone calls to determine hospital visits or death events. In addition, some deaths events were identified when family members or friends called the REGARDS toll‐free numbers to report the death.13 Other death events were identified through searches of the Social Security Administration's Master Death File. For the calculation of mortality rates, we included all REGARDS participants with valid follow‐up (n=29 704; Figure 1). Through December 31, 2013, 5418 REGARDS participants had died. To assess agreement between NDI and adjudicated causes of death, we included REGARDS participants who died on/before December 31, 2013 and had NDI records and additional information available to adjudicate the cause of death. We excluded 295 cases that did not have both REGARDS adjudication and NDI records available, and an additional 1113 REGARDS cases adjudicated with only NDI information, leading to a final analytic sample size of 4010. Participants included in the analysis were more likely to be white, higher income, reside in higher income areas, and to have a history of CHD and less likely to die out of hospital compared with excluded participants (Table 1).

Figure 1

Exclusion cascade to define analytic samples. NDI indicates National Death Index; REGARDS, REasons for Geographic And Racial Differences in Stroke.

Table 1

Comparison of Included and Excluded Deaths Among REGARDS Participants

Characteristic	All Deaths Among REGARDS Participants Through December 31, 2013 (n=5418)	Deaths With NDI and Other Information Included in Current Analysis (n=4010)	NDI Only and Deaths Missing NDI Excluded From Current Analysis (n=1408)	P Valuea
	N (%)	N (%)	N (%)
Age at death, y (%)				0.16
45 to 54	84 (1.6)	59 (1.5)	25 (1.8)
55 to 64	617 (11.4)	442 (11.0)	175 (12.4)
65 to 74	1538 (28.4)	1122 (28.0)	416 (29.5)
75+	3179 (58.7)	2387 (59.5)	792 (56.3)
Race‐sex group (%)				<0.001
White men	1888 (34.8)	1537 (38.3)	351 (24.9)
Black men	1174 (21.7)	818 (20.4)	356 (25.3)
White women	1165 (21.5)	901 (22.5)	264 (18.8)
Black women	1191 (22.0)	754 (18.8)	437 (31.0)
Income, $USD (%)				<0.001
<$20 000	1527 (32.7)	1021 (29.4)	506 (42.0)
$20 to $34 000	1575 (33.7)	1147 (33.1)	428 (35.5)
$35 to $74 000	1196 (25.6)	981 (28.3)	215 (17.8)
≥$75 000	377 (8.1)	321 (9.3)	56 (4.6)
Not reported	743 (13.7)	540 (13.5)	203 (14.4)
Poverty tertile (census tract) (%)				<0.001
Low poverty	1676 (34.1)	1324 (36.3)	352 (27.8)
Medium	1653 (33.7)	1222 (33.5)	431 (34.0)
High poverty	1583 (32.2)	1098 (30.1)	485 (38.2)
Diabetes mellitus (%)	2002 (37.0)	1466 (36.6)	536 (38.1)	0.31
Hypertension (%)	3872 (71.5)	2866 (71.5)	1006 (71.4)	0.99
Hyperlipidemia (%)	4127 (76.2)	3074 (76.7)	1053 (74.8)	0.16
History of CHD (%)	2285 (42.2)	1772 (44.2)	513 (36.4)	<0.001
Site of death (%)				<0.001
Out of hospital	3921 (72.9)	2639 (65.8)	1282 (93.4)
In hospital	1461 (27.1)	1371 (34.2)	90 (6.6)

CHD indicates coronary heart disease; NDI, National Death Index; REGARDS, REasons for Geographic And Racial Differences in Stroke; USD, US dollars.

P value denotes chi‐squared tests to assess statistical significance between groups.

Characteristics of Participants

During the baseline telephone interview, participants reported age, sex, race, and annual household income (<$20 000/year, $20 000–$34 999/year, $35 000–$74 999/year, ≥$75 000/year, or declined to report income). Geographical location of residence was derived from participants' addresses and was linked to Census tract information on percent of the population living in poverty (grouped into tertiles). We assessed baseline hypertension (self‐reported diagnosis or medication use, or elevated blood pressure), hyperlipidemia (low‐density lipoprotein cholesterol ≥100 mg/dL or statin use), and diabetes mellitus (self‐reported diagnosis or elevated blood glucose). History of CHD was defined as self‐reported myocardial infarction, coronary artery revascularization, or evidence of myocardial infarction on ECG at baseline, or myocardial infarction or coronary artery revascularization ascertained during follow‐up. History of CVD was defined as history of CHD described above, self‐reported stroke, aortic aneurysm, or peripheral arterial disease at baseline, or heart failure hospitalization ascertained during follow‐up. Site of death (in‐hospital or out‐of‐hospital) was derived from proxy interviews and death certificates.

Clinician Adjudication of Cause of Death

Death records and medical charts were retrieved from hospitals based on the information on site of death. Participant proxies (next of kin, family members, or close friends) were also interviewed to obtain information about the events surrounding the death.13 Adjudication was carried out by clinicians (general internists, cardiologists, and physician assistants) who had undergone training to identify causes of death. The methods used in REGARDS are similar to those used in other studies, such as the Atherosclerosis Risk in Communities Study14 and the Women's Health Initiative,15 and those recommended in national consensus guidelines.16 Each event was adjudicated independently by 2 expert clinician adjudicators who conducted their reviews using all information available at the time of review, including the participant's clinical baseline characteristics (100%), death certificates (32.9%), proxy interviews (71.9%), emergency medical services report (1.2%), autopsy findings (1.1%), NDI (100%), and medical records from recent hospital admissions (30.8%). After careful individual review to establish the main underlying cause of death, adjudicators had to agree on the primary cause of death. Disagreements were resolved by consensus of the entire end points adjudication committee.

Underlying causes of death were categorized as: (1) definite, probable, or possible myocardial infarction as defined in national guidelines16, 17; (2) stroke, World Health Organization–defined, clinical, and probable stroke18; (3) sudden death defined as sudden unexpected death without evidence of a noncardiac cause13; (4) heart failure; (5) other cardiac causes like myocarditis; (6) not cardiac, but other cardiovascular diseases (eg, ruptured aortic aneurysm); (7) cancer; (8) accident/injury/suicide/homicide; (9) liver disease; (10) infection; (11) end‐stage renal disease; (11) dementia; (12) chronic lung diseases; (13) pulmonary embolism; (14) other noncardiac, nonstroke death that required evidence of a noncoronary and nonstroke cause of death; and (15) unclassifiable, which was used when there was insufficient information to determine whether the death was a CVD death or a non‐CVD death; other ill‐defined and unknown causes of morbidity or mortality; and no evidence of a noncoronary cause of death. Of the above categories, causes of death that comprised CHD included definite, probable, or possible myocardial infarction and sudden death, whereas CVD mortality categories included World Health Organization–defined, clinical, and probable stroke; definite, probable, or possible myocardial infarction; sudden death, heart failure, pulmonary embolism, other cardiac (eg, myocarditis); and not cardiac, but other cardiovascular diseases (eg, ruptured aortic aneurysm).

NDI Cause of Death

Annually, participant Social Security Numbers, names, and dates of birth were submitted to NDI. For deceased participants, the NDI supplied the name, date of birth, age, sex, race, date of death, and the underlying cause of death as International Classification of Diseases, Tenth Revision codes. The participant's Social Security number was used as the primary matching criterion whereas names and dates of birth were used in cases of inconsistency in the reported Social Security number. The NDI derived cause of death by using the super medical indexing, classification, and retrieval (SuperMICAR) computer software system. This system automatically coded multiple cause of death text reported on the death certificate and determined the underlying cause of death in accordance to World Health Organization rules and regulations.1 International Classification of Diseases, Tenth Revision codes used for CHD cause of death were I20–I25, I46, and I49, whereas those used for CVD cause of death were I00 to I99.

Statistical Analysis

We calculated the sensitivity (SN), specificity, positive predictive value (PPV), and negative predictive value (NPV) for NDI‐determined CHD death, with REGARDS adjudicated cause of death as the gold standard. We calculated descriptive statistics of participant characteristics by categories of agreement/disagreement between the REGARDS expert‐adjudicated CHD death and the NDI‐determined CHD death. We considered events as false negative when REGARDS adjudication adjudicated cause of death as CHD and the NDI listed a different cause and false positive when NDI listed cause of death as CHD and REGARDS adjudicated a different cause. We considered events as true positive when both the NDI and REGARDS adjudication listed CHD as cause of death and true negatives when they both did not. Chi‐square tests were used to assess statistical significance of differences between groups. Then, we identified the cause of death for disagreements between REGARDS and NDI where 1 source identified CHD as the cause of death. We used logistic regression to examine characteristics associated with disagreement, mutually adjusting for age at death, race‐sex groups, income, poverty level, chronic medical conditions (diabetes mellitus, hypertension, and hyperlipidemia), history of CHD at baseline or during follow‐up, and site of death. In the first set of models, we limited the analysis cohort to individuals with REGARDS adjudicated CHD deaths; the outcome of interest was having a different cause of death in NDI records (false‐negative CHD deaths). In the second set of models, the analysis cohort was limited to individuals with cause of death listed as CHD in NDI; the outcome of interest was a REGARDS‐adjudicated cause of death other than CHD (false‐positive CHD‐deaths). Rates of CHD mortality as determined by REGARDS and NDI were calculated as the number of cases divided by the person‐time at risk in the 29 704 REGARDS participants with valid follow‐up. Mortality rates were calculated overall and by age at baseline (<75 or ≥75 years), hypertension, diabetes mellitus, and history of CHD. Analyses were repeated for CVD mortality.

Two‐sided P values of <0.05 were considered statistically significant. All analyses were carried out using SAS software (version 9.3; SAS Institute Inc, Cary, NC).

Results

CHD Mortality

There were 379 NDI‐CHD deaths among 707 REGARDS‐adjudicated CHD deaths (SN 53.6%), 2983 NDI non‐CHD deaths among 3303 adjudicated non‐CHD deaths (specificity 90.3%), and PPV and NPV were 54.2% and 90.1%, respectively (Table 2). Overall concordance percentage was 84%. Test characteristics were similar in black and white participants.

Table 2

Operating Characteristics for NDI Determination of CHD and CVD Compared to Expert Adjudication

Overall
	REGARDS=CHD	REGARDS=OTHER
NDI=CHD	379	320	699 PPV 54.2%
NDI=OTHER	328	2983	3311 NPV 90.1%
	707 Sensitivity 53.6%	3303 Specificity 90.3%

CHD indicates coronary heart disease; CVD, cardiovascular disease; NDI, National Death Index; NPV, negative predictive value; PPV, positive predictive value; REGARDS, REasons for Geographic And Racial Differences in Stroke.

Characteristics associated with disagreement between adjudicated and NDI‐determined CHD mortality included age, race‐sex groups, diabetes mellitus, hypertension, hyperlipidemia, history of CHD, and site of death (Table 3). In multivariable‐adjusted analyses, the probability of the NDI reporting a false‐negative CHD death was higher among participants with hypertension and lower among participants with a history of CHD (Table 4). Among NDI‐derived CHD deaths, older age at death was associated with a higher probability of disagreement with REGARDS‐adjudicated cause of death.

Table 3

Characteristics of Deceased REGARDS Participants Comparing Expert Adjudication and the NDI Determined Main Underlying Cause of Death

Characteristic	Overall (n=4010)	Agreement		Disagreement		P Valuea	SN	SP	PPV	NPV
		True Positives (n=379)	True Negatives (n=2983)	False Negatives (n=328)	False Positives (n=320)
	n	n (%)	n (%)	n (%)	n (%)
Age at death, y						<0.001
45 to 54	59	7 (1.8)	42 (1.4)	8 (2.4)	2 (0.6)		46.7	95.5	77.8	84.0
55 to 64	442	62 (16.4)	309 (10.4)	49 (14.9)	22 (6.9)		55.9	93.4	73.8	86.3
65 to 74	1122	109 (28.8)	847 (28.4)	101 (30.8)	65 (20.3)		51.9	92.9	62.6	89.3
75+	2387	201 (53.0)	1785 (59.8)	170 (51.8)	231 (72.2)		54.2	88.5	46.5	91.3
Race‐sex group						<0.001
White men	1537	165 (43.5)	1122 (37.6)	118 (36.0)	132 (41.3)		58.3	89.5	55.6	90.5
Black men	818	83 (21.9)	589 (19.7)	80 (24.4)	66 (20.6)		50.9	89.9	55.7	88.0
White women	901	54 (14.2)	739 (24.8)	50 (15.2)	58 (18.1)		51.9	92.7	48.2	93.7
Black women	754	77 (20.3)	533 (17.9)	80 (24.4)	64 (20.0)		49.0	89.3	54.6	86.9
Income, $USD						0.47
<$20 000	1021	97 (25.6)	756 (25.3)	91 (27.7)	77 (24.1)		51.6	90.8	55.7	89.3
$20 to $34 000	1147	121 (31.9)	828 (27.8)	97 (29.6)	101 (31.6)		55.5	89.1	54.5	89.5
$35 to $74 000	981	83 (21.9)	748 (25.1)	72 (22.0)	78 (24.4)		53.4	90.6	51.5	91.2
≥$75 000	321	34 (9.0)	236 (7.9)	31 (9.5)	20 (6.3)		52.3	92.2	63.0	88.4
Not reported	540	44 (11.6)	415 (13.9)	37 (11.3)	44 (13.8)		54.3	90.4	50.0	91.8
Poverty tertile (census tract)						0.38
Low poverty	1324	132 (38.6)	989 (36.5)	90 (30.3)	113 (38.6)		59.5	89.7	53.9	91.7
Medium	1222	112 (32.7)	904 (33.3)	108 (36.4)	98 (33.4)		50.9	90.2	53.3	89.3
High poverty	1098	98 (28.7)	819 (30.2)	99 (33.3)	82 (28.0)		47.9	90.9	54.4	89.2
Diabetes mellitus	1466	163 (43.0)	1017 (34.1)	154 (47.0)	132 (41.3)	<0.001	51.4	88.5	55.3	86.8
Hypertension	2866	281 (74.1)	2074 (69.5)	271 (82.6)	240 (75.0)	<0.001	50.9	89.6	53.9	88.4
Hyperlipidemia	3074	312 (82.3)	2253 (75.5)	252 (76.8)	257 (80.3)	0.01	55.3	89.8	54.8	89.9
History of CHD	1772	247 (65.2)	1153 (38.7)	175 (53.4)	197 (61.6)	<0.001	58.5	85.4	55.6	86.8
Site of death						0.009
Out of hospital	2639	269 (71.0)	1922 (64.4)	234 (71.3)	214 (66.9)		53.5	90.0	55.7	89.1
In hospital	1371	110 (29.0)	1061 (35.6)	94 (28.7)	106 (33.1)		53.9	90.9	50.9	91.9

Percentages are column percentages. CHD indicates coronary heart disease; NDI, National Death Index; NPV, negative predictive value; PPV, positive predictive value; REGARDS, REasons for Geographic And Racial Differences in Stroke; SN, sensitivity; SP, specificity; USD, US dollars.

P value denotes chi‐squared tests to assess statistical significance between groups.

Table 4

Characteristics Associated With Disagreement Between REGARDS Adjudication and NDI‐Derived Cause of Death

Characteristics	False‐Negative CHD (n=707)a	False‐Positive CHD (n=699)b	False‐Negative CVD (n=1351)c	False‐Positive CVD (n=1403)d
	OR (95% CI)e	OR (95% CI)e	OR (95% CI)e	OR (95% CI)e
Age at death, per 1‐y increase	1.01 (0.99, 1.02)	1.06 (1.04, 1.08)	0.99 (0.98, 1.01)	1.04 (1.02, 1.05)
Race‐sex group
White men	1 (ref)	1 (ref)	1 (ref)	1 (ref)
Black men	1.05 (0.68, 1.61)	1.12 (0.72, 1.74)	1.26 (0.88, 1.79)	1.29 (0.92, 1.82)
White women	1.23 (0.76, 1.98)	1.25 (0.77, 2.02)	1.42 (0.99, 2.04)	1.06 (0.75, 1.50)
Black women	1.16 (0.74, 1.83)	1.20 (0.75, 1.92)	1.06 (0.72, 1.55)	1.07 (0.75, 1.53)
Annual household income
≥$75 000	1 (ref)	1 (ref)	1 (ref)	1 (ref)
$35 000 to $74 999	0.80 (0.44, 1.44)	1.06 (0.55, 2.04)	0.79 (0.48, 1.31)	1.64 (0.92, 2.94)
$20 000 to $34 999	0.69 (0.38, 1.25)	0.89 (0.46, 1.71)	0.65 (0.39, 1.07)	1.36 (0.75, 2.43)
<$20 000	0.69 (0.37, 1.28)	0.78 (0.39, 1.59)	0.85 (0.49, 1.47)	1.77 (0.96, 3.24)
Poverty tertile, medium‐high vs low	1.43 (0.98, 2.09)	1.12 (0.78, 1.60)	1.18 (0.87, 1.61)	1.23 (0.92, 1.63)
Diabetes mellitus	1.22 (0.89, 1.67)	1.08 (0.78, 1.49)	1.38 (1.07, 1.78)	1.15 (0.89, 1.47)
Hypertension	1.67 (1.14, 2.45)	1.19 (0.82, 1.72)	1.14 (0.84, 1.54)	0.92 (0.70, 1.22)
Hyperlipidemia	0.78 (0.53, 1.14)	0.82 (0.55, 1.24)	0.97 (0.72, 1.31)	0.87 (0.65, 1.15)
History of CHD/CVDf	0.59 (0.42, 0.82)	0.84 (0.60, 1.18)	0.67 (0.51, 0.89)	0.60 (0.46, 0.79)
Site of death, in‐hospital vs out‐of‐hospital	1.11 (0.79, 1.57)	1.29 (0.92, 1.82)	0.93 (0.19, 1.90)	1.11 (0.87, 1.41)

CHD indicates coronary heart disease; CVD, cardiovascular disease; NDI, National Death Index; OR, odds ratio; REGARDS, REasons for Geographic And Racial Differences in Stroke.

Model restricted to participants with REGARDS‐adjudicated CHD death. NDI‐derived death other than CHD was the outcome of interest (false negative).

Model restricted to participants with NDI‐derived CHD death. REGARDS‐adjudicated death other than CHD was the outcome of interest (false positive).

Model restricted to participants with REGARDS‐adjudicated CVD death. NDI‐derived death other than CVD was the outcome of interest (false negative).

Model restricted to participants with NDI‐derived CVD death. REGARDS‐adjudicated death other than CVD was the outcome of interest (false positive).

Model is simultaneously adjusted for all characteristics.

History of CHD when outcome is CHD death, history of CVD when outcome is CVD death.

Among the false‐negative CHD cases, diseases of the circulatory system (excluding CHD), endocrine/nutritional/metabolic diseases, and diseases of the respiratory systems were mostly responsible for the disagreement, whereas heart failure, unclassifiable causes, and infections were mostly responsible for the disagreement among the false‐positive CHD deaths (Table 5).

Table 5

Causes of Death for Disagreements Between REGARDS‐Adjudicated and NDI‐Derived CHD Deaths

Causes of Death for Disagreements
When REGARDS=CHD (n=328)	n	%	When NDI=CHD (n=320)	n	%
NDI Cause of Death			REGARDS Cause of Death
Diseases of circulatory system (excluding CHD)	117	35.7	Heart failure	95	29.7
Endocrine/nutritional/metabolic diseases	90	27.4	Unclassifiable	57	17.8
Diseases of respiratory system	36	11	Infection	44	13.8
Diseases of genitourinary system	24	7.3	Other noncardiac, nonstroke death	19	5.9
Neoplasms	17	5.2	Dementia	16	5
Symptoms, signs, and abnormal clinical lab findings	13	4	Stroke	16	5
Diseases of nervous system	9	2.7	Not cardiac but other cardiovascular, eg, ruptured aortic aneurysm	15	4.7
External causes of morbidity/mortality	7	2.1	Cancer	14	4.4
Mental/behavioral disorders	6	1.8	End‐stage renal disease	14	4.4
Diseases of digestive system	5	1.5	Chronic lung disease	11	3.4
Infectious/parasitic diseases	2	0.6	Accident/injury/suicide/homicide	10	3.1
Diseases of blood/blood‐forming organs and immune disorders	1	0.3	Pulmonary embolism	5	1.6
Diseases of musculoskeletal systems/connective tissue	1	0.3	Other cardiac, eg, myocarditis	3	0.9
			Liver disease	1	0.3
Total	328	100	Total	320	100

CHD indicates coronary heart disease; NDI, National Death Index; REGARDS, REasons for Geographic And Racial Differences in Stroke.

The overall CHD mortality rate determined by both REGARDS adjudication and NDI were similar (4.4 and 4.2 per 1000 person‐years, respectively); however, differences were found when rates were found by demographic and clinical characteristics (Figure 2). Mortality rates were highest in participants with age ≥75 years at baseline and those participants with diabetes mellitus, with differences in the rates between REGARDS‐adjudicated and NDI‐derived deaths (for age ≥75, 10.6, and 12.3 deaths per 1000 person‐years, respectively; and for those with diabetes mellitus, 8.2 and 7.1 deaths per 1000 person‐years, respectively).

Figure 2

Mortality rates per 1000 person years (95% CI) for CHD in REGARDS and NDI, overall and by selected statistics. CHD indicates coronary heart disease; NDI, National Death Index; REGARDS, REasons for Geographic And Racial Differences in Stroke.

CVD Mortality

There were 991 NDI‐CVD deaths among 1351 adjudicated CVD deaths (SN 73.4%) and 2247 NDI non‐CVD deaths among 2659 adjudicated non‐CVD deaths (specificity 84.5%; Table 2). PPV was 70.6% and NPV and was 86.2%. Overall concordance percentage was 81%. Test characteristics were similar in black and white participants.

Characteristics associated with disagreement between adjudicated and NDI‐determined CVD mortality included age, race‐sex group, income, diabetes mellitus, hypertension, hyperlipidemia, history of CVD, and site of death (Table 6).

Table 6

Characteristics of Deceased REGARDS Participants Comparing Expert Adjudication and the NDI Determined CVD Mortality

Characteristic	Overall (n=4010)	Agreement		Disagreement		P Valuea	SN	SP	PPV	NPV
		True Positives (n=991)	True Negatives (n=2247)	False Negative (n=360)	False Positive (n=412)
	n	n (%)	n (%)	n (%)	n (%)
Age at death, y						<0.001
45 to 54	59	11 (1.1)	36 (1.6)	9 (2.5)	3 (0.7)		55.0	92.3	78.6	86.7
55 to 64	442	118 (11.9)	242 (10.8)	50 (13.9)	32 (7.8)		70.2	88.3	78.7	82.9
65 to 74	1122	270 (27.2)	671 (29.9)	93 (25.8)	88 (21.4)		74.4	88.4	75.4	87.8
75+	2387	592 (59.7)	1298 (57.8)	208 (57.8)	289 (70.1)		74.0	81.8	67.2	86.2
Race‐sex group						<0.001
White men	1537	402 (40.6)	873 (38.9)	117 (32.5)	145 (35.2)		77.5	85.0	73.5	88.2
Black men	818	196 (19.8)	445 (19.8)	86 (23.9)	91 (22.1)		69.5	83.0	68.3	83.8
White women	901	185 (18.7)	551 (24.5)	79 (21.9)	86 (20.9)		70.1	86.5	68.3	87.5
Black women	754	208 (21.0)	378 (16.8)	78 (21.7)	90 (21.8)		72.7	80.8	69.8	82.9
Income, $USD						0.003
<$20 000	1021	237 (23.9)	548 (24.4)	111 (30.8)	125 (30.3)		68.1	81.4	65.5	83.2
$20 to $34 000	1147	312 (31.5)	622 (27.7)	96 (26.7)	117 (28.4)		76.5	84.2	72.7	86.6
$35 to $74 000	981	228 (23.0)	567 (25.2)	83 (23.1)	103 (25.0)		73.3	84.6	68.9	87.2
≥$75 000	321	72 (7.3)	198 (8.8)	33 (9.2)	18 (4.4)		68.6	91.7	80.0	85.7
Not reported	540	142 (14.3)	312 (13.9)	37 (10.3)	49 (11.9)		79.3	86.4	74.3	89.4
Poverty tertile (census tract)						0.16
Low poverty	1324	333 (37.0)	772 (37.7)	104 (31.9)	115 (31.2)		76.2	87.0	74.3	88.1
Medium	1222	297 (33.0)	679 (33.2)	114 (35.0)	132 (35.8)		72.3	83.7	69.2	85.6
High poverty	1098	271 (30.1)	597 (29.2)	108 (33.1)	122 (33.1)		71.5	83.0	69.0	84.7
Diabetes mellitus	1466	393 (39.7)	739 (32.9)	171 (47.5)	163 (39.6)	<0.001	69.7	81.9	70.7	81.2
Hypertension	2866	758 (76.5)	1515 (67.4)	286 (79.4)	307 (74.5)	<0.001	72.6	83.2	71.2	84.1
Hyperlipidemia	3074	790 (79.7)	1687 (75.1)	282 (78.3)	315 (76.5)	0.03	73.7	84.3	71.5	99.5
History of CVD	1772	750 (75.7)	1062 (47.3)	244 (67.8)	271 (65.8)	<0.001	75.5	79.7	73.5	81.3
Site of death						0.047
Out of hospital	2639	627 (63.3)	1519 (67.6)	235 (65.3)	258 (62.6)		72.7	85.5	70.8	86.6
In hospital	1371	364 (36.7)	728 (32.4)	125 (34.7)	154 (37.4)		74.4	82.5	70.3	85.3

Percentages are column percentages. CVD indicates cerebrovascular disease; NDI, National Death Index; NPV, negative predictive value; PPV, positive predictive value; REGARDS, REasons for Geographic And Racial Differences in Stroke; SN, sensitivity; SP, specificity; USD, US dollars.

P value denotes chi‐squared tests to assess statistical significance between groups.

In multivariable‐adjusted analyses, the probability of the NDI reporting a false‐negative CVD death was higher among participants with diabetes mellitus and lower among participants with a history of CVD (Table 4). Among NDI‐derived CVD deaths, older age at death was associated with a higher probability of disagreement and history of CVD was associated with a lower probability of disagreement with REGARDS‐adjudicated cause of death.

Among false‐negative CVD cases, endocrine/nutritional/metabolic diseases, diseases of respiratory system, diseases of genitourinary system, and neoplasms were mostly responsible for the disagreements, whereas infections, unclassifiable causes, and other noncardiac, nonstroke deaths were responsible for the disagreements among the false‐positive CVD deaths (Table 7). Overall CVD mortality for both REGARDS adjudication and NDI were 8.4 and 8.5 deaths per 1000 person‐years, respectively, with highest rates noted in participants with age ≥75 years at baseline (23.0 and 24.8 deaths per 1000 person years in REGARDS and NDI, respectively; Figure 3).

Table 7

Causes of Death for Disagreements Between REGARDS‐Adjudicated and NDI‐Derived CVD Death

Causes of Death for Disagreements
REGARDS‐Adjudicated CVD Deaths (n=360)	n	%	NDI‐Derived CVD Deaths (n=412)	n	%
NDI Cause of Death			REGARDS Cause of Death
Endocrine/nutritional/metabolic diseases	122	33.9	Infection	114	27.7
Diseases of respiratory system	68	18.9	Unclassifiable	110	26.7
Diseases of genitourinary system	36	10	Other noncardiac, nonstroke death	51	12.4
Neoplasms	34	9.4	End‐stage renal disease	33	8
Symptoms, signs, and abnormal clinical lab findings	20	5.6	Cancer	28	6.8
Diseases of digestive system	18	5	Accident/injury/suicide/homicide	24	5.8
Diseases of nervous system	15	4.2	Chronic lung disease	24	5.8
Infectious/parasitic diseases	12	3.3	Dementia	23	5.6
Mental/behavioral disorders	12	3.3	Liver disease	5	1.2
External causes of morbidity/mortality	10	2.8
Diseases of blood/blood‐forming organs and immune disorders	6	1.7
Diseases of musculoskeletal systems/connective tissue	3	0.8
Congenital malformations	2	0.6
Diseases of skin/subcutaneous tissue	2	0.6
Total	360	100	Total	412	100

CVD indicates cerebrovascular disease; NDI, National Death Index; REGARDS, REasons for Geographic And Racial Differences in Stroke.

Figure 3

Mortality rates per 1000 person years (95% CI) for CVD in REGARDS and NDI, overall and by selected statistics. CVD indicates cardiovascular disease; NDI, National Death Index; REGARDS, REasons for Geographic And Racial Differences in Stroke.

Discussion

Of 4010 REGARDS participants' deaths through December 2013, we found that the SN and PPV of the NDI compared to expert adjudication for detecting CHD mortality were modest, whereas specificity and NPV were high. The SN and PPV of the NDI for detecting the broader category of CVD mortality were slightly better than for CHD mortality. The agreement between the NDI and adjudicated cause of death varied by age at demise, given that deaths among older individuals were more likely to be coded as a CHD death by NDI when the actual cause of death was something other than CHD. Because the majority of deaths occurred in this age group, caution may be warranted when using the NDI to describe causes of death in this subset of people. Disagreements between the NDI and adjudicated cause of death were less likely when participants had a history of CHD or CVD, because a history of either CHD or CVD provides insight in situations where there was little or no information about events surrounding death. When the 2 sources of information on cause of death disagreed, REGARDS investigators most often adjudicated causes of death as heart failure, infections, and unclassifiable when the NDI cause of death was CHD. Differential performance of NDI cause of death determination by decedent characteristics could bias analyses of risk factors. Although inaccuracy in the NDI could lead to bias when assessing the burden of CHD and CVD death in the United States,19 we observed that the overall mortality rates between REGARDS and NDI were approximately the same.

The main underlying causes of death reported on death certificates are not concordant with expert adjudication in up to 30% to 40% of cases.20, 21, 22 A study assessed the accuracy of death certificate completion using standardized case reports prepared by the National Center for Health Statistics and reported only a 56% agreement between the underlying cause of death reported by physicians and the actual cause published by the Center of Health Statistics.23 Studies have also shown that medical students and house staff are more likely to make mistakes in completing a death certificate compared with more‐senior medical personnel.10 In the majority of teaching hospitals, residents have the responsibility of completing death certificates and only a small percentage of them have formal training in how to do this.24 In other cases, a coroner or covering physician may complete a death certificate for someone he or she has little information about. This can lead to misclassification and under‐ or overdetection of various potential causes of death in NDI data and, by extension, mortality statistics in the United States. Differential misclassification of causes of death could lead to biases in associations with risk factors for CHD and CVD mortality when relying on NDI or death certificates. In contrast, adjudication of cause of death in research studies is usually carried out by teams of clinical researchers who are trained in the process of careful review of all available medical information on a deceased patient. Judgments made by expert adjudicators are expected to be more reliable, but the process can be time‐consuming.

Because death certificates are the primary source of information when generating reports of national mortality statistics, there may be an overestimation of CHD as an underlying cause of death in the elderly population (≥75 years), which also comprises the majority of deaths in the United States. With CHD being the leading cause of death in the United States,25 misrepresentation of the cause of death in this age group may cause a substantial shift in death statistics, especially given the aging of the population in the United States. In cases of out‐of‐hospital death in elderly people, heart disease is often given as the cause of death when the circumstances surrounding the death are unclear enough to preclude classification of the cause of death.26

Other studies like ours have also relied on all available clinical information to determine the cause of death. Although there were some effects on cause‐specific mortality rates with secular changes in International Classification of Diseases coding of death certificates, the accuracy remained unchanged over decades of follow‐up.27, 28, 29, 30

Strengths of our study include a large, nation‐wide, community‐based cohort that includes a high proportion of blacks and women and use of expert adjudication to determine the cause of death. Limitations include self‐report for some covariates, and that the REGARDS cohort includes only blacks and whites, possibly limiting generalizability. However, the subgroup of REGARDS participants aged 65 years and older with Medicare coverage had very similar health status to a 5% random sample of Medicare beneficiaries after accounting for demographics.31 It is noteworthy that we found no differences by race, lessening this concern. We only included deaths that had been both adjudicated (using all available sources) and had NDI data, and we found some differences in participant characteristics between those included and excluded. This could result in an underestimation of the CHD and CVD mortality rates given that not all deaths occurring in the cohort were included. Finally, the REGARDS determination of cause of death was not fully independent of NDI, because adjudicators considered the NDI cause of death during adjudication.

In conclusion, our findings suggest that using NDI to classify CHD and CVD mortality may miss 46% of CHD deaths and 27% of CVD deaths, and expert adjudicators did not concur with 45% of deaths classified as CHD and 30% classified as CVD by NDI. We also found variations in the reliability of NDI by participant characteristics, including age. However, mortality rates were remarkably similar despite these discrepancies. Nevertheless, reliability of the NDI may have repercussions for national death statistics and policy decisions that rely on these statistics. Careful consideration of the potential for misclassification should also be taken in research studies solely using NDI or death certificates to determine the cause of death.

Sources of Funding

This research project is supported by a cooperative agreement U01 NS041588 from the National Institute of Neurological Disorders and Stroke, National Institutes of Health, Department of Health and Human Service (Bethesda, MD). Additional sources of support include grants R01 HL080477 and K24 HL111154, both from the National Heart, Lung, and Blood Institute (Bethesda, MD).

Disclosures

Levitan reports research funding from Amgen and consulting for Amgen and Novartis. Safford reports research funding from Amgen. Brown receives research funding from Amgen and is the site PI for a clinical trial funded by Omthera Pharmaceuticals. Durant reports research funding from Amgen and Amarin. Other authors report no disclosures.