Concordance between the underlying causes of death on death certificates written by three emergency physicians.

OBJECTIVE: This study was conducted to evaluate the concordance between the underlying causes of death (UCOD) on the death certificates written by three emergency physicians (EPs). We investigated errors on the death certificates committed by each EP.
METHODS: This study included 106 patients issued a death certificate in the emergency department of an academic hospital. Three EPs reviewed the medical records retrospectively and completed 106 death certificates independently. The selection of the UCOD on the death certificates by each EP (EP-UCOD) was based on the general principle or selection rules. The gold standard UCOD (GS-UCOD) was determined for each patient by unanimous consent between three EPs. We also compared between the EP-UCOD and the GS-UCOD. In addition, we compared between UCODs of three EPs. The errors on the death certificates were investigated by each EP.
RESULTS: The rates of concordance between EP-UCOD and the GS-UCOD were 86%, 81%, and 67% for EP-A, EP-B, and EP-C, respectively. The concordance rates between EP-A and EP-B were the highest overall percent agreement (0.783), and those between EP-A and EP-C were the lowest overall percent agreement (0.651). Although each EP had differences in the errors they committed, none of them listed the mode of dying as UCOD.
CONCLUSION: This study confirmed that each EP wrote death certificates indicating different causes of death for the same decedents; however, the three EPs made fewer errors on the patients' death certificates compared with those reported in previous studies.

INTRODUCTION

In 1967, the Twentieth World Health Assembly defined the causes of death (COD) entered on the medical certificate as “all those diseases, morbid conditions or injuries which either resulted in or contributed to death and the circumstances of the accident or violence which produced any such injuries.” The underlying cause has been defined as “(a) the disease or injury which initiated the chain of morbid events leading directly to death, or (b) the circumstances of the accident or violence which produced the fatal injury.” [1]

Mortality statistics on causes of death are one of the principal sources of health information; in many countries, they are the most reliable type of health data [1]. The World Health Assembly adopted rules and guidelines regarding the selection of a single cause or condition for routine tabulation from death certificates and morbidity records. The cause or condition is usually the underlying cause of death (UCOD) used for tabulation [2]. The accuracy of mortality statistics on causes of death depends on the quality of the death certificate. A high-quality death certificate is defined as logically describing the causes of death according to the proper sequences and compatible causal relationships without fatal errors. Therefore, physicians have a responsibility to indicate which morbid conditions led directly to death and to state any antecedent conditions giving rise to this cause on the death certificate with clinical judgement.

The quality of death certificates is improving in the Republic of Korea. From the World Health Organization mortality database, the percentage of R-codes (R00–R99), which refer to the symptoms, signs, and ill-defined conditions, among all causes of mortality decreased from 9.2% in 2013 to 8.8% in 2015. However, 76% of the death certificates and the postmortem examination certificates had at least one or more errors and lost value as medical evidence documents in a forensic study [3]. These facts suggested that more efforts are needed to achieve a higher quality of mortality statistics.

Previous studies reported a discrepancy between the COD on death certificates and those on autopsy reports [4,5], analyzed the errors on the death certificates [6], and evaluated the improvement points for death certificates in accordance with the guidelines of the Korean Medical Association [7]. However, no attempt has been made to evaluate the concordance of the CODs on the death certificates written by several medical physicians, assuming that there may be different opinions about the COD of the same decedent. If each physician suspects completely different COD with the same clinical information, it suggests that the interpretation of key clinical information is misleading, or the previous medical certificate guidelines do not have the appropriate protocol to write the most accurate death certificates.

Therefore, this study compared the UCOD on the death certificates completed by three emergency physicians (EPs) with the gold standard UCOD (GS-UCOD) and evaluated the concordance rates of the death certificates between the three EPs. Furthermore, the errors were investigated by type, total number, and all other details for each of the EPs.

METHODS

Study design

The study subjects included 106 patients issued death certificates at the emergency department of an academic hospital from May 2016 to December 2016. Patients who died upon arrival without cardiopulmonary resuscitation were excluded, because they must have a postmortem examination certificate instead of a death certificate, in principle. Moreover, patients who died while waiting for admission to another department were excluded, because their death certificates were written by another department’s physician, not EPs. A total of 106 patients were included in this study. After reviewing the patients’ medical records, results of diagnostic laboratory tests, and radiological imaging studies, three board-certified EPs (EP-A, EP-B, and EP-C) independently completed the 106 death certificates in the form of a medical certificate recommended by the World Health Assembly. The three EPs were advised in advance that it would not be possible for them to cheat by viewing the existing death certificate written by another physician during the review of the medical records or to refer to the guidelines, books, and leaflets on how to write the death certificates. The EPs completed the 106 death certificates in the usual manner using only their previous knowledge on how to write a certificate. In the next step, the COD on the lowest line in Part I of each death certificate was selected as EP’s UCOD (EP-UCOD) based on the general principles or selection rules 1, 2, and 3 for selection of UCOD [1]. If we only compared the concordance rates between the UCODs of the three EPs, it would be impossible to distinguish between the correct UCOD from all three EPs and the incorrect UCOD from all three EPs when all UCOD of three EPs were concordant. This was the reason for comparing the EP-UCOD with the GS-UCOD. Finally, the GS-UCODs were determined for each of the 106 decedents through sufficient discussion and agreement between the three EPs. Next, we compared each EP-UCOD and GS-UCOD and evaluated the concordance rates of the cause of death determinations across the three EPs by comparing the UCOD for each decedent (Fig. 1). The manner of death was divided into three categories: natural death, defined as death due to diseases; unnatural death, defined as death due to an external cause (e.g., accident, suicide, or homicide); and undetermined death, defined as death due to unknown diseases or external causes. According to the manner of death, the rates of concordance between the three EPs were analyzed. In addition, we investigated the types of errors, the total number of errors, and all other details according to the types of errors committed by each EP.

Fig. 1.

The comparison between underlying cause of death (UCOD) of emergency physicians (EPs) and gold standard UCOD for 106 decedents, and the comparison between UCOD of three EPs by physician-to-physician matching.

This study was approved by the relevant institutional review board of Ulsan University Hospital (2018-03-043). All data were analyzed anonymously. The requirement for informed consent was waived by the board.

Definition of errors on the death certificate

Errors on the death certificates occur when it is difficult to select the UCOD based on the information included in the death certificates and when the selection process did not follow the guidelines of the International statistical classification of diseases and related health problems (ICD-10) or when the formal terms recommended by the Korean standard classification of diseases (KCD-7) were not used. We reviewed all errors based on the guidelines established in the previous study. The errors were classified as major or minor errors according to their influence on the selection of the UCOD. The errors were evaluated by two EPs who did not complete the certificate. We investigated all errors to determine whether both major and minor errors were simultaneously present.

Major errors

The major errors were divided into those that (1) listed only the mode of dying without including other UCODs (e.g., cardiopulmonary arrest, heart failure, and respiratory failure); (2) included obviously secondary conditions without an antecedent COD (e.g., pulmonary embolism, aspiration pneumonia, and disseminated intravascular coagulation); (3) included only ill-defined conditions as the UCOD (e.g., sudden cardiac death, senility, and ICD-10 codes R00–R94 and R96–R99, i.e., symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified); (4) indicated an improper sequence or sequence of events which were implausible; (5) listed an incompatible causal relationship, such as two or more unrelated CODs; (6) listed more than one COD on a single line in Part I; (7) included a blank line between causes or duplicated the same COD; (8) indicated an incorrect manner of death by identifying unnatural death or undetermined death as natural death; and (9) indicated an unacceptable COD with evidence of an illogical decision (Table 1) [8-13].

Table 1.

The definition of major and minor errors on the death certificates

Type of error	Definition
Major errors
Mode of dying as UCOD	Listed only the mode of dying without including other UCOD
Secondary condition as UCOD	Included obviously secondary conditions without an antecedent COD
Ill-defined conditions as UCOD	Included only the ill-defined conditions as underlying COD
Improper sequence	Indicated an improper sequence or sequence of events did not make sense
Incompatible causal relationship	Listed an incompatible causal relationship such as two or more unrelated COD
≥ 1 COD on a single line	Listed more than one COD on a single line in Part I
Blank/duplication	Included a blank line between causes or duplicated the same COD
Incorrect manner of death	Indicated an incorrect manner of death by determining unnatural death or undetermined death to be natural death
Unacceptable COD	Indicating an unacceptable COD with evidence of an illogical decision
Minor errors
Mode of dying followed by a legitimate COD	Included mode of dying followed by a legitimate COD
Incorrect time interval	Listed an incorrect time interval in Part I
Absence of other significant conditions	Listed no other significant conditions in Part II
English terms/abbreviations	Used English medical terms or abbreviations to identify diseases in Parts I and II
S00-T98 code without V01-Y89 code	Listed only the S00-T98 codes as the underlying COD without V01-Y89 codes in Part I
Incomplete information of external cause	Included incomplete information regarding other conditions as the external cause

UCOD, underlying cause of death; COD, cause of death.

Minor errors

Minor errors were divided into those that (1) included mode of dying followed by a legitimate COD, (2) listed an incorrect time interval in Part I, (3) listed no other significant conditions in Part II, (4) used English medical terms or abbreviations to identify diseases in Parts I and II, (5) listed only the S00-T98 codes as the UCOD without V01-Y89 codes in Part I, and (6) included incomplete information regarding other conditions as the external causes (classification, intentionality, date and time, and place) (Table 1) [9-12].

Determination of the GS-UCOD

For determining a more accurate GS-UCOD, the EPs studied in depth the ICD-10 [1], KCD-7 [14], and the medical certificate guidelines published by the Korean Medical Association 6 months following the creation of the first death certificate. In addition, the EPs mastered their professional knowledge on errors on death certificates by reviewing previous articles and asking the experts from Statistics Korea and other specialists for advice. The three EPs conducted a second review of the medical records of each of the 106 decedents together and shared their individual clinical judgements to determine the GS-UCOD. The EPs reassessed the GS-UCOD based on the medical evidence and knowledge, not insisting that their own UCOD was appropriate as the GS-UCOD. The GS-UCOD was unanimously determined after a panel discussion of the three EPs.

Level of concordance between the UCODs

For international comparison, the analysis by ICD should, as appropriate, be performed in accordance with the detailed list of three-character categories, with or without four-character subcategories [1]. Because the titles of ICD four-character subcategories are meaningful, the subcategories have a high diagnostic value. In fact Statistics Korea has investigated all four-character subcategories of the UCOD. Accordingly, the level of concordance between the EP-UCOD and GS-UCOD was divided into three categories: concordant, similar, or discordant. First, we assigned the KCD-7 disease code to each EP-UCOD and GS-UCOD [14]. If the EP-UCOD code was the same as that of the GS-UCOD code up to the four-character category, it was categorized as “concordant”. If the EP-UCOD was a condition involving the same organ but with a different stage of disease, or a similar etiology but not exactly the same with the GS-UCOD code, it was categorized as “similar”. Finally, if the EP-UCOD was completely different from the GS-UCOD code, it was categorized as “discordant”. The level of concordance between the UCODs selected by the three EPs for each of the 106 decedents was also evaluated using the same categories by physician-to-physician matching (Fig. 1).

Statistical analysis

The rates of concordance between the EP-UCOD and the GS-UCOD were calculated for each EP as a percentage of the decedents who were categorized as “concordant” per the total number of decedents. Based on the GS-UCOD, a comparison of the level of concordance between UCOD of the three EPs designations was performed using Pearson’s chi-square Fisher’s exact test. The concordance rates between UCODs of the three EPs were analyzed to determine the overall percent agreement. IBM SPSS Statistics ver. 21.0 (IBM, Armonk, NY, USA) was used for the statistical analysis, and statistical significance was defined as P<0.05.

RESULTS

Of 106 total decedents whose death certificates were issued by EPs in the emergency department for the study period, 71 had natural deaths, 30 had unnatural deaths, and five had undetermined deaths, based on the GS-UCOD. Among the three EPs, EP-A had the highest “concordant” rate, but had the lowest “discordant” rate. In contrast, EP-C had the lowest “concordant” rate, but had the highest “discordant” rate after comparing the EP-UCOD and GS-UCOD (Table 2).

Table 2.

The rate of concordance between the EP-UCOD and the GS-UCOD

	EP-A	EP-B	EP-C	P-value
Concordant	91 (85.8)	86 (81.1)	71 (67.0)	0.012
Similar	4 (3.8)	6 (5.7)	7 (6.6)
Discordant	11 (10.4)	14 (13.2)	28 (26.4)

Values are presented as number (%).

EP, emergency physician; UCOD, underlying cause of death; GS, gold standard.

Regarding the comparison between the UCODs of three EPs, the overall percent agreement between EP-A and EP-B was the highest (0.783). And the overall percent agreement between EP-A and EP-C was the lowest (0.651) (Table 3).

Table 3.

Overall percent agreement of the level of concordance to gold standard underlying cause of death between emergency physicians

		Cc	Sm	Dc	Overall percent agreement[a)]
Intersection EP-A and EP-B	Cc	77	3	6	0.783
	Sm	5	1	0
	Dc	9	0	5
Intersection EP-B and EP-C	Cc	65	2	4
	Sm	4	2	1	0.717
	Dc	17	2	9
Intersection EP-A and EP-C	Cc	64	2	5
	Sm	6	0	1
	Dc	21	2	5	0.651

EP, emergency physician; Cc, concordant; Sm, similar; Dc, discordant; N, number.

Overall percent agreement=N(Cc)+N(Sm)+N(Dc)/N of total death.

In the case of natural death, the difference between the percentages of level of concordance between EP-UCODs and GS-UCODs were considered significant (P=0.005). However, the percentages of findings designated as “concordant” were almost the same between the three EPs in case of unnatural death (P=1.000) (Table 4).

Table 4.

The rate of concordance between the EP-underlying causes of death and the gold standard underlying causes of death according to the manner of death

	Natural death				Unnatural death
	EP-A	EP-B	EP-C	P-value	EP-A	EP-B	EP-C	P-value
Concordant	59 (83.1)	52 (73.2)	39 (54.9)	0.005	29 (96.7)	29 (96.7)	30 (100.0)	1.000
Similar	4 (5.6)	6 (8.5)	7 (9.9)		0 (0.0)	0 (0.0)	0 (0.0)
Discordant	8 (11.3)	13 (18.3)	25 (35.2)		1 (3.3)	1 (3.3)	0 (0.0)

Values are presented as number (%).

EP, emergency physician.

The total number of errors and minor errors committed by EP-B were the smallest, and the number of major errors committed by EP-A was the smallest. The most frequent major errors committed by EP-B and EP-C were the inclusion of obviously secondary conditions as the UCOD. The most frequent minor errors committed by each EP were all different. The three most frequent minor errors were as follows: (1) listing an incorrect time interval in Part I, (2) listing only the S00-T98 codes as the UCOD without V01-Y89 codes in Part I for unnatural death, and (3) not listing other significant conditions in Part II. Although EP-C made the largest number of major and minor errors, EP-C was more likely to list accurately V01-Y89 codes in Part I as the UCODs compared with EP-A and EP-B (Table 5).

Table 5.

The numbers of major and minor errors on the death certificates written by three EPs

Type of error	EP-A	EP-B	EP-C
Major errors
Mode of dying as UCOD	0	0	0
Secondary condition as UCOD	3	8	17
Ill-defined conditions as UCOD	0	6	4
Improper sequence	0	0	0
Incompatible causal relationship	0	0	0
≥ 1 COD on a single line	0	0	0
Blank/duplication	0	0	0
Incorrect manner of death	1	0	3
Unacceptable COD	3	2	7
No. of major errors	7	16	31
Minor errors
Mode of dying followed by a legitimate COD	0	0	0
Incorrect time interval	35	1	7
Absence of other significant conditions	0	2	65
English terms/abbreviations	0	0	0
S00–T98 code without V01–Y89 code	27	25	16
Incomplete information of the external cause	0	0	0
No. of minor errors	62	28	88
Total no. of errors	69	44	129

EP, emergency physician; UCOD, underlying cause of death; COD, cause of death.

DISCUSSION

The viewpoints of every physician can be a little different even if the death certificates were written using the same clinical information. The difference should be within the acceptable range of most physicians to minimize the controversy. Ideally, the UCOD on the death certificate should not vary between physicians. However, the UCODs judged by physicians are not unequivocal in professional practice and previous research reports. Several studies have shown considerable differences in the way that physicians handle death certifications [15,16]. Even external post-mortems, which are expected to reveal more accurate causes of death, do not always provide conclusive evidence of the manner and COD [17]. In this study, the “concordant” rates between EP-UCOD and GS-UCOD were 67% to 86%, which were higher than the 66.5% for adults reported in a Mexican study [16]. If the UCOD selected from the death certificate written by physicians is inappropriate, the Statistics Korea office manually checks contents of the certificate and reviews the linked administrative data, and then replaces UCOD with a different disease or condition. In this process, only 54% to 62% of CODs written in the original death certificates remained as the final UCODs selected by Statistics Korea from 2007 to 2012, according to the information presented by Statistics Korea in a lecture on how to write a death certificate. Therefore, the “concordant” rate (67% to 86%) in this study was much higher than that reported by the Statistics Korea. Specifically, the category “concordant” indicates that the EP-UCOD code and the GS-UCOD code were the same up to the fourth-character. Despite the conservative measurements, the UCOD of EP-A and EP-B had more than 80% rates of concordance with the GS-UCOD. Additionally, if the modification rules are applied for selecting the UCOD, the rate of concordance between EP-UCOD and GS-UCOD is expected to be higher.

Mortality statistics require consistent quality improvement as they are vital statistics that represent the level of public health [2]. Statistics Korea has refined the UCOD in connection with 22 administrative data bases including those of the national health insurance, cancer registration data, police investigation data, and autopsy data for more accurate statistics. Nonetheless, the percentage of deaths for which COD was ‘well-certified of death’ was calculated to be only 81% of the data reported in the 2016 Global Burden of Disease Study. Korea is the 29th among the 35 Organization for Economic Cooperation and Development member countries. The percentage of these substantiated deaths in New Zealand (95.7%) was the highest, and those reported in the United Kingdom (91.3%), Australia (90.3%), Canada (90.1%), United States (86.9%), and Germany (84.0%) were higher than that reported in Korea. In other words, the exact cause of 19% of deaths in Korea were unknown [18].

The World Health Organization published time series estimates of deaths by cause, age, and sex from 2000 to 2015 among the member countries. This technical study of the country-level CODs excluded the data from countries that reported an average proportion of ill-defined causes of more than 25% [2]. The codes of symptoms, signs, and ill-defined conditions; injuries that were not determined as intentional or unintentional; and ill-defined cancers and cardiovascular diseases listed as UCOD were defined as “garbage” codes. In this study, the mode of dying, obviously secondary condition, and ill-defined conditions listed as UCOD were defined as major errors. Although each EP had differences in the number of errors, it is encouraging that none of them listed the mode of dying. The errors listed as either obviously secondary conditions or ill-defined conditions as the UCOD were relatively frequent in this study. However, this was a small number of errors compared to that reported in previous studies regarding types of errors on death certificates [7,12]. It was presumed that the three EPs had written accurate death certificate with few errors. Considering that, in principle, only EPs can write medical certificates in the emergency department of the study hospital, the EPs have accumulated a great deal of experience and knowledge in writing death certificates.

The concordance and accuracy are critical metrics to evaluate the quality of the death certificate. This study showed that the rate of concordance between UCOD and GS-UCOD is inversely related to the number of errors. Improvements in the knowledge of the guidelines on how to write death certificates can help not only increase the concordance but also reduce errors, hence creating more accurate death certificates. Therefore, the physicians should undergo continuing education on how to write the death certificate in accordance with the ICD and KCD revisions. Moreover, it is necessary to regularly review the contents and appropriateness of the guidelines for writing death certificates with medical students and to evaluate whether the effect of the education is linked and applied to clinical practice after medical students become physicians [19]. For example, clashes occurred between police and protesters gathered in Gwanghwamun Square in Seoul, Korea, in November 2015. A male farmer hit by a water cannon shot by the police in this violent scene lost consciousness from a traumatic subdural hematoma. After approximately 1 year of treatment, he died in September 2016 without recovering consciousness. At the time, not only his family but also diverse sectors of society were paying attention to the COD listed on his death certificate. As soon as the death certificate, on which “acute renal failure” was written as the COD and “natural death” as the manner of death, was released to the press, medical students, alumni, and the Korean Medical Association expressed opposition to the certificate. Eventually, the hospital convened a medical ethics committee, which took 9 months to revise the COD from “acute renal failure due to natural death” to “traumatic subdural hematoma due to the water cannon shot due to unnatural death.” The differences in every physician’s opinions may lead to a delay in revising the COD because of a lack of understanding of the guidelines and selecting a COD on the death certificate [6,7]. This representative example illustrates the importance of the obligation to write a death certificate and the realistic gap between physicians and medical students in terms of writing a death certificate.

In addition, there may be few hospitals where the medical record administrators or other physicians evaluate the appropriateness of the COD listed on the death certificate written by physicians. Na et al. [5] pointed out that it is impossible for EPs to learn to predict the COD because of the absence of a feedback system for autopsy results. The results of this study indicate that each physician had their own frequent errors and that specific errors occurred more frequently depending on the manner of death. Therefore, a feedback system to evaluate the appropriateness of the death certificates in the hospital may reduce all errors and improve the quality of death certificates [20,21].

The Republic of Korea has a high level of access to health care and one of the best level of medical services in the world; however, the collection of basic data related to public health appears to be relatively neglected, with insufficient administrative support in contrast to rapid technological growth. Physicians should make an effort to provide more accurate mortality statistics as the supplier and the largest consumer of these statistics.

This study had some limitations. First, the participants were from a single academic hospital; hence, the findings cannot be generalized to other health institutions or the whole country. Second, it is possible that the death certificates were written more accurately than usual, because the three EPs were aware of the purpose of this study before writing the certificates. Third, the GS-UCODs selected by the three EPs may be different from those actually registered in the mortality statistics of Statistics Korea. Finally, even though there was a time interval, the same three EPs who wrote the death certificates determined the GS-UCOD; hence, the individual opinions of the EPs may be projected excessively.

In conclusion, the levels of concordance of the three EPs were different compared with the GS-UCOD. However, the “concordant” rate between the three EP-UCODs and the GS-UCOD was between by 67% and 86%. The three EPs can be considered to have written the most accurate death certificate with few errors. Education on the guidelines on how to write the death certificates can help increase the concordance and reduce errors.