Provider reported challenges with completing death certificates: A focus group study demonstrating potential sources of error.

OBJECTIVES: To characterize the experiences of providers in completing the cause of death section on death certificates, with particular reference to deaths in people who have cancer.
METHODS: Focus groups were conducted until thematic saturation was reached, resulting in four groups over three months. Participants were from a variety of specialties and levels and types of training. Focus groups were recorded and transcribed verbatim and analyzed using constant comparison analysis.
RESULTS: Three types of challenges to case classification were identified. 1) Infrastructural and procedural challenges encountered when completing death certificates, including the rigid structure of the form, lack of training in its completion, and lack of real-time feedback. 2) Clinical uncertainty and the varied approaches providers take to determine the cause of death based on their perception of the purpose of the death certificate. 3) Choosing cause of death in decedents with a history of cancer.
CONCLUSIONS: There are specific and substantial challenges in the death certification process that lead to errors in documenting the cause of death, but many of these challenges could be addressed with structural change to the forms or mechanism of training. Using these data to inform change could improve the death certification process and reliability of this data.

Introduction

Death certificates have been used to track public health data in the United States since 1844 when Massachusetts passed the first law requiring cause of death to be reported to the state [1]. Since then, death certificates have become the basis for mortality statistics and directly influence medical practice, research and public health policy [2]. Taken individually, these forms can provide closure for next of kin and assist in estate and insurance settlements. When taken collectively, death certificates describe the health of the nation, reveal public health threats and progress, and demonstrate disparities between communities or populations [3-10].

Despite their crucial importance, numerous studies have demonstrated high rates of error in death certificates, especially in the cause of death section [11-19]. For example, a 2017 study in Vermont found that 51% of death certificates had a major error that impacted the interpretation of the primary cause or contributing causes of death [20]. In a 2010 study, deaths from cardiovascular disease in New York City were overestimated by 51% in adults ages 35–74 [21]. These errors have significant impacts on the public health data that underlie disease specific mortality rates, track health disparities and inform research and policy priorities. For example, in a study by Yin et al, misclassification of colon and rectal cancer deaths on the death certificate led to an inflation of the 5-year cause-specific survival rate for rectal cancer from 64.9% to 81.2% [22]. Despite ample evidence of errors in death certificates in the United States [11-19], and evidence that these errors are increasing in frequency [23], it is not clear what is causing these errors or how to prevent them. This study aims to better understand the challenges faced by providers when they complete the cause of death section of death certificates, how those challenges might contribute to sources of error, and any specific challenges relating to cancer-related deaths.

Methods

In the Spring of 2019 four in-person focus groups were conducted with 2–5 participants in each group. Focus groups were chosen for this study to replicate the discussions that may occur between providers when faced with challenges in death certification, and to investigate the degree of consensus between providers on the challenges involved in determining the cause of death. After the fourth focus group, thematic saturation was reached, a finding in line with Guest et. al who found that 80% of themes are captured in 2–3 focus groups and 90% are captured in 3–6 focus groups [24]. Purposive sampling [25] was used to recruit providers from a variety of specialties, levels of training and type of practice, using group email invitations. The study was conducted at a single academic institution in rural New England. A moderator (AM) oversaw each group and two additional researchers (SG and VL) took notes on participant tone and body language. The moderator was a medical student with training in qualitative research methods. Her role was to provide the group with standardized discussion questions and ask follow-up questions on new ideas presented, while managing the time and ensuring all focus group members had the chance to speak. Group bias was minimized by creating groups where providers did not know one another and by masking the provider type to avoid the potential of presumed hierarchy influencing provider responses. The 60–90 minute groups were conducted in private hospital conference rooms with participants seated at a single table to facilitate discussion. The focus groups were recorded with an audio recording device and then transcribed verbatim for analysis by the study team. The COREQ checklist was followed to present the study findings [26].

Each group was asked to consider eight questions and one case study adapted from publicly available training materials to reflect a situation of clinical uncertainty (S1 and S2 Appendices). Each participant was asked to complete the cause of death section of a blank U.S Standard Certificate of Death for the patient or to describe what approach they would take if they were uncertain. This was then used to start a discussion about what specific challenges the providers faced as they completed the death certificate.

Providers were offered a $30 Amazon gift card and a free meal during the focus group. IRB approval was granted by the Dartmouth College Committee for the Protection of Human Subjects. We were granted IRB exemption for this study, IRB number: STUDY00031374. Written informed consent was obtained from all participants prior to their participation in the focus groups. No minors were included in this study.

Focus group transcripts were analyzed using constant comparison analysis and QDA Miner software, a research method based in grounded theory that groups data into units then categories and finally a larger theme [25]. The iterative process of constant comparison analysis allows for identification of broad themes and patterns that can be used to generate theories, an approach that is particularly useful in qualitative studies aimed at exploring new ideas. AM analyzed the data with review during the process together with JR, any discrepancies in interpretation were discussed with the research team and resolved by consensus. Representative quotations for each area of challenge were selected from the transcripts to provide examples of statements made by participants. The causes of death chosen by participants in the case study were summarized.

Results

45 providers were invited, 25 providers responded, and 14 participated in one of four scheduled focus groups. Every focus group included both in-patient and out-patient providers and both men and women. Internal medicine was the most commonly represented specialty followed by palliative care and then oncology/hematology (Table 1).

Table 1

Characteristics of participants (N = 14).

		N
Gender	Male	7
	Female	7
Provider Type	Resident, MD/DO	5
	Attending, MD/DO	7
	Nurse Practitioner	2
Specialty	Oncology	2
	Palliative Medicine	4
	Internal Medicine	7
	Family Medicine	1
Type of Practice	Primarily out-patient	5
	Primarily in-patient	9

Three related challenges leading to misclassification on death certificates were identified: (1) infrastructural and procedural challenges encountered when completing death certificates; (2) clinical uncertainty and the approaches providers take to determine the cause of death; and (3) unique challenges posed by decedents with a recent or remote history of cancer.

Infrastructural and procedural challenges

Participants described several factors relating to the infrastructure and procedure involved in death certification; the structure of the form and its impact on data that can be entered, the lack of training and feedback, and the influence of the next of kin (Fig 1).

Fig 1

Factors that can introduce error in death certificates and solutions discussed by participants.

A major challenge described in all the focus groups was the restrictiveness of the death certificate form whose rigid structure may force providers to alter the sequence of events and omit important information as they describe the cause of death. Another commonly discussed challenge was the lack of standardized training or formal feedback for death certificates. Ten of 14 providers reported not having received any training on death certification in medical school or residency and five did not complete their first death certificate until after residency. Only two of 14 providers reported receiving formal feedback from state or hospital organizations on the quality of their death certificates. An additional topic that was discussed in a smaller number of focus groups was the impact of the next of kin on the information included in the death certificate. Although providers did not report changing the death certificate based on family wishes, they did note the challenging circumstances that arose when factors such as life insurance, military exposures to carcinogenic agents and disease stigma were raised during the death certification process because of perceived financial or personal implications for the next of kin.

Providers identified several potential solutions to the infrastructural and procedural challenges that were raised during the focus groups. Most targeted the lack of training or feedback, with several providers emphasizing that both training and consistent feedback are necessary for a document that is widely used for research and public health. Providers supported two main solutions: electronic death certification, and access to hospital death coordinators for deaths occurring in the hospital. Electronic death certification is used in several states already and was seen by many providers as a way to build instantaneous feedback into the process. Some providers described electronic systems in other states that would reject invalid diagnoses, such as mechanisms of death, and would catch errors such as fields left blank or information entered in the wrong area. Providers working in the inpatient setting described the benefit of a hospital death coordinator for proving feedback and ongoing training. Death coordinators use their expertise in certification to screen for errors and provide real-time feedback and training to staff when they encounter challenges.

Clinical uncertainty and approaches to determining the cause of death

Clinical uncertainty, described in many situations, posed a significant challenge for providers. Two main scenarios led to clinical uncertainty for both community and hospitalist providers; unexpected deaths and deaths following a prolonged period without medical care. Both of these challenges were present in the case study presented during the focus group (S2 Appendix) and resulted in a wide variety of causes of death reported by the providers at the conclusion of the case study (Table 2).

Table 2

Cause of death selected for the case study.

Immediate Cause of Death	Responses
Cardiac arrest	3
Cardiovascular disease	2
Colon cancer	2
Cerebrovascular disease	2
Respiratory failure	1
Cannot determine–would request autopsy	3
Cannot determine–no cause of death provided	1

Providers reported two main approaches to determining the cause of death in situations of clinical uncertainty (Fig 2), depending on their perceptions of the purpose of death certificates. Eight of 14 providers viewed death certificates as being legal documents or data for research and tended to value accuracy over specificity, listing the broadest cause of death that is accurate, even if it is a mechanism of death such as cardiac arrest, or listing the admission diagnosis if the patient dies while in the hospital. In contrast, six of 14 providers viewed the death certificate as a document intended for the medical record or for use by families and they tended to value a more specific diagnosis, even if it may be inaccurate. These providers described a strategy of making an educated guess on the cause of death based on a wide variety of tools including the medical record, conversations with family members and demographic information.

Fig 2

Relationship between participant perception of the purpose of death certificates and their strategies for determining cause of death when faced with clinical uncertainty.

Patients with cancer

Three specific challenges were identified in completing death certificates for patients with cancer (Fig 3). One was deciding if cancer should be listed as the immediate or underlying cause of death. Some providers believed that if a patient died from an aggressive cancer, the cancer should be listed as the immediate cause of death. Other providers had been taught that cancer should not be listed as the immediate cause of death. The second challenge was fitting the complexities of the diagnosis or patient history into the death certificate in a meaningful way. Many providers described patients with long histories of cancer with periods of remission and multiple recurrences which did not fit easily into the confines of the death certificate. The third challenge reported by providers was the heightened documentation requirements that some providers perceived exist for cancer. Many felt that exact tissue diagnosis, location of the primary tumor and sequence of metastatic spread should be included in the death certificate. Some oncologists and palliative care providers also included pathology information and a list of remissions and recurrences when applicable. This degree of detail was challenging for many of the hospitalist and community providers that did not have extensive records of patients’ cancer history. Five providers reported receiving queries from the state cancer registry regarding death certificates listing cancer as a cause of death. Because of this impression of heightened documentation requirements, many participants felt that cancer should not be included on a death certificate unless there was clinical certainty.

Fig 3

Challenges encountered when documenting cancer on the death certificate.

Discussion

This study identified three main types of challenges encountered by providers when completing death certificates; infrastructural and procedural challenges, clinical uncertainty and the unique challenges posed by patients with cancer. Focus group participants tied these challenges to potential sources of error in death certificates, both directly and indirectly.

An overarching challenge identified by providers was a lack of formal training or feedback about a certification process that is often inflexible and, with the exception of new electronic systems, seldom provides real-time feedback or training to optimize the data. Focus group participants agreed that the certification process can lead to inconsistency and error, and it has long been recognized that these ultimately contribute to unreliable public health data [18-22]. Several of the quotations from our focus group discussions illustrate common areas of confusion in the certification process including the use of mechanisms of death, uncertainty around how to handle cancer, and when to refer for an autopsy. All of these areas of confusion and consequent errors in death certification could be addressed by uniform, standardized training and ongoing feedback.

There were also several potential sources of error uncovered in conversations around how providers determined the cause of death. Providers that valued the accuracy of the death certificate were careful to only document diseases, conditions or processes that they could verify; this approach was problematic when there was clinical uncertainty around the cause of death. It was concerning that physicians’ views of the role and importance of death certificates to public health and to families might affect the approach they took to select a cause of death. In addition, several participants reported that, when unable to make a definitive diagnosis, they would list mechanisms of death such as cardiac or respiratory arrest; however, these mechanisms simply attest to the fact of death and they do not describe its etiology. If used at all, mechanisms of death must always be accompanied by cause of death: “the disease, abnormality, injury, or poisoning that caused the death” [27]. When the cause of death was not definitively diagnosed, it is acceptable to list it as “probable” or “likely”. Contrary to the understanding of some focus group participants, it is also acceptable to list “old age” as a cause of death in appropriate circumstances [28].

Providers faced with clinical uncertainty about the cause of death described how they combined clinical judgement with their expectations based on demographics and epidemiological trends; they acknowledged that this approach was a potential source of misclassification in the cause of death. Several group members noted that cardiovascular causes of death are often chosen because of expectations based on demographics. For example, one participant mentioned being asked to follow an algorithm for unwitnessed or sudden events that would attribute deaths to cardiovascular disease in all males over age 50 and to females over age 60 with a history of hypertension. In a concerning form of circular logic, this approach is likely to perpetuate and amplify inaccuracies in the data, exaggerating the importance of one cause of death over others and potentially obscuring important trends. This finding is supported by several studies that have demonstrated that death certificates substantially overestimate cardiovascular mortality [21, 29, 30]. Since these data are important to track diseases over time, determine research and funding priorities and inform new clinical guidelines, death certification data must be accurate and trustworthy.

Our study highlighted uncertainty over how to document cancer on the death certificate. In the hypothetical and intentionally vague case study, participants took varied approaches when the role of cancer was uncertain and the death was unwitnessed. Some participants described cancer as the immediate cause of death while others described it as the underlying cause of death and several participants omitted it completely. One focus group discussion centered around the idea that people don’t die of cancer, but of something else (e.g. anorexia-cachexia, infection) with cancer as a contributing factor. However, when cancer causes death through such a process, it should be listed as a proximate cause of death and not a contributing cause [27]. Many participants believed that in order to include cancer on the death certificate, a tissue biopsy was necessary. Ideally, a death certificate that indicates neoplasm as a cause of death should include the primary site, behavior (benign/malignant), cell type, grade, and the part or lobe of organ affected [27]. But again, a “probable” or “likely” diagnosis is also acceptable when such data are not available. Our results also highlighted variability in distinguishing deaths “from cancer” and deaths “with cancer” and raise concerns about the reliability of national cancer mortality statistics [18, 22, 31].

A 2005 focus group study by McAllum et al. examined the completion of death certificates by general practitioners in New Zealand [32]. Consistent with our findings, they demonstrated that clinical uncertainty posed a significant challenge for providers in the completion of death certificates. Specific challenges were related to gaps in care and sudden deaths, and there was a tendency to fall back on a diagnosis of cardiovascular disease or myocardial infarction as the cause of death when faced with clinical uncertainty. A second theme was the important role of the deceased’s family in determining the cause of death; this was a less prominent issue in our study although some providers described anger from family members over the certification of stigmatizing conditions such as alcoholism. McAllum et al. reported the same circular logic seen in our study whereby providers who do not know the true cause of death may assign one based on their expectations according to patient characteristics such as age, gender, and ethnic group. Uncertainty is likely when physicians are asked to certify deaths of patients they have not treated, or if the patient has not recently sought medical attention. In recent decades, and particularly since the Joint Commission removed the requirement of a 20% autopsy rate by hospitals, there has been a dramatic decline in autopsy rates to only 8.5% in 2007 [33]. “Gold standard” data from autopsies would help stop this cycle of uncertainty and allow death certificates to more accurately reflect important trends in population mortality [14, 34, 35].

Our findings highlight the need for improvement of specific aspects of the death certification process. Several studies have shown significant reductions in errors with standardized training programs, especially interactive programs that are periodically reinforced [36, 37]. For example, a training program in Spain reduced the error rate among a group of residents from 71% to 9% [37]. The Centers for Disease Control [27], World Health Organization [38], and others have established educational materials on death certificates that can be used to build a structured training program for all providers that are eligible to complete death certificates. A mechanism for structured, timely feedback has also been shown to reduce error in death certificates [39].

One of the quickest and least resource intensive methods of providing feedback is through an electronic death reporting system (EDRS), such as the New Hampshire Electronic Cause of Death mobile application [40]. Unlike paper forms, electronic systems can be structured to provide real-time feedback, for example by rejecting mechanism of death such as cardiac arrest, and by offering “help” menus to assist in areas of doubt [2]. Feedback is particularly important during public health crises such as the opioid epidemic and Covid-19 pandemic to ensure that deaths are certified using consistent methods, and that timely, accurate data are made available for public health officials and policy makers to monitor the crisis and its management. However, electronic certification systems cannot be expected to overcome errors relating to poor training and uncertainty over causes of death, and the use of dropdown menus and restrictions within the software may even introduce new errors and biases. Variation in software and in the time at which states and territories have adopted electronic certification technology [41] may also impair the interpretation of trends in causes of death. These limitations must be studied within and between states that use different EDRS software to quantify the impacts of “old” and “new” biases on misclassification of causes of death, but without autopsy confirmation this task may be difficult.

A significant strength of this study was the variety of providers that participated, including physicians and nurse practitioners from both in-patient and outpatient settings with backgrounds in four specialties and a wide variety of experience levels including residents and attendings. Another strength was the ability to conduct multiple focus groups to the point of redundancy which reduced the likelihood of missing major themes. Limitations of the study included its focus on two health systems in New England, potentially limiting the generalizability of these findings to other U.S. settings. The relatively small size of some of the focus groups and greater proportions of hospital-based participants may have limited the extent of novel ideas and quality of group discussion, and the use of focus groups rather than semi-structured interviews may have introduced group biases.

We propose the following strategies to improve the accuracy of death certificates. (1) Expand the use of electronic death certification software to provide guidance, validation and feedback in real time to improve incoming data. It will be critical to design and assess such software carefully to prevent the introduction of new biases. Immediate feedback via prompts within the software would promote the collection of more accurate data on specific diseases such as cancer and heart disease, including whether these diseases caused, contributed to, or were not relevant to the death. (2) Train all physicians and associate providers in the completion of death certificates both before graduation and as part of continuing education. (3) Change sudden death classification to include a place to record whether the death was witnessed or unwitnessed to provide information relating to certainty of the diagnosis. (4) Reinstate autopsy requirements in order to validate causes of death when diagnoses are uncertain in order to improve confidence in mortality statistics.

In conclusion, focus group discussion highlighted substantial challenges and potential sources of error in the death certification process, and provided a map for improvement to this crucial piece of information that we use to make large scale policy and public health decisions. It is critically important that the medical community recognize the impact of inaccurate death certification on population statistics and act to address it.

Questions used in the focus groups.

(DOCX)

Click here for additional data file.

Case study.

(DOCX)

Click here for additional data file.

(DOCX)

Click here for additional data file.

8 Feb 2022

10 Mar 2022

Reviewer #1: Dear, this is an important topic. However I have some comments:

- the background section reveals interesting papers. However the WHO has published important documents on this issue and I think it is important toe read them and see what can be worthwhile (1979 en 2020°. As I was told WHO has an e-learning on this topic. Please control it and give comments on what it can do on the education topic

We have added a statement in our discussion describing the tools available from the CDC and WHO for training providers on death certification (lines 287-89).

- line 71: 4 groups is a low figure and 2 tot 5 participants is also a low figure. comment on that in the discussion section

Although the number of participants was relatively small, we conducted groups until saturation was reached – that is, the ideas generated during the discussions were no longer “new”, despite representation of a range of medical specialties among the participants. In the methods section we cite a paper from Guest et. al who found that 80% of themes are captured in 2-3 focus groups and 90% are captured in 3-6 focus groups in support of our decision to conduct 4 focus groups. A statement was added to the discussion to highlight that the low number of participants (2-5 per group) is a potential limitation of this study (lines 313-16).

- method: check for the Coreq criteria for the publication of Q research.

We reviewed the Coreq criteria for qualitative research and the methods section is in line with the recommendations for the study design including a description of the theoretical framework, participant selection (purposive sampling), setting and data collection. The methods section has been revised in several places to reflect this.

- line 99 it is predominantly in-patient 9 versus 5 it is almost the double. Please mention this as a weak point

We have added this discussion to line 313: “The relatively small size of some of the focus groups and greater proportions of hospital-based participants may have limited the extent of novel ideas and quality of group discussion”.

- there are important papers explaining, like for cancer, that dementia is a problem: dying with or dying caused by dementia. Can you discuss that in the discussion section

The particular focus of the paper was cancer (we have now revised to make this clear in line 69). The case example was cancer-related, and our grant funding was cancer-related. While we agree that dementia can also be a challenging diagnosis for providers to navigate when completing a death certificate, dementia did not come up as a theme in our focus groups and would not have been expected to given the focus of the study, and therefore falls outside of the scope of detailed discussion in the article.

- Line 141 are data available on the effectiveness of the electronic certification other than what you already mentioned?

We discuss some additional points about electronic certification in the discussion section (beginning on line 291), but outside of the studies referenced in this section there is a paucity of data directly comparing electronic certification vs paper death certificates in the United States.

- I was wandering that nothing came out of the results as to the problem of death certification in case of a doctor on duty, who does not know the patient and has only an (incomplete) medical file. Can you discuss this issue, because in other studies it is a problem.

This issue was brought up by participants in the second section of the results starting on line 6-162, and we have now included this in the discussion on lines 277-278.

- line 164-170 if doctors have different views on the importance of death certificates, it has an impact on the motivation to do some training and effort to do i properly. What is you idea about it?

As a legal document that providers are required to complete, there should be an emphasis on providing all trainees either in medical school or residency with adequate training on how to accurately complete these documents. In the paragraph beginning on line 171 following table 2, there is discussion about the differing approached that providers took to resolving uncertainty in determining the cause of death. The differing approaches come from differing views on the purpose of the document rather than differing views on the importance of the document. Overall, participants found death certificates to be very important documents regardless of their approach to determining the cause of death and most supported the idea of additional training as described in the paragraph at the end of section 1: “Most targeted the lack of training or feedback, with several providers emphasizing that both training and consistent feedback are necessary for a document that is widely used for research and public health.” We have added language in the discussion on this topic on lines 225-27: “It was concerning that physicians’ views of the role and importance of death certificates to public health and to families might affect the approach they took to select a cause of death.”.

- discussion section: I think a table with all the possible errors is useful

We are not certain what the reviewer intends here but feel that the three figures provided in the results section present a reasonable summary of the study’s findings.

- line 273-275: is doing more autopsies a way to have better death certificates? In my clinical situation autopsies hardly are done (except in cases of criminality.

We agree. In the US and elsewhere, the rate of autopsies has decreased substantially for a variety of reasons, as discussed in lines 277-81 and it seems likely that this has influenced diagnostic accuracy on the death certificate.

- As to spending of public funding, the epidemiology derived form death certificates, is enourmously important. It is cleat that cardiovascular and cancer deaths are far overestimated. Can you discuss the consequences of all this? What is the effect on the value of some so called evidence based figures and guidelines?

We discuss the consequences of these errors in the introduction in order to describe why this study was undertaken: “In a 2010 study, deaths from cardiovascular disease in New York City were overestimated by 51% in adults ages 35-74.21 These errors have significant impacts on the public health data that underlie disease specific mortality rates, track health disparities and inform research and policy priorities. For example, in a study by Yin et al, misclassification of colon and rectal cancer deaths on the death certificate led to an inflation of the 5-year cause-specific survival rate for rectal cancer from 64.9% to 81.2%.22”

We have also included discussion on these topics on the potential impact of these errors on guidelines (lines 239-49).

We did a research, not yet published where 92 physicians filled death certificates of 5 case vignettes: I put the results from the abstract:

91 participants filled in a death certificate for 5 cases. 90% stated the correct nature of death. The chain of death causes was correct in 30% of cases, partially correct in 45% of cases and false in 25% of cases.

Thank you for telling us about this interesting study supporting the premise that cause of death is poorly documented in death certificates.

Reviewer #2: This is a very sound research article. You present a concise but thorough methodology, specifically one that includes exceptional analyses of the issue at hand. Particularly your discussion of the infrastructural challenges and the issues surrounding patients with cancer was refreshing and much needed in the discipline. The Background at the beginning of the paper is somewhat brief but what’s lacking in the Background section, I think you make up for in the Discussion section. You do an excellent job at laying out the importance of your findings, strengths, weaknesses, applicable recommendations, and ideas for future exploration in the Discussion.

In lines 285 – 287 you mention “Feedback is particularly important during public 286 health crises such as the opioid epidemic and Covid-19 pandemic to ensure that deaths are 287 certified using consistent methods…” I think it might be worth adding a few sentences to this statement. No need to add in a new section of the Discussion, as we don’t want to distract from the purpose of your paper. However, as these two occurrences are highly vulnerable to the cause death determining/certification process, it may be worth adding a few sentences discussing this issue further or adding in some recent citations on the issue. Overall, this was a very solid and impactful contribution. Thank you.

Thank you for this feedback. We added discussion on this beginning on line 293 to further explain the potential role that the real-time feedback of electronic medical records might have in improving accuracy without sacrificing timeliness in public health crises.

Reviewer #3: Summary of the research

Cause of death certification is a timely and important topic to study. I congratulate the authors for choosing this all-important topic.

This study has been conducted to characterize the experiences of the certifiers in completing the cause of death section of death certificates. Four focus group discussions were conducted over a period of three months until the thematic saturation was reached. Three challenges leading to misclassification of causes of deaths in death certificates have been identified during the study.

The rationale for the study is clear and valid. The researchers have used a technically sound protocol and a feasible methodology to achieve the study's aims effectively.

However, I do have the following concerns about their findings and conclusions.

The international standard death certificate format that WHO recommends is also the form used in the USA. This form has two parts, namely part 1 and part 2. Part 1 has four lines for reporting conditions leading to death in a logical sequence. Part 2 is for reporting other significant conditions that may have contributed to the death. The column to the right is for reporting the approximate time interval between the onset of each of these conditions in the certificate and death.

The design of the death certificate form itself is prescriptive for the correct reporting of causes of deaths in a logical sequence. The provider’s perception of the restrictive nature of the form is due to their ignorance of the international standard death certificate format and the best practice death certification standards required to select the proper underlying cause of death.

Electronic certification can improve certification quality by avoiding common death certification errors such as illegibility, use of non-standard abbreviations, etc. However, an electronic death certificate can also be restrictive to the providers due to suggestive texts, drop-down cause lists, and other functionalities included in the format.

Therefore the solution suggested by the providers may be due to their misunderstanding of the benefits of using electronic formats. In other words, the electronic form will never reduce the misclassification of causes of deaths caused by other certification errors such as reporting ill-defined underlying causes, incorrect sequencing of causes in part 1, reporting competing causes, etc.

The appointment and providing access to death coordinators within hospitals suggested by the providers could be a meaningful solution to improve the certification practices of doctors.

Lack of training and lack of real-time feedback are important challenges for the certifying physicians, and they have been described in the published literature as well. Clinical uncertainty and certifying deaths due to cancer are already known sources of certification errors, and in this study, those issues have been reiterated from a certifier's perspective.

We have revised the discussion (lines 290-305) to reflect these additional perspectives.

I also have the following concerns regarding their methodology;

1.The research question is relevant to the present context, and the rationale given in the introduction is satisfactory. However, a clear justification should be given for selecting FGD as the method of investigation.

We have added the rationale to the beginning of the methods section (lines 72-76).

2.Recruitment, sampling, and approach: The authors used a single case study and advocated the FGD at a single academic institution in rural New England. This selection can cause issues on the representativeness of the information obtained during FGD, therefore the generalizability of findings.

We agree this is a limitation of the study and have described the difficulty with generalizability in our limitations section (lines 310-12).

3.Strategic group bias is a recognized issue arising during organizing the FGD. The authors should explain the measures taken to minimize this bias.

We have added a sentence describing the measures taken to minimize group bias to the first paragraph of the methods section 16-(lines 87-89) and added this as a potential limitation to the discussion (lines 314-15).

4.The authors should elaborate in the methods section on the experience and the role of facilitators specifically related to the FGD.

We have added a sentence describing the role of the facilitators to the first paragraph of the methods section (lines 80-85).

5.The authors should explain why they use the constant comparison method as the method of thematic analysis (https://files.eric.ed.gov/fulltext/EJ1004995.pdf).

We have added a sentence explaining the selection of constant comparison analysis to the final paragraph of the methods section (lines 104-6).

6.The authors have not discussed the limitations of the study in the discussion section.

Our discussion of limitations is now expanded to include several additional limitations including the small sample size of the study (lines 310-15).

Reviewer #4: Undoubtedly the public health significance of death reports is important and unrealized by many trainees. As such the premise for this manuscript is relevant and of significance. I liked their focus groups and interventions and breaking it down to three challenges. They focused on unwitnessed deaths and cancer patients -- it would be nice for some estimates of the proportion of this. I wish they discussed the Electronic Death Certification better, more than just real-time feedback. Is this something in pipeline for all states - what are the roadblocks?

We have added more discussion of this in lines 290-305. One of the main roadblocks to wider implementation in the US is that each state is responsible for determining the process of death certification and therefore the US has not been able to roll out a nation-wide electronic death reporting system as many other countries have been able to. On the state level, one of the main roadblocks is funding the development and maintenance of these systems.

Accuracy vs Specificity was another great question, but it didn't provide an answer which is more important or how to balance them in uncertain and cancer cases.

While our paper identified this as a challenge for providers, the topic and potential solutions are likely complex and we did not have space to do them justice within the paper.

The accuracy/specificity conundrum stems from the clinician’s understanding of the role of the death certificate. Eight participants correctly view the death certificate as a legal/epidemiological tool, giving accuracy predominance. For example, the lung cancer patient who dies with one or more complications of their primary disease is properly certified as having died of complications of lung cancer, as opposed to “bronchopneumonia” or “pulmonary embolism.” Unfortunately, applying this principle too broadly leads to technically accurate, but etiologically nonsensical causes of death such as “cardiac arrest.” Six misidentify the death certificate as an element of the medical record, thus leading to the certification of sequelae of the underlying cause of death as the proximate cause. As stated above, while a legitimate question for further study, this discussion exceeds the scope of and purpose of this study beyond what is included in the discussion section.

Finally, the autopsy requirement was an interesting area. Outside scope of paper but would love to hear reasons behind this change - cost, resources, invasiveness. Is the accuracy of death certificates worth changing this policy for?

We believe that the change in Joint Commission guidelines (referenced in the paper) may underlie this change, driven primarily by cost.

Not for nothing was the term “gold standard” used in conjunction with the autopsy informing the accuracy of death certification. It is simply a myth that current diagnostics render the classic autopsy obsolete. There is an extensive literature on this subject. This flawed argument is the only medically related justification for the Joint Commission abandonment of autopsy as a quality control/quality assurance tool. The others, largely denied, are financial (hospital pathologists are loathe to spend billable time on non-reimbursed procedures that could take them hours because of their lack of autopsy experience) and hospital risk management departments inordinate fear of litigation. The latter is truly ironic in that a quality hospital autopsy prevents more litigation than it supports. The death of the hospital autopsy is largely confirmed by the fact that new hospital construction and/or renovation eliminates or excludes an autopsy suite. The expectation that the Joint Commission will reinstate a meaningful autopsy requirement is akin to expecting extruded toothpaste to be forced back into the empty tube.

Overall, very interesting topic with great provider identification and focus groups. I think the most useful intervention is improved education in preparing death documents with systems for real-time feedback (either on phone or electronic.) The largest interventions are a leap and would need more nuanced discussion, outside scope of this article.

Thank you for this feedback.

Submitted filename: PLOS ONE Response to Reviewers.docx

Click here for additional data file.

3 May 2022

Provider Reported Challenges with Completing Death Certificates: A Focus Group Study Demonstrating Potential Sources of Error

PONE-D-21-40129R1

Dear Dr. Morgan,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Benjamin P. Geisler, M.D., M.P.H., F.A.C.P., M.R.C.P. (London), F.H.M.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: You answered succesfully to my remarks and I hope that this paper will have an interesting impact. About line 69 dying with or dying caused by it was not my goal to comment on dementia (it was just an example of my own research). It is important to make the difference between the two concepts.

Reviewer #2: Thank you for your revisions. Your additions add nicely to the quality of your study and help to position your findings in contemporary contexts.

Reviewer #3: You have addressed all my previous comments adequately. Thanks for choosing such an interesting topic for your study.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: De Lepeleire Jan

Reviewer #2: No

Reviewer #3: No

11 May 2022

PONE-D-21-40129R1

Provider Reported Challenges with Completing Death Certificates: A Focus Group Study Demonstrating Potential Sources of Error

Dear Dr. Morgan:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Benjamin P. Geisler

Academic Editor

PLOS ONE