Comparison of the Causes of Death Identified Using Automated Verbal Autopsy and Complete Autopsy among Brought-in-Dead Cases at a Tertiary Hospital in Sub-Sahara Africa.

BACKGROUND: Over one-third of deaths recorded at health facilities in Zambia are brought in dead (BID) and the causes of death (CODs) are not fully analyzed. The use of automated verbal autopsy (VA) has reportedly determined the CODs of more BID cases than the death notification form issued by the hospital. However, the validity of automated VA is yet to be fully investigated.
OBJECTIVES: To compare the CODs identified by automated VA with those by complete autopsy to examine the validity of a VA tool.
METHODS: The study site was the tertiary hospital in the capital city of Zambia. From September 2019 to January 2020, all BID cases aged 13 years and older brought to the hospital during the daytime on weekdays were enrolled in this study. External COD cases were excluded. The deceased's relatives were interviewed using the 2016 World Health Organization VA questionnaire. The data were analyzed using InterVA, an automated VA tool, to determine the CODs, which were compared with the results of complete autopsies.
RESULTS: A total of 63 cases were included. The CODs of 50 BID cases were determined by both InterVA and complete autopsies. The positive predictive value of InterVA was 22%. InterVA determined the CODs correctly in 100% cases of maternal CODs, 27.5% cases of noncommunicable disease CODs, and 5.3% cases of communicable disease CODs. Using the three broader disease groups, 56.0% cases were classified in the same groups by both methods.
CONCLUSION: While the positive predictive value was low, more than half of the cases were categorized into the same broader categories. However, there are several limitations in this study, including small sample size. More research is required to investigate the factors leading to discrepancies between the CODs determined by both methods to optimize the use of automated VA in Zambia. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Background and Significance

Accurate and timely cause of death (COD) data are critical for guiding health policies and effectively mobilizing resources based on priority health issues. 1 Ideally, all deaths should be registered, and the CODs should be obtained from accurate death certificates. 2 However, a United Nations Statistics Division report demonstrated that only 55% of countries and regions worldwide achieve death registration rates ≥90%. 3 Approximately 60% of African countries achieved death registration rates <50%. 3 Therefore, improving civil registration and vital statistics (CRVS), including death registration, has become a key topic in global public health. Goal 16 of the Sustainable Development Goals 4 5 includes a target related to CRVS to promote human security. To address these issues and improve coverage of the CRVS, the health sector could play a key role by acting as an entry point, collecting accurate vital data, and utilizing information from CRVS. 6

The Republic of Zambia is facing substantial challenges in collecting accurate death information. According to the Central Statistics Office, the 2016 death registration rate was 20%. 7 The low death registration rate may be attributed to the large proportion of deaths that occur outside of health facilities. According to the sample vital registration with the verbal autopsy (VA), approximately 50% of deaths occur at home. 8 Further, the information system of the Zambian Ministry of Health in 2016 revealed that more than one-third of deaths in health facilities occurred before arrival at a facility; these deaths are termed brought-in-dead (BID). The CODs of BID cases are not thoroughly analyzed.

VA is a method of gathering information about deceased individuals' symptoms and circumstances to determine their COD. Health information and a description of events prior to death are acquired from conversations or interviews with a person or persons familiar with the deceased. 9 VA is considered a realistic alternative to death certificates to classify the CODs of BID cases. 10 11 The World Health Organization (WHO) has recommended using VA to accumulate data regarding death statistics and to capture CODs to identify trends. 12 While many countries have incorporated VA into the public health information system, 13 14 15 16 physician-based VA for all BID cases may not be feasible due to workload and costs and the automation of VA reporting could be useful to improve data collection regarding mortality. 17 Therefore, automated VA tools have been developed to enable trained personnel other than health care workers to reliably determine the COD of BID cases using a computer program that can assign the most probable COD using background information obtained during standardized interviews.

Based on literature review of previous publications, several automated VA programs such as InterVA, Naïve Bayes Classifier, and InSilicoVA 18 19 20 21 22 have been used to analyze COD in previous studies. 23 24 25 26 27 According to WHO, these automated VA programs have strengths and weaknesses, depending on the setting and the target cause. Currently, there are no recommendations for any particular algorithm. We previously evaluated the automated VA program Tariff Method 2.0 to identify the COD of BID cases from the main health facility in Zambia. 28 We compared COD data obtained using automated VA with the COD data on the death notification form based on insufficient background information and observed that the Tariff Method could determine the COD among more BID cases than the death notification form. Regarding the accuracy of COD data, a previous report indicated that there is a 50.0% chance that the Tariff Method automated VA can correctly identify the COD in adult BID cases. 29 However, any of the automated VA programs' validity has not yet been thoroughly investigated, as it was not compared with the results of complete autopsies. 22 28 30 31 32 33 34 35 The applicability of automated VA in Zambia to determine the COD of BID cases must be determined by comparing the VA results with those of actual autopsies. Therefore, this study investigates an automated VA tool's validity by comparing the results obtained using an automated VA tool with those obtained using a complete autopsy in BID cases in Zambia.

Objectives

To compare the COD identified by an automated VA with those by complete autopsy to examine the validity of a VA tool.

Methods

Study Design

This cross-sectional study aimed to compare the COD of BID cases identified by an automated VA and complete autopsies at the University Teaching Hospital, a tertiary hospital in Lusaka, the Republic of Zambia.

Lusaka has a population of approximately 2.5 million. 36 As this was the only tertiary hospital in Lusaka with a forensic pathologist when this study began, most BID cases in the city were referred to this hospital. Due to limited resources and a high number of deaths, the cases that died from external causes, such as injuries, suicide, and homicide, were excluded; only BID cases of individuals >13 years of age who were brought to the facility during the daytime hours from Monday to Friday from September 2019 to January 2020 were included. BID was defined as a patient who had died before they arrived at the hospital.

Data Collection of Automated VA

The staff who were employed by the Ministry of Home Affairs collected VA data to determine the CODs of the BID cases. First, they asked triage questions to exclude cases who died from external causes and informed consent was obtained from the relatives of each BID case with internal CODs. The staff conducted interviews regarding the death's background using a standardized questionnaire of the 2016 WHO VA instrument (in English). 37 During the interview, the staff used the Android application Open Data Kit (ODK) Collect to convert the collected data to an electronic format. 38 The data stored in ODK Collect were transferred to the Civil Registration Office in Zambia, where they were converted to comma-separated value files. These files were used with InterVA-5 software to identify the CODs as 1 of 53 categories for adults. The Civil Registration Office shared the COD results from InterVA-5 software with our team of researchers. InterVA-5 was chosen as the automated VA tool because the Civil Registration Office already used InterVA-5 to determine CODs among BID cases.

InterVA-5

The InterVA model 21 39 40 is a Bayesian probability theory-based algorithm developed in 2003 and has since been revised. Bayes' theorem defines the conditional probability of a cause in the presence of a particular indicator. 34 39 41 In building the model, prior probabilities were assigned by expert panels using a semiqualitative scale for each indicator and COD. The likelihood of each COD is calculated by applying the aforementioned theorem, considering each pertinent indicator reported in the VA interview. 42 InterVA can also determine the COD with the highest propensity in a population, resulting in population-level data regarding COD. For each case, InterVA reports single-value point estimates for the three CODs with the highest propensities if they fall above a set threshold. If any COD does not meet the threshold, the death is ruled indeterminate. InterVA-5 has been developed to accommodate the 2016 WHO VA standards and is currently under testing. 22 The product information of InterVA-5 (version 5.1) is available at http://www.byass.uk/interva/ .

Data Collection of Autopsies

A forensic pathologist trained in both Zambia and Canada and a surgical pathologist conducted all autopsies, including the pathological investigations, and recorded the CODs in the study registry. Basic demographic data including sex, age, and research identity were also included. Each COD was categorized into the most appropriate category on the COD list in the 2016 WHO VA instrument by mutual agreement among the three physicians so that the results of CODs by automated VA and complete autopsy could be directly compared.

Data Verification

We verified that the cases with consent for participation were located in the official civil registry using the patient names, ages, and death dates. If we could not confirm the data's consistency, the case was excluded from the data analysis.

Statistical Analyses

The 10 most common CODs obtained using the automated VA and complete autopsy were listed and compared. The positive predictive value of automated VA was calculated using four broader disease categories: communicable disease, noncommunicable disease (NCD), injury, and maternal CODs. These results were also described by a visual graph. STATA version 16 statistical software (StataCorp., College Station, Texas, United States) and Microsoft Excel 2013 (Microsoft, Redmond, Washington, United States) were used for the data analyses. The concordance of CODs between the two methods was estimated using Cohen's kappa coefficient and was considered excellent if the coefficient was from 1.0 to 0.8; good if the coefficient was from 0.6 to 0.8; moderate if the coefficient was from 0.4 to 0.6; weak if the coefficient was from 0.4 to 0.6, minimal if the coefficient was from 0.2 to 0.4, and none if the coefficient was less than 0.2. 43

Results

Demographic Data

Fig. 1 shows the flowchart of BID cases enrolled in this study. A total of 152 cases were eligible for this study; 47 cases were excluded as injury cases, the relatives of 33 BID cases chose not to participate, and 9 cases were not verified with the Civil Registration Office. Therefore, the final analysis included 63 BID cases. The demographic data for the BID cases are presented in Table 1 . The median age of the BID cases was 41.0 years (interquartile range: 32.0–52.0 years) ( Fig. 2 ) and 76.2% were male.

Fig. 1

Flowchart of participant enrollment.

Table 1

Characteristics of eligible participants

Total number	63
Male (%)	76.2%
Age (y) a	41 (32.0–52.0)

Age is presented as median and interquartile range.

Fig. 2

Age distribution of the participants.

CODs by Automated VA and Autopsy

Of the 63 BID cases included in this study, the CODs were determined by both automated VA and autopsy in 50 cases (79.4%) ( Table 2 ). The 10 most common CODs determined by each method are shown in Table 3 . The most common CODs determined by autopsy were different from those determined by automated VA. Supplementary Table S1 (available in the online version) shows the full lists of the CODs determined by each method. Table 4 shows the number of CODs in each of the four broad categories used in this study. The positive predictive value of InterVA-5 in the total population was 20.0% (10/50) and the kappa coefficient was less than 0.2. While the positive predictive value of InterVA-5 for maternal CODs was high (100%; 2/2), it was low for NCD CODs (25.0%; 7/28) and communicable disease CODs (5.0%; 1/20). The kappa coefficient of each broader category was less than 0.2. The proportions of CODs between InterVA-5 and autopsy in four disease categories are shown in Table 5 and they indicate that 56.0% of all cases were diagnosed in the same disease categories by both methods. Fig. 3 also describes these results in a bar graph.

Table 2

The number of cases whose CODs could not be determined

	Number	Percentage
Undetermined by both VA and autopsy	4	6.3%
Undetermined by VA	6	9.5%
Undetermined by autopsy	3	4.8%
Determined by both VA and autopsy	50	79.4%

Abbreviations: COD, cause of death; VA, verbal autopsy.

Table 3

Top CODs identified by InterVA and autopsy corresponding to VA COD list

Automated VA tool ( N  = 50)				Autopsy ( N  = 50)
Rank	Cause of death	No	%	Rank	Cause of death	No	%
1	Acute cardiac disease	18	36.0%	1	Other CVD	10	20.0%
2	HIV/AIDS	9	18.0%	2	Pneumonia	6	12.0%
3	Epilepsy	5	10.0%	3	Acute cardiac disease	6	12.0%
4	Pneumonia	4	8.0%	4	Poisoning	5	10.0%
5	Pulmonary TB	4	8.0%	5	Other infectious disease	5	10.0%
6	Stroke	3	6.0%	6	Unspecified infectious disease	5	10.0%
7	Ectopic pregnancy	2	4.0%	7	Stroke	4	8.0%
8	Diarrheal disease	1	2.0%	8	Meningitis	3	6.0%
8	Other CVD	1	2.0%	9	Other NCD	3	6.0%
8	Meningitis	1	2.0%	10	Ectopic pregnancy	2	4.0%
8	Other infectious disease	1	2.0%	10	Epilepsy	2	4.0%
8	Other NCD	1	2.0%

Abbreviations: COD: cause of death; CVDs: cardiovascular diseases; NCDs: noncommunicable diseases; TB: tuberculosis; VA: verbal autopsy.

Table 4

Proportion of cases with CODs as determined by InterVA and autopsy

Disease category	No. of automated VA	No. of matched cases with autopsy	PPR(%)	Kappa coefficient
Communicable diseases	20	1	5.0%	<0.2
NCD	28	7	25.0%	<0.2
Maternal reasons	2	2	100%	NA a
Total	50	11	20.0%	<0.2

Abbreviations: NCD, noncommunicable disease; PCOD, cause of death.

NA: Not analyzed due to too few samples

Table 5

Proportion of cases with disease categories as determined by InterVA and autopsy

Disease category	No. of automated VA	No. of matched cases with autopsy	%
Communicable diseases	20	9	45.0%
NCD	28	17	60.7%
Maternal reasons	2	2	100%
Total	50	28	56.0%

Abbreviations: COD, cause of deaths; NCD, noncommunicable disease.

Fig. 3

Matched proportion of CODs between InterVA and complete autopsy by InterVA and Broad categories. CODs, causes of death.

Discussion

This study compares the CODs determined by complete autopsy with those determined by automated VA in 63 BID cases. As the forensic pathologist who conducted the autopsies is an expert who received professional training in Zambia and Canada with several years of experience at the target hospital, and another surgical pathologist worked with him for quality control purposes, the autopsy results can be assumed to be reliable sources of CODs. According to the results, the positive predictive values of InterVA-5 on CODs determined by complete autopsy were as low as 22% for the total population.

There are several factors related to weaknesses of automated VA that may have led to the low positive predictive value observed in our study. Regarding the methodology of VA, there may be challenges in differentiating some diseases with similar symptoms such as acute cardiac disease and stroke, acute cardiac disease and pneumonia, and epilepsy and convulsion from poisoning. To accurately identify the COD of these cases, detailed information of clinical presentation before the deaths needs to be collected. However, in the circumstances where VA is conducted, it may be hard to obtain the accurate data because some important information is limited to the deceased and their closest relations. Therefore, the selection of interviewees may have affected the unfavorable results of this study. Since the surveyors interviewed the persons who brought the deceased to the health facility, the interviewees were not necessarily those who were the most informed regarding the deceased's health course.

In addition, the categorization of CODs in the 2016 WHO VA instrument may have contributed to the low positive predictive value in this study, as some CODs determined by autopsy could not be represented well in the categories of the instrument. For example, four BID cases identified with disseminated tuberculosis (TB) could not be categorized accurately on the 2016 WHO VA instrument, as it only includes pulmonary TB. 37 As pulmonary TB and disseminated TB correspond to different ICD-10 (International Classification of Diseases 10 th Revision) codes (A15–16 and A19, respectively 44 ), we categorized disseminated TB as “other infectious disease.” Additionally, several BID cases with a ruptured aortic aneurysm were identified on autopsy, including some that were classified as acute cardiac disease by InterVA. According to the 2016 WHO VA instrument, acute cardiac disease does not cover ruptured aortic aneurysm. However, as the two diseases' presentations are similar, InterVA misclassified the disease as acute cardiac disease instead of other and unspecified cardiac disease, lowering the positive predictive value. These diseases could be classified in the same category since the interventions are also similar.

Furthermore, there could be other factors related to the automation of VA. First, InterVA may have internal problems leading to a low positive predictive value for COD. 45 The propensity used by InterVA to assign CODs is based only on the presence of signs or symptoms and does not account for the absence of signs or symptoms. For example, a recent negative human immunodeficiency virus (HIV) test result is not considered when determining COD, even though it is helpful. In a previous study, InterVA diagnosed HIV/acquired immune deficiency syndrome (AIDS) more frequently than physician-certified VA (PCVA). 46 Moreover, in this study, for three BID cases with meningitis as the COD, according to the autopsy, InterVA identified HIV/AIDS as the COD. In addition, even under the best circumstances, there is variation in the presentation of symptoms for a given COD. In the context of VAs, this variation is compounded by the added variability that arises from the interviewee's ability to recall and correctly identify signs or symptoms. Therefore, a broad variability in the ability to correctly answer the interview questions likely exists.

Second, the automated VA was not validated by directly comparing with the CODs determined using complete autopsies. For the validation, some studies have compared results obtained using automated VA with PCVA reports, which is a method of determining COD via reaching a consensus among physicians based on data obtained using the structured VA questionnaire. The reported concordance between the automated VA tool and PCVA in previous studies ranged from 44.0 to 83.1%. 30 33 47 48 49 By disease categories, there is a Kenyan study reporting that the concordance for noncommunicable disease CODs and communicable disease CODs based on PCVA was 71.5 and 71.7%, respectively. 49 However, the gold standard for determining CODs may not be the PVCA report, but complete autopsy. 50 Although few studies investigated the validity of PCVA based on complete autopsy, 51 Misganaw et al compared PCVA to medical data from hospital records in Ethiopia and revealed that the concordance between the two sources was 68% for communicable disease CODs and 79% for NCD CODs. 52 Furthermore, CODs determined using hospital data reviews and complete autopsy results had 30 to 63% discrepancy among 18 previous studies. 53 Taken together, multiplication of these data suggests that estimated concordance between automated VA and complete autopsy should be approximately 20 to 45%. However, direct investigation to compare both methods is necessary to estimate the true concordance.

In addition to the factors possibly responsible for the low positive predictive value mentioned above, there are several limitations to the current study. First, the study population may not represent the general population of BID cases due to the small sample size. Especially, there were only two maternal cases. Also, the case selection could be biased as a substantial number of BID cases that arrived outside of the study period were not included and a large proportion of eligible BID cases were not included due to the lack of consent. This may account for the most common CODs in our study compared with the previous study. 28 Therefore, our results may not be fully generalizable. Furthermore, some subgroups were excluded from this study. In Zambia, cases of death due to external causes require legal procedures for forensic autopsy and the manner of death such as accident, suicide, or homicide is recorded in the forensic report. Since these data were confidential, which meant that the CODs of external cases could not be adequately collected within our research framework, this study focused only on BID cases associated with natural causes of death. Pediatric cases were also excluded owing to the potential challenge of obtaining the consent for autopsy from their parents. Therefore, the validity of results obtained through the InterVA-5 tool compared with those obtained from complete autopsy could not be comprehensively evaluated. Lastly, the quality of the interviewers could also cause the low positive predictive value. For example, data were missing in some cases, especially regarding the duration or the severity of symptoms. While all of the interviewers were trained to conduct VA interviews, the training was provided several years before the research period. As the 2016 WHO VA instrument strongly recommends structured training for interviewers before conducting VA interviews, 37 a lack of refreshment training for interviewers prior to the study period may account for the low positive predictive value.

Considering the factors and the limitations mentioned above, more research is required to investigate the discrepancy between the CODs determined by automated VA and complete autopsy; this work should use larger sample sizes and carry out extensive subgroup analysis. Moreover, updated training for interviewers may significantly improve the two methods' positive predictive values. The effects of refreshment training for interviewers on the results of automated VA in Zambia should indicate the improvements that are necessary for the use of this system in routine national information systems. Automated VA should be primarily utilized to replace PCVA where health resources are limited. The objectives of using automated VA are to provide better COD statistics in places with resource scarcity and implicate the public health interventions required to address CODs. As Table 5 shows, automated VA successfully assigned CODs among more than half of BID cases into the same broader disease categories as did autopsy. Classifying community deaths using the broader categories can still be very useful when prioritizing public health issues. The strengths and weaknesses of automated VA programs must be recognized before these processes are put into use in the real-world setting.

Conclusion

This study compares the CODs assigned to BID cases by automated VA with those determined by complete autopsy to validate the results of automated VA. Several reasons for the low agreement may exist, including the validation method of automated VA, the quality of VA interviews, and the limited lists of CODs available in InterVA-5. Meanwhile, since the concordance under the broader categories was more than 50%, automated VA could be still useful to capture the trend of disease burdens for the public health policy formulation. However, there are several limitations in this study such as small sample size, insufficient training of interviewers, and exclusion of particular subgroups. More research is required to investigate the factors leading to discrepancies between the CODs determined by both methods to optimize the use of automated VA in Zambia.

Clinical Relevance Statement

Automated VA is useful to classify CODs as per disease conditions among people who died in the communities. However, to scrutinize the CODs, the limitations of automated VA need to be understood. Further research should be considered to optimize the use of automated VA in Zambia.

Select the answer from a to d, which indicates the sets of events to be collected for vital statistics purpose by the United Nations.

➀ Marriage

➁ Death

➂ Legitimation

➃ Annulment

All,

➀ and ➁

➀ to ➂

➀, ➁, and ➃

Correct Answer: The correct answer is option a. The United Nations defines all events for vital statistical purposes. Refer to https://unstats.un.org/unsd/demographic/standmeth/principles/m19rev3en.pdf .

What percentage of the countries, territories, and areas in the world has at least 90% coverage of death registration?

34%

51%

68%

85%

Correct Answer: The correct answer is option c. According to the United Nations, 68% of countries, territories, and areas in the world have more than 90% of death registration coverage. Refer to https://unstats.un.org/unsd/demographic-social/crvs/#coverage .