Validation of the verbal autopsy questionnaire for adult deaths in Iran.

Background: Verbal Autopsy Questionnaire (VA) is an important tool to estimate the cause of death among those populations lacking an appropriate death registration system. In this study, the validity and reliability of verbal autopsy were assessed.
Methods: The Persian version of the questionnaire was prepared using the translation and back- translation method. In the first and second phases of the study, 213 and 198 families of deceased persons accepted an invitation to complete the questionnaire. A physician determined the cause of death. These causes were compared with the registered cause of death on the death certificate. Sensitivity, specificity, positive predictive value (PPV), as well as the kappa statistic (between the first verbal autopsy questioning and death registry) were calculated to determine the validity of the questionnaire. Kappa statistic was also used to determine the reliability between the first and second questioning.
Results: The sensitivity of the questionnaire varied from 75% among deaths due to diabetes to 100% Due to breast cancer. The specificity of the questionnaire for all causes of death was higher than 97%. PPV varied from 62.5% to 100% for all causes of death. The kappa statistic between causes of death derived from death certificate, and the first VA questioning for all causes of death was above 0.7 (p<0.001), and it was above 0.78 for the first and second verbal autopsy questioning (p<0.001).
Conclusion: Although the Verbal Autopsy Questionnaire does not fully identify all causes of death, it can be a useful tool for diagnosing causes of death for those deceased persons who have no death certificate.

↑ What is “already known” in this topic:

The accurate record of the cause of death can be important for policy- makers. Many deaths occur with unknown causes, and Verbal Autopsy Tool can help determine those causes of death that are unclear. In this study, the validity and reliability of the Verbal Autopsy tool was examined.

→ What this article adds:

This study showed that Verbal Autopsy Tool has an almost good validity and reliability to detect causes of adult deaths.

Introduction

Information on causes of death is essential for policy- makers, planners, and managers to plan, implement, monitor, and evaluate public health at every level (local and district, provincial, national and international levels) (1).

Inadequate reliable information on mortality rate and cause of poor inhabitants in the world causes limitation of all efforts to establish strong evidences for health policymaking, planning, monitoring, and evaluating. In those regions where the majority of deaths happen at home and there is a lack of vital incidences registry systems, there is little chance to document deaths that occur far from health centers, let alone registering their causes (2). Many countries with the highest burden of disease lack a routine and high-quality information on causes of death (3). Therefore, there is a vital need for a tool to determine the cause of death. Verbal Autopsy (VA) Questionnaire is a suitable tool to determine death causes in those regions with or without poor registry system (4).

Verbal autopsy is a method to determine the cause of death and for this purpose, care takers, friends, or family members are asked for signs and symptoms of the patient prior to death. Further autopsy interviews will be conducted using standard questionnaires to gather details of signs and symptoms, complaints, and any previous medical history and events. The cause of death or the sequences of death causes are determined based on gathered information through this questionnaire and any available information. To avoid recall bias among respondents, it is recommended to assess registered deaths in the past years, choose respondents who were near the deceased person at the time of death, and avoid asking those who were not beside the deceased person prior to death (5). This questionnaire is being used in various countries, especially in less developed countries (2, 6-8). It has also been used in a cohort study in Golestan, Iran, and the results were acceptable; however, further studies are recommended (9).

Each tool entered into a new language and culture should have suitable validity and reliability (10, 11). Validity and reliability of each tool indicate its accuracy and precision. The low error rate increases confidence in the tool (12).

In Iran, death registration system was started as a pilot project in 1998 in Bushehr, Iran, and it is being implemented regularly in all provinces of Iran (31 provinces) since 2004. Completing death certificate and death registry is being implemented in hospitals, health centers, cemeteries, and forensics of each city (13); however, similar to all other countries, under numeration, misclassification, and incorrect registry are 3 problems in death registry in Iran (14). Also, to collect accurate information on the causes of death in all countries and for comparability of countries, The WHO has recommended the same death certificates and verbal autopsy tools (15). Thus, verbal autopsy is required to be recorded more accurately. Considering the need for VA tool in Iran, this study was conducted to translate this tool and evaluate its validity and reliability in Iran.

Methods

In this 2-stage cross-sectional survey, the English version of the Verbal Autopsy Questionnaire was translated into Persian (16) by an epidemiologist and a linguist. The Persian translation was back-retranslated into English by a different team including a physician and a linguist; then, the Persian version of Verbal Autopsy Questionnaire was prepared.

Those who have died in the last year (1239 deaths) were selected by simple random sampling from the archive of Lorestan Health Centre, Fars province, Iran. Inclusion criteria for the study were as follow: age older than 15 years at the time of death, deaths occurring within the past year, and the cause of death had to be accurately recorded in the hospital. Two nurses were selected as interviewers and were trained for questioning skills. They visited the homes of the deceased persons after obtaining oral consent from the family members and explaining the aim of the study. After questioning, the cause of death was determined by a trained physician using the completed questionnaires.

Sample size

According to the theory of Fire et al., 100 to 400 people are needed to assess the validity of the questionnaire (17). In the current study, 250 adult cases (over 15 years old at time of death) were selected for the VA Questionnaire by random sampling from 1012 recorded deaths in the health centre in 2013. After explaining the aim and the method of the study, 213 family members agreed to cooperate. In the second stage of the study, 198 family members agreed to cooperate. The purpose of the first phase of the study was to determine the validity of the questionnaire, and in the second phase of study, the reliability of the questionnaire was assessed. The second phase was performed 2 weeks after the first phase. Before starting the study, informed consent was obtained from all participants, and they were assured of the confidentiality of their personal information.

Validity and reliability assessment

Sensitivity, specificity, and positive predictive value were used to determine the construct validity, and kappa index was used to assess the criterion validity of the questionnaire. The gold standard diagnosis of the cause of death for assessing the validity of the VA was the cause of death recorded in hospital medical records.

The ability of the questionnaire to correctly diagnose the cause of death due to a particular cause is called sensitivity. In the current study, specificity is defined as the ability to correctly diagnose that someone has not died of a specific cause. Positive predictive value is defined as the correctness probability of the determined cause of death for a person (14). The agreement between the registered death cause and the death cause obtained from the first stage of the VA Questionnaire was used to measure validity, and agreement between the first and second stages of VA Questionnaire was used to measure reliability.

Statistical analyses

Sensitivity, specificity, positive predictive value, accuracy index, and kappa index were computed with Microsoft Office Excel 2007. A 95% binomial proportion confidence interval was calculated for all indicators (sensitivity, specificity, and positive predictive value) by an exact method in Stata 11 software.

Results

The study was conducted on 213 deaths, 134 (62.9%) males and 79(37.1%) females. The time for interview and competing the questionnaire was 32±11 minute. The greatest sensitivity was in the diagnosis of breast cancer (100%) and the lowest was in the diagnosis of diabetes (75%). Table 1 demonstrates the values of sensitivity, specificity, and positive predictive value of each cause of death.

Table 1

Validation indicator (sensitivity, specificity, and positive predictive value) of VA cause of death

Cause of death	Death certificate	VA *	Death certificate & VA *	Sensitivity (95% CI)	Specificity (95% CI)	PPV ** (95% CI)	Accuracy	Kappa
Brest cancer	6	7	6	100	99.5(97.33-99.9)	85.7 (42.1- 99.6)	99.53	0.79
Cerebrovascular disorders	25	28	23	92.0 (73.9- 99.0)	97.3(93.9-99.1)	82.1 (63.1- 93.9)	97.65	0.84
Transport accident	10	11	9	90.0 (55.4- 99.7)	99.0 (96.4- 99.8)	81.8 (48.2- 97.7)	98.59	0.84
Colorectal cancer	7	8	6	85.7 (42.1- 99.6)	99.0(96.5-99.8)	75.0 (34.9- 96.8)	98.59	0.79
Lung cancer	7	9	6	85.7 (42.1- 99.6)	98.5(95.8-99.6)	66.6 (29.9- 92.5)	98.12	0.74
Other cancers	12	12	10	83.3 (51.5- 97.9)	99.0(96.4-99.8)	83.3 (51.5- 97.9)	98.12	0.82
Stomach cancer	6	8	5	83.3(35.8-99.5)	98.5 (95.8-99.7)	62.5 (24.4-91.4)	98.12	0.70
Respiratory disease	16	13	13	81.2 (54.3-95.9)	100	100	98.59	0.88
Other causes	22	19	18	81.8 (59.7- 94.8)	98.4 (95.4- 99.6)	94.7 (73.9- 99.8)	97.65	0.86
Ischemic heart disease	32	29	28	87.5 (71.0- 96.4)	99.4(96.9-99.9)	96.5 (82.2- 99.9)	97.65	0.90
Liver disease	16	14	14	87.5 (61.6- 98.4)	100	100	99.06	0.92
COPD	16	18	14	87.5 (61.6 98.4)	97.9(94.8-99.4)	77.7 (52.3- 93.5)	97.18	0.80
Infectious disease	13	10	10	76.9 (46.1- 94.9)	100	100	98.59	0.86
Kidney disease	13	14	10	76.9 (46.1- 94.9)	98.0(94.9-99.4)	71.4 (41.8- 91.6)	96.71	0.72
Diabetes	12	13	9	75.0 (42.8- 94.5)	98.0(94.9-99.4)	69.2 (38.5- 90.9)	96.71	0.70
Total	213	213	181

VA*: verbal autopsy

PPV**: positive predictive value

From 213 deaths in the first phase of the study, 198 (92.9%) cases participated in the second VA phase. The cause of death in the first and second phases and kappa index are presented in Table 2. The 183 (92.42%) deaths in both phases had the same cause of death. Kappa index for each cause of death is presented in Table 2.

Table 2

Kappa statistic between first and second verbal autopsy questioning to measure reliability

Cause of death	VA * 1	VA * 2	VA * 1& VA * 2	Kappa statistic	P-Value
Brest cancer	7	7	7	1	<0.001
Ischemic heart disease	20	21	20	0.97	<0.001
Liver disease	13	14	13	0.96	<0.001
Transport accident	11	12	11	0.95	<0.001
Stomach cancer	8	7	7	0.93	<0.001
Colorectal cancer	8	9	8	0.93	<0.001
Cerebrovascular disease	25	26	24	0.93	<0.001
Respiratory disease	13	15	13	0.92	<0.001
Kidney disease	14	14	13	0.92	<0.001
COPD	16	14	14	0.92	<0.001
Other causes	19	18	17	0.91	<0.001
Total cancer	44	41	39	0.89	<0.001
Lung cancer	9	7	7	0.87	<0.001
Other cancers	12	11	10	0.86	<0.001
Infectious disease	10	11	9	0.84	<0.001
Diabetes	13	12	10	0.78	<0.001
Total	198	198	183

VA*: verbal autopsy

Discussion

In this study, causes of 85% of deaths were correctly diagnosed using VA Questionnaire, which can almost be considered a good estimation. In a study by Yang et al., in China, 96% of deaths were detected by VA questionnaires (2). Khademi et al., in Golestan, Iran, stated that nearly 80% of deaths, compared with the gold standard, were truly identified, which was almost similar to our study (9). Incompleteness in determining the cause of death could be due to the similarity of some symptoms of fatal diseases, which may cause misclassification.

Sensitivity for different causes of death varied and ranged from 75% to 100%. Diagnosis of breast cancer had the most sensitivity; in other words, all cases of death from breast cancer were detected using verbal autopsy, and this could be due to the specificity of the symptoms of breast cancer. The lowest sensitivity was related to diagnosis of diabetes. In other studies in other regions of the world, the sensitivity and specificity for different causes of death varied and had no similar pattern, and in each study, one or more causes of death showed higher sensitivity (2, 6-8).

Specificity is another indicator used in the assessment of diagnostic tests, which was calculated in this study. Specificity for all causes of death was higher than 97.3, which is very good. In other studies, the level of specificity has been acceptable and all reports have been higher than 90% (2, 7, 9). Because many deaths occurred with no particular cause, the amount of specificity was high.

Another important indicator calculated was positive predictive value; the maximum PPV was related to infectious and respiratory diseases (100%), and the lowest rate was associated with gastric cancer (62.5%). Kappa statistics among the causes listed on the death certificate and causes diagnosed by VA varied from 0.70 for diabetes and gastric cancer to 0.88 for respiratory diseases, which was appropriate and showed high agreement among the causes listed on the death certificate and VA. Also, kappa statistics between the first and second stages of VA to assess the reliability of the VA method revealed a very good agreement. Kappa statistics for all causes of death was higher than 0.84. In general, kappa statistic between 0.61 and 0.80 indicates considerable agreement and higher than 0.81 shows ideal agreement (18). In a systematic review on 19 studies around the world, the parameters of sensitivity and specificity for various causes of death varied in different study (6).

The skills of the interviewers and physicians can also lead to increased sensitivity and other indicators. Also, another most important limitation of this study and probably that of other studies is that the causes of death recorded in the death certificates in hospitals are considered as the gold standard, which may cause some problems. One is doubts about the authenticity of the recorded causes of death, and the other is that deaths recorded in hospitals are not representative of all deaths that occur in the society. Moreover, in this study, the sample size was small, which was due to such problems as limited financial resources. Lack of cooperation was another problem in this study. These limitations might have reduced the accuracy of the results.

Conclusion

The results of this study indicated that VA Questionnaire may not be an appropriate tool to identify some of the causes of death. However, it can be used in areas where there is no precise death registration system, or in areas where despite having a death registration system, death causes are not clear. VA Questionnaire can also be used for deaths that occur in places other than hospitals.

‏

Acknowledgements

We would like to acknowledge the research deputy of Kerman University of Medical Sciences and HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Study in Health, Kerman University of Medical Sciences for funding this study and all respondents to the questionnaire.

Funding/support

Research deputy of Kerman University of Medical Sciences and HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Study in Health, Kerman University of Medical Sciences, Kerman, Iran

Conflict of Interests

The authors declare that they have no competing interests.