Accuracy of mortality statistics in Palestine: a retrospective cohort study.

OBJECTIVE: To examine the accuracy of mortality statistics in Palestine, to identify gaps and to provide evidence-based recommendations to improve mortality statistics in Palestine. STUDY DESIGN AND
SETTING: A retrospective death registry-based study that examined a stratified random sample of death notification forms (DNFs) of patients who died in hospitals in Palestine was reported in 2012. We randomly selected 600 deceased from the Cause of Death Registry: 400 from the West Bank and 200 from the Gaza Strip. Analysis was based on the randomly selected deaths that we were able to retrieve the medical records for; 371 deaths in the West Bank and 199 deaths in the Gaza Strip.
RESULTS: Data in the Palestinian Health Information Centre (PHIC) registry had a low degree of accuracy: less than half of the underlying causes stated the correct cause of death. In general, deaths due to malignant neoplasms were more accurately reported on DNFs than other causes of death, and metabolic diseases (including diabetes) were the most problematic. Issues with coding and classification at the PHIC were most apparent for perinatal conditions and congenital anomalies.
CONCLUSION: Procedures for coding and classification at the PHIC deviate considerably from the international norms defined in the International Statistical Classification of Diseases and Related Health Problems (ICD) and account to a considerable extent for the discrepancies between the cause of death determined on the medical data on the death extracted from the deceased patient's hospital records and the cause of death coded by the PHIC. We recommend the introduction of international coding software for coding and classification, and a review to improve data handling in hospitals, especially those with electronic patient records. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The present study is the first assessment study of the accuracy of mortality statistics in Palestine and to the best of our knowledge; it is the first assessment study in the Middle East and North Africa region.

The present study is the first one to examine the completeness and accuracy of death notification forms.

The study was limited to hospital deaths.

The presence of deaths that occurred several decades ago in the sample indicates that reporting is incomplete. It is important to estimate the number of non-reported deaths and determine if the causes of death in such cases differ significantly from those actually reported to the authorities.

Introduction

Statistics on causes of death are essential indicators of the overall health and quality of life of a population. They can be used to monitor the status of health, estimate the burden of major diseases over time and across geographical regions, and to assess the impact of health interventions. However, this requires accurate statements on the cause of death and accurate statistics collected in accordance with international standards.1

In Palestine, the Cause of Death Registry (CoDR) held by the Palestinian Health Information Centre (PHIC) at the Ministry of Health (MoH) is part of the national vital statistics database and holds information on deaths and the causes of deaths since 1994. Prior to that date, causes of death were registered with the Israeli Civil Administration. International Statistical Classification of Diseases and Related Health Problems (ICD) has been used for coding since 1960s and in early 1999, PHIC started using ICD-10 instead of ICD-9.

The main source of data for the CoDR is the death notification form (DNF), issued by the attending physician. The DNF includes information about the main and underlying causes of death. The DNF is given to the family of the deceased, who are the ones responsible for notifying the death to the Primary Health Care (PHC) Directorate in each district. In the Gaza Strip, some hospitals keep a copy of the DNF but others do not. The PHC or the family of the deceased sends the DNF to the Ministry of Interior. At the end of each month, the PHC Directorate compiles all DNFs and mails them to the PHIC for coding and registration. The PHIC also receives some DNFs from government hospitals (figure 1). For neonatal deaths, there are no DNFs; instead, a monthly report of neonatal deaths is sent by each hospital to the PHIC. The PHIC codes the underlying causes of death and enters the death into the CoDR.

Figure 1

Palestinian cause of death statistical system.

We carried out this study to examine the accuracy of mortality statistics in Palestine. We examined hospital deaths, which accounts for 50%–60% of deaths in Palestine. This study will serve to identify information gaps and to provide evidence-based recommendations to improve mortality statistics in Palestine.

Methodology

Study design

This retrospective death registry-based study examined a stratified random sample of DNFs of patients who died in hospitals in the West Bank and Gaza Strip was reported in 2012. There are 81 hospitals in total in Palestine, with 51 in the West Bank (7 of them are in Jerusalem) and 30 in the Gaza Strip, serving around 4.78 million Palestinians (2.88 million in the West Bank and 1.9 million in the Gaza Strip). Around 33% of hospitals in Palestine are governmental. The MoH accounts for 44% of the bed capacity in the West Bank and 69% of the bed capacity in the Gaza Strip. We randomly selected 600 deceased from the CoDR: 400 from the West Bank and 200 from the Gaza Strip. The sample size was estimated using the SampleXS program2 based on the following assumptions: the number of registered deaths in 2012 was 12 000; the completeness of death certificates was 60% based on the recent Palestinian Central Bureau of Statistics (PCBS) study of the completeness of the CoDR3; the margin of error 5% and the design effect 1.5.

About 70% of the selected deaths in the West Bank occurred in government hospitals, while in the Gaza Strip, all deaths were in government hospitals. We took a random sample of hospital deaths from the following four age groups: less than 1 year, 1–17 years, 18–64 years and 65+ years from CoDR at PHIC. The study was carried out in 2014 in close collaboration with the MoH in the West Bank and Gaza Strip. The response rate was 100%; all selected hospitals in the West Bank and Gaza Strip agreed to participate in the study.

Material

We used several sources of data to assess the accuracy and completeness of the mortality data (figure 2). For each death in the sample, we collected the following data:

Figure 2

Study design flowchart. ICD-10, International Statistical Classification of Diseases and Related Health Problems, 10th revision.

Scanned image of the DNF.

Formal assessment of the DNF, such as legibility of the handwriting and if the administrative data had been filled out correctly with the certifier’s signature.

The causes of death in free text and administrative information were extracted from the DNF.

Date on which the death was registered.

ICD-10 codes for underlying causes of death as recorded by the PHIC in the death registry.

Medical extraction forms (EDC)

Medical data on the death extracted from the deceased patient’s hospital records. A team of specially trained physicians from each hospital in the study extracted data from the hospital record on the train of events leading to death. The extraction file contained data on the reason for hospitalisation, previous medical history and the course of events during hospitalisation.

Scanned image of the original hospital case summary.

Case summary based on hospital records, completed by specially trained physicians in each hospital.

Hospital case summaries

Figures 3 and 4 summarise data availability and attrition in the West Bank and Gaza Strip. In the West Bank, hospital records could be found for 371 cases of the 400 randomly selected deaths. In the Gaza Strip, we found hospital records for 199 of the 200 randomly selected deaths. We included all deaths reported in 2012 irrespective of the date of death. The PHIC registry gave an ICD code for the underlying cause of death in 358 of the 371 cases in the West Bank sample for which hospital records were available. In the Gaza Strip, 189 of the 199 cases in the sample had an ICD code for the underlying cause of death and 70 cases also included multiple causes of death registered by the PHIC. No multiple causes were present in the West Bank sample. Scanned images of the DNFs were available for 320 deaths in the West Bank sample. EDC data on file were available for 365 West Bank deaths and for all Gaza Strip deaths. Table 1 summarises the characteristics of the data.

Table 1

Data characteristics of the records available for the study

Characteristic	N (%)
	West Bank	Gaza Strip
Sample size	371	199
Government hospitals	265 (71)	199 (100)
Hospitals using electronic patient records	85 (23)	0 (0)
Age (years)
<1	75 (20)	54 (27)
1–17	45 (12)	19 (10)
18–64	149 (40)	72 (36)
65+	102 (27)	54 (27)
Gender
Male	200 (54)	99 (50)
Female	171 (46)	100 (50%)

Data availability and attrition in the West Bank. DNFs, death notification forms; PHIC, Palestinian Health Information Centre.

Data availability and attrition in the Gaza Strip. DNFs, death notification forms; PHIC, Palestinian Health Information Centre.

Statistical analysis

Completeness and timeliness

We did all analysis by region, due to the legislative and physical division of the occupied Palestinian territory, in terms of the separation of the Gaza Strip from the West Bank; resulting in two separate entities with different health systems and registries. We examined the proportion of complete DNFs in terms of personal data, immediate and underlying causes of death, and the attending physician and notifying person. We also examined the timeliness of registration, measured from the time of death to the time the deceased was registered in the CoDR.

Underlying cause of death based on the patient’s hospital file (EDC) and on the DNF

Using the EDC, we determined a ‘true’ underlying cause of death based on the amended EDC dataset (please see the Selecting the underlying cause of death section) and coded it using ICD-10. All coding, both of the original DNFs and of the EDC data, was done using Iris coding software.4 The version used (V4) corresponds to the 2013 edition of ICD-10. Iris fetches an ICD code for each diagnostic expression from a dictionary. Next, Iris prepares an input string of ICD codes for the Automatic Classification of Medical Entry (ACME) module (developed by the US National Center for Health Statistics). Finally, ACME selects an underlying cause according to the ICD rules and guidelines. After review and coding, we compared the three underlying causes—the one coded by PHIC; the one stated on the DNF form, but coded and classified by Iris and the one according to the EDC and coded by researcher using Iris—pair by pair:

Underlying cause on the DNF, coded by Iris versus underlying cause in the PHIC registry.

This comparison measures PHIC compliance with international rules for the selection of the underlying cause of death.

Underlying cause on the DNF, coded by Iris versus underlying cause according to the EDC.

This measures the accuracy of the original DNF.

Underlying cause according to the EDC versus underlying cause in the PHIC registry.

This comparison measures the overall accuracy of the underlying cause registered in the PHIC registry.

We compared the underlying causes at two levels of detail: ICD detailed level (henceforth referred to as ICD 4-character level) and ICD block level. The principal use of measurements at detailed level data is to estimate the precision of the coding and classification, while measurements at ICD block level provide a broad assessment of the general trustworthiness for public health purposes.

For cross tabulation, we examined the accuracy of mortality statistics by gender and age group of the deceased patient, hospital affiliation (governmental and non-governmental), type of hospital records and underlying cause of death using χ2 test. Tests of significance were two-sided with p value≤0.05. We divided the underlying causes according to the EDC into eight main diagnostic groups based on the frequency of the most common underlying causes according to the EDCs.

Neoplasms (ICD-10 chapter II).

Metabolic diseases (ICD-10 chapter IV).

Cardiac diseases (ICD-10 I00–I51).

Cerebrovascular diseases (ICD-10 I60–I69).

Perinatal conditions (ICD-10 chapter XVI).

Congenital anomalies (ICD-10 chapter XVII).

External causes (ICD-10 chapter XX).

Other causes of death (ICD-10 chapters and codes not included elsewhere).

(‘Other’ includes infectious diseases, diseases of the blood, neurological diseases, diseases affecting vision and hearing, respiratory diseases, gastrointestinal diseases, skin diseases, musculoskeletal diseases, urogenital diseases and symptoms with no clear connection to a single underlying disease.)

Selecting the underlying cause of death

The classification expert (LAJ) checked the diagnoses registered in the hospital patients’ files against the DNFs and corrected the registered text if it differed from the DNF. Similarly, LAJ checked the causes of death recorded in the EDC dataset against the case summaries and against the scanned images of the hospital summaries. If an EDC entry was inconsistent with the case summary or the hospital summary, the text in the EDC dataset was corrected according to the summaries. The underlying causes of death derived from this amended EDC dataset were considered as the true underlying causes of death.

Also, for each DNF, the following markers were used to record non-compliance with WHO instructions for completion of the death certificate: abbreviations used in the medical section, illegible writing in the medical section, sequencing errors in Part 1, symptomatic or secondary condition reported as underlying cause of death, cause of death insufficiently specified and other reporting errors (such as reporting several causes on the lowest completed line in Part 1, placing the underlying cause in Part 2 of the certificate or using the wrong field on the DNF to report causes of death).

Patient and public involvement

Not applicable for this study as we used mortality statistics and data were de-identified prior to the analysis.

Results

Completeness of DNF forms and other characteristics

Table 2 gives a summary of other characteristics of DNFs unrelated to the accuracy of the underlying cause of death. Although most deaths had been reported within a year (median of 89 days; range excluding extreme values 0–365 days), occasionally deaths from the 1980s and 1990s were only reported to the authorities in 2012. A fairly high number of DNFs (23% West Bank and 26% Gaza Strip) reported an underlying cause of death that presumably developed as a complication of some other conditions, yet the underlying condition is not recorded. For example, in several DNFs, kidney failure was reported as the underlying cause of death, but there was no mention of the reason why kidney failure developed, such as diabetes, glomerulonephritis or urinary obstruction. Administrative data (Part 1 of the DNF) were complete in only 7% and 2% of DNFs from hospitals with electronic patient records and paper patient records, respectively. Other problems with electronic records included several deaths that had been registered twice or more, and some of the deceased recorded in the hospital files were still alive.

Table 2

Completeness and other DNF characteristics

	West Bank (n=320)		Gaza Strip (n=199)
	Cases	%	Cases	%
No medical data on DNF	23	7.2	1	0.5
Electronic records (71)	15	21.1	–	–
Paper records (249)	8	3.2	1	0.5
Administrative data complete (Part 1)	9	2.8	108	54.3
Electronic records (71)	5	7.0	–	–
Paper records (249)	4	1.6	–	–
Abbreviations used	132	41.2	76	38.2
Illegible writing	96	30.0	11	5.5
Sequence errors	64	20.0	38	19.1
Non-informative UC	74	23.1	51	25.6
UC lacking specificity	36	11.2	12	6.0
Irrelevant information	105	32.8	22	11.0
Other certification errors	92	28.8	40	20.1
Competing causes of death	36	11.2	23	11.6
Certifier’s signature complete	159	59.7	50	25.1
Registry delay (days; median)	89		57

DNF, death notification form; UC, underlying cause of death.

Agreement between the three underlying causes

DNF–EDC agreement

Agreement on the underlying cause was highest between the underlying cause derived from the original DNF and the medical extraction forms (DNF–EDC). At the most detailed level, the agreement was 56% for the West Bank sample and 52% for the Gaza Strip sample. The difference between the two sample groups was not statistically significant (p value: 0.37) (table 3).

Table 3

Agreement on the underlying cause by ICD level: ICD 4-character level and ICD block level, point estimates (PE), and 95% CI

		ICD 4-character level		ICD block level
		PE (%)	95% CI	PE (%)	95% CI
DNF–EDC	West Bank	56	51 to 62	68	63 to 73
	Gaza Strip	52	44 to 59	62	55 to 68
DNF–PHIC	West Bank	23	19 to 28	46	41 to 51
	Gaza Strip	39	31 to 46	53	46 to 60
EDC–PHIC	West Bank	19	15 to 23	44	38 to 49
	Gaza Strip	31	24 to 38	44	37 to 51

DNF, death notification form; ICD, International Statistical Classification of Diseases and Related Health Problems; PE, point estimates; PHIC, Palestinian Health Information Centre.

DNF–PHIC agreement

DNF–PHIC agreement between the DNF coded in line with international standards and the underlying cause as coded by the PHIC was 23% in the West Bank sample at ICD 4-character level and 39% in the Gaza Strip sample. The difference between the samples was statistically significant (p value: 0.0001). At ICD block level, the West Bank sample had a DNF–PHIC agreement of 46% and the Gaza Strip sample had 53%, a difference that was not statistically significant (p value:0.12).

EDC–PHIC agreement

Agreement was lowest between the EDC and the underlying cause recorded in the PHIC registry. At ICD 4-character level, EDC–PHIC agreement was 19% in the West Bank sample and 31% in the Gaza Strip sample.

Agreement by sex, age, type of hospital and type of deceased hospital records

In both samples, there was no noticeable relationship between the sex and age of the decedent and the accuracy of the cause of death There were no statistically significant differences between males and females in underlying cause agreement (21% vs 16%, respectively, p value: 0.16). Also, there were few significant differences between the four age groups, either at ICD 4-character level or ICD block level. At a detailed level, there was significantly lower agreement for the youngest age groups in the two comparisons involving the codes in the PHIC registry (DNF–PHIC and EDC–PHIC) (7% and 8%) compared with 27% and 24% at age group 18–64 years (both p value: <0.0001). At block level, only the DNF–PHIC comparison showed a significant difference between children <1 year and older people >64 years (43% vs 60%, respectively, p value: 0.0002), probably reflecting difficulties in the coding of congenital anomalies and perinatal causes.

Also, the West Bank sample showed only minor differences between governmental and non-governmental hospitals, and no differences in accuracy between DNFs from hospitals using an electronic patient record system or those using paper-based documentation. The only statistically significant difference between governmental and non-governmental hospitals was in DNF–EDC agreement at ICD 4-character level (53% and 64%, respectively, p value: 0.013), but this difference was not significant at ICD block level (data not shown). With the exception of EDC–PHIC comparison at a detailed level (11% using electronic records and 21% using paper records, p value: 0.0003), the use of an electronic patient record system or traditional paper medical records by the hospital had no impact on agreement. The EDC–PHIC difference disappeared at block level.

Agreement by diagnostic group

There were considerable differences in agreement between diagnostic groups. At both the ICD 4-character level and block level, the accuracy of the DNFs (DNF–EDC agreement) was best for external causes and neoplasms. The weakest agreement was for metabolic conditions. The differences between the West Bank and Gaza Strip samples were not statistically significant (56% and 52%, respectively, p value: 0.37) (table 4).

Table 4

Agreement in the underlying cause by diagnostic group: ICD 4-character level, point estimates and 95% CIs

	DNF–EDC				DNF–PHIC
	West Bank		Gaza Strip		West Bank		Gaza Strip
	PE (%)	95% CI	PE (%)	95% CI	PE (%)	95% CI	PE (%)	95% CI
Neoplasm	76	65 to 87	84	73 to 95	39	26 to 51	64	49 to 79
Metabolic	32	12 to 52	29	8 to 50	16	1 to 31	19	1 to 37
Cardiac	49	38 to 61	37	13 to 61	42	30 to 54	29	5 to 54
Cerebrovascular	53	38 to 68	45	21 to 69	16	5 to 27	63	39 to 87
Perinatal	46	27 to 66	45	23 to 68	11	0 to 23	5	0 to 15
Anomaly	51	34 to 69	47	21 to 74	3	0 to 9	29	5 to 54
External	78	60 to 96	–		9	0 to 22	–
Other	56	43 to 69	45	32 to 59	16	6 to 26	36	22 to 50

DNF, death notification form; ICD, International Statistical Classification of Diseases and Related Health Problems; PE, point estimates; PHIC, Palestinian Health Information Centre.

At both ICD detailed level and ICD block level, agreement in the West Bank sample between the original DNFs and PHIC registry data was best for cardiac diseases, but still weak. It was generally better in the Gaza Strip sample, with the highest value for neoplasms (39% and 64%, respectively, p value: <0.0001) and cerebrovascular conditions (16% and 63%, respectively, p value: <0.0001), both significantly better than in the West Bank sample. The lowest DNF–PHIC agreement in the West Bank sample was for anomalies and external causes significantly lower than in the Gaza Strip sample (3% and 29%, respectively, p value: <0.0001). In the Gaza Strip sample, the lowest DNF–PHIC agreement was for perinatal causes.

The accuracy of the underlying causes in the PHIC register, measured by EDC–PHIC agreement, was highest, but still moderate, for neoplasms and cardiac conditions in the West Bank sample. In the Gaza Strip sample, the highest agreement was for neoplasms (62%, significantly higher than in the West Bank sample 36%, p value: <0.0001). The lowest agreement was for metabolic conditions in the West Bank sample and for perinatal causes in the Gaza Strip sample.

Discussion

This is the first baseline study of the accuracy of mortality statistics in Palestine. On the basis of the study findings, the accuracy is low, mostly due to inaccuracies in the PHIC coding of the underlying cause of death. But deficient certification of causes of death also contributed significantly to the inaccuracy. Both the West Bank and Gaza Strip, 30%–40% of DNFs had sequence-related errors combined with incorrect order or entries and competing causes of death. From an international perspective, this is a high figure.5 Agreement between the underlying cause according to the EDC and the underlying cause of death actually registered by the PHIC was low: 19% (West Bank) and 31% (Gaza Strip) at the most detailed level and 44% (both samples) even at ICD block level. In some comparisons, the Gaza Strip performed better than the West Bank, in others, it was the other way round. Further research is needed to understand the differences in accuracy between the Gaza Strip and West Bank.

On the basis of the study findings, the accuracy of mortality statistics is lower than in other published similar studies. For example, a validation of mortality statistics in Cape Town showed an accuracy of 55% at WHO tabulation list 1 level (103 groups, roughly corresponding to block level).6 Other studies vary widely in their assessment of the accuracy of mortality statistics,7 but since they generally refer to mortality statistics in high-resource countries and to specific causes of death, the results are difficult to compare to the present study.

As in several other studies on the quality of mortality statistics, accuracy varies between diagnostic groups.7–9 Accuracy in the causes of death in the PHIC register, indicated by EDC–PHIC agreement, was best, but still moderate, at 69% for cerebrovascular diseases in the West Bank sample. This is surprising since most studies show the highest agreement for neoplasms, which is also the case in the Gaza Strip sample (83%).9 In the West Bank sample, neoplasms show 54% of agreement, which is low in comparison with other studies. Both samples show low agreement for metabolic conditions, which is consistent with the findings from other studies.

Although the present study adds to the accuracy of mortality statistics registries’ literature, it has several limitations. First, the study was limited to hospital deaths. A future study is needed to examine the accuracy of DNFs for deaths outside the hospital context. Second, pairwise per cent agreement used in data analysis does not take into account that agreement may occur by chance. Therefore, the per cent agreement may be over-reported. Third, we do not know the number of deaths that are not reported to the authorities at all. The presence of deaths that occurred several decades ago in the sample indicates that some deaths are unreported. It is important to estimate the number of non-reported deaths and determine if the causes of death in such cases differ significantly from those actually reported to the authorities.

Conclusions

On the basis of the study, the PHIC CoDR gives a poor picture of the causes of death in both the West Bank and Gaza Strip. This hampers the assessment of important aspects of public health, such as the burden of disease, maternal mortality and the prevalence of congenital anomalies. The conclusion is that PHIC mortality statistics are not comparable to statistics from other countries that adhere to the ICD instructions more closely, and also that the medical precision of the statistics (measured by EDC–PHIC agreement) would be improved if the international coding and classification rules were adhered to. To address these points, the following measures are proposed:

Use of international coding software for coding and classification.10

The use of internationally recognised coding software would bring several benefits. The software automatically applies ICD instructions for the selection and classification of causes of death. It also covers less common cases that coders may be unfamiliar with and brings coding and classification in line with the ICD instructions. A dictionary of medical terms is included in the software, contributing further to consistent coding. Further, coding software speeds up the coding process since it reduces the burden on the limited staff at PHIC who is currently responsible for all the coding. At present, there can be a delay of several months from the time the DNF is received by the PHIC until it is coded and entered in the CoDR. A further advantage of automated coding is that with automated coding, the PHIC registry would contain multiple causes of death for all deaths, and the potential for detailed analysis and monitoring would be considerably enhanced.

Training and supervision of cause of death certification at hospitals.

Studies of the quality of mortality statistics often suggest that training for certifiers on how to complete the death certificate would resolve issues related to quality. An automated coding system would facilitate the training of new coders. Another proposed approach is that each death certificate should be written and signed by two physicians,11 12 or that a senior physician should review all death certificates issued at the hospital. In some locations, a specialised committee has been set up to review all hospitalisations that end with the death of the patient, and also to verify the accuracy of the death certificate.12 13

Feedback from PHIC to the individual hospital on problems listed in table 2.14 15 It would be even more efficient if the hospital or certifier could be contacted in such cases and asked to amend the DNF.

In addition, steps should also be taken to improve the overall completeness of the CoDR. First, review and improve data handling at hospitals with electronic patient records. Electronic patient record systems are supposed to improve data availability and facilitate the administration of hospital care, but appeared to have the opposite result in our study. Second, develop a digital DNF in the electronic system. Third, enforce completion of the death report in the electronic system by adding specific restrictions in the electronic system.

According to a study conducted by the PCBS, it now includes only about 60% of all deaths.3 Consequently, the statistics would still provide a poor picture of the causes of death even if the accuracy of reported cases was considerably improved.