Differences in reporting of maternal and child health indicators: A comparison between routine and survey data in Guizhou Province, China.

BACKGROUND: The quality of routine data, such as the maternal mortality ratio (MMR), infant mortality rate (IMR), and under-five mortality rate (U5MR) is often questioned. The objective of this study was to compare routine and survey data on key maternal and child health indicators, including the MMR, IMR, and U5MR in the Guizhou Province of China.
METHODS: In 2008, an urban area and a rural area in the Guizhou Province were randomly selected. All households in the selected areas were included and, of the total 5466 households therein, 5459 were visited. The response rate was 99.9%. Survey data were collected from mothers (46.0%), fathers (32.5%), grandmothers (11.1%), grandfathers (9.0%), and other caregivers (1.4%). Data from routine records of the health bureaus in selected areas were reviewed for the same indicators. The Chi-square test was used to study the differences between routine data and survey data.
RESULTS: We found the differences between the routine and survey data live births in the survey data (68) was fewer than in the routine data (94) in the rural area, while live births in the survey data (106) was larger than in the routine data (96) in the urban area. The IMR was higher in the survey data (51.7 per thousand) as compared with routine data (31.6 per thousand). The U5MR was higher (69.0 per thousand) in the survey data than in the routine data (42.1 per thousand). Indicators related to the coverage of maternal and child health interventions were over-reported in routine data.
CONCLUSION: Small differences were observed between routine data and survey data in Guizhou, one of the poorest areas of China. The quality of routine data in urban areas was better than in rural areas.

Introduction

The maternal mortality ratio (MMR), infant mortality rate (IMR), and under-five mortality rate (U5MR) are common indicators used to monitor the health status of a country or region and progress towards the achievement of millennium development goals 4 and 5.1–3 Despite being widely used for this purpose, the quality of MMR, IMR, and U5MR surveillance data is often not well known.4,5

Since 1980, China’s family planning policy restricts the number of children per family. This policy allows Han Chinese (Hanzu) couples in urban areas to have only one child, and rural Hanzu couples to have two children if the first child is a girl. Ethnic minorities are permitted two children, and ethnic minorities in pastoral areas are permitted three children. Violation of family planning policies results in heavy fines and may cost the parents their jobs.6 To avoid being penalized, noncompliant families tend to have home births. As a result, many home births and deaths go unreported, leading to differences between routine and survey data.

The most common methods of checking the quality of routine data in China include interviewing household members, and checking hospital, public security, and family planning department records.7

In China, current maternal and child health (MCH) indicators come from routine reporting data sources. Figure 1 shows the structure of the Maternal and Child Health Reporting System in China. The Ministry of Health (MOH) estimates the mortality at the national level and the coverage of health interventions based on these routine data. Reporting forms for MCH indicators are developed by the MOH and National Bureau of Statistics of China, and are considered to be statutory reports. Every county and district in all the 31 provinces (autonomous regions and municipalities) in Mainland China are therefore obliged to fill out these forms.8

Figure 1

The structure of maternal and child health reporting system in China.

China established a national maternal mortality surveillance system in 1989. Initially it covered 247 maternal mortality surveillance sites and a population of 100 million individuals, which was approximately 8% of the total population in China. A national child mortality surveillance system was established in 1991, initially covering 81 surveillance sites and a population of 8.5 million. Birth defect surveillance was established in 1986, and was a hospital-based surveillance system. In 1996, the MOH incorporated maternal mortality surveillance, child mortality surveillance, and birth defect surveillance into a national Maternal and Child Mortality Surveillance System (MCMS). The number of surveillance sites has been expanded from 116 in 1996 to 336 as of today, comprising 126 sites in urban areas and 210 sites in rural areas.9 The MCMS reports the number of live births, the number and causes of maternal and child deaths, and provides scientific evidence for health officials and MCH workers through data analysis and utilization.

The MOH has carried out national health service surveys every 5 years since 1993 (excluding Hong Kong, Macau, and Taiwan) to provide data for health planning. The MCH is one part of this survey, and includes main MCH indicators, such as the MMR, IMR, hospital delivery rate, and antenatal care coverage. In addition, a few international cooperation projects conduct baseline and endline surveys to evaluate the effect of MCH interventions.10 MCMS is assumed to be the most reliable data source for estimation of maternal and child mortality in China. In 2008, the MMR in China was 34.2 per 100,000 live births, the IMR was 14.9 per 1000 live births, and the U5MR was 18.5 per 1000 live births.9

Guizhou is a remote and poor province with 37.8% of ethnic minority groups in southwest China.11 We conducted a study in Guizhou with the aim of comparing maternal and child mortality as well as other selected maternal and child health indicators between routine data and survey data. This is the first internationally published study in China that compares key maternal and child health indicators from routine data and those from survey data.

Materials and methods

Study area

Guizhou is a mountainous province located in southwest China. It is among the poorest provinces in China, and its per capita gross regional product ranks the lowest in the country.12 Maternal and child mortality rates in Guizhou are ranked the highest in China, with poor maternal and child health status and health service provision.13,14

The district of Honghuagang was randomly selected as the sample urban area from a possible eight urban surveillance sites. Similarly, the county of Pingtang was chosen as the sample rural area from 18 possible rural surveillance sites. One street from Honghuagang and one township from Pingtang were once again randomly identified and every household within these areas was surveyed.

Study population

The study was carried out during April–November 2008 in Guizhou. All households within a selected township/street were included as participants in a household survey. A total of 5459 households from 5466 were visited, with 3563 being urban and 1896 being rural. The response rate was 99.9%. Figure 2 shows a flow chart of the sampling framework and study population.

Figure 2

Flow chart of sampling framework and study population in Guizhou province.

Data collection

This study was approved by the National Center for Women and Children’s Health and Guizhou Provincial Health Bureau. Hard copies of original local records of routine MCH data were obtained from the local health bureaus in Pingtang and Honghuagang. Three questionnaires covering the same MCH indicators as the routine data source were used to collect survey data by household interview from October 1, 2007 to September 30, 2008 (time period was consistent with that of the routine data source) in the study areas. Questions about socioeconomic characteristics were also included. All questionnaires were in the Chinese Mandarin language, but for about 10% of elderly minority respondents who could not understand Chinese Mandarin, the questions were translated into minority languages by local translators. Differences in translation were discussed and corrected by the investigators.

We trained 24 local MCH workers as investigators in the rural and urban areas on the contents of the questionnaires, interpretation of each question, classification of maternal and child deaths according to the ICD-10,15 as well as skills for conducting the interview. The questionnaires were pretested by local investigators, problems were identified, and questionnaires were modified accordingly.

The investigators visited a household three times, and if they could not find a respondent in the household by the third visit, the household was excluded. The respondents in our sample were mothers (46.0%), fathers (32.5%), grandmothers (11.1%), grandfathers (9.0%), and other caregivers (1.4%). Written informed consent was obtained from each respondent before the interview. The data were collected during August–November 2008. The trained investigators were responsible for entering the answers. They also checked each other’s forms for errors and completeness daily, and questionnaires with omissions were investigated further and corrected within one week. We personally checked all questionnaires each month after the data collection was completed. Doubts and further questions, if any, were marked and sent back for correction. The survey results were then compared with the routine data obtained from the local County and District Health Bureau.

Selection and calculation of indicators

We selected main MCH indicators from the routine data source for comparison of the routine data and survey data, including demographics, maternal health, newborn health, and child health and nutrition. Sociodemographic and socioeconomic indicators from our survey data were selected for comparison between the rural and urban areas. We calculated the selected MCH indicators using the same method as that used by the routine surveillance system. Table 1 shows the definitions of the main selected indicators.

Table 1

Definitions of main selected indicators

Indicator	Definition
Live birth	As defined by the WHO, a live birth is any baby who upon delivery from its mother breathes or shows any other evidence of life, such as a heartbeat, pulsation of the umbilical cord, or definite movement of voluntary muscles, irrespective of duration of the pregnancy. In China, a live birth is defined as any newborn with a gestational age of ≥ 28 weeks or birth weight > 1000 g, who upon delivery from its mother has any of the four vital signs (heartbeat, breath, pulsation of the umbilical cord, and voluntary muscle contraction).8,16
Maternal death	Death of a woman while pregnant or within 42 days of termination of pregnancy, irrespective of the duration and site of the pregnancy, from any cause related to or aggravated by pregnancy or its management, but not from accidental or incidental causes.8
MMR	Number of maternal deaths during a given time period per 100,000 live births during the same period.17
IMR	Number of infant deaths during a given time period per 1000 live births during the same period.3
U5MR	Number of child deaths under five years during a given time period per 1000 live births during the same period.3
Hospital delivery rate	Number of births attended by skilled health personnel in medical institutions at the township level and above in a given time period per 100 live births in the same period.8
Antenatal care rate	Number of women with a live birth who received antenatal care provided by skilled health personnel at least once during pregnancy in a given time period per 100 live births in the same period.8
Postnatal visit rate	Number of women with a live birth who received postnatal visit provided by skilled health personnel at least once during the first 28 completed days after delivery in a given time period per 100 live births in the same period.8

Abbreviations: WHO, World Health Organization; IMR, infant mortality rate; MMR, maternal mortality ratio; U5MR, under-five mortality rate.

Data analysis

Statistical analyses were performed using SPSS 15.0 Statistical Software for Windows (SPSS Inc, Chicago, IL). Frequency analysis was used for descriptive statistics. The Chi-square test was used for comparative analysis.

Results

Table 2 shows the basic characteristics of Honghuagang district (urban area) and Pingtang county (rural area). The urban area had a larger population but fewer deaths compared with the rural area (Table 2). The rural area was poorer than the urban area (Table 3). Children under five years (83.8%) and pregnant women (72.7%) in the rural area were mainly from the ethnic minorities, while those in the urban area were mainly Han Chinese (Table 3). A larger proportion of pregnant women (10.4%) in the rural area were unmarried (Table 3). Pregnant women in the urban area were more educated than those in the rural area (Table 3). Nearly 40% of children under five years in the urban area belonged to the migrant population (Table 3).

Table 2

Basic characteristics of study areas in Guizhou for 2008 from routine dataset

	Area (km2)	Total population	Children under five years	Live births	Maternal deaths		Infant deaths		Child deaths under five years
					n	Rate (MMR)	n	Rate (IMR)	n	Rate (U5MR)
Pingtang county	2815	313,287	16,204	2973	5	168.2	110	37.0	138	46.4
Ganzhai township (rural area)	–	9252	438	94	0	0	6	63.8	7	74.5
Honghuagang district	624	476,514	23,290	4846	2	41.3	55	11.4	71	14.7
Beijing street (urban area)	–	13,745	884	96	0	0	0	0	1	10.4

Abbreviations: IMR, infant mortality rate; MMR, maternal mortality ratio; U5MR, under-five mortality rate.

Table 3

Basic characteristics of study areas in Guizhou for 2008 from survey dataset

	Rural area		Urban area
Total population	9346		13,762
Children under five years	488		900
	n	%	n	%
Gender
Male	282	57.8	485	53.9
Female	206	42.2	415	46.1
Ethnicity*
Han	79	16.2	862	95.8
Minority	409	83.8	38	4.2
Migrant status**
No	484	99.2	561	62.3
Yes	4	0.8	339	37.7
Birth registration
Yes	470	96.3	869	96.6
No	18	3.7	31	3.4
Annual family income per capita* (CNY)
<2000	407	83.4	466	51.8
2000–3999	73	15.0	137	15.2
4000–8000	8	1.6	174	19.3
>8000	0	0.0	123	13.7
Pregnant women	77		136
Ethnicity*
Han	21	27.3	128	94.1
Minority	56	72.7	8	5.9
Age groups, years
20–24	35	45.5	47	34.6
25–29	17	22.0	53	38.9
30–34	16	20.8	21	15.5
35–39	4	5.2	6	4.4
≥40	0	0.0	5	3.7
Unknown	5	6.5	4	2.9
Marital status*
Married	69	89.6	134	98.5
Unmarried	8	10.4	2	1.5
Educational background*
Primary school and below	37	48.1	8	5.9
Junior high school	39	50.6	72	52.9
Senior high school	1	1.3	26	19.1
Junior college and above	0	0.0	30	22.1
Annual family income per capita* (CNY)
<2000	57	74.0	56	41.2
2000–3999	16	20.8	38	27.9
4000–8000	4	5.2	18	13.2
>8000	0	0.0	24	17.6
	n	Rate	n	Rate
Live births	68	–	106	–
Maternal deaths	1	1470.6	0	0
Infant deaths	8	117.7	1	9.4
Child deaths under five years	9	132.4	3	28.3

Notes:

Significant difference between rural and urban area using Chi-square testing, P < 0.01;

migrant status means a person coming from another province to live in the current province for less than a year, according to the definition of the routine reporting system.

Abbreviation: CNY, Chinese Yuan.

The number of live births (94) was larger than the number of infants (88) in the routine data for the rural area (Table 4). There was a difference in number of live births in both the rural and urban areas, ie, live births in the survey data (68) was fewer than in the routine data (94) in the rural area, but live births in the survey data (106) was larger than in the routine data (96) in the urban area (Table 4). Because the number of live births is the denominator for the mortality and hospital delivery rate, over-reporting of live births is the reason for the lower hospital delivery rate in the rural area. Most statistics in the coverage indicators of maternal and child health interventions were over-reported in the routine data. In the urban area, all indicators showed discrepancies between the routine data and survey data, except for the number of maternal deaths (0) and management rate of high-risk pregnant women (100%). We found one missing maternal death in the rural area and four missing child deaths in two study areas, three of which were neonatal deaths. Table 4 shows the differences between the routine data and survey data for the selected indicators.

Table 4

Comparison of routine and survey data on selected indicators in selected areas for Guizhou in 2008

Indicators	Rural area			Urban area
	Routine data	Survey data	Difference between survey and routine data	Routine data	Survey data	Difference between survey and routine data
Demographics
Live births (n)	94	68	−26	96	106	10
Infants (n)	88	127	39	209	234	25
Children under 3 years (n)	264	300	36	505	530	25
Children under 5 years (n)	438	488	50	884	900	16
Maternal and child deaths
Maternal deaths (n)	0	1	1	0	0	0
Neonatal deaths (n)	3	5	2	0	1	1
Infant deaths (n)	6	8	2	0	1	1
Child deaths under 5 years (n)	7	9	2	1	3	2
Maternal and child health
Maternal and newborn health
Antenatal care (at least one visit) rate (%)	80.65	84.42	3.77	100.00	96.12	−3.88
Hospital delivery rate (%)	65.59	72.73	7.14	100.00	97.09	−2.91
Clean delivery rate (%)	100.00	91.04	−8.96	100.00	99.03	−0.97
Highrisk pregnancy management rate (%)	58.33	50.00	−8.33	100.00	100.00	0.00
Postnatal visit rate (%)	81.91	75.00	−6.91	90.80	53.27	−37.53
Child health and nutrition
Systematic management rate for children under 3 years (%)	37.55	9.60	−27.95	92.67	83.99	−8.68
Breastfeeding rate (<6 months, %)	100.00	93.75	−6.25	90.10	77.36	−12.74
Exclusive breastfeeding rate (<6 months, %)	97.53	84.38	−13.15	68.81	69.81	1.00

We compared routine data between Pingtang county and Honghuagang district, and the quality of data in Honghuagang district was better than in Pingtang county in general, except for the postnatal visit rate, which was poorer in Honghuagang district than in Pingtang county (Table 5).

Table 5

Over-reporting and under-reporting of routine data compared with survey data between rural area and urban area of Guizhou in 2008

	Rural area (over/under reporting compared with survey data) %	Urban area (over/under reporting compared with survey data) %	Chi-square P value
Live births (n)	38.24	−9.43	0.001a
Infants (n)	−30.71	−10.68	0.000a
Children under 3 years (n)	−12.00	−4.72	0.000a
Children under 5 years (n)	−10.25	−1.78	0.000a
Maternal deaths (n)	−100.00	0	–
Neonatal deaths (n)	−40.00	−100.00	1.000
Infant deaths (n)	−25.00	−100.00	0.333
Child deaths under 5 years (n)	−22.22	−66.67	0.236
Antenatal care (at least one visit) rate (%)	−4.47	4.04	0.133
Hospital delivery rate (%)	−9.82	3.00	0.282
Clean delivery rate (%)	9.84	0.98	0.019a
High-risk pregnancy management rate (%)	16.66	0	0.000a
Postnatal visit rate (%)	9.21	70.45	0.000a
Systematic management rate for children under 3 years (%)	291.15	10.33	0.000a
Breastfeeding rate (<6 months) (%)	6.67	16.47	0.095
Exclusive breastfeeding rate (<6 months) (%)	15.58	−1.43	0.002a

Notes:

P value < 0.05 means significant difference between rural and urban areas. Village health workers are responsible for maintaining monthly records of births and deaths in the village, the number of pregnant women and the number of children under five years. Charts with the data are displayed on the walls of the village clinic. This chart is from a village clinic in the Guizhou province.

Discussion

We found the number of births and deaths in our survey were larger than those in the national reporting data in both urban and rural areas, the largest gap being for the number of live births and neonatal deaths. Several published papers have highlighted the problem of under-reporting of live births and neonatal deaths internationally, and this was verified by our results.17–20

Economic reforms in China have led to a gradual decentralization from central government level to the provinces. However, this is related to economic development and administrative management only. China is still centralized when it comes to major policies and systems. Therefore, health system policies and the routine reporting system for the entire country remain the same.

The law of the land requires families to go to the registration office to register a new birth. Under the very strict Chinese Hukou system, every birth has to be registered in the local residence area. For this, a permit from the family planning department and a birth certificate from a medical institution are mandatory. Hukou is also necessary for children’s education and social benefits.

With increasing migration from the rural areas to the cities, returning back to rural home towns for the sake of registering births and deaths is on the decline, given time and cost constraints. Parents prefer to wait until their children reach school age.5,21 Strict family planning policies, fines, and other penalties for exceeding the permitted number of children leads to under-reporting. Under-reporting from the villages is due to lack of understanding of the importance of the need to register and poor access to registration facilities.22 The successful procedures used for mobile registration systems from other countries might be used to improve civil registration in remote areas.23 The significance and necessity of a births and deaths register for newborns is not well recognized in rural China.24,25

The primary reporting of maternal and child deaths is done by village or community health workers. They are required to report maternal and child deaths to the township hospital as soon as possible. The increasing drift of the labor force from rural to urban areas in China poses a difficulty for village and community health workers. Home births are preferred in the rural areas of western China, and especially by ethnic minority groups. Social norms prevent families from reporting maternal and neonatal deaths that occur at home.26 This necessitates extra vigilance on the part of village health workers for the sake of accuracy. Nearly half of births are not registered within the legally prescribed time period in both the international and Chinese contexts.22,27

Every birth and death counts towards MCH policy-making, health planning, and reasonable health resource allocation at all levels, and helps to improve maternal and child survival. Furthermore, monitoring and evaluation of indicators for millennium development goals 4 and 5 will remain objective only if estimations are made based on good reporting systems.28–30

We found that most coverage rates for MCH indicators in our survey were lower than in the routine data for both urban and rural areas of Guizhou, with the largest gap being for postnatal visits and systematic management of children under three years of age. Similar findings were reported by three other studies.31–33 It is difficult to implement systematic management for children under three years under the present Chinese health system, especially in rural areas, due to inadequacy of human resources, poverty, and unawareness on the part of caregivers.34–38 The reason for the low postnatal visit rate in the urban area may have been the migration-linked population explosion, leading to overworked community health workers.31

MCMS is designed to monitor maternal and child mortality and causes of death, provide scientific evidence for MCH interventions, and to make policy. Quality control is fundamental to ensure the accuracy and reliability of data reporting systems. China establishes its own quality control mechanisms and methods, and MCMS data are used to adjust the MMR and U5MR in China. The completeness of data reporting is required to be 100%, with an error rate of less than 1% when supplying information, an error rate less 10% for reporting live births, and birth and death omission rates less than 15%.7 If the under-reporting rate is less than 20%, the adjusted mortality rate would be:

and if the under-reporting rate is at least 20%, the adjusted mortality rate would be:

Quality control is done regularly at every level. The counties carry out quarterly quality control checks, and annually at provincial and national levels. Original reporting forms and related records are reviewed at county, township, and village levels, and death records for women of reproductive age and children under five years are maintained by the health department, public security department, and family planning committees. Causes of death, and missing births and deaths information are identified. The Ministry of Health then makes the final adjustments to national maternal and child mortality rates.

Study strengths and limitations

To our knowledge, this study is the first internationally published investigation of the quality of routine MCH data in China. The support of the National Center for Women and Children’s Health and Guizhou Provincial Health Bureau was very valuable. This governmental initiative facilitated the collection of routine data and the household interviews. Random sampling and a high response rate in the study helped to increase the reliability of the results.

This was a retrospective study, so we cannot rule out recall bias, because some of the respondents were asked to recall information that was more than one year old. While maternal or child death is an unforgettable event for the family, information bias in the utilization of maternal and child health services cannot be ruled out. This was corroborated with other records and documents available in the households. Original questionnaires were used for the first time, without translation into dialects that lack written formats, like Buyizu and Miaozu. Spoken dialect translation might cause small information loss or differences. The study was conducted to compare key maternal and child health indicators between routine data and survey data, and the study was located in Guizhou Province, so it is impossible to address data quality for the whole country.

Conclusion

Our survey results do not show a large gap between routine and survey data in Guizhou. The situation in Guizhou might reflect the situation in most provinces of China. Based on this study, we can infer that the routine MCH reporting system is in place and functional in China. However, more studies are needed to confirm this. It is recommended that health personnel at all levels take note of the under-reporting of births and deaths and over-reporting of health interventions. Accurate and reliable data collection is essential for Chinese policies aimed at women, children, and public health. While the current data show that China has achieved millennium development goal 4 ahead of time, the quality of the routine reporting system for maternal and child health in China needs to be further improved to confirm this.