Evaluation of maternal and neonatal hospital care: quality index of completeness.

OBJECTIVE: Develop an index to evaluate the maternal and neonatal hospital care of the Brazilian Unified Health System.
METHODS: This descriptive cross-sectional study of national scope was based on the structure-process-outcome framework proposed by Donabedian and on comprehensive health care. Data from the Hospital Information System and the National Registry of Health Establishments were used. The maternal and neonatal network of Brazilian Unified Health System consisted of 3,400 hospitals that performed at least 12 deliveries in 2009 or whose number of deliveries represented 10.0% or more of the total admissions in 2009. Relevance and reliability were defined as criteria for the selection of variables. Simple and composite indicators and the index of completeness were constructed and evaluated, and the distribution of maternal and neonatal hospital care was assessed in different regions of the country.
RESULTS: A total of 40 variables were selected, from which 27 single indicators, five composite indicators, and the index of completeness of care were built. Composite indicators were constructed by grouping simple indicators and included the following variables: hospital size, level of complexity, delivery care practice, recommended hospital practice, and epidemiological practice. The index of completeness of care grouped the five variables and classified them in ascending order, thereby yielding five levels of completeness of maternal and neonatal hospital care: very low, low, intermediate, high, and very high. The hospital network was predominantly of small size and low complexity, with inadequate child delivery care and poor development of recommended and epidemiological practices. The index showed that more than 80.0% hospitals had a low index of completeness of care and that most qualified heath care services were concentrated in the more developed regions of the country.
CONCLUSIONS: The index of completeness proved to be of great value for monitoring the maternal and neonatal hospital care of Brazilian Unified Health System and indicated that the quality of health care was unsatisfactory. However, its application does not replace specific evaluations.

INTRODUCTION

Several studies have investigated the health assistance to women during pregnancy and childbirth and to children in the first year of life, particularly in relation to maternal, perinatal, and neonatal mortality. , ,

Donabedian , reported that the quality of health care can be assessed by measuring its efficacy, effectiveness, efficiency, optimization, acceptability, legitimacy, and equity (the seven pillars of quality) as well as the structure-process-outcome framework. The structure is defined as the resources to perform health care services. Processes are the set of activities developed in the relationship between medical staff and patients and the dynamics of this relationship. The results are the effects of health care activities.

The conditions that allow the establishment of a correlation of perinatal and neonatal deaths with maternal and neonatal care are as follows: number of neonatal deaths in the first hours of life, frequency of fetal deaths at the end of pregnancy, occurrence of neonatal deaths in hospitals without adequate neonatal care, and prevalence of preventable (or reducible) causes for most perinatal deaths, among others. , , ,

The elements and circumstances of the structure and process of obstetric care directly affect the prevalence of maternal and perinatal mortality. The structure of hospital services includes several factors: insufficient number of hospital beds, even those used for general risk procedures, and incubators; inadequacy in the relationship between the number of health professionals and number of neonatal beds; neonatal unit overcrowding; inadequacy in the transport of newborns; unavailability of obstetric and neonatal intensive care units; lack of high-risk infirmaries equipped with fetal care services to treat severe cases; complementary exams; and cardiological support. , , , The aspects related to obstetric care are as follows: inappropriate diagnosis and treatment; failures in obstetric management and/or deficiencies in nursery/newborn care; neglect of prenatal care; limited access of pregnant women and newborns to health services; difficulties in the regionalization and ranking of health care services; absent or incipient association between prenatal care and childbirth; distinction between primary care services and child delivery care; and disqualification of health professionals, among others. , , , , ,

Maternal and perinatal mortality can be significantly reduced through adequate access of mothers and newborns to comprehensive and quality health care. , , The quality of health care services, measured by comparing appropriate parameters, is determined by the level of adequacy or excellence achieved in the execution of health care activities.

Considering the growing need to assess the health status and compare the performance of health systems and services, these indicators are valuable tools capable of providing relevant information to support decision making.

Indices are indicators used to group and summarize information from a large number of indicators, which would be difficult to interpret in isolation. These indices are obtained by grouping two or more simple indicators and are assigned to the same dimension or to different dimensions of reality by defining relative weights or scores.

Several studies have evaluated the maternal and neonatal care , , of Brazilian Unified Health System (SUS), but few studies have defined parameters to evaluate this care. The aim of the present study was to develop an index to evaluate the quality of maternal and neonatal hospital care in SUS.

METHODS

This descriptive and cross-sectional study of national coverage analyzed data obtained from the Sistema de Informações Hospitalares (SIH/SUS – Hospital Information System) and the Cadastro Nacional de Estabelecimentos de Saúde (CNES – National Registry of Health Establishments).

The SUS network included 5,312 hospitals. The inclusion criteria were as follows: at least 12 deliveries performed in 2009 or the number of deliveries represented at least 10.0% of the total number of hospitalizations in 2009. Of these, 3,400 hospitals were selected, including 16 units that had no obstetric beds registered at CNES but showed a significant number of obstetric hospitalizations.

The database was organized by collecting the Authorization Records for Hospitalization of each federative state in Brazil for every month of 2009, in addition to CNES files related to the number of health establishments and hospital beds, accreditations, and equipment corresponding to the month of December 2009.

The theoretical foundations proposed by Donabedian (1980) were adopted for preparation of the evaluation method and suggested that analysis should be performed using the structure-process-outcome framework because of the interdependence among these three dimensions. The integrality in health care was incorporated to ensure a more comprehensive analysis and included various levels of health care (promotion, prevention, treatment, and rehabilitation in maternal and neonatal hospital care).

The subdimensions evaluated were hospital size, level of complexity, child delivery care, and recommended hospital and epidemiological practices, and were established as composite indicators, reflecting the ability of the hospitals to provide comprehensive and quality care to mothers and newborns through SUS.

Relevance and reliability were defined as criteria for the selection of variables for construction of indicators. A variable was considered relevant when it correlated with any of the dimensions of quality care and comprehensive health care. Reliability was defined as the accuracy of each variable in reflecting the phenomenon with which it was correlated and was considered desirable when it showed no distortions or contradictions with reality.

The list of indicators was evaluated by researchers from the Laboratory of Analysis of Health Information Systems of the Aggeu Magalhães Research Center (Fiocruz-PE), in the state of Pernambuco, Northeastern Brazil, and by professional obstetricians and pediatricians of the Integrative Medicine Institute of Pernambuco, which is reference in obstetrics, neonatal care, and medical teaching. Only the indicators evaluated by the professionals of both institutions were included.

The following variables were equally important in indicator construction: data availability, simplicity of interpretation, reproducibility, discriminatory power, clarity of purpose, and representation.

The index was constructed in the following three steps:

1) Definition of simple indicators: definition of score ranges (final score between 0 and 1).

2) Development of composite indicators by summing the scores of the simple indicators that form the composite indicators (dimensions), thereby establishing score ranges and a final score for each dimension (between 0 and 5).

3) Construction of the index of completeness of maternal and neonatal hospital care through the sum of scores obtained by the evaluation of composite indicators, thereby forming five score ranges that express five levels of completeness of care.

These indicators and the index were applied to different geographical regions to evaluate the quality of care.

These data were processed using the tabulation software TABWIN, analyzed in Microsoft Office Excel software, and presented in the form of charts and tables.

The present study followed the Declaration of Helsinki guidelines and was approved by the Human Research Ethics Committee of Aggeu Magalhães Research Center/FIOCRUZ-PE, under protocol CAAE 0068.0.095.000-07/2007.

RESULTS

A total of 294 variables related to maternal and neonatal care were identified, of which 113 were related to the evaluation of the quality of maternal and neonatal care of SUS.

Individual analyses were performed to measure the accuracy of each variable in identifying the phenomenon with which it was related and to ensure the quality and applicability of the index to be developed. A total of 30 variables with distortions of reality (poor reliability) were excluded.

In addition, 83 variables were analyzed according to relevance and resulted in 40 variables associated with quality and relevance.

A total of 27 simple indicators were constructed to identify and describe general aspects of maternal and neonatal hospital care that would form the composite indicators. The calculation method involved simple frequency calculation, identification of the presence of certain health care services, programs, or accreditations, and the calculation of proportions.

Subsequently, the scoring system and score range for each single indicator were defined, and a score between 0 and 1 was assigned for each indicator (Table 1).

Table 1

Scoring criteria of simple indicators of maternal and neonatal care in the Brazilian hospital network. Brazil, 2009.

Composite indicator	Simple indicators	Score/Range
		0	0.2	0.4	0.6	0.8	1
Size	Total number of neonatal beds	0	1 to 4	5 to 9	10 to 19	20 to 29	≥ 30
	Total number of obstetric beds	0	1 to 9	10 to 29	30 to 49	50 to 69	≥ 70
	Total number of neonatal admissions	0	1 to 49	50 to 99	100 to 299	300 to 499	≥ 500
	Total number of obstetric procedures	0	1 to 499	500 to 999	1,000 to 2,999	3,000 to 4,999	≥ 5,000
Complexity	Number of beds in adult ICU	0	1 to 5	6 to 10	11 to 20	21 to 30	≥ 31
	Number of beds in neonatal ICU	0	1 to 5	6 to 10	11 to 15	16 to 20	≥ 21
	Number of beds in pediatric ICU	0	1 to 5	6 to 10	11 to 15	16 to 20	≥ 21
	Number of incubators	0	1 to 5	6 to 10	11 to 15	16 to 20	≥ 21
	Tertiary assistance to high-risk pregnacy	No	–	–	–	–	Yes
	Enteral and parenteral nutritional service	No	–	–	–	–	Yes
	Cleft lip treatment	No	–	–	–	–	Yes
	Referral center for neonatal screening	No	–	–	–	–	Yes
Delivery care	High-risk vaginal delivery	No					Yes
	High-risk cesarean delivery	No					Yes
	% vaginal deliveries with hospital permanence of 24 hours	< 50.0	50.0 to < 60.0	60.0 to < 70.0	70.0 to < 80.0	80.0 to < 90.0	90.0 to 100
	% cesarean deliveries with hospital permanence of 48 hours	< 50.0	50.0 to < 60,0	60.0 to < 70.0	70.0 to < 80.0	80.0 to < 90.0	90.0 to 100
	% cesarean deliveries/year	0 or 80.1 to 100	0.1 to 10.0 or 70.1 to 80.0	10.1 to 20.0 or 60.1 to 70.0	20.1 to 30.0 or 50.1 to 60.0	40.1 to 50.0	30.1 to 40.0
Recommended practices	Shared accommodation	No	–	–	–	–	Yes
	Creation of a milk bank	No	–	–	–	–	Yes
	Accreditation in a baby friendly hospital	No	–	–	–	–	Yes
	Accreditation in an AIDS day hospital	No	–	–	–	–	Yes
	Accreditation for neonatal intermediate care	No	–	–	–	–	Yes
	Accreditation for tubal ligation	No	–	–	–	–	Yes
	Accreditation for organ procurement	No	–	–	–	–	Yes
Epidemiological practices	Commission for fatality analysis	No	–	–	–	–	Yes
	Commission for hospital infection control	No	–	–	–	–	Yes
	Commission for disease notification and investigation	No	–	–	–	–	Yes

Sixteen indicators were dichotomized, resulting in a score of 0 for “No” and 1 for “Yes”. The remaining 11 indicators represented numerical frequencies and showed a large variation of results, with a minimum score of 0 and a maximum score of 1 and subdivided into the subscores 0, 0.2, 0.4, 0.6, 0.8, and 1, which required the definition of ranges that corresponded to each subscore. Each indicator reflected a particular feature of the hospital network. Therefore, the number or percentage of each score range was assigned individually according to each feature.

The number of neonatal beds ranged between 0 and 57. This frequency was fractionated, yielding the following ranges: 0, < 5, 5-9, 10-19, 20-29, and > 30, which corresponded to the subscores 0, 0.2, 0.4, 0.6, 0.8, and 1, respectively.

Each dimension consisted of distinct numbers of simple indicators. The total value of the dimension “hospital size” and “epidemiological practice” corresponded to the number of simple indicators. The dimensions “complexity” (with seven indicators) “delivery care” (with five indicators) and “recommended hospital practice” (with seven indicators) obtained maximum scores of 5, 4, and 4, respectively, because less than 0.5% of the hospitals were situated in the highest score ranges. Therefore, the maximum possible score was 21 points, not 27, in the sum of simple indicators (Table 2).

Table 2

Scoring criteria of composite indicators and index of completeness of maternal and neonatal hospital care in the Brazilian hospital network. Brazil, 2009.

Composite indicator	Score/Range						Total
	0	1	2	3	4	5
Size	–	> 0 to 1	> 1 to 2	> 2 to 3	> 3	–	4
Complexity	0	> 0 to 1	> 1 to 2	> 2 to 3	> 3 to 4	> 4	5
Child delivery care	0	> 0 to 1	> 1 to 2	> 2 to 3	> 3	–	4
Recommended practices	0	1	2	3	4	≥ 5	5
Epidemiological practices	0	1	2	3	–	–	3
Index	Scores/Levels/Classification
Completeness of care		1 to 4	5 to 8	9 to 12	13 to 16	≥ 17
		Level I	Level II	Level III	Level IV	Level V
		Very low	Low	Intermediate	High	Very high

The index of completeness grouped the five dimensions and reached a maximum score of 21 points. It was fractionated in ascending order in five ranges and regular intervals, forming the following five levels of completeness of maternal and neonatal hospital care: I: very low, II: low, III: intermediate, IV: high, and V: very high (Table 2).

Evaluation of simple indicators that formed the “hospital size” dimension (Table 3) revealed that 87.4% of the hospitals did not have neonatal beds, 90.2% had a maximum of 29 obstetric beds, and 70.0% had less than 500 obstetric admissions in 2009.

Table 3

Number of hospitals with the score obtained using simple indicators of maternal and neonatal care. Brazil, 2009.

Dimension	Indicator	Hospital units/Score
		0	0.2	0.4	0.6	0.8	1
Size	Neonatal beds	2,971	172	136	75	31	15
	Obstetric beds	16a	1,843	1,209	226	57	49
	Neonatal hospitalizations	613	2,117	176	258	129	107
	Obstetric procedures	0	2,323	433	449	140	55
Complexity	Adult ICU	2,837	60	284	148	35	36
	Neonatal ICU	3,025	60	197	49	30	39
	Pediatric ICU	3,187	102	78	15	9	9
	Incubators	728	2,188	241	96	55	92
	Tertiary assistance to high-risk pregnacy	3,322	–	–	–	–	78
	Enteral and parenteral nutritional service	3,003	–	–	–	–	397
	Cleft lip treatment	3,390	–	–	–	–	10
	Reference in specialized neonatal care	3,391	–	–	–	–	9
Delivery care	% vaginal deliveries with hospital permanence of 24 hours	1,286	392	469	515	442	296
	% cesarean deliveries with hospital permanence of 48 hours	1,283	348	432	474	456	407
	% cesarean deliveries	607	280	572	991	390	560
	High-risk vaginal deliveries	3,248	–	–	–	–	152
	High-risk cesarean deliveries	3,245	–	–	–	–	155
Recommended practices	Shared accommodation	2,018	–	–	–	–	1,382
	Milk bank	3,182	–	–	–	–	218
	Accreditation in a baby friendly hospital	3,108	–	–	–	–	292
	Accreditation in an AIDS day hospital	3,366	–	–	–	–	34
	Accreditation for neonatal intermediate care	3,278	–	–	–	–	122
	Accreditation for tubal ligation	2,602	–	–	–	–	798
	Accreditation for organ procurement	3,116	–	–	–	–	284
Epidemiological practices	Commission for fatality analysis	2,907	–	–	–	–	493
	Commission for hospital infection control	1,291	–	–	–	–	2,109
	Commission for disease notification and investigation	1,567	–	–	–	–	1,833

ICU: intensive care unit

a hospital units with no obstetric beds registered in the National Registry of Health Establishments in 2009, but with a significant number of obstetric admissions.

More than 83.0% of the hospitals had no adult, pediatric, or neonatal ICU beds, and less than 2.3% were qualified for tertiary care of high-risk pregancies, cleft lip treatment, and specialized neonatal care.

With regard to delivery, 21.7% and 25.4% of the hospitals had the expected period of hospitalization for more than 80.0% of vaginal and cesarean deliveries, respectively. However, less than 4.6% were qualified to perform high-risk vaginal and cesarean deliveries.

Approximately 40.6% of the hospitals had shared accommodation, and 23.5% were qualified to perform tubal ligation. With regard to epidemiological hospital practices, 62.0% had commissions for nosocomial infection control, and 53.9% had commission for disease notification and investigation.

In addition, 76.0% of the hospitals in the network were of small size, 241 were of larger size (levels 3 and 4), 3.7% were high-complexity hospitals (levels 3 and 4), and 85.2% had low adequacy (levels 0 and 1) and intermediate adequacy (level 2) for child delivery care. Approximately 8.9% belonged to the three highest levels of achievement of recommended hospital practices, and 44.2% belonged to higher levels of achievement (levels 2 and 3) of epidemiological practices (Table 4).

Table 4

Number of hospitals with scores obtained using composite indicators of maternal and neonatal care. Brazil, 2009.

Dimension/Composite indicator	Level/Number of hospitals
	0	1	2	3	4	5
Size	–	2,585	573	200	41	–
Complexity	710	2,228	183	152	75	52
Delivery care	120	1,259	1,517	558	46	–
Recommended practices	982	1,528	588	195	69	38
Epidemiological practices	835	1,063	1,134	368	–	–

Analysis of the index of completeness showed that 79.7% of the hospitals had either very low or low indices of completeness of maternal and neonatal care. In addition, 5.5% had the highest indices of completeness, and of these, 52.2% were located in the Southeast of Brazil (Table 3).

The network included 3,400 hospitals, distributed as follows: 10.0% in the North, 35.2% in the Northeast, 9.7% in the Midwest, 17.9% in the South, and 27.2% in the Southeast (Table 5). More than 89.3% of the hospitals showed low or very low indices of completeness of care in the North, Northeast, and Midwest, and less than 2.7% had high or very high indices of completeness. The Southeast region had the lowest percentage (9.4%) of hospitals with very low indices of completeness, more hospitals with intermediate indices (19.9%), and the largest percentage (10.5%) of hospitals with either high or very high indices of completeness.

Table 5

Distribution of hospitals by region, according to the index of completeness of maternal and neonatal care. Brazil, 2009.

Region	Completeness of care										Total
	Very low		Low		Intermediate		High		Very high
	n	%	n	%	n	%	n	%	n	%
North	111	32.6	197	57.9	26	7.6	4	1.2	2	0.6	340
Northeast	440	36.8	623	52.1	98	8.2	25	2.1	10	0.8	1,196
Midwest	73	22.1	218	66.1	26	9.0	9	2.7	4	1.2	330
South	100	16.4	395	64.9	79	13.0	29	4.8	6	1.0	609
Southeast	87	9.4	557	60.2	184	19.9	76	8.2	21	2.3	925
Brazil	811	23.9	1,990	58.5	413	12.1	143	4.2	43	1.3	3,400

DISCUSSION

The process of elaboration of the quality index required a lot of testing and analysis and the definition of criteria and parameters. The development of a simple and easily reproducible index was a complex process. In addition to data availability, the validity and quality of indicators and the application of non-predetermined parameters needed to be ensured. The development of composite indices of living conditions using socioeconomic and environmental variables is considered one of the most important tools for the diagnosis of health conditions. , ,

The adoption of reliability and relevance criteria was fundamental for developing the index using data originated from HIS and eliminating problems related to variable incompleteness and inconsistency.

This ensured quality and validity of the indices and showed that HIS is an important tool for health care assessment. The relevance of HIS as a tool to assess the quality of hospital care has been reported by Escosteguy et al (2005).

Adoption of the dimensions from the study by Donabedian allowed the assessment of the health care network according to size and complexity. Analysis of the comprehensiveness of care enabled the establishment of a correlation of the composite indicators with delivery care and recommended and epidemiological practices. This expanded the focus of quality of care beyond the variables associated with size and complexity.

This multidimensional framework is widely used in the preparation of composite indices of social deprivation that group several indicators and statistical analyses and explain intraurban differences. ,

Another study used a global indicator, focusing not on the characterization of living conditions and population intraurban differences but on the quality of primary care in municipal management as the synthetic indicator. This study analyzed the inefficiencies in the management of activities related to the access to and provision of health care in 63.9% of the municipalities in the state of Santa Catarina, Southern Brazil.

The index of completeness of maternal and neonatal care in SUS indicated that the hospital network was predominantly of small size and low complexity, with poor performance of delivery care and poor development of recommended and epidemiological practices.

The small size of the network attests its difficulty in offering quality services to the population with minimal technological incorporation in health care. Rosa & Hortale (2000) found that deficiencies in physical infrastructure, services and equipment, staff, and organizational characteristics were correlated with the prevalence of potentially preventable perinatal deaths in the metropolitan region of Rio de Janeiro.

Child delivery care, usually, does not require highly complex health care support. However, this assistance should be adequate and accessible to the population when needed. The health network had unexpected very low and low complexity levels and many hospital units without any complexity. This compromises low-risk delivery care, and the low number of hospitals with improved technological innovation compromises the assistance to more serious cases. The limited access to higher levels of care was also observed in Belo Horizonte, MG, Southeastern Brazil, and in Paraná, Southern Brazil, and was associated with the prevalence of perinatal deaths. ,

The inappropriate performance of delivery care practices highlights the need for an adequate network to provide such care. The inadequate length of stay and excess of cesarean deliveries greatly influence clinical outcomes and neonatal and maternal deaths. Inadequate obstetric care conditions and/or deficiencies in nursery/newborn care were associated with the prevalence of preventable perinatal deaths in Piauí and Pernambuco. ,

Poor development of recommended and epidemiological practices in the hospital network suggested that some health care services have disregarded the practice of humane care, quality monitoring, infection control, and disease notification.

The index of completeness reflected the ability of hospitals to provide comprehensive and quality care to mothers and newborns through SUS. The development of this index allowed the creation of a score system for overall evaluation and establishment of a hierarchical classification among hospitals. Accordingly, this index can be an important tool for the management, surveillance, and monitoring of maternal and neonatal care.

Costa et al (2004) evaluated hospital resources for perinatal care and developed a classification and evaluation system using point scores. This system was designated “levels of complexity and potential safety of perinatal units in maternity hospitals” and established a classification system for the quality of care.

Application of this index indicated that most of the health care network had a low index of completeness. This result was expected, considering the quality of obstetric care, which usually requires procedures and interventions with little technological development. Hospitals with very low index of completeness were present in all regions but concentrated mostly in the North, Northeast, and Midwest regions. This situation is undesirable even for the performance of low-risk deliveries and highlights the presence of regional inequalities.

A similar situation was observed in Belo Horizonte, where 17.9% of maternity hospitals lacked both complexity and minimum security for the usual risk care of mothers and newborns, and 46.4% lacked conditions for medium- and high-risk assistance.

The Southeast region had a distribution close to ideal: 60.0% of the units had a low index of completeness, 20.0% had intermediate completeness, and 10.0% had a high index of completeness, although 9.4% of the hospitals had a very low index. Health services with the best quality of care in the maternal and neonatal network were available in the more developed regions of the country (Southeast and South).

The inequalities in the access of pregnant women and newborn to obstetric care in a timely manner have been reported previously and were higher in less developed regions with poor socioeconomic conditions. , ,

The index of completeness included multidimensional variables and revealed that the public hospital network provided limited care services and experienced structural problems (lack of material resources and facilities) and unavailability of certain strategic services and programs for the maternal and neonatal care segment. This highlights the need for interventions aimed at restructuring and organizing health services to reduce regional differences in the access and use of obstetric products and care and ensure the appropriate and timely care for pregnant women and newborns in Brazil.

The index of completeness proved to be sensitive (it could detect activities with different frequencies and amplitudes in hospital care), accessible (the data needed for its calculation are publicly available and can be rapidly obtained through the internet with minimal cost), easily available (the data needed for its calculation are easily accessible and can be obtained using routine services), and simple (the method of calculation is the sum of the variables).

Despite the high sensitivity in evaluation of the quality of care, this index lacks specificity. Therefore, its use should not eliminate or replace the use of other specific assessments and should be used as a management tool for preliminary assessment of the quality of maternal and neonatal hospital care in Brazil. However, it does not eliminate the need for further in loco assessments.