Development of a clinical prediction model for perinatal deaths in low resource settings.

BACKGROUND: Most pregnancy-related deaths in low and middle income countries occur around the time of birth and are avoidable with timely care. This study aimed to develop a prognostic model to identify women at risk of intrapartum-related perinatal deaths in low-resourced settings, by (1) external validation of an existing prediction model, and subsequently (2) development of a novel model.
METHODS: A prospective cohort study was conducted among pregnant women who presented consecutively for delivery at the maternity unit of Zanzibar's tertiary hospital, Mnazi Mmoja Hospital, the Republic of Tanzania between October 2017 and May 2018. Candidate predictors of perinatal deaths included maternal and foetal characteristics obtained from routine history and physical examination at the time of admission to the labour ward. The outcomes were intrapartum stillbirths and neonatal death before hospital discharge. An existing stillbirth prediction model with six predictors from Nigeria was applied to the Zanzibar cohort to assess its discrimination and calibration performance. Subsequently, a new prediction model was developed using multivariable logistic regression. Model performance was evaluated through internal validation and corrected for overfitting using bootstrapping methods.
FINDINGS: 5747 mother-baby pairs were analysed. The existing model showed poor discrimination performance (c-statistic 0·57). The new model included 15 clinical predictors and showed promising discriminative and calibration performance after internal validation (optimism adjusted c-statistic of 0·78, optimism adjusted calibration slope =0·94).
INTERPRETATION: The new model consisted of predictors easily obtained through history-taking and physical examination at the time of admission to the labour ward. It had good performance in predicting risk of perinatal death in women admitted in labour wards. Therefore, it has the potential to assist skilled birth attendance to triage women for appropriate management during labour. Before routine implementation, external validation and usefulness should be determined in future studies. FUNDING: The study received funding from Laerdal Foundation, Otto Kranendonk Fund and UMC Global Health Fellowship. TD acknowledges financial support from the Netherlands Organisation for Health Research and Development (grant 91617050).

Evidence before this study

Foetal monitoring is often a neglected sphere of childbirth care. We conducted a systematic review on fetal monitoring techniques in low- and middle- income countries (LMICs) using searches in five electronic databases (Pubmed/MEDLINE, Cochrane Library, EMBASE, POPLINE, and Global Health Library) to identify studies with a title or abstract containing MeSH/Emtree terms related to “intrapartum,” “fetal surveillance,” “outcomes,” and “low- and middle-income countries.” Out of 38 studies on intrapartum foetal heart rate monitoring, 14 were observational studies of low-moderate quality on “admission tests” and early intrapartum (cardiotocography: n = 7, intermittent auscultation: n= 1, other methods: n = 6). The findings showed association between abnormal "admission tests” and higher rates of intrapartum foetal distress, low Apgar score, neonatal intensive care unit and perinatal mortality and increased caesarean section suggesting that "admission tests” could be good screening tools to identify and triage high-risk fetuses in settings with scarce (human) resources.

Added value of this study

In this study, we tested the value of an “admission test” resulting from a Delphi consensus procedure to predict perinatal deaths. Findings suggest that a combination of basic routinely collected maternal and foetal characteristics on admission (including gestational age estimation, maternal perception of fetal movement, maternal medical history, vaginal bleeding, intermittent auscultation by hand-held Doppler or Pinard and meconium-staining of amniotic fluid when the membranes have ruptured) have high predictive performance for intrapartum-related perinatal death.

Implications of all available evidence

Models developed to predict perinatal outcomes in low resource settings are limited and show variable performance. The newly developed model and point score system developed here have potential clinical value in settings with limited resources as it may assist birth attendants improve birth care.

Alt-text: Unlabelled box

Introduction

The majority of the five million perinatal deaths globally are related to intrapartum care in resource-poor countries and are avoidable., Most of intrapartum-related deaths can be averted by identification and appropriate management of women at high risk of labour complications, skilled birth attendance for monitoring throughout childbirth and effective interventions such as emergency obstetric and new-born care. However, skilled birth attendants (SBA) in low-resource settings encounter substantial challenges due to the high volume of labouring women, inadequate number of trained staff, insufficient amount of equipment and supplies, and lack of space., As a result, labour support, monitoring and timely management for all women is often impossible in routine clinical practice.

The first contact with an SBA on admission to the labour ward is a key moment in intrapartum care. Evidence suggests that clinical tests performed on admission such as cardiotocography, intermittent auscultation and maternal perception of foetal movement could be useful, especially in low and middle income countries (LMIC). This is because these countries tend to have inadequate care and screening during antenatal period, resulting in a higher incidence of intrapartum-related morbidity and mortality.

Clinical tests performed on admission could be used in LMIC to quickly identify foetuses at high risk of mortality, and to triage them for appropriate monitoring and management strategies. More generally, risk stratification upon admission may help to optimize resource allocation in settings with heavy workload and scarce (human) workforce. Ideally, risk stratification is not merely based on the results from a single test, but also accounts for multiple maternal and foetal characteristics.6, 7, 8, 9 Yet, prognosis models that combine multiple predictors to identify women at risk of adverse birth outcomes for prompt interventions are extremely rare in LMIC.

The overall aim of this study was to contribute to the development and evaluation of low-cost, easily-applicable prognosis models for predicting perinatal deaths, through 1) external validation of an existing model for stillbirth; 2) revision of this existing model in the new clinical setting if required; and 3) development and internal validation of a new prognosis model for perinatal deaths, if necessary.

Methods

Description of the original dataset

The reporting of the study adheres to the TRIPOD guidelines (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis). First, we evaluated and updated an existing prognosis model for stillbirths in LMIC., Briefly, this model was developed in a retrospective cohort of 6,573 pregnant women and their babies in the Federal Medical Centre Bida, a tertiary hospital in Niger state, Nigeria, from January 2010 to December 2013. There were 443/6,956 (6·4%) stillbirths, defined as birth of a baby who died intrauterine after 20 completed gestational weeks. The original prediction model was developed using multivariable logistic regression and comprised of six predictors. After internal validation, the model showed excellent performance in terms of discrimination (C-statistic: 0·80, 95 % CI 0·78-0·83) and calibration in predicting stillbirths.

Description of validation and new model development dataset

The dataset collected for this study comprised a prospective cohort of labouring women at gestational age of ≥ 28 weeks, consecutively recruited as they presented for delivery at the maternity unit of Zanzibar's tertiary hospital, Mnazi Mmoja Hospital, the Republic of Tanzania between October 2017 and May 2018. The following women were excluded from the study: confirmed intrauterine foetal death before or at the time of admission to the maternity unit and women who did not undergo intrapartum care in the hospital, i.e. women admitted for elective or emergency caesarean section or post-delivery.

Sources of data

At the time of admission, trained research nurses collected routinely-measured predictors from antenatal care (ANC) card, history from the patient, in-patient file, and results of physical examination as assessed by routine nurses and they assessed outcomes mainly using in-patient files in the maternity and neonatal care units. For the predictor of maternal perception of foetal movement, a specific questionnaire was newly-developed. Data were recorded on a pilot-tested paper form, and visually inspected for inconsistencies and missing information before entry into a password-protected preformed electronic database (KobotoolBox).

Outcomes

For validation of the existing model, the outcome was stillbirth, and defined as intrapartum death ≥28 weeks gestational age, in line with the WHO definition as this is more applicable in LMICs. We focused on intrapartum stillbirths (i.e. stillbirths who had a positive foetal heart rate on admission) because we aimed to build a model to reduce intrapartum-related deaths. For the new model development, the outcome was perinatal deaths, i.e. stillbirths and neonatal deaths before hospital discharge.

Predictors

Only prenatal and pre-delivery maternal and foetal characteristics were considered for prediction of intrapartum-related deaths (Table 1). For evaluation of the existing model, this study included all six predictors considered in the Nigerian model (i.e. place of residence, maternal occupation, maternal parity, bleeding and fetal presentation and maternal comorbidity) and were similarly defined (Table 1). In the original model, maternal comorbidity was an additive score of the following medical conditions: hypertension, pre-eclampsia, diabetes, impaired glucose tolerance, sickle cell disease, renal disease, thyroid disease, syphilis and pelvic inflammatory disease (PID) but in the validation dataset, all maternal comorbid conditions were captured except PID which was not available.

Table 1

Candidate predictors of perinatal death for new model development.

	Predictors	Definition	Operationalised definitions and coding of variables
1.	Place of residence	The place where the patient resides permanently	0 = Urban
			1 = Rural
2.	Maternal occupation	Main occupation	0 = Unemployed
			1 = Self-employed
			2 = Employed
3.	Parity	Number of previous pregnancies carried beyond 28 weeks gestational age	0 = up to four previous pregnancies
			1= more than four previous pregnancies (grand multipara)
4.	Maternal comorbidity score which consisted of:
	Hypertensive disorders	Blood pressure of 140/90 mmHg and above as measured in antenatal care visits and/or on admission	0 = No hypertensive disorders
			1 = Mild hypertension/preeclampsia: systolic blood pressure of 140-159 or diastolic 90-109 mm Hg, plus proteinuria ≥ ++ for pre-eclampsia
			2 = Severe hypertension/(pre)eclampsia: systolic blood pressure ≥160mmHg or diastolic blood pressure ≥110mmHg, plus proteinuria ≥ ++ and/or seizures for (pre)eclampsia
	Diabetes	Blood Sugar (Fasting blood sugar) > 7 mmol/L or Random blood sugar> 11·1 mmol/L*	0 = No
			1 = Yes
	Sickle cell disease	Presence of haemoglobin SS, SC or S β-thalassemia*	0 = No
			1 = Yes
	Renal disease	Presence of clinical features, ultrasound findings, and elevated serum urea and creatinine*	0 = No
			1 = Yes
	Thyroid disease	Presence of clinical manifestations and elevated serum free thyroxine and triiodothyroxine concentration*	0 = No
			1 = Yes
	HIV status	HIV positive*	0 = No
			1 = Yes
	Syphilis	Diagnosed using Venereal Disease Research Laboratory test*	0 = No
			1 = Yes
5.	Bleeding	Vaginal bleeding during the current pregnancy	0 = No
			1 = Yes
6.	Foetal presentation	Part of the foetus closest to the pelvic inlet	0 = Cephalic
			1 = Abnormal Presentation (Breech/oblique or transverse)
7.	Previous caesarean section	A previous delivery via caesarean section	0 = No
			1 = Yes
8.	Multiple gestations	Number of babies carried in this pregnancy	0 = Singleton
			1 = Multiple
9.	Gestational age	Duration of pregnancy on admission as estimated by ultrasound or last menstrual period	1 = Very preterm: 28 to <32 weeks
			2 = Moderate to late preterm: 32 to <37 weeks
			0 = Term: 37-42 weeks
			3 = Post-term: ≥42 weeks 16,17
10.	Number of ANC visits	Number of ANC visits	Continuous
11.	Fundal height	The distance on the longitudinal axis of the abdomen from the top of the fundus to the upper border of the symphysis pubis	1 = Small: ≤30cm (i.e.<2500g)
			0 = Normal: 31-38cm
			2 = Large: >38cm (i.e. >4000g).18
12.	Prolonged rupture of membranes	Rupture of membranes ≥24 hours before the onset of labor19	0 = No
			1 = Yes
13.	Foetal heart rate	Foetal heart beat as measured on admission using intermittent auscultation (Pinard/hand-held Doppler)	0 = Normal: 110-160 beats/minute
			1 = Abnormal: <110 beats/minute or >160 beats/minute.20
14.	Maternal perception of foetal movement	Maternal sensation of any discrete kick, flutter, swish or roll of the foetus	0 = Normal
			1 = Reduced
			2 = Absent
			3 = Not sure
15.	Meconium stained liquor	Yellow or green discolouration of amniotic fluid21	0 = No
			1 = Yes

Predictors were mostly patient-reported or diagnosis was documented in patient-held records; diabetes, HIV and syphilis were routinely screened during ANC.

Abbreviations: ANC = Antenatal care, HIV = Human Immunodeficiency Virus,

As recommended in methodological papers, we based variable selection on “background knowledge …from previous studies in the same field of research, from expert knowledge or from common sense.” As such, for new model development the following candidate predictors were considered: all recorded six predictors from the Nigerian model as mentioned above, and five additional predictors that were identified from the literature and an international expert-based Delphi consensus (foetal movement by maternal perception, gestational age, fundal height, foetal heart rate on admission, meconium stained liquor),, and five from clinical reasoning (previous caesarean, multiple gestation, number of ANC visits, prolonged rupture of membranes and antepartum haemorrage) (Table 1). The latter category were factors identified in the clinical setting through direct observations of the quality of labour care and the development process of the PartoMa labour management guidelines (the PartoMa Project).

Determination of gestational age is notably challenging in low resource settings as most pregnant women do not have (an early) antenatal ultrasound and may also not recall their last menstrual period accurately. Thus, estimation of gestational age reflected the clinical reality whereby the most accurate available method of determination was used in the following order: 1) early ultrasound (up to 12 weeks), 2) the last menstrual period, 3) second trimester (up to 22 weeks), 4) 3rd trimester ultrasound. When none of these methods were available, gestational age was considered unknown and multiple imputation was used (see section on missing data). Precise data e.g. for gestational age, fundal height and foetal heart rate is difficult to obtain in these settings and thus categorisation of continuous data was used as a crude scale for these measurements, (Table 1).

Sample size

It has been recommended thatexternal validation studies should include at least 100, but preferably 200 or more outcome events. We aimed to include at least 200 events, in order to allow a sufficient sample size to develop a new model using more predictors with at least 10 events per candidate predictor. At MMH the stillbirth incidence was around 3.8%. Thus, the required sample size was 5,263 participants. With 12,000 births annually, this roughly corresponds to a seven-month period of data collection.

Missing data

Multiple imputation was applied to account for missing data using the MICE package in R. The imputation accounted for all candidate predictors and outcomes in the dataset. This resulted in 20 multiply imputed datasets., All analyses were repeated across the 20 datasets with pooling of estimates and their uncertainty measures using Rubin's rules.

Statistical analysis

Categorical variables were described using frequencies and percentages. As in the Nigerian study, all continuous data were summarized using medians and interquartile ranges (IQR) which allowed comparison of baseline characteristics between the Nigerian and Zanzibar datasets. Descriptive statistics were generated for the original data (before imputation) and the proportion of missing values was calculated for all candidate predictors. For the continuous variable of number of antenatal care visits, non-linear predictor-outcome association was explored using restricted cubic splines.

Predictive performance

For all (existing and newly developed) models, we assessed calibration and discrimination performance. Calibration was visually assessed using a calibration plot, comparing the agreement between observed frequencies of stillbirth (original and updated models) and perinatal deaths (new model) in the new dataset and the predicted risks. The ability of the models to discriminate between women with and without stillbirth (original and updated models) and perinatal death (new model) was assessed using the concordance (c)-statistic, which is equivalent to the area under a receiver operating characteristic (ROC) curve for prognostic models with binary outcomes.,32, 33, 34, 35

Updating the original model

The original model was adjusted to the new cohort using recalibration methods (adjustment of the intercept and adjustment of both the intercept and slope) previously described.,

Prognostic modelling

For the new model development, multivariable logistic regression was used with all candidate predictors. This strategy is may be preferred over stepwise selection methods, which often lead to model instability and overfitting. All predictors (including all comorbidities) were entered individually in the initial model. Subsequently, hypertensive disorder and sickle cell were presented as individual predictors in the final model because they were the maternal co-morbidity with highest estimated risk. The remaining maternal conditions were combined into a comorbidity score (i.e. adding up of comorbidities). No interactions were identified clinically and so an additive model was used. This also reduces the risk of overfitting.

Internal validation and shrinkage

Model optimism was assessed via bootstrap resampling. Briefly, the aforementioned prediction model was refitted (i.e. re-estimation of the coefficients) in 200 bootstrap samples, and the performance of these models was then evaluated in the original sample. This yielded a shrinkage factor which was used to adjust both the regression coefficients and c-statistic of the original model for optimism.

All analyses were performed in R version 3·5·3 (The R Foundation for Statistical Computing, 2019).

Presentation of the model

Methods previously described were used to derive a point score system for the newly developed prognostic model. Risk estimates were organised into clinically meaningful categories. An example is given which also illustrates the correspondence between the risks estimated by the multivariable model directly and those approximated by the points system.

Ethics approval and consent to participate

The study was approved by the Zanzibar Medical Research Ethical Committee (ZAMREC/0004/AGUST/17). Upon arrival to the admission room, a research nurse assessed the eligibility criteria of women. Written information in Kiswahili about the study was read out to the women by a research nurse. Women were then asked for their voluntary consent to participate in the study. Hardcopies of the data were stored in a locked office and electronic data were password protected.

Role of the funding source

The funders of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study, and all the authors accept final responsibility for the decision to submit for publication.

Results

Participants

Between October 2017 and May 2018, 7708 pregnant women delivered at the maternity ward of Mnazi Mmoja hospital and 5610 women were included in the study (a total of 5747 babies, (Figure 1). Table 2 depicts the characteristics of these women and compares them to the women of the Nigerian dataset. The majority of women in both datasets lived in urban areas. The women in Zanzibar cohort had a lower median age and parity, but higher literacy and unemployment rates. While the majority of women in both datasets had singleton pregnancies with cephalic presentation, the Nigerian dataset had more multiple gestations and abnormal presentations. The median gestational age was higher than the derivative dataset and could not be determined in 12·9% (n=721) of cases.

Figure 1

Flowchart of participant inclusion

Table 2

General characteristics of the study population.

Characteristics	Development cohort of the existing model12	Cohort for external validation, updating and new development
	All women N=6573	All women N=5610	Missing data N (%)	Live infants N=5454 (94·9)	Stillbirths N=191 (3·3)	Neonatal deaths N=102 (1·8)
Socio-demographic and socio-economic characteristics
Maternal age in years	27 [24-30]	26 [22-31]	8 (0·1)	26 [22-31]	28 [23-32]	26 [23-31]
Maternal height (cm)	156 [153-160]	158 [154-161]	997 (17·8)	158 [154-161]	157 [153-161]	158 [155-160]
Maternal weight on admission (kg)	65 [57-75]	67[60-76]	496(8·8)	68[60-76]	66[60-76]	69[60-79]
Place of residence
Urban	5707 (89·1)	4515(80·9)	29(0·5)	4395((95.3)	149(3.2)	68(1.5)
Rural	700 (10·9)	1065(19·1)		1037(93.9)	39(3.5)	28(2.5)
Maternal education
Educated	3284 (63·8)	5340(96·4)	67(1·2)	5257(95.2)	161(3.0%)	100(1.8)
Not educated	1866 (36·2)	202(3·6)		197(94.9)	8(3.9)	2(1.0)
Maternal occupation
Not-employed	2894 (49·6)	4164(74·8)	73(1·3)	4052(94.9)	152(3.6)	65(1.5)
Self-employed	1969 (33·8)	966(17·4)		938(95.0)	26(2.6)	23(2.3)
Employee	968 (16·7)	406(7·3)		403(97.3)	8(1.9)	3(0.7)
Marital status
Married	N/A	5401(96·7)	22(0·4)	5254(95.0)	182(3.3)	97(1.8)
Not married	N/A	186(3·2)		179(93.7)	9(4·7)	3(1.6)
Obstetrics history
Parity	2(0-3)	1(0-4)	112(2·0)	1[0-3]	1[0-3]	1[0-3]
0-4	N/A	4810(87·5)		4677(94.9)	160(3.2)	93(1.9)
≥5	N/A	687 (12·5)		667(94.7)	28(4.0)	9(1.3)
Previous foetal loss
No	N/A	4570(82.4)	66(1·2)	4445(94.9)	154(3.3)	85(1.8)
Yes		974(17.6)		1009(94.9)	37(3.5)	17(1.6)
Number of previous foetal loss	0 [0-1]	0[0-0]		0[0-0]	0[0-0]	0[0-0]
Previous caesarean section
No	N/A	5224(95.3)	128(2.3)	5091(95.1)	165(3.2)	90(1.7)
Yes		258(4.7)		240(91.6)	16(6.2)	6(2.3)
Number of previous caesarean sections	0 [0-0]	0[0-0]		0[0-0]	0[0-0]	0[0-0]
Pregnancy characteristics
Number of antenatal visits	N/A	4[3-4]	74(1·3)	4[3-4]	3[2·5-4]	4[2·8-4]
Maternal comorbidity score*	0 [0-0]	0[0-1]	72(1·3)	0[0-1]	0[0-1]	0[0-1]
0	N/A			5022(95.5)	153(2.9)	83(1.6)
1				148(94.9)	7(4.5)	1(0.6)
≥2				1(50.0)	1(50.0)	0
Bleeding in pregnancy
No	6,406 (94·9)	5552(99·0)	0	5434(95.6)	167(2·9)	84(1.5)
Yes	341 (5·1)	57(1·0)		20 (32.3)	24(38.7)	18(29.0)
Gestational age (days)	265 [137-276]	280[266-280]	721(12·9)	278[259-280]	267[231-280]	268[228-280]
Very preterm	N/A	118(2·4)		92(71.9)	21(16.4)	15(11.7)
Moderate to late preterm	N/A	715(14·6)		714(94.9)	29(3.9)	9(1.2)
Term	N/A	3820(78·1)		3754(96.6)	85(2.2)	46(1.2)
Post-term	N/A	236(4·8)		229(95.4)	6(2.5)	5(2.1)
Fundal height(cm)	N/A	34[32-36]	260(4·6)	34[32-36]	33[30-36]	34[30-38]
Small	N/A	673(12·6)		604(88.8)	52(7.6)	24(3.5)
Normal	N/A	4218(78·8)		4120(96.5)	109(2.6)	42(1.0)
Large	N/A	459(8·6)		485(92.7)	21(4.0)	17(3.3)
Birthweight (kg)	3·1[2·7-3·4]	3·1[2·8-3·4]	211(3·8)	3·1[2·8-3·4]	2·5[1·8-3·1]	2·6[1·7-3·1]
Number of babies
Singleton	6201 (89·2)	5474 (97·6)	0	5214(95.3)	180(3.3)	80(1.5)
Multiple**	754 (10·8)	136(2·4)		240(87.9)	11(4.0)	22(8.1)
Foetal presentation
Cephalic	6506 (93·7)	5359(97·4)	107(1·9)	5207(95.2)	171(3.1)	91(1.7)
Breech	334 (4·8)	137(2·5)		144(92.3)	6(3.8)	6(3.6)
Others	100(1·4)	6(0·1)		4(57.1)	2(28.6)	1(14.3)
Sex of neonate
Male	3,506 (51·4)	2704(49·5)	150(2·7)	2640(95.0)	94(3.4)	46(1.7)
Female	3,310 (48·6)	2755(50·5)		2679(95.2)	83(2.9)	52(1.8)
Cervical dilatation on admission	N/A	4[2-6]	244(4·3)	4[2-6]	3[2-5]	3[2-5]
Abnormal foetal heart rate on admission
No	N/A	4897(98·6)	597(10·6)	4860(96.2)	113(2.2)	78(1,5)
Yes	N/A	94(1·4)		36(51.4)	32(45.7)	2(2.9)
Prolonged rupture of membranes
No	N/A	5483 (99·1)	74(1·3)	5343(95.2)	182(3.2)	89(1.6)
Yes	N/A	52(0·9)		46(8·7)	5(8.8)	6(10.5)
Meconium-staining of amniotic fluid
No	N/A	5349(99·2)		205(95.1)	178(3.3)	93(1.7)
Yes	N/A	43(0·8)	217(3·9)	37(84.1)	5(11.4)	2(4.5)
Maternal perception of foetal movement
Normal	N/A	4631(83·4)	53(0·9)	4583(96.6)	101(2.1)	58(1.2)
Reduced	N/A	544(9·8)		495(88.7)	44(7.9)	19(3.4)
Absent	N/A	38(0·7)		23(56.1)	17(41.5)	1(2.4)
Not sure	N/A	343(6·2)		325(92.9)	18(5.1)	7(2.0)
Referral pathway
Home	N/A	5325(99·0)	228((4·1)	5179(94.9)	177(3.2)	100(1.8)
Referred	N/A	56(1·0)		54(93.1)	3(5.2)	1(1·7)

Continuous variables are given as median [Interquartile range] and categorical variables as frequency (percentages).

*conditions considered in the comorbidity score: diabetes, HIV, thyroid disease, renal disease, syphilis.

**n=273 in the current cohort.

N/A = information not available.

In addition, 22.9% (n=1282) of women had co-existing medical conditions (see supplementary file for prevalence of all comorbidities). Number of antenatal care (ANC) visits significantly increased with the following measures of socioeconomic status: living in an urban region, higher levels of education, being married and being employed and was linearly related to the outcome. Therefore, the number of ANC visits was included as a linear term in the model and as a single proxy for socioeconomic status, indicating an important pathway in which socioeconomic status influences pregnancy outcome. The majority of women presented in early phases of labour (median 4 (IQR:2-6) cm dilatation). The caesarean section rate was 7% (n=392), and there were 16 vacuum deliveries (0·3%). There were 191 intrapartum stillbirths (3·3%) and 102 neonatal deaths (1·8%) that occurred before hospital discharge in the Zanzibar cohort. Of all live births, 8·7% (n=468/5556) had an Apgar score between 1-6 at one minute, which decreased to 2·2% (n=120/5556) at five minutes. There were five women who died postpartum in the cohort. Missing data ranged from 0-17·8% (maternal height).

Performance of the existing model

After applying the previously published Nigerian model to the data from Zanzibar (Tanzania), the predicted probabilities were systematically too high and did not discriminate well between babies who died and survived (Table 3 and Figure 2A-C). The discriminative ability of the model was much lower in the validation set (0·57(95% CI 0·56-0·58)) than in the original development dataset (0·80 (95 % CI 0·78–0·83)).

Table 3

Regression coefficients for the predictors and intercepts of the original and updated models.

	Original model	Update 1 (intercept only)	Update 2 (intercept and slope)
Calibration intercept	-	-1·053 (standard error 0·006)	-1·894 (standard error 0·043)
Calibration slope	-	-	0·609 (standard error0·007)
Intercept	‐3·6486	-4·6948	-5·5426
Maternal co-morbidity	0·7077	0·7077	0·4305
Place of residence
Urban
Rural	1·3047	1·3047	0·7937
Employment
Homemaker/unemployed
Self-employed	-0·3022	-0·3022	-0·1838
Public/private employment	-0·3788	-0·3788	-0·2304
Parity	0·0797	0·0797	0·0485
Bleeding
No
Yes	2·1579	2·1579	1·3127
Presentation
Cephalic
Breech	0·9616	0·9616	0·5850
Other presentation	2·0588	2·0588	1·2524

Figure 2

The grey line (plots A-C) and dashed line (plot D) show a perfect calibration where predicted probabilities are equal to observed probabilities.

The solid line (plots A-C) and dotted lines (plot D) shows the logistic calibration curve and is derived by estimating the relation between the linear predictor and observed outcomes using a logistic regression model.

The dotted line (plots A-C) shows the non-parametric calibration curve and is derived by describing the relation between the linear predictor and observed outcomes using a LOWESS smoother.

The solid line on plot D is the bias-corrected (overfitting- corrected) estimates of predicted vs. observed values.

The ticks at the bottom of the chart along the x-axis represent the frequency distribution of the predicted probabilities.

Updating of original model

Model performance remained unsatisfactory after updating the intercept term and common slope (Figure 2C). These adjustments (by definition) could not improve the low c-statistic index. This motivated de novo model development in this study.

Development of new model

All 5747 babies were used to develop the new prognostic model with 15 predictors for perinatal deaths (i.e. stillbirths and neonatal deaths). Bootstrap validation yielded a shrinkage factor of 0·95, which was applied to shrink the coefficients of the final model. (Table 4) Model performance was very good in terms of calibration and discriminative ability with optimism adjusted calibration slope of 0·94 and c-statistic of 0·80 and 0·78 before and after internal validation. (Figure 2D).

Table 4

Final new model.

Coefficient	Unadjusted coefficient*	Standard error	Adjusted coefficient	Adjusted Odds ratio*
Intercept	-3·651	0·261	-3·593
Number of antenatal care visits⁎⁎	-0·112	0·062	-0·105	0·894
Hypertensive disorders
No hypertensive disorders	Reference category
Mild hypertension	0·128	0·252	0·120	1·137
Severe hypertension	0·720	0·180	0·677	2·054
Sickle cell anaemia	2·649	1·003	2·491	14·139
Other comorbidities†,⁎⁎⁎	0·723	0·434	0·680	2·062
Gestational age
Term	Reference category
very preterm	1·780	0·255	1·674	5·931
mild-moderate preterm	0·177	0·196	0·167	1·194
Post-term	0·297	0·331	0·280	1·346
Fundal height
Normal	Reference category
Small	0·749	0·178	0·704	2·115
Large	0·353	0·230	0·332	1·424
Abnormal/non-reassuring FHR	2·520	0·296	2·370	12·425
Rupture of membranes >24hours	0·916	0·437	0·862	2·500
Meconium-staining of amniotic fluid	1·237	0·486	1·163	3·445
Maternal perception of foetal movement
Normal	Reference category
Reduced	1·060	0·177	0·997	2·885
Absent	1·830	0·440	1·721	6·236
Unsure	0·759	0·241	0·714	2·137
Bleeding	3·322	0·334	3·124	27·720
Grand-multiparity (≥5 parity)	0·059	0·209	0·055	1·061
Previous scar	0·403	0·283	0·379	1·497
Abnormal presentation	0·056	0·388	0·053	1·058
Multiple pregnancy	0·621	0·266	0·584	1·860

Unadjusted coefficient. denotes coefficient before shrinkage; Adjusted denotes coefficient after shrinkage.

Odds ratio per one antenatal care visit.

Odds ratio per comorbidity.

Other comorbidities: diabetes, HIV, thyroid disease, renal disease, syphilis

Model specification:

Risk of perinatal death = 1/(1+exp (-(-3·593 + -0·105 *(Number of antenatal care visits)+ 0·120 * (Mild hypertension) + 0·677 * (Severe hypertension) + 2·491 * (Sickle cell anaemia) + 0·680 * (Other comorbidities) + 1·674 * (very preterm) + 0·167*(Mild-moderate preterm) + 0·280 * (post-term) +0·704 * (small) + 0·332 * (large) + 2·370 * (Abnormal Foetal heart rate) + 0·862 * (rupture of membranes >24hours) + 1·163 * (meconium-staining of amniotic fluid) + 0·997 * (Reduced foetal movement) + 1·721 * (Absent foetal movement) + 0·714 * (Unsure of foetal movement) + 3·124 * (Bleeding) + 0·055 * (grand-multiparity) + 0·379 * (Previous scar) + 0·053 * (Abnormal presentation) + 0·584 * (Multiple pregnancy))))

Worked example: for a para 6 with 4 antenatal care visits who presented at term with preeclampsia, history of bleeding and reduced foetal movement

Risk of perinatal death = 1/(1+exp(-(-3·593 + (-0·105 * 4 + 0·120*0 + 0·677*1 + 2·491*0 + 0·680*0 + 1·674*0 + 0·167*0 + 0·280*0 + 0·704*0 + 0·332*0+ 2·370*0+ 0·862*0+ 1·163*0+ 0·997*1 + 1·721* 0+ 0·714*0 + 3·124*1+ 0·055* 1+ 0·379*0 + 0·053*0 + 0·584*0))))

=1/(1+exp (-0·84))

Risk of perinatal death = 0·70, which corresponds to 70%.

A simple scoring system is presented in Tables 5 and 6. There was good agreement between predicted and observed risks. Risk stratification assigned 30%, 40% and 30% of babies as low (predicted probability ≤ 2.0%), moderate (predicted probability 2.1-5.0%) and high risk (predicted probability > 5.0) of perinatal death respectively. Around three quarters of all perinatal deaths occurred in the high risk group.

Table 5

Point score system for risk of perinatal death on admission to the labour ward.

Risk Factor	Categories	Points
Number of antenatal care visits*
	0-3	1
	4-7	0
	≥8	-2
Hypertensive disorders	None	0
	Mild	0
	Severe	2
Sickle cell anaemia	No	0
	Yes	7
Other comorbidities	0-1	0
	2-3	4
Gestational age	Normal	0
	very preterm	4
	mild-moderate	0
	Post-term	1
Fundal height	Normal	0
	Small	2
	Large	1
Abnormal/non-reassuring FHR	No	0
	Yes	6
Rupture of membranes >24hours	No	0
	Yes	2
Meconium-staining of amniotic fluid	No	0
	Yes	3
Maternal perception of foetal movement	Normal	0
	Reduced	3
	Absent	5
	Unsure	2
Bleeding	No	0
	Yes	8
Grand-multiparity (≥5 parity)	No	0
	Yes	0
Previous scar	No	0
	Yes	1
Abnormal presentation	No	0
	Yes	0
Multiple pregnancy	No	0
	Yes	2

In women who have attended eight or more antenatal visit, points should only be deducted if no risk factor has been identified.

Table 6

Point score system for risk of perinatal death on admission to the labour ward.

Point total	Estimate of risk	Number of perinatal deaths	Total Number of babies	Observed incidence of perinatal deaths
-2 to 0	0.015	32	1735	0.018
1	0.031	27	1500	0.018
2	0.044	18	794	0.023
3	0.063	39	777	0.050
4	0.09	20	362	0.055
5	0.126	20	206	0.097
6	0.174	17	102	0.167
7	0.236	17	81	0.210
8	0.31	17	49	0.347
9	0.397	13	35	0.371
10	0.49	5	14	0.357
11	0.584	13	20	0.650
12	0.672	10	16	0.625
13	0.75	5	10	0.500
14	0.814	13	16	0.813
15	0.865	8	10	0.800
≥16	0.966	19	20	0.950

Worked example: for a para 6 with 4 antenatal care visits who presented at term with severe preeclampsia, history of bleeding and reduced foetal movement: 0+0+0+2+8+3=13; risk probability=0.75.

Discussion

We carried out a prospective cohort study of 5747 mother-baby pairs to develop a model for predicting perinatal deaths in women who arrive for delivery at a tertiary hospital in a low-resource setting. A crucial step in this process was the evaluation and updating of an existing stillbirth model which pointed out the need to develop a new model. We therefore used the information gained from this previous model, a literature review, consensus-based recommendations and clinical reasoning to develop and internally validate a new model to prognosticate both intrapartum stillbirths and early neonatal deaths. The new model consisted of 15 predictors and showed good calibration and discriminative ability after internal validation (c-statistic of 0·78). Our results further emphasize the importance of antenatal care, and comprehensive assessment on admission including accurate assessment of gestational age, blood pressure, history of sickle cell anaemia, bleeding, maternal perception of foetal movement and foetal heart rate in predicting perinatal outcomes. A simple and low-tech point score system was developed for ease of use and rapid risk assessment and stratification of women on admission to the labour ward.

Strengths and Limitations

This is one of the few clinical prediction models developed for use in pregnant women in LMIC. The comprehensiveness of the model by including both maternal and foetal characteristics makes it relevant to neonatal and maternal survival. Predictor selection was based on prior knowledge and predictors can be obtained easily through history-taking and physical examination at the time of admission making the model directly applicable for low-resource settings. The prospective design reduced missing data and allowed the inclusion of maternal perception of foetal movement – an important yet often neglected predictor in the clinical setting., It may seem more clinically useful to develop a prognostic model to predict adverse perinatal outcomes in women with seemingly normal pregnancy (e.g. term, singleton foetus, cephalic presentation), whose risks are harder to predict. However, we chose broad inclusion criteria for more generalisability of the model to all women admitted to the labour ward. Since admission is usually the first contact point with an SBA, it is paramount for all women to be examined for conditions that may not have been detected in ANC as well as newly-arising problems. Supporting this approach are results from our prior study that showed inadequate risk assessment on admission, with delayed detection of problems such as twins, breech presentation, and intrauterine foetal death until close to delivery. This emphasises the need to strengthen assessment of all women on admission to the labour ward in these types of settings.

Our study methods have several limitations. This is a single-centre study and thus generalisation of the model to other setting is questionable. Thus external validation, update and/or revision are recommended in other settings. This could include context-informed determination of appropriate risk classification cut-off points. Ideally, the generalisability of existing models could be improved by sharing data across countries and health care settings. Also, data quality is a major concern because the data was collected within the inherent limitations of the clinical situation. Thus, inaccuracies existed in measuring continuous predictors such as gestational age, fundal height and foetal heart rate, which led to simplifying these measurements to a cruder scale. Categorisation of these continuous variables may have caused loss of information about the relation between the predictors and the outcome and reduced performance of the model. Moreover, measurement error may have occurred because many maternal comorbidities such as diabetes mellitus, renal disease and sickle cell disease were not tested but depended on patient-report or absence of documented diagnosis. Maternal infection screening was commonly not done. Hence, there was probable misclassification of maternal conditions which may have affected the effects of predictors. However, pelvic inflammatory disease was not an important variable in the derivation of the original model and therefore we do not think that the absence of pelvic inflammatory disease is sufficient not to consider this study as an external validation of the Nigerian model.

Determination of stillbirth as intrapartum depends on the accurate auscultation of foetal heart rate on admission. It was highly probable that there was misclassification of stillbirths because of unchecked or inaccurate FHR assessment (false positive and negative FHR detection). The definition of perinatal death included late neonatal deaths (occurring after seven days), whose causes are more likely to differ to those of stillbirths and early neonatal deaths (<7 days). However, during the same study period, a study found that the overwhelming majority of neonatal deaths born in this hospital were early neonatal deaths, and only about 5% were late neonatal deaths. In addition, we included a large number of predictors in the new model which may lead to model instability. To limit the potential impact of overfitting, we adopted shrinkage methods and adjusted estimates of model performance for over-optimism. While 15 is a large number of model predictors, these are easily available, rapid to assess and are all necessary for the assessment of every woman who is admitted in the labour ward. Incorporating other additional prenatal risk factors such as maternal age, and indicators of socioeconomic status (e.g. maternal education, occupation and residence) may improve the predictive ability of the model but risks the model becoming more complex and unstable. Thus, antenatal care visit was used to reflect socioeconomic status since studies have consistently shown socioeconomic status to influence antenatal care utilisation and quality among pregnant women. However, such proxy indicator may be associated with perinatal outcomes through independent mechanisms not related to antenatal care. Lastly, the unit of analysis in this study is the baby and this raises concern that multiple pregnancy may violate the assumption of independency of observations that underpin our analyses. However, an additional analysis using the mother as the unit of analysis, with a stillbirth or neonatal death in any baby as the outcome showed the same predictive performance.

Interpretation

Simple prediction tools which are based on clinical maternal and foetal characteristics have high predictive ability for the risk of adverse birth outcomes., Unfortunately, the Nigerian model we evaluated here performed poorly when predicting stillbirths of ≥28 weeks gestation. Multiple factors may help explain this finding: 1) the observed difference in case mix of predictor variables and outcome occurrence between the development and validation samples; 2) the original model omitted important predictor variables and; 3) the validation study indicates that the model's predictive mechanisms are completely different in the validation population, perhaps due to major differences in outcome definitions and routine care. Yet, the model provided useful information which we leveraged for a new model development.

Two large and multi-centre predictive studies have recently been carried out in general populations of South Asian countries to predict stillbirths and/or neonatal deaths. They created several prediction models using prenatal, predelivery, delivery and postdelivery variables. Our model is more comparable to the models that use predelivery and delivery variables to predict perinatal death as we used mostly clinical predictors that are identifiable before or during birth. Like in our study, prematurity/gestational age, multiple gestation, antepartum haemorrhage, and hypertensive disorders were important predictors of perinatal death. The overall predictive ability of these models were lower than in our study with AUC values 0.73 or less.,

Labour monitoring is known to be time-consuming, labour-intensive and poorly performed in low-resourced busy labour wards. There are simply not enough doctors and staff to provide one-to-one care. The high incidence of perinatal death in the study setting signifies the need to improve baseline quality of care as well as prioritisation and intensified monitoring of high risk babies. Also, women with obstetric complications may be at higher risk of being overlooked, delayed and neglected as resources and screening are redirected towards COVID-19, the symptoms of which mimic obstetric emergencies. Thus, there is a need for obstetric risk assessment tools to aid safe triaging.,, Such simplified prediction tools are only useful when coupled with management guidelines. Therefore, these tools need to be externally validated and tested both quantitatively and qualitative alongside context-specific clinical guidelines to determine their impact on birth outcomes, implementation issues and user experience. Interventions such as anti-hypertensive and anti-convulsive treatments and operative delivery may lower the estimated risk of perinatal death, whereas other treatment such as oxytocin augmentation may increase the risk and therefore alter model performance. It is also highly plausible that current routine care includes some form of prioritisation of identified high(er) risk women which allows closer monitoring/follow-up and quicker intervention. For example, a higher proportion of pregnancies with abnormal presentation (30%) and previous caesarean section (40%) were delivered by a caesarean section which may also have reduced the effect size of these predictors. Thus, future studies will need to continue to explore the development of dynamic models, in which intrapartum interventions and treatment quality that alter model performance can be incorporated so that models can be routinely updated based on developing clinical information.

We externally validated a Nigerian prognostic model for stillbirths and found a substantial reduction in predictive performance. Subsequently, we developed and internally validated a prognostic model with higher predictive ability for perinatal death using easily-available parameters in low-resourced, busy labour wards. Before the model can be implemented, further validation and implementation studies need to be carried out to determine whether model predictions can also improve outcomes in newborns and clinical practices in different settings. Future studies should also consider dynamic modelling strategies to account for the various interventions during labour which modify the risk of adverse birth outcomes.

Contributors

NH conceived and designed the experiment, managed data acquisition, analysed the data, interpreted the results, drafted the first version of the manuscript and led the drafting process of the manuscript.

MJR conceived and designed the experiment, contributed to interpretation of the results and critically revised the manuscript.

KW contributed to design of experiment, managed data acquisition and critically revised the manuscript.

NHN carried out data acquisition and critically revised the manuscript.

KG contributed to design of experiment, data analysis and critically revised the manuscript.

CD contributed to data analysis, interpretation of the results and critically revised the manuscript.

TD contributed to data analysis, interpretation of the results and critically revised the manuscript.

TM contributed to management of data acquisition, interpretation of the results and critically revised the manuscript.

AF conceived and designed the experiment, contributed to interpretation of the results and critically revised the manuscript

DEG conceived and designed the experiment, contributed to interpretation of the results and critically revised the manuscript.

JLB conceived and designed the experiment, contributed to interpretation of the results and critically revised the manuscript.

Declaration of interests

NH received funding from Laerdal Foundation and Otto Kranendonk Fund for the described work. TD reports consulting fees from Biogen, unrelated to this work.