Machine learning prediction of non-attendance to postpartum glucose screening and subsequent risk of type 2 diabetes following gestational diabetes.

OBJECTIVE: The aim of the present study was to identify the factors associated with non-attendance of immediate postpartum glucose test using a machine learning algorithm following gestational diabetes mellitus (GDM) pregnancy.
METHOD: A retrospective cohort study of all GDM women (n = 607) for postpartum glucose test due between January 2016 and December 2019 at the George Eliot Hospital NHS Trust, UK.
RESULTS: Sixty-five percent of women attended postpartum glucose test. Type 2 diabetes was diagnosed in 2.8% and 21.6% had persistent dysglycaemia at 6-13 weeks post-delivery. Those who did not attend postpartum glucose test seem to be younger, multiparous, obese, and continued to smoke during pregnancy. They also had higher fasting glucose at antenatal oral glucose tolerance test. Our machine learning algorithm predicted postpartum glucose non-attendance with an area under the receiver operating characteristic curve of 0.72. The model could achieve a sensitivity of 70% with 66% specificity at a risk score threshold of 0.46. A total of 233 (38.4%) women attended subsequent glucose test at least once within the first two years of delivery and 24% had dysglycaemia. Compared to women who attended postpartum glucose test, those who did not attend had higher conversion rate to type 2 diabetes (2.5% vs 11.4%; p = 0.005).
CONCLUSION: Postpartum screening following GDM is still poor. Women who did not attend postpartum screening appear to have higher metabolic risk and higher conversion to type 2 diabetes by two years post-delivery. Machine learning model can predict women who are unlikely to attend postpartum glucose test using simple antenatal factors. Enhanced, personalised education of these women may improve postpartum glucose screening.

Introduction

Gestational diabetes mellitus (GDM) is associated with adverse maternal and offspring outcomes both in the short and long-term. Incidence of type 2 diabetes (T2D) in women with history of GDM using real-world data can be 20-fold higher [1]. The conversion to T2D following the pregnancy seems to happen early in the postpartum period with highest risk within 3–6 years of index pregnancy; by 10–14 years, 50% of women have dysglycaemia [2, 3]. GDM is also associated with at least two-fold greater risk for developing hypertension and cardiovascular disease (CVD) [1, 4]. This highlights the importance of early identification of women at risk to implement any preventive strategies [5, 6]. Most international guidelines recommend 75-g oral glucose tolerance test (OGTT) or HbA1c for all GDM women in the postpartum period, and then at least once every 1–3 years [7, 8].

In 2015, the National Institute of Health and Care Excellence (NICE) changed the screening guidelines from every 3 years to every year and recommended a fasting plasma glucose (FPG) test at 6 to 13 weeks after delivery or HbA1c at 13 weeks onwards instead of OGTT [9]. Despite these recommendations and evidence of diabetes risk, postpartum testing for dysglycaemia (by any form of glucose testing; OGTT, FPG or HbA1c) is low and reaches only about 30% by year three following a GDM pregnancy [1, 10]. While some centres were able to improve their uptake to 70% by dedicated coordinators, this has not been replicated by others and uptake remained poor [11, 12]. System and patient barriers including the lack of awareness (‘it does not affect me’), discomfort and duration of the test (especially for OGTT), other socio- economic factors and poor transition of information from secondary to primary care are primary reasons for poor uptake of postpartum screening [13, 14].

We had earlier shown that women who did not attend postpartum glucose testing (ppGT) in three centres from our region had higher metabolic risk factors [10]. Subsequently, in one of the centres, we introduced a dedicated advanced nurse practitioner to encourage women to attend the postpartum testing and send personalized letters to women and their GPs for annual screening. However, there has been limited data on an individualised approach for targeting women who are unlikely to attend the ppGT. The primary aim of this study was to identify patient characteristics of women who did not attend the immediate ppGT in a real-world setting and to assess their subsequent T2D risk within 24-months post-delivery. Further, we built a predictive model to identify women who are less likely to attend ppGT using machine learning.

Methods

Study population

Detailed demographic, clinical and anthropometric data for all pregnant women who had GDM and ppGT due between January 2016 and December 2019 (n = 607) in a district general hospital (George Eliot Hospital NHS Trust, Nuneaton, UK) were collected. No personal information of the subjects was obtained. This study was conducted as a service evaluation audit and ethical approval was not necessary. The audit was approved by the GEH Diabetes and Audit department. A selective screening was done based on NICE 2015 criteria, using 75g OGTT between 24 and 28 weeks of gestation. GDM diagnosis was made if FPG ≥5.6 mmol/L or 2- hour plasma glucose ≥7.8 mmol/L. The risk assessment for screening includes women with higher pre-pregnancy BMI (≥30 kg/m2), family history of diabetes, previous GDM, previous macrosomic baby (birth weight ≥ 4500g) and from ethnic minority groups [9]. At the time of discharge, all GDM women were instructed to schedule a ppGT at 6 to 13 weeks and received a phone reminder in the previous week of appointment by an advanced nurse practitioner.

In addition to the detailed antenatal data, data at birth including initiation of breastfeeding and birth centile of the infant, assessed by ethnicity specific UK reference chart were collected [15]. Women received a personalized letter to attend the ppGT (OGTT for those who had normal FPG and abnormal 2-hr glucose during antenatal OGTT or HbA1c for other abnormalities) (S1 Appendix). If they had difficulty in attending OGTT, they were invited again for postpartum HbA1c test. Following the results, GDM women and their General Practitioner (GP) received a second letter indicating their results and highlighting their future risk of T2D and CVD and advising them to visit their GP for annual HbA1c tests. All the follow-up glucose/HbA1c data (up to 24 months from the date of delivery) were extracted from the hospital electronic database. During follow-up, dysglycaemia (diabetes and prediabetes) was identified using American Diabetes Association (ADA) criteria [diabetes: HbA1c ≥48 mmol/mol (≥ 6.5%) or FPG ≥7.0mmol/L and 2-hr ≥11.1mmol/L post 75g OGTT; prediabetes: HbA1c ≥39 and <48 mmol/mol (≥5.7 and 6.4%) or FPG ≥5.6 and ≤6.9mmol/L and 2-hr ≥7.8 to ≤11.0 mmol/L post 75g OGTT] [16].

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics for Windows, version 27 (IBM Corp., Armonk, NY, USA). Baseline characteristics were expressed in percentages for categorical variables and mean ± standard deviation (SD) for continuous variables. Univariate and multivariate hazard ratios (HR) were estimated using Cox proportional hazard model in a subgroup of women who were followed for two years from the date of delivery. All the potential predictors of dysglycaemia during the two years follow-up period were adjusted in the final model (S1 Table).

Machine learning analysis

Machine Learning (ML) analysis was performed in Python version 3.7 (www.python.org). Lasso regularization was used for feature selection. Nested standard 5-fold cross validation (CV1) was used for model evaluation [17]. An internal stratified 10-fold cross validation (CV2) was performed on each of the five training folds of CV1 for optimizing the shrinkage parameter in lasso (S1 Fig). Missing values were imputed using Multivariate Imputation by Chained Equations (MICE) technique, using other non-missing covariates, separately for the training and testing folds of CV1 to avoid leakage of information from the testing data into the training data. The training folds in CV1 were resampled using adaptive synthetic resampling technique to ensure equal representation of both the binary classes. The resampled training data was normalized.

Logistic regression model was fitted on the training folds in CV1 using the selected features from lasso. The model predictions on each of the test folds in 5 iterations of CV1 were aggregated and the Receiver Operating Characteristic (ROC) curve was plotted for this aggregated set. The area under the ROC curve (AUROC) was used to assess the performance of the method. The concept diagram of this method is illustrated in S2 Fig. After getting assurance of acceptable performance of this method, logistic regression was applied stepwise on the full data to obtain the final model for deployment. Stepwise details of our proposed method are given below:

Step 1: Lasso hyperparameter optimization–Lasso regularization embeds feature selection in the form of an L1 penalty on the magnitude of the feature coefficients, causing irrelevant feature coefficients to shrink to zero. The shrinkage hyperparameter, i.e., the magnitude of the penalty, is optimized using a stratified 10-fold cross validation (CV2) on the training folds of CV1 in each iteration i. This process gives us an optimal hyperparameter value C for each fold i.

Step 2: Feature selection- All baseline features (except history of alcohol in current pregnancy and previous pregnancy GDM history) were considered for feature selection. Converting the categorical features to binary using one-hot-encoding, there were 27 features in total. Lasso with the tuned hyperparameter C from step 1 was used to select the best feature set f from the training data in iteration i.

Step 3: Model training- Logistic regression model m was learned from the features f, selected in step 2, for each fold i of CV1.

Step 4: Model evaluation- The logistic regression model m in step 3 was used to make predictions on the held-out test data of the corresponding fold i of CV1. The predictions on each of the 5 test folds of CV1 were aggregated to plot the ROC and calculate the AUROC.

Finally, Steps 1 to 3 were applied on the full data to obtain the final model for practical use. That is, lasso regularization hyperparameter was optimized using stratified 10-fold cross validation on the full data, features were selected from the full data using lasso with optimized C, and logistic regression model was learned on the selected features of the full data.

Analysis of Specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), F1-score, and Accuracy was performed for five predetermined values of Sensitivity (60%, 70%, 75%, 80%, and 90%) for the optimal selected model. Using the coefficients from the final fitted logistic regression model on the full data, a composite risk scoring system was developed using the best selected antenatal predictors to predict the probability of GDM women to miss ppGT. Composite risk score was calculated from the equation, 1/[1+exp(−b)], where b = b + bx + bx + … + bx where b is the intercept and b coefficient of nth predictor (x), respectively. A Decision Curve Analysis (DCA) was used to evaluate and compare the performance of our model in comparison to ‘target all’ and ‘target none’ approaches [18]. Finally, the correctly identified non-attenders (sensitivity) vs follow-ups avoided (the true negatives + false negatives, obtained from the optimal selected model) were used to calculate the number of women requiring enhanced care, to maximize the postpartum follow-up care.

Results

In total, 607 pregnant women were diagnosed with GDM. Diagnosis of GDM was made at mean gestational week of 27.9±4.4 weeks. 7.4% (n = 45) of women were diagnosed by the FPG alone, 58.3% (n = 354) by 2-hrs glucose alone and 12.4% (n = 75) had abnormal values for both. Further, 21.9% (n = 133) GDM diagnosed women had missing data on antenatal OGTT which were imputed using MICE technique. The prevalence of large for gestational age (LGA) was 13.8% and small for gestational age (SGA) was 14.7%.

Overall, 64.9% (n = 394) GDM women attended the ppGT, including postnatal OGTT (n = 340) and HbA1c (n = 54). Subsequently, 233 women had follow-up glucose testing at least once within 24-months of index delivery. At the immediate ppGT, 21.6% had dysglycaemia/impaired glucose regulation (abnormal impaired fasting glucose (IFG), impaired glucose tolerance (IGT) or HbA1c ≥39 and <48 mmol/mol) and 2.8% had been diagnosed with T2D (Fig 1). Comparison of the anthropometric, clinical, and demographic characteristics of women who did and did not attend the ppGT is shown in Table 1.

Fig 1

Consort diagram of immediate postpartum glucose testing attendance and follow-up for 24-months post-delivery.

The flow chart displayed the proportion of GDM women attended vs not attended ppGT. The diagnosis of dysglycaemia/impaired glucose regulation and Type 2 diabetes was performed as, Normal glycaemia: FPG <5.6 and 2-hr glucose <7.8 at postpartum OGTT or HbA1c <39 mmol/mol (<5.7%); Impaired glucose regulation: IFG, IGT and/or Prediabetes [Impaired Fasting glucose: FPG ≥5.6mmol/L; Impaired Glucose tolerance: 2-hr glucose ≥7.8 mmol/L; Prediabetes: HbA1c ≥39 to <48 mmol/mol (≥5.7 and <6.4%)]; Type 2 diabetes: FPG ≥7.0mmol/L and/or 2-hr ≥11.1mmol/L post 75g OGTT or HbA1c ≥48mmol/mol (≥6.5%).

Table 1

Antenatal, delivery and postnatal characteristics of GDM women with postpartum glucose screening attendance.

Characteristics	All women n = 607	Attended n = 394	Did not attend n = 213	p-value
At booking
Age**	31.59±5.76	32.21±5.40	30.45±6.22	0.001
Parity***	1.98±1.17	1.78±0.98	2.35±1.38	<0.001
Multiparous (≥2)**	339/604 (56.1%)	200/392 (51%)	139/212 (65.6%)	0.001
Weight (Kg)**	81.86±20.51	79.78±19.79	85.58±21.27	0.001
Height (m)	1.64±0.07	1.64±0.07	1.64±0.08	0.613
BMI (Kg/m 2 )**	30.49±7.14	29.76±6.81	31.79±7.53	0.001
• < 18.5	10/557 (1.8%)	6/357 (1.7%)	4/200 (2.0%)	0.004
• 18.5 to 24.9	125/557 (22.4%)	92/357 (25.8%)	33/200 (16.5%)
• 25 to 29.9	141/557 (25.3%)	99/357 (27.7%)	42/200 (21.0%)
• ≥30	281/557 (50.4%)	160/357 (44.8%)	121/200 (60.5%)
Blood Pressure (BP; mmHg)
• Systolic BP	115.8±13.23	115.71±13.62	115.97±12.54	0.827
• Diastolic BP	70.26±9.46	69.98±9.39	70.74±9.57	0.366
Ethnicity
• White European	481/607 (79.2%)	303/394 (76.9%)	178/213 (83.6%)	0.104
• South Asian	67/607 (11%)	46/394 (11.7%)	21/213 (9.9%)
• Other (Black African/ Caribbean or mixed ethnicity)	59/607 (9.7%)	45/394 (11.4%)	14/213 (6.6%)
Smoking in pregnancy***
• Never smoked	270/583 (46.3%)	190/373 (50.9%)	80/210 (38.1%)	<0.001
• Ex-smoker (stopped before or during pregnancy)	216/583 (37.0%)	147/373 (39.4%)	69/210 (32.9%)
• Current smoker	97/583 (16.6%)	36/373 (9.7%)	61/210 (29.0%)
Alcohol in pregnancy
• Never drank	264/567 (46.6%)	160/361 (43.2%)	104/206 (50.5%)	0.098
• Ex-drinker (stopped before or during pregnancy)	288/567 (50.8%)	194/361 (53.7%)	94/206 (45.6%)
• Current drinker	15/567 (2.6%)	7/361 (1.9%)	8/206 (3.9%)
Marital status**
• Single	47/572 (8.2%)	21/366 (5.7%)	26/206 (12.6%)	0.004
Employment status
• Unemployed	18/547 (3.3%)	9/352 (2.6%)	9/195 (4.6%)	0.196
Previous GDM pregnancy	50/607 (8.2%)	26/394 (6.6%)	24/213 (11.3%)	0.545
At OGTT and Intrapartum
Gestational age at diagnosis (weeks)	27.94±4.41	28.16±4.21	27.53±4.74	0.093
Fasting glucose (mmol/l)*	5.02±0.89	4.96± 0.89	5.12±0.89	0.042
2hrs glucose (mmol/l)**	8.41±1.76	8.55±1.75	8.13±1.75	0.008
HbA1c (mmol/mol)	35.74±5.05	35.52±4.69	36.17±5.70	0.154
Gestational age at birth (weeks)	37.92±1.32	37.91±1.27	37.95±1.39	0.711
Preterm (GA <37 weeks)	85/599 (14.2%)	53/390 (13.6%)	32/209 (15.3%)	0.565
Mode of delivery
• Spontaneous	309/600 (51.5%)	197/391 (50.4%)	112/209 (53.6%)	0.688
• Instrument assisted	50/600 (8.3%)	32/391 (8.2%)	18/209 (8.6%)
• Caesarean delivery	241/600 (40.2%)	162/391 (41.4%)	79/209 (37.8%)
Birthweight (gms)	3209.23±490.39	3211.95±467.75	3204.14±531.49	0.853
Birth centiles
• AGA (10-90thcentile)	394/551 (71.5%)	264/356 (74.2%)	130/195 (66.7%)	0.058
• SGA (<10 centile)	81/551 (14.7%)	43/356(12.1%)	38/195 (19.5%)
• LGA (>90 centile)	76/551 (13.8%)	49/356 (13.8%)	27/195 (13.8%)
Male baby n (%)**	307/599 (51.3%)	183/390 (46.9%)	124/209 (59.3%)	0.004
Breastfeeding initiated**	293/545 (53.8%)	207/355 (58.3%)	86/190 (45.3%)	0.004
Timing of ppGT (days)	-	67.63±18.32	-

Values are expressed in mean ± standard deviation and n (%) as appropriate; BW, Birthweight; AGA, Appropriate for gestational age; SGA, Small for gestational age; LGA, Large for gestational age.

Women who did not attend ppGT were younger, unmarried, multiparous, had higher BMI and continued to smoke during pregnancy at the antenatal booking visit compared to those who attended ppGT (Table 1). The factors independently associated with non-attendance of ppGT are shown in Table 2. The factors associated with non-attendance of ppGT, selected in each of the 5 iterations of cross validation are shown in S3 Fig.

Table 2

Factors associated with non-attendance to postpartum screening identified by logistic regression.

Predictors	β (SE)	OR (95% CI)
Intercept	-3.1599 (3.337)	-
Maternal age	-0.0797 (0.019)	0.92 (-0.116, -0.043)
Antenatal fasting glucose	0.1926 (0.145)	1.21 (-0.092, 0.478)
Antenatal 2-hrs Glucose	-0.1415 (0.062)	0.87 (-0.262, -0.021)
Antenatal HbA1c	0.0195 (0.024)	1.02 (-0.028, -0.067)
Gestational age at GDM diagnosis	-0.0410 (0.022)	0.96 (-0.084, 0.002)
Booking BMI	0.0027 (0.016)	1.00 (-0.028, 0.033)
Continuing to smoke at booking	0.8312 (0.258)	2.30 (0.325, 1.338)
Unmarried at booking	-0.8447 (0.345)	0.43 (-1.521, -0.168)
Diastolic BP at booking	0.0116 (0.011)	1.01 (-0.010, 0.034)
Other Ethnicity (Black African/ Caribbean or mixed ethnicity)	-0.2392 (0.351)	0.79 (-0.928, 0.450)
Gestational age at birth	0.1062 (0.076)	1.11 (-0.043, 0.255)
Instrument assisted delivery	0.5630 (0.348)	1.76 (-0.118, 1.244)
Women delivered SGA infants	0.7418 (0.275)	2.10 (0.204, 1.280)
Women delivered male babies	0.4761 (0.194)	1.61 (0.096, 0.856)
Breastfeeding initiation before discharge	-0.2115 (0.210)	0.81 (-0.622, 0.199)
Parity	0.5486 (0.096)	1.73 (0.361, 0.736)

ML: Machine learning; OR: Odds ratio; 95%CI, confidence interval. Logistic regression model was fit using features selected from lasso by machine learning algorithm.

The AUROC computed from aggregating the test predictions from 5 test folds is 0.72 (Fig 2). The optimal threshold was determined as 0.46 (sensitivity of 0.70, specificity of 0.66 and, maximal F1 score). Forty six out of 100 women were above this threshold of 0.46 and focusing on these women could improve the ppGT testing (S4 Fig). Table 3 shows the sensitivity, specificity, PPV, NPV, F1 score and accuracy at other probability thresholds. The F1 graph is shown in S5 Fig.

Fig 2

AUROC for prediction of non-attendance at ppGT.

AUROC was used to evaluate the performance of our machine learning based algorithm using logistic regression model on the validation cohort, n = 607 by aggregating the predictions from the 5 test folds of CV1. The area under ROC was 0.72. The dotted line indicates optimal threshold. The grey line indicates ‘target none’ approach and black line indicates ‘target all’ approach. The blue line indicates the net benefit of the proposed ML prediction model.

Table 3

Sensitivity and specificity of postpartum glucose attendance by ML algorithm at various probability thresholds.

Probability threshold	Sensitivity	Specificity	PPV	NPV	F1	Accuracy	Proportion attended ppGT
0.27	0.90	0.32	0.42	0.85	0.57	0.52	25
0.36	0.80	0.48	0.45	0.82	0.58	0.59	38
0.39	0.75	0.53	0.46	0.80	0.57	0.60	43
0.46*	0.70	0.66	0.52	0.80	0.60	0.67	54
0.53	0.60	0.72	0.54	0.77	0.57	0.68	61

PPV: Positive predictive value; NPV: Negative predictive value;

* Optimal threshold with maximal F1 score that shows sensitivity of 0.70 and specificity of 0.66 to determine the number of GDM women to be focused for postpartum glucose testing.

In the decision curve analysis, by comparing the ‘target all’ and ‘target none’ approaches, ML algorithm obtained higher standardized net benefit as compared to the follow-up of all GDM women for ppGT (Fig 3). Our proposed ML prediction model has the highest benefit across various probability thresholds and specifically identified 46% GDM women who are unlikely to attend ppGT (optimal threshold = 0.464). Based on our proposed final model, the composite risk score, P (non-attendance to ppGT), was calculated as 1/[1+exp(−b)], where b = −3.1599 + (0.1926×antenatal fasting glucose) + (−0.1415×antenatal postprandial glucose) + (0.0195×antenatal HbA1c) + (−0.0410×gestational age at antenatal OGTT) + (−0.0797×maternal age) + (0.0027×booking BMI) + (0.5486×parity) + (−0.8447×married) + (−0.2392×other ethnicity) + (0.8312×current smoker) + (0.0116×diastolic BP) + (0.1062×gestational age at birth) + (0.7418×women delivered SGA infants) + (0.4761×male sex of the baby) + (0.5630×instrumental delivery) + (−0.2115×initiated breastfeeding) (S2 Appendix).

Fig 3

Decision curve analysis for the standardized net benefit obtained from the proposed ML model.

The DCA (Decision curve analysis) showed the net benefit obtained from the ML (blue line) prediction model compared to the target all (solid black line) or target none (solid grey line). Net benefit by implementing our model in a clinical setting is larger when compared to the follow-up of all GDM women for ppGT. DCA was derived from the equation, Net benefit = , where TP and FP are the true positives and false positives respectively, p is the probability threshold, and N is the total number of participants in the validation cohort, N = 607.

During the subsequent follow-up period between 4 and 24 months post-delivery, two hundred and thirty- three GDM women had at least one HbA1c tested value (Fig 1). Among those tested (n = 233 out of 607), additional 24% (n = 56) women had dysglycaemia. Twelve women had converted to T2D between ppGT and 24 months with higher proportion in women who did not attend the immediate ppGT (attended vs non-attended: 2.5 vs. 11.4%; p = 0.005). Survival analysis showed that women from south Asian ethnicity, those with higher booking BMI and antenatal HbA1c had increased hazard ratio for dysglycaemia in both ppGT groups (S1 Table).

Discussion

Our study highlights that unmarried status, younger age, higher BMI, multiparity, and continued smoking during pregnancy are risk factors of poor postpartum attendance for glucose testing, following a GDM pregnancy. Using an unbiased, data-driven machine learning approach, we propose a composite risk score based on easily available antenatal parameters for women who are less likely to attend postpartum screening. Worryingly, it appears that those who did not attend the immediate postpartum screening, have higher conversion rate to T2D within two years of index pregnancy, although the numbers were small.

The attendance to postpartum glucose tolerance screening in our centre has improved since the introduction of individualised letter at the time of discharge in 2015 (64.9% vs 49%) [10]. Although the uptake is similar [12], or better than others [1, 19], it is still not satisfactory. Interestingly, 33% of women who did not attend the immediate testing, subsequently were tested at least once within two years of index pregnancy. This suggests that the message of annual screening to the patient and/or their GP is filtering through, albeit in a small proportion of individuals. However, we did not have the information on how many women were invited for this subsequent testing.

Previous studies have attempted to understand the barriers for poor attendance. Lack of awareness and competing challenges with childcare are some of the common barriers [14]. In addition, younger women may perceive their risk is low [20]. While obese women may understand their risk, they might not attend for other reasons including fear or the stigma of diagnosis of T2D [21]. Antenatal educational interventions highlighting the subsequent risk of T2D and flexible postnatal lifestyle services by incorporating health visitors for glucose testing have been suggested as potential strategies to improve these barriers [13, 22]. Although implementation of a recall register system and/or coordinator institution enhanced the ppGT uptake rate in some countries, our ML based risk stratification could provide a better understanding for personalised education specifically, to capture non-attenders at the time of hospital discharge [11, 12]. While our letter is designed to have individualised approach, and seem to have improved the overall testing, it is possible that some women felt the information provided is standard and is ‘not specific to them’.

Our proposed machine learning based individualised probability threshold identified 46% of women who are unlikely to attend postpartum testing using simple, routinely available characteristics. Healthcare professionals can identify and target these women antenatally and provide enhanced education on the importance of the postpartum glucose testing both during pregnancy and at the time of discharge from the hospital. This approach may also facilitate healthcare professionals’ perception of ppGT by emphasizing the significance of screening that may help women to identify their risk and seek professional care for timely intervention. Alternatively, doing FPG just before discharge could also be carried out in these women, to ensure some form of glucose testing is done [23]. To our knowledge this is the first study to attempt to create an individualised composite risk score for postpartum non-attendance, using a machine learning algorithm. A simple, Microsoft Excel based individualised risk score can be easily calculated by the healthcare professionals to identify the women who is less likely to attend that may benefit from enhanced education (S2 Appendix).

The overall conversion rate to abnormal glucose tolerance in our study was high i.e., 24.4% at the ppGT and an additional 24% in the subsequent 18 months. This is worryingly higher than previous reports [1, 3], and highlights the importance of ensuring testing happens not only in immediate post-delivery, but annually thereafter. Recent evidence also suggests that the cardiovascular risk is higher in these women [4, 22] and, perhaps the annual screening should include other common CVD risk factors such as smoking, blood pressure and lipid profile. Individualised risk calculation for abnormal glucose tolerance similar to the one proposed for non-attendance could potentially improve attendance, utilize the resources effectively and enable targeted education for prevention. As our composite risk score can be calculated in a simple Excel based calculator (S2 Appendix) our approach can also be easily implemented in low resource settings.

Our study had two key strengths: follow-up data on women who did not attend the immediate post-partum testing and machine learning based individualised composite risk score for non-attendance. However, we note that although this work included all the women who were diagnosed to have GDM in the study period, this was a retrospective study. We therefore were limited by the routinely collected data. In addition to some missing data, we did not have information about treatment modalities, which has been shown to be associated with postpartum screening uptake, albeit with mixed results [24-26].

Other factors could conceivably be relevant in predicting women’s participation in follow-up, for example, proximity of the healthcare centre, their employment status and the nature of their employment, flexibility in having an appointment. However, in this study, all participants were proximal to the hospital, had their antenatal care and delivery in the hospital, women were able to reschedule and pick appointment date/time that is suitable for them. We also found no statistical difference between employed and unemployed women. But these and other socioeconomic parameters may prove relevant in larger studies.

While this information might have improved the predictive power of the ML algorithm, we believe that using our model it is easy to incorporate additional maternal characteristics based on their availability. We also did not have information on how many women were invited for the subsequent annual testing. Finally, while we are confident about the beneficial role of advanced nurse practitioner for the improvement in our postpartum testing, we were unable to ascertain the role of the standard letter sent to the women.

Conclusion

Our study highlights that a simple machine learning algorithm can accurately identify women who are unlikely to attend postpartum testing, based on routinely collected clinical parameters. Such knowledge can enable healthcare professionals to provide enhanced education throughout the antenatal period, at the time of discharge, and during the immediate postpartum period using health visitors. However, this will require an RCT to confirm the usefulness of ML based risk prediction in the evaluation of postpartum glucose screening uptake rate. In addition, with the improved accuracy of point of care HbA1c testing kits, this can also be utilized to improve the testing and identification of abnormal glucose tolerance [27]. The cost effectiveness of such strategies will also require well designed prospective studies.

Mean CV-accuracy as a function of lasso regularization hyperparameter C.

S1 Fig shows the variation in the mean stratified 10-fold cross validation accuracy as a function of the lasso regularization hyperparameter C for the final model. Maximum CV-accuracy of 0.7065 is obtained for C = 0.1325.

(TIF)

Click here for additional data file.

Concept diagram of the proposed ML algorithm.

The processed data is divided into 5 folds. The light grey region shows the training data, and the dark grey shows the testing data. Within each iteration i of the outer 5-fold CV, the training folds further undergo internal 10-fold CV in Step 1 for lasso hyperparameter optimization. This is known as nested cross validation. In step 2, feature selection is performed on the training folds in iteration i using lasso with optimized regularization hyperparameter. Logistic regression model with selected features is fit on the training folds in iteration i in Step 3. The fit model is used for prediction on the exclusively held out test data in iteration i. The test predictions from all 5 iterations are aggregated to plot and calculate the area under the ROC curve for evaluating the performance of our method.

(TIF)

Click here for additional data file.

Feature selection using lasso regularization in each of the 5 iterations of CV1 and the final model.

S3 Fig gives a visual understanding of the features selected in each iteration of CV1 and as well as in the final model. Pink cells show the selected features in CV1, green cells show the selected features in the final model built on the full data and red cells show the feature coefficients shrank to zero in both CV1 and the final model. The sets of features selected in all folds of CV1 (except fold 3, where all features except two are selected) and those selected in the final model are similar. The number of samples in the training and testing folds, value of the optimized hyperparameter C, number of features selected and area under the ROC curve for the test fold for each iteration i of CV1 as well as for the final model are provided below each respective column.

(TIF)

Click here for additional data file.

Comparison of correct prediction of non-attendance versus follow-ups avoided for different probability thresholds using ML proposed model.

S4 Fig shows the comparison of correct prediction of non-attendance versus follow-ups avoided for different probability thresholds using our proposed model. Based on our proposed final model, the composite risk score, P(non-attendance), is calculated as 1/1+exp(-b), where b = -3.1599 + (0.1926*antenatal fasting glucose) + (-0.1415*antenatal postprandial glucose) + (0.0195*antenatal HbA1c) + (-0.0410*gestational age at antenatal OGTT) + (-0.0797*maternal age) + (0.0027*booking BMI) + (0.5486*parity) + (-0.8447*married) + (-0.2392*other ethnicity) + (0.8312*current smoker) + (0.0116*diastolic BP) + (0.1062*gestational age at birth) + (0.7418*women delivered SGA infants) + (0.4761*male sex of the baby) + (0.5630*instrumental delivery) + (-0.2115*initiated breastfeeding). SGA baby, smoker, marital status, male sex of baby, and other ethnicity are binary variables, taking value of either 0 or 1 depending on their absence or presence respectively.

(TIF)

Click here for additional data file.

F1 graph of optimal probability threshold.

F1 graph showing the model sensitivity and specificity at variable thresholds.

(TIF)

Click here for additional data file.

Cox proportional hazard ratio for dysglycaemia within 2 years of index pregnancy.

(DOCX)

Click here for additional data file.

Standard letter format for postpartum screening to all GDM women issued from the Diabetes clinic, GEH-NHS Trust, UK.

(PDF)

Click here for additional data file.

Microsoft Excel based risk stratification of ppGT non-attendance in GDM women.

(XLSX)

Click here for additional data file.

Supporting information file consisting minimal anonymized dataset of postpartum attendance in GDM women.

Full dataset is available on request following completion of suitable confidentiality agreement.

(CSV)

Click here for additional data file.

15 Dec 2021

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Rajakumar Anbazhagan, Ph. D

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Thank you for stating in your Funding Statement:

[PS and YW are part funded by Medical Research Council, UK Grant number:  MR/R020981/1 Funder: MRC- UK URL: https://mrc.ukri.org/funding/

NP is funded by Chancellor's international Scholarship for doctoral research.

Funder: University of Warwick

URL: https://warwick.ac.uk/services/dc/schols_fund/scholarships_and_funding/chancellors_int/

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.].

Please provide an amended statement that declares *all* the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now.  Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement.

Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf.

3. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere. [A subset of findings were presented as abstract at the 56th Annual Meeting of the European Association for the study of Diabetes (EASD) in 21-25th September 2020. No data tables and/or figures from this manuscript were published or presented elsewhere.] Please clarify whether this conference proceeding was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

4. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: I Don't Know

Reviewer #4: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript authored by Nishanthi Periyathambi et al aimed to identify the factors associated with non attendance of postpartum glucose test by developing a machine leaning algorithm using available information on a cohort of gestational diabetes mellitus women (607). Their algorithm was able to predict which individuals are unlikely to attend postpartum glucose testing. The study is particularly interesting because it gives an opportunity to the health care system to adapt their strategy to improve the follow up of these individuals using simple tools.

The manuscript is well written and the data are supporting the conclusion. There is minor comments to improve the paper:

1/ It would be very helpful to further extend the discussion to include what other factors the current model is not taking in consideration and which may be very interesting to improve the accuracy of the prediction (My first thought goes to the proximity to a testing center, the flexibility in having an appointment, the nature of jobs that these individuals have...etc . How this can be addressed in future studies,...etc.

2/ The font is not homogenous in the discussion section and need to be adjusted.

Reviewer #2: In this manuscript by Nishanthi Periyathambi et al., describe factors who did not attend gestational type 2 diabetes (T2D) screening in the postpartum period using machine learning algorithms. This study is aimed to identify patient characteristics who did not attend the immediate ppGT and assess their subsequent T2D risk. Authors built a predictive model using machine learning algorithms and proposed factors associated with non-attendance of immediate postpartum glucose test. Further authors think, improved personalised education may improve postpartum glucose screening. Overall factors associated with the gestational T2D, are interesting. I have a few questions on this study, authors have to address these following questions.

Major comments

- Authors should highlight significant values in tables with asterisk marks or underline etc., for younger, unmarried, multiparous, BMI and smoking during pregnancy.

- Authors should describe the details of model analysis in the methods section. Readers benefited by including the S1 file which mentioned "Stepwise 1 description of the Machine learning" into methods section.

- Results sections of the study were not well explained, what is the concept diagram used in this study, how does it generate helpful outcomes of the figure S2 for next steps. The area under ROC was 0.72. How does this value help to decide the factors associated with non attendance of glucose tests? The outcome results of decision curve analysis is not explained in detail. How does decision curve analysis can be used to find factors or glucose levels in women who did not attend gestational type 2 diabetes (T2D) screening in the postpartum period.

- Authors should provide any previous examples or references for the machine learning (ML) used in this study. How does this ML help in the example disease predictive models?

- Authors mention in line 148 “After getting assurance of acceptable performance of this method” how does the assurance of method was confirmed by authors in this study, please explain features that help to get confidence of the model.

Minor comments

- Authors should report how hospitals can be improved the personalised education for postpartum glucose screening using this study, whether this can be applied to hospitals in many countries or specific to a country. How it can be improved could be described in discussion or if any country has better screening for gestational type 2 diabetes (T2D) in the postpartum period, additional details of these elements could be helpful to improve the study or personalised education.

- Add a short form for the area under the ROC as AUROC for figure2 legend or results section describe figure2.

- Label x and y axis in fig S5.

Reviewer #3: Reviewer’s comment

In the study titled ‘Machine learning prediction of nonattendance to postpartum glucose screening and subsequent risk of type 2 diabetes following gestational diabetes,’ the authors Periyathambi et al. have attempted to find the characteristics/habits of the women who did not attend postpartum glucose testing (ppGT) despite the reminder from the hospital staff. This is a retrospective cohort study of all 607 GDM women who were ppGT due between January 2016 and December 2019 at the George Eliot Hospital NHS Trust, UK. This study indicates that the women who did not attend ppGT were younger, multiparous, obese, and continued smoking during pregnancy. Most of these characteristics of individuals are comparable with their previous study (Venkararaman et al., 2015). The authors have used machine learning (Python version 3.7) for predicting the type of women who were more likely to have nonattendance for ppGT. This study has many limitations such as small sample size, uniformity in reminding the individuals, etc. However, this preliminary study can encourage researchers to do a study on a larger sample size in the future. Following are a few suggestions that might improve the article.

1. The authors may explain how different machine learning results are compared to other analyses such as manual analysis.

2. The in-person consultation with hospital staff at the time of discharging from the hospital after delivery can play an important role in whether the woman will attend ppGT or not. Also, the amount of importance given for the consultation may vary among hospital staff.

3. The authors may consider including a future direction such as ‘A study with larger sample size would give better clarity on machine learning for nonattendance for ppGT.

Reviewer #4: The manuscript by Periyathambi et al, titled " Machine learning prediction of non-attendance to postpartum glucose screening and subsequent risk of type 2 diabetes following gestational diabetes" studies the social and physiological factors associated with non-attendance of immediate postpartum glucose test using machine learning algorithm in women with gestational diabetes mellitus.

The study also presents correlation between non-attendance and higher risk of conversion to type 2 diabetes mellitus.

The study is well designed and executed and the import of the report is evident in its findings. However, I would like the authors to discuss the effectiveness of their machine learning model and its predictive power in a different socio-economic setting – for example in under-developed or developing societies. This would certainly enhance the quality and relevance of the manuscript.

Overall, the authors did an excellent job in organizing their workflow, and the manuscript is easily understandable even for non-expert readers. I applaud the authors for their commendable effort.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Anil Mathew Tharappel

Reviewer #4: Yes: Madhurima Dhara

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Reviewers comment.docx

Click here for additional data file.

Submitted filename: Review_Periyathambi et al_PONE-D-21-35154.docx

Click here for additional data file.

7 Feb 2022

PONE-D-21-35154

Machine learning prediction of non-attendance to postpartum glucose screening and subsequent risk of type 2 diabetes following gestational diabetes

Dear Reviewers,

REVIEWER 1 COMMENTS:

1. It would be very helpful to further extend the discussion to include what other factors the current model is not taking in consideration, and which may be very interesting to improve the accuracy of the prediction (My first thought goes to the proximity to a testing center, the flexibility in having an appointment, the nature of jobs that these individuals have...etc. How this can be addressed in future studies, etc.

Response:

We thank the reviewers for this important point.

· We agree that factors such as access to nearest healthcare centre and travel distance could impact women’s participation in screening (Cullinan et al, 2011). However, all the study participants in our cohort had easy access to the testing centre (the catchment area is small, less than 5 miles for those who live in the city and easy/free parking at the hospital). They all are given the option to choose their appointments too if the offered appointment date/time is not suitable. We acknowledge that our anonymous data collection process could not incorporate otherwise identifiable records, i.e personal address and postcode.

· We have supplemented the S1 Appendix section of the invite that implies flexibility to have an appointment at the client’s earliest availability. In addition, all the participants had telephone reminders regarding their appointment booking by one of the specialist nurse practitioners where it provides an opportunity for the women to amend their screening date and/or time.

· Although we found higher proportion of employed women (64.8%) attending the postpartum glucose testing compared to unemployed women (50%), it was not statistically significant. We have also incorporated our clarification in discussion in page 15, lines 335:341. We agree further research on the barriers and facilitators of postpartum glucose testing is needed to provide additional insights.

2. The font is not homogenous in the discussion section and need to be adjusted.

Response:

We have addressed this comment by modifying the font size to 11pts in page, 11:12 and a spacing in line 318 on page 14.

REVIEWER 2 COMMENTS:

3. Authors should highlight significant values in tables with asterisk marks or underline etc., for younger, unmarried, multiparous, BMI and smoking during pregnancy.

Response:

Thank you for this suggestion. We have incorporated this comment by including a separate column for p-values and asterisk marks in the ‘Characteristics’ column to show significance in Table.1 in ‘Revised Manuscript with Track Changes’.

4. Authors should describe the details of model analysis in the methods section. Readers benefited by including the S1 file which mentioned "Stepwise 1 description of the Machine learning" into methods section.

Response:

We agree with your comments and have incorporated this suggestion in the Method section in page 6:7, lines 150:168.

5. a) Results sections of the study were not well explained, what is the concept diagram used in this study, how does it generate helpful outcomes of the Figure S2 for next steps.

b) The area under ROC was 0.72. How does this value help to decide the factors associated with non-attendance of glucose tests?

c) The outcome results of decision curve analysis is not explained in detail.

d) How does decision curve analysis can be used to find factors or glucose levels in women who did not attend gestational type 2 diabetes (T2D) screening in the postpartum period.

Response:

a) We would like to clarify how the concept diagram used in this study as follows: The processed data is divided into 5 folds. The light grey region shows the training data, and the dark grey shows the testing data. Within each iteration i of the outer 5-fold CV, the training folds further undergo internal 10-fold CV in Step 1 for lasso hyperparameter optimization. This is known as nested cross validation. In step 2, feature selection is performed on the training folds in iteration i using lasso with optimized regularization hyperparameter. Logistic regression model with selected features is fit on the training folds in iteration i in Step 3. The fit model is used for prediction on the exclusively held out test data in iteration i. The test predictions from all 5 iterations are aggregated to plot and calculate the area under the ROC curve for evaluating the performance of our method. We have updated the figure caption to explain this.

b) A non-informative classifier which picks the label at random, will have a 0.5 area under the ROC (AUROC), while a perfect one will have an AUROC of 1.0. A value of 0.72 on an unseen (test) dataset denotes that the classifier is informative, although not perfect. Different classifiers will result in different areas, the 0.72 was the best possible area among the methods we tried. The factors that give the highest area under the curve on an unseen dataset are listed as the most prominent factors associated with non-attendance of glucose tests.

c) We have redrafted the result section by incorporating decision curve analysis (DCA) outcome with the optimal threshold (0.464) to target non-attenders based on our prediction model in page 12, lines 248: 250 to be more in line with your comments. The DCA has higher net benefit than the conventional strategy to focus all the women or targeting none which would lead to improved glucose screening rate in any clinical settings. We have also updated the Fig.3 legend to show the DCA comparisons between the prediction model vs ‘target all’ or ‘target none’ lines in page 13, lines 260:261.

d) The DCA evaluated our prediction model which encompasses all the variables that were used to construct the final model (antenatal OGTT values, i.e, fasting glucose and 2-hrs glucose and gestational age at the time of screening, antenatal HbA1c, age, booking BMI, parity, marital status, ethnicity, smoking status, booking BP, gestational age at birth, infant birth size and sex, type of delivery and breastfeeding initiation assessed before the hospital discharge). Therefore, our DCA has overcome the limitation of discriminating women who would not have been screened for gestational diabetes and otherwise targeted via other predictive factors. We would like to further address your comments by highlighting the S2 Appendix, an Excel based calculator which shows the combination of all the predictive factors to identify women who are less likely to attend the ppGT. In simple terms, DCA was not implemented to find factors associated with non-attendance; DCA helps to understand that the classifier built with the chosen factors (corresponding to highest AUROC) is more beneficial than follow-up of all GDM women assuming none will attend.

6. Authors should provide any previous examples or references for the machine learning (ML) used in this study. How does this ML help in the example disease predictive models?

Response:

Thank you for your suggestion. We have included a reference for machine learning approach to predict a disease (Type 2 Diabetes after GDM pregnancy) in page, 6, line number 134. However, while the basic methods such as logistic regression and lasso are standard, the detailed framework as presented here is our own, and approaches like nested cross validation to find optimal hyper-parameters have not been used before in such studies, to the best of our knowledge.

7. Authors mention in line 148 “After getting assurance of acceptable performance of this method” how does the assurance of method was confirmed by authors in this study, please explain features that help to get confidence of the model.

Response:

The term “acceptable performance” refers to the AUROC of 0.72 over the unseen data. A value consistently higher than 0.7 i.e., better than the 0.5 a random classifier would achieve assures us of its performance. Along with AUROC, high sensitivity as well as specificity together help to get confidence from the model.

8. Authors should report how hospitals can be improved the personalised education for postpartum glucose screening using this study, whether this can be applied to hospitals in many countries or specific to a country. How it can be improved could be described in discussion or if any country has better screening for gestational type 2 diabetes (T2D) in the postpartum period, additional details of these elements could be helpful to improve the study or personalised education.

Response:

We have elaborated the usefulness of our ML predictive model in better understanding of personalised education that could improve the ppGT uptake rate in page 14, lines 298:301. We hope these revisions provide a more balanced discussion on the implementation of individualised education in addition to the reminder or recall systems described in few previous studies, Benhalima et al (2017) and Carmody et al (2015).

9. Add a short form for the area under the ROC as AUROC for figure2 legend or results section describe figure2.

Response:

We have addressed your suggestion in page 6 in line 145 and in page 11 in line 225 to rewrite the short form of area under the ROC.

10. Label x and y axis in fig S5.

Response:

We have addressed this comment by including ‘x’ and ‘y’ axis label as ‘Precision’ and ‘Recall’ for the S5 Fig.

REVIEWER 3 COMMENTS:

11. The authors may explain how different machine learning results are compared to other analyses such as manual analysis.

Response:

We are unsure of what the reviewer means by manual analysis. We assume the reviewer means standard statistical analysis. ML methods used here are of the predictive kind, where the eventual goal is to predict the class of a new case (not seen thus far by the model). Statistical analyses will identify differences across individual predictors, between the classes (the p-value listed in Table.1), but do not give a score or a prediction based on them. We note that the final formula learned by the method, that does the best among all methods we tried, can in fact be applied using a simple calculator (as provided in the S2 Appendix) by the healthcare professional at his/her own clinic.

12. The in-person consultation with hospital staff at the time of discharging from the hospital after delivery can play an important role in whether the woman will attend ppGT or not. Also, the amount of importance given for the consultation may vary among hospital staff.

Response:

We agree with you and have incorporated this suggestion in page 14, lines 309:311.

13. The authors may consider including a future direction such as ‘A study with larger sample size would give better clarity on machine learning for nonattendance for ppGT.

Response:

Thank you for your suggestion and we have revised the text in page 15, lines 353:355 to address the usefulness of ML prediction model.

REVIEWER 4 COMMENTS:

14. The study is well designed and executed, and the import of the report is evident in its findings. However, I would like the authors to discuss the effectiveness of their machine learning model and its predictive power in a different socio-economic setting – for example in under-developed or developing societies. This would certainly enhance the quality and relevance of the manuscript.

Response:

We have incorporated your comments in page 14:15, lines 325:327 to reflect the easily identifiable factors for the evaluation of non-attenders for postpartum screening. Our Excel based calculator- S2 Appendix could be used in any low resource setting to target women who are less likely to turn up for the postpartum screening.

Submitted filename: Response to Reviewers_Final.docx

Click here for additional data file.

15 Feb 2022

Machine learning prediction of non-attendance to postpartum glucose screening and subsequent risk of type 2 diabetes following gestational diabetes

PONE-D-21-35154R1

Dear Dr. Saravanan,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: No

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: All the comments were addressed by the authors.

Reviewer #3: In the revised article, titled 'Machine learning prediction of non-attendance to postpartum glucose screening and

subsequent risk of type 2 diabetes following gestational diabetes' the authors response and changes made are satisfactory.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

24 Feb 2022

PONE-D-21-35154R1

Machine learning prediction of non-attendance to postpartum glucose screening and subsequent risk of type 2 diabetes following gestational diabetes

Dear Dr. Saravanan:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Rajakumar Anbazhagan

Academic Editor

PLOS ONE