Quantitative fibronectin to help decision-making in women with symptoms of preterm labour (QUIDS) part 1: Individual participant data meta-analysis and health economic analysis.

INTRODUCTION: The aim of the QUIDS study is to develop a decision support tool for the management of women with symptoms and signs of preterm labour, based on a validated prognostic model using quantitative fetal fibronectin (qfFN) concentration, in combination with clinical risk factors. METHODS AND ANALYSIS: The study will evaluate the Rapid fFN 10Q System (Hologic, Marlborough, Massachusetts) which quantifies fFN in a vaginal swab. In part 1 of the study, we will develop and internally validate a prognostic model using an individual participant data (IPD) meta-analysis of existing studies containing women with symptoms of preterm labour alongside fFN measurements and pregnancy outcome. An economic analysis will be undertaken to assess potential cost-effectiveness of the qfFN prognostic model. The primary endpoint will be the ability of the prognostic model to rule out spontaneous preterm birth within 7 days. Six eligible studies were identified by systematic review of the literature and five agreed to provide their IPD (n=5 studies, 1783 women and 139 events of preterm delivery within 7 days of testing). ETHICS AND DISSEMINATION: The study is funded by the National Institute of Healthcare Research Health Technology Assessment (HTA 14/32/01). It has been approved by the West of Scotland Research Ethics Committee (16/WS/0068). PROSPERO REGISTRATION NUMBER: CRD42015027590. VERSION: Protocol version 2, date 1 November 2016. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Development of prognostic model and for validation in a separate prospective cohort study.

Health economic analysis to determine cost-effectiveness from National Health Service perspective.

Not a randomised control trial to test effectiveness of the model on improved patient outcomes.

Introduction

The overall aim of the QUIDS study is to develop a decision support tool for the management of women with symptoms and signs of preterm labour, based on a validated prognostic model using quantitative fetal fibronectin (qfFN) testing. The study has been conceptually divided into two parts. In this, the protocol for QUIDS part 1, we detail the protocol for development and internal validation of the prognostic model. In the protocol for QUIDS part 2, we detail the protocol for the prospective cohort for external validation of the prognostic model and acceptability testing.1

Preterm delivery (before 37 weeks) occurs in 7.1% of pregnancies in the UK (>50 000 deliveries per annum), with the majority the result of preterm labour.2 3 It remains the leading cause of neonatal morbidity and mortality, but timely interventions, such as antenatal steroids to promote lung maturity, magnesium sulfate for neuroprotection and delivery in a unit with appropriate neonatal care facilities, can improve neonatal outcome. Establishing a diagnosis of preterm labour is, however, difficult. Clinical signs are non-specific and false-positive diagnoses are common, with up to 80% of women with signs and symptoms of preterm labour remaining pregnant after 7 days.4 5 Such diagnostic uncertainty means a large proportion of women with symptoms of preterm labour are treated unnecessarily to ensure benefits to the small proportion of babies that do actually deliver preterm.

It is understandable that both clinicians and pregnant women may prefer a ‘treat-all’ approach in women with symptoms of preterm labour, particularly in a setting remote from an appropriate neonatal unit; and in order to ensure steroid prophylaxis in case preterm delivery occurs. However, unnecessary interventions result in both a substantial economic burden to health services and in potential adverse maternal and neonatal events. Hospital admission and interhospital transfer have considerable cost implications and can be associated with enormous problems for women and their families due to physical separation and emotional stress.6 7 Neonatal cots become ‘blocked’ in order to accept a preterm baby just in case delivery occurs; negatively impacting the efficiency of already stretched neonatal units and networks. This frequently has knock-on effects to other women and babies, who may need transfer to another unit due to lack of cot availability despite an empty, but ‘blocked’, cot. It also may increase the number of ex utero transfers, which are associated with poorer outcomes than in utero transfers.8 If preterm labour has been wrongly diagnosed, and delivery does not occur, steroids may also have adverse long-term consequences for the baby, especially if multiple courses are given.9 Tocolytic therapy, even when appropriate, can have serious side effects for both mother and baby.10 Lastly, uncertainty of outcome may contribute to the high anxiety scores seen in women with threatened preterm labour and their partners.11

Diagnostic tests for preterm labour are available and used in many units in the UK. fFN (Hologic, Marlborough, Massachusetts, USA) is a biochemical marker of preterm labour that can be measured in samples of cervicovaginal secretions collected at a speculum examination. It has potential to help improve diagnosis of impending preterm delivery.12 Other biochemical tests available include Actim Partus (Medixbiochemica, Espoo, Finland), which measures phosphorylated insulin-like growth factor binding protein-1, and PartoSure (Parsagen Diagnostics, Boston, Massachusetts, USA), which measures placental alpha microglobulin-1. An alternative approach (which can be combined with fFN) is to measure the cervical length using transvaginal ultrasound, as the longer the cervix is, the less likely a preterm delivery.12

As part of a National Institute of Healthcare Research Health Technology Assessment (NIHR HTA) report, Honest et al found that a qualitative fFN test (giving a positive or negative result based on a single threshold of 50 ng/mL) was potentially useful in the prediction of preterm delivery <34 weeks gestation, with its main benefit relating to its high negative predictive value, that is, its ability to rule out impending delivery.12 A more recent HTA-funded review found that qualitative fFN testing has moderate accuracy for predicting preterm birth with overall sensitivity and specificity estimates of 76.7% and 82.7% for delivery within 7–10 days.13 These estimates suggest that qualitative testing on its own would not have the sensitivity to rule out preterm delivery adequately, although in systematic review of clinical trials, no increase in neonatal morbidity or mortality was seen in association with false-negative fFN results.13 The authors concluded that this observation is likely to relate to the multifactorial nature of assessment of the risk of preterm delivery, where, in practice, fFN is just one component of the clinical assessment on which management decisions are based.13

Both HTA reviews described above examined the performance of a qualitative fFN test, which provided a positive or negative result on the basis of a single threshold of 50 ng/mL. Recently, this test has been replaced in the UK with the Rapid fFN 10Q System, which provides a concentration of fFN within 10 min, and thus may be a more useful predictor of preterm delivery (qfFN). We surveyed current practice in UK maternity units (response rate 66% (137/207); March–July 2014).14 135/137 units (98.5%) use some sort of diagnostic test of preterm labour. The most common test is fFN (84/137 units; 61.3%). fFN is now only available with a quantitative analyser in the UK, but there is no consensus as to which women to use the test in, or how to interpret the results. Developing and evaluating a decision support for qfFN is thus likely to improve decision-making, even if qfFN is already available in clinical practice. Evidence about the potential value of the new qfFN is required, along with guidance about how to interpret results. The QUIDS study will address this evidence gap.

Methods and analysis

Aims and methodologies

The aim of the QUIDS study is to develop a decision support tool for the management of women with symptoms and signs of preterm labour, based on a validated prognostic model using qfFN testing.

The study protocol has been divided into two parts (see flow chart in figure 1). The protocols for parts 1 and 2 are reported in separate manuscripts.

Figure 1

Flow chart illustrating the design of QUIDS study and conceptual division into part 1 and part 2.1 qfFN, quantitative fetal fibronectin.

Part 1: Development and internal validation of prognostic model

(1) Individual participant data (IPD) meta-analysis to develop a prognostic model using qfFN and other risk (prognostic) factors and to evaluate the added value of qfFN towards this prognostic model performance. A prognostic model will be developed and internally validated15 16 based on a meta-analysis of IPD from existing prospective cohort studies where qfFN results and pregnancy outcome details are available. The primary outcome will be delivery within 7 days, although other endpoints will be included if recommended by focus groups.

(2)Economic analysis: To provide an economic rationale for the prognostic model and analyse its cost-effectiveness from the perspective of the National Health Service (NHS) to provide an economic rationale for the prognostic model and the risk factors included in it.

Part 2: Validation and refinement of prognostic model

Part 2 involves a prospective cohort study and acceptability testing, with external validation (and, if necessary, refinement) of the prognostic model, and update of health economic model.1

Endpoints

The primary endpoint is spontaneous preterm delivery within 7 days of qfFN test, in women tested at less than 36 weeks gestation. This is both an important endpoint for women and caregivers (determined in QUIDS Qualitative study—a preceding qualitative study to identify the decisional needs of women, their partners and clinicians; online supplementary material) as well as a clinically important endpoint. Antenatal steroids (which significantly reduce morbidity and mortality in preterm babies17) are most effective if delivery occurs within 7 days of administration. As repeated doses of antenatal steroids may be harmful, it is crucial to ensure steroids are timed correctly.

A secondary endpoint suggested by the preceding QUIDS Qualitative study consultation (online supplementary material) was delivery within 48 hours of qfFN test. This analysis will be performed if feasible to do so within the constraints of the data available for model development.

Health technologies being assessed

The study will evaluate the Rapid fFN 10Q System (Hologic), which provides a concentration of fFN (ng/mL or invalid) in a vaginal swab sample. Further details about the system and recommended sampling technique are provided in the QUIDS Protocol part 2.1

Target population

The target population is pregnant women attending hospital with signs and symptoms of preterm labour.

How patients are involved in this study

Patient representatives were consulted during the protocol development and have been invited to join the project management group (PMG) and the trial steering committee (TSC). Prior to commencing QUIDS, we performed a qualitative study to determine the decisional needs of pregnant women with signs and symptoms of preterm labour, their partners and their caregivers. This is described in the separate protocol ‘QUIDS Qualitative’ (online supplementary material). The end product of QUIDS will be a decision support aid to help clinicians, women and their partners decide on management of threatened preterm labour, based on the results of the qfFN. In QUIDS Qualitative women and clinicians indicated that they would prefer this to be on web-based or mobile app-based format, presenting the risk of preterm birth within 7 days of testing.

Development of prognostic model

IPD meta-analysis

The proposed IPD meta-analysis was registered on International Prospective Register of Systematic Reviews (CRD42015027590). Our IPD meta-analytical approach will follow existing guidelines, and our output will comply with TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) statement.18

Inclusion criteria

We prespecified inclusion of prospective cohort studies or randomised controlled trials of women with signs and symptom of preterm labour (as defined by investigators) that include qfFN results determined by Rapid fFN 10Q System and pregnancy outcome data; and the principal investigator (PI) of which has agreed to collaborate and provide data.

Exclusion criteria

We will exclude studies where fFN concentration was measured by ELISA and studies where IPD is not available for meta-analysis.

Search strategy

When applying for funding for this study (April 2014), we performed a literature search for completed and ongoing cohort studies of qfFN using search terms for quantitative fetal/foetal fibronectin and preterm birth, including databases (MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects and HTA Database) and clinical trial registries (Cochrane Central Register of Controlled Trials, ClinicalTrials.gov), general search engines (such as Google: https://www.google.co.uk) and systematic reviews. We also consulted preterm birth researchers and networks Royal College of Obstetricains and Gynaecologists Clinical Study Groups (RCOG CSG) British Maternal Fetal Medicine Society (BMFMS), Preterm Brith International Collaborative (PREBIC) and the manufacturers of qfFN, (Hologic) to help ensure capture of all relevant studies.

Study manuscripts and/or protocols were screened by two researchers. We identified a total of 10 studies of qfFN that were potentially eligible. Four early datasets (in three manuscripts) used ELISA to determine the concentration of fFN and were excluded as the different method of analysis and earlier period of study would increase heterogeneity.5 19 20 Therefore, six studies fulfilled the eligibility criteria (see table 1).

Table 1

Details of studies contributing data to IPD meta-analysis

	PI	Setting	N	Events	Dates	Inclusion	Primary outcome
Studies with data available
EQUIPP30 31	Professor AS	5 UK centres	452	14	2010–2012	22–35 weeks with symptoms of preterm labour	Delivery <34 weeks gestation
EUFIS*32	Professor BM	10 European hospitals	452	48	2012–2014	24–34 weeks with preterm contractions and intact membranes	Delivery within 7 days of test
APOSTEL I*33	van Baaren	10 Dutch hospitals	528	70	2009–2012	24–34 weeks with preterm contractions and intact membranes	Days to delivery truncated at 7 days
QFCAPS  (unpublished)	Dr AK	London teaching hospital	86	2	2012–2014	24–34 weeks with symptoms of preterm labour Singletons only	Delivery within 7 days of test
UCLH/Whit  (unpublished)	Dr ALD	2 UK centres	262	5	2009–2010	22–35 weeks with symptoms of preterm labour	Delivery within 7 days of test
	Total	5 studies	1783	139
Studies where data may be available in future
STOP study  (https://clinicaltrials.gov/ct2/show/NCT01868308)	Professor M Elovitz	USA teaching hospital	700	NK	2011–2015	22–34 weeks Symptomatic women with singleton pregnancy	Delivery before 37 weeks

*Study unpublished at time of search in April 2014; manuscript now published.

APOSTEL 1, Alleviation of Pregnancy Outcome by Suspending of Tocolysis in Early Labour; EQUIPP, Evaluation of Fetal Fibronectin with a novel bedside Quantitative Instrument for the Prediction of Preterm birth; EUFIS, European Fibronectin Study; IPD, individual participant data; NK, not known; PI, principal investigator; QFCAPS, Quantitative fetalfibronectin, Cervical length and ActimPartus for the prediction of Preterm birth in Symptomatic women; STOP, Screening to Obviate Preterm Birth; UCLH/WHIT, Univesity College London Hospital/Whittington Hospital Study.

Establishment of the qfFN IPD collaboration

We contacted the PIs of the six eligible studies of qfFN and invited them to participate (see table 1). Five of these agreed to provide their IPD as evidenced by their involvement as coapplicants on the funding application and/or coauthorship of this protocol (BM, van Baaren, AK, AS, ALD). The PI of the sixth study (Elovitz) indicated IPD may be available after publication of her study.

The five included studies (table 1) are European studies of women with symptoms of preterm labour, comprising 1783 women and 139 events of preterm delivery within 7 days of testing. They are from consultant-led maternity units in the UK (three studies) and Europe (two studies). All women in the included trials provided informed consent for participation in clinical trials, and for their IPD to be used in subsequent analyses.

Study quality assessment and data collection

IPD will be stored in a bespoke database on a secure server at the University of Edinburgh. PIs will be asked to provide de-identified data, and consider all recorded variables (even if not reported publications). We will assess study quality according to Quality assessment of diagnostic accuracy studies (QUADAS-2),21 Quality in Prognostic Studies (QUIPS)22 and Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS)23 guidelines.

Sample size considerations

The size of the IPD meta-analysis is limited by the number of studies with data available (table 1). In model development the number of covariates that can be considered is limited by the number of events, with guidance suggesting at least 10 events required for each covariate.24 25 In our IPD meta-analysis data, we have 139 events (preterm labour within 7 days of testing) and therefore deemed that it was sensible to evaluate qfFN and up to 13 other factors (covariates) for potential inclusion in our model.

Data items

The following factors which are thought to influence risk of spontaneous preterm birth will be requested and considered for inclusion as covariates in the prognostic model: qfFN concentration, previous spontaneous preterm labour, gestation at fFN test, age, ethnicity, body mass index, smoking, deprivation index, number of uterine contractions in set time period, cervical dilatation, vaginal bleeding, previous cervical treatment for cervical intraepithelial neoplasia, cervical length (measured by transvaginal cervical length), singleton/multiple pregnancy, tocolysis and fetal sex. Up to 13 of these will be prespecified for inclusion, based on available data (we will only use variables which are available in each study), and ranking for likely clinical relevance as agreed by consensus of the project management team.

Data cleaning

Prior to analysis data will be checked for outliers and missing data will be identified. Descriptive statistics will be performed to summarise data. Problems identified will be discussed with the PI of the original study, and amended as indicated by consensus discussion.

Data analysis and prognostic model development

Multivariable logistic regression modelling will be the primary method of analysis. The primary endpoint for the prognostic model will be delivery within 7 days. Another endpoint found to be important in focus group consultations performed in QUIDS Qualitative (online supplementary material) included delivery within 48 hours, and we will use this as a secondary endpoint if feasible (ie, if sufficient number of cases with delivery within 48 hours). We will develop an initial model with qfFN concentration, and then consider a model with other predefined clinical predictor variables (see the Data items section).

Tocolysis (which may delay onset of labour, although likely not beyond 48 hours) will be included as a categorical variable (administered/not administered). We will explore treatment effect by sensitivity analysis with and without the assumption that tocolysis could delay delivery within 48 hours by a maximum OR of 5.39, 95% CI 2.14 to 12.34, based on data in Haas et al.26

As the outcome is binary, a logistic regression modelling framework will be used to develop the model. A multilevel structure will be used to account for clustering of patients within studies, and heterogeneity of the effects of included factors (hereafter called ‘predictors’) will be accounted for using random effects, with between-study heterogeneity quantified using the estimated variance (τ2) and the I2 statistic. A separate intercept term per study will be included in the model, to account for the clustering and also gauge how predictions may require tailoring to different populations. Predictors with large heterogeneity in the prognostic effect across studies may be removed to ensure summary beta terms in the model are meaningful (accurate) for individual populations.16

In the primary analysis, we will use data from the first recorded attendance with signs and symptoms of preterm labour to determine the relationship between that individual episode and outcome. Data from subsequent attendances will be analysed subsequently, and may be included in an appropriate model. As a parsimonious model is sought, to reduce the factors included in the model that may otherwise delay its use, we will use backward stepwise selection based on an information criterion (eg, Akaike’s information criterion p<0.15) to identify a parsimonious set of factors to be included in the model; hereafter these are referred to as included ‘predictors’. Further, an approach of adding specialist tests, such as cervical length, only after considering simpler clinical assessment will be used, to maximise the utility of the model by ensuring that extra tests with their additional costs are only be included if they add to the predictive power.

Linearity between continuous variables and outcome will be assessed using cubic spline plots and data will be transformed where appropriate before inclusion in multivariable analysis (eg, using fractional polynomial methods). Missing data will be assessed to determine whether missing at random is appropriate, and if so, multiple imputation of observed participant characteristics will be used, with missing data imputed within each original study separately, before the meta-analysis. The results of these analyses will be compared with a complete case analysis.

Assessing apparent model performance

The apparent performance of the model will be assessed by its overall fit, and the observed discrimination and calibration in the IPD used to develop the model. Overall, fit of the models will be expressed with Nagelkerke R2. The ability of the models to discriminate between women with and without spontaneous preterm birth will be determined by the area under the receiver operating characteristics (AUC) curve, also known as the C statistic. Agreement between predicted and observed proportions of women with spontaneous preterm birth will be visualised using a calibration plot, and measured using calibration slope and calibration in the large.

Internal validation: assessing optimism in model performance

Apparent performance is likely to be optimistic, as it is examined in the same data used for model development. Therefore, internal validation will also be undertaken using a non-parametric bootstrap resampling technique in which each modelling step is repeated in each bootstrap sample, to obtain a new model in each bootstrap sample, and then its apparent performance (AUC and calibration slope) in the bootstrap sample is compared with its performance in the original dataset. The ‘optimism’ is the mean difference (across all bootstrap samples) between the apparent value in the bootstrap sample and the observed value in the original dataset. This optimism estimate is then subtracted from the original model’s apparent performance, to give an optimism-adjusted estimate of each measure of performance for the original model (eg, R2, C statistic, calibration slope).

Production of final model from IPD meta-analysis via uniform shrinkage

The optimism-adjusted calibration slope will be used as a uniform shrinkage factor, to adjust the parameter estimates (log ORs) of the original model. The beta coefficients in the original model will be multiplied by the shrinkage factor, and the study intercept terms re-estimated to ensure perfect overall calibration is maintained (across all studies and, ideally, in each study separately). This will thereby produce a final model containing the updated intercepts and the shrunken beta coefficients.27 With multiple intercepts, a strategy (or strategies) will be developed among the study investigators for which intercept should be chosen for use when externally validating the model in a new population (eg, choose intercept from study that most closely resembles the population of application); each strategy will be evaluated and compared in the cohort study external validation phase.

Added value of qfFN

The added value of qfFN will be examined throughout the whole model process, in particular its improvement on discrimination, calibration and other meaningful factors (such as clinical decisions) using appropriate techniques (such as net reclassification improvement and decision analysis methods).

Subgroup analyses

Subgroup analysis will be performed for multiple pregnancy, women with a previous preterm labour, gestation and those with criteria that are suggested to indicate preterm labour (number of uterine contractions in a set time period and/or cervical change). This will allow us to do a subgroup analysis in which we assess whether the predictive capacity of qfFN is similar in all subgroups.

Health economic analysis

An early-stage decision-analytical model will be built using evidence from current literature and from the IPD meta-analysis to explore the potential cost-effectiveness of different prognostic models including qfFN.

A literature review will be undertaken to inform model design and identify additional model parameters with searches of MEDLINE, EMBASE, Cochrane Library and the Paediatric Economic Database Evaluation for economic analyses including the use of fFN testing in woman with threatened preterm labour. Any evidence on resource use (test administration, treatments for preterm labour, hospital stay, hospital transfers), quality of life and diagnostic outcome data from the IPD meta-analysis will be synthesised with the wider evidence based on current practice for women attending hospital with signs and symptoms of preterm labour. The economic analysis will be undertaken from the perspective of the UK NHS adhering to good practice guidelines and the National Institute for Health and Care Excellence reference case.28 A decision tree will be developed to model the clinical pathway. The model will be used to explore potential cost-effectiveness of the prognostic model at different thresholds on the receiver operator characteristic curve, providing an economic rationale for the chosen prognostic model.

Ethics and dissemination

Trial management and oversight arrangements

Project management group

The trial will be coordinated by a PMG, consisting of the grant holders (chief investigator and coapplicants), the trial manager, representatives from the Study Office and the Centre for Healthcare Randomised Trials (CHaRT-the supporting Clinical Trials Unit), plus service user representatives (from the patient advisory group). The PMG will meet approximately every 4 months by teleconference or face to face.

TSC and data monitoring committee

A combined TSC and data monitoring committee (DMC) will oversee the conduct and progress of the study. The terms of reference of the committee will be developed separately. Members of the TSC/DMC will consist of experts and two patient representatives.

Good clinical practice

The study will be conducted in accordance with the principles of Good clinical practice. Local research and development approval will be obtained prior to commencement of the study at each site.

Dissemination

On completion of the study, the study data will be analysed and tabulated, and a clinical study report will be prepared. Results will be communicated to the academic community via the scientific literature, attendance at conferences and invited presentations. TRIPOD reporting guidelines will be adhered to.18 Summaries of results will also be made available to investigators for dissemination within clinics. Social media will be used to signpost publications and conference presentations and highlight important findings. Twitter and Facebook will be used to disseminate findings to professional organisations, charities, stakeholders and the public. Communication to the general public will further be facilitated by our close links with charities such as Tommy’s.29

Peer review

The study was extensively peer reviewed as part of the process of gaining grant funding from the National Institute of Healthcare Research (NIHR) HTA (14/32/01).