Vision Amniotic Leak Detector (ALD) to Eliminate Amniotic Fluid Leakage as a Cause of Vaginal Wetness in Pregnancy: A NICE Medical Technology Guidance.

In prelabour rupture of membranes (PROM) or preterm PROM the amniotic membranes rupture prior to labour. Where this is not overt a speculum examination is undertaken to confirm diagnosis. The Vision Amniotic Leak Detector (ALD) is a panty liner that can diagnose amniotic fluid as a cause of vaginal wetness. It was evaluated by the UK National Institute for Health and Care Excellence (NICE) as part of the Medical Technologies Evaluation Programme. The sponsor (CommonSense Ltd) identified five studies, of which three were deemed within scope by the External Assessment Centre (EAC). Two of these three used an inappropriate comparator. The EAC recalculated the diagnostic accuracy of Vision ALD using speculum examination as the comparator: sensitivity of 97% (95% CI 93-99%), negative predictive value of 96% (95% CI 92-98%). A negative result would therefore allow patients to be discharged with confidence. In the sponsor's cost-consequence model only patients with a positive Vision ALD result would have a speculum examination, producing a cost saving of around £10 per patient. The EAC felt that some costs were unjustified and the model did not include infection outcomes or use in a community setting. The EAC revised the sponsor's model and found the results were most sensitive to clinician costs. Vision ALD was associated with savings of around £15-£25 per patient when administration in lower-cost community healthcare avoided a referral to a higher-cost secondary-care centre. NICE published guidance MTG15 in July 2013 recommending that the case for adopting Vision ALD was supported by the evidence.

Key Points for Decision Makers

Introduction

This is one in a series of articles in Applied Health Economics and Health Policy summarising guidance produced by the National Institute for Health and Care Excellence (NICE) Medical Technologies Evaluation Programme (MTEP) [1-6]. This evaluation programme provides guidance on medical devices and diagnostic technologies to the UK National Health Service (NHS) and supports the adoption of technologies that improve clinical outcomes or the patient experience and/or that result in cost savings. The NICE MTEP process is explained by Campbell and Campbell in the first publication in this Applied Health Economics and Health Policy series [7]. This article summarises the External Assessment Centre (EAC) report and Medical Technology Guidance (MTG) for the use of the Vision® Amniotic Leak Detector (ALD) as a diagnostic tool for vaginal wetness during pregnancy [8].

Background

Premature or prelabour rupture of membranes (PROM) is a condition in which the amniotic membrane ruptures before the onset of labour and where the pregnancy is 37 weeks’ gestation or more. PROM occurs in 8–10 % of full-term pregnancies [9, 10] and is followed by labour within 24 h in 60 % of cases [11]. Where this does not happen labour may be induced, as membrane rupture increases the risk of a serious neonatal infection from 0.5 to 1 % [11]. If PROM is diagnosed the woman should be provided with advice about the risks and options and appropriate clinical monitoring should be administered. Preterm PROM (PPROM) is membrane rupture in pregnancies of less than 37 weeks’ gestation. It affects around 3 % of all pregnancies and approximately a third of preterm births [9, 10, 12]. Approximately one-third of patients with PPROM have a positive amniotic fluid culture, indicating infection which poses a risk to both the woman and the baby [12]. The NICE clinical guideline for the Induction of Labour states that induction should not take place in PPROM before 34 weeks unless there are additional obstetric indications. The decision to induce labour in such cases is influenced by the gestational age, additional obstetric indications, availability of neonatal critical care facilities, and the risks to the mother and neonate [13].

Membrane rupture is often accompanied by a gush of liquor and the diagnosis is therefore unequivocal. However, in some cases the presenting symptom may be an unexplained increase in vaginal wetness. Alternative causes for this symptom are leaking urine and vaginal infection, but these are of lesser clinical urgency. Differential diagnosis of vaginal wetness in pregnancy is therefore important and the early detection of leaking amniotic fluid may lead to a reduction in maternal and neonatal morbidity [14]. In the NHS currently, diagnosis is based on a sterile speculum examination in which the patient lies supine for up to an hour, following which a midwife or obstetrician will look for pooled liquor in the posterior fornix. Additional diagnostic tests may be used including nitrazine pH, ultrasound examination of the volume of fluid in the uterus and the fern test (crystallisation of amniotic fluid on a microscope slide). However, all of these require a vaginal examination and/or skilled clinical assessment. Diagnostic accuracy for these tests are less than ideal with various ranges of values quoted in the published literature [9, 12, 15, 16].

Amniotic fluid normally has a pH of 6.5 or above whereas vaginal pH is 3.5–4.5. Vision ALD (Common Sense Ltd, UK) is a panty-liner with a removable indicator strip that turns blue-green on contact with fluid with a pH higher than 5.2. The panty-liner is worn until wet or for up to 12 h and the indicator strip is removed and placed in a plastic drying box for 30 min. The colour of the indicator strip is then assessed. Urine has a pH of 7.0 and will also change the colour of the indicator strip initially. However, high concentrations of ammonia, such as in urine, cause the colour to fade within 30 min. Infection (including bacterial vaginosis, trichomoniasis or desquamative inflammatory vaginitis) can also raise vaginal pH and produce a positive result. Infected vaginal discharge does not contain high levels of ammonia so the colour change will remain after 30 min. The device is also known as AmnioSense, Al-Sense, Meitest and Amniscreen.

Decision Problem (Scope)

NICE defines the scope of the evaluation prior to the sponsor’s submission of evidence.

The benefits of Vision ALD to pregnant women claimed by the sponsor [17] are:

A reduction in unnecessary speculum examinations.

A reduction in the time spent in hospital. Women using Vision ALD will not require a hospital bed and can undergo the test as an outpatient or in a community setting; there is no requirement for them to lie down whilst wearing the liner.

A reduced risk of infection from speculum examination, particularly if repeat examinations are required.

The incidental detection of possible vaginal infection.

The benefits of Vision ALD to the healthcare system claimed by the sponsor are:

A reduction in the costs associated with the avoidance of speculum examination.

A reduction in staff time and hospital bed use.

Population

The population of interest was pregnant women with unidentified vaginal wetness which may be leaking amniotic fluid caused by ruptured membranes. The use of Vision ALD as a screening test in women with a history of PROM or PPROM and its routine use following amniocentesis were excluded.

Intervention (Vision ALD)

The intervention was Vision ALD administered by a healthcare professional in a community or acute setting. The use of Vision ALD by the patient herself was excluded. In their submission the sponsor (CommonSense Ltd., Israel) only included the use of Vision ALD in a hospital setting in their economic model and did not address its use in a community setting.

Comparator (Current Practice)

The comparator was speculum examination alone. In the NICE clinical guideline for Intrapartum Care [11] a speculum examination is only required if rupture of the membranes at term is uncertain. The Royal College of Obstetricians and Gynaecologists guideline on Preterm Prelabour Rupture of Membranes [12] recommends the use of clinical history followed by a sterile speculum examination. Ultrasound examination is considered useful in some cases.

Speculum examination may result in a false negative result if the leak of amniotic fluid is particularly slow or intermittent or if insufficient time is allowed for pooling. In their submission the sponsor stated that the speculum examination has a low sensitivity and quoted false negative rates of 12–30 %. However, the EAC could only identify one experimental study evaluating the diagnostic performance of speculum examination alone, which reported a 12 % false negative rate [18]. The EAC considered that this had not been calculated correctly and re-calculated the false negative rate for speculum examination alone as 3.8 or 3.9 % [19].

Outcomes

The outcomes specified for consideration in the scope included incidence of speculum examinations, diagnostic performance of Vision ALD in the identification of amniotic fluid leak, identification of vaginal infection, incidence of speculum-associated cross-infection, bed utilisation and staff time, and device-related adverse events. Only incidence of speculum examinations and the diagnostic performance of Vision ALD for rupture of membranes (ROM) were addressed by the sponsor’s submission. Additional work by the EAC also addressed the identification of vaginal infection.

Review of Clinical and Economic Evidence

The sponsor provided an evidence submission to NICE that reviewed the clinical and cost evidence for Vision ALD and presented a de novo cost-consequences model. Cedar a collaboration between Cardiff and Vale University Health Board and Cardiff University, was the EAC commissioned by NICE to produce the assessment report for this technology. The role of the EAC is to review and critique the sponsor’s submission: to ensure completeness and relevance of the evidence presented, to appraise its quality and to check the interpretation of this evidence with respect to the scope. The aim is to evaluate whether Vision ALD carries a diagnostic advantage and/or a reduced cost in comparison to current NHS standard care. The EAC was able to call upon clinical experts (research midwives and obstetricians) where clarification or advice on current UK practice was required.

Sponsor’s Submission of Clinical Effectiveness Evidence

The sponsor presented four studies, one of which was a laboratory study that assessed the ability of Vision ALD to detect small amounts of amniotic fluid [20]. The other three studies were clinical trials of Vision ALD in which diagnostic accuracy was assessed against a reference test [21-23]. The EAC’s literature search and grey literature search did not identify any studies not included in the sponsor’s submission. The three clinical studies are summarised in Table 1. All three studies recruited women who presented at a labour or antenatal unit in secondary care. Comparative results were assessed in women presenting with unexplained vaginal wetness or a history suggestive of membrane rupture. In Bornstein et al. [21], two additional groups were recruited: a positive diagnosis group of women with overt spontaneous or artificial rupture of the membranes (n = 42) and a negative diagnosis group of women presenting for routine antenatal checks with no complaint of vaginal wetness (n = 27).

Table 1

Methodology summary of the included studies. EAC External Assessment Centre, n number of subjects

References	Study design and details	Patients	Intervention (i), comparator (c)	Outcomes	Study issues
Bornstein et al. [21]	Prospective diagnostic accuracy feasibility study; plus negative and positive test groupsSingle centre (Israel)>20 weeks gestationAssessors were independent and blinded	n = 108 (103 analysed)Study = 34Positive = 42Negative = 27	i = Visionc = speculum exam alone, OR both fern AND nitrazine pH tests	Sensitivity and specificityκ statistic for agreement of vision reading between two clinicians	Small study groupDifferential verification bias in reference testsFive participants excluded for protocol violationsInconsistent reference tests
Bornstein et al. [22]	Prospective diagnostic accuracy studyMulticentre (Israel, USA)>16 weeks’ gestationVision results were assessed by two study assessors and the patient. All were independent and blinded from each others’ results	n = 339 (309 analysed)Israel 1 = 110Israel 2 = 179USA = 50	i = Visionc = speculum exam alone, OR both fern AND nitrazine pH tests	Sensitivity and specificity using patient-read Vision resultsκ statistic for agreement of Vision reading between the patient and clinician	Difference in vaginal infection rates between centresDifferential verification bias in reference testsThirty participants excluded for protocol violationsAdditional protocol violations found by EAC in study data
Mulhair et al. [23]	Prospective diagnostic accuracy studySingle centre (UK)18–42 weeks’ gestationAssessors were independent and blinded	n = 157 (139 analysed)	i = Visionc = speculum exam alone	Sensitivity, specificity, positive predictive value, negative predictive value	12 false positive readings unaccounted for18 exclusions

Only one of the studies (Mulhair et al.) used speculum examination alone as the comparator, as specified in the NICE scope. In the other two studies [21, 22] a positive comparator test was defined as a positive result from either speculum examination alone, or from both nitrazine pH and ferning tests.1 In Bornstein et al. [22], the primary outcome was the comparison between the patient’s reading of the Vision ALD result and the initial clinical diagnosis. In this study, if the Vision ALD result was positive and the comparator was negative then the woman was tested for vaginal infection and underwent a second comparator test within 48 h.

The results as submitted by the sponsor indicate that Vision ALD has a high sensitivity ranging from 95.7 to 100.0 % and a reasonably high specificity of 75.0–84.5 %. (A lower specificity of 65.0 % was reported by Mulhair et al. [23] but was excluded by the sponsor.) The sponsor did not report any other outcomes but two studies also reported positive and negative predictive values: 67.4–87.0 % and 94.7–98.1 %, respectively [22, 23]. Mulhair et al. [23] estimated a 38 % reduction in referrals to their unit if the device were used in the community and also carried out a subgroup analysis on preterm subjects: sensitivity of 93 %, specificity of 76 % (n = 61).

Critique of Sponsor’s Submission of Clinical Evidence

The EAC excluded the laboratory study from the analysis as outside the scope of the evaluation [20]. Each of the three clinical studies was consistent with the patient population in the NICE scope. Only Mulhair et al. [23] was also consistent with the scope intervention and comparator. The interventions varied slightly in the other two studies as both the patient and one [22] or two [21] investigators independently read the Vision ALD result. The Bornstein et al. studies [21, 22] used ‘clinical diagnosis’ as the comparator so that different reference tests were used for different patients (differential verification bias). Other outcomes identified in the scope were not reported: incidence of speculum examinations, speculum cross-infection, bed utilisation and staff time.

All three of the selected studies classed positive Vision ALD results due to vaginal infections as false positives, whereas the scope includes identification of vaginal infections as an outcome measure. In Bornstein et al. [21], six out of 34 women in the study group had a false positive Vision ALD result. Of these, four were diagnosed with bacterial vaginosis and the other two developed clinically identifiable PROM within hours of the initial examination. In the negative diagnosis group, two out of 27 women with no suspicion of ruptured membranes had a positive Vision ALD result but were subsequently identified as having ruptured membranes. In Bornstein et al. [22], 23 out of 309 women initially had a false positive Vision ALD result. Of these, four were rediagnosed as positive using another comparator test within 48 h.

The sponsor used an inappropriate appraisal tool, intended for randomised controlled trials, for reporting the quality of the included studies. The EAC used the QUADAS tool (QUality Assessment tool for Diagnostic Accuracy Studies) [24] to appraise the three diagnostic accuracy studies and rated them as good quality. Overall, the clinical evidence is good in terms of study design, but low in volume and only includes comparative diagnostic accuracy studies. It therefore provides little information about patient outcomes where clinical decision-making is based on Vision ALD readings. It is also unable to demonstrate whether Vision ALD is superior to the comparator.

Additional Work Carried Out by the External Assessment Centre (EAC)

The authors of the two of the Bornstein et al. studies [21, 22] provided the EAC with their study data upon request. With this the EAC was able to recalculate the diagnostic accuracy of Vision ALD using the intervention (Vision ALD assessed by a healthcare professional) and comparator (speculum alone) specified in the scope and to account for the re-classification of vaginal infections as true positives.

The diagnostic accuracy of Vision ALD was determined by the EAC using three different models: These results are shown in Tables 2 and 3.

Table 2

Diagnostic accuracy parameters recalculated by the External Assessment Centre, by analytic model. PPV positive predictive value, NPV negative predictive value, CI confidence intervals, for sensitivity, specificity, PPV and NPV these were calculated using Meta-DiSc [25]. Prevalence is the rate of positive speculum tests. CIs for this were calculated using MetaXL [26]

	References	Sensitivity % (95 % CI)	Specificity % (95 % CI)	PPV % (95 % CI)	NPV % (95 % CI)	Prevalence (%)
Primary analysis	Bornstein et al. (n = 34) [21]	100.0 (63.1–100.0)	69.2 (48.2–85.7)	50.0 (24.7–75.3)	100.0 (81.5–100.0)	23.5
	Bornstein et al. (n = 307) [22]	95.4 (90.7–98.1)	80.0 (72.8–86.0)	82.4 (75.9–87.7)	94.7 (89.3–97.8)	49.5
	Mulhair et al. (n = 139) [23]	98.3 (90.9–100.0)	65.0 (53.5–75.3)	67.4 (56.5–77.2)	98.1 (89.9–100.0)	42.4
	Weighted mean	96.3 (92.9–98.4)	74.3 (68.6–79.5)	75.9 (70.4–80.8)	96.0 (92.3–98.3)	45.6 (32.7–58.6)
Post-hoc analysis	Bornstein et al. [21]	100.0 (69.2–100.0)	75.0 (53.3–90.2)	62.5 (35.4–84.8)	100.0 (81.5–100.0)	29.4
	Bornstein et al. [22]	95.5 (91.0–98.2)	82.1 (75.1–87.9)	84.7 (78.5–89.6)	94.7 (89.3–97.8)	50.8
	Mulhair et al. [23]	98.5 (92.1–100.0)	73.2 (61.4–83.1)	77.9 (67.7–86.1)	98.1 (89.9–100.0)	48.9
	Weighted mean	96.6 (93.4–98.5)	78.9 (73.2–83.8)	81.3 (76.2–85.7)	96.0 (92.3–98.3)	48.7 (39.1–58.4)
Infection analysis	Bornstein et al. [21]	100.0 (76.8–100.0)	90.0 (68.3–98.8)	87.5 (61.7–98.4)	100.0 (81.5–100.0)	41.2
	Bornstein et al. [22]	95.6 (91.1–98.2)	83.2 (76.2–88.8)	85.8 (79.7–90.6)	94.7 (89.3–97.8)	51.5
	Mulhair et al. [23]	98.7 (92.8–100.0)	81.3 (69.5–89.9)	86.0 (76.9–92.6)	98.1 (89.9–100.0)	54.0
	Weighted mean	96.8 (93.7–98.6)	83.3 (77.8–87.8)	86.0 (81.3–89.8)	96.0 (92.3–98.3)	51.5 (47.0–55.9)

Table 3

Diagnostic accuracy parameters (per 1000 initial cases), generic to analytic models

References	Speculum examinations avoided	Vision patients sent home erroneously
Bornstein et al. [21]	529	0
Bornstein et al. [22]	427	23
Mulhair et al. [23]	381	7
Weighted mean	421	17

Primary analysis: Intervention—clinician reading of Vision ALD. Comparator—initial clinical diagnosis using only speculum examination.

Post-hoc analysis: As for primary analysis, but women with initially false positive Vision ALD results who were later diagnosed with ROM or who gave birth within 72 h of the study are re-designated as true positives.

Infection as positive analysis: As for the post-hoc analysis, but false positives due to infections are re-designated as true positives. This is in accordance with the NICE scope in which identification of infections is an outcome measure.

Sensitivity, specificity, positive predictive value and negative predictive value were calculated for each study, according to the model being used. The comparator was speculum examination only. Two patients from Bornstein et al. [22] were excluded from this analysis due to protocol violations. No patients were excluded from the other studies. Weighted means for sensitivity, specificity, positive predictive value and negative predictive values were used to take into account the sample sizes in the three studies.

The weighted mean false negative rate (1-sensitivity) for Vision ALD for the three models varied from 3.2–3.7 %, which is comparable to that for a speculum examination of 3.9 % [18, 19]. The number of speculum examinations avoided by using Vision ALD is the number who would be sent home following the Vision ALD result. This is derived from the negative Vision ALD test rate: Bornstein et al. [21] 18/34, Bornstein et al. [22] 131/307 and Mulhair et al. [23] 53/139. The number of patients sent home erroneously following a Vision ALD test is determined from the number of patients with a false negative Vision ALD result as a proportion of all negative Vision ALD results: Bornstein et al. [21] 0/34, Bornstein et al. [22] 7/307 and Mulhair et al. [23] 1/139. (This is not the same as the false negative rate, which is the number of patients with a false negative Vision ALD result as a proportion of all patients with a positive speculum result.)

The EAC recalculations show that Vision ALD has high sensitivity and reasonably high specificity and therefore shows promise in terms of non-inferiority to the reference standard. The high sensitivity (97 %) and high negative predictive value (96 %) means it can reliably rule out negative cases of PROM and PPROM in the populations included in these studies. If all women presenting with unidentified vaginal wetness were initially diagnosed with Vision ALD and only those with a positive result were then given a speculum examination, this indicates that around 42 % could avoid this procedure.

Cost Evidence

The sponsor identified Mulhair et al. [23] as a published economic study in their economic submission. In this the authors use their diagnostic accuracy results to speculate that the use of Vision ALD in the community to rule out PROM/PPROM could avoid 38 % of such referrals to the antenatal day unit (ADU) at a cost of £147 each. This is extrapolated to an approximate annual saving of £33,000 for that unit. However, the EAC considered this was not a valid economic study of Vision ALD.

The sponsor also provided a de novo economic model with an NHS perspective. The model structure was shown in a flow diagram comparing two diagnosis options for pregnant women with unexpected vaginal wetness who present at an ADU. The comparator is standard speculum diagnosis, in which the sponsor included a cardiotocograph trace (CTG, for detecting foetal heart rate and uterine contractions). In the intervention, Vision ALD is used as the initial diagnostic test. Patients with a positive Vision ALD result then go on to have a speculum examination to confirm membrane rupture (or infection), and a CTG. Those with a negative result are sent home. The timeframe for this model was the diagnostic tests only. No treatment or labour costs or costs associated with infections were included. The model was presented as a simple Excel spreadsheet.

The base-case inputs into this model and their sources are listed in Table 4. For the intervention the model resulted in a cost of £33.93 per patient. For the comparator, the model resulted in a cost of £43.94 per patient. Therefore the potential saving of using Vision ALD from the sponsor’s model is £10.01 per patient.

Table 4

Inputs to the sponsor’s base-case economic model. EAC External Assessment Centre, PSSRU Personal Social Services Research Unit, PROM prelabour rupture of membranes, CTG cardiotocograph, ROM rupture of membranes

Item	Value	Source	EAC comments
Cost of Vision ALD per pad	£1.60	Manufacturer	Appropriate
Midwife cost per min	£1.37	Cost of a nurse in a day ward excluding qualification costs, PSSRU [27]	Based on Band 5 for qualified nurses. This is low for midwives (Band 5–8)
Cost of a sterile disposable speculum	£0.84	Oncall Medical Supplies [28]	Appropriate
Cost of a bed day (24 h) in an antenatal day unit	£364	Reference cost for antenatal false labour including PROM excess bed day	This should be cost per attendance and cannot be equated to a bed-day. Source is unknown
Time for midwife to undertake a CTG trace	20 min	Clinical advice	Only requires midwife time during set-up and reading, not the full 20 min trace time
Time for midwife to undertake a speculum exam	5 min	Clinical advice	Appropriate
Time spent laying on bed before speculum	30 min	Clinical advice	Clinical advisers indicate this can be <60 min, therefore appropriate
Time to administer Vision	5 min	Clinical advice	Appropriate
Percentage of patients with negative ROM following Vision	42 %	Published study [23]	This is the prevalence rate. It should be the Vision ALD negative test rate of 38 % from the same study

The sponsor conducted a one-way sensitivity and threshold analysis. The cost of the Vision ALD pads and disposable speculums were fixed, the range for bed day costs was taken from a published source (not identified) and other parameters were varied by ±50 %. For the threshold analysis the sponsor combined the midwife’s time to conduct a CTG trace and a speculum examination. Vision ALD remained cost saving over the one-way sensitivity ranges. Threshold analysis indicated that Vision ALD would no longer be cost saving if the cost of a pad exceeded £11.60, the time for a midwife to perform a speculum examination and CTG trace fell below 7.6 min, the time to administer Vision ALD increased above 12.3 min or the proportion of women with a negative Vision result fell below 19 %. Multiway analysis indicated the ‘best case’ (combination of inputs most favourable to Vision ALD) and ‘worst case’ (least favourable combination) scenarios for the maximum cost saved and minimum saved/maximum incurred, respectively, by using Vision ALD. The best-case scenario produced a cost saving of £54.60 per patient and the worst-case scenario incurred an additional cost of £3.52 per patient.

Critique and Interpretation of Cost Evidence

The sponsor’s de novo cost model was conservative; however, there were limitations and assumptions that weaken the model. Table 4 includes the EAC’s comments on the appropriateness of each input. The EAC conducted a more detailed sensitivity, threshold and impact analysis and determined that the cost savings were most sensitive to the midwife time for a CTG trace, followed by the time to administer a Vision ALD test and the proportion of patients with a negative Vision ALD test result. The higher cost of standard care was primarily due to the cost of midwife time to undertake the CTG trace. A clinical adviser confirmed that a CTG trace does not require 20 min of dedicated midwife time as modelled by the sponsor. The midwife would only stay to assess the trace if anything adverse is initially observed. The use of a CTG trace was not included in the scope, or in the sponsor’s clinical evidence or description of the clinical pathway. Additionally it could also be used in the intervention option, during the Vision ALD wear time. The EAC also considered the cost of a bed day in an ADU (£364 for 24 h) to be very high. The source of this value was unclear but is based on inpatient bed costs rather than outpatient or day unit; however, the impact of this is mitigated by the short stay-time (30–60 min). The sponsor obtained clinical advice regarding inputs from a single clinician and did not have their model checked by clinical advisers. The sponsor’s model does not account for false negative Vision ALD test results nor for any costs relating to the identification or treatment of infections as included in the scope.

The NICE scope states that Vision ALD can be administered in the community (primary healthcare) setting, which was not explored in the sponsor’s model. In light of this, NICE requested that the EAC model additional scenarios and inputs, including community use.

EAC Revisions to the Sponsor’s Economic Model

The EAC’s revisions to the economic model are separated into PROM and PPROM models, as the two conditions vary considerably in treatment, infection rates and costs. Subgroup analysis in Mulhair et al. [23] indicates that Vision ALD diagnostic accuracy is not different between these populations. In standard diagnosis, the speculum examination can be performed at an ADU, by a general practitioner (GP) or a midwife at the GP practice. In the intervention arm, Vision ALD can be administered in these same settings, or additionally by a midwife visiting the patient’s home. A positive Vision ALD result is followed up with a speculum examination, as in the sponsor’s model. The EAC’s model includes infection, with and without ROM, as a diagnostic outcome, and the timeframe includes infection treatment but not labour. The decision tree for PROM is shown in Fig. 1.

Fig. 1

Decision tree for the External Assessment Centre’s revision to the economic model. Figures in italics are probabilities—shown for prelabour rupture of membrane (PROM). The figure was originally a TIFF file created using the TreeAge Pro modelling software

Base-case inputs attached to the model are given in Table 5. All inputs and the model structure were checked by clinical advisers. The positive speculum and Vision ALD result rates were determined as weighted means from the three clinical studies in the clinical evidence. We used the EAC post-hoc analysis to determine the rates for the speculum examination because this model provides the most complete estimate of the underlying prevalence rate. We used published infection rates rather than those from the clinical studies as infection testing was not universally conducted. We used the post-hoc plus infection analysis for the PPV for Vision ALD (86 %) as infection was included as an outcome. False positive results for the speculum examination are not accounted for as the PPV for this is very high (97 or 100 % [19]). The EAC model excluded Vision ALD false negative results because: (1) it has a very high negative predictive value, and (2) the false negative rates for Vision ALD and for standard diagnosis (speculum examination) are very similar. Thus any cost differences for these patients between the two arms of the model would be negligible.

Table 5

Resources used in the External Assessment Centre’s revisions to the economic model. PPV positive predictive value, PROM prelabour rupture of membranes, PPROM premature prelabour rupture of membranes, ADU antenatal day unit, GP general practitioner, PSSRU Personal Social Services Research Unit

Item	Value	Source
Positive speculum result	48.8 %	Clinical evidence
Positive Vision result	57.9 %	Clinical evidence
Vision false positive rate (1-PPV)	14 %	Clinical evidence
Infection rate in women with PROM	1.0 %	[11]
Infection rate in women with PPROM	28 %	[29]
Infection rate in women without PROM	0.5 %	[11]
Time to administer Vision in clinical setting	10 min	Clinical advice
Time to administer Vision in patient’s home	15 min	Clinical advice
Time for speculum exam	15 min	Clinical advice
GP cost per minute	£3.68	PSSRU [27]
Midwife at GP surgery, cost per min	£0.88	PSSRU [27]
Midwife at patient’s home, cost per min	£1.18	PSSRU [27]
Referral cost for an ADU	£81.00	NHS Reference Costs, NZ05C [30]
Cost of Vision ALD per pad	£1.60	Sponsor
Cost of a sterile disposable speculum	£0.84	NHS Supply Chain

There is no evidence or claim that Vision ALD increases the rate of diagnosis of ROM. The proportion of patients at each payoff should therefore remain constant, irrespective of whether Vision ALD or standard diagnosis is used. In practice there were small (<0.8 %) differences in the patient distributions that were considered to be due to rounding errors and to the assumption that the post-hoc analysis represents the true prevalence rate for ROM. Although we modelled treatment costs, these were not influential in the cost-consequence analysis due to these very small differences between outcome prevalences and are not shown here for simplicity. Infection rates were similarly unimportant in the cost differences between diagnosis options. Savings from the introduction of Vision ALD would therefore be realised if the cost of administering Vision ALD to all patients is offset by the savings from the number of speculum examinations avoided.

Results and Sensitivity Analysis from the External Assessment Centre (EAC)’s Revisions to the Economic Model

Base-case results are presented in Table 6 as per patient cost differences: Vision ALD minus standard care. Vision ALD is cost-incurring when administered in the same healthcare setting that a patient would attend for a speculum examination. Additional costs range from £1.28 to £38.28 depending on the clinician administering the tests and whether the condition is PROM or PPROM. Much higher treatment costs for PPROM patients meant that the small differences in outcome prevalences between the standard and Vision ALD arms of the model were more significant in this model. This leads to a consistent additional £5.75 cost for the Vision ALD arm as seen in the more positive values for PPROM in Table 6. Vision ALD was determined to be cost-saving only when it is administered at a lower clinician cost (for example, by a midwife at the GP practice or the patient’s home) when the patient would otherwise have gone to a high-cost scenario for a speculum examination (GP or ADU). One-way sensitivity analysis shows that clinician time is the most significant cost in the model in both PROM and PPROM, which supports the conclusions of the base-case analysis.

Table 6

Base-case cost analysis of Vision ALD administration in different healthcare settings. Positive numbers represent a per-patient cost incurred by the Vision ALD arm of the model, rows in bold indicate scenarios where Vision ALD is cost saving. PROM prelabour rupture of membranes, PPROM premature prelabour rupture of membranes, GP general practitioner, ADU antenatal day unit

Vision ALD arm		Standard care arm (speculum only)	PROM	PPROM
Initial Vision ALD test	Speculum follow-up
GP	GP	GP	£14.85	£20.60
Midwife at GP	Midwife at GP	Midwife at GP	£4.52	£10.28
Midwife at home	Midwife at GP	Midwife at GP	£7.52	£13.28
ADU	ADU	ADU	£1.28	£7.03
GP	Midwife at GP	Midwife at GP	£32.52	£38.28
GP	ADU	ADU	£3.99	£9.75
Midwife at GP	GP	GP	−£13.15	−£7.40
Midwife at GP	ADU	ADU	−£24.01	−£18.25
Midwife at home	GP	GP	−£10.15	−£4.40
Midwife at home	ADU	ADU	−£21.01	−£15.25

NICE Guidance

Preliminary Guidance

The NICE MTAC met in February 2013 and considered evidence from a range of sources, including the sponsor’s submission, the EAC report and additional economic modelling, and testimony from clinical experts. The Committee provisionally decided that:

“The case for adopting the Vision Amniotic Leak Detector (ALD), when issued by a midwife or other healthcare worker, is supported by the evidence. The available evidence suggests that the device can reliably exclude amniotic fluid leak as a cause of vaginal wetness in pregnancy, avoiding the need for a speculum examination. Using the device in the community can prevent unnecessary referrals to secondary care antenatal day units or maternity triage services for speculum examinations, releasing clinical time.

The Vision ALD should be considered for use in pregnant women with unexplained vaginal wetness.

Based on cost modelling, the use of the Vision ALD is estimated to be cost saving in scenarios considered to be clinically likely. When issued by a midwife or other healthcare worker in a primary-care setting, cost savings per woman of up to £24.01 (for prelabour rupture of membranes; PROM) and £18.25 (for premature prelabour rupture of membranes; PPROM) could be achieved, through avoiding the need for referral to an antenatal day unit. When issued by a community midwife in a woman’s home, Vision ALD is associated with an estimated cost saving of up to £21.01 per woman for PROM and £15.25 per woman for PPROM”.

The Committee heard advice from clinical experts indicating that in the community setting Vision ALD would most likely be administered by a practice-based or community midwife, followed by a speculum examination in an ADU for positive results. In these scenarios, the EAC determined that Vision ALD would be cost saving. Expert clinical advice also highlighted that Vision ALD could be issued by a healthcare worker at a GP practice, and that women contacting an ADU by telephone could be advised to contact a community or primary care based midwife for administration of Vision ALD. The use of Vision ALD in an ADU would be cost-neutral if the consumable costs for the Vision ALD pad and speculum were included in the referral costs [30]. The Committee concluded that a change in practice to avoid secondary care referrals would be acceptable to clinicians in the care pathway, and that Vision ALD would allow patients to avoid inconvenient, invasive and unnecessary speculum examinations.

Consultation Response

The preliminary guidance was available for comment on the NICE website as a public consultation document between 22 March and 19 April 2013. Few responses were received and only one comment from the sponsor resulted in a minor change to Recommendation 1.1. This was altered to include reference to the patient benefit of avoiding the discomfort associated with a speculum examination. NICE Medical Technology Guidance on Vision ALD to assess unexplained vaginal wetness in pregnancy was published on 17 July 2013 as MTG15 [8].

Key Challenges and Learning Points

Original Sources for Parameter Values Should be Checked

It is easy for errors to perpetuate through the published literature by citation when articles are not checked thoroughly. In attempting to verify the sponsor’s quoted false negative rate for speculum examination, the EAC located an original reference [18]. It was only by close examination of this short paper that it was realised that the rate was being quoted incorrectly. The patient population in Ladfors et al. [18] (women with equivocal ROM on speculum examination) was subtly different to that in the scope for Vision ALD and therefore their results required re-interpretation rather than quoting. The EAC was able to recalculate the appropriate parameter which improved the diagnostic performance of the comparator in the evaluation of Vision ALD [19].

Diagnostic Accuracy Studies Should be Supplemented by Implementation Studies

Comparative studies between a novel diagnostic intervention and a ‘gold standard’ reference comparator, such as those included in this assessment, can only demonstrate non-inferiority of the intervention. The sensitivity of Vision ALD for the diagnosis of ROM is very high and comparable to that for speculum examination. Only a controlled trial that implements treatment decisions based on the results of Vision ALD could demonstrate superiority over the current NHS standard diagnosis. It would also provide additional data for economic modelling and may help to encourage adoption, especially in such cases as this where changes in the patient pathway are necessary in order to realise cost-savings.

Implementation can be Key to the Realisation of Cost Savings for Medical Devices

In the case of Vision ALD, nearly half of women presenting with unexplained vaginal wetness could avoid an unnecessary speculum examination, which some women may find uncomfortable or embarrassing. However, by implementing Vision ALD in the same setting as current standard diagnosis the savings from avoided referrals did not outweigh the cost of additional Vision ALD tests. Cost savings were only realisable if the care pathway was altered so that the initial testing could be conducted by a lower cost healthcare professional. This is possible because speculum examinations require skilled nursing or medical staff plus a consulting space, whereas Vision ALD could be administered (e.g.) at the patient’s home.

Medical devices and diagnostic tests differ significantly from pharmaceuticals in that the costs and benefits of using them are dependent on their mode of use. Also, new healthcare technology in general is likely to increase healthcare costs [31, 32]. As the emphasis of NICE medical technologies guidance is to evaluate technologies that either provide additional benefit at similar cost or are cost saving compared to current standard care [33], the clinical pathway into which the technology fits may need to be altered in order to achieve any financial savings. Manufacturers, evaluators and adopters should not be shy of investigating alternative pathways to identify the greatest benefit to patients and the healthcare system.

Treatment Costs were Essentially Irrelevant to the Cost-Consequences Modelling

A new diagnostic technology may result in earlier detection of a condition or in the detection of additional cases. In such cases treatment costs are likely to be increased following implementation. However, in the case of Vision ALD, no change in the diagnostic outcomes was expected. Therefore, costs were either saved or incurred solely due to the diagnostic tests themselves, or the healthcare setting/clinician time used to administer them.

Clinical Advisers Should Reflect Both the Current Care Pathway and the Proposed Care Pathway in a Technology Assessment

Clinical advice was provided to the EAC by secondary-care professionals, but no representation from primary care was available. Medical technologies can be cost saving through disruption of care pathways and by changing the settings in which care is delivered. Therefore advisers are needed for both the standard care and proposed new settings.

The available evidence indicates that Vision ALD has a high sensitivity and a high negative predictive value. This means that patients can be reliably discharged with a negative PROM/PPROM diagnosis if they have a negative Vision ALD result.

The false negative rate of Vision ALD is almost identical to that of speculum examination (approximately 3–4 %), so the number of patients wrongly discharged with PROM/PPROM will be similar to those with the speculum examination.

Vision ALD can reduce the number of unnecessary speculum examinations by around 42 %, and it is also less invasive than the speculum examination, which may benefit patients.

Referrals to secondary-care clinic may be avoided by using Vision ALD in the community setting, which could be cost saving and release clinic resources.