Implementation of field detection devices for antimalarial quality screening in Lao PDR-A cost-effectiveness analysis.

Substandard and falsified (SF) antimalarials have devastating consequences including increased morbidity, mortality and economic losses. Portable medicine quality screening devices are increasingly available, but whether their use for the detection of SF antimalarials is cost-effective is not known. We evaluated the cost-effectiveness of introducing such devices in post-market surveillance in pharmacies in Laos, conservatively focusing on their outcome in detecting SF artemisinin-based combination therapies (ACTs). We simulated the deployment of six portable screening devices: two handheld near-infrared [MicroPHAZIR RX, NIR-S-G1], two handheld Raman [Progeny, TruScan RM]; one portable mid-infrared [4500a FTIR] spectrometers, and single-use disposable paper analytical devices [PADs]. We considered two scenarios with high and low levels of SF ACTs. Different sampling strategies in which medicine inspectors would test 1, 2, or 3 sample(s) of each brand of ACT were evaluated. Costs of inspection including device procurement, inspector time, reagents, reference testing, and replacement with genuine ACTs were estimated. Outcomes were measured as disability adjusted life years (DALYs) and incremental cost-effectiveness ratios were estimated for each device compared with a baseline of visual inspections alone. In the scenario with high levels of SF ACTs, all devices were cost-effective with a 1-sample strategy. In the scenario of low levels of SF ACTs, only four devices (MicroPHAZIR RX, 4500a FTIR, NIR-S-G1, and PADs) were cost-effective with a 1-sample strategy. In the multi-way comparative analysis, in both scenarios the NIR-S-G1 testing 2 samples was the most cost-effective option. Routine inspection of ACT quality using portable screening devices is likely to be cost-effective in the Laos context. This work should encourage policy-makers or regulators to further investigate investment in portable screening devices to detect SF medicines and reduce their associated undesired health and economic burdens.

Introduction

Substandard and falsified (SF) medicines have devastating health and economic implications [1,2]. Falsified medicines are medical products that are produced to deliberately and fraudulently misrepresent their identity, composition or source, while substandard medicines are those that are produced by authorised manufacturers but fail to meet quality standards or specifications [3,4]. The distribution of SF medicines is an increasing global concern [5-7]. The World Health Organization (WHO) estimated that in low- and middle-income countries (LMICs), approximately 10% of medicines are substandard or falsified. Reasons why LMICs are highly affected include poor governance, weak technical capacity and poor supply chain management [2,8]. The economic impact of SF medicines in LMICs has been estimated at US$30.5 billion per year [1].

Despite declines in transmission, malaria is still ranked fourth in amongst infectious diseases in terms of its global burden, following diarrheal diseases, HIV, and tuberculosis, respectively [9]. The incidence of malaria cases and deaths in Laos were reported as 10.2 cases per 1,000 population and less than 50 deaths in 2015 [10]. WHO reported the number of confirmed malaria cases in Laos in 2015 as 36,056 [10]. Malaria is curable but in the absence of effective treatment can be fatal. Artemisinin-based combination therapies (ACT) are now recommended as first-line therapy for Plasmodium falciparum malaria worldwide [11]. A recent systematic review suggested that SF antimalarial drugs, including ACTs, are widely distributed across LMICs with a mean prevalence of 19.1% [12]. The distribution of SF ACTs endangers patients and in the longer term can threaten malaria control and elimination campaigns by promoting the development of drug resistance [13-15].

In a national survey conducted in Laos in 2003, 22 out of 25 pharmacies across the country were found to supply falsified artesunate [16]. In a repeat survey with similar methodology in 2012, a quarter of ACT samples tested were outside the 90–110% pharmacopeial limits claimed on the label, but none of the samples were classed as falsified [17]. While routine pharmacy post-market surveillances are performed by regulatory bodies to detect SF medicines, these rely on an initial screening by visual inspection, followed by a longer and expensive process for centralized pharmacopeial testing.

New field detection devices are increasingly available and could enhance the surveillance process with prompt and actionable results at the point of inspection. These devices vary widely in cost and underlying technology, with varying advantages and disadvantages when used in the field [18].

This is the fourth paper in the Collection ‘A multi-phase evaluation of portable screening devices to assess medicines quality for national Medicines Regulatory Authorities’ [19]. Six devices deemed ‘field-suitable’ in the laboratory evaluation were included in a field evaluation in the Lao PDR (Laos) during which their utility and usability for pharmacy inspections were investigated [20]. This study aims to evaluate the costs and consequences of introducing these six portable devices for the screening of antimalarials during pharmacy inspections and assess whether investment in these devices would be cost-effective from a healthcare provider’s perspective in the Laos setting. This analysis conservatively focuses only on the cost-effectiveness of these devices in detecting substandard and falsified ACTs, with health gains measured in terms of clinical outcomes for malaria patients obtaining treatment from pharmacies that undergo routine post-market surveillance.

Methods

Strategies

Six field detection devices deemed suitable based on laboratory evaluation and field-testing in Laos [21] were included in this analysis. These consist of two handheld near-infrared spectrometers, two handheld Raman spectrometers, one portable mid-infrared device, and single-use disposable paper analytical devices (PADs). All devices were compared with a baseline of visual inspections alone, that was assumed to detect 25% of substandard and 50% of falsified ACTs. This analysis estimated the incremental costs of the six devices if incorporated within these inspection visits, and their outcomes measured in disability adjusted life years (DALYs) averted, taking a healthcare provider’s perspectives with a one year timeframe without discounting. A key assumption is that the introduction of the devices facilitating on-site detection of SF medicines will allow for the temporary removal of SF medicines from distribution in the pharmacies where they are detected.

The actual prevalence of SF ACTs in Laos is not well described, although the available evidence indicates a large decline in recent years in the prevalence of falsified antimalarials and modest falls in the prevalence of substandard antimalarials [17]. We therefore modelled two hypothetical scenarios with varying levels of SF ACTs in circulation. In Scenario 1, 60% of ACTs are genuine, 20% substandard, and 20% falsified. In Scenario 2, 85% are genuine, 10% substandard, and 5% falsified.

Almost all falciparum malaria cases in Laos are concentrated in 42 districts across five southern provinces (Savannakhet, Salavan, Sekong, Champasak, and Attapeu). Patients are assumed to be equally distributed across the districts and have equal access to 10 pharmacies per district. The key intervention to reduce the burden of malaria is the provision of long-lasting insecticide nets [22].

Model structure

A decision tree model was developed to simulate post-market surveillance scenarios at the pharmacy level with screening devices as compared with visual inspection alone. Each pharmacy was assumed to stock three ACT brands which are used with equal frequency amongst malaria patients obtaining treatment from the pharmacy. The first component of the decision tree simulates the inspection process where three ACT brands are screened by inspectors in each pharmacy (Fig 1A). The second component models health outcomes for malaria patients prescribed an ACT, which can be genuine, substandard, or falsified (Fig 1B).

Fig 1

Decision analytic model for drug sampling at pharmacy (A Medicines Model) and malarial illness (B Patients Model).

The modelled scenarios assumed that one device is available for each of the 42 malaria endemic districts for biannual post-market surveillances of 10 pharmacies per district. In each pharmacy and for each medicine, the inspectors test either one, two, or three samples per ACT brand. Testing higher numbers of samples implies a higher probability of the device identifying SF medicines, but also an increased probability of mistakenly indicating that a sample is not genuine. Performance estimates for the six devices were derived from their evaluation in a laboratory setting [19] determining the probabilities that they provide a correct result for either genuine, substandard or falsified antimalarials. The accuracy estimates were derived from the samples tested after removal from their packaging (Table 1). For sampling strategies in which two or three samples are tested, the probability of the device indicating a SF sample was raised to the power of the number of samples taken.

Table 1

Probability with which each device will identify genuine, substandard (between 50% and 80% API) and falsified (wrong/0% API) medicines with strategies of 1/2/3 samples per brand of ACTs.

Device	Medicine quality *	1-sample		2- sample **		3-sample **
		Device: Fail†	Device: Pass‡	Device: Fail†	Device: Pass‡	Device: Fail†	Device: Pass‡
	Genuine	0	1	0	1	0	1
TruScan RM	Substandard	0.42	0.58	0.66	0.34	0.80	0.20
	Falsified	1	0	1	0	1	0
MicroPHAZIR RX	Genuine	0	1	0	1	0	1
	Substandard	0.5	0.5	0.75	0.25	0.88	0.13
	Falsified	1	0	1	0	1	0
	Genuine	0	1	0	1	0	1
4500a FTIR	Substandard	0.33	0.67	0.56	0.44	0.70	0.30
	Falsified	1	0	1	0	1	0
	Genuine	0	1	0	1	0	1
Progeny	Substandard	0.08	0.92	0.16	0.84	0.23	0.77
	Falsified	1	0	1	0	1	0
	Genuine	0	1	0	1	0	1
NIR-S-G1	Substandard	0.33	0.67	0.56	0.44	0.70	0.30
	Falsified	0.95	0.05	1	0	1	0
	Genuine	0	1	0	1	0	1
PADs	Substandard	0	1	0	1	0	1
	Falsified	1	0	1	0	1	0

*Genuine medicine (90–110% API), Substandard (50% and 80% API) and Falsified medicine (wrong/0% API)

**The probability of detecting SF medicine under 2- and 3-sample strategies are estimated with each repeat sample test having the same probability as a single sample test independently.

Device:Fail refers to the probability that the device indicates that the medicine is substandard or falsified.

Device:Pass refers to the probability that the device indicates that the medicine is genuine.

Where the screening devices indicates that a brand of ACT is SF, that batch is assumed to be replaced with genuine ACTs, implying a temporary improvement in the proportion of genuine medicines at the pharmacy. This was assumed to last for one month before returning to the scenario-specific baseline prevalence of SF ACTs. Samples that fail the device screening are assumed to be sent for formal reference laboratory testing by high-performance liquid chromatography (HPLC), a standardized and high cost analytical chemistry technique. Incorrectly classifying a genuine sample as a fail by the portable devices incurs the unnecessary costs of reference HPLC testing and replacement ACTs. If the device indicates a genuine medicine, no further action is taken. Therefore, if a sample classed as genuine was in fact substandard or falsified, patients remain at higher risk of severe outcomes.

Parameter inputs

Costs

The total cost of introducing the screening devices was composed of costs for procurement and delivery, inspector visits, sample test reagents, reference testing by HPLC, and the cost of drug replacement if the results from a tested sample failed the screening test. A micro-costing approach was used to estimate these costs. To estimate the cost of devices and delivery, fixed and variable costs for the devices and their consumables were collected from either manufacturer responses to a list of questions sent by email or the supplier quotation (Text A in S1 Text). Fixed costs comprised of the instrument purchase and maintenance assuming a five-year shelf life for all devices, other than the PADs which are single-use disposable tests. Variable costs include those for reagents and materials used for each assay, as well as for the additional time spent per sample by inspectors.

For the costs of inspections, five inspectors per district were assumed to perform inspections at 10 pharmacies biannually. Based on informal interviews with staff from the Lao Bureau of Food and Drug Inspection, all five inspectors were assumed to visit two pharmacies per field trip. The total number of visits and hours spent for inspections was multiplied by the inspectors’ salary and per diem rates to calculate the total cost per inspection. The cost of additional inspection time for use of the devices was derived from a time-and-motion study in a simulated pharmacy conducted with Lao medicine inspectors [19] (Table A in S1 Text). The total variable costs per inspection were determined by the sampling strategy of either one, two, or three samples per ACT brand for each of the three ACT brands. The costs of HPLC reference testing and ACT replacement were calculated assuming that all samples failing a device test were tested with HPLC, and SF stocks replaced with genuine ACTs.

Probabilities and consequences

Patients treated with a substandard or falsified medicine are assumed to be at higher risk of becoming severely ill (24%) [27] and dying of malaria (15% of those severely ill) [23]. These adverse outcomes are converted into disability adjusted life years (DALYs), using the assumption that the event of illness due to severe malaria would last for six weeks with 0.133 disability weight [24] and the average age of patients infected with malaria was conservatively assumed to be 48 years. Therefore, those who died of malaria would lose 20 life-years, based on the average life expectancy in Laos [25]. (Table 2).

Table 2

List of parameters used in the cost-effectiveness analysis model.

Parameters	Values	Reference
Total malaria cases per year (Lao PDR, year 2015)	36,056	[10]
Number of districts where malaria cases were reported	42
Number of pharmacies inspected per district	10	Laos Food and Drug Department (FDD)* (current practice)
Number of ACT brands per pharmacy	3	Assumed
Total number of malaria cases, per pharmacy per year	86	[10]
Total ACT (blisters) stock of all brands, per pharmacy	258	Assumed
Number of samples tested per brand	1–3	Assumed
Number of inspections, per pharmacy per year	2	Lao FDD
Number of months genuine replacement ACTs in place until returning to baseline levels	1	Assumed
Economic data
Number of inspectors, per visit (five inspectors per district to perform inspections at 10 pharmacies)	5	Laos FDD
Hours per inspection, per pharmacy	1	Assumed
Number of pharmacy visit, per day	2	Assumed
Inspector’s salary per hour (US$ 144 or 1.2 million LAK per month)	0.9	Bureau of Food and Drug Inspection (BFDI), Lao PDR
Per diem (per day) (250,000 LAK)	30	Bureau of Food and Drug Inspection (BFDI), Lao PDR
Cost of device (up front and subsequent costs over 5 years)	See Table 3	[20]
Cost of test, per sample (consumable material and reagents)	See Table 3	[20]
Cost of reference analysis with HPLC (1,245 million LAK), per sample	US$ 149.4	Bureau of Food and Drug Inspection (BFDI), Lao PDR
Cost of ACT, per tablet	US$ 0.78	[23]
Years of life with disability (YLD) due to severe malaria	0.02	Disability weight of 0.133 [24] and an assumed duration of 6 weeks
Years of life lost (YLL)	20	Assumed based on life expectancy in Laos [25] and most patients in low endemic settings being adults
Willingness to pay (GDP per Capita) threshold (Lao)	US$ 2,353	[26]
Transition Probability
Risk of severe malaria when treated with genuine ACT	0	[27]
Risk of severe malaria when treated with substandard and falsified ACT	0.24	[27] (Assumed to be equal to untreated malaria; average for children and adults)
Risk of death in severe malaria	0.15	[21]
Risk of death in non-severe malaria	0	[21]

*MRA–Medicines Regulatory Authority

Cost-effectiveness analysis

The costs and DALYs were used to estimate the incremental cost-effectiveness ratios (ICER) of each device in both scenarios of high and low prevalence of SF antimalarials. Devices are considered cost-effective when the incremental cost per DALY averted is below the assumed willingness to pay threshold (WTP) of US$ 2,353, the 2016 Laos GDP per capita [26], as recommended by the WHO [28]. ICERs were also estimated for different sampling strategies, whereby the inspectors select either one, two, or three samples per brand of ACT.

The ICERs were initially calculated for each device individually as compared with a baseline of visual inspections alone. To facilitate the comparison of multiple devices and sampling strategies we use the net-monetary benefit (NMB). The NMB provides a simpler indicator of relative cost-effectiveness than ICERs, whereby the device and sampling strategy with the highest NMB is identified as optimal. NMB is calculated by multiplying the effectiveness of the intervention (in this instance measured in DALYs averted) by the WTP threshold and deducting any incremental costs associated with use of the device.

A series of one-way sensitivity analyses were performed to determine the effect on results if parameter values deviated from the initial point estimates. For brevity, this was carried out only in the lower prevalence scenario. Plausible ranges for key parameters, including the cost of the devices (-50% and +20%), test performance (-30% and +30%), and number of years of life lost due to malaria (-20% and +20%), were applied to the model. The results are presented in a tornado diagram to show the magnitude of the effect of varying these parameter estimates on the cost-effectiveness of each device. In addition, an alternative approach of purchasing one device per province instead of one per district (i.e. purchasing five instead of 42 devices), was also evaluated; the costs of transporting the devices between districts was not accounted for. As the assumption regarding the detection rates of visual inspection for substandard and falsified are central to the analysis (25% and 50%, respectively), two further scenario analyses were performed with detection rates of 12.5% and 5% for substandard medicines, and 25% and 10% for falsified medicines.

Finally, a budget impact analysis estimating the financial impact when introducing these devices at the country level was also carried out. This consisted of the initial purchase costs for 42 devices (1 per district), and the annual costs for using the devices in the post market surveillances over a duration of 5 years.

Results

Cost of screening device

The TruScan RM had the highest fixed cost, followed by Progeny, MicroPHAZIR RX, 4500a FTIR, NIR-S-G1, and PADs (Table 3). The largest proportion of the total costs for all devices other than the PADs was the initial purchase cost. The PADs had the highest variable costs per sample, estimated at US$ 3.06 with no capital costs. For the other devices the variable cost per sample were below US$ 0.10 (Table 3).

Table 3

Fixed and variable costs of the devices.

Costs (US$, 2017)	TruScan RM	Micro PHAZIR RX	4500a FTIR	Progeny	NIR-S-G1	PADs
Initial cost
- Cost of devices*	68,750	52,250	34,724	67,449	1,539	0
Subsequent cost (over 5 years)
- Replacement cost of the battery	112	506	N/A	580	30	N/A
- Light bulb	N/A	300	N/A	N/A	N/A	N/A
- Other material, solvent, and maintenance	N/A	300	N/A	N/A	N/A	N/A
Shipment Cost **	138	147	358	163	126	126
Fixed total over 5 years	69,000	53,503	35,082	68,192	1,695	126
Variable unit cost per sample	0.04	0.04	0.09	0.04	0.04	3.06

*Device costs are inclusive of Lao PDR VAT rate of 10%

**Shipment cost was estimated from the average price of DHL Express Worldwide service from Europe (UK) and the USA to Lao PDR based on device weight (June, 2018).

Cost of implementation

To implement the post-market surveillance with these devices nationally with a 1-sample strategy would incur an initial cost ranging between US$5,308 to US$2,893,292 in both prevalence scenarios (Table 4). The TruScan RM had the highest total initial cost followed by Progeny, MicroPHAZIR RX, 4500a FTIR, NIR-S-G1 and PADs, respectively. The total annual costs ranged from US$220,706 to US$380,317 under the high prevalence scenario and from US$139,925 to US$196,302 under the lower prevalence scenario. The MicroPHAZIR RX had the highest annual costs followed by the TruScan RM, 4500a FTIR, NIR-S-G1, PADs, and Progeny.

Table 4

Cost of implementation at country level with two prevalence scenarios.

a) Cost of implementation at country level under a high prevalence scenario (20% substandard and 20% falsified)
Cost US$ (2017)	TruScan RM	Micro PHAZIR RX	4500a FTIR	Progeny	NIR-S-G1	PADs
Initial Cost
Cost of Devices*	2,887,500	2,194,500	1,458,414	2,832,855	64,634	0
Shipping Cost**	5,792	6,173	15,047	6,864	5,308	5,308
Total Initial Cost	2,893,292	2,200,673	1,473,461	2,839,719	69,942	5,308
Annual Cost
Maintenance cost	1,176	11,613	-	6,090	315	-
Cost of Inspections§	81,993	81,984	82,099	82,072	81,959	82,290
Cost of Consumablesß	492	474	1,050	648	423	23,917
Cost of Reference analysis by HPLC†	183,538	197,656	169,420	127,065	164,286	79,062
Cost of Replacement of suspected SF ACTs∑	82,262	88,590	75,934	56,950	73,633	35,436
Total Annual Cost	349,460	380,317	328,503	272,825	320,615	220,706
b) Cost of implementation at country level under a low prevalence scenario (10% substandard and 5% falsified)
Cost US$ (2017)	TruScan RM	Micro PHAZIR RX	4500a FTIR	Progeny	NIR-S-G1	PADs
Initial Cost
Cost of Devices*	2,887,500	2,194,500	1,458,414	2,832,855	64,634	0
Shipping Cost**	5,792	6,173	15,047	6,864	5,308	5,308
Total Initial Cost	2,893,292	2,200,673	1,473,461	2,839,719	69,942	5,308
Annual Cost
Maintenance cost	1,176	11,613	-	6,090	315	-
Cost of Inspections§	81,993	81,984	82,099	82,072	81,959	82,290
Cost of Consumablesß	491	474	1,050	648	423	23,917
Cost of Reference analysis by HPLC†	63,532	70,592	56,473	35,296	55,190	28,237
Cost of Replacement of suspected SF ACTs∑	28,475	31,639	25,311	15,820	24,736	12,656
Total Annual Cost	175,667	196,302	164,934	139,925	162,623	147,099

*Device costs are inclusive of Lao PDR VAT rate at 10%.

** Shipping cost was estimated from the average price of DHL Express Worldwide service from Europe (UK) and the USA to Lao PDR based on device weight.

§Cost of inspections was estimated based on the total time spent for overall inspections (visual inspections) and additional time spent for the test by each device.

ßCost of consumables was estimated from additional material used including reagent and cleaning wipes for the test by each device.

†Cost of reference analysis was estimated from the number of samples sent to validate with HPLC from suspected substandard and falsified (SF) samples as suggested by the device screening result.

∑ Cost of replacement was estimated from cost of the whole batch of ACTs that require replacement with genuine ACTs at the pharmacy outlet due to a suspected SF batch as indicated by the device screening results.

The procurement cost of devices is the largest cost category for all devices under both scenarios (Fig 2A and 2B) except NIR-S-G1 and PADs, for which the largest cost category is the cost of reference testing and the cost of inspections, respectively, under the high prevalence scenario (Fig 2A) and the cost of inspections for both devices under the lower prevalence scenario (Fig 2C). Without accounting for the cost of devices, the reference testing cost category is the largest for all devices under high prevalence scenario except for PADs for which the cost of inspections is the largest cost category (Fig 2B). The cost of inspections are the largest cost category for all devices under the lower prevalence scenario except NIR-S-G1 where the cost of reference testing is highest (Fig 2D).

Fig 2

Cost of introducing post-market surveillance at country level by cost category under (A) Cost of introduction by component under a high prevalence scenario, (B) Cost of introduction by component excluding procurement costs under a high prevalence scenario, (C) Cost of introduction by component under the lower prevalence scenario, (D) Cost of introduction by component excluding procurement costs under the lower prevalence scenario.

Cost-effectiveness of the devices individually compared with visual inspections alone

In the high prevalence scenario and using a single-sample strategy, the devices averted between 445 and 778 DALYs per year across the malaria endemic areas in Laos, compared with a baseline of visual inspections alone, with the MicroPHAZIR RX being the most effective device. All devices were cost-effective when compared with the baseline of visual inspections alone, with an ICER well below the WTP threshold (indicated by the blue line in Fig 3A), ranging between US$391 and US$1,514 per DALY averted.

Fig 3

Incremental costs and effects of inspection with a 1-sample strategy in (A) high prevalence scenario; 20% substandard and 20% falsified and in (B) lower prevalence scenario; substandard 10% and falsified 5%, compared with visual inspection. The diagonal line represents the Willingness to pay threshold at US$ 2,353, Lao PDR GDP per capita).

In the lower prevalence scenario, the devices averted between 111 and 278 DALYs with a single-sample strategy, compared with the baseline of visual inspections alone; the MicroPHAZIR RX remained the most effective device. Four devices, the NIR-S-G1, PADs, 4500a FTIR, and MicroPHAZIR RX were cost-effective in this scenario with an ICER below the WTP threshold (Fig 3B) and a positive NMB (Table 5).

Table 5

Country level costs and effects in high and lower prevalence scenario per device with a 1-sample strategy compared with visual inspection ranked by descending net monetary benefit (NMB, US$).

DALYs—disability adjusted life years. ICER—incremental cost-effectiveness ratio.

Device name	Cost	DALYs	Incremental Cost	DALY averted	ICER	NMB
High prevalence scenario
Baseline	81,900	1,111.7
NIR-S-G1	334,541	464.9	252,641	647	391	1,269,333
MicroPHAZIR RX	818,129	333.5	736,229	778	946	1,094,896
4500a FTIR	623,195	444.7	541,295	667	811	1,028,241
PADs	270,838	667.0	188,938	445	425	857,419
TruScan RM	927,883	389.1	845,983	723	1,171	854,348
Progeny	839,551	611.4	757,651	500	1,514	419,501
Low prevalence scenario
Baseline	81,900	444.7
NIR-S-G1	176,548	227.4	94,648	217	436	416,640
PADs	148,161	333.5	66,261	111	596	195,328
4500a FTIR	459,626	222.3	377,726	222	1,699	145,452
MicroPHAZIR RX	634,114	166.8	552,214	278	1,987	101,759
TruScan RM	754,091	194.6	672,191	250	2,687	-83,615
Progeny	706,651	305.7	624,751	139	4,496	-297,765

Multiway comparison of all devices and sampling strategies

When comparing all possible options (six devices with 1/2/3-sample strategy) with visual inspections in Scenario 1, all options were cost-effective (Fig 4A). The comparative cost-effectiveness analysis suggests that the best option may be the NIR-S-G1 with a 2-sample strategy followed by a 3- and 1-sample strategy (Table 6). In most cases a 2-sample strategy outperformed the 3-sample and single sample strategies (except in the case of PADs and Progeny where a single sample strategy outperformed 2/3-sample strategies).

Fig 4

Incremental costs and effects of all sampling strategies (1, 2, or 3-samples per drug per inspection) under (A) high prevalence scenario and (B) lower prevalence scenario. The diagonal line represents the Willingness to pay threshold at US$ 2,353, Lao PDR GDP per capita.

Table 6

Country level costs and effects in high and lower prevalence scenario per device and per different sample strategy compared with visual inspection ranked by descending net monetary benefits (NMB, US$).

DALY—disability adjusted life year. ICER—incremental cost-effectiveness ratio.

High prevalence scenario
Rank	Device	Strategy	Incremental Cost	DALY averted	ICER	NMB
1	NIR-S-G1	2-sample	521,578	814.4	640	1,394,582
2	NIR-S-G1	3-sample	816,210	914	893	1,334,537
3	NIR-S-G1	1-sample	252,641	646.8	391	1,269,333
4	MicroPHAZIR RX	2-sample	1,038,137	945	1,099	1,185,372
5	4500a FTIR	2-sample	804,136	815.3	986	1,114,185
6	MicroPHAZIR RX	1-sample	736,229	778.2	946	1,094,896
7	MicroPHAZIR RX	3-sample	1,351,730	1,028.40	1,314	1,067,970
8	4500a FTIR	3-sample	1,099,001	914.1	1,202	1,051,844
9	4500a FTIR	1-sample	541,295	667	811	1,028,241
10	TruScan RM	2-sample	1,130,917	884.8	1,278	950,897
11	TruScan RM	3-sample	1,439,046	979.3	1,469	865,301
12	PADs	1-sample	188,938	444.7	425	857,419
13	TruScan RM	1-sample	845,983	722.6	1,171	854,348
14	PADs	2-sample	326,192	444.7	734	720,166
15	PADs	3-sample	463,445	444.7	1,042	582,912
16	Progeny	1-sample	757,651	500.3	1,514	419,501
17	Progeny	2-sample	917,220	551.2	1,664	379,827
18	Progeny	3-sample	1,098,954	597.9	1,838	307,997
Low prevalence scenario
Rank	Device	Strategy	Incremental Cost	DALY averted	ICER	NMB
1	NIR-S-G1	2-sample	199,405	296.2	673	497,626
2	NIR-S-G1	3-sample	318,591	345.9	921	495,217
3	NIR-S-G1	1-sample	94,648	217.3	436	416,640
4	4500a FTIR	2-sample	481,535	296.5	1,624	216,037
5	4500a FTIR	3-sample	601,355	345.9	1,739	212,478
6	PADs	1-sample	66,261	111.2	596	195,328
7	MicroPHAZIR RX	2-sample	675,210	361.3	1,869	174,955
8	4500a FTIR	1-sample	377,726	222.3	1,699	145,452
9	MicroPHAZIR RX	3-sample	804,050	403	1,995	144,211
10	PADs	2-sample	118,805	111.2	1,069	142,784
11	MicroPHAZIR RX	1-sample	552,214	277.9	1,987	101,759
12	PADs	3-sample	171,349	111.2	1,541	90,240
13	TruScan RM	2-sample	786,713	331.2	2,375	-7,395
14	TruScan RM	3-sample	912,833	378.5	2,412	-22,249
15	TruScan RM	1-sample	672,191	250.1	2,687	-83,615
16	Progeny	2-sample	676,709	164.4	4,115	-289,775
17	Progeny	1-sample	624,751	139	4,496	-297,765
18	Progeny	3-sample	739,749	187.8	3,939	-297,863

In Scenario 2, when comparing all possible options (six devices and 1/2/3-sample strategy) with visual inspections, 12 out of 18 options were cost-effective (Fig 4B). The comparative cost-effectiveness analysis suggests that in this scenario too the best option would be using NIR-S-G1 with 2-samples, followed by 3-samples, and 1-sample (Table 6).

Sensitivity analysis

Fig 5 shows a tornado diagram for the case of the NIR-S-G1 under Scenario 2, illustrating the change in NMB when key model parameters were changed above or below their point estimates for the NIR-S-G1. The number of months after detecting a SF batch of ACTs before returning to baseline SF prevalence has the highest impact on the NMB, followed by the device performance in detecting genuine ACTs. Results of the one-way sensitivity analysis for all other devices in the lower prevalence scenario are provided in Fig A in S1 Text. With lower detection rates of visual inspection in both scenarios, all but one of the devices (the Progeny) were cost-effective with lower ICERs in both high and lower prevalence scenarios (Table B in S1 Text).

Fig 5

Tornado diagram illustrating one-way sensitivity analyses with a range of plausible parameter values for key model parameters, in a lower prevalence scenario for the NIR-S-G1 device (blue bars represents scenarios with lower parameter estimates, orange bars represent scenarios with the higher parameter estimates, and dotted vertical line represents the cost-effectiveness threshold).

When the total number of devices across the country is reduced to 5 (from 42), all devices, and especially devices with high upfront costs such as the 4500a FTIR, MicroPHAZIR RX, TruScan RM, and Progeny were highly cost-effective. In this scenario, MicroPHAZIR RX was estimated to be the most cost-effective option (Table 7 and Table C in S1 Text)).

Table 7

Country level costs and effects in sensitivity analysis when using one device per province instead of one per district per device in lower prevalence scenario with a 1-sample strategy compared with visual inspection in descending order of net monetary benefit (NMB, US$).

DALY—disability adjusted life year. ICER—incremental cost-effectiveness ratio.

Name	Cost	DALYs	Incremental Cost	DALY averted	ICER	NMB
Baseline	81,900	444.7
MicroPHAZIR RX	238,192	166.8	156,292	277.9	562	497,681
NIR-S-G1	164,003	227.4	82,103	217.3	378	429,185
TruScan RM	243,491	194.6	161,591	250.1	646	426,985
4500a FTIR	200,016	222.3	118,116	222.3	531	405,062
Progeny	202,028	305.7	120,128	139.0	864	206,859
PADs	147,226	333.5	65,326	111.2	588	196,263

Budget impact analysis

The total budget to implement the routine medicine inspections with these screening devices with a 1-sample strategy across all 42 districts over five years ranged from US$1,108,836 to US$4,640,594 under high prevalence scenario and from US$704,806 to US$3,771,629 under lower prevalence scenario. TruScan RM was associated with the highest 5-year total cost followed by Progeny, MicroPHAZIR RX, 4500a FTIR, NIR-S-G1, and PADs, respectively. (Table 8).

Table 8

Budget impact analysis under high and low prevalence scenario with a 1-sample strategy across all 42 districts.

Cost US$ (2017)	TruScan RM	Micro PHAZIR RX	4500a FTIR	Progeny	NIR-S-G1	PADs
High prevalence scenario
Cost for the 1st year	3,242,752	2,580,990	1,801,964	3,112,544	390,558	226,014
Cost for subsequent years (2nd-5th year)	349,460	380,317	328,503	272,825	320,615	220,706
Total cost (five years)	4,640,594	4,102,258	3,115,976	4,203,845	1,673,019	1,108,836
Low prevalence scenario
Cost for the 1st year	3,068,959	2,396,975	1,638,395	2,979,644	232,565	152,408
Cost for subsequent years (2nd-5th year)	175,667	196,302	164,934	139,925	162,623	147,099
Total cost (five years)	3,771,629	3,182,183	2,298,131	3,539,346	883,055	740,806

Discussion

Our results indicate that introduction of any of the devices included here into routine pharmacy post market surveillances for the detection of SF ACTs would be cost-effective in a scenario where SF antimalarials are highly prevalent. In a scenario where substandard medicines are prevalent but falsified medicines are less frequent, there is a clear advantage for devices that can detect both forms of poor quality medicines. In both scenarios, the NIR-S-G1 appeared to be the most cost-effective option, with a 2-sample strategy being more cost-effective compared with 3- and 1- sample tests per brand of ACT, with the incremental costs of the 3rd sample as compared with the first 2 samples not being justified by the incremental gains. The MicroPHAZIR was associated with the best performance in identifying SF antimalarials, but due to its high initial costs as compared with the variable costs, it was the most cost-effective option only when fewer devices are procured for use across several districts, which could be logistically impractical. The budget impact analysis also suggests that some of these devices could be affordable for local authorities, in particular the 4500 Agilent, NIR-5-G1, and PADs. Our findings show that for the majority of devices, the initial procurement cost alone is much higher than the annual cost of visual ACT inspection alone (ranging from 18 times higher with the 4500 Agilent to 35 times higher with TruScan RM), with the exception of PADs and NIR-5-G1where the costs of device are lower. This could act as a deterrent from their adoption, despite being potentially highly cost-effective.

While in many of the scenarios the devices appeared to be cost-effective, it is important to emphasize that cost-effectiveness is to a great extent contextual and will depend on how the devices are implemented and how their use alters pre-existing practices. For example, it was generally the case that devices with the ability to detect substandard medicines as well as falsified ones outperformed those that could detect falsified medicines alone; how big an advantage this represents in a real-life setting will depend mostly on the relative prevalence of substandard and falsified medicines. In a context with a large percentage of substandard medicines but lower falsified ones, only those devices with the capacity to detect substandard medicines are likely to be cost-effective. As it is increasingly apparent that medicine quality is highly variable through time and space, conclusions on cost-effectiveness will also be dynamic as the prevalence of SF medicines change and as regulatory systems, pharmaceutical supply chains, and medicine use patterns change.

The cost-effectiveness results were highly dependent on the assumptions made on how the devices would be integrated within the medicine inspection environment. For instance, the number of devices required per province and how inspectors would respond to samples that fail a test. We assumed that for logistical reasons each district requires its own device, and that when a substandard or falsified medicine is detected, the batch of medicine is replaced with a genuine one lasting for one month before returning to baseline levels of SF medicines. The detection of SF medicines may have much wider effects for public health and cost-effectiveness if the information is shared appropriately with other neighboring districts, provinces and countries which would be alerted to the problem, facilitating further investigation and response. All these factors will vary considerably through time and space, therefore at a later stage if a decision is made to proceed with introducing these devices it will be imperative to refine the assumptions and parameter estimates. A refined analysis will then be more informative as to the choice of device, and how best to utilise it in the field. The approaches and parameter estimates used in the cost-effectiveness analysis were mostly conservative and our results, therefore, are likely to underestimate rather than overestimate cost-effectiveness. Most importantly, we focused only on the outcomes of detecting SF ACTs and ignored other essential medicines, particularly antimicrobial agents such as amoxicillin, acyclovir, doxycycline, ethambutol and isoniazid that can cure many widely prevalent infections, and ensuing genuine medicines are used can help slow the development of antimicrobial resistant to these agents in the long run. [29,30] For some devices for which the reagents are costly and the range of API detection is limited, this was appropriate. Other devices able to detect a broad range of medicines at no added cost will offer greater potential health outcomes than those accounted for in our analysis.

The costs per test for each device were derived from capital purchase costs of the device, reagents and other consumable costs that are dependent on the number of samples tested, and maintenance costs that are mostly fixed (although are likely to rise with the number of tests performed). We assumed that the devices are used relatively infrequently, up to 180 samples per device per year, across a district’s 10 pharmacies. For some devices such as the PADs or those with high reagent costs, the cost per sample tested will scale with the frequency of testing. Other devices with high purchase costs but low consumable costs could be far more cost-effective than they appeared in the analysis if used more frequently than we assumed. It is important; however, not to overlook the limit on the number of sample tests that can be performed in a single inspection, and the opportunity costs of using up inspection time that could be dedicated to other activities (e.g. visual inspections of larger volumes of samples).

The number of previous studies estimating the burden and economic impact of poor quality antimalarials in low- and middle-income countries is very limited [31-33] and only one of these assessed the economic impact of interventions that might address this impact, focusing on eliminating antimalarial stock-outs, increasing care seeking rates for malaria treatment, and encouraging better adherence to antimalarial treatment [27]. The annual loss due to SF antimalarials was estimated to be US$ 892 million in Nigeria [29] and at US$ 614 million for the population aged under five in Uganda [31]. Our work is one of the first attempts to evaluate the cost-effectiveness of interventions to mitigate the impact of SF ACTs (and SF medicines more broadly) and to our knowledge the first to consider the cost-effectiveness of devices to detect medicine quality in field inspections. We based our analysis in the context of Laos, but our findings might be informative for other countries where malaria endemicity and the prevalence of poor quality medicine in considering these devices for post-market surveillance.

This study has several limitations. First, our analysis contains many assumptions about how devices might be utilised in the inspections and about contextual factors such as the total number of ACT brands and stock level at the pharmacy. This may not be accurate and is very context specific. We used imagined prevalences of SF ACTs that are unlikely to reflect reality. In addition, the scope of this study considers only the costs and outcomes if devices are deployed at final pharmacy points, rather than higher up the pharmaceutical supply chain. Evaluations with wider scope would help with better understanding of the broader usage and impact of these devices. Possible indirect effects of using these devices in field inspections were not accounted for, notably higher vigilance on the part of retailers in their medicine purchasing and storage practices. Lower prevalence of SF ACTs could also slow the spread of artemisinin resistance with long term health and economic gains, which are unaccounted for here. The probabilities used to measure the effectiveness of the devices was based on a study performed in the laboratory evaluation. Devices were tested ‘out-of-the-box’ but their performances could have been improved if further upfront work had been performed [20,21]. This may have induced an under-estimation of the probabilities to identify SF (especially substandard medicines) in the current study. The upfront work required would also have had increased their costs. Finally, we were surprised to discover a very meagre evidence base in the Greater Mekong subregion of the economics of post-market surveillance of medical products. More data, collected in standardized formats across the subregion, are needed to allow more objective comparative analysis of the economics of post-market surveillance and how such devices and other interventions would vary in cost-effectiveness in different communities.

In conclusion, our findings establish that routine use of field detection devices in post marketing surveillance in a resource limited setting could be cost-effective, but further evaluation will be needed to identify optimal strategies for their use. This information can aid policy-makers or regulators considering investment in handheld screening devices to improve medicine quality and reduce the undesired health and economic burden associated with poor quality medicines.

Additional information for methods and results.

Text A. List of information acquired from manufacturers for device cost estimation. Table A. Median total time taken per sample in sample set testing. Fig A. Results of sensitivity analyses from the cost-effectiveness analysis. Table B. Results from sensitivity analyses with lower detection rate with visual inspection (A) 12.5% and 25% reduction for substandard and falsified medicines, respectively and (B) 5% and 10% reduction under both high and lower prevalence scenarios. Table C. Results from a sensitivity analysis when using one device per province instead of one per district per device in high prevalence scenario with a 1-sample strategy compared with visual inspection.

(DOCX)

Click here for additional data file.

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

6 Feb 2021

Dear Dr Luangasanatip,

Thank you very much for submitting your manuscript "Implementation of Field Detection Devices in Medicine Quality Screening in Lao PDR – A Cost-Effectiveness Analysis" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments.

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Vahid Yazdi-Feyzabadi, PhD

Deputy Editor

PLOS Neglected Tropical Diseases

Vahid Yazdi-Feyzabadi

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: First, I think that te paper needs to more clearly present the objectives of the study rather than only saying that they are evaluating the cost-effectiveness of introducing the screening devices. They should state what the testable hypothesis is for the study, the perspective of the study, and the target audience. Also, since the study focused on testing of ACTs, it appears to be focused on antimalarials rather than other types of drugs. In that case, the title of the study should be changed to reflect the focus on antimalarials as opposed to all medicines. Iis not clear whether the findings can be generalized to other types of medicines.

It would also be helpful to describe the prevalence and incidence of malaria in Lao PDR what interventions are currently being implemented against malaria, etc.

The study design is described but focuses largely on the assumptions of the decision analysis and effectiveness of the screening devices. It requires more detail on data collection and costing methods. It is unclear whether actual data collection was conducted in Lao PDR and, if so, who conducted the analysis. Also, it does not include a description of the cost categories and costing methods used. It does not appear that capital costs were depreciated as would be expected. Also, the total costs of introducing the devices is not described, i.e. implementation costs as well as the costs of the devices, are not presented except in Table 7. It would be good to see a table with total costs as well as proportions spent on each cost category since delivery costs are important in making decisions on introduction of a new technology.

I found Table 1 difficult to interpret. It would help to explain this table in more detail - e.g. what device fail and device pass means. This information is in the decision analytic model but should also be stated in the table.

Reviewer #2: Yes, the methods used, the statistical analysis and data presentation is well indicated. The study met its objectives with sufficient sample size.

Reviewer #3: Use Strategies instead of overview heading.

In the parameter inputs section of the methods, it is better to include subtitles such as probabilities, costs and consequences, which in each section, how to calculate each item is explained in detail.

Given that the measure of outcome is DALY averted in this study and the main parameters of DALY such as YLL and disability weight are assumed and there is no specific source or reference for them. Therefore, the results cannot be trusted.

The perspective of cost calculating, time horizon and discount rate are not mentioned in the method.

--------------------

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: The analysis presented matches the analysis plan to some extent. However, as noted above, it does not provide sufficient information on the costs of implementation and only presents this information in the budget impact. This information should be presented in the general results section for the two scenarios and some charts developed with the shares of total costs for each cost category with and without the costs of the devices

Reviewer #2: Yes

Reviewer #3: The ACER column in Tables 4 and 5 should be changed to ICER because ACER is calculated for a strategy while the numbers in this column are incremental cost/ incremental effect for a strategy compare with baseline.

Probability sensitivity analysis(PSA) should be performed according to its importance in policy decisions

--------------------

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: In the discussion section, it would be helpful to compare the findings with other studies in the literature if there are any. Also, it would be good to compare the delivery costs with and without the cost of devices with current expenditures on ACT screening if this information is available.

Reviewer #2: Discussion is poorly written.

Reviewer #3: (No Response)

--------------------

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: (No Response)

Reviewer #2: Minor revision

Reviewer #3: (No Response)

--------------------

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: The paper discusses the cost-effectiveness of of portable screening devices to assess the quality of medicines in Lao PDR. The paper indicates that insufficient screening of medicine quality is being conducted, particularly to assess substandard medicines. However, the paper needs to improve its situation analysis, presentation of methods and results, as well as discuss other literature on the subject. It is unclear whether the study is focusing on antimalarials rather than all medicines, the title does not seem to match the content, and the costing methods are not well-described and presented.

Reviewer #2: The authors sought to determine the Cost-Effectiveness of Field Detection Devices in Medicine Quality Screening in Lao PDR. Indeed, Substandard and falsified medicines pose a significant threat to the health system nowadays.

The authors did an outstanding work which add knowledge to the scientific community.

The methods used, the statistical analysis and data presentation is well indicated. The English write-up was also good.

It is a breakthrough to assess the cost-effectiveness of intervention methods to mitigate the impact of SF ACTs.

Listed below are minor comments:

• Your study focused on market surveillance among pharmacies in Laos. The study could have been more plausible if it is multi-centered including various facilities to draw better concluding remarks.

• You have tested about 6 different devices to evaluate cost-effectiveness; however, device specific conclusions were not made lastly.

• The discussion is poorly written; It looks simply a repetition of your methodology and result part. Moreover, it states facts and I didn’t see comparisons with other similar studies. Suggestion: better to mention a portion of your result, compare with other similar (may not be exactly same) reports, then draw justification.

Reviewer #3: In the abstract, instead of the analysis scenario, it is better to emphasize the type of costs that have been calculated and the outcome measurement index.

Depending on the type of economic evaluation study (cost utility analysis), it is better to use the term outcome or effect instead of benefit in all parts of the manuscript.

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Melaku Ashagrie Belete

Reviewer #3: Yes: Reza Goudarzi

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, PLOS recommends that you deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see https://journals.plos.org/plosntds/s/submission-guidelines#loc-methods

Submitted filename: Reviewer report .docx

Click here for additional data file.

1 Apr 2021

Submitted filename: Response to reviewers_MedQualScreening.docx

Click here for additional data file.

2 May 2021

Dear Dr Luangasanatip,

Thank you very much for submitting your manuscript "Implementation of Field Detection Devices for Antimalarial Quality Screening in Lao PDR – A Cost-Effectiveness Analysis" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you modify the manuscript according to the review recommendations.

Please prepare and submit your revised manuscript within 10 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Vahid Yazdi-Feyzabadi, PhD

Deputy Editor

PLOS Neglected Tropical Diseases

Vahid Yazdi-Feyzabadi

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: The objectives of the study are clearly stated. The study design seems appropriate. In general, the methods seem appropriate. I question, though, why the study authors did not use discounting for their capital costs. In a cost-effectiveness study, it is recommended to estimate economic costs rather than financial costs and the capital costs should be discounted. So, I suggest that the authors should discount the capital costs.

Reviewer #3: (No Response)

--------------------

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: (No Response)

Reviewer #3: (No Response)

--------------------

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: I have some comments about the conclusions. First, for the budget impact analysis, I don't think it makes sense to say that donor support to purchase some of the devices would be required since that would not be supporting sustainability. Instead, it would be better to say what the most affordable options are, given the available budget.

It seems obvious that cost-effectiveness is not an 'inherent' feature of the devices since cost-effectiveness will vary with the probability of SF, the costs, and effectiveness. So I would reword that sentence.

I also think that it would be useful to discuss whether there are any other uses of the devices that would increase their cost-effectiveness.

Reviewer #3: (No Response)

--------------------

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: I see that the authors added a paragraph on malaria incidence to the paper. However, I think that it should be in the introduction section rather than the methods section. Also, they should say 'incidence' rather than 'incidences.' I also think that they should add a sentence on the burden of malaria illness compared to other diseases and why it would be important to ensure that ACTs are of high quality and not SF.

The phrasing of some sentences is awkward. In several places, it would be preferable to use the term 'introducing' rather than 'implementing. - e.g. pg. 5, line 123.

Reviewer #3: (No Response)

--------------------

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: In general, the revised paper has improved.

Reviewer #3: (No Response)

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: Yes: Reza Goudarzi

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

References

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article's retracted status in the References list and also include a citation and full reference for the retraction notice.

12 May 2021

Submitted filename: Response to reviewers_R2 PLoS Neg_.docx

Click here for additional data file.

14 May 2021

Dear Dr Luangasanatip,

Thank you very much for submitting your manuscript "Implementation of Field Detection Devices for Antimalarial Quality Screening in Lao PDR – A Cost-Effectiveness Analysis" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you modify the manuscript according to the review recommendations.

Please prepare and submit your revised manuscript within 7 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Vahid Yazdi-Feyzabadi, PhD

Deputy Editor

PLOS Neglected Tropical Diseases

Vahid Yazdi-Feyzabadi

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

References

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article's retracted status in the References list and also include a citation and full reference for the retraction notice.

20 May 2021

Submitted filename: Response to reviewers_R2 PLoS Neg_.docx

Click here for additional data file.

28 May 2021

Dear Dr Luangasanatip,

Thank you very much for submitting your manuscript "Implementation of Field Detection Devices for Antimalarial Quality Screening in Lao PDR – A Cost-Effectiveness Analysis" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you modify the manuscript according to the review recommendations.

We did receive no response or any required change for below comments in last review.

Please kindly response to these comments and amend based on the reviewer comments.

1. I see that the authors added a paragraph on malaria incidence to the paper. However, I think that it should be in the introduction section rather than the methods section. Also, they should say 'incidence' rather than 'incidences.' I also think that they should add a sentence on the burden of malaria illness compared to other diseases and why it would be important to ensure that ACTs are of high quality and not SF.

2. The phrasing of some sentences is awkward. In several places, it would be preferable to use the term 'introducing' rather than 'implementing. - e.g. pg. 5, line 123.

Editor's Comments to Author

Please prepare and submit your revised manuscript within 30 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Vahid Yazdi-Feyzabadi, PhD

Deputy Editor

PLOS Neglected Tropical Diseases

Vahid Yazdi-Feyzabadi

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

We did receive no response or required change to below comments in last review.

Please kindly response to these comments and amend based on the reviewer comments.

I see that the authors added a paragraph on malaria incidence to the paper. However, I think that it should be in the introduction section rather than the methods section. Also, they should say 'incidence' rather than 'incidences.' I also think that they should add a sentence on the burden of malaria illness compared to other diseases and why it would be important to ensure that ACTs are of high quality and not SF.

The phrasing of some sentences is awkward. In several places, it would be preferable to use the term 'introducing' rather than 'implementing. - e.g. pg. 5, line 123.

Editor's Comments to Author

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

References

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article's retracted status in the References list and also include a citation and full reference for the retraction notice.

2 Jun 2021

Submitted filename: Response to reviewers_R3 PLoS Neg_final.docx

Click here for additional data file.

4 Jun 2021

Dear Dr Luangasanatip,

We are pleased to inform you that your manuscript 'Implementation of Field Detection Devices for Antimalarial Quality Screening in Lao PDR – A Cost-Effectiveness Analysis' has been provisionally accepted for publication in PLOS Neglected Tropical Diseases.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Vahid Yazdi-Feyzabadi, PhD

Deputy Editor

PLOS Neglected Tropical Diseases

Vahid Yazdi-Feyzabadi

Deputy Editor

PLOS Neglected Tropical Diseases

***********************************************************

23 Aug 2021

Dear Dr Luangasanatip,

We are delighted to inform you that your manuscript, "Implementation of Field Detection Devices for Antimalarial Quality Screening in Lao PDR – A Cost-Effectiveness Analysis," has been formally accepted for publication in PLOS Neglected Tropical Diseases.

We have now passed your article onto the PLOS Production Department who will complete the rest of the publication process. All authors will receive a confirmation email upon publication.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any scientific or type-setting errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript. Note: Proofs for Front Matter articles (Editorial, Viewpoint, Symposium, Review, etc...) are generated on a different schedule and may not be made available as quickly.

Soon after your final files are uploaded, the early version of your manuscript will be published online unless you opted out of this process. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Shaden Kamhawi

co-Editor-in-Chief

PLOS Neglected Tropical Diseases

Paul Brindley

co-Editor-in-Chief

PLOS Neglected Tropical Diseases