Diagnostic performance of a Recombinant Polymerase Amplification Test-Lateral Flow (RPA-LF) for cutaneous leishmaniasis in an endemic setting of Colombia.

BACKGROUND: Control of cutaneous leishmaniasis by public health systems in the Americas relies on case identification and treatment. Point-of-care diagnostics that can be performed by health workers within or near affected communities could effectively bring the health system to the resource-limited sites providing early diagnosis and treatment, reducing morbidity and the burden of disease. METHODOLOGY/PRINCIPAL
FINDINGS: A cross-sectional study was undertaken to evaluate the diagnostic test performance of Isothermal Recombinase Polymerase Amplification (RPA) targeting Leishmania kinetoplast DNA, coupled with a lateral flow (LF) immunochromatographic strip, in a field setting and a laboratory reference center. Minimally invasive swab and FTA filter paper samples were obtained by community health workers and highly trained technicians from ulcerated lesions of > 2 weeks' evolution from 118 patients' ≥ 2 years of age in the municipality of Tumaco, Nariño. Extracted DNA was processed by RPA-LF at a reference center or in a primary health facility in the field. Evaluation was based on a composite "gold standard" that included microscopy, culture, biopsy and real-time polymerase chain reaction detection of Leishmania 18S rDNA. Standard of care routine diagnostic tests were explored as comparators. Sensitivity and specificity of RPA-LF in the reference lab scenario were 87% (95%CI 74-94) and 86% (95%CI 74-97), respectively. In the field scenario, the sensitivity was 75% (95%CI 65-84) and specificity 89% (95%CI 78-99). Positive likelihood ratios in both scenarios were higher than 6 while negative likelihood ratios ranged to 0.2-0.3 supporting the usefulness of RPA-LF to rule-in and potentially to rule-out infection.
CONCLUSIONS/SIGNIFICANCE: The low complexity requirements of RPA-LF combined with non-invasive sampling support the feasibility of its utilization by community health workers with the goal of strengthening the diagnostic capacity for cutaneous leishmaniasis in Colombia. TRIAL REGISTRATION: ClinicalTrials.gov NCT04500873.

Introduction

Cutaneous leishmaniasis (CL) is a recognized public health challenge in the Americas, with an average of 55,000 cases per year between 2001–2018 in 17 countries of the region [1]. Most cases (79.4%) have been acquired in and continue to occur in rural areas [2]. Colombia is second in reported cases in Latin America and one of the ten countries worldwide reporting the highest number of cases [3]. In 2018, 6273 new cases of CL were reported in Colombia [1], and Tumaco was among the most affected municipalities [4]. Leishmaniasis control efforts rely on case identification and treatment, both of which are challenging in rural areas where access to health services is often scarce. Hence, one of the goals of the Pan American Health Organization for control of leishmaniasis in the Americas 2017–2022 is to improve the opportunity and access to diagnosis, treatment, rehabilitation and adequate follow-up of leishmaniasis cases [5]. According to Colombian national guidelines, provision of treatment requires parasitological diagnosis [6]. Sensitive, specific and accessible diagnostic methods are needed to meet this goal, to prevent progression to mucosal leishmaniasis, and to avoid overtreatment and potential adverse events in patients having disease due to other etiologies.

Currently available diagnostic methods for CL have several limitations. Light microscopic analysis of smears obtained from cutaneous lesions is the most commonly used diagnostic method for CL because of its low cost, yet its sensitivity varies widely depending on the experience and skill of the operator. In addition, sensitivity of microscopy is diminished in lesions of longer duration in chronic lesions with their notorious low parasite burden [7,8]. The combination of lesion smear, culture of lesion aspirate or biopsy and histopathology (25%–50% sensitivity) are diagnostic alternatives for chronic lesions [9], but their cost and restricted availability in reference centers, limit their use in rural settings. The Montenegro Skin Test is sensitive and specific but does not distinguish current from past infections [9-11], and since the requirement of production under standards of Good Manufacturing Practice, skin test antigen (leishmanin) is no longer available in the Americas.

A rapid test based on antigen detection for the diagnosis of CL at the point-of-care aimed principally at detection of Leishmania major and Leishmania tropica has been evaluated in the Old World, showing acceptable sensitivity (65%–68%) and specificity (80%–100%) [12,13]. Additionally, several molecular diagnostic tests have been developed for CL, achieving higher sensitivity (80% to 98%) and specificity (from 87% to 100%) than conventional diagnostic methods and establishing the feasibility of less invasive sampling [7,9,14-20].

Notwithstanding the challenges, the development of molecular tests for deployment where the disease occurs is a priority for improving access to care. Advances in non-invasive sampling for these molecular methods, such as lesion swabs, are particularly promising since they facilitate sample procurement for diagnosis in remote areas [15]. Nevertheless, the technical requirements and costs of sample processing by conventional or quantitative polymerase chain reaction (PCR) preclude their routine use in primary care facilities in resource-constrained settings.

Our team recently developed a method of Isothermal Recombinase Polymerase Amplification (RPA) targeting Leishmania kinetoplast DNA, coupled with a lateral flow (LF) immunochromatographic strip that has shown high accuracy in detecting Leishmania Viannia spp. [21]. This methodological approach allows sample processing and visual readout of results by naked eye using procedures amenable to local settings such as primary care centers. We reported the evaluation of the diagnostic performance of this RPA-LF test in a reference center laboratory, and in a field scenario where transmission is endemic, together with community participation.

Methods

Ethics statement

This research was approved and monitored by the Centro Internacional de Entrenamiento e Investigaciones Médicas (CIDEIM) Institutional Ethical Review Board (Approval number: 1275) in accordance with national and international regulations. Written Informed Consent was obtained from all participants or guardians of subjects <18 years of age. Assent was obtained from children ≥ 7 years of age. The study was registered under Clinical Trial Register NCT04500873. Medical decisions were based on the results of microscopic evaluation of lesion smear and culture, since at the time of the study, molecular methods were not yet recommended by the Colombian national guidelines for CL.

Study design and population

We conducted a cross-sectional study of diagnostic test performance between January 2018 and July 2019. The study was carried out at CIDEIM in the reference laboratory in Cali, Valle del Cauca (3°26’14.0"N 76°31’21.0"W) and its primary health care facility in the municipality of Tumaco, Nariño, Colombia (1°47’55.0"N 78°48’56.0"W). CL is endemic in Tumaco but not in Cali, where suspected CL cases arriving from or who have visited endemic areas are referred for diagnosis and treatment. Participants ≥2 years of age, with ulcerated skin lesions of more than two weeks’ duration were eligible and enrolled consecutively either in their residence (in rural Tumaco) by community health workers, or when seeking care at the primary health facility in urban Tumaco or the reference center laboratory in Cali. Sample size was calculated using a formula to estimate a proportion [22,23]. One hundred fifteen participants were obtained with an expected 95% sensitivity for RPA-LF, 4% error and confidence interval of 95%. It was adjusted by 5% loss of enrollment patients, reaching a final sample size of 121 participants [24].

We evaluated the performance of the RPA-LF test to diagnose CL considering two scenarios. 1) Reference laboratory scenario: samples were obtained by highly trained technicians (auxiliary nurses) in a Primary Health facility in the urban area of Tumaco, then sent to the reference center in Cali where samples were processed by an expert microbiologist. This scenario also includes samples obtained from subjects seeking care directly at the reference lab in Cali (Fig 1A and S1 Fig). 2) Field scenario: samples were obtained in rural areas of Tumaco by trained community health workers (CHW), sent to and processed by a non-expert lab technician in the primary health facility in Tumaco who was trained by the research team to perform RPA-LF (Fig 1B and S1 Fig). The gold standard was constituted by the combination of several tests: lesion smear evaluated by microscopy, culture, histopathology (when the results of the two previously mentioned tests were negative), and real-time polymerase chain reaction detection of Leishmania 18S rDNA (qPCR-18S). The same composite gold standard was used to evaluate the performance of RPA-LF test in both previously described scenarios. Since this composite gold standard is not usually available outside the research settings, additionally secondary evaluations were done with non-reference standard tests (Fig 1C and S1 Fig).

Fig 1

Schematic summary of patient enrollment, and subsequent, sampling, diagnostic procedures, and performance of RPA-LF evaluation in two scenarios.

A) Reference lab scenario: samples were obtained by highly trained field technician in Tumaco and processed in a reference center in Cali. In Cali samples were obtained and processed by an expert microbiologist. B) Field scenario: samples were obtained by trained community health workers (CHW), RPA-LF was processed by a non-expert field technician in Tumaco in primary health facility (PHF), and swabs were sent to reference center to be processed. C. Performance of RPA-LF test.

Data and clinical sample collection

Samples from enrolled patients were obtained by both the CHW at the place of residence in the rural area and by the trained technician at the primary health facility in the urban area of Tumaco, regardless of the site of initial enrollment.

Reference laboratory scenario

The study physician collected clinical and demographic information by electronic data capture. Direct smear and culture, swabs and Whatman FTA filter papers from ulcerated lesions were obtained by highly trained field technician according with PAHO and other guidelines [21,25]. The order of these procedures was randomly allocated (block randomization). When smear and culture were negative, and based on physician assessment, a lesion biopsy was obtained and histopathologic evaluation was also performed for differential diagnosis. All swabs and Whatman filter paper samples obtained in the primary health facility in Tumaco were transported to the CIDEIM reference laboratory in Cali. Swab samples were stored at −20°C and Whatman filter paper discs at room temperature. Then they were processed by an expert microbiologist using q-PCR-18S of rDNA and RPA-LF testing, respectively (Fig 1).

Field scenario

Ten CHW were selected to participate in the study according to the following criteria: being a member of and living in a rural community of the municipality of Tumaco, literacy, experience working in health, and acceptability by the community in accordance with WHO Guidelines for Community Health Worker Programmes [26,27]. CHW and two technicians of the primary health facility were trained using a structured program, with lectures and practical sessions. The training program included the following topics: ethics, basic concepts of CL and study procedures (research protocol, informed consent process, and collection, storage, and transportation of study samples). Additionally, field technicians were trained in the utilization of RPA-LF. Follow-up training at quarterly (3–4 month) intervals was carried out to re-enforce the technical capacity of the study team.

Samples were taken from the most recent ulcerated lesion in the following order: swabs for qPCR-18S and Whatman FTA filter paper for RPA-LF test. After cleaning and disinfecting the lesion and surrounding skin (iodized solution followed by 70% antiseptic alcohol and rinsing with sterile saline solution), two swab samples were obtained for qPCR-18S by gently rubbing over the surface of the ulcer ~10 times [15]. Afterwards, six 3 mm diameter Whatman FTA filter paper discs were applied to the lesion to absorb the tissue fluid and cells present on its surface. All samples (Whatman FTA filter paper and swabs) were stored and transported at room temperature to the local laboratory of the primary health facility in Tumaco where they were processed (Fig 1). Patients were also referred to this primary health facility for other laboratory tests within 7 days of the sampling procedure.

Reference tests

Composite gold standard

The “gold standard” for diagnosis of cutaneous leishmaniasis is visualization of amastigotes or isolation of Leishmania, yet individual methods to achieve this are not highly sensitive while molecular tests particularly those that amplify genetic material of Leishmania surpass the sensitivity of parasitological tests. For this reason, we utilized a composite “gold standard” based on microscopy of lesion smear, culture, histopathology of biopsy and qPCR18S as a reference. Additionally, recognizing that at the point-of-care such a standard is not feasible, we also conducted comparative analyses of the performance of RPA-LF in relation with individual routinely used diagnostic tests: 1) microscopy of lesion smear, 2) direct smear and culture of aspirates 3) together with histopathology of biopsy and 4) qPCR-18S alone (Fig 1).

In accordance with the recommendations of the FDA for comparisons of new diagnostic tests with measures other than the “gold standard”, comparisons with routine diagnostic procedures were made based on percent positive and negative agreement [28].

Smear, culture and histopathology

Microscopic evaluation of the standard direct smear was performed. A smear was considered positive when at least one intra- or extracellular amastigote was observed by microscopy. A result was negative when amastigotes were not observed in any of the examined fields. Four aspirates were obtained from the lesion border using a tuberculin syringe and 27G needle. Aspirates were cultured in Senekjie’s diphasic culture medium [8] and parasite growth was evaluated at weekly intervals for up to one month. Histopathology was considered positive when amastigotes were visualized or the inflammatory pattern suggested leishmaniasis. All of these procedures were performed following the guidelines of PAHO [25]. Leishmania strains isolated from cutaneous lesions were identified using monoclonal antibodies [29]. Strains that were not identified with monoclonal antibodies, were analyzed by isoenzyme electrophoresis for species identification [30,31].

DNA extraction and molecular amplification of Leishmania 18S rDNA

DNA for qPCR of 18SrDNA was extracted from samples using Qiagen DNeasy Blood & Tissue Kit (Qiagen, USA) according to manufacturer’s protocol. The resultant DNA was eluted in 50 μL AE buffer. Extraction controls without DNA samples were included to verify the absence of contamination during this process. The quantity and quality of nucleic acids was evaluated using a NanoDrop™ spectrophotometer. All DNA samples were stored at −20°C until processing [15]. qPCR amplification of 18S r-DNA from Leishmania spp. was performed in a total volume of 12.5 μL (1.25 μL of total sample, 6.25 μL PCR Mastermix (BioRad), 1 μM of each oligonucleotide primer and 0.25 μM of the Leishmania 18S rDNA-specific FAM-labelled TaqMan probe and 2.75 μL Nuclease-Free Water [32]). The qPCR was carried out in the reference laboratory by an expert microbiologist. Comparisons between experiments were made using a standard curve for 18S r-DNA amplification of L. (V) panamensis DNA. Negative and positive controls were included in each PCR assay [15,33].

Recombinase Polymerase Amplification Lateral Flow (RPA-LF)

The RPA-LF was conducted according to the methodology described by Saldarriaga and collaborators [21]. DNA extraction: 3 mm diameter disks were transferred to a 1.5 mL Eppendorf microcentrifuge tube using clean forceps, then washed three times for 5 minutes each at room temperature with 200 μL FTA Purification Reagent. Subsequently, the disks were washed once with 200 μL TE buffer (10 mM Tris-HCl, 1 mM EDTA) for 5 minutes at room temperature. After discarding the TE buffer, the disks were resuspended in 50 μL TE buffer and placed in a heat block at 95°C for 30 minutes. The supernatant containing DNA was transferred to a prelabelled 1.5 mL microcentrifuge Eppendorf tube from which aliquots were drawn for molecular amplification. In this study we utilized the primer sequences and probe described by Saldarriaga et al (2016) [21]. The amplification mixture contained the forward primer-(5μM), biotinylated reverse primer (5μM), FAM-labeled probe (5μM), and the rehydrated cocktail (TwistAmp nfo RPA kit -TwistDx, UK). Two microliters of DNA extracted from the clinical sample and 1.25μL magnesium acetate 280nM were added to the mixture and amplified at 42°C for 40 minutes using a dry bath. Then, 2μL of the RPA product were added to 98 μL of dilution buffer in a 1.5 Eppendorf tube and the lower edge of the lateral flow strip (Ustar Biotechonologies Hangzhou Ltda, China) was immersed in the solution. The amplification product migrated upwards by capillarity. Parasite DNA amplification was visually confirmed within 10 minutes by the appearance of the corresponding 2 red bands in the lower portion of the strip (Fig 2). Negative and positive controls were included in each amplification batch [21]. The same procedure was used by the technician at the primary health facility in Tumaco and the expert microbiologist in the reference lab in Cali.

Fig 2

Results of RPA-LF test

Statistical analysis

We performed a descriptive analysis for clinical, socio-demographic and lesion characteristics. Acute CL was defined as individuals having lesions of <6 months of evolution and chronic those with lesions ≥ 6 months. We defined true positives based on at least one of the composite gold standard tests (smear, culture, histopathology or q-PCR-18S) and RPA-LF being positive. A result was considered true negative when all tests were negative. Similar definitions were used for each comparator Table 1.

Table 1

Definitions of true positives and negatives according to reference tests.

Reference tests	True positive	True negative
Composite gold standard (smear, culture, histopathology and q-PCR-18S) vs. RPA-LF	At least one of the constituent tests of composite gold standard and RPA-LF were positive	All composite gold standard tests and RPA-LF were negatives
Smear vs. RPA-LF	Smear and RPA-LF tests were both positive	Smear and RPA-LF were both negative
Smear and culture vs. RPA-LF	At least one test and RPA-LF were positive	Smear and culture were negatives and RPA-LF was negative
Smear, culture, and histopathology vs. RPA-LF	At least one test and RPA-LF were positive	At least two reference tests and RPA-LF were negative
qPCR-18S vs. RPA-LF	q-PCR-18S and RPA-LF test were both positive	q-PCR-18S and RPA-LF were both negative

When results evidenced contamination, double sequential RPA reactions were conducted using an aliquot of the first amplification for a second RPA. Samples of 29 participants, 25 from the primary health facility and 4 from the reference laboratory, were repeated due to contamination of the negative control, which invalidated the batch of samples. In this study, results were considered equivocal as defined by FDA guidelines [28].

We calculated sensitivity, specificity, predictive values (positive and negative) and likelihood ratios (positive and negative) with the corresponding 95% confidence intervals for both scenarios, reference lab and field separately [34]. A McNemar test for paired data was used to identify differences in sensitivity and specificity and predictive values between reference lab and field scenarios [35-37]. Likelihood ratios were compared using the Differences in Diagnostic Likelihood Ratios Test, a P-value <0.05 was considered as statistically significant. Additionally, RPA-LF was compared with other test non- gold reference standard, estimating the positive and negative percent agreement with its confidence intervals in both scenarios following the FDA guideline [28]. Differences between reference laboratory and field were calculated using the McNemar test. Finally, sources of heterogeneity in the sensitivity of RPA-LF test compared with the composite gold standard were estimated using stratified analysis by sex, ethnicity, age, previous episode of leishmaniasis, medications received, number of lesions and duration of the oldest lesion. Stata, version 12 and DTCompair R software were used for these analyses.

Results

Study participants and gold standard results

After the initial enrollment of 128 CL suspected cases by the CHW, the physician at the primary care facility excluded ten participants due to lesion healing (n = 7) at the time of consultation or consent withdrawal (n = 3) (Fig 3 and S2 Fig).

Fig 3

Flow chart of participants by scenario.

Most participants were 18 years of age or older (78%), male (70.3%) and Afro-Colombians (62.7%). There was a wide range in the number of ulcers per patient (1–11), but single lesions were the most common presentation (53.3%). Most of the skin lesions (94.3%) were small ulcers (median: 15 mm diameter) localized on the arms (42%) or legs (37.6%). At the time of enrollment, the majority of lesions (89.3%) had less than six months of evolution. Only 5% of patients had a previous episode of CL as determined by the presence of typical scar or by clinical history Table 2 and S1 and S2 Tables.

Table 2

Clinical and demographic characteristics of study participants.

Characteristics	n = 118
Demographic
Sex, n (%)
Male	83	(70.3)
Female	35	(29.7)
Ethnicity, n (%)
Afro-Colombian	74	(62.7)
Mestizo	29	(24.6)
Indigenous	15	(12.7)
Age, median (range), years	26	(2–85)
Department (state) of origin, n (%)
Nariño	106	(89.8)
Valle del Cauca	7	(6.0)
Others	5	(4.2)
Clinical
Previous episode of leishmaniasis, n (%)	6	(5.1)
Leishmania species, (n = 69), n (%)
L.(V) panamensis	64	(94.8)
L.(V) braziliensis	5	(7.2)
Lesions (n = 245)
Number of lesions. Median (range)	1	(1–11)
Duration of the oldest lesion, months, n(%)
0–5.9	106	(89.8)
≥ 6	12	(10.2)
Type of lesions, n (%)
Ulcer	231	(94.3)
Plaque	10	(4.1)
Other	4	(1.6)
Location in the body
Arms	103	(42)
Legs	92	(37.6)
Face-neck	26	(10.6)
Trunk	24	(9.8)
Presence of satellite lesions, n (%)	33	(13.5)
Presence of lymphadenopathy, n (%)	13	(5.3)
Maximum diameter of lesions (mm). Median (Range) (n = 236)	18.6	(3.2–78.4)
Maximum diameter of ulcers (mm). Median (Range) (n = 227)	15	(0.9–99.1)

The standard of care diagnostic protocol in Colombia is based on Giemsa-stained smears from skin lesions complemented by biopsy and histopathology when lesion smears are negative by microscopy. Seventy-nine patients presented positive smears, 69 were positive by culture and only one patient required confirmation by histopathology for a total of 80/118 (67.8%) patients being parasitologically confirmed. Additionally, q-PCR-18S identified three more cases, establishing that 83 (70.3%) of enrolled patients were positive for Leishmania spp. Among the 35 participants who were negative by the composite gold standard, 18 were diagnosed as having soft tissue infections by the treating physician hence biopsy and histopathology were not performed; the remaining 17 were evaluated by histopathology. The diagnoses achieved by histopathology were vascular diseases (n = 12 71%), carcinoma (n = 3 17%) and soft tissue infections (n = 2 12%). Sixty-nine positive cultures allowed the identification of L. (V.) panamensis as the most frequent species (n = 64 94.8%) and the rest were L. (V.) braziliensis Table 2 and S1 Table.

Performance of RPA-LF test

We determined that the RPA-LF test was capable of confirming or ruling out CL in both the lab reference center and field scenario using the composite gold standard as comparator. In the reference laboratory scenario, the sensitivity of RPA-LF was significantly higher at 87% (95%CI 79–94) than in the field scenario at 75% (95%CI 65–84), p = 0.04. This suggests that technical or logistic factors may have diminished test efficacy in the field. Specificity was similar in both scenarios: 86% (95%CI 74–97) and 89% (95%CI 78–99). Positive likelihood ratios were higher than 6, consequently RPA-LF is a good test to confirm CL in the reference center and field scenario. Likewise, negative likelihood ratios were between 0.2–0.3 showing that this test could be useful to discard CL Table 3 and S3 Table.

Table 3

Diagnostic performance of RPA-LF in reference laboratory and field scenarios compared with composite gold standard (n = 118).

Scenario	TP	FP	FN	TN	Sensitivity	p	Specificity	p	PPV	p	NPV	p	LR+	p	LR-	p
					% (95% CI)		% (95% CI)		% (95% CI)		% (95% CI)		% (95% CI)		% (95% CI)
Reference laboratory	72	5	11	30	87	0.04*	86	0.7*	94	0.9£	73	0.06£	6.1	0.90	0.2	0.7¶
					(79–94)		(74–97)		(88–99)		(60–87)		(2.7–13.7)		(0.1–0.3)
Field	62	4	21	31	75		89		94		60		6.5		0.3
					(65–84)		(78–99)		(88–100)		(46–73)		(2.6–16.6)		(0.2–0.42)

PPV: Positive predictive value. NPV: Negative predictive value. LR: Likelihood Ratio.

* McNemar Test.

Relative Predictive Values Test.

Differences in Diagnostic Likelihood Ratio Test

Comparison of the RPA-LF test with standards other than the composite gold standard that are used at the point-of-care to diagnose CL, reveal the positive percent agreement was significantly higher in the reference laboratory scenario than field scenario where agreement ranged between 85%–90% and 74–77% respectively. Negative percent agreement was lower than positive, and results for the reference laboratory and field were similar Table 4 and S4 Table.

Table 4

Agreement between RPA-LF test and other diagnostic tests.

Diagnostic test	RPA-LF scenario	Positive percent agreement			Negative percent agreement
		n/N	% (95%CI)	p*	n/N	% (95%CI)	p*
Smear	Reference lab	67/71	94 (89–100)	0.007	28/33	85 (72–97)	0.3
	Field	55/71	77 (67–87)		25/33	75 (61–90)
Smear + Culture	Reference lab	72/79	91 (84–98)	0.02	34/39	87 (76–98)	1
	Field	61/79	77 (68–87)		34/39	87 (76–98)
Smear + Culture + histopathology	Reference lab	72/80	90 (83–96)	0.04	33/38	86 (76–98)	0.7
	Field	62/80	77 (68–87)		34/38	89 (79–99)
qPCR-18S	Reference lab	63/74	85 (77–93)	0.08	30/44	68 (54–82)	0.3
	Field	55/74	74 (64–85)		33/44	75 (62–88)

* McNemar Test

Sources of variation, Compliance, and adverse events, of RPA-LF test

Sensitivity of RPA-LF in the reference laboratory was similar across of stratified analysis by sex, age, ethnicity, previous episode of leishmaniasis, medication received within the last month and, number of lesions. Most of variables had sensitivity variations between strata of less than 15%. The most relevant factor was duration of the lesion, which presented differences in sensitivity higher than 20%. Sensitivity of RPA-LF for lesions <6 months was 87.5% while lesions > 6 months, 66.7%. S5 Table.

All samples were obtained according to the protocol and no modifications were introduced to the RPA-LF test throughout the course of the study. The seven-day interval between sampling by CHW and at the primary health facility was met in 91% of participants. None of the patients enrolled in the study presented adverse events related with the sampling procedures performed at the local clinic in Tumaco or by the community health workers in the rural areas.

Discussion

This study evaluated the diagnostic performance of RPA-LF test in two scenarios, a primary health facility in a CL endemic area and a laboratory reference center. The test was compared against a composite gold standard that encompassed microscopy, culture, histopathology and qPCR. Use of a combination of highly sensitive diagnostic methods as gold standard sets a “high bar” for the comparative performance of RPA-LF, underrepresenting the benefit of this technology for endemic settings.

Results provided evidence that RPA-LF test is a valid test to confirm or rule out CL in a reference laboratory or field scenario. In a reference laboratory center the RPA-LF test accurately detected 87% of infections identified by the composite gold standard, which was the most stringent comparison. Sensitivity of the RPA-LF test was similar to other molecular tests that achieved values ranging from 81%– 98.7% [15,20].

This is the first study evaluating the performance of the RPA-LF test in which sampling was performed by CHW and processed in a primary health care “field scenario”. Among the performance criteria, only the sensitivity of RPA-LF (75%) was significantly lower than the reference laboratory scenario. The factors responsible for this difference still need to be determined, however we believe RPA-LF sensitivity could be improved by further simplification of the process, (e.g. reducing number of steps and materials involved).

During training sessions, we observed that manipulating the small filter paper discs (3 mm diameter) was challenging for the CHW. We consider that this difficulty could have affected, in part, the sensitivity of RPA-LF and that further improvement of the sampling protocol (i.e. by using swabs or larger filter paper discs) could optimize this point-of-care diagnostic test.

Differences in sensitivity between scenarios could also be attributed to the subjective reading of weak bands of the lateral flow strips when loaded with low concentrations of amplification products. Consequently, we believe that RPA-LF accuracy could be improved by adapting an inexpensive digital system that would add objectivity to the readout of results.

Some samples presented contamination. Yet, the areas of highest contamination risk are those found in labs where daily PCR work produce large numbers of amplicons. In the field, sampling with disposable materials offer lower risk of contamination. However, the local clinic should be aware of amplicons and establish separate areas for obtaining samples, extracting DNA and running the molecular test.

The simplicity of sample procurement using filter paper discs or swabs allowed trained CHW to obtain samples at the patients’ homes, avoiding the need of costly transportation to the clinic, thereby overcoming a key barrier to diagnosis of CL. Furthermore, the implementation of noninvasive sampling is an advantage for all patients and particularly pediatric patients, who represented 9% of the study population and are even more frequent in other areas and settings.

Several studies have shown the importance of CHW participation in the diagnosis of other infectious diseases in areas with limited access to health services [38,39]. The benefit of our approach was that suspected patients were proactively identified by CHWs and samples were obtained and delivered to the local clinic for diagnosis. This strategy demonstrated its potential to reduce the health equity gap in hard-to-reach areas of transmission.

Regarding the comparison of RPA-LF with non-gold standard diagnostic tests that are used at the point-of-care to diagnose CL, we found a high (>85%) positive and negative agreement between them (smear, histopathology and culture in a research center). RPA-LF was capable of diagnosing CL in resource-limited settings, reaching or surpassing the individual and combined sensitivity of microscopy, culture and histopathology (≤80%), which are the most widely used diagnostic methods [9,18-20,40-42]. Additionally, the sensitivity of RPA-LF was higher than the sensitivity of lesion smears which is generally reported to be <60% [8,18,43-45].

Positive and negative agreements decreased when RPA-LF was compared to qPCR-18S, which is expected given that the qPCR is a real-time molecular test performed under ideal conditions. However, the RPA-LF test has practical advantages over qPCR-18S or other molecular methods because it is easier to perform, does not require expensive equipment, extensive training or sophisticated health infrastructure [14,40,46]. Other qualitative isothermal amplification methods, e.g. loop mediated amplification (LAMP), have shown a range of sensitivities for detecting Leishmania species [47-49]. Nevertheless, a drawback of qualitative LAMP is its dependency on indirect detection methods like turbidity or non-specific dyes that could potentially lead to false positive results [50].

Study strengths included randomization of the order in which samples were obtained avoiding potential bias that could have influenced RPA-LF performance. Results of RPA-LF were maintained across the different sources of variation such as sex, age, ethnicity, previous episode of leishmaniasis, number of lesions, ethnicity, age, receipt of medication, number of lesions. Duration of disease was the most relevant variable that can modify the sensitivity of the test.

This study has some limitations. First, the RPA-LF process was not blinded and the operator could have known the smear results in advance, potentially influencing the interpretation of lateral flow reading. Second, not all patients with negative smear were evaluated by histopathology, and hence could have been misclassified as CL negative. Third, CL cases were mostly due to L. (V.) panamensis, consequently the efficacy of RPA-LF to detect other species within the subgenus Viannia requires additional evaluations. However, L panamensis and L braziliensis are the most relevant species in Colombia and the capacity of RPA-LF to detect other species of the subgenus Viannia has been previously determined [46]. People who migrated from other regions of Colombia may harbor other kinetoplastid parasites like L. infantum, T. cruzi or T. rangeli but the RPA-LF test does not cross-react with these parasites [21].

The sampling methods for RPA-LF or qPCR did not result in any adverse events. However, noninvasive samples can be obtained only from ulcerated lesions, limiting their usefulness in non-ulcerated and atypical forms of CL. It is possible that minimally invasive sampling using a fine needle (27G) aspirate and transfer of material to filter papers could overcome this limitation.

Recently, researchers affiliated with DNDi, FIND and WHO proposed the target product profile for a point-of-care CL diagnostic test as a “simple and robust test that can be implemented in resource-limited settings, enabling decentralized diagnosis and treatment of dermal leishmaniasis” [51]. Our results indicate that RPA-LF meets the principal characteristics of the product profile of a point-of-care test since they demonstrated its potential to improve access to CL diagnosis in resource-limited settings. Further studies should focus not only on test performance, but also on the operational requirements, costs, and relevant clinical outcomes. Utilization of this test should lead to a significant increase in the number of patients initiating treatment, and reduction of the time between appearance of disease and treatment [52-55].

In conclusion, RPA-LF is a valid, efficacious test to diagnose CL that could replace or complement microscopy in rural areas. Its combination with non-invasive sampling and low complexity requirements for processing and interpreting results position this test as an achievable alternative for diagnosis of CL at points of care in Colombia.

Sampling and results of diagnostic procedures performed in reference laboratory and field scenarios.

(CSV)

Click here for additional data file.

Results of diagnostic procedures, RPA-LF test in reference laboratory and field as well as results of composite gold standard.

(CSV)

Click here for additional data file.

(CSV)

Click here for additional data file.

Description of the study participants’ lesions.

(CSV)

Click here for additional data file.

RPA-LF results in reference laboratory and field scenarios, as well as data of composite gold standard.

(CSV)

Click here for additional data file.

(CSV)

Click here for additional data file.

Sensitivity of RPA-LF in the reference laboratory scenario stratified by sociodemographic and clinical characteristics.

(DOC)

Click here for additional data file.

8 Jan 2021

Dear Mrs. Cossio,

Thank you very much for submitting your manuscript "Diagnostic performance of a Recombinant Polymerase Amplification Test - Lateral Flow (RPA-LF) for cutaneous leishmaniasis in an endemic setting of Colombia" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you modify the manuscript according to the review recommendations.

Please prepare and submit your revised manuscript within 30 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Philippe Büscher, PhD

Associate Editor

PLOS Neglected Tropical Diseases

Epco Hasker

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: (No Response)

Reviewer #2: This paper describes in details a relatively new technique of combining RPA and latter flow to develop a POC device, in this case for the detection of Leishmania.

The methodology of the study is very well described, and the graphic representation of the study design truly helps to grasp the context of the paper.

Just for the sake of clarity, a few additional details should be added, so that the paper becomes completely clear and will be used a a general refence by others in the future.

1) Line 277: please include primer sequences in the M&M section

2) Line 315 calculation methods (formulas) for sensitivity, specificity and positive / negative predictive should be added, taken that it is not always clear which golden standard is used exactly in the calculation. This reviewer was very pleased by the fact that PPV and NPV were include, something that is very often omitted in experimental diagnostic research papers despite the fact that these are the values that determine if a clinician can 'trust' the test in the field (while sensitivity and specificity are theoretic values that allow very easy masking of serious issues). So, well done !

The reason for this question: see below, result section.

Reviewer #3: lines 148-150 The authors could clarify for which hypothesis and power this sample size is calculated.

line 253 Are 'the samples' the swabs for qPCR?

--------------------

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: (No Response)

Reviewer #2: Very clear presentation of results.

Just one question for additional explanation...

Line 369/370: the difference in 'results' obtained in the lab versus the field. Details are outlined in Table 3.

From these results it is clear that the main differences are TP (72 vs 62) and FN (11 vs 21). of course it is obvious that field analysis will never perform as good as good as a lab context, but the question here is how did these 2 difference occur. By providing the formula used to calculate these values, and indicating which factor contributed to the decrease of TP and increase of FN, the work could become a textbook example of how to go about doing these sort of things. Again, the quality of the work, the methodology and paper is outstanding, and by adding this info, 'I' would even be temped to use this paper as a textbook/lecture example in advanced molecular parasitology classes.

Reviewer #3: lines 334-336 If the sample size was estimated to be 118, why did the authors include 128 patients?

--------------------

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: (No Response)

Reviewer #2: The conclusion is very well written, and this reviewer specially appreciated the critical assessment of the limitations of the study. Again, this is something that is very often missing from so many diagnostic papers, and it just adds to the value of this particular study.

One last question however, RPA-LFA is not the cheapest technique, and by adding a 'cheap' readout tool in the future, the 'practicality' of the cost could further be an issue. Would it be possible for the authors just to add a few details on the current cost of the RPA-LFA (something that should be similar to RPA-LFAs for other diseases such as trypanosomosis) versus the cost of the 'current' most used golden standard in the field. the latter will be very different depending on the disease studied. It's not crucial for the quality of the paper, but it just allows the reader to think about the bigger picture and out everything in perspective.

Reviewer #3: line 433 - 443 Could sensitivity have been impacted by first doing swab sampling for qPCR and only then filter disks sampling for RPA-LF?

line 445 - 449 Is the misinterpretation of banding patterns also linked to the <90% specificity of RPA-LF in both field and ref lab?

--------------------

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: (No Response)

Reviewer #2: See other sections

Reviewer #3: line 59 remove the '('

line 63 duplicated 'in'

line 161 correct 'performed' to 'perform'

line 351 duplicated 'in'

line 397 correct 'Ddiagnotic'

--------------------

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: The authors had previously established a RPA-LF protocol for CL diagnosis (Ref 21). Here, they successfully evaluate its efficiency and usefulness in field conditions by following a rigorous algorithm with multiple comparisons to an exhaustive panel of diagnostic tests (including a challenging composite gold-standard). This is an excellent work, with some real efforts to present the analytic pipelines and their corresponding results in a simple yet precise manner. The manuscript is well-written, the methodology is very rigorous and the discussion is adapted to the results. The results are positive and convincing and they will certainly be of interest for people involved in the control of CL, and possibly in the context of other NTDs for which diagnostic tools require improvement. I only have minor comments and suggestions below.

Minor comments:

• L.33: in the field?

• L.63: in both

• L.225: Leishmania in italic

• L.308-313: The relatively frequent occurrence of contaminations during the RPA protocol, especially in the field, could be explained a bit more in details and discussed in the appropriate section (possible reasons for such contaminations and ways to avoid it in the future).

Reviewer #2: Very clear paper.

Reviewer #3: (No Response)

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Brice Rotureau

Reviewer #2: No

Reviewer #3: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, PLOS recommends that you deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see http://journals.plos.org/plosntds/s/submission-guidelines#loc-materials-and-methods

16 Feb 2021

Submitted filename: Response to reviwers.docx

Click here for additional data file.

5 Mar 2021

Dear Mrs. Cossio,

We are pleased to inform you that your manuscript 'Diagnostic performance of a Recombinant Polymerase Amplification Test - Lateral Flow (RPA-LF) for cutaneous leishmaniasis in an endemic setting of Colombia' has been provisionally accepted for publication in PLOS Neglected Tropical Diseases.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Philippe Büscher, PhD

Associate Editor

PLOS Neglected Tropical Diseases

Epco Hasker

Deputy Editor

PLOS Neglected Tropical Diseases

***********************************************************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: (No Response)

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: (No Response)

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: (No Response)

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

Reviewer #3: (No Response)

**********

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: All suggestions and corrections were taken into account in the revised version.

Reviewer #2: Thank you for addressing the comments made during the first round of reviewing.

Reviewer #3: No further questions.

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Brice Rotureau

Reviewer #2: No

Reviewer #3: No

29 Mar 2021

Dear Mrs. Cossio,

We are delighted to inform you that your manuscript, "Diagnostic performance of a Recombinant Polymerase Amplification Test - Lateral Flow (RPA-LF) for cutaneous leishmaniasis in an endemic setting of Colombia," has been formally accepted for publication in PLOS Neglected Tropical Diseases.

We have now passed your article onto the PLOS Production Department who will complete the rest of the publication process. All authors will receive a confirmation email upon publication.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any scientific or type-setting errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript. Note: Proofs for Front Matter articles (Editorial, Viewpoint, Symposium, Review, etc...) are generated on a different schedule and may not be made available as quickly.

Soon after your final files are uploaded, the early version of your manuscript will be published online unless you opted out of this process. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Shaden Kamhawi

co-Editor-in-Chief

PLOS Neglected Tropical Diseases

Paul Brindley

co-Editor-in-Chief

PLOS Neglected Tropical Diseases