Performance of the Access Bio/CareStart rapid diagnostic test for the detection of glucose-6-phosphate dehydrogenase deficiency: A systematic review and meta-analysis.

BACKGROUND: To reduce the risk of drug-induced haemolysis, all patients should be tested for glucose-6-phosphate dehydrogenase (G6PD) deficiency (G6PDd) prior to prescribing primaquine (PQ)-based radical cure for the treatment of vivax malaria. This systematic review and individual patient meta-analysis assessed the utility of a qualitative lateral flow assay from Access Bio/CareStart (Somerset, NJ) (CareStart Screening test for G6PD deficiency) for the diagnosis of G6PDd compared to the gold standard spectrophotometry (International Prospective Register of Systematic Reviews [PROSPERO]: CRD42019110994). METHODS AND
FINDINGS: Articles published on PubMed between 1 January 2011 and 27 September 2019 were screened. Articles reporting performance of the standard CSG from venous or capillary blood samples collected prospectively and considering spectrophotometry as gold standard (using kits from Trinity Biotech PLC, Wicklow, Ireland) were included. Authors of articles fulfilling the inclusion criteria were contacted to contribute anonymized individual data. Minimal data requested were sex of the participant, CSG result, spectrophotometry result in U/gHb, and haemoglobin (Hb) reading. The adjusted male median (AMM) was calculated per site and defined as 100% G6PD activity. G6PDd was defined as an enzyme activity of less than 30%. Pooled estimates for sensitivity and specificity, unconditional negative predictive value (NPV), positive likelihood ratio (LR+), and negative likelihood ratio (LR-) were calculated comparing CSG results to spectrophotometry using a random-effects bivariate model. Of 11 eligible published articles, individual data were available from 8 studies, 6 from Southeast Asia, 1 from Africa, and 1 from the Americas. A total of 5,815 individual participant data (IPD) were available, of which 5,777 results (99.3%) were considered for analysis, including data from 3,095 (53.6%) females. Overall, the CSG had a pooled sensitivity of 0.96 (95% CI 0.90-0.99) and a specificity of 0.95 (95% CI 0.92-0.96). When the prevalence of G6PDd was varied from 5% to 30%, the unconditional NPV was 0.99 (95% CI 0.94-1.00), with an LR+ and an LR- of 18.23 (95% CI 13.04-25.48) and 0.05 (95% CI 0.02-0.12), respectively. Performance was significantly better in males compared to females (p = 0.027) but did not differ significantly between samples collected from capillary or venous blood (p = 0.547). Limitations of the study include the lack of wide geographical representation of the included data and that the CSG results were generated under research conditions, and therefore may not reflect performance in routine settings.
CONCLUSIONS: The CSG performed well at the 30% threshold. Its high NPV suggests that the test is suitable to guide PQ treatment, and the high LR+ and low LR- render the test suitable to confirm and exclude G6PDd. Further operational studies are needed to confirm the utility of the test in remote endemic settings.

Introduction

Radical cure of Plasmodium vivax and P. ovale malaria requires killing of both the blood and liver stages of the parasite to prevent relapsing malaria and reduce ongoing transmission [1]. Primaquine (PQ) has been used for over 65 years and is currently the only widely available hypnozoitocidal drug for P. vivax and P. ovale. PQ has to be administered in combination with a blood schizontocidal agent over 7 to 14 days to clear hypnozoites [2-6]. While PQ is tolerated in most patients, it can cause haemolysis in patients with glucose-6-phosphate dehydrogenase deficiency (G6PDd), the severity of which is dependent on the underlying genetic variant, the dose of PQ administered, and the age of the patient’s red blood cell (RBC) population [7,8].

To date, 215 genotypes conferring different degrees of G6PDd have been described, and these are most prevalent in areas of past and present malaria endemicity [9-Blood Cells Mol Dis. 2012 ">11]. The G6PD gene is located on the X chromosome (Xq28), therefore males are either hemizygous G6PD deficient or G6PD normal, whereas females can be homozygous G6PD deficient, G6PD normal, or heterozygous for the gene. In heterozygous females, one copy of the G6PD gene is randomly inactivated through a process called lyonization; accordingly, heterozygous females harbour 2 distinct groups of RBCs, a G6PD normal and a G6PD-deficient one [12]. Depending on the ratio of G6PD-normal to G6PD-deficient RBCs, heterozygous females may be at a risk of severe drug-induced haemolysis [13,14].

To reduce the risk of drug-induced haemolysis, WHO recommends that patients be tested routinely for G6PDd prior to administration of PQ-based radical cure [4]. The gold standard method for measuring G6PD activity is quantitative spectrophotometry [15,16], but this method is expensive and requires laboratory facilities that are often unavailable in malaria-endemic communities, especially in remote areas. The fluorescent spot test (FST) is a qualitative alternative; however, it also requires laboratory infrastructure and extensive training for reliable interpretation [17,18]. In 2011, Access Bio (Somerset, NJ) introduced a qualitative, lateral-flow point-of-care assay (CareStart screening test for G6PDd; CSG) [19]. The aim of this article was to undertake a meta-analysis of published studies to determine the performance of the assay in a variety of populations at risk of drug-induced haemolysis (International Prospective Register of Systematic Reviews [PROSPERO]: CRD42019110994).

Methods

Search strategy and eligibility criteria

A PubMed search was undertaken for relevant articles published in English between 1 January 2011, when the test was first introduced [19], and 27 of September 2019. The search terms applied were “G6PD AND (rapid diagnostic test OR carestart)”. Identified articles were first screened for eligibility by title, abstract, and then by the full text by 3 study authors (BL, AWS, and HR) independently. Reference sections of identified articles were screened for additional relevant articles. Eligible articles reported performance indicators of the CSG from samples collected prospectively. Articles describing prototypes of the CSG were excluded. Only studies comparing the CSG results to the gold standard spectrophotometry, using kits from Trinity Biotech PLC (Wicklow, Ireland), were included. Studies were included irrespective of whether blood was collected from capillary or venous sampling.

Corresponding authors of identified articles were contacted and asked to provide anonymized individual participant data (IPD). All corresponding authors were contacted a minimum of 3 times before the study was excluded. Minimal data requested included the sex of the participant, CSG result, spectrophotometry result in U/gHb, and corresponding haemoglobin measurement in U/dL. Data were entered into a customized Excel database (Microsoft Corporation, Redmond, WA) and analysed using Stata software version 14 (release 14; StataCorp, College Station, TX). Analysis was done primarily using the Midas package.

Data preparation

Invalid CSG results were excluded from the analysis. Spectrophotometry results that were missing or extreme (>25 U/gHb) were excluded from analyses because these readings suggested a procedural or data error. Some studies reported an intermediate CSG result; in clinical use, these are more likely to be considered G6PDd results and were defined accordingly. One article reported the results of 2 separate evaluation studies from Laos and Cambodia [20]; because the applied cut-off activities and reported performance were distinct for each country, the results are reported separately.

The adjusted male median (AMM) was calculated from spectrophotometry results separately for each study site and defined as 100% G6PD activity [16]. Because some studies applied different definitions of 100% G6PD activity (for example, by considering genotype [21]), the definitions within this study and the original source articles may sometimes differ. Studies reported spectrophotometry results either from venous and/or capillary blood, and the source of blood could have affected spectrophotometry measurements. One study measured G6PD activity in paired capillary and venous samples by spectrophotometry [21], and the results were compared for significant differences using the Wilcoxon signed-rank test.

Spectrophotometry provides a quantitative result; following the current informal cut-off to guide PQ-based radical cure [15], and the intended cut-off of the CSG [17,20], any sample with less than 30% of the AMM was defined as G6PDd. Study-specific performance was calculated following standard formulae [16,22,23], by comparing the CSG against the reference method spectrophotometry. A positive result was defined as a G6PDd outcome and a negative result as a G6PD normal outcome. Results from the CSG were then classified as true positive (TP), true negative (TN), false positive (FP), and false negative (FN) with reference to the results of spectrophotometry.

Data analysis

To calculate the pooled estimates for sensitivity and specificity, a 2-level model with independent binomial distributions was fitted for the TPs and TNs conditional on the sensitivity and specificity in each study, and a bivariate normal model for the logit transformations of the sensitivity and specificity between the studies was created [24]. A summary receiver operator characteristic (SROC) curve was constructed, and the area under the curve (AUC) was calculated to determine overall test performance.

Unconditional predictive values were calculated for G6PDd prevalence of 5% to 30% reflecting G6PDd prevalence within most malaria-affected populations [25]. Likelihood ratios are a convenient method to determine the usability of a diagnostic test. In the case of the CSG, the positive likelihood ratio (LR+) describes how many times more likely a G6PDd test result is to occur in a G6PD-deficient individual compared to in a G6PD-normal individual. The negative likelihood ratio (LR−) is defined as the inverse of this, or how much less likely a G6PD-deficient result will occur in a G6PD-normal person compared to a G6PD-deficient individual [26]. In general, tests with an LR+ above 10 are considered suitable for the diagnosis of a condition, and an LR− of less than 0.1 is considered suitable to exclude a condition [27]. The LR+ and LR− were calculated, and the practical utility of the CSG was evaluated by constructing likelihood ratio diagrams. The quality of the included publications was assessed using the QUADAS-2 tool [28].

Model validation

I was calculated as a measure of heterogeneity for sensitivity and specificity. Publication bias was assessed by a funnel plot, and a linear regression model was fitted to the log odds ratio of the inverse root of effective sample sizes as a test for funnel plot asymmetry.

Sensitivity analyses

We tested whether the sensitivity and the specificity of the tests varied by type of blood collected (capillary or venous) and sex by fitting separate multilevel models. In the first, we included a covariate for blood type and allowed both sensitivity and specificity to vary by blood type; we then repeated the analysis by instead including a covariate for sex. Each of these models was compared to a model without covariates using a likelihood ratio test. Additional sensitivity analyses were undertaken in which the pooled performance was recalculated excluding studies that were at high risk of bias due to participant selection or laboratory methods. The pooled performance was recalculated applying a pooled AMM across all included studies rather than the study-specific AMM. In response to a reviewer’s request, the analysis was repeated including all data as well as the aggregated data extracted from eligible articles for which individual patients’ data were not available. The definition of TP, TN, FP, and FN for articles in which no IPD were available was based on definitions applied in the respective studies. Whenever a discrepancy between reported performance and numbers of TPs and FPs and TNs and FNs was found, the latter was considered.

Results

Identified studies and participants

A total of 42 articles were identified in the literature review, of which 11 met the inclusion and exclusion criteria. Individual data were available from 8 studies (Fig 1) enrolling a total of 5,815 participants with paired CSG and spectrophotometry measurements (S1 Table).

Fig 1

Flow chart on article selection.

All of the studies included were undertaken between 2014 and 2018. Six studies were conducted in Southeast Asia [20,21,29-32], one in Africa [33], and one in the Americas [34]. In total, 3 studies (4 countries, 2,845 participants) assessed G6PD status from capillary blood [20,31,33] and 3 from venous blood (3 countries, 2,066 participants) [30,32,34]. In 1 study, CSG and spectrophotometry were performed on both venous and capillary samples [21], and in 1 study CSG was performed on both venous and capillary samples; however, spectrophotometry was only performed on capillary blood [29] (Table 1). Results from 14 (0.2%) participants were excluded because the spectrophotometry result was missing or had an extreme value (>25 U/gHb), and results from 24 (0.4%) participants were excluded due to an invalid CSG result. A total of 5,777 (99.3%) results were included in the analysis (Table 1), of which 3,095 (53.6%) were from females. The majority of samples were collected from healthy volunteers (Table 2).

Table 1

Origin, source of blood, and results included.

Article	Blood	Country	Original sample size	G6PD > 25 U/gHb or missing (%)	Invalid CSG result (%)	Total included (%)
Bancone, 2015* [21]	Capillary	Thailand	150	0 (0.0)	12 (8.0)	138 (92.0)
Bancone, 2015* [21]	Venous	Thailand	150	0 (0.0)	1 (0.6)	149 (99.3)
Espino, 2016** [29]	Capillary	Philippines	302	1 (0.3)	0 (0.0)	301 (99.7)
Espino, 2016** [29]	Venous	Philippines	302	1 (0.3)	0 (0.0)	301 (99.7)
Henriques, 2018*** [20]	Capillary	Cambodia	505	0 (0.0)	7 (1.4)	498 (98.6)
Henriques, 2018*** [20]	Capillary	Laos	757	4 (0.6)	4 (0.6)	749 (98.9)
Oo, 2016 [30]	Venous	Myanmar	1,000	0 (0.0)	0 (0.0)	1,000 (100.0)
Roca-Feltrer, 2014 [31]	Capillary	Cambodia	938	5 (0.5)	0 (0.0)	933 (99.5)
Roh, 2016, Uganda [33]	Capillary	Uganda	645	2 (0.3)	0 (0.0)	643 (99.7)
Satyagraha, 2016 [32]	Venous	Indonesia	610	1 (0.2)	0 (0.0)	609 (99.8)
von Fricken, 2014 [34]	Venous	Haiti	456	0 (0.0)	0 (0.0)	456 (100.0)
Total			5,815	14 (0.2)	24 (0.4)	5,777 (99.3)

*Paired CSG and spectrophotometry results from venous and capillary blood.

**Paired CSG results from venous and capillary blood, spectrophotometry results from venous blood.

***Same publication but different sites.

Abbreviations: CSG, CareStart Screening test for G6PD deficiency; G6PD, glucose-6-phosphate dehydrogenase

Table 2

Details on studies included.

Article	Blood	Country	Study population	n With malaria (%)*	Females included (%)	Males included (%)	Calculated local AMM (100% G6PD activity) in U/gHb; G6PD activity at 30%**	n of Study population included with <30% G6PD activity based on local AMM (%)	n of Study population included with <30% G6PD activity based on pooled AMM (%)
Bancone, 2015 [21]	Capillary	Thailand	Healthy volunteers	0 (0.0)	95 (68.8)	43 (31.2)	6.6; 2.0	41 (29.7)	44 (31.9)
Bancone, 2015 [21]	Venous	Thailand	Healthy volunteers	0 (0.0)	99 (66.4)	50 (33.6)	6.6; 2.0	45 (30.2)	51 (34.2)
Espino, 2016 [29]	Capillary	Philippines	High school students from cross-sectional survey	Not provided	197 (65.5)	104 (34.6)	11.1; 3.3	17 (5.7)	16 (5.3)
Espino, 2016 [29]	Venous	Philippines	High school students from cross-sectional survey	Not provided	197 (65.5)	104 (34.6)	11.1; 3.3	17 (5.7)	16 (5.3)
Henriques, 2018 [20]	Capillary	Cambodia	Participants of cross-sectional survey	Not provided	248 (49.8)	250 (50.2)	7.6; 2.3	117 (23.5)	124 (24.9)
Henriques, 2018 [20]	Capillary	Laos	Purposively selected community members	Not provided	366 (48.9)	383 (51.1)	11.5; 3.5	39 (5.2)	38 (5.07)
Oo, 2016 [30]	Venous	Myanmar	Healthy volunteers	0 (0.0)	476 (47.6)	524 (52.4)	8.3; 2.5	68 (6.8)	68 (6.8)
Roca-Feltrer, 2014 [31]	Capillary	Cambodia	Adults >18 years, nonpregnant from cross-sectional survey	0 (0.0)	484 (51.9)	449 (48.1)	12.0; 3.6	74 (7.9)	70 (7.5)
Roh, 2016, Uganda [33]	Capillary	Uganda	Children 6–59 months from cross-sectional survey (3.5% with microscopic malaria)	22 (3.4)	317 (49.3)	326 (50.7)	6.4; 1.9	10 (1.6)	24 (3.73)
Satyagraha, 2016 [32]	Venous	Indonesia	All ages from cross-sectional survey (2.5% with malaria)	15 (2.5)	349 (57.3)	260 (42.7)	9.3; 2.8	30 (4.9)	30 (4.9)
von Fricken, 2014 [34]	Venous	Haiti	Primary school children from cross-sectional survey	Not provided	267 (58.6)	189 (41.5)	9.1	46 (10.1)	46 (10.1)

*Based on publication.

**Calculated cut-offs and cut-offs published in source article do not necessarily match due to different definitions.

Abbreviations: AMM, adjusted male median; G6PD, glucose-6-phosphate dehydrogenase

Definition of 100% G6PD activity

In the study with paired spectrophotometry measures of patients with both capillary and venous sampling, there was no significant difference in G6PD activity (p = 0.292) [21]. Results for capillary and venous spectrophotometry were therefore pooled. The site-specific AMM ranged from 6.6 U/gHb to 12.3 U/gHb. When results were pooled across all studies, the overall AMM was 9.2 U/gHb (interquartile range [IQR] 7.2–11.5) (Table 2).

Pooled performance

The pooled sensitivity was 0.96 (95% CI 0.90–0.99) (Fig 2), and the specificity was 0.95 (95% CI 0.92–0.96) (Fig 3). The number of invalid results was significantly higher for capillary samples (12/3,274) compared to venous samples (2/2,517, p = 0.022) (Table 1); the AUC of the SROC was 0.98 (95% CI 0.97–0.99) (S1 Fig).

Fig 2

Forest plot: Sensitivity.

Threshold for G6PDd is calculated based on the site-specific AMM. Study ID is identified by first author, country of sample collection, and type of blood used. AMM, adjusted male median; G6PDd, glucose-6-phosphate dehydrogenase deficiency.

Fig 3

Forest plot: Specificity.

Threshold for G6PDd is calculated based on the site-specific AMM. Study ID is identified by first author, country of sample collection, and type of blood used. AMM, adjusted male median; G6PDd, glucose-6-phosphate dehydrogenase deficiency.

Utility of the CSG

When the prevalence of G6PDd was varied from 5% to 30%, the unconditional negative predictive value (NPV) was 0.97 (95% CI:0.94–1.00), and the positive predictive value (PPV) was 0.76 (95% CI 0.72–0.81). The LR+ and LR− were 18.2 (95% CI 13.0–25.5) and 0.05 (95% CI 0.02–0.12), respectively (S2 Fig).

Publication bias

Three of the 11 eligible studies (enrolling 1,280 participants from Brazil, Yemen, and Ghana) were not included because the corresponding authors did not reply [35-37]. These studies had a higher proportion of malaria patients. The characteristics of studies included and excluded in the individual data analysis are presented in S2 Table. No significant publication bias was detected among included studies (p = 0.41); 3 studies were identified as yielding a high risk of bias, 2 due to purposive selection of participants [20,21] and 1 due to lack of temperature-controlled spectrophotometry [34] (Fig 4, S3 Fig).

Fig 4

Qualitative assessment of included studies (QUADAS2).

QUADAS2, Quality Assessment of Diagnostic Accuracy Studies.

In the a priori sensitivity analyses, the pooled performance did not vary significantly irrespective of whether capillary or venous blood was collected (p = 0.547). For capillary samples, the sensitivity was 0.99 (95% CI 0.80–1.00), and specificity was 0.94 (95% CI 0.90–0.97) compared to 0.93 (95% CI 0.87–0.96) and 0.94 (95% CI 0.92–0.96), respectively, for venous samples. However, performance differed significantly between males and females (p = 0.027). In males, the sensitivity was 0.97 (95% CI 0.92–0.99), and specificity was 0.98 (95% CI 0.96–0.99), significantly higher than in females, who had a sensitivity of 0.92 (95% CI 0.80–0.97) and a specificity of 0.93 (95% CI 0.89–0.96) (Table 3).

Table 3

Results of pooled and sensitivity analysis.

Analysis	Sensitivity (95% CI)	Specificity (95% CI)	Sample size
Primary analysis	0.96 (0.90–0.99)	0.95 (0.92–0.96)	5,777
Capillary only	0.99 (0.80–1.00)	0.94 (0.90–0.97)	3,262
Venous only	0.93 (0.87–0.96)	0.94 (0.90–0.97)	2,515
Males only	0.97 (0.92–0.99)	0.98 (0.96–0.99)	2,682
Females only	0.92 (0.80–0.97)	0.93 (0.89–0.96)	3,095
Excluding studies with purposively selected participants	0.95 (0.86–0.99)	0.94 (0.93–0.96)	4,243
Excluding studies without temperature-controlled spectrophotometry	0.96 (0.89–0.98)	0.95 (0.93–0.96)	5,321
Applying pooled AMM	0.96 (0.89–0.98)	0.95 (0.93–0.96)	5,777
Considering aggregate data from all eligible studies	0.96 (0.90–0.99)	0.95 (0.92–0.96)	7,057

Abbreviation: AMM, adjusted male median

When 2 studies enrolling purposively selected participants were excluded, the pooled performance was slightly lower (sensitivity 0.95, 95% CI 0.86–0.99; specificity 0.94, 95% CI 0.93–0.96) [20,21]. When one study using venous samples in which spectrophotometry was not temperature controlled was excluded, the overall pooled performance was unchanged (sensitivity 0.96, 95% CI 0.89–0.98; specificity 0.95, 95% CI 0.93–0.96), although the performance for venous samples was slightly lower (sensitivity 0.95, 95% CI 0.89–0.98; specificity 0.95, 95% CI 0.92–0.97).

When the analysis was repeated using an AMM derived from pooled spectrophotometry data rather than the site-specific AMM, the pooled performance did not differ (sensitivity 0.96, 95% CI 0.89–0.98; specificity 0.95, 95% CI 0.93–0.96) (Table 3 and S4 Fig). When aggregated data were included from the 3 studies that fulfilled the inclusion criteria, but for which no IPD were available, the performance did not change (Table 3, S5 Fig).

Discussion

In this meta-analysis, we observed an overall sensitivity and specificity of the CSG of more than 95%; the NPV was almost 100% across a wide range of G6PDd prevalences. The high LR+ and low LR− suggest that the CSG is suitable for confirmation as well as exclusion of G6PDd at a 30% threshold level.

The CSG performed significantly better in males compared to females. The CSG performs best at an approximate 30% cut-off activity [20]; however, the absolute cut-off of the CSG and the absolute cut-off calculated from spectrophotometry do not necessarily match. Since males are either hemizygous normal or deficient, their enzyme activity will be either below or well above the 30% cut-off, and small discrepancies between the 2 thresholds will not affect the calculated performance. However, females can be either homozygous or heterozygous for the G6PD gene, the latter manifesting phenotypically with enzyme activities ranging from almost normal to G6PD deficient [12-14]. Therefore, in heterozygous females, small differences between the inherent test and the calculated cut-off activity will affect the test’s performance adversely.

The performance of the CSG was slightly better in samples collected from capillary compared to venous blood, although this did not reach statistical significance. However, the overall performance was more reliable in studies using venous blood, which had a lower number of invalid results. Bancone and colleagues previously compared CSG results from paired venous and capillary samples, with 11% discrepancy between samples, with a sensitivity at the 30% threshold of 100% in capillary samples compared to 89% in venous samples [21]. In the same study, the authors also correlated their findings with haematological parameters and found that RBC concentration, haemoglobin, haematocrit, mean corpuscular volume, and platelet count varied slightly between venous and capillary samples; however, they concluded that these differences were unlikely to have a major effect on the performance of the CSG [21]. In contrast, a study conducted by Espino and colleagues reported lower sensitivities for diagnosing deficiency at the 30% threshold among capillary samples (69% sensitivity) compared to their paired venous counterparts (94% sensitivity) [29].

Despite the significantly higher number of invalid results, the CSG is more likely to be performed on capillary blood from a finger prick, following the same procedures as for malaria rapid diagnostic tests. The observed good performance of the CSG on capillary blood is therefore reassuring; the pooled sensitivity is similar to the widely used FST [38]. While the CSG and the FST can be applied to screen patients for G6PDd prior to administering PQ, the recommended criteria for the recently licensed 8-aminoquinoline drug tafenoquine (TQ) are more stringent and require diagnosis of G6PDd at a 70% threshold, which requires a quantitative assay [39,40].

In reality, G6DP testing is rarely available in malaria-endemic communities, and therefore PQ is often not prescribed due to fear of inducing haemolysis in vulnerable patients [2]. The availability of a robust point-of-care G6PD test to screen patients prior to treatment provides a significant advance that will enhance the uptake of radical cure into routine practice. Unfortunately, the CSG does not have a control line, and this has implications for implementation into routine practice. Previous studies have shown that, at a cost of US$1.75, the use of the CSG is a cost-effective strategy at enhancing safe and effective radical cure with PQ [41].

Limitations

Our study has a number of limitations. The geographical spread of results included was limited, with most studies being conducted in Southeast Asia. It is likely that the performance, including PPV and NPV of the tests, will vary with the local context, including the prevalence and variants of G6PDd and the training and education of the clinic staff.

Only a few data variables were collated from all studies, and therefore our covariate analysis was limited to the haemoglobin concentration, the sex of the participant, and the country of sample collection. Other factors that may also have influenced the test results could have included batch to batch variability in test kits, the temperature at which the tests were performed, and training and ability of individuals undertaking the tests.

Spectrophotometry remains the gold standard for the diagnosis of G6PDd and was used as the reference for the current analysis [16]. Alternative approaches, such as molecular analysis for G6PD variants correlate poorly with G6PD phenotype, precluding use of this approach as reliable reference [33,42-44]. In a comparison between Trinity spectrophotometry kits, considered for this analysis, and another spectrophotometry kit (Pointe Scientific, Canton, MI), both assays showed a very good correlation (r = 0.9799, p < 0.001) [45].

The AMM was calculated for each site specifically; consequently, the absolute cut-off activity in U/gHb of the reference method spectrophotometry varied across sites. To assess whether this had an impact on the pooled performance, the analysis was repeated calculating a universal AMM across all sites; reassuringly, the results of the pooled performance did not differ.

IPD from 3 eligible studies, enrolling 1,280 participants, were not available [35-37]. In contrast to the included studies, the proportion of malaria patients among the excluded studies was higher. It is possible that malaria influences G6PD activity, although it is unlikely that this would have impacted the observed performance because CSG and spectrophotometry testing were done on the same sample.

Reassuringly, when the analysis was repeated including aggregated data, the test performance did not change. Finally, all studies included were performed under research conditions and by well-trained study staff; in real-life settings, the performance of the CSG could be lower.

Conclusion

The results from this pooled analysis suggest that the CSG provides a reliable method to identify individuals with less than 30% G6PD enzyme activities; based on these findings, the test is suitable for introduction into routine treatment prior to PQ but not TQ treatment. Further operational research is required to assess how the test performs under real-life conditions.

PRISMA IPD checklist.

PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analyses.

(DOCX)

Click here for additional data file.

Test methods applied.

(DOCX)

Click here for additional data file.

Details on studies not included.

(DOCX)

Click here for additional data file.

Contact details from which data were obtained.

(DOCX)

Click here for additional data file.

Summary ROC.

ROC, Receiver Operating Characteristics curve.

(TIF)

Click here for additional data file.

Likelihood ratio scatter diagram.

(TIF)

Click here for additional data file.

Deeks’ funnel plot asymmetry test.

(TIF)

Click here for additional data file.

Forest plot: Sensitivity and specificity, pooled AMM.

Threshold for G6PDd is calculated based on the pooled AMM.

(TIF)

Click here for additional data file.

Forest plot: Sensitivity and specificity, all studies fitting the inclusion criteria.

Threshold for G6PDd is based on definitions provided from aggregate data (first 3 studies from top) and calculated based on the site-specific AMM for IPD (all other studies). Study ID is identified by first author, country of sample collection, and type of blood used.

(TIF)

Click here for additional data file.

26 Sep 2019

Dear Dr. Ley,

Thank you very much for submitting your manuscript "A meta-analysis of the performance of the Carestart™ rapid diagnostic test for the detection of glucose 6 phosphate dehydrogenase deficiency" (PMEDICINE-D-19-02311) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to three independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Oct 17 2019 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see http://journals.plos.org/plosmedicine/s/submission-guidelines#loc-methods.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Thomas McBride, PhD

Senior Editor

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

1- In addition to the data statement, it would be helpful to provide a list of the data contacts for the included studies in a supplemental file.

2- Please revise your Title to PLOS Medicine style, placing the study design at the end, after a colon “Performance of the Carestart rapid diagnostic test for the detection of glucose 6 phosphate dehydrogenase deficiency: a Systematic review and meta-analysis”

3- PLOS style does not permit trademarks, please remove the (TM) in the title and throughout the manuscript.

4- Please combine the Abstract “Methods” and “Results” sections into one section, titled “Methods and Results”.

5- Please include the full dates (day, month, year) of the search in the Abstract, as well as the data sources, eligibility criteria, and synthesis/appraisal methods

6- Alongside the 95% CIs, please include p-values.

7- In the last sentence of the Abstract Methods and Findings section, please describe the main limitation(s) of the study's methodology.

8- At this stage, we ask that you include a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary

9- Please update your search to the present time.

10- Please evaluate study quality and risk of bias.

11- Line 97, I believe you mean “sex” rather than “gender”. Please make sure the usage is accurate.

12- In the first paragraph of the Discussion and in the Discussion conclusions, please address the study implications without overreaching what can be concluded from the data; the phrase "In this study, we observed ..." may be useful.

Comments from the reviewers:

Reviewer #1: Thank you for the opportunity to review this interesting paper. This study presented an IPD meta-analysis of the diagnostic accuracy of the Carestart test for the detection of glucose 6 phosphate dehydrogenase deficiency (GPDDd) which showed good accuracy (according to diagnostic test metrics). The clinical message is simple but rather important - this is a rapid diagnostic test which can be deployed cheaply in the field to identify to GPDDd in individuals with malaria prior to being treated with primaquine to prevent primaquin-inducted can cause haemolysis. The methods are appropriate and I commend the authors for using an IPD approach for a diagnostic test accuracy study which can be challenging to synthesize. The methods are appropriate. The authors have utilised the appropriate approach of a two level model with a bivariate normal model with a logit transformation of sensitivity and specificity between studies. See https://www.ncbi.nlm.nih.gov/pubmed/18816508 - this may be a useful reference for the authors. The results are presented clearly and succinctly, and the conclusions support the results. Thus, I have only a few minor suggestions:

- Lines 133-136: the explanation of likelihood ratios is not very clear and they are described as probabilities in the text. LR+ and LR- are not probabilities - they are ratios, correct interpretation: LR+: quantifies how much more likely the positive test of G6PDd is to occur in subjects with the condition compare those without; LR-: quantifies how much less likely the negative test of G6PDd will occur in subjects without disease

- Lines 136-137: this statement needs be benchmarked to accepted criteria. see - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4975285/. Generally rule is that good test have LR+ > 10 and LR- < 0.1. The results show that the Carestart test satisfies this benchmark.

- Lines 194-196: the authors excluded three studies from the IPD due to not providing study data. Given that three studies did not reply but did meet eligibility criteria, it would be helpful to consider pooling the aggregate study reported outcomes together for 11 studies. This will allow for a comparison with the aggregate samples, given that the three eligible studies contribute about 1280 participants with a higher proportion of malaria patients. That's quite a large number of patients to not include in any of the analyses.

- Discussion section: It would be helpful if the authors could benchmark the Carestart diagnostic accuracy results from their IPD against the standard approach of mass spec from the literature. It would also be helpful for the authors to state if there are any of comparable reviews in this area and if so, how do their results compare. If not, they should also state the novelty of their current study.

- Limitations section: understand there was a limited data request from the study authors to data providers and hence there were a limited number of covariates. The authors should address this issue as a limitation, and speak to if any potential covariates which have not been captured could influence the results.

Reviewer #2: A timely and useful analysis. A few minor remarks for the authors to consider.

1. Lines 63, 64. This seems unhelpfully vague. What are the possible clinical consequences? Also, "can cause"? This seems to infer inconsistency of the effect, i.e., sometimes it does not. If that's true, say so and cite the evidence. Otherwise, the evidence available would lead this reviewer to believe "invariably causes" to be the more appropriate terminology.

2. Line 65. "More than 215"? How many more? This seems a peculiar expression given the precision of the number.

3. Line 72. They are certainly at significant risk simply by the genetics. The authors perhaps mean "at risk of significant harm".

4. Line 77. The expressed affordability of the FST may be challenged. It is about $5 a test just for the reagents and requires a cold-chain and specialised equipment. It is certainly not sufficiently affordable to even referral hospitals in endemic zones, where it is almost never found (because it costs too much).

5. Line 226-227. Doubtful that the CSG was engineered to perform poorly above 30% of normal, i.e., "designed". It is likely to be an inherent limitation of the technology.

Reviewer #3: This is a metadata analysis on field evaluations of a G6PD deficiency (G6PDd) point-of-care (POC) RDT CareStart produced by AccessBio. This analysis included 8 publications, which had both RDT and quantification of G6PD activity. Given that the use of primaquine and tafenoquine for radical cure of vivax malaria requires the screening of G6PDd, the need for a reliable POC device to detect G6PDd is important. Overall, the analysis is well done. Since one aim of the journal is to have an impact on medical practice, it would be important not to include a lot of statistical jargons without explanation in the abstract.

1. The authors used unconditional negative predictive value (NPV), positive (LR+) and negative (LR-) likelihood ratios in the abstract. However, for the general audience, they are not immediately understandable (though they are explained in the main text). Suggest to include a short description of what the meanings of the results are.

2. Line 76: the gold standard quantitative method is expensive - it should be clarified that there are many G6PD quantitative kits available, but the one from Trinity is much more expensive.

3. Some studies compared the assay results from both venous and capillary blood. I wonder whether there is any basis suggesting that the two sources would be different in G6PD activity?

4. In sensitivity analysis, the sensitivity for males was significantly higher than for females. Is there a potential explanation for this?

Some editorial suggestions:

1. For consistency with published literature, please use PQ as abbreviation for primaquine and use it throughout the text (see lines 74, 254).

2. I suggest to use CareStart in the abstract, since it is difficult to see why Carestart is abbreviated into CSG.

3. In the abstract Introduction section, "a qualitative lateral flow assay from Carestart, USA" should be "CarestartTM G6PD RDT, a qualitative lateral flow assay from AccessBio, USA".

4. Line 222 , use abbreviation NPV or spell out it in the entire text.

Any attachments provided with reviews can be seen via the following link:

[LINK]

16 Oct 2019

Submitted filename: Response to rev. 0.1.docx

Click here for additional data file.

25 Oct 2019

Dear Dr. Ley,

Thank you very much for re-submitting your manuscript "Performance of the Accessbio/Caretstart rapid diagnostic test for the detection of glucose-6-phosphate dehydrogenase deficiency: a systematic review and meta-analysis" (PMEDICINE-D-19-02311R1) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

Our publications team (plosmedicine@plos.org) will be in touch shortly about the production requirements for your paper, and the link and deadline for resubmission. DO NOT RESUBMIT BEFORE YOU'VE RECEIVED THE PRODUCTION REQUIREMENTS.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper.

We expect to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.

We look forward to receiving the revised manuscript by Nov 01 2019 11:59PM.

Sincerely,

Thomas McBride, PhD

Senior Editor

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

1- Please add this statement to the manuscript's Competing Interests: "LVS receives a stipend as a Specialty Consulting Editor for PLOS Medicine and serves on the journal's editorial board."

2- Thank you for providing the contact information for accessing data from the primary studies in table S3. However, PLOS does not allow authors of the current manuscript to be the primary contact for data access. Please make these data available (either in the supplemental files or in a data repository) or provide an alternate contact for data access.

3- Apologies for my mistake, please rename the Abstract “Methods and results” section “Methods and findings”. Please also rename the Abstract “Discussion and conclusion” section “Conclusions”.

4- In the Abstract, please briefly note the regions giving rise to the studies and at least broad details of the participants (adults vs children).

5- Was the start date for the pubmed search Jan 1, 2011? Please specify the month and date in the Abstract and the Methods.

6- Thank you for including an author summary. It may be useful to readers to explain the significance of the 30% cutoff for G6PD activity.

7- In your response to Reviewer 1, point 1, it may be more clear to state: “. In the case of the CSG the positive likelihood ratio (LR+) describes how many times more likely a G6PDd test result is to occur in a G6PDd individual compared to in a G6PD normal individual. The negative likelihood ratio (LR-) is defined as the inverse of this, or how much less likely the negative test of G6PDd is to occur in a G6PD normal person compared to a G6PDd individual (26).”

8- Please preface the description of the analysis that includes aggregate data from eligible articles without IPD with “In response to a reviewer request…” or similar.

9- The limited geographical span of the studies could be listed as a limitation, both in the Abstract and the Discussion.

10- There is a p= 0.00 in the forest plot. Please report as p < 0.001.

11- Line 336 (of the marked up manuscript): “*In a comparison between* Trinity spectrophotometry kits, considered for this analysis, *and* another spectrophotometry kit…”. To make clear that the comparison was not part of the current study.

Comments from the Academic Editor:

Line 53: should be "95%CI" not "955CI"

Line 97: Use "haemolysis" to be consistent with the British spelling (the manuscript mostly used British spelling)

Line 86: spell out Plasmodium for both species

Line 147, 291: use "PQ" for primaquine since it is abbreviated earlier (check the entire text)

Line 218-226: all "95%CI" were written as "95CI"

Line 220: use AUC since it was abbreviated earlier

Line 265-267: this sentence does not read well, suggest to revise (remove "and thus"?)

Line 278: use "haem" for all three words in this line

Any attachments provided with reviews can be seen via the following link:

[LINK]

8 Nov 2019

Dear Dr Ley,

On behalf of my colleagues and the academic editor, Dr. Liwang Cui, I am delighted to inform you that your manuscript entitled "Performance of the Accessbio/Caretstart rapid diagnostic test for the detection of glucose-6-phosphate dehydrogenase deficiency: a systematic review and meta-analysis" (PMEDICINE-D-19-02311R2) has been accepted for publication in PLOS Medicine.

PRODUCTION PROCESS

Before publication you will see the copyedited word document (in around 1-2 weeks from now) and a PDF galley proof shortly after that. The copyeditor will be in touch shortly before sending you the copyedited Word document. We will make some revisions at the copyediting stage to conform to our general style, and for clarification. When you receive this version you should check and revise it very carefully, including figures, tables, references, and supporting information, because corrections at the next stage (proofs) will be strictly limited to (1) errors in author names or affiliations, (2) errors of scientific fact that would cause misunderstandings to readers, and (3) printer's (introduced) errors.

If you are likely to be away when either this document or the proof is sent, please ensure we have contact information of a second person, as we will need you to respond quickly at each point.

PRESS

A selection of our articles each week are press released by the journal. You will be contacted nearer the time if we are press releasing your article in order to approve the content and check the contact information for journalists is correct. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact.

PROFILE INFORMATION

Now that your manuscript has been accepted, please log into EM and update your profile. Go to https://www.editorialmanager.com/pmedicine, log in, and click on the "Update My Information" link at the top of the page. Please update your user information to ensure an efficient production and billing process.

Thank you again for submitting the manuscript to PLOS Medicine. We look forward to publishing it.

Best wishes,

Thomas McBride, PhD

Senior Editor

PLOS Medicine

plosmedicine.org