Accuracy of malaria diagnostic tests performed on non-invasively collected samples: a systematic review and meta-analysis.

BACKGROUND: During the last decade, many studies have assessed the performance of malaria tests on non-invasively collected specimens, but no systematic review has hitherto estimated the overall performance of these tests. We report here the first meta-analysis estimating the diagnostic performance of malaria diagnostic tests performed on saliva, urine, faeces, skin odour ('sniff and tell') and hair, using either microscopy or PCR on blood sample as reference test.
METHODS: We searched on PubMed, EMBASE, African Journals Online and Cochrane Infectious Diseases from inception until 19 January 2021 for relevant primary studies. A random effects model was used to estimate the overall performance of various diagnostic methods on different types of specimen.
RESULTS: Eighteen studies providing 30 data sets were included in the meta-analysis. The overall sensitivity, specificity and diagnostic OR (DOR) of PCR were 84.5% (95% CI 79.3% to 88.6%), 97.3% (95% CI 95.3% to 98.5%) and 184.9 (95% CI 95.8 to 356.9) in saliva, respectively; 57.4% (95% CI 41.4% to 72.1%), 98.6% (95% CI 97.3% to 99.3%) and 47.2 (95% CI 22.1 to 101.1) in urine, respectively. The overall sensitivity, specificity and DOR of rapid diagnostic test for malaria in urine was 59.8% (95% CI 40.0% to 76.9%), 96.9% (95% CI 91.0% to 99.0%) and 30.8 (95% CI:23.5 to 40.4).
CONCLUSION: In settings where PCR is available, saliva and urine samples should be considered for PCR-based malaria diagnosis only if blood samples cannot be collected. The performance of rapid diagnostic testing in the urine is limited, especially its sensitivity. Malaria testing on non-invasively collected specimen still needs substantial improvement. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Malaria diagnostic can be performed on non-invasively collected specimens

Blood is the biological fluid of choice for malaria diagnosis.

The meta-analysis suggested that sensitivity of PCR in saliva and urine is lower than that reported in the literature when PCR is performed on blood.

The performance of RDT on urine is lower than the one observed in blood.

Malaria testing on non-invasively collected specimen still needs substantial improvement.

In settings where PCR is available, saliva and urine samples should be considered for PCR-based malaria diagnosis only if blood samples cannot be collected.

Introduction

Malaria remains a global public health problem with a substantial mortality especially in woman and children under 5 years.1–3 According to the World Malaria Report 2020, there has been a significant reduction in the burden of malaria over the last two decades, although the Malaria Millennium Development Goal of 90% reduction in global malaria incidence and mortality by 2030 is far to be achieved.1 4 The facies of malaria transmission and endemicity has changed thoroughly during the last two decades, with some regions like the great Mekong being close to elimination, while others like sub-Saharan Africa still have countries with high endemicity and heterogenous annual transmission pattern.1

Accurate diagnosis of malaria is a pillar of malaria control and elimination.5 6 Prior to the dissemination of rapid diagnostic tests, microscopy and, to a lesser extent, PCR were among the most used methods in the diagnosis of malaria. These methods had the drawback that they require well-trained personnel, ongoing training of the workforce, logistic and equipment that are not always available in developing countries. Although microscopy remains the gold standard, the advent of rapid diagnostic tests has greatly improved case detection and treatment rates.7 However, the current diagnostic tests are done on blood samples collected invasively. In some areas, especially in sub-Saharan Africa, the collection of blood sample on which malaria testing is performed, is not an easy task because of blood taboos related to local cultural beliefs, fear of needles and beliefs that HIV test will be conducted on blood collected without consent of the participants when the amount of blood collected is high.8–11 Moreover, in countries that are in elimination phase, the willingness of asymptomatic patients to go for an invasive test for surveillance purposes may become challenging over time, hence a need for malaria diagnostic tests performed on non-invasively collected specimens.12 13 These non-invasively collected specimens are also more convenient for research purposes, to support decision making and can be used in management of patients with malaria in hospital.

Recently, several studies have evaluated the accuracy of diagnosing malaria using PCR, ELISA or rapid diagnostic testing (RDT) on non-invasively collected human specimens such as saliva, urine, faeces and hair.14 15 The current study aimed to systematically review these studies and performed a meta-analysis to determine the overall diagnostic accuracy of malaria diagnostic tests performed on saliva, urine, faeces and hair.

Methods

This review was registered with PROSPERO (International Prospective Register of Sytematic Reviews) and is reported in accordance with the Preferred Reporting Items for a Systematic Reviews and Meta-analyses of Diagnostic Test Accuracy guidelines.16

Search strategy and eligibility criteria

PubMed, EMBASE, Cochrane Infectious Diseases Group Specialised Register and African Journals Online were searched from inception to 19 January 2021 using predefined search strategies adapted for each database (online supplemental tables 1 and 2). We included studies with at least 20 participants reporting on malaria tests performed on non-invasively collected samples regardless of the language, year of publication, design or country. Were considered as non-invasively collected samples all specimens that were obtained without cutting the skin or penetrating any part of the body as defined in the Cambridge dictionary.17 The ‘sniff and tell’ method refers to the diagnosis of malaria with dogs. We excluded reviews, letters, commentaries and editorials.

Records retrieved from bibliographic searches were imported in Rayyan online software.18 After removal of duplicates, the titles and abstracts of remaining records were independently screened for potential inclusion by two reviewers (CD, JJNN). Full texts were then downloaded and assessed for final inclusion. Disagreements were solved through discussion and consensus.

Data extraction and quality assessment

Data were extracted using a preconceived form. They included first author’s name, year of publication, country, characteristics of the study population (age distribution and symptoms), index test, reference standard test, type of non-invasive sample, number of true positive, true negative, false positive and false negative cases.

Records reporting the estimation of diagnostic accuracy on two subpopulations, or the ones stratifying the analysis according to a specific criterion, for example, the index test or reference standard used, were splitted into different data sets in order to obtain a single estimation per data set. Thus, the term ‘record’ refers to one study or article, while ‘data set’ refers to a substudy.

The Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) was used for the assessment of the risk of bias and applicability of included studies.19 The QUADAS-2 tool is divided into four sections: patient selection, index test, reference standard, flow and timing. All the four sections are rated in the risk of bias assessment, while all except ‘flow and timing’ are rated in the applicability concern.19 20

An extensive description of the different methods of malaria diagnosis, their principles and techniques are discussed elsewhere.21–24

Statistical analysis

All analyses were conducted in R software V.4.0.2. Random effects meta-analysis was performed to determine separately the pooled sensitivity, specificity and diagnostic OR (DOR) using the ‘meta’ package and the summary receiver operating characteristic curve within the ‘MADA’ package.25 26 A subgroup analysis was conducted according to the following variables: the type of specimen (urine, saliva, stool, ‘sniff and tell’), the index test used on the non-invasively collected sample, the reference test used on blood and the age of participants. The presence of heterogeneity was assessed with the Cochran statistic and quantified by the I2.27 28 Values between 0%–40%, 30%–60%, 50%–90%, 75%–100% were considered as indicative of low, moderate, substantial, considerable heterogeneity, respectively.29 A p≤0.05 was considered as statistically significant.

Results

Search results

We retrieved 1607 records from bibliographic searches. Eighteen studies14 15 30–45 were included, contributing to a total of 36 data sets included in the systematic review and 30 in the meta-analysis (figure 1).

Figure 1

PRISMA flow chart. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Characteristics of studies in the meta-analysis

Data sets included in the meta-analysis were from 10 countries, mostly from Iran (10 data sets), India (5 data sets) and the Gambia (4 data sets) (online supplemental table 3). The studies were conducted between 2009 and 2020, and they included participants aged between 1 and 80 years. The proportion of male ranged from 29.0% to 82.4%. Online supplemental table 4 presents the individual characteristics of included studies. The data sets predominantly (17 out of 30) had a moderate risk of bias (figure 2).

Figure 2

Quality assessment of studies included in the meta-analysis.

Diagnostic accuracy of malaria diagnostic tests performed on saliva

Fourteen data sets were included in the meta-analysis of diagnostic accuracy of tests performed on saliva (online supplemental figures 1–3). Overall (irrespective of the reference test), the pooled sensitivity, specificity and DOR of PCR on saliva were 84.5% (95% CI 79.3% to 88.6%), 97.3% (95% CI 95.3% to 98.5%) and 184.9 (95% CI 95.8 to 356.9), respectively. With PCR on a blood sample as the reference test, PCR on saliva had a pooled sensitivity, specificity and DOR of 87.0% (95% CI 81.8% to 90.9%), 98.6% (95% CI 95.7% to 99.5%), 395.5 (95% CI 117.1 to 1335.8), respectively. When microscopy on a blood sample was considered as the reference test, the pooled sensitivity, specificity and DOR of PCR on saliva were respectively 83.2% (95% CI 76.0% to 88.6%), 96.9% (95% CI 94.3% to 98.3%), 153.4 (95% CI 72.6 to 323.8) (table 1, online supplemental figures 1–3).

Table 1

Meta-analysis of diagnostic accuracy of malaria diagnostic tests performed on non-invasively collected samples

	Pooled sensitivity, % (95% CI)	Pooled specificity, % (95% CI)	Pooled DOR, OR (95% CI)	N studies	Heterogeneity for sensitivity (I², %)	Heterogeneity for specificity (I², %)	Heterogeneity for DOR (I², %)
Saliva
Studies conducted on saliva with PCR as the index test and,	84.5 (79.3 to 88.6)	97.3 (95.3 to 98.5)	184.9 (95.8 to 356.9)	10	68.3	60.1	49.4
PCR in a blood sample as the reference test	87.0 (81.8 to 90.9)	98.6 (95.7 to 99.5)	395.5 (117.1 to 1335.8)	3	0.0	0.0	0.0
Microscopy of a blood sample as the reference test	83.2 (76.0 to 88.6)	96.9 (94.3 to 98.3)	153.4 (72.6 to 323.8)	7	73.9	64.3	56.0
Urine
Studies conducted on urine with PCR as the index test and,	57.4 (41.4 to 72.1)	98.6 (97.3 to 99.3)	47.2 (22.1 to 101.1)	5	87.9	0.0	0.0
PCR in a blood sample as the reference test	70.1 (61.9 to 77.1)	98.6 (90.6 to 99.8)	99.5 (18.8 to 526.2)	2	0.0	0.0	0.0
Microscopy of a blood sample as the reference test	48.2 (28.5 to 68.4)	98.6 (97.1 to 99.3)	46.4 (15.2 to 141.7)	3	88.3	0.0	23.1
Studies conducted on urine with RDT as the index test and,	59.8 (40.0 to 76.9)	96.9 (91.0 to 99.0)	30.8 (23.5 to 40.4)	9	95.8	95.6	0.0
Microscopy of a blood sample as the reference test	71.7 (44.9 to 88.7)	89.9 (83.9 to 93.8)	30.0 (22.5 to 40.0)	5	96.1	81.2	0.0

DOR, diagnostic OR; RDT, Rapid diagnostic test.

Diagnostic accuracy of malaria diagnostic tests performed on urine

Thirteen data sets were included in the assessment of the diagnostic performance of tests conducted on urine (online supplemental figures 4–6). Irrespective of the reference test, the pooled sensitivity, specificity and DOR of, PCR on a urine sample were 57.4% (95% CI 41.4% to 72.1%), 98.6% (95% CI 97.3% to 99.3%) and 47.2 (95% CI 22.1 to 101.1), respectively. With PCR on a blood sample as the reference test, PCR on urine had a pooled sensitivity, specificity and DOR of 70.1% (95% CI: 61.9% to 77.1%), 98.6% (95% CI: 90.6% to 99.8%), 99.5 (95% CI 18.8 to 526.2), respectively. When microscopy of a blood sample was considered as the reference test, the pooled sensitivity, specificity and DOR of PCR on urine were respectively 48.2% (95% CI 28.5% to 68.4%), 98.6% (95% CI 97.1 to 99.3), 46.4 (95% CI 15.2 to 141.7) (table 1).

The pooled sensitivity, specificity and DOR for RDT on urine (irrespective of the reference test) were 59.8% (95% CI 40.0% to 76.9%), 96.9% (95% CI 91.0% to 99.0%) and 30.8 (95% CI 23.5 to 40.4), respectively (table 1). With microscopy of a blood sample as the reference test, RDT on urine had pooled a sensitivity, specificity and DOR of 71.7% (95% CI 44.9% to 88.7%), 89.9% (95% CI 83.9% to 93.8%), 30.0 (95% CI 22.5 to 40.0), respectively (table 1).

Diagnostic accuracy of the ‘sniff and tell’ method in children

Five out of the 36 data sets were derived from studies done in children, with two reporting the performance of ‘sniff and tell’ (including two dogs) method. In studies reporting on ‘sniff and tell’, malaria positivity was assessed by microscopy while malaria negativity was confirmed by qPCR on blood samples. The pooled sensitivity, specificity and DOR of ‘sniff and tell’ were 71.7% (95% CI 59.1% to 81.6%), 90.7% (95% CI 86.8% to 93.5%) and 24.6 (95% CI 12.4 to 48.9), respectively (online supplemental figures 7–9).

In leave-one-out analysis, regardless of the reference test, the exclusion of none of the studies significantly changed the pooled diagnostic accuracy of tests performed on the urine or saliva (online supplemental figures 10–16).

Data sets from studies that used ELISA, PCR and RDT on saliva in children were not considered for meta-analysis due to small sample size (less than 20 participants). These data are summarised in online supplemental table 4.

Discussion

This meta-analysis of studies on the performance of malaria diagnostic tests on non-invasively collected samples revealed a lower overall sensitivity of PCR in saliva and urine compared with that reported in the literature when PCR is performed on blood. PCR performance in urine and saliva was better when the reference test in blood was PCR. Probably because only two studies included in the meta-analysis of urine/saliva were conducted in patients that were not symptomatic. Thus, the diagnostic performance estimates are probably representative of those that would be observed among clinical infections with parasite densities above 100/μL. Moreover, the performance of the tests performed on saliva was better than that of the tests conducted on urine. Probably because most saliva studies have used PCR as the index test (71.4% vs 38.5% for urine). When the studies were stratified according to the index test performed on the non-invasively collected sample, regardless of the type of sample, PCR had a higher pooled sensitivity than RDT, LAMP and ELISA tests. In addition, PCR performed better in saliva than in urine when the reference test on blood was PCR.

The higher performance of PCR compared with other tests in the diagnosis of malaria is well established and has been published in several studies and reviews. A meta-analysis showed a pooled sensitivity of PCR of about 98% (95% CI 90% to 99%) when performed on blood samples, which is higher than the sensitivity in saliva found in our meta-analysis.46 The high sensitivity of PCR in the saliva compare to urine can be due to blood contamination of the saliva as a result of microbleeding in the oral cavity.37 47 48 It is paramount for research purposes to compare the performance of PCR on saliva samples in which the presence of blood has been formally excluded with the ones in which it has not. One of the alternatives to deal with blood contamination in saliva may be to use supernatant of spun saliva instead of whole saliva to test for malaria as reported in some studies.37 In addition, a better understanding of the mechanisms of malaria detection in saliva is needed to improve the performance of malaria diagnostic methods at point-of-care.

The performance of RDT on urine appears to be lower than the one observed in blood. Indeed, the average sensitivity and specificity of histidine-rich protein II (HRP2) based RDT of malaria in blood regardless of the reference test are estimated to be 95.0% (95% CI 93.5% to 96.2%) and 95.2% (95% CI 93.4% to 99.4%) respectively,7 compared with 58.7% (95% CI 25.8% to 85.3%) and 96.5% (95% CI 82.8% to 99.4%) in urine as determined by our meta-analysis. Given that rapid diagnostic tests are among the most accessible and user-friendly methods of malaria diagnostic, the development of highly sensitive and polyvalent tests that can be performed with comparable sensitivity on urine, saliva and blood would significantly increase adherence to diagnostic testing of asymptomatic individuals, particularly in resource-constrained settings and in countries that are in the elimination phase.

In addition to help in the management of malaria cases in hospitals, malaria diagnosis can be done for many purposes, such as to assess the prevalence of malaria in communities, for research activities or to support decision-making in countries or areas that are in the elimination phase and where the detection of the human parasite reservoir can be useful to tailor interventions. Therefore, the diagnosis of malaria and the interpretation of the current findings cannot be made from the sole prism of hospital case management but should be integrated into a broader context.

The molecules detected in non-invasive samples are the same as in blood. For nPCR in saliva for example, 18S rRNA genes, or mitochondrial cytochrome b gene, of Plasmodium falciparum and Plasmodium vivax were targeted and amplified in most of the studies,30 31 whereas RDT in saliva and urine target PfHRP2 and pLDH antigens.14 36 This suggests that the issues faced by blood based RDT tests (regarding PfHRP detection) are the same for tests conducted on non-invasive sample.

This review is written under the premise that non-invasive tests for malaria would be preferable if they were at least as good as the currently available point-of-care tests that use finger-prick blood (RDT or microscopy). However, many of these tests still face the same logistical challenges as blood-based tests, as they require sophisticated equipment and trained personnel. Nevertheless, the use of RDT on non-invasive samples has a substantial advantage over blood as they do not require any expertise for sampling since urine and saliva are directly available, while blood collection requires knowledge of asepsis and good knowledge of finger or phlebotomy blood sampling methods. The samples are painless and do not require psychological preparation of patients to cope with pain as is the case for blood sampling. However, when these tests are not performed immediately after the collection of the non-invasive material, the necessity to store the samples at low temperatures, makes it difficult to perform the tests in routine practice in the communities or at malaria point of care in low/middle-income countries where electricity is often lacking and the number of patients to be tested is large. It is essential that the stability of non-invasive specimens when stored at room temperature is assessed to determine whether their storage at room temperature does not compromise the performance of malaria tests performed on these specimens.

The findings of the current review suggest that the performance of malaria diagnostic tests on non-invasively collected samples still needs to be improved to be comparable with the performance on blood. They call for further research to develop highly sensitive rapid diagnostic tests based on non-invasively collected samples, particularly saliva which can be easily obtained, and for more studies to assess the performance of available tests on saliva and urine.

This review is mainly limited by some heterogeneity observed in the meta-analysis, the source being the multiplicity of reference tests used in the blood, and index tests used on the non-invasive sample, and perhaps the difference in nucleic acid stability in saliva and urine. However, this study is the first meta-analysis on the diagnostic accuracy of malaria tests performed on non-invasively collected samples. A subgroup analysis was conducted by type of specimen, reference blood test and index test to give a broad overview on the performance of this approach in different contexts.

Conclusion

In settings where PCR is available, saliva and urine samples may be considered for PCR-based malaria diagnosis only if blood samples cannot be collected, given the lower sensitivity found. The performance of RDT in the urine remain limited, especially its sensitivity. Malaria testing on non-invasively collected specimen still needs substantial improvement, especially for RDT, in order to be considered for widespread use.