Accuracy of an Immunochromatographic Diagnostic Test (ICT Malaria Combo Cassette Test) Compared to Microscopy among under Five-Year-Old Children when Diagnosing Malaria in Equatorial Guinea.

Conventional malaria diagnosis based on microscopy raises serious difficulties in weak health systems. Cost-effective and sensitive rapid diagnostic tests have been recently proposed as alternatives to microscopy. In Equatorial Guinea, a study was conducted to assess the reliability of a rapid diagnostic test compared to microscopy. The study was designed in accordance with the directives of the Standards for Reporting Diagnostic Accuracy Initiative (STARD). Peripheral thick and thin films for the microscopy diagnosis and a rapid immunochromatographic test (ICT Malaria Combo Cassette Test) were performed on under five-year-old children with malaria suspicion. The ICT test detected Plasmodium spp. infection with a sensitivity of 81.5% and a specificity of 81.9% while P. falciparum diagnosis occurred with a sensitivity of 69.7% and a specificity of 73.7%. The sensitivity of the ICT test increased with higher parasitemias. The general results showed little concordance between the ICT test and microscopy (kappa = 0.28, se: 0.04). In Equatorial Guinea, the ICT Malaria Combo Cassette Test has proven to be an acceptable test to detect high P. falciparum parasitemias. However, the decrease of sensitivity at medium and low parasitemias hampers that ICT can replace properly performed microscopy at present in the diagnosis of malaria in children.

1. Background

The current malaria control strategies are mainly based on early diagnosis and a correct treatment of the cases. These are essential to reduce the fatal outcome of the disease [1]. However, the weakness of the health systems in many endemic countries, particularly at the peripheral level, means that the malaria diagnosis has to be based on clinical criteria. Taking into account that other infectious diseases course with signs and symptoms like malaria, a high percentage of overdiagnosis can be expected in a tropical area [2-4].

The growing resistance to drugs commonly used for malaria treatment (chloroquine, quinine, and sulphadoxine-pyrimethamine), due to their abusive use in the past, and the arrival of artemisinin-based combination therapies (ACTs), which are more expensive than the former, mean that the methods to diagnose malaria are once again back in the spotlight. Microscopy and the use of rapid diagnostic tests (RDTs) are currently considered to be the two diagnostic procedures with the greatest impact on controlling malaria [5].

Microscopy can be a highly useful diagnostic tool, as in expert hands it can detect up to 50 parasites per μl (0.001% parasitemia) and identify the plasmodia in 98% of the cases [6, 7]. However, this procedure is not simple and time consuming, requiring the sample to be stained quickly to maintain its reliability. It also demands well-trained and motivated human resources, along with properly maintained equipment [8]. Thus, there are several evidences of the low diagnostic capacity of microscopy services under real field conditions [9-12].

Therefore, the availability of cost effective, sensitive, and accurate rapid diagnostic tests would enable the malaria diagnosis to be better handled.

The new rapid malaria diagnostic tests are based on immunochromatographic techniques using conjugated monoclonal antibodies as infection indicators. The detected antigens are preferably those present in all forms of the parasite (either sexual or asexual), such as histidine rich protein II (HRPII), lactate dehydrogenase (LDH), and aldolase.

The World Health Organization (WHO) establishes that RDT sensitivity must be close to 100% in parasitemias over 100 parasites/μl [13, 14]. The RDTs that are currently on the market can detect parasitemias over 100 parasites/μl in 15–20 minutes without the need for equipment, which simplifies the procedure with respect to other diagnostic methods [6, 7].

Malaria is one of the most important public health problems in Equatorial Guinea, and it is responsible for the high morbidity and mortality rates, particularly among the under five-year-old children and the pregnant women [15-18]. In the last 5 years, two important malaria control projects have been developed in Equatorial Guinea mainly focused on the indoor spraying with residual effect insecticides (IRS) [16, 19], but also seeking to improve the diagnostic quality of malaria in health centers by means of introducing RDTs.

This study aims to assess the sensitivity and specificity of a rapid diagnostic test, which is going to be introduced in Equatorial Guinea to diagnose malaria, compared to microscopy (diagnostic reference test or “gold standard”).

2. Material and Methods

2.1. Study Design

The study was carried out in the Malaria Reference Laboratory (MRL) located on the island of Bioko, Equatorial Guinea. Malaria in Equatorial Guinea is hyperholoendemic [15]. Entomological studies conducted on the island of Bioko have shown high transmission values for the main vector species: 242.7–281 infective bites per year for Anopheles gambiae s.s. and 317–787.6 for An. funestus s.l. [17, 18]. In the last years, a sustained decrease of malaria transmission on the island of Bioko has been observed due to the implementation of a major malaria control project [16, 19]. Previous papers have deeply described both the geographical characteristics of this island and its malaria transmission pattern [15, 16].

Research methodology was in accordance with the Standards for Reporting Diagnostic Accuracy (STARD) initiative guidelines to assess the validity of diagnostic tests [20]. During the months of September and October 2007 (end of the local rainy season), under five-year-old children suspected of suffering from malaria and referred to the MRL by their physicians were consecutively enrolled until the established sample was reached. Posttreatment monitoring and relapsing cases were not included in the analysis.

The legal guardians of the recruited children were informed about the study, and after that their consent to participate was requested. The test outcome was shared with the physicians to help with the final diagnosis. The study was approved by the National Malaria Control Programme authorities (Ministry of Health and Social Welfare), and no bioethical impediments were found to disallow the study.

The sample size was calculated with an alpha error of 5% and a statistical power of 80%. The estimated prevalence was established from the frequency of positive diagnostics by microscopy in the MRL in 2006: 31% by Plasmodium spp. of which more than 90% corresponded to P. falciparum infections. A 10% of children were added to the sample size to offset drop outs or errors. The estimated initial sample was 360 children, which was subsequently increased to 400 children.

2.2. Blood Samples Collection and Analysing

A blood sample was taken from the finger of the under five-year-old children recruited in the study. Giemsa-stained thick and thin blood films were performed for each sample. Parasite density (parasites/μl of blood) was established by comparing the ratio of counted parasites within a hundred microscopic fields against counted white blood cells (WBCs). Two lab technicians using a double-blind procedure performed the microscopic diagnosis. Regarding diagnostic result, it was considered a discordant result; there is no concordance on the species differentiation, either simple or mixed infections, and/or there was a difference of more than 25% on the parasite count. In that case, a third technician was involved in the diagnosis. When the discrepancy criteria kept on, the three lab technicians reassessed the sample and settled a final diagnosis.

The diagnostic capacity (sensitivity and specificity) of the lab technicians was previously certified by means of reading a slide bank (20 positives for Plasmodium spp. and 10 negatives for plasmodia) that had previously been validated by experts [21]. After gathering the blood sample to perform the microscopic diagnosis, 5 μl were collected to carry out the rapid immunochromatographic test (ICT Malaria Combo Cassette Test; ICT Diagnostics, Cape Town, South Africa; http://www.ictdiagnostics.co.za). ICT Malaria Combo Cassette Test is a rapid, in vitro diagnostic test for the detection of circulating P. falciparum antigens (HRPII antigen) and an antigen that is common to all four species of malaria (aldolase antigen). HRPII is profusely expressed on the surface of the red blood cells parasitized by P. falciparum while aldolase is produced during the glycolytic cycle of the parasite in all the plasmodia species. The test uses specific monoclonal antibodies for each of HRPII and aldolase antigens that have been immobilized across the test strip. A procedural control line is also immobilized across the test strip and will always appear if the test has been correctly performed. The test is positive when 2 or 3 lines appear in the test window. As long as the control line appears the following represents positive test results: P. falciparum positive: a control line and one upper test line (HRPII antigen); P. falciparum infection or a mixed infection (P. vivax., P. malariae, and P. ovale): a control line and 2 test lines (HRPII and aldolase antigens); P. vivax., P. malariae, and P. ovale or a mixed infection of all three: a control line and one lower test line (aldolase antigen). The procedure followed and the reagents used for the reading were those recommended and provided by the test manufacturer.

2.3. Statistical Analysis

The data was analysed using Stata v. 8.2 software (Stata Corporation, College Station Texas, 2007). Taking the microscopy as the diagnostic reference test (“gold standard”), the sensitivity and specificity of the RDT by detected parasitemia ranks were calculated in order to determine its validity. In addition, the reliability of the rapid test when detecting P. falciparum was determined by means of calculating the positive predictive value (PPV), the negative predictive value (NPV), the area under the receiver operator characteristic curve (AUC), and the likelihood ratios.

The Cohen's kappa test was used to analyse the agreement between the two diagnostic tests. Finally, the Spearman correlation was established between the parasitemia and the reading of the intensity of the RDT diagnostic band. All estimated parameters are detailed with a 95% Confidence Interval (CI) unless otherwise stated.

3. Results

The sensitivity of the diagnosis in the reading of the slide bank by the lab technicians that performed the study was 100% (confidence interval, CI90% : 86%—not applicable (NA)) and the specificity 100% (CI90% : 74%—NA). The accuracy percentage to determine the species ranged from 47% to 76% and from 43% to 68% for quantifying the parasitemia.

A total of 400 samples, 207 of which were from males (51.7%) and 193 from females (48.3%), were analysed. The average age of the total of the sample was 1.8 year old (standard deviation, SD  =  1.3 year old). The average packed cell volume (PCV) was 32.2% (SD  =  4.9%).

Out of the total samples analysed by microscopy, 270 (67.5%) were negative for Plasmodium spp., 66 (16.5%) positive for P. falciparum, 30 (7.5%) P. falciparum + P. malariae, 20 (5.0%) P. falciparum + P. ovale, 12 (3.0%) P. malariae, and 2 (0.5%) P. ovale. The median of the parasitemia was 5,360 parasites per μl of total blood (Interquartile range (IR): [80-11,160] parasites per μl of total blood). P. falciparum gametocytes were observed in 10% of the positive samples (13/130) and schizonts in 6.9% (9/130).

The ICT test diagnosed 245 (61.25%) samples as negative, 117 (29.25%) as mixed infections that contained P. falciparum, 21 (5.25%) as infection by other plasmodia species, and 17 (4.25%) as simple infection by P. falciparum.

The control band of all the ICT tests performed indicated that the test had been performed correctly. The reading of the band was considered visible in 58.7% of the total of positive samples (91/155), very weak in 20.7% (32/155), weak in 13.5% (21/155), and very visible in 7.1% (11/155).

The sensitivity of the ICT test to detect Plasmodium spp. compared to microscopy was 81.5% (CI95%  =  73.8%–87.8%) and the specificity 81.9% (CI95%  =  76.7%–86.3%). The AUC was 0.82 (CI95%  =  0.78–0.86). The PPV was 68.5% (CI95%  =  60.4%–75.6%) and the NPV 90.2% (CI95%  =  85.8%–93.6%). On the other hand, the LR+ was 4.49 (CI95%  =  3.44–5.86) and the LR− 0.27 (CI95%  =  0.16–0.33).

In order to analyse the reliability of the RDT when detecting P. falciparum monoinfection, the mixed infections diagnosed by microscopy were not taken into account while in the ICT test, both the P. falciparum monoinfection (HRPII-specific antigen) and the mixed infections were included in the analysis. Only 5 out of 66 (7,6%) P. falciparum monoinfections diagnosed by microscopy were identified as such by the ICT test.

The sensitivity of the ICT test to detect P. falciparum monoinfection compared to microscopy was 69.7% (CI95% = 57.1%–80.4%) and the specificity 73.7% (CI95% = 68.6%–78.3%). The AUC was 0.72 (CI95% = 0.66%–0.78%). Nonetheless, when P. falciparum monoinfections that exclusively expressed the HRPII antigen were considered, the sensitivity fell to 17%.

The PPV was 34.3% (CI95% = 26.3%–43.0%) and the NPV 92.5% (CI95% = 88.6%–95.3%). The LR+ was 2.65 (CI95% = 2.08–3.36) and the LR− 0.41 (CI95% = 0.28–0.59).

All the above values differed when they are analysed using parasitemia ranges (see Table 1).

Table 1

Sensitivity and specificity of the ICT test compared to microscopy by Plasmodium falciparum monoinfection and parasitemia ranges.

Parasites/μl	N*	Sn	Sp	PPV	NPV	LR+	LR−	AUC
<101	27	33.3 (54.0–16.5)#	88.8 (92.2–84.4)	22.5 (38.5–10.8)	93.2 (95.9–89.4)	2.9 (5.6–1.6)	0.7 (0.9–0.6)	0.6 (0.7–0.5)
101–500	4	50.0 (93.2–6.7)	88.3 (83.9–91.8)	5.8 (19.7–0.7)	99.2 (99.9–97.1)	4.3 (12–1.5)	0.5 (1.5–0.2)	0.7 (0.9–0.4)
>500	35	100.0 (100.0–90.0)	73.8 (78.5–68.7)	29.2 (38.2–21.2)	100.0 (100.0–98.5)	3.8 (4.6–3.2)	0	0.8 (0.9–0.8)

Sn:  sensitivity.

Sp: specificity.

PPV: positive predictive value.

NPV: negative predictive value.

LR+: Likelihood Ratio of positive test.

LR−: Likelihood Ratio of negative test.

AUC: Area Under the Receiver Operator Characteristic Curve.

*Number of individuals with monoinfection by Plasmodium falciparumin each parasitemia group.

#All values in brackets indicate 95% Confidence Intervals.

Both tests agreed in their diagnostics 67.2% of the time (with the expected coincidence being equal to 54.7%). The concordance measured by the Cohen's kappa test was 0.28 (se: 0.04).

The intensity of the reading band of the ICT test was directly correlated with the intensity of the parasitemia observed by microscopy (Spearman rho = 0.73).

4. Discussion

The sensitivity of the ICT Malaria Combo Cassette Test to detect Plasmodium spp. was similar to that observed in other studies with RDT based on the detection of the HRPII antigen. Moreover the specificity of obtained values ranked slightly under the published data where it was usually over 90% [6–8, 22–25], although this result should be considered with caution due to limited sample size. The ICT Combo Cassette Test was tested in the Round 1 WHO Product Testing of RDTs (2008) performing a detection rate against wild type P. falciparum samples (n = 79) of 86.08% at low parasite density (<200 parasites/μl) and 100% at higher parasitemias (2000–5000 parasites/μl) while it only identified 15 out of 20 (75%) cultured P. falciparum lines when parasitemia was below 200 parasites/μl [26]. Nonetheless, the fact that the RDT type, target population, and epidemiological context influence the feasibility and reliability of these immunochromatographic tests must be taken into account. In addition, exposure to high temperatures during transport and storage, commonplace in tropical countries, can degrade the tests and bias the results [27]. In the present assessment, the RDT storage and transport conditions were controlled throughout the study, and the storage temperature recommended by the manufacturer was not exceeded.

The ICT rapid test stood out for its NPV, in other words, for its greater capacity to detect precisely the absence of malaria while the PPV was lower, particularly for the specific diagnostics, which demonstrated the weakness of the test to diagnose positive cases by microscopy.

The diagnostic capacity of the ICT test to detect P. falciparum was marked by the presence of false positives (65.7% compared to 31.6% false positive in the detection of Plasmodium spp.) but it does not significantly bias the assessment of the tests in clinical practice [28, 29]. In extensive reviews on malaria RDT, these false positives have been linked with individuals that had been recently treated with antimalaria drugs and with the presence of the serum rheumatoid factor [6, 7]. In our study, it is likely that many of the children had received treatment in the days prior to the sample collection without being reported by their guardians, and therefore they presented residual circulating HRPII antigens. On the other hand, mistakes in microscopy slide reading, although unlikely due to the participation of three trained microscopists, can not be excluded.

False positives could also be attributed to chronic infections [30]. Even expert lab technicians find it difficult to detect chronic infections in high endemic zones, characterised by low parasitemias and few or nonexistent clinical manifestations [31]. However, this fact was unlikely in this study since children suspected of suffering from malaria (with clinical criteria) were only recruited.

In any case, it has been indicated that the detection and treatment of these cases do not have great clinical relevance, albeit they are important from the transmission point of view as the majority of them present gametocytemia [32]. ICT tests are not able to discriminate persisting asexual parasitemia from gametocytemia. The detection of residual HRPII antigen detection could be an advantage as it widens the diagnostic opportunities of fluctuant parasitemias.

In the same way as in other studies where the parasitemia subgroups have been analysed, variations were observed in the feasibility and reliability of the test to detect P. falciparum according to parasitemia [6, 7].

Particularly noteworthy is that the ICT test was only capable of identifying 7.6% of the P. falciparum monoinfections diagnosed by microscopy. Therefore, the antigens present in all the plasmodium species seemed to play a greater role in the diagnostics than the specific HRPII antigen of the P. falciparum. Subsequent controlled studies would be necessary to assess properly these facts. Nonetheless, it has been put forward that the diversity of the genes that codify the HRPII antigens in P. falciparum from different geographical areas may explain the disparity in the results obtained in the different field tests [33]. However, the sequences that codify the aldolase are well conserved in the different plasmodium populations [34].

Bearing in mind the cost that the ICT tests can represent for the national health system of Equatorial Guinea, its cost effectiveness needs to be carefully considered [35]. Many of the current RDT have been shown to have a lower cost per unit than a three-day cycle with the majority of the artemisinin-based combination therapies (ACTs) [36]. The introduction of the ACTs in the country probably will increase the cost effectiveness of the rapid tests if the over-prescribing of these drugs is reduced [37].

As has already been discussed, the malaria transmission levels condition the diagnostic reliability using RDT. On the island of Bioko, the IRS-based control programme and growing urban development, among other factors, have managed to significantly reduce the prevalence of the disease [16]. Thus, when the malaria transmission levels are low or moderate, the RDTs could be more cost effective than low-quality microscopy, usual in resource-poor health systems [38]. In high prevalence areas (over 70%), the empirical treatment is the first option in cost effectiveness terms [39].

Models developed in conjunction with the WHO consider cost effective the RDT in comparison with the presumptive treatment up to high prevalence values of P. falciparum infection. This model is subordinated to the prescribers basing their therapy decisions in line with the test outcome [40]. Therefore, adherence to the diagnostic test outcome determines the cost effectiveness of the RDT, and it must condition the health policies that regulate its use.

The WHO recommends that all patients are diagnosed as suffering from malaria by means of a test that shows the parasite infection, with the possible exception of children in high epidemic areas where the high mortality of the badly diagnosed cases does not allow the weaknesses of the diagnostic tests to be assumed [41, 42]. This last point, which directly affects the cohort studied in this analysis, would clearly limit the use of the RDT and the strict following of their outcomes to decide the treatment for the under five year-old children.

5. Conclusions

In Equatorial Guinea, the rapid ICT Malaria Pf. Pv. Po. Pm Combo Cassette Test has proven to be an acceptable test to detect high P. falciparum parasitemias in peripheral blood under experimental conditions. Therefore, it can be used as a support tool to diagnose malaria in resource-poor health care settings, where quality microscopy diagnosis is either not present or not guaranteed. However, the existing shortcomings regarding its upkeep and handling should be taken into account. On the other hand, the decrease of sensitivity at medium and low parasitemias hampers that ICT can replace properly performed microscopy diagnosis at present in under five-year-old children, the most affected by malaria in a high prevalence country as Equatorial Guinea.