Additional Evaluation of the Point-of-Contact Circulating Cathodic Antigen Assay for Schistosoma mansoni Infection.

Studies of the urine-based point-of-contact cathodic circulating antigen test (POC-CCA) in Schistosoma mansoni-endemic settings in Africa indicate it has good sensitivity in detecting infections, but in areas of low prevalence, the POC-CCA can be positive for persons who are egg-negative by Kato-Katz stool assays. We examined the POC-CCA assay for: (a) batch-to-batch stability; (b) intra-reader and inter-reader variability; (c) day-to-day variability compared to Kato-Katz stool assays, and (d) to see if praziquantel (PZQ) treatment converted Kato-Katz-negative/POC-CCA positive individuals to POC-CCA negativity. We found essentially no batch-to-batch variation, negligible intra-reader variability (2%), and substantial agreement for inter-reader reliability. Some day-to-day variation was observed over 5 days of urine collection, but less than the variation in Kato-Katz stool assays over 3 days. To evaluate the effect of treatment on Kato-Katz(-)/POC-CCA(+) children, 149 children in an area of 10-15% prevalence who were Kato-Katz(-) based on 3 stool samples but POC-CCA(+) were enrolled. Seven days after treatment (PZQ 40 mg/kg) samples were again collected and tested. Almost half (47%) POC-CCA positive children turned negative. Those still POC-CCA positive received a second treatment, and 34% of them turned POC-CCA negative upon this second treatment. Most who remained POC-CCA positive shifted each time to a "lesser" POC-CCA "level of positivity." The data suggest that most Kato-Katz-negative/POC-CCA positive individuals harbor low-intensity infections, and each treatment kills all or some of their adult worms. The data also suggest that when evaluated by a more sensitive assay, the effective cure rates for PZQ are significantly less than those inferred from fecal testing. These findings have public health significance for the mapping and monitoring of Schistosoma infections and in planning the transition from schistosomiasis morbidity control to elimination of transmission.

Introduction

WHO guidelines for control and elimination of schistosomiasis involve pre-treatment and periodic post-treatment evaluations of the prevalence of Schistosoma infections to inform programmatic decisions on whom, and how often to treat within endemic areas (1). For Schistosoma mansoni, this has usually involved mapping by examinations for parasite eggs in the stool, most often by the Kato-Katz thick-smear microscopic technique (2). By making stool exams more feasible on a population level, the Kato-Katz technique revolutionized the process of obtaining intensity and prevalence data from large numbers of people. This assay worked very well for control programs in areas of high intensity and corresponding high prevalence. It was used extensively and became established as part of WHO guidelines for morbidity control programs, with prevalence estimates usually based on examination of one stool per subject, using two separate Kato-Katz slides. In situations with high-mean intensities of infection (and of corresponding high prevalence), the Kato-Katz is believed to provide close to true prevalence estimates of local infection for schistosomiasis control programs (3). Research programs also use it extensively – most often testing three consecutive stools, two Kato-Katz slides on each stool for greater accuracy of diagnosis. However, the Kato-Katz assay has documented day-to-day and intra-stool variability, especially in communities with moderate-to-low intensity of infection and corresponding low prevalence. Day-to-day variation is biological (4), and intra-stool variation is related to sampling error based on the amount of stool evaluated (5). Thus, the Kato-Katz test is acknowledged to be relatively insensitive at lower levels of intensity of infection, often corresponding to lower prevalence as well. Over the course of a control program, the ability of the Kato-Katz assay to provide true prevalence estimates will become less and less as the population mean intensity of infection decreases (6–14). Over 20 years ago, de Vlas et al. (3, 15, 16) published a series of statistical reviews that showed that programmatic dependence on a single stool examination led to underestimates of true prevalence and that observations based on such surveys could result in inaccurate allocation of control measures.

Currently, an ideal diagnostic tool for implementing schistosomiasis control programs would be one that could be easily used for regional mapping to initiate a “gaining control” program. In this scenario, low sensitivity may be acceptable but high specificity is good. A tool for monitoring the post-treatment impact of an initial “gaining” or ongoing “sustaining” control program would need to have high sensitivity for low intensity infections and retain high specificity. It would serve as the tool that determines if it is time to switch MDA protocols or add adjunct strategies, such as snail control, behavioral change, or implementation of water and sanitation measures, as prevalence and intensities decrease. The tool used to determine when it is time to switch from a standard control program to a more aggressive elimination program must have very high sensitivity, and ideally high specificity too. A post-elimination surveillance tool would require excellent sensitivity and excellent specificity based on exposure or infection and allow high throughput.

Over the last several years, multiple studies have evaluated a commercially available, urine-based, point-of-contact (POC) cassette assay that detects circulating cathodic antigen (CCA). CCA is derived from the gut of adult schistosomes that are established in the human circulation (8, 17–22). The antigen is eliminated by the kidneys into the urine where it can be detected by a standardized immunoassay. In general, the results of earlier evaluations indicate that the POC-CCA assay is suitable for the prevalence mapping required for regional schistosomiasis morbidity control programs (21). For this purpose, POC-CCA testing appears just as good as or better than using Kato/Katz assays. With the additional advantage of being urine-based, it is able to be done on-site, and appears to be more sensitive for detection of S. mansoni infection at low egg intensities (8, 21, 22). The POC-CCA is now being used in several countries, but its performance should be continually reassessed to determine its potential flaws and limitations as control programs mature. Because there is no sensitive and specific “gold standard” for diagnosis of active S. mansoni infection, there remain questions about the overall specificity and sensitivity of the POC-CCA assay and thus about its use in place of the Kato-Katz assay for programmatic decision making. One of the questions revolves around observer readings of faint diagnostic bands, considered positive by the manufacturer and called “trace” results. However, it has also been suggested that information can be obtained based on the differential intensity of a positive POC-CCA band to indicate, in semi-quantitative fashion, levels of intensity of infection (13, 21, 23). In addition, it is questioned if the POC-CCA assay could prove more reliable than the Kato-Katz as an indicator of the effectiveness of drug treatment of S. mansoni infections (23). In the present report, we address some of these questions based on prospective laboratory and field trials in an S. mansoni-endemic area of Kenya, on additional testing data from a non-endemic site in coastal Ecuador, and previously reported data (21) from a non-endemic area of Ethiopia.

Materials and Methods

Ethics statement and participant selection

The study protocol was approved by the scientific steering committee (SSC) of the Kenya Medical Research Institute (KEMRI, SSC number 1820), the Ethical Review Committee (ERC) of KEMRI, the Institutional Review Boards (IRB) of the Centers for Disease Control and Prevention (CDC), the Pontificia Universidad Catolica del Ecuador, Quito, Ecuador, the Aklilu Lemma Institute of Pathobiology, Addis Ababa University of Ethiopia and of the University of Georgia. Parents/guardians, and participating children were informed about the purpose and procedures of the study. Written informed consent was obtained from the parents/guardians and the children participating in the study gave assent prior to study enrollment. All samples obtained in the study were coded and treated confidentially.

Study population and sample collection

Most of this study was conducted in schools near the shores of Lake Victoria, western Kenya from January 2013 to April 2014. Studies in areas known not to be endemic for schistosomiasis but endemic for soil-transmitted helminths (STHs) were done in Ethiopia [as previously reported; (21)], and in the coastal region of Ecuador. From each school, children above the age of 6 years for whom consent and assent had been obtained were selected for study inclusion. All children who were positive for schistosome infection were treated with praziquantel (PZQ) (40 mg/kg) and children positive for STHs were given 400 mg Albendazole as a single dose. All contact with the children was done at the school.

Laboratory methods and study designs

Kato-Katz testing

Stool samples collected were processed within 24 h of collection using the Kato-Katz thick-smear technique in duplicate (normally three stools, six slides) for the quantitative determination of S. mansoni and STHs (2). Each slide was read within 1 h of preparation to diagnose presence of hookworm, Ascaris lumbricoides, and Trichuris trichiura by two well-trained microscopists. The S. mansoni readings were performed at least 24 h after slide preparation and recorded as number of eggs per gram of stool (EPG) (2). The results of all slides read were averaged and the intensity of infection classified according to the WHO guidelines (24). STH infections were simply categorized as being either positive or negative.

POC-CCA testing

Urine samples were collected and tested in the laboratory for positivity and band intensity by the commercially available POC-CCA assay, according to manufacturer’s instructions (Rapid Diagnostics, Pretoria). Briefly, one drop of urine was added to the well of the testing cassette and allowed to absorb. Once it was absorbed, a drop of buffer (included in the kit) was added into the same well and the assay was allowed to develop for 20 min, at which time the results were read. The tests were considered invalid if an internal control band did not appear or if the tests were left to develop for more than 25 min after addition of the buffer before being read. In these few cases, the samples were re-run with a new test cassette and scored appropriately. To score the intensity of the POC-CCA assay results, the intensity of the test band was compared to that of the control band. Positive results were given a score of “trace” if the band was barely visible, 1+ if the test band was less intense than the control band, 2+ if the test band was of equal intensity as the control band, and 3+ if the test band was more intense than the control band.

Cassette batch variation

Five separate batches of CCA test kits were used for this study. Three readers were provided with 10 urine samples from subjects who either did not have S. mansoni infections or had infections that ranged from low to moderate intensity based on Kato-Katz stool examinations. Each of the 3 readers read the 10 urine samples using 10 POC-CCA cassettes from Box # 1. The urine samples were shuffled and then read again using 10 cassettes from Box # 2; this retesting of the 10 test urines continued using cassettes from the remaining boxes, 10 per box, to Box # 5.

Intra-reader reliability

Five readers were provided with 10 urines that ranged from negative to moderately positive, as described above. In Round 1, each reader read urine results using 10 individual POC-CCA cassettes read at 20 min. For Round 2, the cassettes were randomly rearranged, and each reader re-read and re-scored the same (masked) cassettes for a second time within ~5 min, to keep within the time limit for valid cassette readings.

Inter-observer reliability

Five readers were provided with 30 urines that ranged from negative to moderately positive, as described above. The five readers read the urines on POC-CCA cassettes independently and the results were recorded without discussion.

Day-to-day variability

Seventy-three participants were recruited for this study. Five consecutive urine samples for POC-CCA and three consecutive stool samples for Kato-Katz were collected and assessed to determine if there were any fluctuations in the POC-CCA or Kato-Katz readings based on the different days of collection.

Post-treatment readings and re-treatments

One hundred forty-nine children from schools that had received two or three rounds of annual MDA with PZQ and who were POC-CCA positive but Kato-Katz negative (based on a single urine sample and three stool samples) were enrolled at baseline, and treated with PZQ based on the POC-CCA positive results. Seven days later, a single urine and stool sample were collected and tested by POC-CCA and Kato-Katz, respectively. Children who were still POC-CCA positive were treated for a second time, and 7 days later urine and stool samples were again collected and assayed. Children who were still POC-CCA positive were again treated for a third time without further testing. A schematic diagram of the post-treatment study is shown in Figure 1.

Figure 1

Flow diagram of post-praziquantel treatment study, Kenya.

Data analyses

Data collected from evaluations of school children included egg intensity readings and POC-CCA cassette band readings and were entered into an Excel database and analyzed using SAS statistical software (SAS, Cary, NC, USA). Fleiss kappa statistical analysis (25) was used to measure the level of inter-reader agreement among five readers, with values of 0.00–0.20 considered slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1.00 almost perfect agreement (26). The estimated diagnostic accuracy of the Kato-Katz and POC-CCA tests was compared using Bayesian latent class modeling software (27), or by measuring detection accuracy against a “combined gold standard.” The combined gold standard was defined as any one positive test out of all eight tests (five consecutive CCA and three Kato/Katz tests) for any individual subject. In a separate analysis, Spearman’s rank correlation was used to compare trace, 1+, 2+, 3+ band intensity on the POC-CCA assay with estimated infection intensity determined by Kato-Katz egg counts. Fleiss kappa and Spearman’s rho were analyzed using IBM SPSS 21.

Results

Cassette batch variation

When different batches of the cassettes provided by the manufacturer were tested on the same 10 urine samples, there was no real variation between them. Nine of 10 samples showed the same results across all 5 batches for all 3 readers. Only one sample gave varying results when cassettes were taken from different batches, being negative for some lots and trace (positive) on others. This same variation occurred for all three readers.

Intra-reader reliability

When 5 trained readers read two rounds of the same 10 specimens, there was only 1 reading of the 50 pairs of repeat readings that was not replicated. That one was read as negative the first time and as trace (positive) when it was re-read by one of the readers. This corresponds to a 2% intra-reader variation.

Inter-reader reliability

When 5 trained readers were evaluated in regard to the reliability of their readings of the same 30 specimens and the results analyzed by Fleiss kappa statistics the overall Fleiss kappa value for inter-reader agreement of the POC-CCA scores was 0.70 (95% confidence interval: 0.58, 0.81; P < 001). This indicates substantial agreement between the five different readers.

Day-to-day variability

Seventy-three participants each had POC-CCA tests on five consecutive days’ urines; and Kato-Katz tests on three consecutive days’ stools (two slides each) to evaluate S. mansoni prevalence. This study sample was representative of a moderate-to-high prevalence area (44% on the first Kato-Katz stool/two slides). As per the manufacturer’s instructions, “Trace” readings were considered positive. For each assay, prevalence estimates based on single tests did not vary significantly between days, ranging between 81 and 88% for the five consecutive POC-CCA and between 41 and 44% for three consecutive Kato/Katz tests. Almost all (94%) of children were positive for at least one of the five POC-CCA tests while 70% were positive for the at least one of Kato/Katz tests (Table 1).

Table 1

Schistosomiasis prevalence based on one or several POC-CCA and Kato-Katz tests among 73 school children, Kenya.

		Prevalence, N (%)
POC-CCA
Day	1	59 (81)
	2	62 (85)
	3	62 (85)
	4	62 (85)
	5	64 (88)
At least 1 of 5 CCA tests positive		69 (94)
Kato-Katz
Day	1	32 (44)
	2	31 (43)
	3	30 (41)
At least 1 of 3 K-K tests positive		51 (70)

When the first three CCA consecutive tests were taken together, 82% of participants had consistent daily results compared to only 49% of participants with consistent daily results by Kato-Katz assays. Mixed results (i.e., both positive and negative results for the same participant) were observed in 18% of the participants for the POC-CCA tests but in 48% of Kato-Katz tests. Thus, while there was some day-to-day variation for both assays, the Kato-Katz varied much more in terms of diagnosing the presence of infection than the POC-CCA (Table 2).

Table 2

Consistency of results for three consecutive diagnostic tests for schistosomiasis on 73 school children, Kenya.

	N (%)
Result of 3 POC-CCA tests
All 3 negative	5 (7)
All 3 positive (Trace/1/2/3)	55 (75)
Mixed positive and negative results	13 (18)
Result of 3 Kato-Katz tests (2 slides taken together)
All 3 negative	21 (29)
All 3 positive	15 (20)
Mixed positive and negative results	35 (48)

The sensitivity and specificity of the POC-CCA assay were evaluated against an artificial “gold standard” of any one positive test of the combined eight tests (five POC-CCA and three Kato-Katz). These analyses indicated that the POC-CCA assay provided high levels of both sensitivity and specificity, as shown in Table 3.

Table 3

Sensitivity and specificity analysis of POC-CCA one day testing evaluated against a combined gold standard of at least 1 of 8 tests (5 POC-CCA and 3 Kato-Katz) being positive.

	Sensitivity, % (95% CI)	Specificity, % (95% CI)
CCA day 1	85.5 (74.9–2.8)	100 (40.2–100)
CCA day 2	89.9 (80.2–95.8)	100 (40.2–100)
CCA day 3	89.9 (80.2–95.8)	100 (40.2–100)
CCA day 4	89.9 (80.2–95.8)	100 (40.2–100)
CCA day 5	92.7 (83.9–97.6)	100 (40.2–100)

When the POC-CCA assay was tested in areas in Ethiopia and Ecuador that had never been endemic for schistosomiasis but were endemic for STH, only one presumably false positive test was found among 100 participants in Ethiopia (21), while none in the 143 participants from Ecuador was positive.

A Bayesian latent class model (BLCM) was developed using data from all 73 subjects in this part of the study, assessing the most likely performance characteristics of 1 up to 5 daily POC-CCA results, and 1 up to 3 daily Kato-Katz stool results, guided by the POC-CCA specificity data Ethiopian and Ecuadorian children described above (Figure 2). By this analysis, it was estimated that the sensitivity of a single POC-CCA assay was likely to be much greater than that of a single Kato-Katz test, and even more sensitive than performing three Kato-Katz tests in detecting S. mansoni infection in the study population. The estimated specificity for both POC-CCA and Kato-Katz assays approached 100%.

Figure 2

Bayesian latent class modeling values for sensitivity and specificity of single and multiple POC-CCA tests versus one or more Kato-Katz tests, Kenya.

Infection intensity using POC-CCA can be determined only semi-quantitatively by comparing the density of the test line that develops to the control line provided in the cassette. Yet, we found a good correlation between egg intensity determined by Kato-Katz as compared to POC-CCA scores based on readings the intensities of the positive bands. All the 10 participants who had moderate or high egg counts by Kato-Katz had an unambiguously positive PCO-CCA result. There was a significant correlation between a participant’s mean EPG and their CCA score on all of the tested days (Figure 3).

Figure 3

Correlation between intensity of infection by Kato-Katz (EPG) and band intensity by POC-CCA. Representative boxplot is shown from day 5 POC-CCA test. There is a significant association between the EPG and CCA band intensity: Spearman’s rho = 0.601 (P < 0.001).

Post-treatment readings and re-treatments

For this phase of the study, 149 children who had tested negative by Kato-Katz (3 stools/2 slides each) but positive by POC-CCA (tested on a single urine) were selected from an area that had achieved 10–15% overall S. mansoni prevalence (by Kato-Katz) following 2 or 3 rounds of MDA with PZQ. One week after testing, 126 of these children were treated with PZQ. Of those not lost to follow-up, 55 of 117 (47%) became POC-CCA negative on repeat testing performed 1 week post-treatment. Those who remained POC-CCA positive (and were available) were treated a second time, and 7 days later, 19 of 56 (34%) children were negative based on follow-up POC-CCA testing at that time. The Kato-Katz assays of these children, who were selected as Kato-Katz negative, remained negative after each treatment round.

Discussion

Many schistosomiasis control programs in Sub-Saharan Africa are now at the point of mapping and moving toward gaining control by mass drug administration. Some are sustaining control at a moderate level but will eventually wish to move toward elimination of transmission of S. mansoni infection. The sensitivity of the assay used to evaluate prevalence in these different phases becomes critical, as the current Kato-Katz test is known to become significantly less sensitive as egg count intensities decrease (23, 24, 28–33). Our data and those from other studies indicate that the POC-CCA assay is more sensitive than the Kato-Katz assay (13, 22). Therefore, in moderate to lower prevalence and intensity areas, POC-CCA will indicate higher prevalence levels than the Kato-Katz. Since prevalence estimates impact programmatic guidelines for coverage, questions have been raised about the true sensitivity and specificity of POC-CCA readings. For this reason, we undertook further evaluation of this urine-based, commercially available assay.

Our study found little or no variability of performance between batches of the POC-CCA cassettes, and intra-reader variability was minimal when trained technicians performed the assay. In addition, Fleiss kappa statistics indicated substantial agreement in regard to inter-reader reliability among 5 trained readers reading 30 specimens (Fleiss kappa value = 0.70). We had previously observed that when non-technician field workers were minimally trained and then asked to read multiple POC-CCA tests, there was considerable variation in the readings. Since there was little or no variability seen when trained readers (laboratory technicians) were used, this highlights the need for adequate training prior to use of the POC-CCA assay.

Day-to-day variability in per-subject prevalence, based on readings from 3 successive days of Kato-Katz or 5 days of POC-CCA, was minimal and almost all variation occurred at the very low intensity end of the CCA test spectrum, involving “trace” readings. Also, based on the first 3 days of testing run in parallel, the day-to-day variability of the POC-CCA readings (18%) was much less than those of the Kato-Katz (48%).

Our post-treatment data suggest that, based on the estimated sensitivity of the POC-CCA assay, many of the Kato-Katz-negative/POC-CCA positive results reflect continued infection, i.e., these subjects have remaining worms that continue to make detectable levels of CCA. This means that currently accepted PZQ cure rates for Schistosoma infection are likely to be significantly overestimated, based on the post-treatment inaccuracy of a lower-sensitivity assay, the Kato-Katz. PZQ is typically said to produce 70–90% cure rates as evaluated by Kato-Katz assays (34–37). If, in fact, many treated individuals continue to harbor low–intensity infection (as determined by a more sensitive assay such as the POC-CCA) this suggests the need to treat many more people, more often, for persistent infection. Such a shift in recommendations would constitute a significant alteration in programmatic decision-making algorithms from when Kato-Katz testing is used as the sole diagnostic criterion.

Our data and those of others (13) indicate that the POC-CCA assay can be used in post-treatment studies and programs, and is indeed more suitable for post-treatment evaluations where the goal is to detect uncured persons. In an earlier drug efficacy study, the POC-CCA was found to be more sensitive than six Kato-Katz slides (two/slides on three stools) at 4-weeks-post-praziquantel, and again at 6 months post-praziquantel (13). We have shown that, following treatment, there are still those with positive POC-CCA readings who are K-K negative – and these drop by roughly half upon a second treatment and again after another retreatment. Those who do not convert to POC-CCA negative upon these treatments decreased the intensity of their CCA bands, perhaps indicating successive partial cures (Figure 4).

Figure 4

POC-CCA results following two rounds of praziquantel treatment, Kenya.

Locations that were endemic for STH but not human schistosomes (areas of Ethiopian and Ecuador) yielded only one “trace” POC-CCA positive (from Ethiopia) child out of 243 children tested, confirming the high specificity of this assay.

In conclusion, our data show that most (if not all) Kato-Katz-negative/POC-CCA positive individuals in endemic areas are likely to harbor low level infections with S. mansoni. While not a perfect test (as seen in the day-to-day variability) the POC-CCA assay appears to be highly suitable for initial mapping of S. mansoni areas, and for subsequent post-treatment programmatic decision making. The fact that it indicates higher prevalences than the less sensitive Kato-Katz in areas with low-average intensity should now be factored into how treatment decisions will be made. Our data bring up the possibility that many people in endemic areas are harboring low levels of worms that are producing few, if any, eggs on a daily basis. This raises an issue that is seldom voiced, but may be quite important in programmatic thinking. What are the consequences of such a scenario? If few or no eggs are being released every day, does that level of infection still lead to morbidity? If few or no eggs are being released, it would seem that the implications for transmission would be important. We do not yet have answers to these questions, but believe they need to be discussed. They will continue to arise as the POC-CCA and other more sensitive assays are introduced and employed more widely in the field.

Author Contributions

The study was designed by DC, NK, CK, CC, and PM. EO and PC coordinated field activities and laboratory work. NK and CK analyzed the data. All authors actively contributed to the interpretation of the findings, data compilation, and manuscript preparation. All authors read and approved the final manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.