Predicting Plasmodium falciparum infection status in blood using a multiplexed bead-based antigen detection assay and machine learning approaches.

BACKGROUND: Plasmodium blood-stage infections can be identified by assaying for protein products expressed by the parasites. While the binary result of an antigen test is sufficient for a clinical result, greater nuance can be gathered for malaria infection status based on quantitative and sensitive detection of Plasmodium antigens and machine learning analytical approaches.
METHODS: Three independent malaria studies performed in Angola and Haiti enrolled persons at health facilities and collected a blood sample. Presence and parasite density of P. falciparum infection was determined by microscopy for a study in Angola in 2015 (n = 193), by qRT-PCR for a 2016 study in Angola (n = 208), and by qPCR for a 2012-2013 Haiti study (n = 425). All samples also had bead-based detection and quantification of three Plasmodium antigens: pAldolase, pLDH, and HRP2. Decision trees and principal component analysis (PCA) were conducted in attempt to categorize P. falciparum parasitemia density status based on continuous antigen concentrations.
RESULTS: Conditional inference trees were trained using the known P. falciparum infection status and corresponding antigen concentrations, and PCR infection status was predicted with accuracies ranging from 73-96%, while level of parasite density was predicted with accuracies ranging from 59-72%. Multiple decision nodes were created for both pAldolase and HRP2 antigens. For all datasets, dichotomous infectious status was more accurately predicted when compared to categorization of different levels of parasite densities. PCA was able to account for a high level of variance (>80%), and distinct clustering was found in both dichotomous and categorical infection status.
CONCLUSIONS: This pilot study offers a proof-of-principle of the utility of machine learning approaches to assess P. falciparum infection status based on continuous concentrations of multiple Plasmodium antigens.

Introduction

Malaria remains a significant global public health burden that is responsible for an estimated 229 million infections worldwide and 409,000 deaths annually, with the vast majority of malaria cases and deaths occurring in Sub-Saharan Africa [1]. The knowledge of malaria epidemiology and implementation of control measures in any endemic setting are imperative for the reduction of transmission and eventual transition to elimination efforts [2]. Utilizing rapid diagnostic tests (RDTs), detection of Plasmodium antigens directly from human blood provides an effective measure of active malaria infection [3]. In 2019, 348 million RDTs were sold by global manufacturers with the most commonly used RDTs detecting the presence of Plasmodium falciparum histidine rich protein 2 (HRP2), though tests are available which also detect Plasmodium aldolase and lactate dehydrogenase (LDH) [1]. These RDTs are evaluated at a sensitivity of detection of 200 parasites/μL, although actual field results can be influenced by a number of test, operator, and parasite factors [4]. A positive HRP2-based RDT result could indicate an active P. falciparum infection (clinical or subpatent) or a recently-cleared P. falciparum infection with HRP2 antigen still in systemic circulation. Due to the length of time for post-treatment clearance of HRP2, HRP2-based RDTs can be positive for weeks after resolution of infection [5, 6]. Clearance of aldolase and LDH is substantially quicker and their presence is more indicative of active infection [7]. A negative result for any type of RDT can indicate a true negative, or false negative due to a low-density Plasmodium infection or low production of the antigen target (or non-production of HRP2 due to a gene deletion, [8]). RDTs provide a qualitative, point-of-care measurement for specific Plasmodium antigens (presence or absence of the antigen), and quantitative measurement of these malaria antigens can occur in the laboratory setting through different immunoassay platforms [9-11].

Further molecular testing, such as real-time polymerase chain reaction can be used as a sensitive method to detect the presence/absence of active infections for all malaria species and inform estimates of malaria prevalence [12]. However, testing is more laborious and costly as multiple steps and assays are needed for confirmation of Plasmodium DNA. Initial screening of samples using antigen detection can serve as a more economical and high-throughput method to screen samples to predict parasite presence/absence status using PCR. Additionally, beyond the simple presence or absence of a Plasmodium antigen in a sample, using lab assays for detection of multiple antigens can provide quantitative estimates for each target, as well as generate an antigen profile (interpretation for +/- to multiple targets) for a specimen [9].

Recent advances in statistics and computing power have seen the increase of use of sophisticated machine learning approaches for classification in the context of complex datasets [13, 14], including random forest machine learning approaches to predict protection to malaria based on antibody profiles [15]. In this study, we evaluated the use of machine learning approaches using continuous concentration of antigen data to predict PCR presence/absence classification. Specifically, we evaluated the use of conditional inference trees using antigen concentration and log concentration to predict the presence/absence of infection and classification of five parasite density levels using dried blood spot samples from high and low transmission areas in Angola and Haiti. Predictive models can provide greater nuance to epidemiological estimates and inform the selection of samples as a screening method for further downstream molecular testing.

Materials and methods

Samples and ethics statement

Dried blood spot (DBS) samples used in this study were previously collected for a therapeutic efficacy study (TES) in Angola in 2015 (n = 193) [16], an Angolan health facility (HF) survey in 2016 (n = 208) [17], and a bednet study in Haiti enrolling persons seeking care in health facilities from 2012–2013 (n = 425) [18]. The TES samples were from symptomatic children seeking care at health facilities with microscopy confirmed P. falciparum infection. The 2016 health facility survey samples were from a representative sample of febrile and afebrile outpatients of all ages in Angola. The Angolan TES activity was classified as non-research by human subjects research boards at CDC (#2014-233b) and the Angolan Ministry of Health. Blood sample collection during the Angolan health facility survey was approved by the Angolan Ministry of Health and further laboratory investigation approved by the Office of the Associate Director for Science in the Center for Global Health at the CDC (#2018–034). The Haiti bednet study enrolled febrile patients presenting to health facilities and capillary blood was collected blood for an RDT and also spotted onto filter paper. The Haiti study protocol was approved by the National Bioethics Committee of Haiti and the Institutional Review Board at the CDC. For all studies, written consent was obtained from all participants, and assent obtained from minors upon consent from minor parent or guardian.

Plasmodium falciparum detection and parasite density calculation for different studies

Molecular detection of P. falciparum infection and parasite density estimation were determined using real-time PCR and/or microscopy. Parasitemia for samples collected during the 2015 TES in Angola was determined using traditional microscopy [16]. Parasitemia for samples collected during the 2016 health facility survey in Angola was determined using sensitive quantitative PCR (sen-qPCR) using methods previously described [9], with an analytical sensitivity of 0.02 parasites/μL [19]. Parasitemia for samples collected in Haiti was determined using PET-PCR using methods previously described [18].

PCR assays and multiplex antigen detection

For samples with PCR results, total DNA was extracted from blood specimens by column purification with the Qiagen DNA easy kit according to manufacturer’s protocol (Qiagen, Valencia, CA), and purified DNA subjected to either PET-PCR or sen-qPCR as denoted above. To translate from real-time PCR signal to estimated parasite density, appropriate standard curves were prepared specific for each assay as described previously [20].

Concentrations of HRP2, pan-Plasmodium lactate dehydrogenase (pLDH), and pan-Plasmodium aldolase (pAldolase) were calculated for each sample using the multiplexed antigen bead-based assay and extrapolation from assay signal to antigen concentration performed using methods previously described [9]. For all laboratory data collected for analyses, it was assumed there was no sample contamination.

Data analysis and malaria infection status classification

Principal components analysis (PCA) was performed using antigen concentration and log concentration for PCR presence/absence and infection level parasitemia (parasites/μL) based on five categories (none- 0, lowest—> 0–20, low—> 20–200, mid- > 200–2,000, high- > 2,000) using the prcomp function in R (R Foundation for Statistical Computing). Categories were selected on a log10 scale with the 200 p/μL as the benchmark, being the minimum parasite density RDT product testing employs [4]. As the Angola (microscopy dataset) were nearly all high density infections, Infection level categories for that dataset were the following: lowest = > 0–5,000; low = > 5,000–10,000; mid = >10,000–15,000; high = > 15,000. Conditional inference trees were constructed using the ctree function in the party package in R. Conditional inference trees were selected as a non-parametric regression analysis, which uses unbiased recursive partitioning on continuous, multivariate data to identify the most informative features (e.g., antigenic concentration) and quantitative thresholds for prediction. The decision trees were trained using leave-one-out cross validation using antigen concentration and log concentration as features to classify PCR presence/absence and infection level status as described above. Accuracy, sensitivity, and specificity of conditional inference trees was calculated using the following: accuracy = (true positive (tp) + true negative (tn)) / (tp + false positive (fp) + tn + false negative (fn)); sensitivity = tp / (tp + fn); specificity = tn / (fp + tn). All figures were created using the ggplot2 and cowplot libraries, unless otherwise stated. All R code for data analysis and figure generation is available at https://github.com/SESchmedes/plasmodium_falciparum_infection_status.

Results

The P. falciparum prevalence of the study population was 28% from the 2016 Angola survey [17], and 4.0% from the Haiti study [18]. The mean age of participants for the 2015 Angola TES was 2.8 years with a median 3 years and range 7 months– 12 years. The mean age of participants for the 2016 Angola HF survey was 21 years with a median 15 years and range 1 month– 90 years. The mean age of participants for the 2012–2013 Haiti HF study was 17.4 years with a median 11 years and range 0–99 years.

HRP2, pLDH, and pAldolase concentrations were generated from a total of 826 dried blood spot samples collected in P. falciparum high-transmission (Angola) or low-transmission (Haiti) areas. Persons from the Angola TES had parasitemia determined by microscopy with a range of 2,175 to 184,464 parasites/μL (mean 49,230 parasites/μL); parasite densities for the Angolan health facility samples ranged from 0 to 43,290 parasites/μL (mean 618 parasites/μL); parasite densities for Haitian samples ranged from 0 to 18,463 parasites/μL (mean 908 parasites/μL) (Fig 1).

Fig 1

Parasitemia distribution for each countries’ dataset.

For each study, parasite density depicted as parasites/μL blood. Middle bar = median. Upper box hinge = 75th percentile. Lower box hinge = 25th percentile. Upper whisker = largest value no further than 1.5 * IQR (inter-quartile range or distance from first and third quartiles) from the hinge. Lower whisker = smallest value at most 1.5 * IQR from the hinge.

Conditional inference trees were trained using leave-one-out cross validation with HRP2, pAldolase, and pLDH concentrations (and log-transformed concentrations) for classification of PCR presence/absence status (Fig 2) and infection level (Fig 3, S1 Fig). Both HRP2 and pAldolase informed the models for PCR presence or absence for the full Angolan and Haitian datasets, but pLDH concentrations did not. Using the non-transformed antigen concentrations, the Angolan sen-qPCR data from the health facility survey only predicted one node (bifurcation point) at a pAldolase concentration greater than 325.3 pg/mL to predict P. falciparum presence or absence (Fig 2A), and the Haitian PET-PCR provided two nodes with the first at a HRP2 concentration of 183 pg/mL and the second at pAldolase at 274 pg/mL (Fig 2C). When log-transforming the antigen data, additional prediction nodes were generated with the Angolan sen-qPCR predicting two nodes with the first of pAldolase at 325.1 pg/mL and second of HRP2 concentration at 595.7 pg/mL (Fig 2B). The log-transformed Haitian antigen data also provided two nodes for infection presence/absence, the first at HRP2 concentration of 182.8 pg/mL, and the second at a higher HRP2 concentration of 779.8 pg/mL (Fig 2D). PCR infection status was predicted with accuracies ranging from 73–96%, while infection level was predicted with accuracies ranging from 59–66% (Table 1).

Fig 2

Conditional inference trees using HRP2, pLDH, and pAldolase antigen concentration for classification of Plasmodium falciparum presence or absence as determined by PCR assay.

A) Angola (sen-qPCR). B) Angola (sen-qPCR), log scale. C) Haiti (PET-PCR). D) Haiti (PET), log scale. Y-axes at base of trees indicate probability of correct classification on a scale of 0.0 to 1.0.

Fig 3

Conditional inference trees using HRP2, pLDH, and pAldolase antigen concentration and log concentration for malaria infection level classification.

Infection level categories: None = 0 parasites/μL; Very low = > 0–20; Low = > 20–200; Mid = > 200–2,000; High = > 2,000. A) Angola (sen-qPCR). B) Angola (sen-qPCR), log. C) Haiti (PET-PCR). D) Haiti (PET), log. Y-axes at base of trees indicate probability of correct classification on a scale of 0.0 to 1.0.

Table 1

Percent accuracies for malaria infection status prediction.

Country/Dataset	Attribute	Presence/Absence	Infection Level % accurate
		% accurate (Se, Sp)*
	Concentration	NA	70
Angola (microscopy)	Log concentration	NA	72
Angola (sen-qPCR)	Concentration	73 (94%, 67%)	66
	Log concentration	75 (73%, 78%)	66
Haiti (PET-PCR)	Concentration	96 (97%, 95%)	59
	Log concentration	96 (97%, 95%)	66

* Se: sensitivity; Sp: specificity; Accuracy, Se, and Sp are based on correct classification of malaria parasite presence/absence by utilizing PCR result as gold standard

P. falciparum infection data was further modeled by conditional inference trees after sub-dividing into five categories based on levels of estimated parasite densities. As the Angola TES only enrolled participants based on a microscopically-confirmed parasite density above 2,000 p/μL, those data were not able to be evaluated using the same categorization scheme as the Angola (sen-qPCR) and Haiti (PET-PCR) datasets; therefore, higher concentration infection levels were used for resolution of level of infection (S1 Fig). In assessing the non-transformed antigen data, the Angolan sen-qPCR generated four nodes for infection level, with the first three from pAldolase and the fourth node from HRP2 (Fig 3A). The Haitian dataset provided further resolution with a primary node at 311 pg/mL of pAldolase, and further downstream nodes based on HRP2 or pAldolase concentrations (Fig 3C). As was the case for infection presence/absence, log-transformed antigen data provided more nodes for level of infection. For Angolan sen-qPCR data, log-transformed antigen data provided five nodes with the first three based on pAldolase concentration and the final two on HRP2 concentration (Fig 3B). Notably, log-transforming the Haitian antigen data now provided the first node at HRP2 (concentration of 182.8 pg/mL) with downstream nodes involving both HRP2 and pAldolase (Fig 3D). Percent accuracy for predicting P. falciparum infection level ranged between 59 and 72%, all which were lower than the accuracies of predicting simple presence/absence (Table 1).

For both presence/absence and infection level analyses with PCR data, pLDH did not provide significant decision nodes in the full dataset. However, if pAldolase data was removed, leaving only HRP2 and pLDH, then pLDH did provide nodes. An example is shown for the Angolan sen-qPCR dataset with pAldolase removed where pLDH provides the first nodes for both the non-transformed and transformed antigen concentrations (S2 Fig). The corresponding classification accuracy was approximately the same for both the non-transformed (75%) and log-transformed (75%) antigen data when compared to the full dataset with all three antigens included (Table 1). For all three studies, the pLDH and pAldolase concentrations were shown to correspond with each other (S3 Fig).

When performing principal component analysis (PCA) on the datasets, scatterplots displaying the concordance between the first and second principal components (PC) showed a degree of clustering based on infection presence/absence as well as level of infection. For data from PCR assays, no discernable clustering was observed with non-transformed antigen data for infection presence/absence (S4A and S4C Fig), but when antigen data were log-transformed, the Haiti PET-PCR infection presence was strongly connected to lower values of PC1 (which explained 81.1% of variance) (S4D Fig). When assessing data by level of infection, the non-transformed antigen data again did not show defined visual clustering (Fig 4A and 4C). Higher PC1 values were strongly connected to higher parasite densities for the log-transformed antigen data from Angola (Fig 4B), but the inverse was true for the Haiti dataset with lower PC1 values connected with the ‘mid’ and ‘high’ infection levels. Infection level categories by microscopy did not show any visual clustering by scatterplots of PC1 and PC2 (S5 Fig).

Fig 4

Principal components analysis of HRP2, pLDH, and pAldolase concentrations and log concentrations for infection level using qPCR.

Infection level categories: None = 0 parasites/μL; Very low = > 0–20; Low = > 20–200; Mid = > 200–2,000; High = > 2,000. A) Angola (sen-qPCR). B) Angola (sen-qPCR), log scale. C) Haiti (PET-PCR). D) Haiti (PET-PCR), log scale.

Discussion

Our results suggest that machine learning algorithms can be trained using quantitative malaria antigen data to reliably predict P. falciparum presence/absence and as well as different levels of peripheral parasite densities. Antigen detection is utilized globally for diagnosis of malaria by RDTs, but these are designed to detect clinically-relevant parasite densities, and only provide a binary result [3, 4]. Additionally, standard RDT use would also require a point-of-contact action (i.e. administering anti-malarial drugs) upon a positive result, and the multiplex antigen data and analyses presented here would be more utilized for epidemiological purposes. By being able to categorize sample sets into levels of estimated parasite densities based on multiplex antigen data alone, an additional benefit could arise by being able to select samples within these higher levels for greater success with DNA-based assays. The ability to use quantitative antigen concentrations to train machine learning algorithms to predict peripheral parasite densities represents a novel step forward for these efforts.

Collection of quantitative multiplex antigen data presents many advantages to other laboratory assays for the estimation of malaria status from a patient sample. These immunoassays are formatted to a 96-well format, the per-sample cost is approximately an order of magnitude less than nucleic-acid based assays, and hands-on time in the laboratory is short due to the simplicity of the antigen detection assays [9, 10]. For the datasets available for this study, quantitative data for three Plasmodium antigens was available: pan-Plasmodium LDH and aldolase antigens (pLDH and pAldolase, respectively), and P. falciparum-specific HRP2. As all three datasets were specifically capturing P. falciparum infections by microscopy or PCR assays, this panel of three antigens was appropriate for this investigation. However, non-falciparum infections have been reported in both Angola [9, 21] and Haiti [22], so the possibility also exists that non-detected mixed infections with P. vivax, P. malariae, or P. ovale (only in Angola) could have skewed the pan-Plasmodium antigen concentrations beyond what would be expected for a P. falciparum-only infection. However, as a proportion of the total malaria burden on these populations, non-falciparum infections are rare in these two countries, so presence of mixed infections would likely not have influenced the models. With quantitative data potentially available for other malaria antigen targets not utilized here, these similar machine learning approaches could be expanded to be even more robust in predicting P. falciparum infection or be modeled against infection with another of the human malarias.

In assessing the model input of the pan-Plasmodium antigens for the complete datasets, pLDH was only informative for the Angola TES study (infection detected by microscopy), whereas pAldolase was informative for nearly every other decision tree with both non-transformed and log-transformed antigen concentrations. It may not be surprising that one of these pan- antigens would “out-compete” the other as high collinearity was observed in absolute concentrations between these two targets in the same sample (S3 Fig) [9]. As an example, when the pAldolase data was removed from the Angolan health facility dataset, the pLDH replaced pAldolase as the first node on the decision tree. As both are metabolic enzymes in Plasmodia spp., though concentrations of these two antigens would be expected to be largely concordant, they both may provide unique information for different strains of P. falciparum parasites which may express slightly different isoforms of these two antigens [23, 24]. By far, the most informative input for the models creating the most nodes was the HRP2 antigen, which is abundantly expressed during blood-stage P. falciparum infection [25]. This was true in training models for both P. falciparum presence/absence as well as level of parasite density during infection. The added advantage of being able to measure signals for multiple antigens at a time is consistent with previous reports showing that HRP2/LDH ratios are predictive of determining active from recently cleared infection [26].

When compared to models for parasite presence/absence, modeling for discrete infection levels had noticeably lower accuracy. This was not surprising as blood-stage malaria infection is characteristic of billions of parasites and high amounts of antigen being produced, so an identified infection typically has very high amounts of these antigens in the host without precise gradations. Additionally, the “very low” (1–20 parasites/μL) and “low” (20–200 parasites/μL) categories of P. falciparum infection are both under the parasite density levels evaluated by the World Health Organization RDT evaluation program for product qualification [4]. The highest accuracy, up to 96%, was observed for prediction of PCR presence/absence in Haiti, and this could be explained by the low-transmission setting in this country; individuals would have been less likely to have had a recent infection with antigen concentrations creating “noise” that makes it more difficult to distinguish from active infection.

A limitation to this study was that datasets from each of the three studies provided different sample sizes in terms of number of persons infected with P. falciparum and utilized different enrollment criteria and samples from persons with different exposure histories. Additionally, the only sample type utilized in these surveys was DBS, and during drying and storage, potential degradation of antigen or DNA may have occurred to understate the quantity of these biomarkers. Different PCR assays were used for Haiti and Angola, and while both estimated parasite densities from quantity of DNA in the samples, comparison of classification trees between these two sample sets should consider the differences in PCR assays. A more recently detected phenomenon of P. falciparum strains with deletions or alterations of the pfhrp2 gene has been seen in numerous countries but was not evaluated in this study [8]. However, these deletions have not been reported in Haiti [27], and only reported at very low levels in Angola [9], so these potential deletions likely did not affect our analyses. High-transmission areas (like Angola) might not perform as well using this model compared to low-transmission areas (like Haiti) due to lingering HRP2 antigen in circulation [5, 7], which could negatively impact specificity estimates.

Future studies on larger datasets should address optimal statistical tests and machine learning models for infection status prediction, as well as employ methods to correct for dataset imbalance. Conditional inference trees were selected for use in this study to perform a nonparametric regression analysis as a method for unbiased recursive partitioning to easily identify the most informative antigen for the model, in addition to predictive quantitative thresholds of antigenic concentrations. As such, it could not be stated that this approach utilized here would produce optimal accuracy, and further investigation of other statistical approaches, such as k-nearest neighbor regression, linear discrimination analysis, random forest, gradient boosting, or finite mixture models, should be conducted on future antigenic concentration datasets collected addressing the limitations stated above.

This study provides a pilot methodology and the results can be used to design and conduct additional studies. Specifically, future validation studies should have datasets with: substantial numbers of negative and positive samples with a wide range of parasite densities; molecular detection/parasite density measured using a variety of quantitative PCR techniques; and samples from different geographical areas (including pre-elimination, low-transmission and high-transmission countries/regions). Further investigation of machine learning approaches could provide greater resolution for determination of infection status from quantitative antigen data to support malaria surveillance activities and epidemiologic studies.

Conditional inference trees using HRP2, pLDH, and pAldolase antigen concentration for malaria infection level classification by microscopy.

Infection level categories: Lowest = > 0–5,000; Low = > 5,000–10,000; Mid = >10,000–15,000; High = > 15,000. A) Angola (microscopy). B) Angola (microscopy), log scale. Y-axes at for all plots indicate probability of correct classification on a scale of 0.0 to 1.0.

(TIF)

Click here for additional data file.

Conditional inference trees using only HRP2 and pLDH data with pAldolase removed for classification of Plasmodium falciparum presence or absence as determined by PCR assay.

Data shown for Angola (sen-qPCR) classification with antigen concentrations on non-transformed scale on left and log-transformed on right. Y-axes at base of trees indicate probability of correct classification on a scale of 0.0 to 1.0.

(TIF)

Click here for additional data file.

Comparison of concentrations of pLDH and pAldolase antigens from persons enrolled among the three surveys.

Antigen concentration data shown for the Angola 2015 TES (A), Angola 2016 health facility (B), and Haiti bednet (C) studies with concentration of pAldolase on x-axis and pLDH on y-axis for each.

(TIF)

Click here for additional data file.

Principal components analysis of HRP2, pLDH, and pAldolase concentrations and log concentrations for qPCR presence/absence.

A) Angola (sen-qPCR). B) Angola (sen-qPCR), log scale. C) Haiti (PET-PCR). D) Haiti (PET), log scale.

(TIF)

Click here for additional data file.

Principal components analysis of HRP2, pLDH, and pAldolase concentrations for infection level using microscopy.

Infection level categories: Lowest = > 0–5,000; Low = > 5,000–10,000; Mid = >10,000–15,000; High = > 15,000. A) Angola (microscopy). B) Angola (microscopy), log scale.

(TIF)

Click here for additional data file.

Inclusivity in global research.

(DOCX)

Click here for additional data file.

19 Dec 2021

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Danny W Wilson

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include a complete copy of PLOS’ questionnaire on inclusivity in global research in your revised manuscript. Our policy for research in this area aims to improve transparency in the reporting of research performed outside of researchers’ own country or community. The policy applies to researchers who have travelled to a different country to conduct research, research with Indigenous populations or their lands, and research on cultural artefacts. The questionnaire can also be requested at the journal’s discretion for any other submissions, even if these conditions are not met.  Please find more information on the policy and a link to download a blank copy of the questionnaire here: https://journals.plos.org/plosone/s/best-practices-in-research-reporting. Please upload a completed version of your questionnaire as Supporting Information when you resubmit your manuscript

3. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

4. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

5. Thank you for stating the following in the Acknowledgments Section of your manuscript:

The authors acknowledge partial support from the Bioinformatics Fellowship Program administered by the Association of Public Health Laboratories (APHL) and funded by the CDC. S.S. is supported in part by the Bioinformatics Fellowship Program administered by the APHL and funded by the CDC.

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

MP was supported by the U.S. President’s Malaria Initiative

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

6. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

Additional Editor Comments:

Figures were quite blurred. Please review the PLOS One figure requirements and see whether Figures can be uploaded in a higher resolution format.

Do you intend to provide the code and data that underline this study?

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This study examines whether the quantitative measurements of Plasmodium antigen in human exposed blood samples can be used to train machine learning algorithms to categorize patients by parasite presence or absence and parasite density level in order to achieve greater nuance in malaria diagnostics using antigen based assays rather than more costly DNA amplification methodologies. Blood samples from different cohort comprising symptomatic children and febrile or non-febrile adults were used to determine parasitemia by different quantitative methods, and antigen levels for three different antigens were determine by a bead-based quantitative assay. Conditional inference trees and PCA were performed for classification into presence or absence of parasites and into pre-defined parasite levels based on transformed and un-transformed antigen levels. Prediction of PCR infection status and parasitemia levels varied depending on antigen transformation of data.

This pilot study provides encouraging results to show that machine learning algorithms can be trained using quantitative antigen data in order to predict infection status and parasitemia levels. Results and limitations of the study and study cohort were appropriately discussed.

Minor points:

Given that the machine based learning algorithms are trained by continuous antigen data derived from dried blood spots, could the authors comment or speculate on how this would impact on the assessment of fresh blood samples in POC health care settings (i.e. would antigen measurement and DNA extraction from dried blood spots potentially underestimate parasite density due to poor antigen and DNA extraction?).

Results, lines 146ff describe the mean number of parasites/µl whereas figure 1 shows median. Could the authors make this consistent, i.e. quote median in text or show mean in figure.

Table 1: Can the authors please define sensitivity and specificity in the table notes

Font size in all figures and tables should be increased

Figure S3: Is this a Spearman or Pearson’s correlation

Reviewer #2: The paper entitled “Predicting Plasmodium falciparum infection status in blood using a multiplexed bead-based antigen detection assay and machine learning approaches” is an application of conditional inference trees to malaria infection data. The paper is interesting and, as far as I know, it is the first time that this type of methodology was applied to malaria data; other machine learning techniques such as random forests were used in searching protective immunity against clinical malaria (doi: 10.1371/journal.pcbi.1005812) and this should be acknowledged in the Introduction. However, I think an extensive revision of the paper is needed in order to make the take-home message more compelling and crystal clear. Below please find my specific comments:

- The title does not reflect the content of the paper. First, infection levels were also predicted in the study. Second, it was only applied a single method (i.e., conditional inference trees) to predict infection status and levels and hence the use of the plural ''approaches'' is not appropriate. Third, if one tracks down the theoretical developments of conditional inference trees, they were published in statistical rather machine learning journals. Therefore, conditional inference trees are more statistical learning techniques than machine learning ones.

- The motivation of this study should be rephrased more clearly. Is the motivation on the use of the methodology or the use of data from multiplex bead assays? I guess it is the methodology but then why to use conditional inference trees and exclude other existing methodologies to tackle the same classification problem?

- In the introduction, it is important to be clear that absent production of the antigen target (line 71) is mostly for the HRP2 case. As far as I know, there are no reports of gene deletions for Aldolase and LDH. This might be obvious for most malaria researchers but for less specialized audience, I would write that LDH, Aldolase, and HRP2 are the proteins used in current pan-malaria and pf-malaria RDTs.

- Please provide data about the prevalence of infection for the Haiti study and the second study from Angola. It would be useful to provide information (mean, median, range, etc) about the age of the participants from each study. This increases the interpretability of the results.

- In Figure 1 (Angola sen qPCR), there are 25% of the infections that have parasitemia below 1 mu/l. Can we trust these low levels of parasite density? What is the lower level of detection above which one can trust the respective parasitemia quantification? This point is important to clarify given that the limited performance of the conditional inference trees might be caused by these infections with low parasitemia.

- Is there any rationale to divide infection levels into the 5 categories used? If this categorization is completely arbitrary, this should be clearly stated. Otherwise, provide a rationale (maybe related to the expected sensitivity to RDT as function of parasitemia).

- In the materials & methods, provide a brief explanation about conditional inference trees, how they are constructed and interpreted. This increases readability of the paper to a less specialized audience.

- It is worth mentioning that conditional inference trees are dependent on the scale of covariates/features. This is a limitation of the methodology that should be acknowledged. This limitation could have been avoided by using other methodologies, such as random forest or XGBoost, which are invariant to change of scale. Why were not these methodologies applied to the same data?

- It is also unclear whether simpler and more common approaches such as logistic regression, probit regression or other generalized models for binary/categorical data could perform equally well in the same data. Linear discrimination analysis is also another population alternative for classification problems using multivariate data.

- It was used a leave-one-out cross-validation procedure. This allows the estimation of the sensitivity (Se) and specificity (Sp) shown in Table 1. But I think 5-fold or 10-fold cross-validation provides a better idea of how robust (or uncertain) accuracy, Sp and Se estimates are. Please define accuracy, Sp and Se for a less specialized audience.

- To complement the presented accuracy measures of the model predictions, the ROC curves should be also presented (in the main text) and the respective area under the curve calculated.

- I like the idea of having cutoffs in the covariates/features. This reminds what malaria epidemiologist do in serological data analysis where a cutoff is used to define seronegative and seropositive population. From a perspective of responsible and explicable machine learning, I recommend to fit finite mixture models (Gaussian or non-Gaussian) to the antigen data, check whether there are multiple latent populations (e.g, antigen-negative plus multiple antigen-positive levels), and whether the cutoffs derived from conditional inference trees are related to the discrimination between these latent populations. Particularly flexible finite mixture models are the ones based on the Skew-normal and Skew-t distributions as described in Domingues et al (doi: 10.1101/2021.03.08.21252807). This additional analysis takes the paper into a whole new level.

- In Table 1, accuracy for the infection level data should be discriminated per infection level. I bet misclassification comes mostly from categories related to low parasitemia infections.

- In Table 1, it is interesting that Sp seems to be lower in Angola than in Haiti. I bet this is related to a higher transmission in Angola than in Haiti. This is an interesting finding that deserves exploration and discussion.

- In the text, it says the accuracy for the infection levels ranged from 59% and 72%, but the estimates in Table 1 do not show any estimate equal to 72%. Hopefully, this is just a typo.

- I am confused that “As the Angola TES 185 only enrolled participants based on a microscopically-confirmed parasite density above 2,000 p/�  L, those data were not able to be evaluated in this categorization scheme” (lines 184—186). It seems this dataset was not used at all for prediction given that it could not also be used for infection status prediction . If that is the case, the paper needs to be totally revised to remove any reference to this dataset (including Figures and Supplementary Figures).

- What was the rationale to include a principal component analysis (PCA) as it is not use to predict infection status and levels? Given that the objective is related to a classification problem, why not to use a related multivariate technique such as linear discriminant analysis as suggested above?

- With respect to PCA, I cannot observe that higher values of PC1 reflect high infection levels for Figure 4D (lines 224-225).

- Figures: unfortunately, I could not make a better assessment of the figures due to their low resolution. However, I think violin plots or related plots provide a more informative way of visualizing the data instead of the boxplots shown in Figure 1. In the same Figure, it should be clearly stated in the figure legend that non-infected individuals are not represented in the plots. The remaining Figures are unreadable. What is it plotted in the y axis of the plots at the bottom of the trees? All figure legends should be expanded to be more informative.

- Code and data sharing: to increase replication of the study by other researchers, authors should consider to share their data and code with the community.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

1 Jul 2022

Response to Editor and Reviewers

Additional Editor Comments:

• Figures were quite blurred. Please review the PLOS One figure requirements and see whether Figures can be uploaded in a higher resolution format.

- This has been done.

• Do you intend to provide the code and data that underline this study?

- Yes, we have now provided the code on Github, and indicated this at the end of Methods. We have also added a statement to the end of Methods that “Data used in these analyses is available upon request to the corresponding author”.

Reviewer #1: This study examines whether the quantitative measurements of Plasmodium antigen in human exposed blood samples can be used to train machine learning algorithms to categorize patients by parasite presence or absence and parasite density level in order to achieve greater nuance in malaria diagnostics using antigen based assays rather than more costly DNA amplification methodologies. Blood samples from different cohort comprising symptomatic children and febrile or non-febrile adults were used to determine parasitemia by different quantitative methods, and antigen levels for three different antigens were determine by a bead-based quantitative assay. Conditional inference trees and PCA were performed for classification into presence or absence of parasites and into pre-defined parasite levels based on transformed and un-transformed antigen levels. Prediction of PCR infection status and parasitemia levels varied depending on antigen transformation of data.

This pilot study provides encouraging results to show that machine learning algorithms can be trained using quantitative antigen data in order to predict infection status and parasitemia levels. Results and limitations of the study and study cohort were appropriately discussed.

Minor points:

Given that the machine based learning algorithms are trained by continuous antigen data derived from dried blood spots, could the authors comment or speculate on how this would impact on the assessment of fresh blood samples in POC health care settings (i.e. would antigen measurement and DNA extraction from dried blood spots potentially underestimate parasite density due to poor antigen and DNA extraction?).

- The reviewer brings up an important point here regarding protein and DNA integrity from different sample types. Additional text has been added to the limitations section of Discussion (Line 297) to point out that only DBS were used in these studies, and the potential for antigen/DNA degradation: “ Additionally, the only sample type utilized in these surveys was DBS, and during drying and storage, potential degradation of antigen or DNA may have occurred to understate the quantity of these biomarkers.”

Results, lines 146ff describe the mean number of parasites/µl whereas figure 1 shows median. Could the authors make this consistent, i.e. quote median in text or show mean in figure.

- We report the mean in the text because it is more informative than the median (for example, some of the medians are 0). Please note that the y-axis in the Figure is on a log-scale.

Table 1: Can the authors please define sensitivity and specificity in the table notes

- Additional text has been added to the Table 1 footnote to explain this further: “Accuracy, Se, and Sp are based on correct classification of malaria parasite presence/absence by utilizing PCR result as gold standard”.

Font size in all figures and tables should be increased

- This has been done

Figure S3: Is this a Spearman or Pearson’s correlation

- The use of the word ‘correlation’ is not the most appropriate here, as we do not provide statistical tests to assess true correlation and significance. This word has been replaced in both the text and figure legend.

Reviewer #2: The paper entitled “Predicting Plasmodium falciparum infection status in blood using a multiplexed bead-based antigen detection assay and machine learning approaches” is an application of conditional inference trees to malaria infection data. The paper is interesting and, as far as I know, it is the first time that this type of methodology was applied to malaria data; other machine learning techniques such as random forests were used in searching protective immunity against clinical malaria (doi: 10.1371/journal.pcbi.1005812) and this should be acknowledged in the Introduction.

- We thank the reviewer for this suggestion, and this study has been acknowledged in the Introduction and is now reference #15.

However, I think an extensive revision of the paper is needed in order to make the take-home message more compelling and crystal clear. Below please find my specific comments:

- The title does not reflect the content of the paper. First, infection levels were also predicted in the study. Second, it was only applied a single method (i.e., conditional inference trees) to predict infection status and levels and hence the use of the plural ''approaches'' is not appropriate. Third, if one tracks down the theoretical developments of conditional inference trees, they were published in statistical rather machine learning journals. Therefore, conditional inference trees are more statistical learning techniques than machine learning ones.

- The authors thank the reviewers for the insight. We agree with the reviewer that the main narrative of the paper is not to assess the ideal machine learning approach to this sort of dataset. Our objective was to show that a machine learning approach (like conditional inference trees) can be effective at classifying samples by infection status. There is a large overlap between “statistical approach” and “machine learning approach”, and both could apply here. From our perspective, “infection status” is inclusive of both positivity/negativity as well as infection level.

- The motivation of this study should be rephrased more clearly. Is the motivation on the use of the methodology or the use of data from multiplex bead assays? I guess it is the methodology but then why to use conditional inference trees and exclude other existing methodologies to tackle the same classification problem?

- Thank you for pointing out this lack of clarity in our report. Our motivation was to apply an appropriate statistical tool (conditional inference trees) that was chosen in advance and apply it to a classification problem using data from multiplex bead assays for antigen detection. Our motivation was decidedly not to compare different methodologies and choose the most appropriate one – that would be well outside the scope of this work. However, we appreciate this comment and have added an additional paragraph near the end of Discussion explicitly stating the possibility of using alternative statistical and machine learning tools to address the same classification problem. We hope to explore other methods in future work.

- In the introduction, it is important to be clear that absent production of the antigen target (line 71) is mostly for the HRP2 case. As far as I know, there are no reports of gene deletions for Aldolase and LDH. This might be obvious for most malaria researchers but for less specialized audience, I would write that LDH, Aldolase, and HRP2 are the proteins used in current pan-malaria and pf-malaria RDTs.

- The authors agree with this comment and that this should be better clarified. We add additional text here to accentuate that these three antigens are the current targets for malaria RDTs, and also clarify that only HRP2 is known to have gene deletions that would affect RDT results.

- Please provide data about the prevalence of infection for the Haiti study and the second study from Angola. It would be useful to provide information (mean, median, range, etc) about the age of the participants from each study. This increases the interpretability of the results.

- The authors agree this is useful information for the readers’ interpretation. We have now added the requested information to the beginning of the Results section.

- In Figure 1 (Angola sen qPCR), there are 25% of the infections that have parasitemia below 1 mu/l. Can we trust these low levels of parasite density? What is the lower level of detection above which one can trust the respective parasitemia quantification? This point is important to clarify given that the limited performance of the conditional inference trees might be caused by these infections with low parasitemia.

- To provide the reader with additional information, we have added the analytical sensitivity of this qPCR assay to line 119 of Methods with appropriate reference: “…, with an analytical sensitivity of 0.02 parasites/µL (19).”

- Is there any rationale to divide infection levels into the 5 categories used? If this categorization is completely arbitrary, this should be clearly stated. Otherwise, provide a rationale (maybe related to the expected sensitivity to RDT as function of parasitemia).

- The reviewer is correct regarding the RDT test reliability assumption, and the categories were selected on a log10 scale with the 200p/uL as the benchmark being the minimum parasite density RDT product testing employs. The authors have added additional text to line 137 in Methods to explain this.

- In the materials & methods, provide a brief explanation about conditional inference trees, how they are constructed and interpreted. This increases readability of the paper to a less specialized audience.

- The authors have added an enhanced explanation of conditional inference trees in the “Data analysis and malaria infection status classification” section.

- It is worth mentioning that conditional inference trees are dependent on the scale of covariates/features. This is a limitation of the methodology that should be acknowledged. This limitation could have been avoided by using other methodologies, such as random forest or XGBoost, which are invariant to change of scale. Why were not these methodologies applied to the same data?

- The authors appreciate the comment and have added an additional paragraph to Discussion that proposes future alternate statistical approaches including k-nearest neighbor regression, linear discrimination analysis, random forest, gradient boosting, or finite mixture models.

- It is also unclear whether simpler and more common approaches such as logistic regression, probit regression or other generalized models for binary/categorical data could perform equally well in the same data. Linear discrimination analysis is also another population alternative for classification problems using multivariate data.

- The authors appreciate this comment, and will refer the reviewer to the comment above regarding other statistical approaches. The authors have updated Discussion to explicitly mention alternative statistical approaches.

- It was used a leave-one-out cross-validation procedure. This allows the estimation of the sensitivity (Se) and specificity (Sp) shown in Table 1. But I think 5-fold or 10-fold cross-validation provides a better idea of how robust (or uncertain) accuracy, Sp and Se estimates are. Please define accuracy, Sp and Se for a less specialized audience.

- In this study, leave-one-out cross validation was used to maximize the training set since our datasets were relatively small. 10-fold validations will be used on future, larger datasets to minimize overfitting. Definitions of accuracy, Sp, and Se are now fully described in Methods section.

- To complement the presented accuracy measures of the model predictions, the ROC curves should be also presented (in the main text) and the respective area under the curve calculated.

- To our knowledge, it is not possible to calculate ROC curves for classification problems where there is more than one predictor variable.

- I like the idea of having cutoffs in the covariates/features. This reminds what malaria epidemiologist do in serological data analysis where a cutoff is used to define seronegative and seropositive population. From a perspective of responsible and explicable machine learning, I recommend to fit finite mixture models (Gaussian or non-Gaussian) to the antigen data, check whether there are multiple latent populations (e.g, antigen-negative plus multiple antigen-positive levels), and whether the cutoffs derived from conditional inference trees are related to the discrimination between these latent populations. Particularly flexible finite mixture models are the ones based on the Skew-normal and Skew-t distributions as described in Domingues et al (doi: 10.1101/2021.03.08.21252807). This additional analysis takes the paper into a whole new level.

- We appreciate this suggestion from the reviewer and will refer to the added comment above which the authors have updated the Discussion section to explicitly mention alternative statistical approaches. Using and/or comparing alternative approaches is outside the scope of this work. Specifically, this current unit of work attempts to move away from more simple antigen positive/negative approaches and maintain antigen concentrations as continuous.

- In Table 1, accuracy for the infection level data should be discriminated per infection level. I bet misclassification comes mostly from categories related to low parasitemia infections.

- We thank the reviewer for the suggestion here, but due to relatively smaller datasets utilized here, these accuracy estimates are unable to be reliably generated when sub-dividing the dataset by the 5 infection levels. We have added in Discussion the need to take these (and alternative) approaches on larger datasets.

- In Table 1, it is interesting that Sp seems to be lower in Angola than in Haiti. I bet this is related to a higher transmission in Angola than in Haiti. This is an interesting finding that deserves exploration and discussion.

- The authors agree with the reviewer, and have revised the last sentence in the limitations paragraph to point this out directly: “High-transmission areas (like Angola) might not perform as well using this model compared to low-transmission areas (like Haiti) due to lingering HRP2 antigen in circulation (5, 7), which could negatively impact specificity estimates.”

- In the text, it says the accuracy for the infection levels ranged from 59% and 72%, but the estimates in Table 1 do not show any estimate equal to 72%. Hopefully, this is just a typo.

- Yes, this was a typo, and has been corrected.

- I am confused that “As the Angola TES 185 only enrolled participants based on a microscopically-confirmed parasite density above 2,000 p/�  L, those data were not able to be evaluated in this categorization scheme” (lines 184—186). It seems this dataset was not used at all for prediction given that it could not also be used for infection status prediction. If that is the case, the paper needs to be totally revised to remove any reference to this dataset (including Figures and Supplementary Figures).

- Because the Angola microscopy dataset only contained samples >2,000, that was automatically in our “high” infection level status compared to the other Angola (sen-PCR) and Haiti datasets. As the Angola (microscopy) dataset used a different category, those results were only include in the Supplemental. The authors have edited the figure legend to reflect the correct category levels and the text referred to in this comment. The authors have now included the accuracy for this dataset in the main Table 1.

- What was the rationale to include a principal component analysis (PCA) as it is not use to predict infection status and levels? Given that the objective is related to a classification problem, why not to use a related multivariate technique such as linear discriminant analysis as suggested above?

- Please see comment above – the authors have updated the discussion to explicitly mention alternative statistical approaches, including linear discriminant analysis. For this current study, PCA was utilized to visualize sub-populations of infection status (as determined by PCR data) as a factor of antigen concentrations.

- With respect to PCA, I cannot observe that higher values of PC1 reflect high infection levels for Figure 4D (lines 224-225).

- The authors apologize for the poor resolution of the originally submitted figures, and we have uploaded new figures with higher resolution. The reviewer is correct in that lower PC1 values correspond to higher PCR-determined parasite densities in Figure 4D, and have revised Results text to reflect this.

- Figures: unfortunately, I could not make a better assessment of the figures due to their low resolution. However, I think violin plots or related plots provide a more informative way of visualizing the data instead of the boxplots shown in Figure 1. In the same Figure, it should be clearly stated in the figure legend that non-infected individuals are not represented in the plots. The remaining Figures are unreadable. What is it plotted in the y axis of the plots at the bottom of the trees? All figure legends should be expanded to be more informative.

- The authors apologize for the poor figure resolution, and have resubmitted with higher resolution figures. The authors appreciate the suggestion for violin plots, but prefer to stick with boxplots as they accurately depict the data and are more intuitive for the reader. The y-axis for the trees has been added to the figure legends.

- Code and data sharing: to increase replication of the study by other researchers, authors should consider to share their data and code with the community.

- The authors thank the reviewer for this comment, and have added all code to the Github, and indicated this at the end of Methods.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

18 Aug 2022

24 Aug 2022

Schmedes, et al.

PONE-D-21-34503R1

Response to Reviewers

All references have been cross-checked for accuracy.

1. In lines 339-346, it should be written more clearly that optimal accuracy might not have been achieved in this study, because this study did not attempt to do it so. To achieve optimal accuracy, other statistical and machine learning methods should have been used and compared with the conditional inference trees.

- The authors are in agreement with the reviewer’s assessment, and as not every possible method was tested here (with the same dataset), we could not make a statement that the greatest accuracy was achieved. We have revised this section on Line 343 to include the statement: “As such, it could not be stated that this approach utilized here would produce optimal accuracy, and further investigation…”

2. With respect to my point about cutoffs, I agree that we should aim for using more advanced analytical techniques. However, given that malaria (sero-)epidemiologists are well aware of cutoff-based methods, the paper benefits of explicitly suggesting the link between the cutoffs suggested by the conditional inference trees and those that can be derived from finite mixture models. This suggestion connects the study with the existing literature. This connection can be done by revising the paragraph in lines 339-346.

- The authors thank the reviewer for this suggestion, but do not agree with the parallels being drawn between the antigen detection data being presented in this current study and antibody detection data (what is utilized for malaria seroepidemiology). Statistical methods for dichotomization, including FMM, have been widely discussed in the antibody (almost unanimously IgG) literature, but this does not apply to levels of Plasmodium antigens in blood samples – a completely different biological phenomenon. Our group has casually investigated cutoffs for antigen data in previous studies (cited here), but there is no real precedent in the literature outside of our previous work.

3. In the Material & Methods, it is important to state that it was assumed that there was no sample contamination in the data analysed.

- The authors agree with this statement, and have added text to Line 134: “For all laboratory data collected for analyses, it was assumed there was no sample contamination.”

4. Sorry for raising this comment at this stage, but it seems that dataset is imbalanced and this might affect the accuracy predicted by conditional inference trees. This should be briefly discussed by acknowledging that there are (machine learning) methods that could have been used to correct for that.

- For analyses presented here, datasets from separate studies were analyzed independent of one another, and predicted accuracies were presented for each unique study/analysis. So while the authors would agree that categories are not equally represented within each dataset, we provide datasets from different malaria transmission settings as examples of how these outputs may be interpreted from different endemic settings. The authors feel they currently describe the limitations of the current datasets in Discussion, but we have now added additional text to explicitly state that ML methods exist to help deal with imbalanced datasets on Line 340: “Future studies on larger datasets should address optimal statistical tests and machine learning models for infection status prediction, as well as employ methods to correct for dataset imbalance.”

Submitted filename: Response to Reviewers_R1.docx

Click here for additional data file.

12 Sep 2022

Predicting Plasmodium falciparum infection status in blood using a multiplexed bead-based antigen detection assay and machine learning approaches

PONE-D-21-34503R2

Dear Dr. Rogier,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Danny W Wilson

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: Thank for trying to accommodate my comments in the revised version. I am happy with the revisions done.

**********

20 Sep 2022

PONE-D-21-34503R2

Predicting Plasmodium falciparum infection status in blood using a multiplexed bead-based antigen detection assay and machine learning approaches

Dear Dr. Rogier:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Danny W Wilson

Academic Editor

PLOS ONE