Cost-effectiveness analysis protocol of the Smart Triage program: A point-of-care digital triage platform for pediatric sepsis in Eastern Uganda.

BACKGROUND: Sepsis is a clinical syndrome characterized by organ dysfunction due to presumed or proven infection. Severe cases can have case fatality ratio 25% or higher in low-middle income countries, but early diagnosis and timely treatment have a proven benefit. The Smart Triage program in Jinja Regional Referral Hospital in Uganda will provide expedited sepsis treatment in children through a data-driven electronic patient triage system. To complement the ongoing Smart Triage interventional trial, we propose methods for a concurrent cost-effectiveness analysis of the Smart Triage platform.
METHODS: We will use a decision-analytic model taking a societal perspective, combining government and out-of-pocket costs, as patients bear a sizeable portion of healthcare costs in Uganda due to the lack of universal health coverage. Previously published secondary data will be used to link healthcare utilization with costs and intermediate outcomes with mortality. We will model uncertainty via probabilistic sensitivity analysis and present findings at various willingness-to-pay thresholds using a cost-effectiveness acceptability curve. DISCUSSION: Our proposed analysis represents a first step in evaluating the cost-effectiveness of an innovative digital triage platform designed to improve clinical outcomes in pediatric sepsis through expediting care in low-resource settings. Our use of a decision analytic model to link secondary costing data, incorporate post-discharge healthcare utilization, and model clinical endpoints is also novel in the pediatric sepsis triage literature for low-middle income countries. Our analysis, together with subsequent analyses modelling budget impact and scale up, will inform future modifications to the Smart Triage platform, as well as motivate scale-up to the district and national levels. TRIAL REGISTRATION: Trial registration of parent clinical trial: NCT04304235, https://clinicaltrials.gov/ct2/show/NCT04304235. Registered 11 March 2020.

Introduction

Sepsis, the Importance of Triage, and the Smart Triage program

Sepsis is a syndrome characterized by an inflammatory state resulting from a presumed or proven infection that results in organ dysfunction and/or death [1], with global disease-specific mortality of 20% and a case fatality ratio of 25% or higher in low-middle income countries (LMICs) [2,3]. Earlier treatment results in better outcomes; therefore, international guidelines for timely treatment were shortened from 3 hours to 1 hour in 2016 [4]. Treatment comprises a “sepsis bundle” of antimicrobials, fluid, and oxygen as clinically indicated [5].

While high-income countries have employed evidence-based patient triage strategies to expedite sepsis treatment [6], these strategies involve various laboratory investigations rarely available in LMIC settings, where delays in treatment leading to a high case fatality ratio are commonplace [7]. LMIC-specific guidelines such as the Emergency Triage Assessment and Treatment (ETAT) [8] and others [9,10], show good predictive power for mortality and improve clinical outcomes when successfully implemented [11-13]. However, the complexity of these guidelines introduces challenges in everyday usage and staff training in an environment where patient burden and new-staff turnover is high [14,15]. Infrastructural barriers to patient care organization and medication shortages further delay sepsis treatment even when a sick child has been identified through effective triage [15].

The Smart Triage program [16] tackles guideline complexity via a prediction model for admission to facilitate the identification of children at risk of deterioration. This model classifies children into three levels of risk categories, using clinical variables found in ETAT and other triage studies that are then used as inputs in an algorithm derived from machine learning techniques. Our methods for initial consideration of variables have been published [17], and the details of the final algorithm are under peer review. This model was developed using data from the trial’s pre-intervention phase, and has been integrated into a mobile application to reduce cognitive error and training time compared to a manual triage process. The triage results from the mobile application will be linked to a Clinician Dashboard local web page and a locally hosted Bluetooth treatment-tracking system to improve patient care organization. The Smart Triage program contains all of these components in a package, which will be implemented at the Jinja Regional Referral Hospital (JRRH) in Uganda, in an interventional trial for which the protocol is published [16].

Economic evaluation of LMIC triage programs

When considering new programs in low-resource settings, economic evaluations such as cost-effectiveness analyses can inform policy decisions and motivate further research. While cost-effectiveness analyses of quality improvement initiatives that deliver earlier sepsis care have demonstrated reduced cost and increased benefit over standard of care in high-income settings [18,19], such analyses are sparse for triage programs in LMICs. In the literature, studies may report partial costs without a formal economic evaluation [14], or perform cost-effectiveness analyses on intermediate outcomes without linkage to clinical endpoints [20]. One analysis used a robust decision-analytic model, but interventions evaluated spanned across the inpatient and post-discharge timeframes and were not isolated to the triage process [21]. Lastly, many analyses focused on the perspective of the healthcare system [13,20,22], but given the lack of universal health coverage in many LMICs, a societal perspective that also includes out-of-pocket (OOP) costs and time/productivity loses may be more appropriate. Therefore, in light of the literature gaps presented, we propose a cost-effectiveness analysis alongside the ongoing Smart Triage trial, using a decision analytic model, costing data from previous trials [23], linkage of intermediate outcomes to clinical endpoints [24], and a societal perspective. Our proposed economic evaluation serves to complement the previously published protocol of the Smart Triage trial [16], and will play a role in shaping policy and encouraging follow-up research surrounding the Smart Triage platform.

Calculate the incremental cost relative to years-of-life lost (YLL) averted from the Smart Triage program compared to the standard of care, using the pre-intervention phase at JRRH as the control.

Determine the probability of cost-effectiveness of the Smart Triage program under different willingness-to-pay thresholds, when implemented in a government regional referral hospital in Uganda, using a decision-analytic model from the societal perspective. The comparator will be an ETAT-based manual system widely used in Uganda, including our study site prior to implementation of Smart Triage.

Investigate the impact of OOP costs on cost-effectiveness by secondary analyses from the government and patient perspectives.

Materials and methods

Description of parent clinical trial

The Smart Triage program and a corresponding clinical trial will be implemented in the pediatric outpatient department at JRRH in Uganda, which serves children up to age 18 presenting from the community at various degrees of disease severity. The outpatient department serves as a gateway to admission if necessary. All presenting children, except those with scheduled appointments, are eligible to participate. However, enrollment of all children is not possible due to a high patient load, and therefore a sampling framework as published in the parent trial’s protocol is used to obtain a representative sample [17]. The Smart Triage trial was designed as a difference in differences trial, with a Kenyan site for control. However, we will restrict this cost-effectiveness analysis to the Ugandan site at JRRH, to minimize the impact on our analysis due to variations in baseline ETAT implementation between countries. Therefore, we will compare the post-intervention data at JRRH with that of the pre-intervention phase, with the latter serving as control for our analysis.

The primary outcome of the Smart Triage trial is the difference in the proportion of children who receive a sepsis bundle of care within one hour before and after Smart Triage’s implementation. Enrolled children will be followed up to assess in-hospital mortality (if admitted), 7-day post-discharge mortality, and 7-day post-discharge healthcare utilization rates. Of note, Smart Triage does not aim to indiscriminately increase the number or proportion of children receiving sepsis bundle. Indication for sepsis treatment is determined by local clinicians both before and after Smart Triage implementation. Smart Triage simply facilitates the timelier delivery of sepsis treatment, if deemed indicated by local clinicians, through a data-driven quality improvement process.

Further details of the parent clinical trial such as program implementation, clinical trial protocol, and sample size calculations have been published [16] with appropriate clinical trial registration on clinicaltrials.gov (NCT04304235). Ethics approval has been obtained from Makerere University School of Public Health (MUSPH) Higher Degrees, Research and Ethics Committee. MUSPH provided approval for the parent study and this economic evaluation to be conducted at Jinja Regional Referral Hospital in Jinja, Uganda. Written informed consent for the parent clinical trial as well as follow-up analyses such as this economic evaluation will be obtained from all caregivers/participants prior to enrollment. Patient recruitment has commenced and program implementation is underway, with the earliest projected end-date of data collection being late 2021. While the parent trial’s protocol aims to establish clinical efficacy of the Smart Triage platform, this protocol aims to establish cost-effectiveness, and will be conducted alongside the parent clinical trial.

Decision analytic model

We will perform the economic evaluation of Smart Triage using a decision analytic model (Fig 1). The model structure is based on a similar model used to evaluate the cost-effectiveness of a post-discharge program in Uganda named Smart Discharges [25], and the model care pathways have been shown to have face validity by collaborators at JRRH. As patients in Uganda bear a significant portion of healthcare costs due to the lack of universal healthcare, we will take a societal perspective for our base case analysis [26]. Also, since Smart Triage is a program that facilitates immediate treatment for an acute illness, it would likely exert its strongest effects on short-term outcomes, and therefore we will use a 7-day horizon for our analysis. This 7-day horizon is also congruent with the recommended duration of antibiotic administration for a typical episode of sepsis [4]. Model structure and key assumptions are presented in Fig 1 and Table 1, respectively.

Fig 1

Decision analytic model of post-triage and post-discharge care pathways, used for the cost-effectiveness analysis of Smart Triage.

Table 1

Key assumptions of the decision analytic model.

Assumption	Rationale
Smart Triage exerts its strongest effects on mortality and short-term morbidity within the initial 7 days.	Smart Triage facilitates an acute intervention to target short-term outcomes (sepsis-induced mortality and morbidity during the immediate illness) and will likely exert its strongest effects in this timeframe. The model may be updated when long-term disability data and disability weights for sepsis survivors become available in LMICs.
The odds ratio of death for receiving a sepsis bundle within 1 hour in high-income setting may be used to link the Smart Triage study’s primary outcome to mortality in an LMIC setting.	LMIC-specific data in this regard is not yet available. However, due to various factors predisposing LMIC patients to a higher disease severity upon presentation and the lack of expensive intensive care support, a timely sepsis bundle may be even more crucial and exert a stronger effect in these patients. Our use of a high-income setting odds ratio may therefore not only be appropriate but a conservative estimate of the true value for LMICs. Please see the “discussion” section for details. Our model may be updated with an LMIC-specific odds ratio when available.
Costs of care in pneumonia can be used to approximate costs of care in sepsis.	Detailed costing data on sepsis-specific healthcare utilization are not available in Uganda, to our knowledge. Pneumonia features similar healthcare needs as sepsis (antibiotic, oxygen, fluids) and may progress to sepsis. Therefore, we will use previously published Ugandan costing data for pneumonia to approximate the care costs of sepsis [23], and update our model with all-cause sepsis costs when available.
The cost of death is taken to be zero in our model. (Note that care costs leading up to death are not taken to be zero and are counted in our costing data.)	In costing studies [25], OOP costing for death, such as funeral expenses, was foregone due to emotional trauma experienced by caregivers when asked such questions. Therefore, this data is not available.
Estimates of life expectancy from the World Health Organization, used to calculate YLL averted, representatively captures individuals with a history of childhood infection or sepsis.	Sepsis survivors are likely well-represented in census-based estimations of population life expectancy given a high childhood incidence of sepsis in Uganda. In addition, patients presenting to JRRH with infection vary widely in their initial severity of disease, and given JRRH’s wide catchment area, likely reflects the spectrum of childhood infection severity. Therefore, use of census-based estimates of life expectancy to calculate YLL averted for the JRRH sample seems appropriate.

Model inputs

Probabilities

Probabilities for admission, receiving sepsis bundle within one hour, post-discharge healthcare utilization, and post-discharge mortality will be obtained from the Smart Triage clinical trial.

Program costs

Equipment costs of Smart Triage will be recorded as prices in Ugandan Shilling (UGX) if a component was available in the country; otherwise, the imported price by the research team will be used. Wages will be recorded in local currency. Since the results of this cost-effectiveness analysis are meant to inform implementation of Smart Triage as a new standard of care, we will model for continuation of Smart Triage in our base case analysis. Specifically, one-time costs will be annuitized with a 3% discount rate as per international modelling guidelines [27], assuming a five-year lifetime for equipment, while annual costs will be distributed evenly among each patient.

As Smart Triage is being implemented at JRRH, a public center, program costs borne by the research team will be regarded as costs from the government perspective. We will assume a +/- 25% variation in a deterministic distribution to account for uncertainty in a probabilistic sensitivity analysis.

Healthcare utilization costs

Government and OOP costs of care from inpatient and outpatient cases are available from a Ugandan costing study for pneumonia in children under five years, spearheaded by the Decade of Vaccine Economics (DOVE) project at Johns Hopkins University [23]. As costs of care for all-cause sepsis are unavailable and pneumonia has similar treatment needs to sepsis, such as oxygen and antibiotics, we will assume that DOVE’s costs approximate that of all-cause sepsis. The DOVE costing study surveyed 16 public, 15 private for-profit, and 17 non-profit non-government facilities from four geographically diverse districts in Uganda (Gulu, Jinja, Mbarara, and Wakiso) using a top-down approach. This multi-district data provides an estimation of the national average costs of care in the public, for-profit, and non-profit sectors for inpatients and outpatients in Uganda, which we will link to care-seeking probabilities obtained from the Smart Triage trial. Per diem costs of inpatient care will be inferred in each sector by dividing average inpatient costs by the median length of stay. To maximize the external validity of our estimate for wages lost due to illness, we will use the national average daily wage in Uganda from the International Labor Organization [28] multiplied by the average days of missed work by caregivers in our trial.

Adjusting costs for inflation

Cost data from secondary sources will be adjusted to current-year costs through the Ugandan gross domestic product (GDP) deflator from the World Bank. Program costs will be adjusted through the Ugandan GDP deflator if accrued locally, or through the GDP deflator of the country of purchase if imported, as per World Health Organization (WHO) guidelines for the economic evaluation of interventions [29].

Clinical effect–years of life lost

As the Smart Triage trial is not powered to detect a difference in mortality, the difference in the proportion of children who receive a sepsis bundle within one hour will be linked to mortality via existing literature describing the impact of timely sepsis bundle on mortality [24]. We will then calculate years of life lost (YLL) from the modelled mortality rate using life expectancy data from WHO [30]. Due to the lack of disability weights and longitudinal disability data in LMIC sepsis survivors, we are unable to model years lived with disability for calculation of disability-adjusted life years (DALYs) averted, and therefore will restrict our analysis to YLLs averted.

Model and parameter uncertainty

One-way sensitivity analyses

We will explore the assumption of the continuation of Smart Triage beyond trial completion via a one-way sensitivity analysis where one-time costs are incurred within the study duration without annuitization. One-way sensitivity analyses using discount rates of 0%, 1% and 5% will also be performed as per international modelling guidelines [27].

In addition, the patient load of the JRRH outpatient department was significantly decreased during the first two months of the Smart Triage study due to the coronavirus (COVID) pandemic. Local collaborators noted that travel restrictions resulted in caregiver hesitancy to seek care for their children and estimated a more than 50% decrease in number of presenting patients per day. As Smart Triage’s primary outcome, the proportion of children receiving sepsis bundle within one hour, is a process outcome that would be dependent on patient load, we will perform a sensitivity analysis to investigate any bias introduced by a reduced patient load. Specifically, we will perform this sensitivity analysis by excluding the initial two months of patient recruitment along with any subsequent timeframe where patient load is judged by local clinicians to decrease by more than 50%.

Probabilistic sensitivity analysis

A probabilistic sensitivity analysis will be conducted using a Monte Carlo simulation with 10,000 iterations. Values from odds ratios and patient data will be drawn from a log-normal distribution and bootstrapping, respectively. Program costs will be drawn from a deterministic distribution with limits of +/-25%. Healthcare utilization costs will be drawn from mean and standard deviations via the method of moments, assuming a gamma distribution. If only a point estimate is available from the literature, the standard deviation will be inferred from a conservatively estimated coefficient of variation in consultation with expert opinion, followed by the method of moments as described. All statistical analyses and modelling will be performed using R [31].

Results to be reported

The proportion of children who receive a sepsis bundle of care within one hour will be reported for both the pre-intervention and post-intervention phases. Likewise, all other probabilities and costs used as model inputs will be reported, including program costs. The results of the Monte Carlo simulation will be presented on a cost-effectiveness plane, in order to illustrate the percentage of simulations that result in added effectiveness for an added cost. We will summarize these results on a cost-effectiveness acceptability curve, which will illustrate the confidence of Smart Triage being cost-effective at different thresholds of willingness to pay per YLL-averted. All results will be reported in multiples of Uganda’s GDP and in United States dollars (USD) for ease of interpretability by local policymakers and the international research community alike.

Discussion

Strengths & implications of study

The Smart Triage program is a pioneering triage infrastructure to expedite sepsis treatment using a mobile-based algorithm derived from existing guidelines combined with organizational assistance through an automated patient tracking system and an electronic clinician dashboard. The results of our proposed economic evaluation can further encourage follow-up trials and governmental scale-up efforts, especially when combined with subsequent budget impact analyses to model a district-level or national-level scale-up in Uganda [32]. To this end, we have designed Smart Triage to be of low technical maintenance after initial set up and have heavily involved local stakeholders during program implementation to ensure sustainability after study termination. This model of sustainability has been successfully used in similar projects by our group [33]. In addition, our implementation of Smart Triage as a package allows the program’s various components to work in tandem to tackle multiple coexisting barriers to care. Although this implementation strategy prevents the isolated cost-effectiveness analysis of each package component, we are conceptually unlikely to exert a clinically significant effect by tackling one barrier to care but not others. Finally, our unpublished study data thus far indicates a high follow-up rate of 98.3%, minimizing the impact of incomplete outcome data in our analysis. As we relied exclusively on telephone follow-up, we optimized follow-up through rigorous checks for phone number accuracy, repeat calls to unreached participants through a standardized process, and use of shared phone number, such as that of a neighbor, should a personal number not be available. Our final follow-up rate will be published upon completion of the trial.

To our knowledge, our analysis is the first proposed study to employ a decision-analytic model in evaluating the cost-effectiveness of a sepsis program focused on triage in an LMIC setting. This technique allows for a more comprehensive evaluation of cost-effectiveness compared to previous economic analyses of LMIC triage programs [13,20], and allows for comparison of cost-effectiveness to analyses of sepsis triage programs in high income settings [18,19]. Specifically, our model allows linkage of receiving sepsis bundle within one hour to mortality based on published data, thereby increasing the clinical interpretability of our results. Additionally, a decision-analytic model allows us to incorporate post-discharge follow-up data, such as healthcare utilization and post-discharge mortality, rather than artificially restricting our analyses to the inpatient timeframe. Lastly, the use of a probabilistic sensitivity analysis incorporates uncertainty into our analysis following international modelling guidelines [27].

To enhance external validity, JRRH was selected for our study site due to the similarity of its triage infrastructure and existing use of ETAT to many Ugandan centers of comparable catchment size [34]. Therefore, despite the single-center nature of our analysis, our estimates of clinical efficacy may be suggestive of that in a typical Ugandan regional referral center, a hypothesis awaiting confirmation through a multicenter follow-up trial. Additionally, our use of a costing dataset sampled from various geographically diverse regions across Uganda also increases external validity outside of JRRH [23]. Although this costing data arises from pneumonia cases, the similar treatment requirements to sepsis make this data the best option in the absence of data for all-cause sepsis. While healthcare utilization and associated costs of pneumonia may not be identical to sepsis, the two disease processes have significant overlap as pneumonia may progress to sepsis if sub-optimally treated, particularly in resource-constrained settings. Therefore, we chose to use secondary data for pneumonia care costs in their entirety, rather than an item-by-item selection of costs. To this end, our model could be updated with LMIC-specific all-cause sepsis costing data when available. Finally, to further optimize external validity, we used the national Ugandan average in daily wages, rather than the local average, to calculate the cost of lost wages from days of missed work [35]. Through these uses of multiregional and national linkage data to maximize external validity, our analysis is well-situated to inform further multicenter research from a clinical and economic standpoint.

As Smart Triage is being implemented during the COVID-19 pandemic, we have also taken steps to maximize our analysis’s external validity to the post-pandemic setting. To this end, we will perform a sensitivity analysis where time periods with decreases in estimated patient load greater than 50% will be excluded. These time periods may bias our results by artificially increasing proportion of children receiving sepsis bundle within one hour due to a lower triage burden not seen outside of a pandemic setting, or may conversely result in more extensive delays due to lower staffing ratios from staff redeployment. According to local staff, these decreases in patient load only occurred within the first two months of patient recruitment as of this protocol’s writing and may have resulted from caregivers’ hesitancy to seek care due to travel restrictions early in the pandemic. We were also informed by local administrative and clinical staff that while JRRH has put in place infectious disease precautions in response to the pandemic, the triage process remains unchanged from the pre-pandemic process. Therefore, changes observed in the clinical trial may be more easily attributable to Smart Triage rather than another change to the triage system made in response to the pandemic.

Limitations of study

Our linkage of receiving sepsis bundle within one hour to mortality provides a more clinically relevant endpoint to our analysis by calculating cost per YLL averted. However, we use data from high-income settings for this linkage, as LMIC-specific data is not yet available due to the recency of the one-hour recommendation for defining timely treatment [4]. Despite this apparent limitation, we postulate that this substitution is appropriate, as the relationship between timely sepsis bundle and mortality in a high-income setting is likely a conservative estimate of that in the LMIC setting for several reasons. Firstly, children are more likely to present later in their disease course due to pre-hospital barriers to care such as delayed caregiver recognition of illness or inefficient transportation [36]. Secondly, children in LMICs have a lower physiological reserve from a higher comorbidity burden and malnutrition rate. Lastly, these children may present in greater need of resuscitation than a high-income setting, due to the preponderance of dehydration from diarrheal diseases [3] aggregated by decreased access to clean drinking water. For reasons such as these, early sepsis bundle may conceptually be more crucial and exert an even greater mortality benefit on children in LMICs than high-income settings. Therefore, as a corollary, not only is our substitution with high-income setting linkage data appropriate, but a future repeated analysis with LMIC-specific data may reveal even greater cost-effectiveness than our proposed analysis. On the other hand, if we had foregone the linkage of a timely sepsis bundle to mortality due to the current unavailability of LMIC-specific linkage data, our analysis would generate only the cost per child receiving timely bundle, as opposed to cost per YLL-averted, with a significantly reduced clinical interpretability.

Another area where LMIC-specific evidence is lacking is long-term quality-of-life data and corresponding disability weights in sepsis survivors. Therefore, while cost per DALY averted is a widely used metric to calculate cost-effectiveness in the LMIC setting, we are unable to estimate years lived with disability for calculating DALYs. Extrapolating quality-of-life evidence from high-income settings into the LMIC setting would be inappropriate given vast differences in social support systems. We also lacked the resources to carry out our own LMIC-based valuation study where different health states are methodically assigned disability weights. When such evidence with appropriate valuation data becomes available in the LMIC setting, long-term outcomes may be incorporated into our economic evaluation through a Markov model. An estimate of the cost per DALY averted from such a model would enhance comparability with estimates of the cost-effectiveness of other common health systems interventions in LMICs.

Lastly, while our cost-effectiveness analysis will be an important first step in shaping local policy towards scaling up of Smart Triage should cost effectiveness be demonstrated, our analysis does not take into account the impact of scale up on issues such as reallocation of resources, impact on healthcare budget given sepsis’ burden of disease [32], and multicenter scale-up costs [37]. To this end, future directions of study may include a budget impact analysis factoring in scale-up costs, the number of centers for scale up, and the large burden of disease of sepsis in these centers.

Conclusions

Conducting a cost effectiveness analysis of a program like Smart Triage is an important aspect of any parent study, especially in resource constrained settings. To complement the Smart Triage interventional trial, aimed at providing expedited sepsis treatment in children, we propose methods for a cost-effectiveness analysis of the platform. Our analysis, together with subsequent analyses modelling budget impact and scale up, will inform future modifications to the Smart Triage platform, as well as motivate scale-up to the district and national levels. This protocol may also serve as a template for the economic evaluation of similar technologies in LMICs.

(DOCX)

Click here for additional data file.

2 Sep 2021

PONE-D-21-21174

Cost-effectiveness analysis protocol of the Smart Triage program: a point-of-care digital triage platform for pediatric sepsis in Eastern Uganda

PLOS ONE

Dear Dr. Grays,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Oct 16 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Tai-Heng Chen, M.D.

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thanks for the opportunity to review this manuscript entitled "Cost-effectiveness analysis protocol of the Smart Triage program: a point-of-care digital triage platform for pediatric sepsis in Eastern Uganda". This is a protocol for an important research question. Overall, the manuscript is well written with enough details in different sections. Tables are informative. Below are comments/concerns for the authors to consider.

- Include a results section with expected outcomes and results to be reported

- Include a conclusions section

- Discuss alternative strategies to address some potential limitations

- Discussion, the authors should expand and elaborate more on how their findings support or contrast available literature and provide suggestions for future research directions that would address existing knowledge gaps.

Reviewer #2: Manuscript title: Cost-effectiveness analysis protocol of the Smart Triage program: a point-of-care digital triage platform for pediatric sepsis in Eastern Uganda

The manuscript reports a study protocol. Sepsis is an important cause of mortality in children and early treatment results in better outcomes. ETAT and other guidelines when properly implemented show good predictive power for mortality and improved clinical outcomes.

The SMART Triage program tackles guideline complexity to predict, at admission, those children most likely to deteriorate. The SMART triage program consists of a mobile app linked to a dashboard and a locally hosted Bluetooth treatment tracking mechanism to improve patient organization.

The study is part of a larger study examining pediatric sepsis treatment and is done in Jinga Teaching and Referral Hospital in Uganda. Figure 1 is a useful aid in understanding where the data analysis decision points will be made.

The decision analytic model which includes a societal perspective is a valuable inclusion as out of pocket costs calculations recognize that patients’ bear a sizable portion of health care costs.

Previously published secondary data will be used to link healthcare utilization with costs and intermediate outcomes with mortality. This data is in the Harvard Dataverse repository. It requires permissions for downloading and as such, it was not available to the reviewer in a timely fashion. That said, the variables for calculation of costs seems reasonable based on the brief description available.

Monte Carlo simulations seem very appropriate for this type of analysis-though this type of analysis is outside my specific areas of expertise.

Comments:

This protocol is clearly written. Strengths and limitations are well stated. A major strength is the linking of the sepsis bundle within one hour to longitudinal clinical data and including the 7-day post discharge outpatient time frame. Another strength is the inclusion of cost data from out of pocket expenditures.

1. Please consider a wording change-(patient care organization may be better wording than patient organization)

2. The asserted high follow up rate in the pilot data is a very reassuring, but also quite uncommon (98.5%). Please describe what methods are used to ensure so little loss to follow up.

3. YLL averted is measured by a comparison of data from Jinga hospital where SMART Triage program in utilized (intervention) + one Kenyan hospital and data from one hospitals in Kenya (control).

Have the authors considered using a within Jinga hospital pre intervention time frame as a control for that comparison as well? YLL averted is the critical variable on which all other cost data is framed. Variation in implementation of the ETAT protocol between countries may be greater than hoped and could influence assessment of YLL averted.

Recommendation: Accept with minor revision

Reviewer #3: Thanks for the opportunity to review this interesting protocol. The protocol is well written and detailed, and I make a few suggestions that may make the work easier to follow.

Materials and Methods

1. I suggest you summarize the predictive model the Smart Triage is based on – unless its proprietary. It would be helpful for readers. If proprietary, please say so.

2. Some abbreviations are still not written in full at first use e.g. OOP

3. Description of parent trial – provide more detail on the trial. Are patients randomized? How? Are all patients at the outpatient department screened and enrolled or is there a sampling framework? Does the trial collect data on hard outcomes such as mortality, even if it is not powered to assess them?

4. It seems that the methods make a fundamental assumption – that more children receiving the sepsis bundle is good, even if it is not indicated. Please clarify.

5. Pneumonia patients often require imaging. Will the cost data be item by item, allowing the exclusion of items that may not be relevant? Or are we assuming that it the costs where pneumonia and sepsis differ may not meaningfully matter since pneumonia often leads to sepsis too.

6. Are the cost data from public or private settings? How much variation is factored in?

7. The models assume that the life expectancy of sepsis survivors is similar to the life expectancy of the general population. Without data to back it up, this assumption seems fraught.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PONE-D-21-21174_reviewer-commentsMA.docx

Click here for additional data file.

14 Oct 2021

Responses to reviewer and editor comments have been written in a response letter attached to this submission.

Submitted filename: R1_Response letter 12Oct2021.docx

Click here for additional data file.

2 Nov 2021

Cost-effectiveness analysis protocol of the Smart Triage program: A point-of-care digital triage platform for pediatric sepsis in Eastern Uganda

PONE-D-21-21174R1

Dear Dr. Grays,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Tai-Heng Chen, M.D.

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

Reviewer #3: Yes

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors addressed all my earlier concerns. I have no further or additional comments. The authors a great job improving the manuscript.

Reviewer #3: Thanks for addressing the comments from previous review. Looking forward to seeing the article in the world.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

5 Nov 2021

PONE-D-21-21174R1

Cost-effectiveness analysis protocol of the Smart Triage program:  A point-of-care digital triage platform for pediatric sepsis in Eastern Uganda

Dear Dr. Grays:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Tai-Heng Chen

Academic Editor

PLOS ONE