Common data elements for predictors of pediatric sepsis: A framework to standardize data collection.

BACKGROUND: Standardized collection of predictors of pediatric sepsis has enormous potential to increase data compatibility across research studies. The Pediatric Sepsis Predictor Standardization Working Group collaborated to define common data elements for pediatric sepsis predictors at the point of triage to serve as a standardized framework for data collection in resource-limited settings.
METHODS: A preliminary list of pediatric sepsis predictor variables was compiled through a systematic literature review and examination of global guideline documents. A 5-round modified Delphi that involved independent voting and active group discussions was conducted to select, standardize, and prioritize predictors. Considerations included the perceived predictive value of the candidate predictor at the point of triage, intra- and inter-rater measurement reliability, and the amount of time and material resources required to reliably collect the predictor in resource-limited settings.
RESULTS: We generated 116 common data elements for implementation in future studies. Each common data element includes a standardized prompt, suggested response values, and prioritization as tier 1 (essential), tier 2 (important), or tier 3 (exploratory). Branching logic was added to the predictors list to facilitate the design of efficient data collection methods, such as low-cost electronic case report forms on a mobile application. The set of common data elements are freely available on the Pediatric Sepsis CoLab Dataverse and a web-based feedback survey is available through the Pediatric Sepsis CoLab. Updated iterations will continuously be released based on feedback from the pediatric sepsis research community and emergence of new information.
CONCLUSION: Routine use of the common data elements in future studies can allow data sharing between studies and contribute to development of powerful risk prediction algorithms. These algorithms may then be used to support clinical decision making at triage in resource-limited settings. Continued collaboration, engagement, and feedback from the pediatric sepsis research community will be important to ensure the common data elements remain applicable across a broad range of geographical and sociocultural settings.

Introduction

Sepsis is life-threatening organ dysfunction due to a dysregulated host response to infection [1]. Despite a global trend of decreasing incidence and mortality, sepsis remains a major cause of health loss worldwide and has an especially high burden in low-and-middle-income countries [2]. In recognizing the enormity of the global burden of sepsis in terms of mortality, morbidity, and social and economic consequences, the World Health Assembly, the decision-making body for the World Health Organization highlighted the urgent need for strengthened efforts to identify, prevent, and treat sepsis [3].

Improved prioritization, coordination, and timely identification of critically ill (or at risk of becoming critically ill) children has been recognized as cornerstone in the efforts to improve sepsis outcomes [4,5]. Clinical decision making in these areas can be supported and enhanced with a data driven precision health approach and patient-centred, personalised risk prediction recommendations. The importance of powerful data driven prediction algorithms is underlined by the fact that they can be used by healthcare workers with less experience and training, a common scenario in resource-limited settings where the burden of sepsis is highest. Early identification of the at-risk patient can improve healthcare delivery by facilitating timely administration of treatment before significant deterioration occurs, and early referral when a higher level of care is required.

Progress in the adoption of this data driven approach to optimize health and healthcare has been limited by the lack of robust, high quality sources of data and validated clinical outcomes [6]. It is becoming increasingly clear that multi-center collaborations and combining data from multiple settings are necessary to generate the highest level of evidence to guide clinical practice [6]. However, the linking and aggregation of collected data are often limited by the varying degrees of data compatibility across studies. One way to ensure meaningful data exchange and compatibility into the future is to support harmonized use of a recommended set of common data elements. A common data element is a combination of a precisely defined question (predictor variable) paired with a specified set of clearly defined responses to the question [7]. Common data elements can also be classified into tiers of importance to document the relevance of each data element for future studies and to advise a minimum set of routinely collected data across studies [8]. By increasing consistency of data collected across studies common data elements can facilitate data aggregation, meta-analyses, cross-study comparisons, and simplify training operations to increase overall efficiency and quality of data collection [7].

The purpose of this study was to create a universally accepted living framework to increase standardization of data collection in future studies of pediatric sepsis prediction. The Pediatric Sepsis Predictor Standardization (PS2) working group, a subgroup within the Pediatric Sepsis CoLab, has collaborated to develop a set of common data elements for predictors of pediatric sepsis at the point of triage in resource-limited settings. Standardized outcomes are being addressed by a related working group [9,10]. Routine use of the framework within the pediatric sepsis research community would promote an environment of collaboration, improve compatibility of datasets across studies, and yield robust, high quality sources of data that can be used to develop powerful risk prediction algorithms and support clinical decision making.

Methods

Ethics approval was obtained from the Child and Women’s Research Ethics Board at the University of British Columbia (H17-01893). All expert contributors are members of the Pediatric Sepsis CoLab and voluntary involvement in the Delphi process implied consent.

Pediatric sepsis predictor standardization (PS2) working group

The PS2 working group is a subgroup within the Pediatric Sepsis CoLab, a global network for collaboration and data sharing to address pediatric sepsis morbidity and mortality [11,12]. The PS2 working group is currently comprised of six experts in pediatric sepsis with representation from Canada, the United States of America, East Africa, and Cambodia; and clinical expertise including pediatric critical care, infectious disease management, and anesthesia.

Study design

A preliminary list of potential predictor variables was compiled based on a systematic review [13] followed by a 5-step modified Delphi approach [14] to develop a set of common data elements relevant to prediction of pediatric sepsis at resource-limited health facilities. Standardized predictors, definitions, prompts, accepted values, and importance tiers were developed using four 60-minute conferences, separated by rigorous revisions based on literature review and expert opinion. An 80% threshold was used to define consensus. The final set of common data elements and guidelines for their use have been published to the Pediatric Sepsis CoLab’s Dataverse [15], a data-sharing platform used by the CoLab to share resources and study data internationally among members.

Systematic review

A prior literature review conducted by our group [13] was used to assemble a preliminary list of candidate predictor variables. The systematic search was conducted using the Ovid MEDLINE database, and the search strategy included terms such as “child”, “sepsis”, “triage”, “prediction”, and “resource-limited settings”. The complete search strategy is available in the S1 Appendix. The most up-to-date versions of published triage guidelines such as Integrated Management of Childhood Illness, Emergency Triage Assessment and Treatment, and the South African Triage Scale were also reviewed for candidate predictors. In addition, reference lists of included studies were screened for potential manuscripts which may have yielded additional candidate predictors. Lastly, case-report forms from large prediction studies in progress by collaborators and working group members were reviewed in order to capture currently used sepsis predictors [16-20]. The systematic review adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (S1 Checklist) [21].

A master list was created using Microsoft Excel to track each predictor variable and its frequency of use across studies. Variables were classified by field type (binary, continuous, categorical), and organized into seven core domains: patient characteristics/history, pregnancy/birth details, sociodemographic information, anthropometric data, vital signs, clinical signs and symptoms, and laboratory tests. The clinical signs and symptoms domain was further divided into seven sub-categories: respiratory, circulation/perfusion/dehydration, neurological, infection, gastrointestinal/genitourinary, malnutrition, and trauma. The working spreadsheet documenting the results of the systematic review and source studies for each predictor is available on the Pediatric Sepsis CoLab Dataverse [22].

Delphi process

A 5-step modified Delphi process was used to determine the final list of predictors, develop standardized definitions, response values and terminology for data collection; as well as to assign a tier of importance to each variable (Fig 1).

Fig 1

Flowchart depicting the modified Delphi process used to develop the case report form for predictors of pediatric sepsis.

Step 1: Independent review of the preliminary list of potential sepsis predictors to comprise the core common data element set.

Members of the PS2 working group independently reviewed the master list of predictors and flagged predictors to exclude from the common data element set. The decision to exclude was based on considerations of the perceived predictive value of the candidate predictor at the point of triage, intra- and inter-rater measurement reliability, and amount of time and material resources required to reliably collect the predictor in resource-limited settings. Predictors with 100% consensus for exclusion were excluded. Predictors that did not reach this consensus agreement were discussed in Step 2.

Step 2: Consensus meeting to determine final list of predictors to comprise the common data element set.

Step 2 occurred as a 60-minute web conference during which predictors without unanimous exclusion consensus were openly discussed until consensus was reached. At that time, members were encouraged to suggest additional variables and reach consensus as a group on whether each of these variables should be included in the common data element set. By the end of the meeting, the list of included predictors for common data element set was agreed upon and finalized.

Step 3: Two web conferences to develop standardized definitions, prompts and responses for each variable.

For each predictor on the consensus list, standardized prompts and suggested response values were systematically constructed using a rigorous combination of literature sources and expert opinion. Operational definitions of predictors from the literature were consulted where available. Sources of operational definitions included study publications, case report forms from ongoing studies, and terminology standardization organizations such as SNOMED [23]. Where no operational definition was available from the literature, members of the working group constructed prompts that were easy to follow, objectively interpretable, and culturally compatible based on clinical and research experience. Predictors with relevance limited to a specific subset of participants (e.g. jaundice and bulging fontanelles in neonates) were also identified. Criteria for activating these predictors were selected based on accepted values in the literature and expert consensus.

The resultant list of predictors with standardized prompts and accepted responses was then circulated to the working group for individual voting. Prompts and responses flagged for review were discussed during a 60-minute web conference, after which the flagged items were further refined based on suggestions of the working group. A second round of voting and video conference was conducted to finalize the prompts and responses for each predictor.

Step 4: Independent voting on tier of importance for each predictor.

To guide the inclusion of certain high-yield variables in future studies, each predictor was prioritized using a 3-point tiering scale based on importance from an epidemiological or data management point of view [24]. During Step 4, members of the PS2 working group independently assigned a tier of importance to each predictor. Predictors were ranked as Tier 1 when considered essential for all studies. These predictors comprise the consensus-based minimum data set to be collected across all future studies. Tier 2 predictors were defined as important predictors that should be collected in relevant studies and when resources allow, and Tier 3 predictors were exploratory predictors or granular data necessary for a few specific studies within the field. Consensus agreement was defined as at least 80% among all members. Predictor variables that did not reach 80% agreement were discussed in Step 5.

Step 5: Consensus meeting to finalize importance tiers assigned to each predictor.

In Step 5, members engaged in a 60-minute web conference to openly discuss tiering of variables that did not reach 80% agreement until consensus was reached.

The results of the modified Delphi process were used to develop the common data element set that was then reviewed and approved by all members of the working group.

Dataverse

The set of common data elements and supplementary materials were published on the Pediatric Sepsis CoLab Dataverse, a secure platform that is internationally accessible but exhibits customizable permission control to protect sensitive data [15,22]. This Dataverse is the Pediatric Sepsis CoLab’s main data-sharing platform and allows for data sharing among members with built in access and version control. To this end, patient-level datasets of predictors are also shared on Dataverse, alongside other general resources for CoLab members. Access to resources labelled as restricted files are available through a simple 5-minute online application [25]. Access to patient-level data may be obtained on a per-project basis by contacting the Pediatric Sepsis CoLab coordinator [15].

Results

The final set of 116 common data elements and supplementary materials are published and freely available on the Pediatric Sepsis CoLab Dataverse [22].

Final list of predictors

The literature review [13] yielded a preliminary list of 144 potential predictor variables (S1 Table) (Fig 1). Consensus for exclusion was attained for 50 of these variables during the independent review (S1 Table). The 25 variables that did not reach consensus for exclusion during the independent review were flagged for discussion in the consensus meeting (Step 2), and 17 of these variables were ultimately excluded (S1 Table). Most common reasons for exclusion were inadequate inter- or intra- rater measurement reliability (N = 31) due to subjectivity of the measures or the potential impact of recall bias on data integrity, and lack of perceived predictive value at the point of triage (N = 31) (S1 Table). Predictor variables were also excluded if information could be captured from a more detailed variable (N = 5). For example, the “born at home” predictor was captured as a response option in the more comprehensive “facility of birth” predictor. During the consensus meeting (Step 2), an additional 64 predictors were proposed for inclusion, 39 of which achieved consensus for inclusion (S2 Table). Consensus for inclusion in these guidelines was reached on a final list of 116 predictors (Fig 1). Predictors were grouped by categories and subcategories for ease of navigation of the guideline and to guide data grouping in future studies (Table 1).

Table 1

Schematic of the common data element guidelines with examples.

Recommended predictor	Tier	Standardized question or prompt on data collection platform/form	Possible values	Notes
Clinical Signs and Symptomsa
Infectionb
NEONATAL JAUNDICE branching logicc
Neonatal Jaundice (observed)d	1e	Does the child have yellowed skin?	Yes/no
IF: neonatal jaundice = yesf
Phototherapy needed?	2	Ask: “Did the child need light therapy for yellow skin?”	Yes/no/unknown
Bilirubin measurement	2			See Laboratory section for details.
RASH branching logic
Rash (observed)	2	Does the child have a rash?	Yes/no
IF: rash (observed) = yes
Localized rash (observed)	2	Is the rash localized?	Yes/no
IF: localized rash (observed) = yesg
Rash body part (observed)	3	Record the body part where the rash is localized.	Scalp/face/neck/torso/left upper limb/right upper limb/left lower limb/right lower limb/perineal/other

aPredictor category.

bPredictor subcategory.

cHeading for branching logic tree.

dTier 1 variables are bolded.

eBlue cells indicate variables applicable to young infants (< 2 months).

fRed cells indicate first order branching logic conditions that trigger subsequent variables.

gYellow cells indicate second order branching logic conditions that trigger subsequent variables.

Features of the guidelines

Importance tier

Unanimous agreement was reached for all predictors ranked as Tier 1. Of the predictors ranked as Tier 2 and Tier 3, 16 required group discussion in order to reach consensus (S1 and S2 Tables). The common data element set consisted of 26 Tier 1 predictors, 66 Tier 2 predictors, and 24 Tier 3 predictors. The importance rank of certain variables depended on the geographical setting of the study and local infectious disease epidemiology. For example, a predictor exploring use of insecticide treated nets would rank as Tier 1 in malaria endemic regions and Tier 2 in most other settings (Table 2). Instructions explaining the variation in tier assignment of such predictors have been included in the guidelines.

Table 2

Sample predictors as presented in the common data element guidelines.

Recommended predictor	Tier	Standardized question or prompt on data collection platform/form	Possible values	Notes
Sociodemographic data
Bed-net use (reported)	2	Ask: "How often do you use insecticidal (bug resistant) bed nets for the child?" Read out options.	Never/rarely (<1 per week)/sometimes (1–3 times/week)/often (4–6 times/week)/always/unknown	Consider tier 1 for malaria endemic regions, or if variable is otherwise of interest to the study.
Clinical Signs
Circulation/perfusion/hydration
Capillary refill time > 3 seconds (upper limb)	1	Apply pressure to a thumb or finger for 3 seconds to blanch it. Does it take more than 3 seconds to return to original pink color after you let go? (See SOP).	Yes/no	See SOP for details. 3 second cut-off shown to be applicable for all pediatric age groups (8).

Standardized prompts

A standardized prompt was constructed for each variable (Table 2). The prompts serve to standardize the syntax and operational definitions of predictors, and when applicable, the phraseology to be used by the research team during data collection. The prompts were carefully crafted with terminology comprehensible to respondents and language appropriate for the study setting. For variables captured by means of clinical examination, prompts also provided instructions to standardize measurements. Additional enhancements to the prompts may be required following field testing in various locations and following translation.

Possible values

A suggested set of response values are provided for each variable to facilitate compatibility between shared data sets (Table 2). The values can be used for data cleaning and to enhance opportunities for data linking.

Notes

An additional section is added for each variable that allows for communication of any special suggestions for using a predictor, rationale for how a predictor is presented, or other information that is likely to be of interest to users of the common data element set (Table 2).

Branching logic

Branching logic is used where applicable to avoid redundant questions, elucidate complex history concepts in a standardized manner, and expedite data collection (Table 3). Each branching logic tree is presented under a single heading for clarity (Table 1). The efficiency offered by branching logic can be maximally harnessed by using an electronic data collection interface rather than traditional paper methods [17].

Table 3

Sample branching logic to elucidate diarrhea timeline as per World Health Organization definitions.

Recommended predictor	Tier	Standardized question or prompt on data collection platform/form	Possible values	Notes
GI/GU
DIARRHEA branching logic
Diarrhea (reported)	1	Ask: "Does the child currently have more than 3 loose stools a day?"	Yes/no/unknown	Cut-offs of 3 loose stools are chosen based on WHO definition for diarrhea.
IF: diarrhea (reported) = yes
Persistent Diarrhea (reported)	1	Ask: "Has your child been passing loose stools for more than 2 weeks?"	Yes/no/unknown	Cut-offs of 2 weeks based on WHO definition for persistent diarrhea.
IF: persistent diarrhea (reported) = no
Dysentery (reported)	2	Ask: "Has the child had blood in their stools since getting loose stools?"	Yes/no/unknown	Dysentery only asked for non-persistent diarrhea as per WHO definition.

Standard operating protocols

Standard operating protocols have been developed to provide additional standardization in variables with more involved physical exam procedures, such as anthropometric measurements and vital signs. These are available as supplementary material alongside the common data element guidelines [22].

Feedback survey

A web-based survey is available through the Pediatric Sepsis CoLab for researchers to propose new candidate predictors or suggest modifications to existing variables [26]. Feedback will be taken into account as input into future repetitions of the Delphi process to process to determine inclusion in updated iterations of the common data element set.

Discussion

Summary

Standardized data collection methods across pediatric sepsis prediction studies are lacking. As a step toward increasing consistency and compatibility for aggregation of future datasets, a multidisciplinary team of experts in pediatric sepsis collaboratively generated a set of common data elements and guidelines for inclusion in case report forms or other data collection methods. Even when used within a single study, common data elements can provide consistency and efficiency in establishing data collection infrastructure and minimize variability in training and implementation [7]. Routine use of this standardized framework in future studies can increase the quality and reproducibility of research and results in the field of pediatric sepsis, and contribute to development of well validated risk prediction algorithms to guide clinical care. Thus far, the framework has been implemented in the development of digital case report forms (mobile applications) for two independent studies of pediatric sepsis prediction [16-19].

Adoption and use

The collective effort of a diverse group of expert clinician-researchers in the field of pediatric sepsis provided validation of the guidelines and ensured conformity with the current best standards in structured data curation. The broad representation of the expert panel will ensure applicability across a wide range of geographical settings and healthcare systems. The common data element set is meant to serve as the first iteration of a living guideline that will ultimately become the product of collaborative effort and continued engagement from the pediatric sepsis research community. In addition, the systematic review and Delphi process will be repeated every two years to capture new findings. Updated iterations of the common data element set will be published periodically based on contributions received through the feedback survey and emergence of new information.

Limitations

Although the PS2 working group is comprised of expert clinical researchers diverse with respect to geographic location and healthcare setting, the group is not representative of all experts in pediatric sepsis across all geographic locations. It is possible that some variables excluded may in fact be important for prediction studies in select geographical or clinical settings. Further, the tier assignments have potential to be improved based on feedback from experts in geographic settings outside of those represented in the PS2 working group. Our decisions of inclusion and tier assignments were made in consideration of a delicate balance of achieving sufficient data quantity while avoiding diminished data quality due to study participant or staff fatigue. In future guideline iterations, these decisions will be improved upon based on feedback received from the pediatric sepsis community and new information in the literature.

As with the variability in predictor inclusion and importance, the applicability of the standardized prompts and responses may also vary among different geographical or healthcare settings. Additional response options may become available in specific locations and the availability or acceptability of measuring instruments can differ across facilities. In these cases, common data elements would have to be adapted to ensure relevance to the specific setting in which the data are being collected. We attempted to maximize applicability by utilizing culture-neutral phraseology for history questions and consulting international guidelines for standard operating procedures in clinical examination. Nonetheless, data sets from different settings will not be absolutely and completely consistent in actual practice. Despite this limitation, the use of common data elements can maximize compatibility between datasets and increase the quality of data collected within each dataset.

Lastly, despite the widespread use of the Delphi process as a best-practice method for consensus-based research, there are limitations inherent in its methods. The strength of Delphi methodology lies in its iterative process of independent opinion sharing and group feedback; it is commonly used in health care to gain expert consensus on a topic that may not be amenable to data driven methods of evidence generation [14]. The choice of repeatedly capturing independent opinions followed by active group discussions allowed for each voting round to be conducted anonymously and without potential influence from the opinion of others. The variables that did not reach unanimous agreement during anonymous voting were subsequently discussed amongst the PS2 working group and were susceptible to potential bias introduced by group interpersonal dynamics. Nevertheless, active discussions promoted the use of collaborative thinking, generation of new ideas and solutions, and consideration of different perspectives to reach consensus on variables. To this end, the feedback survey is an extension of this collaborative modified Delphi process that involves the pediatric sepsis research community beyond the PS2 working group.

Conclusion

Using a systematic review and modified Delphi approach, we developed a set of common data elements prioritized as essential (Tier 1), important (Tier 2), or exploratory (Tier 3) to serve as a standardized framework for future studies of pediatric sepsis prediction. This will enable collaboration and coordination in the pediatric sepsis research community and strengthen the quality of risk prediction models for pediatric sepsis to support clinical decision making at the point of triage.

Completed PRISMA checklist.

(PDF)

Click here for additional data file.

Search strategy for systematic review.

(PDF)

Click here for additional data file.

Preliminary list of predictor variables compiled through literature review (N = 144).

(PDF)

Click here for additional data file.

Additional predictor variables proposed in Step 3 (N = 64).

(PDF)

Click here for additional data file.

1 Apr 2021

PONE-D-20-35664

Common data elements for predictors of pediatric sepsis: A framework to standardize data collection

PLOS ONE

Dear Dr. Mawji,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 16 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Valérie Pittet, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

3. Please provide additional details regarding participant consent to collect personal data, including email addresses, names, or phone numbers. In the Methods section, please ensure that you have specified how consent was obtained and how the study met relevant personal data and privacy laws. If data were collected anonymously, please include this information.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript:

[AC is supported by a Wellcome Trust Doctoral Training Fellowship. SA was supported by the Initiative to Develop African Research Leaders (IDeAL) Wellcome Trust award (# 107769).]

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

[The authors received no specific funding for this work.]

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

5. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere. [Figure 1 (the systematic review process) was also published in our related manuscript. It is not dual publication because are just using the results of a literature review we conducted in the previous publication for a new purpose. This is further detailed in the cover letter. ] Please clarify whether this publication was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

6. Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure.

7. Please include captions for ALL your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The article presents a collection of predictors of pediatric sepsis sepsis and describes the process how this set was selected. The selection is based on expert’s opinions, not analysing data. The aim is to make the selected data elements as a standardised way to collect pediatric sepsis data. This would increase compatibility across different studies on the topic. The topic of the study is definitely important and results have a clear impact.

The research methodology is based on systematic literature review and domain experts’ knowledge through Delphi method. Finally, 116 data elements were accepted. The methodology is very well described and suitable for the study. However, the article does not describe whether any data collection has already done. Even a limited evaluation by using existing data would clearly improve the study.

Naturally, predictive power of the data items is difficult to measure since it would require a large amount of data and it also depends on the research questions at hand. If any data is already available, some data quality measures could have been computed. These measures could include basic descriptive statistics such as the number of observations per patient, missing values, correlations, and distributions. This statistical validation could have been an additional step in used methodology but it would also be an additional validation step for the defined set of data items.

The manuscript mentions that the aim is to facilitate the design of efficient data collections tools. Could you explain better what kind of tools these could be? In Conclusion, risk prediction tools are also mentioned. Are these two different sets of tools?

Other comments:

-Two tables S2 and S3 have the same title “Common data elements for predictors of pediatric sepsis: A framework to standardize data collection”

-On these tables a reason for exclusion can be “low perceived predictive values”. Is also based only on expert’s opinions or statistical analysis?

Reviewer #2: This is a very well written and important manuscript. The concept is well-defined and explained in the paper. There are some areas of redundancy that I think can be improved on (discussed more below).

Abstract:

- I really like how the amount of time and material resources to reliably collect the predictor in resource-limited settings was considered -- this is extremely important.

- Conclusion line 72: replace "remains" with "remain"

Introduction:

- Lines 102 and 103: Please standardize how you present "data-driven" vs "data driven", as you spell it with and without a hyphen.

- Line 105: Specify which settings you are referring to here. Low-resource settings?

- Line 106: Improving healthcare delivery seems vague here compared to the following two examples of timely treatment and early referral. Perhaps clarify a bit or omit.

- Line 113: Consider adding a source here.

- Lines 121-123: Use of commas is confusing here, and it ends up being a run-on sentence. I would use ";" to help organize this sentence.

- Line 125: For those unfamiliar with the PS2, I would suggest adding "a subgroup within the Pediatric Sepsis CoLab" after workgroup here. I know you go into more detail in the methods, but I would at least add that here in the Intro.

- Last paragraph: Clarify what the objective of this current paper is. Is it the first sentence? I would suggest making this more clear by stating "the objective of this study" or something like that.

Methods:

- Objective section: This seems a bit repetitive with the objective paragraph in the Intro. Perhaps include this there, instead, and cut where redundant?

- Line 173: Please keep past/present tense consistent throughout as past tense. Here please say "may have yielded".

- Line 174: Also, keep in either active or passive voice. This whole paragraph is in passive voice but the last sentence switches to active voice.

- Lines 188-190: Consider here the use of ";" rather than "," as otherwise this sentence is hard to follow.

- Lines 231-232: For those unfamiliar with the Delphi method, perhaps clarify what "80% threshold required for consensus". Does that mean that 80% of experts had to agree that it was important enough to be included?

- IRB approval was not discussed and should be included.

- If possible to add in a little bit about the experts, like what countries they represent, that would be very interesting.

Results:

- Line 277: Missing a "(" in front of S2 Table.

- Line 279-280: Most common reasons for exclusion "were".

- Line 282: Please standardize spacing before and after "=" as you include spacing here but not elsewhere.

- Lines 331-332: Although I understand the concept of branching logic being maximally harnessed by using electronic data collection, in many low-resource settings this is not a possibility. Is it also appropriate to do this on traditional paper methods?

- Notes section: I might consider moving this to where you discuss Table 2 as otherwise the reader is jumping around quite a bit.

- Standard operating protocols section: This seems a little out of place. I would either cut or elaborate beyond one simple sentence. Perhaps add an example to make this section more relevant and robust.

- Feedback survey: How are the suggestions on the survey incorporated into the common data element set? Are they incorporated immediately or at certain time points? Are the just added in or does consensus have to be achieved on discussion? Perhaps provide some details on this here.

- I may have missed this, but where is the list of the 116 predictors? Perhaps be more clear about where to find these.

Discussion:

- Summary paragraph: This is well written, but it is overall restating a lot of what you have stated above already. I would try to reduce redundancy and streamline a bit if possible here.

- Adoption and use: This paragraph is in future tense, but shouldn't it be past tense since you have already done this? Also, a lot of these 2 paragraphs should go above in the results rather than the discussion. A lot of it is redundant and elaborating on logistics of accessing the Dataverse, so I would say include above where you talk about Dataverse.

Conclusion:

- Final sentence lines 441-444: Instead of saying "This was with the intent..." consider something like "This will enable collaboration", which looks towards the future and proves the benefit of the study.

Reviewer #3: This is well-written and appropriately describes the Delphi process used. The expert consensus group is internationally representative, but is small in number (6 experts). I have no major concerns.

Table 1 is confusing - it might be more helpful to include a sample with variables included, and to describe the significance of bolding and colors in the footnotes. Tables 2 and 3 are more clear.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Tapio Niemi

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

25 Apr 2021

See "Response to Reviewers" uploaded attachment.

Submitted filename: Response to PLOS ONE revieweres 26APR21.docx

Click here for additional data file.

28 May 2021

Common data elements for predictors of pediatric sepsis: A framework to standardize data collection

PONE-D-20-35664R1

Dear Dr. Mawji,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Valérie Pittet, PhD

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: All my comments and suggestions have been taken into account. I have no further concerns and I am happy to recommend the manuscript to be published in Plos One.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

1 Jun 2021

PONE-D-20-35664R1

Common data elements for predictors of pediatric sepsis: A framework to standardize data collection

Dear Dr. Mawji:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

PD Dr. Valérie Pittet

Academic Editor

PLOS ONE