Field testing two existing, standardized respiratory severity scores (LIBSS and ReSViNET) in infants presenting with acute respiratory illness to tertiary hospitals in Rwanda - a validation and inter-rater reliability study.

INTRODUCTION: There is a substantial burden of respiratory disease in infants in the sub-Saharan Africa region. Many health care providers (HCPs) that initially receive infants with respiratory distress may not be adequately skilled to differentiate between mild, moderate and severe respiratory symptoms, which may contribute to poor management and outcome. Therefore, respiratory severity scores have the potential to contributing to address this gap.
OBJECTIVES: to field-test the use of two existing standardized bronchiolitis severity scores (LIBSS and ReSViNET) in a population of Rwandan infants (1-12 months) presenting with respiratory illnesses to urban, tertiary, pediatric hospitals and to assess the severity of respiratory distress in these infants and the treatments used.
METHODS: A cross-sectional, validation study, was conducted in four tertiary hospitals in Rwanda. Infants presenting with difficulty in breathing were included. The LIBSS and ReSViNET scores were independently employed by nurses and residents to assess the severity of disease in each infant.
RESULTS: 100 infants were recruited with a mean age of seven months. Infants presented with pneumonia (n = 51), bronchiolitis (n = 36) and other infectious respiratory illnesses (n = 13). Thirty-three infants had severe disease and survival was 94% using nurse applied LIBSS. Regarding inter-rater reliability, the intra-class correlation coefficient (ICC) for LIBSS and ReSViNET between nurses and residents was 0.985 (95% CI: 0.98-0.99) and 0.980 (0.97-0.99). The convergent validity (Pearson's correlation) between LIBSS and ReSViNET for nurses and residents was R = 0.836 (p<0.001) and R = 0.815 (p<0.001). The area under the Receiver Operator Curve (aROC) for admission to PICU or HDU was 0.956 (CI: 0.92-0.99, p<0.001) and 0.880 (CI: 0.80-0.96, p<0.001) for nurse completed LIBSS and ReSViNET respectively.
CONCLUSION: LIBSS and ReSViNET were designed for infants with bronchiolitis in resource-rich settings. Both LIBSS and ReSViNET demonstrated good reliability and validity results, in this cohort of patients presenting to tertiary level hospitals. This early data demonstrate that these two scores have the potential to be used in conjunction with clinical reasoning to identify infants at increased risk of clinical deterioration and allow timely admission, treatment escalation and therefore support resource allocation in Rwanda.

Introduction

Acute respiratory distress is a frequent cause of pediatric emergency department attendance. Common etiologies include bronchiolitis and pneumonia [1, 2]. The burden of respiratory disease remains in Low-Income Countries (LICs), where mechanical ventilation facilities are limited or unavailable [3]. Deaths from pneumonia and bronchiolitis have been linked to low health coverage, lack of exclusive breastfeeding, malnutrition, incomplete immunization and lack of access to an appropriate health care service [4, 5]. Many health care providers (HCPs) that initially receive infants with respiratory distress may not be adequately skilled to differentiate between mild, moderate and severe respiratory symptoms, which may contribute to poor management and outcome [6]. Untreated or unsupported pediatric respiratory distress can lead to respiratory failure, identified as the main cause of cardiac arrest in children [7]. Continuous positive airway pressure (CPAP) can help to prevent progression to respiratory failure [3]. However, identifying those children who need admission and those that need higher levels of care is dependent upon HCPs with the necessary skills and tools.

Risk prediction models offer the potential to support such clinical decision making, and in respiratory disease can help to identify children that require admission and respiratory support [8]. One such model is a clinical measurement instrument consisting of clinical signs and symptoms that are grouped together to measure an intended construct. The validation of such an instrument should ideally include: assessment of reliability, responsiveness, and usability [9-15]. Instruments could be used by HCPs in their decision making, such as when to admit children, or when to escalate care of infants with respiratory distress. In order to be fit for purpose, such an instrument should be validated, reliable, quick to perform, and straightforward to interpret in the clinical context [15]. It should not involve complex measurements, descriptions or equipment.

There are several respiratory distress instruments from high-income countries (HIC), with many being developed specifically for single pathologies such as bronchiolitis [9, 15–33]. Several models have been developed in resource-limited settings, to assess for primary outcomes of mortality or antibiotic-treatment failure in children with severe bacterial pneumonia, namely; The Mamtani score from India, modified Respiratory Index of Severity in Children (RISC) from Kenya, RISC-Malawi and the original RISC Score from South Africa [15, 34–38]. However, the income and health provisions in these countries are broad, varying between low-income (LIC) Malawi, and upper-middle income (UMIC) South Africa.

The Liverpool Bronchiolitis Severity Score (LIBSS) and the Respiratory Syncytial Virus Network Scale (ReSViNET), are both bronchiolitis specific scores. LIBSS and ReSViNET have both previously been assessed for apparent validity (to develop the model) and internal validity in infants in developed countries with bronchiolitis symptoms [25, 28].

Objectives

This study sought to field-test the use of two scoring instruments (LIBSS and ReSViNET), assessing the severity of respiratory distress in a population of Rwandan infants (1–12 months) presenting to urban, tertiary, pediatric hospitals. Specifically, we assessed the usability, inter-rater reliability and internal consistency of the two instruments. The secondary aim was the description of the severity of respiratory distress in these infants and the treatments used.

Reasoning for choice of LIBSS and ReSViNET scores

Both the LIBSS and ReSViNET instruments share five parameters and employ clinical parameters assessed by HCPs [15]. The parameters of both scores are applicable to all respiratory diseases that can cause respiratory distress in infants in settings such as Rwanda, and can be undertaken with no equipment (ReSViNET) or with just a saturation monitor (LIBSS). Specifically we sought to continue the work of the LIBSS and ReSViNET teams to see if these scores could be used in our setting, without requiring an assessment of HIV, malaria or chronic nutritional status, which are required for RISC scores.

Materials and methods

Study design

This was a cross-sectional, multi-centre, validation study, which took place from September 2018 to February 2019. Reporting of this study has been verified in accordance with the TRIPOD checklist for reporting prediction models [39, 40].

Study sites (data source)

The principal investigator (BH) undertook the study to complete his MMed pediatric residency at the University of Rwanda. The study was therefore conducted at four public university, or university affiliated hospitals in Rwanda, namely; Kigali University Teaching hospital (CHUK), Rwanda Military Hospital (RMH), Butare University Teaching hospital (CHUB) and Ruhengeri Referral Hospital (RRH). Recruitment was planned but unsuccessful at King Faisal Hospital (KFH), a collaborative public-private, university, tertiary hospital. CHUK, KFH and RMH are all located in Kigali city, the capital of Rwanda. CHUB and RRH are in provincial towns. All hospitals are tertiary referral centres. These sites are all located in urban settings but receive patients from both rural and urban settings. Patients using the national health insurance system (“mutuelle de sante”) cannot self-refer to tertiary hospitals and are therefore referred from Health Centers and District Hospitals. A small number of paying patients will self-refer.

Study population

Participants were recruited prospectively from children in both outpatient departments and the pediatric emergency rooms (ER) at the study sites.

Inclusion criteria

infants 1–12 months of age presenting with respiratory distress due to any respiratory illness and whose parents could give consent [41, 42].

Our case definition of symptoms and signs indicative of respiratory distress were: apnea, subcostal or intercostal recession, tracheal tug, nasal flaring, head bobbing, grunting, cyanosis, oxygen desaturation, tachypnea, wheezing, stridor, oxygen requirement, or reduced air-entry [25, 28].

Exclusion criteria

infants with known chronic lung disease (CLD), or infants who presented with a non-respiratory cause of respiratory distress (e.g. cardiac disease).

Sampling

recruitment was opportunistic at the study sites, with infants presenting to the study sites and meeting the case definition of respiratory distress being approached for recruitment.

Patient recruitment

The resident pediatrician and/or resident on duty identified eligible infants. Parents were provided with both verbal and written information about the study. If they agreed to participate in the study, written informed consent was gained and demographic details collected.

Clinical care of infants

All data-collectors were HCPs and were involved in the clinical care of the participants, and so were not blind to the infants’ condition, treatment interventions and patient outcomes. If an infant required stabilization this was done before recruitment and severity scoring. These interventions were administration of oxygen, antibiotics and fluids.

Data collection tool

Five data collection tools were used (https://doi.org/10.7910/DVN/N4O05G):

Unique Patient Identifier Sheet

Study-specific questionnaire: Patient demographics guided by the Demographics and Health Survey (DHS) parameters [41].

LIBSS: A validated score for use in children with bronchiolitis between the ages of 0–12 months. LIBSS has two scoring systems 0–3 months score and 3–12 months to take into account for age-dependent vital signs [28]. The LIBSS includes ten parameters, namely; General condition; Apnea; Increased work of breathing; Sa02; Respiratory Rate; Appearance; Heart Rate; Feeding; Urine output; Capillary refill time.

ReSViNET: a validated clinical severity scale that was developed to assess the severity of illness in infants with bronchiolitis [25, 43]. ReSViNET includes seven parameters, namely; Feeding intolerance; Medical intervention; Respiratory difficulty; Respiratory rate; Apnea; General condition; Fever

Study-specific follow-up questionnaire for final outcomes: admission to pediatric ward; admission to high dependency unit (HDU) or pediatric intensive care unit (PICU); or outpatient treatment. This section of the questionnaire was specifically designed for the purposes of this study.

Instrument translation

Most HCPs in Rwanda speak Kinyarwanda (local language), English and/or French. However, English may not be the first language. Therefore, to overcome language-related barriers, LIBSS and ReSViNET were translated into Kinyarwanda by the principal investigator (PI) and then back-translated for accuracy by a native-Kinyarwanda speaking pediatric resident. Translation discrepancies were reviewed with a third native-Kinyarwanda speaking pediatric resident for consensus. Within the scoring instruments, the Kinyarwanda translation was presented alongside the original English.

Instrument availability

The Translated LIBSS and ReSViNET, used in this study, are available online (https://doi.org/10.7910/DVN/N4O05G). The original ReSVinet was published in PLoS and is therefore available under Creative Commons licence CC BY [25] and the original LIBSS is available under a CC BY attribution [28].

Data-collection

One pediatric resident and one nurse independently assessed each infant using LIBSS and ReSViNET. The resident and nurse were instructed to not share their LIBSS/ReSViNET findings with each other. Infants were recruited and scored within an hour of presentation to hospital.

Training data-collectors/assessors

Prior to undertaking assessments, HCPs were given three hours of training on two consecutive days on how to use the two scores and questionnaires.

Sample size

Data was collected on paper forms, entered into Microsoft Excel and analyzed using SPSS version 24. The ’rule of thumb’ to determine the sample size for a clinical field-test is 10–15 times the number of parameters in a test [44]. This gave a minimum sample size of 100 for LIBSS (10 parameters) and 70 for ReSViNET (7 parameters).

Outcomes, data management and statistical methods

Severity of disease

As there is no “gold standard” for determining the severity of respiratory distress, clinical severity was defined by outpatient management (mild), admission (moderate) and severe (HDU/PICI admission). We determined the severity of illness using the LIBSS and ReSViNET pre-defined cut-points of Mild (0–10); Moderate (11–20); Severe (>21) for LIBSS [28] and 0–6 for mild affection, 7–13 for moderate distress, and 14–20 using ReSViNET [25, 43].

Validity

Criterion and convergent validity were used to assess whether the instrument measures what it was intended to measure. Criterion validity (predictive) testing was determined using the area under aROC based on severity scores of nurse-assessed LIBSS and ReSViNET. aROCs of 0.50–0.70, 0.70–0.90 and >0.90 were pre-defined as indicating low, moderate and high validity respectively [44]. Construct validity was assessed by examining the correlation between LIBSS and other measures of the impact of the disease: hospital admission, length of stay, HDU/PICU admission and mortality. Length of stay was categorised into binary criteria of long/short dependent on the median length of stay as a cut-off.

The two scores use 12 different parameters. Five parameters are shared by the two tools, namely: Apnea; Feeding intolerance; General condition; Increased work of breathing; and respiratory rate. ReSViNET uses two additional parameters (Fever and medical intervention), and LIBSS employs five unique parameters (Appearance, Capillary refil time, heart rate, oxygen requirement and urine output). Therefore, convergent validity of LIBSS and ReSViNET scores was assessed using Pearson’s r correlation coefficient. Coefficients of 0.4–0.59, 0.6–0.79 and 0.8–1.0 indicating moderate, strong and very strong relationships respectively [45].

Reliability testing

Inter-rater reliability (agreement) was evaluated between paired pediatric resident and nurse responses using the Intraclass Correlation Co-efficiency (ICC) with ICC <0.75 poor to moderate, >0.75 is good, >0.9 is excellent [44, 46]. Internal consistency of the parameters within each score was measured using Cronbach alpha, whereby Cronbach’s: <0.70 poor, >0.70 good (if <7 items), interpretation is dependent on number of parameters [44].

Ethical considerations and declarations

Financial disclosure

The author(s) sought nor received any specific funding for this work.

Potential conflict of interest

The authors have declared that no competing interests exist.

Confidentiality

Unique Individual Patient Identifier codes were used when completing the questionnaires. Patient-identifiable information was kept in a separate database to protect confidentiality.

Informed consent

Informed, written consent, was gained from parents or caregivers.

Incentives for subjects

No incentives, financial or otherwise, were offered to participants.

Risk to subjects (including safeguards to mitigate these risks)

No significant physical, social, emotional, legal or financial risks were identified.

Ethical approval

The project proposal was reviewed by the Institutional Review Board (IRB) at the University of Rwanda. Ethical approval was given (Ref: NO237/CMHSIRB/2018). The proposal was then reviewed and approved by the ethics committees of each enrolling site.

Availability of data and materials: The study data-set is available online at https://doi.org/10.7910/DVN/N4O05G/AKSNOZ.

Results

Recruitment

A total of n = 107 eligible infants were recruited (Fig 1): CHUK (n = 47); RRH (n = 46), CHUB (n = 9) and RMH (n = 5). Data from seven infants could not be used due to data collection being incomplete or inadequate, and therefore, these cases were removed. Data most commonly omitted were the heart rate and respiratory rate.

Fig 1

CONSORT diagram.

Data quality

No data points were missing from the final data set.

Demographic characteristics of the participants

Mean age of participants was seven months (201 days, Standard deviation, SD ±114.7), and 63% were male. Mean weight and median length of hospital stay (LoS) were 6.6kg (SD±2.4) and 4.0 days (min = 0, max = 39) respectively (Table 1). Participants were diagnosed with pneumonia (n = 51, 51%), bronchiolitis (n = 36, 36%), viral induced wheezing (n = 6, 6%), upper respiratory tract infection (n = 1, 1%) and other infectious respiratory illnesses (n = 6, 6%). Survival rate was 94% (n = 94). Twenty-eight patients required HDU or PICU management, with 7% (n = 7) requiring outpatient management alone. The most common co-morbidities were: malnutrition (n = 12, 12%), prematurity (n = 5, 5%) and HIV (n = 2, 2%) (Table 1).

Table 1

Demographic characteristics of the participants.

Characteristic
Gender
Male	63 (63%)
Female	37 (37%)
Mean age (days)	201 (SD ±114.73)
Median age (days)	204
Mean weight (kg)	6.6 (SD ± 2.43)
Median weight (kg)	6.0
Social Group (Ubedehe U )
High (3 & 4)	47 (47%)
Low (1 & 2)	53 (53%)
Residence
Urban	32 (32%)
Rural	68 (68%)
Living sibling
No	20 (20%)
Yes	80 (80%)
Vaccinations complete
No	0 (0%)
Yes	100 (100%)
Maternal marital status
Married	84 (84%)
Single, divorced, Widowed	16 (16%)
Maternal age
Young (<25 years)	32 (32%)
Old (≥25years)	68 (68%)
Maternal occupation
Unemployed	19 (19%)
Manual labourer	64 (64%)
Professional	17 (17%)
Maternal educational level E
High	49 (49%)
Low	51 (51%)
Final diagnosis
Pneumonia	51 (51%)
Bronchiolitis	36 (36%)
Viral induced wheezing	6 (6%)
Upper respiratory tract infection	1 (1%)
Other	6 (6%)
Co-morbidities
Malnutrition	12 (12%)
Prematurity	5 (5%)
HIV positive	2 (2%)
Management
Outpatient	7 (7%)
Pediatric ward	65 (65%)
HDU/PICU	28 (28%)
Median length of stay (days)	4.0
Mean length of stay (days)	6.5 (SD ±7.14)
Mortality rate/Died
Yes	6 (6%)
No	94 (94%)

UUbedehe is the Rwandan community based social classification;

E High = secondary or university completed, Low = primary or no formal education.

Severity of disease

ReSViNET described fewer infants (n = 16, 16%) as having severe disease compared to LIBSS (n = 33, 33%) (Table 2). Most patients presented in moderate or severe respiratory distress with 87% (n = 87) and 84% (n = 84) having moderate/severe disease on LIBSS and ReSViNET respectively.

Table 2

Severity of distress LIBSS and ReSViNET.

		Mild	Moderate	Severe
LIBSS	Residents	14	48	38
	Nurses	13	54	33
ReSVinet	Residents	16	67	17
	Nurses	16	68	16

Validity

Convergent validity

The Pearson’s correlation between ReSViNET and LIBSS for residents (R = 0.815) and nurses (R = 0.836) were both very strong (Table 3 and Fig 2) [45].

Table 3

Validity and reliability results.

		LIBSS	ReSVinet
Validity statistics
Convergent validity (LIBSS versus ReSViNET)	Pearson’s correlation (resident)	R = 0.815 (CI: 0.70–0.93) (p<0.001)
	Pearson’s correlation (nurse)	R = 0.836 (CI: 0.73–0.95) (p<0.001)
Criterion Validity for Hospital admission	aROC (nurse)	0.956 (CI: 0.88–1.0) (p<0.001)	0.973 (CI: 0.94–1.0) (p<0.001)
	aROC (resident)	0.955 (CI: 0.87–1.00) (p<0.001)	0.956 (CI: 0.92–0.99) (p<0.001)
Criterion Validity for HDU/PICU	aROC (nurse)	0.956 (CI: 0.92–0.99) (p<0.001)	0.880 (CI: 0.80–0.96) (p<0.001)
	aROC (resident)	0.951 (CI: 0.91–0.99) (p<0.001)	0.872 (CI: 0.787–0.957) (p<0.001)
Criterion Validity for mortality	aROC (nurse)	0.976 (CI: 0.95–1.0) (p<0.001)	0.974 (CI: 0.944–1.0) (p<0.001)
	aROC (resident)	0.974 (CI: 0.94–1.0) (p<0.001)	0.980 (CI: 0.954–1.0) (p<0.001)
Criterion Validity for Length of hospital stay	aROC (nurse)	0.718 (CI: 0.62–0.82) (p<0.001)	0.637 (CI: 0.531–0.747) (p<0.001)
	aROC (resident)	0.722 (CI: 0.62–0. 82) (p<0.001)	0.639 (CI: 0.531–0.747) (p<0.001)
Reliability statistics
Inter-rater reliability	Intra-class correlation (Nurse to resident)	0.985 (CI: 0.98–0.99) (SD±16.741) (p<0.001)	0.980 (CI: 0.97–0.99) (SD±6.899) (p<0.001)

HDU = High dependency unit; PICU = Pediatric Intensive Care Unit.

Suggested Interpretation of Validity and Reliability statistics.

aROC (area under Receiver Operating Characteristic): 0.50 = no different than random (i.e. useless), 0.50–0.70 low; 0.70–0.90 moderate, >0.90 high [44].

Intra-class correlation (ICC): <0.75 poor to moderate, >0.75 is good, >0.9 is excellent [44, 46].

Pearson R correlation: R = 0–0.19 very weak, R = 0.2–0.39 weak, R = 0.40–0.59 moderate, R = 0.6–0.79 strong and R = 0.8–1 very strong correlation [45].

Fig 2

Convergent validity.

footnote: R = Pearson’s coefficient, ICC = Intra-class correlation.

Criterion validity (predictive)

Both LIBSS and ReSViNET performed well for predicting hospital admission, HDU/PICU admission and mortality (Table 4). However, they performed only moderately well for predicting prolonged length of stay (Fig 3).

Table 4

Internal reliability (internal consistency) of LIBSS & ReSViNET.

	Parameters	LIBSS		ReSVinet
		NURSE	RESIDENTS	NURSE	RESIDENTS
Overall Scale Cronbach		0.831	0.823	0.850	0.848
Reliability if item deleted from scale
LIBSS only	Appearance	0.804	0.799	-	-
	Central capillary refill time	0.820	0.810	-	-
	Heart rate	0.826	0.822
	Oxygen requirement	0.809	0.796	-	-
	Urine output	0.804	0.801	-	-
Shared parameters	Apnea	0.833	0.824	0.870	0.864
	Feeding	0.804	0.789	0.810	0.817
	General condition	0.810	0.808	0.823	0.812
	Increased work of breathing	0.809	0.801	0.811	0.808
	Respiratory rate	0.831	0.821	0.812	0.810
ReSVinet only	Fever	-	-	0.844	0.843
	Medical intervention	-	-	0.829	0.825

Suggested Interpretation of Validity and Reliability statistics: Cronbach’s: <0.70 poor, >0.70 good (if <7 items), interpretation is dependent on number of parameters [44].

Fig 3

Criterion validity (aROC) of nurse (green) and resident (blue) performed LIBSS and ReSVinet.

footnote: aROC = area under Receiver Operating Characteristic.

Reliability

Interrater reliability

The inter-rater reliability between residents and nurses was excellent for both LIBSS (ICC = 0.985) and ReSViNET (ICC = 0.980) (Table 3).

Internal reliability (consistency)

Internal consistency for both scores was good with marginally higher internal consistency for data from the ReSViNET score (Cronbach = 0.850 and 0.848 for nurse and resident scoring respectively) (Table 4). Table 4 demonstrates the reliability of each score if each item is deleted from the score, with only the removal of apnea resulting a modest increased reliability in both scores.

Treatment

Most participants were treated with antibiotics (pneumonia 100%, bronchiolitis 72% and other infections 39%) and oxygen therapy (93%) (Table 5). In infants with bronchiolitis, non-standard therapy including adrenaline (47%), salbutamol nebulization (58%) and steroids (14%). In patients with severe disease, 5% required intubation and mechanical ventilation while 8% required CPAP (Table 5).

Table 5

Treatments used.

	Pneumonia (n = 51)	Bronchiolitis (n = 36)	Other (n = 13)
Antibiotic	51 (100%)	26 (72.2%)	5 (38.5%)
Oxygen therapy	50 (98%)	35 (97.2%)	8 (61.5%)
Salbutamol nebulisation	13 (25.5%)	21 (58.3%)	8 (61.5%)
CPAP (continuous positive airway pressure)	5 (9.8%)	3 (8.3%)	0 (0.0%)
Adrenaline nebulization	4 (7.8%)	17 (47%)	4 (30.8%)
Steroid administration	3 (5.9%)	5 (13.9%)	4 (30.8%)
Intubation and ventilation	1 (2%)	3 (8.3%)	1 (7.7%)

Discussion

This study sought to field-test the use of two scoring instruments (LIBSS and ReSViNET), assessing the severity of respiratory distress in a population of 100 Rwandan infants (1–12 months) consulting urban, tertiary, pediatric hospitals. The predictive validity, reliability between raters, and the internal consistency of the two instruments was measured and both instruments performed well.

The majority of patients had moderate or severe disease. This likely represents the location of the field testing and the health structure in Rwanda. Children cannot self-present to tertiary sites and are therefore referred from Health Centers and District Hospitals, where they will likely have provided the necessary care of infants with mild disease, without referring to the tertiary hospitals. The pre-established cut-points of ReSViNET identified fewer infants as having severe disease.

Treatments used

Though assessing severity is important, this study also highlights the importance of the implementation of evidence into clinical practice to ensure that evidence-based treatments are employed. Many patients received unnecessary treatments (Table 5). Antibiotics, adrenaline and/or bronchodilators or not warranted in bronchiolitis but were used frequently in children with this condition and discontinuing the use of these three interventions as a key priority [47-49]. Not only is the efficacy of these medications not backed up by the evidence in the literature, they are also costly for the health care system and families. Further work needs to be undertaken into how to reduce the use of these ineffective treatments. This is not a Rwandan-specific issue, and this problem has also been described in developed [50].

Validity of the scores

Many healthcare facilities in resource-limited settings will not have medical doctors. Therefore, the nurse-assessed LIBSS and ReSViNET were used for the validity analyses. There is no “gold-standard” for assessing respiratory disease. When LIBSS and ReSViNET were measured against each other, there was a strong correlation between the two scores (Pearson’s >0.8). The scores use 12 parameters, and as they share five of parameters (apnoea, feeding intolerance, general condition and respiratory rate), the strong correlation is perhaps not surprising, but it is reassuring that they are scoring severity similarly. Both scores performed highly for predicting hospital admission, HDU/PICU admission and mortality. HDU/ICU are not infrequently needed to optimize respiratory and medical support, especially in sub-Saharan Africa [2], however, such care may require the transfer of the patient, is expensive and labour-intensive. Identifying the right patients for this level of care is, therefore, important. The case-definition used in our inclusion criteria overlapped with the scoring systems that were measured. Therefore, when interpreting the validity of the instruments it is important to consider that the sample taken were done so using this case-definition and therefore it is feasible that the tools will perform very differently in unselected children and in primary care levels.

The inter-rater reliability was good, with lower confidence limits of 0.98 and 0.97 for LIBSS and ReSViNET, whereby scoring >0.9 is “excellent” [44, 46]. In the UK, field-testing of LIBSS, the lowest confidence interval for ICC was 0.75 [28], and the ReSViNET gained lowest ICCs of 0.76 between professionals [25]. It is therefore interesting that the Rwandan professionals scored considerably higher than both these original settings.

Internal consistency

The consistency was better within the ReSViNET, and this is despite it having fewer parameters (seven versus ten). It is well known that some items may particularly affect reliability within the data and removing this item could see an improvement in internal consistency, at the cost of content validity. Our data revealed good internal reliability and only the removal of the measurement of apnoea would have marginally improved the internal reliability. As Apnoea is an important aspect of respiratory illness in infants (content validity) we would not advocate removing it.

Use of LIBSS and ReSViNET in the Rwandan setting

The most common co-morbidities were: malnutrition (12%), prematurity (5%) and HIV (2%) (Table 2). According to the Rwandan DHS 2015 data, 38% of under-5 children are stunted where 18% of infants between 6 to 8 months are stunted [51]. RISC requires and assessment of nutritional status as these infants are particularly prone to worse outcomes. Therefore, using a respiratory distress score including HIV and/or nutritional assessment is likely to be beneficial. However, doing so requires additional skills (nutritional assessment) which can add to the complexity of the scoring.

Comparison of the population with original tools

The major consideration here is that the original scoring tools were designed for infants from resource-rich settings and for infants with bronchiolitis alone, with other respiratory illnesses being assessed. The cut-points for disease severity were therefore determined in a different setting with cohorts of patients very different from ours. These factors may account for the lower criterion validity for HDU/PICU admission with ReSViNET (aROC 0.880) compared to LIBSS (0.956).

Use of valid and reliable clinical scores in research

As well as using scores in clinical practice severity scores are also useful for clinical trials, in order to consistently and reliably assess severity of diseases and the effects of treatments. Both the scores described here are simple to use, without complex investigations or assessments, and have the potential to be used in research activities in this setting and age group.

Bias

Our population of outpatient-managed cases was only small (n = 7), probably reflecting the type of hospitals where assessment was undertaken. Usually tertiary and/or referral hospitals in Rwanda receive critically ill patients transferred from peripheral hospitals. Due to the scarcity of hospital beds, priority is given to the most critically ill patients. But self-referred patients with private financial capacity can consult as outpatients. Therefore our population was biased to either tertiary, referred patients or private self-referring patients.

Limitations

Limitations of our study include the lack of a gold standard for the evaluation of children with acute respiratory distress and the lack of a mechanism to have an ongoing follow-up. The other major limitation was our inability to hire full-time data collector nurses at each study site. This would have potentially increased recruitment and reduced incomplete datasets that led to seven cases being excluded. We attempted to recruit at a University public-private hospital, King Faisal Hospital (KFH), a public-private, but only recruited three cases, which was felt insufficient for inclusion. The reason for poor recruitment was high perceived work load and reported low admissions of patients with respiratory distress.

The sample size was suitable for field-testing the severity scores and to assess if they are feasible for use. There is however a risk of over-fitting in relation to the prediction measures (aROCs). The tests we used were created for assessing bronchiolitis rather than all causes of respiratory illness. They do not assess for HIV, malaria or chronic nutritional status, and these may have important prognostic implications that require additional resources and HCP skills, therefore, may impede utilization both in clinical and research applications.

Next steps

In this study, it was not feasible to undertake repeated assessments, and therefore, responsiveness was not assessed. Testing responsiveness could have allowed us to monitor progress in hospital and see if the scores were helpful to identify deterioration or improvement and guide care. Applying evidence into practice is challenging. These two scores have not been applied in Rwanda and doing so would require additional work, such as assessing stakeholder attitudes, along with addressing training needs of those using scores in primary care facilities. When LIBSS and ReSViNET were measured against each other, there was a strong correlation between the two scores (Pearson’s >0.8). However, ReSViNET described fewer infants (n = 16, 16%) as having severe disease compared to LIBSS (n = 33, 33%) (Table 2). Therefore a larger study to identify appropriate cut-off points, along with responsiveness, would be an important piece of work to undertake.

Conclusion

The findings of this study are important to the Rwandan health system where facilities have limited resources, and the decision to admit and escalate care must be carefully considered. This early data demonstrate that these two scores have the potential to be used in conjunction with clinical reasoning to identify infants at increased risk of clinical deterioration and allow timely admission, treatment escalation and therefore support resource allocation in Rwanda.

TRIPOD checklist for ReSVinet LIBSS field testing.

(DOCX)

Click here for additional data file.

(DOCX)

Click here for additional data file.

(DOCX)

Click here for additional data file.

(DOCX)

Click here for additional data file.

2 Sep 2020

PONE-D-20-20016

Field testing two existing standardised respiratory severity scores (LIBSS and ReSViNET) in infants presenting with acute respiratory illness to tertiary hospitals in Rwanda – a validation and inter-rater reliability study

PLOS ONE

Dear Dr. Cartledge,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has some merit but requires substantial revision to meet PLOS ONE’s publication criteria. Therefore, we invite you to submit a revised version of the manuscript that addresses all the points raised during the review process.

Please submit your revised manuscript by Oct 17 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Brenda M. Morrow, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Overall comments

The paper could benefit from English language editing. It is hard to follow and there are a number of grammatical errors. The main objective of the study is to assess two scores of respiratory distress from HIC one exclusively for bronchiolitis in infants. There is no clear motivation as to why the authors chose not to compare with other tools from LMIC settings taking into account high levels of malnutrition in their population. The patient selection and the motivation for item selection is not clearly articulated. There is no clear impression given to the context of the study. The inclusion and exclusion criteria and limited and the methodology is lacking in defining a number of variables that have been discussed in the document. There is no explanation why some sites failed to recruit any participants. There major limitation is the lack of uniformity of disease classification, 13% of study sample mentioned as "other". The representation of the data requires major revisions and the discussion is limited in its academic discussion of the findings and does not seek to address the primary objective and secondary objective.

Reviewer #2: This is an important research question that is of paramount importance in the care of children in low and middle income countries. There are a few fundamental issues that the authors need to consider in the methods and results section that I have pointed out. I have attached a detailed document with these comments...

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Reviewer comments- PLoS one 20-20016.docx

Click here for additional data file.

Submitted filename: Field testing two existing standardised respiratory severity scores.docx

Click here for additional data file.

29 Jan 2021

Editorial team comments

We noticed minor instances of text overlap with the following previous publication(s), which need to be addressed:

(1) https://academic.oup.com/tropej/article-abstract/66/2/234/5570307?redirectedFrom=fulltext

The text that needs to be addressed involves the Introduction section (lines 90-93).

-Response: Thanks for pointing this out. We have put in alphabetical order of country and add our own paper reference.

Please amend either the title on the online submission form (via Edit Submission) or the title in the manuscript so that they are identical.

-Response: Done

Comments to the Author

Reviewer #1: Overall comments

The paper could benefit from English language editing. It is hard to follow and there are a number of grammatical errors.

-Response: We are native English speakers and have used Grammarly to finish off the language. I’m not sure what else we can do, without having knowledge of the specific sections that are difficult to understand.

The main objective of the study is to assess two scores of respiratory distress from HIC one exclusively for bronchiolitis in infants. There is no clear motivation as to why the authors chose not to compare with other tools from LMIC settings taking into account high levels of malnutrition in their population.

-Response: We have moved around the introduction, which we had hoped explained their use. We have and created a new paragraph after the objectives to give an exaplanation of the reasoning for their use. This paragraph can be moved to the methods section if it is felt to be more appropriate.

The patient selection and the motivation for item selection is not clearly articulated.

-Response: Regarding patient selection, it was opportunistic based on the case-definition of “respiratory distress”. We have added this below the inclusion criteria under “sampling”. Regarding the item selection, we were limited to the items in the LIBSS and ReSViNET which we didn’t modify.

There is no clear impression given to the context of the study.

-Response: We are not sure what the reviewer would want in terms of context that is not described in the introduction?

The inclusion and exclusion criteria are limited

-Response: we’ve reworded this section, but these were the criteria we used, so there really isn’t much else to add.

The methodology is lacking in defining a number of variables that have been discussed in the document.

-Response: all the variables are described in the results and the questionnaires are available at the DOI. Therefore we didn’t want to add repetition.

There is no explanation why some sites failed to recruit any participants.

-Response: We’ve added this into the limitations.

There major limitation is the lack of uniformity of disease classification, 13% of study sample mentioned as "other".

-Response: we have split the other down to include the viral induced wheezing and URTI

The representation of the data requires major revisions and the discussion is limited in its academic discussion of the findings and does not seek to address the primary objective and secondary objective.

-Response: Each of the objectives is described in the discussion with a subheading. We would have welcomed the opportunity to go deeper into discussion, however to maintain a balanced and appropriate word count we have given a description that describes the findings in terms of the study population and a brief synopsis of it’s comparison with other studies. We have made general changes to the discussion and hopefully this is more satisfactory to the reviwere.

REVIEWER 1 COMMENTS (with responses)

Abstract: I would suggest that the ‘introduction’ contain some sort of rationale for the study

-Response: we have added an additional sentence, but fear that the abstract word length may now be too long

Introduction: Line 89. Please check if this is indeed LIC or if it is supposed to be LMIC’s. Please note that these mean different things. For example, while South and Malawi are both LMICs, South Africa is Upper-Middle income, while Malawi is Low income…South Africa and India are not LIC

-Response: Yes, this is an oversight of the terminology. We have amended this and reworded the paragraph.

Materials And Methods

Line 131. So, to be clear, the children were selected on the basis of having respiratory distress. This would mean that the bases on which they were selected overlapped with the scoring system that will access the severity of their illness, right? This is important later given what the limitations are: the sample taken may not be appropriate to tell us how the tools will fare in detecting severe disease in unselected children (and in primary care levels, children arrive unselected).

-Response: yes, this is a very valid point. We have added a section with the “validity” commentary in the discussion, rather then in the limitations

Table 1. It may be useful to add columns that indicate the ranges of points for each item in each score.

-Response: we have come to realise that Table 1 isn’t that helpful, especially as many readers aren’t used to a Table in an introduction. We have therefore removed it. The scoring is all available in the tools which are available at the DOI.

Line 199. I have struggled to find these odd ratios in the paper. Did I miss them? Line 200. It is not clear as to what variables the model was adjusted for. What were these confounders? (I will return to this in the results section).

-Response: Sorry, this was a remnant text from the PIs MMed dissertation. We removed this analysis from the manuscript due to the length and number of tables already presented. We have therefore deleted this from the methodlogy now.

-Response:

Lines 207-209. I am not entirely sure that doing correlation of two scores that have such a large overlap of components makes any sense at all…. In fact one does not need to collect data to figure out that they will almost be guaranteed to be correlated (as long as apnoea means the same thing in both scores, the fact that scores are allocated slightly differently should have little impact on correlation). In fact the correlation might potentially be seen as almost colinear from basic principles. I can therefore accept correlation of either score to the same but independent outcome, but not to each other… I hope my discomfort with this analysis makes sense (ReSVinet=70% of LIBBS’s components while 50% is the reverse) .

-Response: We have added the following text to the statistics section of the methodology “The two scores use 12 different parameters. Five parameters are shared by the two tools, namely: Apnea; Feeding intolerance; General condition; Increased work of breathing; and respiratory rate. ReSViNET uses two additional parameters (Fever and medical intervention), and LIBSS employs five unique parameters (Appearance, Capillary refil time, heart rate, oxygen requirement and urine output). Therefore,…”

Would it not make sense to compare the severity of each score (a categorical analysis) to clinical severity as indicated in lines 193-194 and see how each ‘agree’ with this??

-Response: we did a criterion validity for these surrogate markers of severity (see Table 4 in new version)

Results:

NB. As the study is about the two scoring methods, I expected to see a summary of the scores generated from these two, then see how they perform against clinical severity as mentioned above

-Response: Is this related to the order we presented the results? We did think about this ourselves when writing the manuscript. We put the treatment quite early as we felt it provided more context on the practice of professionals at the study site. We have moved it down.

Lines 219-221: 107-8=99, however the data used in the rest of the paper as well as Fig 1 suggest that 7 (and not 8) were excluded.

-Response: yes, sorry, this is an error, can’t believe we missed that. Sorry.

Line 228 – SD used for the first time here. Suggest writing it in full at first mention.

-Response: amended

By the way, I would have to confess that it is rather unusual for a number of the continuous variables reported here to be ‘normally’ distributed enough to be acceptably summarised with means and SDs.

-Response: amended. We’ve also added into the statistical analysis on how we categorized length of stay using the median.

Were all these data tested for normality (either formally or graphically).

-Response: we reviewed them graphically

Specifically with age, my suggestion would be to summarise this using months rather than days. It is just easier for the reader to immediately comprehend the age of the cohort that way. Hardly do people measure children’s age as days outside the neonatal period.

-Response: amended, added months into the body of the text

NB. While, I really appreciate that for this cohort of 100 children, as long as we are referring to proportions of the total, the number will always equal the percentage, it is otherwise good practice to always report both number and percentage [n(%)] in the text. When part of the report includes subgroup proportions, not having numbers to which the percentages refer gets very confusing.

-Response: amended.

Table 2.

Table includes ‘social group’ which is undefined

-Response: amended, footnote added

‘Vaccination’ what does No and Yes mean?

-Response: amended, sorry

Maternal education: What does high and low mean??

-Response: amended, footnote added

‘Median length of stay’ but result indicate some sort of standard deviation??

-Response: amended

Table 3. By the way, the use of a Kappa statistic is coming here for the first time being absent in methods.

-Response: Sorry, this column should have been removed from the table. It is not needed as the ICC is described in Table 4.

Line 248. I think it should read Table 4 not 3

-Response: amended, now Table 6, see above

NB. When it comes to tables and figures, make sure all acronyms are explained in the figure/table legends. E.g. acronyms such as ‘ICC’ in figures..

-Response: amended

Table 4. What is the difference between the ‘others’ of “Bronchiolitis and others” and ‘Other’?

-Response: sorry, this is an error. The “and others” has been removed”

Table 5. Already alluded to in Methods, I am not sure what is adjusted for here

-Response: see above. The adjusted odds ratios were a separate analysis in the PIs dissertation.

Table 5 - Would it be possible to show the data for each binary outcome?

-Response: It would make very heavy tables, therefore we chose not to. Original data is available

Table 5 - Secondly, in the case of adjustment being done one would need to show both adjusted and unadjusted estimates

-Response: see above. The adjusted odds ratios were a separate analysis in the PIs dissertation.

Table 5 - Length of hospital stay: I can see the others working as ‘binary’ in a Yes-No gold standard to generate

-Response: Yes, we realise now we never described how we did this. We’ve added it to the statistical analysis section.

Some things in the table legend don’t seem to belong to it: Cronbach, SPR

-Response: amended

Discussion

My sense is that the section on treatment (Lines 300-314) is an unwarranted digression that important as it is has little to do with the theme of the paper. If there is a need for it, it really needs to be brief

-Response: We hadn’t initially put this in the “objectives” as the objectives statement was already quite long. We’ve added this back in. Also we have cut down this section in the discussion.

Lines 338-340. I can’t see anything in table 6 indicating the detrimental effect of apnoea to reliability. Maybe if there is anything of the sort the authors could highlight this in the text in the results section before discussing it here.

-Response: We’ve added it to the text above the table (which is now labelled Table 4)

Lines 343-345; Could the authors not have gone around this issue by doing stratified analysis with just bronchiolitis to see if they find results similar to those in HIC?

-Response: We didn’t do this.

Conclusion

Are the authors suggesting that the two scores yield similar results? It just seems to me that table 3 suggests that one score misses half of severe cases (or maybe one wrongly doubles the number of severe cases??) This is worrying and needs explanation, but can not be addressed unless the two scores were assessed against a single ‘gold’ standard as suggested previously rather than to each other.

-Response: We’ve added the following text to the “next steps” section: “When LIBSS and ReSViNET were measured against each other, there was a strong correlation between the two scores (Pearson's >0.8). However, ReSViNET described fewer infants (n=16, 16%) as having severe disease compared to LIBSS (n=33, 33%) (Table 2). Therefore a larger study to identify appropriate cut-off points, along with responsiveness, would be an important piece of work to undertake.”

REVIEWER 2 COMMENTS and Responses

The representation of the data requires major revisions and the discussion is limited in its academic discussion of the findings and does not seek to address the primary objective and secondary objective.

-Response: This is quite a broad feedback, therefore hopefully the points are addressed in the below amendments and responses.

The tables and figures are not comprehensively captioned and there are mistakes when the author makes references to some tables. There are two (Table 1s) in the text.

-Response: amended

The authors do not use the discussion to elaborate fully on their results they mention that findings are interesting and does try to elaborate, compare and make arguments for or against.

-Response: As discussed above, we are limited by the number of words and didn’t want the balance of the paper to fall mainly on the discussion. Most readers, in our opinion, can make their own conclusions and opinions without us going through each result in detail.

References are made to a PhD thesis that is not available in print, textbooks where pages and chapters are not mentioned, references to studies that were apparently done in the UK but when looked up are somewhere else, conclusive remarks supported by comparison to studies with different methodologies (compares apples with oranges).

-Response: The entire PhD is available freely as a DOI. If the reference to the study “apparently” done in the UK was the Griffiths, and Riphagen article then I can confirm it was done in the UK. Of course, the reviewer is right that comparing UK with Rwanda is challenging, but that is the nature of our paper, using a tool developed in UK (LIBSS) for it’s appropriateness in Rwanda.

The reporting of numbers is not the same everywhere at times uses 3 decimal points then uses 2, the results in abstract are not the same as those in the tables in results.

-Response: Yes, this is challenging. For the numbers <1 we tended to use three decimal places. We’ve amended the abstract to use 2 decimal places for the Cis.

The introduction is fair but for the discussion, the author has not tried to put create a followable argument, there are a lot of headings with paragraphs that do not have an introduction, body and conclusion, there are just thoughts put under a heading. So the gist of the article is lost.

-Response: We have made a number of changes to the discussion, but without the word count being significantly increased I think it is unlikely that this will have been fully addressed to the reviewers requirement.

The paper argues for use of a tool developed in resource-rich but does not explain why? e.g. What's wrong with ones developed in LMICs?

-Response: Amended, as per previous peer-reviewer comments

Also if it will be used to use in different resp illnesses it's no longer a validation study but an adaptation or modified tool. or further development of the tool. If the study compared its use in bronchiolitis only it would validate the tool. I am not convinced about the validation argument.

-Response: We were field testing it. We are aware that significantly more work will need to be done to fully validate it. As discussed in the “next steps” section.

Methodology

Bias: participant selection would be those with severe disease if no self-referral system- how can this be applicable to patients with mild or moderate disease

-Response: We have added a new paragraph called “bias” to the discussion and have discussed this within that section

How were the participants selected: this is not clear ??

-Response: Amended, as per reviewer 1 comments

Other relevant exclusions??

-Response: No other exclusions

Table 1; for ease of reading useful to have similar markers for both scores in same line so easy for reader to see which parameters are dissimilar. Also two tables named Table 1.

-Response: We’ve now removed Table 1. And we’ve added the similar markers into the text

There is no explanation of what is meant by Social Group (Ubedehe) High (3 & 4),Low (1 & 2), what levels of schooling etc for parents.

-Response: added into footnote of the tables

What vaccination was asked about, what is the EPI program in Rwanda?

-Response: Sorry, it was just whether they had received vaccines or not. We have added this to the table

Results

The final number stated is 100 but only 99 were analyzed as there were 107 and 8 exclusions? “from eight infants could not be used due to data collection being incomplete or inadequate”

-Response: This was an error. We’ve amended it. There were 100

Table 1 missing after characteristics no explanation of what data is presented

-Response: This “Table 1” has been removed and it’s contents moved into the body of the text.

Table 1: 28 required HDU and ICU but 13 seem to have received interventions in HDU or PICU

-Response: We aren’t exactly sure what the point is that the reviewer is making. Sorry.

Lack of consistency with decimal places, confusing results and explanation of key in tables and figures

-Response: we have used three decimal places for numbers with a value less than 1. Except for confidence intervals, for which we used 2 decimal places. We have added additional information to the keys for tables and figures, as per reviewer 1.

Discussion

There is no logical flow to the discussion and this fails to give a detailed academic discussion on the results of the study and the context.

-Response: as above

Conclusion: considering the study size, variable disease profiles of children I think the conclusion is slightly over-reaching in the abstract, the conclusion in the main document more modest.

-Response: amended

8 Oct 2021

Field testing two existing, standardized respiratory severity scores (LIBSS and ReSViNET) in infants presenting with acute respiratory illness to tertiary hospitals in Rwanda – a validation and inter-rater reliability study

PONE-D-20-20016R1

Dear Dr. Cartledge,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Jamie Males

Staff Editor

PLOS ONE

Additional Editor Comments (optional):

We noticed minor instances of text overlap with the following previous publication(s), which need to be addressed:

(1) https://academic.oup.com/tropej/article-abstract/66/2/234/5570307?redirectedFrom=fulltext

The text that needs to be addressed involves the Introduction section (lines 90-93).

In your revision please ensure you cite all your sources (including your own works), and quote or rephrase any duplicated text outside the methods section. Further consideration is dependent on these concerns being addressed.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

26 Oct 2021

PONE-D-20-20016R1

Field testing two existing, standardized respiratory severity scores (LIBSS and ReSViNET) in infants presenting with acute respiratory illness to tertiary hospitals in Rwanda – a validation and inter-rater reliability study

Dear Dr. Cartledge:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr Jamie Males

Staff Editor

PLOS ONE