Monitoring Severity of Respiratory Syncytial Virus (RSV) in Infants and Young Children Using the Pediatric RSV Electronic Severity and Outcome Rating System (PRESORS): Results of Initial Quantitative Validation.

PURPOSE: PRESORS ClinRO completed by clinicians and ObsRO completed by caregivers were developed to characterize the clinical course of respiratory syncytial virus (RSV) infection. This study describes preliminary analysis of PRESORS' measurement properties using clinical trial data. PATIENTS AND METHODS: PRESORS ClinRO and ObsRO data were collected in a 28-day randomized, double-blind, Phase 1b trial of JNJ-53718678 or placebo in infants and children ≤24 months of age treated for RSV infection in hospitals. PRESORS data were scored and key psychometric properties of scores were evaluated, including ability to discriminate between known groups and to detect change over time. Time to resolution of RSV signs was explored using two responder definitions.
RESULTS: Daily completion rates for PRESORS ClinRO and ObsRO were high for the 44 children in the study (median: 100% and 93%, respectively). Large floor effects were observed at baseline for signs of severe RSV infection that were either absent (cyanosis, fever, apnea) or rarely reported (reduced urination/dehydration, vomiting). Implausible ObsRO ratings suggested some caregivers could not accurately measure heart rate. Known-group validity was confirmed: children in poor health based on baseline ClinRO had mean baseline composite scores that were significantly worse for both ObsRO (p=0.001) and ClinRO (p<0.001) compared to those with better overall health. ObsRO (p=0.009) and ClinRO (p<0.001) composite scores were responsive to change in overall health status from baseline to Day 3. Mean scores for RSV sign dimensions decreased rapidly from baseline to Day 7 except for coughing and sleep ratings by caregivers. Time to recovery varied greatly depending on definitions used.
CONCLUSION: PRESORS ClinRO and ObsRO can inform endpoints and enable monitoring the clinical course of RSV in pediatric trials. Improved alignment between ClinRO and ObsRO and revisions ensuring caregivers can assess all signs will be addressed in revised PRESORS.

Introduction

Respiratory Syncytial Virus (RSV) is a seasonal disease responsible for respiratory tract infection that can require hospitalization, particularly in very young children, the elderly, those with compromised immune function, cardiac, or pulmonary conditions.1–5 From 1993 to 2008, the total RSV hospitalization rate in the United States across all age groups was 55 per 100,000 person-years, but the rate was significantly higher for infants (2345 per 100,000 person-years).6 Therapeutic solutions for RSV are very limited.7 New vaccines are in development to prevent RSV disease as are new treatments to limit the severity and duration of RSV disease in infants and young children.

Currently, there are no published standardized assessments for clinical trial endpoints that allow both clinicians and parents (or other caregivers) to monitor the full clinical course of RSV-related illness. Clinicians routinely monitor vital signs such as heart rate, respiration rate, and blood oxygen levels, but these do not fully characterize the severity of RSV-related disease or its clinical course; moreover, they can only be assessed when the child is attended by the clinician. Endpoints based on clinical outcomes assessments (COA) by the child’s primary caregiver and the clinician are needed to monitor signs of RSV disease in clinical practice and at home. Clinicians rely on the caregiver to provide information on whether the child is not eating, sleeping, behaving, or appearing as usual and how they evolved to evaluate what interventions are needed. Having effective tools for caregivers and clinicians to share information about the child’s illness and recovery may help assess new treatments that can lessen severity and shorten the duration of illness in infants and young children with RSV infection.

There is no consensus on how to quantify severity or monitor the clinical course of RSV-related illness.8–10 As illness due to RSV infection is a common cause of hospitalization in infants, medical guidelines and hospital protocols specify procedures and supportive care required to manage or prevent severe signs of RSV-related illness for infants and young children. These vary from setting to setting, but most focus on identifying and managing signs of respiratory distress, dehydration, and respiratory failure that are the focus of medical intervention in hospitals. They do not systematically characterize all the clinical signs present at admission or throughout the clinical course to disease resolution that may be important for assessing treatment efficacy.

To support development of new RSV treatments, the Pediatric RSV Electronic Severity and Outcome Rating System (PRESORS) caregiver diary (an observer-reported outcome or “ObsRO” measure) was created using recommended best practices for the development of clinical outcomes assessments as clinical trial endpoints.7 An initial item pool was developed based on the medical literature and consultations with clinicians and pediatric COA experts. A Clinician-reported outcome (ClinRO) measure was developed in parallel as a companion instrument to be used during hospitalization or outpatient visits. Once the initial ClinRO and ObsRO assessments were drafted, the reliability, validity, and ability to detect meaningful change in RSV severity with preliminary scores were tested using blinded data from a pediatric clinical trial of JNJ-53718678 (study 53718678RSV1005 - NCT02593851, later referred to as RSV1005).11 This report describes the psychometric analyses of the ObsRO and ClinRO data in RSV1005.

Materials and Methods

53718678RSV1005 Clinical Trial Design

RSV1005 was a Phase Ib, randomized, partially double-blind, placebo-controlled trial in infants and young children hospitalized with RSV infection. Details of the study design, sample, and primary study results were published by Martinón-Torres et al.11 Infants and young children (later referred to as children) of 1 to 24 months of age were followed for 28 days including a 7-day treatment period and a 21-day follow-up period. Per protocol children could be discharged from the hospital as early as Day 3 of treatment if deemed appropriate by the investigator. PRESORS ObsRO measure was translated using best practice standards for clinical outcomes assessment12 and implemented as electronic diary applications on smartphones. Site personnel and caregivers (ie, parents, step-parents, foster parents or grandparents of the children) completed training in how to complete PRESORS instruments on the electronic device prior to first use. The trial was conducted in accordance with Good Clinical Practice guidelines, the Declaration of Helsinki, and local regulatory requirements (the study protocol and procedures were reviewed and approved by relevant health authorities and ethics committees for each site). Written informed consent and parental permission were obtained for all children.

Data Collected

During the clinical trial, an electronic case report form (eCRF) was completed by clinicians for each day during the hospitalization phase and for each follow-up visit (Day 7, Day 14 and Day 28 visits).

PRESORS ObsRO was completed by caregivers once at baseline, three times each day (morning, afternoon, evening) from Day 1 to Day 14, and once each day in the evening from Day 15 to Day 28. Heart and respiration rates and body temperature were measured and reported by caregivers only in the evenings after hospital discharge through Day 14. The instructions to measure the respiration rate were:

Please count how many times the child breathes in and out in 15 seconds. Press the button to start the timer. When you hear the first tone, count each time the child breathes in and out. When you hear the second tone, stop counting and record how many breaths you counted.

This was multiplied by 4 to obtain the child’s breath per minute. Similarly, for the child heart beats, instructions were:

Please count the child’s heartbeats. Place your fingertips on the child as shown. When you feel the child’s heartbeat, press the Start button with your other hand. When you hear the first tone, count how many times you feel the child’s heartbeat. When you hear the second tone, stop counting and record the total number of heartbeats you counted.

This number corresponding to the number of beats in 15 seconds was multiplied by 4 to obtain the number of heart beats per minute.

PRESORS ClinRO was completed by clinicians once at baseline, twice a day during the hospitalization phase (morning and evening), and then once at each follow-up visit (ie, at Day 7, Day 14 and Day 28 visits).

Scoring of the PRESORS Instruments

The PRESORS assesses 12 ObsRO and 13 ClinRO signs that are scored to reflect severity based on recommendations by pediatric infectious disease specialists on a continuum from none/not present (0) to severe (3).13 Tables 1 and 2 provides details about how each item is scored for the ObsRO (Table 1) and ClinRO (Table 2). Most items are scored using an ordinal 3- or 4-level severity scale using the following: absent or none (0), mild (1), moderate (2), or severe (3). A few items are scored as either present or absent with the underlying severity of the sign scored as moderate (ObsRO and ClinRO “Fever”) or severe (ClinRO “Apnea” and “Cyanosis”) if present.13

Table 1

PRESORS ObsRO Dimension Scores

Dimension (PRESORS Item)	0 Points	1 Point	2 Points	3 Points
General Condition (16)	Excellent/Very good	Good	Fair	Poor/Very poor
Activity Level (1)[a]	Alert, normal	A little less active than usual	Much less active than usual	–
Sleep problems (3) [a]	Slept as normally does	–	Slept a lot less than usual	Slept a lot more than usual
Crying (4)	Normal, no more fussing or crying than usual	Fussier than usual or cried a little more than usual but calmed if held or soothed	Cried a lot, difficult to calm even if held of soothed	Cried a lot, would not stop crying even if held or soothed
Wheezing (5)	No	Yes, only at the end of when the child breathes out	Yes, throughout breathing out	Yes, throughout breathing in and out
Breathing problems: other signs (6)	No	Unable to breathe through stuffy or runny nose	–	Gasping for air/ Long pauses between breaths
Cyanosis (lips, skin, fingernails pale or blue) (7)	No	–	Yes, but only when the child was crying	Yes, even when the child had not been crying
Coughing (8)	No coughing	A little coughing	Coughing a lot	Coughing almost all the time
Vomiting (9)	No	Yes, only when coughing	Yes, not only when coughing	–
Feeding problems (10)	Yes	No, ate/nursed a little less than usual	No, ate/nursed a lot less than usual	No, did not eat or nurse at all (Q10) OR Fed only by tubes in the nose or veins (Q10)
Dehydration/Urination (12)	Yes	No, a little less than usual	No, a lot less than usual	No, did not wet a diaper or use the toilet
Fever (13)	<38°C	–	≥38°C	–
Heart rate[b] (15)	Inside normal age range	Outside normal age range but not inside abnormal range	–	Inside abnormal age range
Respiration rate (14)[c]	Inside normal age range	Outside normal age range but not inside abnormal range	–	Inside abnormal age range

Note: [a]ObsRO Q1 “Activity level” response 4 “Sleeping the entire time” will lead to a missing “Activity level” and “Sleep problems” dimension score. [b]“Heart rate” was finally not included in the ObsRO scoring due to suspected reliability issues with these data. [c]Normal and abnormal age ranges for respiration rate and heart rate originate from the clinical trial protocol and are displayed in Appendix.

Table 2

PRESORS ClinRO Dimension Scores

Dimension (PRESORS Item)	0 Points	1 Point	2 Points	3 Points
Activity level (1)	Alert, active	Irritable/ restless/ agitated	Listless, lethargic	Only responds to pain/ Comatose
Sleep problems (2)	Normal	Occasional restlessness/ disturbed	Restless/ disturbed much of time	Comatose
Feeding Problems (8)	≥ 75% of normal feeding via usual route	50–74% of normal feeding via usual route	< 50% of normal feeding via usual route OR Nasogastric tube OR Intravenous	–
Any Retractions (3)	No	Yes	Yes	Yes
--Intercostal retractions (3.1)		None	Mild/Moderate/Severe	Any
--Tracheosternal retractions (3.2)		None	None	Mild/Moderate/Severe
Breathing problems: other signs (3.3)	None of the above	Nasal flaring OR grunting	Head bobbing OR both nasal flaring and grunting	Head bobbing AND at least one of nasal flaring or grunting
Cyanosis (3.3)	No central cyanosis (blue lips/tongue)	-	-	Central cyanosis (blue lips/tongue)
Coughing (5) [a]	Little or no coughing	Occasional strong cough, sometimes productive	Frequent cough, sometimes causing choking, gagging, or vomiting	-
Excess Nasal Secretions (6)	None, or Minimal, easily cleared with suctioning	Moderate, but could be cleared with suctioning	Extensive, could not be completely cleared with suctioning	-
Wheezing (7)	No wheezing	Terminal expiratory wheezing or only with stethoscope	Entire expiration or audible during expiration without stethoscope	Inspiration and expiration without stethoscope
Apnea (4)	None	-	-	Occasional self-correcting apnea/short pauses OR Required stimulation OR Assisted ventilation
General condition 1[b]	No concerns (condition is stable or improving)	-	Some concerns (may become unstable/requires close observation)	Extremely concerned (unstable, requires immediate medical review)
General condition 2 (9)	Excellent/Very good	Good	Fair	Poor/ Very Poor
Fever [b]	<38°C	-	≥38°C	-
Heart rate[b]	Inside normal age range	Outside normal age range but not inside abnormal range	-	Inside abnormal age range
Respiration rate[b]	Inside normal age range	Outside normal age range but not inside abnormal range	-	Inside abnormal age range

Note: [a]ClinRO version 2.0 Q5 “Coughing” response 4 “Required suctioning to stimulate cough and remove secretions” was not included in the dimension score. [b]Data not captured as part of PRESORS ClinRO. Data was retrieved from eCRF. Normal and abnormal age ranges for respiration rate and heart rate originate from the clinical trial protocol and are displayed in Appendix.

As can be seen in Table 3, “Activity level”, “Sleep problems”, “Wheezing”, “Breathing problems other than retractions”, “Cyanosis”, “Coughing” and “Feeding problems” are assessed in both the ObsRO and ClinRO. Three dimensions (“Crying”, “Dehydration/Urination” and “Vomiting”) are only assessed in the ObsRO based on the assumption that caregivers are more likely to observe these signs because they spend more time with the child over a 24-hour period and they are familiar with how the child was before the RSV infection. Three dimensions (“Apnea”, “Breathing problems: retractions”, and “Nasal secretions” that required suctioning) are only assessed in the ClinRO as these signs require training and experience arising from treating many children with RSV. In addition, four ClinRO dimension scores are derived from data collected in the eCRF rather than the PRESORS ClinRO to avoid duplication (“Concerns with condition”, “Heart rate”, “Respiration rate”, and “Fever”) during hospitalization and at follow-up visits. Heart rate, respiration rate and fever were also assessed by the caregiver in PRESORS ObsRO, but heart and respiration rates were measured only after hospital discharge and up to Day 14.

Table 3

Concepts Covered by the PRESORS ObsRO and ClinRO and Included in the Summary Scores and Recovery Endpoint Generation

Concepts/Dimensions	ObsRO (12 Dimensions)	ClinRO (13 Dimensions)
Health status	(X)	(X) (from CRF)
Activity Level	X	X
Sleep problems	X	X
Crying	X	–
Wheezing	X	X
Breathing problems: retractions	–	X
Respiratory difficulty	–	–
Breathing problems: other signs	X	X
Nasal secretion	–	X
Apnea	–	X
Cyanosis	X	X
Cough/Coughing	X	X
Vomiting	X	–
Feeding problems	X	X
Dehydration/Urination	X	–
Fever	X	X (from eCRF)
Respiration Rate	X	X (from eCRF)
Heart rate	(X)	(X) (from eCRF)

Notes: In brackets are items pertaining to the scale but not used in the calculation of the summary score or generation of the recovery endpoint.

A composite score was calculated for both COAs as the mean of sign scores (excluding the ObsRO “Heart rate” that was not reported by caregivers until after hospital discharge and was not reliably reported by caregivers post discharge). The composite score was the average of the 12 sign scores for the ObsRO and the 13 sign scores for the ClinRO (of note, dimension scores related to general health impression were not included in the composite sign score). The composite scores range from 0 to 3 with higher scores reflecting more severe RSV signs.

Statistical Analyses

Daily completion rates – the percentage of children with at least one ObsRO (respectively one ClinRO) completed each day – were described over time to evaluate adequacy of completion by caregivers and clinicians as required by the protocol. Overall completion rates were also calculated as the percentage of days with a completed questionnaire for either the ObsRO or the ClinRO.

PRESORS ObsRO and ClinRO daily dimension scores were calculated as the highest (ie, worst) score for the day for the considered sign, allowing to capture each sign at its most severe level over a 24-hour period.

PRESORS ObsRO and ClinRO psychometric properties were assessed blinded from treatment arm assignment. Each instrument’s clinical validity was assessed by describing and comparing composite scores between groups of children categorized as Excellent/Very good/Good and Fair/Poor/Very poor based on responses to the ObsRO or ClinRO “Health status” questions.

The responsiveness or ability of each PRESORS instrument to detect change over time was assessed by describing and comparing change in dimensions and composite daily scores from baseline to Day 3 in children categorized as unchanged or worsened versus improved based on change over time observed in the corresponding “Health status” item. Additionally, Effect sizes (ES), were calculated (as difference between baseline and Day 3 score means divided by baseline SD) to evaluate the magnitude of change in health status seen in these groups.14 Following Cohen’s guidance for interpreting the magnitude of ES, a value of 0.20 ES was considered a small change, 0.50 a moderate change, and 0.80 a large change.15 Clinical validity and responsiveness were considered to have been demonstrated when statistically significant differences (p<0.05) between clinical severity groups and between clinical change groups were obtained in the expected direction.

As the ClinRO was developed to assess disease severity in children as rated by clinicians and presented similarities with the ObsRO, convergence between the two instruments was assessed by calculation of polychoric correlation coefficients between dimension scores of the two instruments at baseline and Day 3.

Due to sparse ClinRO data after hospital discharge, and the fact that children have remaining symptoms at the time of discharge, time to resolution of RSV signs was generated using ObsRO data only. Time from first study drug intake to resolution of RSV signs was estimated by Kaplan–Meier medians using two endpoints derived from the ObsRO. The first endpoint, time to recovery, was defined considering all signs except “Heart rate” (due to the lack of reliability of caregivers’ measurements for heart rate) as the first time point over a 24-hour period where all signs considered were rated no greater than mild – ie, a score of 0 or 1 for three consecutive assessments from Day 1 to Day 14 and for two consecutive assessments beyond that point. The second endpoint was the time to definitive recovery defined as the first time point where all signs excluding “Heart rate” were rated no greater than mild from that point to the end of follow-up.

Results

Among the 61 children enrolled in the RSV1005 trial, 44 (72.1%) were confirmed to have RSV using qRT-PCR and were randomized to receive either JNJ53718678 or placebo. All 44 patients had data available from at least one PRESORS ObsRO and one ClinRO and were included in the present analyses. Children baseline characteristics are presented in Table 4. The ObsRO had been completed by the same caregiver throughout the trial for 65.9% of children. ObsRO daily completion rates were generally high throughout the trial: 93.2% at baseline and around 80% daily afterwards. Similarly, ClinRO daily completion rates were high throughout the trial (90.9% at baseline and higher than 90% afterwards). The median percentage of days with a completed ObsRO and a completed ClinRO were 93.1% and 100% respectively. At item level, heart and respiration rates measurements, which were to be completed by caregivers only after hospital discharge and until Day 14, presented large percentage of missing data (49% and 38% of missing data at Day 7 and Day 14 respectively).

Table 4

Baseline Characteristics of Infants in RSV1005 Analysis

Characteristics	N=44
Female, n (%)	23 (52.3%)
Age group, n (%)
≥ 6 months and ≤24 months of age	19 (43.2%)
≥ 3 months and <6 months of age	10 (22.7%)
≥ 1 months and <3 months of age	15 (34.1%)
Race, n (%)
Asian	2 (4.5%)
Not reported	5 (11.4%)
Other	7 (15.9%)
White	30 (68.2%)
Time from presentation to hospital until 1st SDI (days), mean (SD)	2.5 (0.73)
Time between start of infection and Day 1 (days), mean (SD)	5.4 (1.81)

Abbreviations: SDI, study drug intake; SD, standard deviance.

PRESORS ObsRO and ClinRO Daily Dimensions and Composite Scores

PRESORS ObsRO and ClinRO dimensions and composite mean (SD) scores at baseline are presented in Table 5, together with the percentage of children with the minimum score of 0 (sign absent). The most prevalent RSV signs at baseline as assessed by clinicians were “Cough”, “Wheezing”, “Sleep problems” and “Retractions”. For most other RSV signs, a substantial proportion of caregivers and clinicians rated the child as not having the considered sign at baseline. This was particularly salient for the ObsRO and ClinRO “Cyanosis” and “Fever”, for the ObsRO “Vomiting”, and for the ClinRO “Apnea” and “Breathing problems: other signs”; all of these signs were absent in all or all but one patient at baseline. Consequently, mean baseline ObsRO and ClinRO dimension scores (range: 0.0 to 1.7; 0.0 to 2.0, respectively) and composite scores (0.7 for both) were low, suggesting mild RSV-related illness for most of the infants in the study at baseline.

Table 5

PRESORS ObsRO and ClinRO Dimension and Composite Scores at Baseline

	ObsRO (N=41)			ClinRO (N=40)
	Score Values	Mean (SD)	% at Floor[a]	Score Values	Mean (SD)	% at Floor[a]
Sign Dimensions
Activity level	0–2	1.0 (0.76)	29.3%	0–3	0.6 (0.74)	55.0%
Crying	0–3	0.8 (0.58)	31.7%	NA	NA	NA
Sleep problems	0 or 2[c]	1.3 (0.97)	34.1%	0–3	1.0 (0.58)	15.0%
Feeding problems	0–3	1.4 (1.05)	24.4%	0–3	1.1 (1.28)	47.5%
Dehydration	0–3	0.3 (0.52)	70.7%	NA	NA	NA
Vomiting	0 or 2[c]	0.1 (0.31)	97.6%	NA	NA	NA
Nasal secretion	NA	NA	NA	0–2	0.7 (0.53)	35.0%
Apnea	NA	NA	NA	0 or 3[c]	0.0 (0.00)	100.0%
Breathing problems: other signs[b]	0,1, or 3	1.2 (1.09)	29.3%	0–3	0.1 (0.40)	90.0%
Breathing problems: retractions	NA	NA	NA	0–3	2.0 (1.14)	20.0%
Wheezing	0–3	1.2 (1.24)	41.5%	0–3	1.1 (0.68)	15.0%
Cough	0–2	0.8 (0.83)	43.9%	0–2	1.3 (0.56)	5.0%
Cyanosis	0, 2, or 3	0.0 (0.00)	100.0%	0 or 3[c]	0.0 (0.00)	100.0%
Respiration rate	0,1, or 3	MD[e]	MD[e]	0, 1, or 3[d]	0.7 (0.72)	37.5%
Heart rate	0, 1, or 3	MD[e]	MD[e]	0, 1, or 3[d]	0.8 (1.04)	50.0%
Fever	0 or 2[c]	0.0 (0.00)	100.0%	0 or 2[c, d]	0.1 (0.32)	97.5%
Composite score	0–3	0.7 (0.40)	2.4%	0–3	0.7 (0.26)	0.0%
Concerns with condition	NA	NA	NA	0, 2, or 3[d]	0.9 (1.01)	55.0%
Health status	0–3	1.7 (0.60)	2.4%	0–3	1.6 (0.74)	5.0%

Notes: [a] Percentage of children with sign not present; [b] other signs: head bobbing, nasal flaring, grunting, and other signs of increased work of breathing (excluding retractions), [c] dichotomous, [d] Dimensions collected through the eCRF, [e] Heart and respiration rates were measured by caregivers only after hospital discharge and until Day 14.

Abbreviation: NA, not applicable (dimension was not available for this instrument); MD, missing data (score could not be computed because no data were available).

The highest average sign dimension scores (means ≥1) were obtained for “Feeding”, “Sleep problems”, “Wheezing”, “Breathing problems: other signs” and “Activity level” for the ObsRO and “Breathing problems: retractions”, “Coughing”, “Wheezing” and “Feeding” for the ClinRO.

PRESORS ObsRO and ClinRO mean daily dimension and composite scores over time are presented in Figure 1. Most signs present at baseline showed a rapid decrease in their mean scores from baseline to Day 7. ObsRO and ClinRO mean composite scores also decreased rapidly from baseline to Day 7 and remained low until the end of the trial. ObsRO “Sleep problems” and “Wheezing” scores, but not ClinRO corresponding scores, persisted after Day 7. Both ObsRO and ClinRO “Heart rate” and to a lesser extent “Respiration rate”, showed higher means compared to the other dimensions suggesting that some children had persistent elevated heart and respiration rates even after all other signs had resolved. This trend was more pronounced for the ObsRO than for the ClinRO. A close look at respiration and heart rates showed, in addition to the large percentage of missing data for the ObsRO at time points where these data should have been recorded, an abnormally larger proportion of children with a score of 3 (corresponding to clinically abnormal levels, these abnormal levels are provided per age class in the table in ) with the ObsRO compared to corresponding scores with the ClinRO. This was particularly true for heart rate. At Day 7, 55% of patients with available heart rate measurement (12 of 22 patients) had a score of 3 on the ObsRO reflecting clinically elevated levels of heart rate while this proportion was 0% (0 of 43 patients) for the ClinRO on the same day. Similar findings were seen at Day 14, where 64% (16 of 25 patients) of patients had a score of 3 on the ObsRO while this proportion was only 2% (1 of 44 patients) for the ClinRO. The proportion of patients with a respiration rate score of 3 for the ObsRO was 23% (5 of 22 patients) and 16% (4 of 25 patients) at Day 7 and Day 14, respectively, and 0% at both time points for the ClinRO.

Figure 1

ObsRO (A) and ClinRO (B) Daily Dimensions and Composite Scores over Time.

Results from qualitative research revealed that caregivers had difficulties counting breaths or heart beats.16 These findings together with the fact that heart rate was not identified as a key sign of RSV in the FDA guidance related to the development of antiviral drugs for RSV10 has led to exclusion of “Heart rate” from the ObsRO composite score calculation, while respiration rate being a Key Sign of RSV was kept in the calculation of the summary composite sign score and recovery endpoints.

ObsRO Psychometric Properties

Clinical validity results (Figure 2A) demonstrated a pattern of higher mean scores reflecting increased severity observed in children with a worse health status for most dimension scores, leading to significant between-group differences for the ObsRO composite score (p<0.001). Dimension scores for “Activity level”, “Feeding problems”, “Crying”, and “Breathing problems other than retractions” were as expected significantly higher in children whose health was rated as fair or poor compared to those with an excellent, very good or good health. Similar patterns were observed for change in dimension scores, although “Sleep problems” replaced “Activity level” as a key sign linked to clinically meaningful change. Larger improvement in ObsRO scores (in absolute values) was observed in children with an improved health status compared to those with unchanged (or worsened) health status, with a significant between-group difference in change in the ObsRO composite score overall (p=0.005) (Figure 2B). The majority of ES were of moderate to large amplitude in the improved children. The generally small to large ES obtained for the unchanged (or worsened) patients were indicative of improvement in general also, but to a lesser degree. These results suggest that the ObsRO composite score is sensitive to clinical differences and to change over time.

Figure 2

PRESORS ObsRO Psychometric Validation. (A) Clinical validity: Description and comparison of PRESORS ObsRO mean (SD) scores at baseline across health status groups. (B) Ability to detect change over time: Description and comparison of PRESORS ObsRO mean score change (SD) from baseline to Day 3 and standardized effect sizes (ES).

ClinRO Psychometric Properties

Analysis of the clinical validity of the ClinRO (Figure 3A) showed a similar pattern of higher mean scores reflecting higher severity in children with a worse health status rating at baseline, and the accumulation of impairment in scores is reflected in significant differences for the ClinRO composite score (p=0.001) when comparing patients with a good to excellent health status to those with poor to fair health status. When describing and comparing the change in PRESORS ClinRO scores from baseline to Day 3, larger reduction in scores were seen in children with improvement in health status than in children rated as unchanged or worsened for “Activity level”, “Retractions”, “Coughing”, and “Wheezing” leading to significant differences observed in the composite score (Figure 3B).

Figure 3

PRESORS ClinRO Psychometric Validation. (A) Clinical validity: Description and comparison of PRESORS ClinRO mean (SD) scores at baseline across health status groups. (B) Ability to detect change over time: Description and comparison of PRESORS ObsRO mean score change (SD) from baseline to Day 3 across health status change groups and standardized effect sizes (ES).

These findings are supported by analysis of ES. Except for heart and respiration rates, ESs were largest in the greatly improved health status group, moderate to small in the slightly improved health status group, and generally smaller in the unchanged/worsened health status group. These results suggest that the ClinRO composite score is sensitive to clinical differences and to change over time.

Correlations Between PRESORS ObsRO and ClinRO Scores

Polychoric correlation coefficients calculated at baseline between ObsRO and ClinRO PRESORS sign scores are presented in Table 6. Dimensions covering similar concepts were expected to show positive large polychoric correlation coefficients (>0.4). “Activity level”, “Cough/Coughing” and “Feeding problems” showed a positive association; “Sleep problems” and “Wheezing” dimension scores presented lower polychoric correlation coefficients (0.20 and 0.28, respectively). Due to no or limited variability in the data (some signs were not seen at baseline), and to missing data (heart and respiration rates were captured by caregivers only after hospital discharge and were therefore missing at baseline), correlation coefficients could not be calculated for the other dimensions covering similar concepts between the two instruments - “Breathing problem: other signs”, “Cyanosis”, “Fever”, “Heart rate”, and “Respiration rate”.

Table 6

Polychoric Correlation Coefficients Between ObsRO Dimensions and ClinRO Dimensions Scores at Baseline (N=40)

		ClinRO Scores
		Activity Level	Sleep Problems	Breathing Problem I	Breathing Problem II	Coughing	Nasal Secretion	Wheezing	Feeding Problems
ObsRO scores	Activity level	0.576	0.463	0.098	−0.068	0.370	0.066	0.148	0.388
	Sleep problems	0.199	0.196	0.183	−0.300	0.450	−0.008	−0.024	0.049
	Crying	0.132	0.316	0.066	−0.643	0.398	−0.103	−0.135	0.439
	Wheezing	−0.098	0.004	0.261	−0.196	0.056	−0.221	0.279	−0.208
	Breathing problem II	0.509	0.323	0.372	−0.480	0.442	−0.055	0.198	0.157
	Cough	0.063	0.286	0.268	−0.140	0.464	−0.005	−0.050	0.303
	Vomiting	0.991	−0.997	0.983	−0.926	−0.309	0.383	0.000	0.098
	Feeding problems	0.308	0.095	0.173	−0.286	0.491	−0.120	−0.215	0.831
	Dehydration	−0.403	0.083	0.021	−0.147	0.418	−0.399	0.257	−0.084

Note: Breathing problems I= breathing problems retractions; Breathing problems II= breathing problems: other signs. In bold are polychoric coefficient correlation above 0.4. In blue are correlations hypothesized to show the largest positive associations. In grey are polychoric correlations involving ObsRO “Vomiting” and “Dehydration” and ClinRO “Breathing problem: other signs” scores. These should be interpreted with caution as correlations involving these scores present spurious findings due to the extremely limited variability of these scores. Results for ObsRO “Cyanosis”, ObsRO “Fever”, ObsRO “Heart rate”, ObsRO “Respiration rate”, ClinRO “Cyanosis”, ClinRO “Apnea” and ClinRO “Fever” dimensions are not presented in this table due to no variability in the data and/or missing data for all patients.

Results at Day 3 indicated similar positive associations (correlation coefficient>0.4) between the two instruments for “Activity level”, “Cough/Coughing”, “Feeding problems”, while correlations for “Sleep problems” and “Wheezing” increased from 0.20 to 0.48 and 0.28 to 0.46 suggesting greater consistency between the two instruments at Day 3 compared to baseline.

These results suggest some consistency between the ObsRO and the ClinRO dimension scores, although correlations were sometimes low between scores assessing the same concept (eg, “Sleep” and “Wheezing” at baseline).

Time to Resolution of RSV Signs (ObsRO)

The percentage over time of children who were considered recovered or definitively recovered is presented in Figure 4. By the end of Day 3, 68% of children were considered recovered, ie, all RSV signs (excluding “Heart rate”) were mild or absent for at least 24 hours, while this percentage reached 91% by Day 7. Kaplan–Meier analysis resulted in a median time to recovery (95% CI) of 45.6 hours (29.7; 52.5) or 1.9 days. When using the more conservative definition of definitive recovery (first time point when all signs excluding “Heart rate” were rated no greater than mild from that point to the end of follow-up), the percentage of definitely recovered children was much lower, ie, 11% at the end of Day 3 increasing to 21% at the end of Day 7. Results from Kaplan–Meier analysis showed that 23% of the children were censored (these patients were not considered definitely recovered at the end of the 28-day study period) and the median time to definitive recovery (95% CI) was 415.0 hours (293.0; 537.0) or 17.3 days.

Figure 4

Percentage of patients recovered and definitively recovered based on ObsRO assessments.

Discussion

PRESORS ObsRO and ClinRO instruments were designed to monitor the clinical course of illness due to RSV infection by evaluating the severity of key as well as less specific RSV signs (eg, sleep). The goal was to monitor RSV signs, derive composite summary sign scores, and identify endpoints marking disease resolution based on the absence of these signs. Quantitative blinded analyses of the PRESORS data from RSV1005 trial were conducted to assess key psychometric properties of preliminary ObsRO and ClinRO versions, identify improvements that may be needed, and analytical strategies to be used in future trials.

The daily completion rates for both the PRESORS ObsRO and ClinRO were generally high (ie, around or superior to 80%) over the entire study period. The median percentage of days with a completed questionnaire was 93.1% and 100% for the ObsRO and the ClinRO, respectively, suggesting acceptability and relative ease of use for both instruments. One exception was the reporting of heart and respiration rates by caregivers, which presented a substantial percentage of missing data.

PRESORS instruments were able to characterize the evolution of the clinical course of RSV-related illness while children required hospitalization, at discharge, and throughout follow-up for several weeks at home by caregivers. As expected, PRESORS ClinRO and ObsRO scores indicated greater severity at baseline and improvement over time, in accordance with the caregivers’ and clinicians’ ratings of the child’s health throughout the trial. Some ObsRO dimension scores indicated some signs lingered beyond Day 7, in particular “Sleep problems” and to a lesser extent “Wheezing”, but this was not observed in the corresponding ClinRO dimension scores. It is important to remember that the study population included many very young children whose lingering “Sleep problems” are not surprising; restless or disturbed sleep from time to time is not uncommon at a very young age, even in healthy children, and is not necessarily due to RSV infection. Lingering “Wheezing” could result from signs noticed by those caregivers who spend most of their time with the child. Clinicians may not have the opportunity to observe these lingering and potentially intermittent signs if they occur outside the clinical examination, for example while the child is asleep, when a caregiver is more likely to be present to observe them.

Even though assessing change in scores over time can be informative, the time to resolution of disease signs or recovery is more relevant in an acute illness like RSV. The ObsRO that was completed daily throughout the trial duration from treatment initiation until Day 28 was the only instrument that could be used to assess time to recovery for all signs. Time to recovery – defined as the first time point over a 24-hour period where all signs considered were rated no greater than mild – was rapid (Kaplan Meier estimated median: 1.9 days). But use of the more conservative definition of definitive recovery, when all signs were rated no greater than mild from that point in time to the end of follow-up, resulted in a substantial proportion of patients not considered definitely recovered over the studied period (22.7%), and a much longer time to definitive recovery (Kaplan Meier estimated median: 17.3 days). When considering those signs responsible for patients not being considered definitively recovered, “Sleep” was by far the most prevalent (9 out of 10 patients not definitively recovered had a sleep score of 2 in the last 2 days of the studied period), while others such as “Coughing” or “Vomiting” were also apparent in some infants. This suggests that time to recovery endpoints need to be defined carefully so that signs not specific to RSV, which may occur even in healthy children, do not lead to misclassifying patients as not recovered.

Although 8 dimensions of the PRESORS ObsRO and ClinRO assessed similar concepts, convergence between scores varied. Some concepts were substantially correlated while others presented lower or more variable levels of association. Some of this may be due to differences in operationalization between ObsRO and ClinRO scores such as timing of assessment, recall period, frequency, questions and answer choices content, and scoring algorithm. Limited convergence may also be due to differences in proximity and time spent with the child, as well as in the level of expertise in recognizing clinical signs of RSV-related illness. This is particularly true for concepts such as “Sleep”, for which a reliable assessment requires the presence of the rater during the sleeping period; crying or coughing, which can vary greatly over a 24-hour period; or wheezing, which may be more prevalent or noticeable at night time, but may also require expertise in recognizing wheezing sounds.

These data also suggest some caregivers encountered difficulty with measuring heart and respiratory rates of the infant. This was confirmed by findings from a qualitative research conducted with parents of children with RSV.16 Some missing data for these scores were expected per protocol, ie, before hospital discharge and after Day 14, but others were not, for example only 50% of children had a respiratory rate reported at Day 7 in the ObsRO, when all children had been discharged from hospital and were therefore expected to have such a measurement conducted in the evening at home. Additionally, when available, some caregivers reported improbable abnormal values leading to higher mean scores compared to scores derived from clinician data on the same days, particularly for heart rate. This ultimately resulted in the decision to omit heart rate from the ObsRO composite score. Providing simple, reliable biosensors that caregivers can be trained to use for heart and respiration rate recording may be the only way to capture these vital signs accurately at home.

Another important finding from this study was that a few signs were absent in all or almost all participants at baseline (eg, fever, apnea, and cyanosis). This may be due to the time elapsed between hospital admission and baseline PRESORS assessments. Indeed, in RSV1005, 5.4 days on average occurred between occurrence of first RSV signs and first treatment intake, and 2.5 days between presentation to the hospital and first trial drug intake. This demonstrates that for many patients substantial time had elapsed from presentation to the hospital until first study medication intake, time during which, abnormal vital signs were identified and supportive standard of care management initiated to normalize respiration rate, heart rate, blood oxygen levels, and hydration. It is likely that the severe signs present in children were addressed upon admission to the hospital and substantially improved by the time the participants were enrolled in the study.

Limitations

These analyses were based on a small sample size that restricted the types of analyses that could be performed. Also, the absence of many signs at baseline led to significant floor effects for several dimensions which made the psychometric analyses of these dimensions impossible or uninterpretable. In particular, correlation-based analyses to assess construct validity or internal consistency reliability could not be generated when there was no variability or led to spurious findings when only a couple of patients showed some variability on a given score.

Therefore, results from these quantitative analyses of the initial version of PRESORS should be considered preliminary and be used to resolve identified issues with the instruments and their implementation for use in future clinical trials. Further quantitative assessment of updated PRESORS will be conducted in future larger scale trials.

Future Directions

Based on these findings, there are several improvements envisioned for PRESORS. Wording of the ObsRO and ClinRO response options will be revised to improve convergence between the two instruments. Revised study designs that enable both instruments to be completed with the same frequency, around the same time, and using the same recall period would likely increase convergence between the two instruments. Finally, just as clinicians have medical devices to aid in monitoring breathing and heart function, parents and other caregivers need simple to use and accurate biosensors to assess these reliably. In the absence of robust, well-validated biosensors to assess these outcomes in infants and young children, it would be better to omit heart and respiration rate assessments from the PRESORS ObsRO. Impressions of rapid breathing or heart beats will be explored to determine whether these can provide any evidence for monitoring RSV-related disease based on caregiver observations.

Since the start of RSV1005, new questionnaires have been described in the medical literature, including the ReSViNET scale17 and the Gilead RSV Caregiver Diary (GRCD).18 The ReSViNET scale was developed to allow classification and monitoring of children without the need for in-patient assessment, but does not attempt to measure all the most common signs of RSV that caregivers may observe or monitor at home. The GRCD was developed based on qualitative research findings and has been evaluated in a non-trial quantitative study following children for 5 days at home. As suggested by the authors, several steps are still needed to finalize this instrument such as item reduction and adjusted scoring algorithm as well as implementation of the GRCD in a clinical trial setting to evaluate its construct validity in that context and characterize what a meaningful change should be.

The present research has also shown that the operational definition of recovery (ie, the selection of signs and the required duration of stabilization of these signs) should focus on respiratory and systemic signs of RSV-related illness. While disturbed sleep, problems with feeding or with activity levels are behavioral signs that a child is unwell, these are not RSV-specific, and sleep in particular has shown to be problematic for defining recovery in infants and small children. Assessment of sleep disturbance may provide valuable evidence that a child has sufficiently recovered from key RSV signs, but may not be specific enough to be used in defining recovery from RSV.

PRESORS ClinRO and ObsRO were revised based on findings from this research and qualitative studies with caregivers and clinical experts. The revised PRESORS instruments are now being studied in multiple clinical observational and interventional trials that will provide information on the clinical course of RSV in infants and children treated for RSV in both hospital and community settings.

Finally, the floor effects seen at baseline resulting in short time to recovery observed in this trial would likely not support demonstration of between-group differences if observed in future trials. While it is unlikely that children with respiratory distress would be unmanaged at presentation at the hospital, these analyses highlight the importance of reducing the time from first presentation for medical care to first study drug administration to better capture the value of RSV treatments.

Conclusion

Administration of PRESORS instruments during a Phase 1 clinical trial provided valuable information on signs of RSV disease and the ability of caregivers to monitor Key Signs using an electronic user-friendly format. These analyses identified ways to improve the PRESORS and clinical outcomes assessments by caregivers:

Focus on signs caregivers can be easily trained to assess accurately

Increase consistency between the PRESORS ObsRO and ClinRO items and

Ensure timing of assessments and recall period used is similar where possible for ClinRO and ObsRO to improve concordance.

Independently of PRESORS, future trials should aim to reduce the time from first presentation to the hospital to baseline assessments to limit baseline floor effects.

In conclusion, PRESORS ObsRO shows promise as a clinical outcome assessment in studies of new treatments for RSV-related illness in infants and young children.

Table 7

RSV1005 Participating Study Sites and Ethics Committees with Date of Ethics Approval

Country	Site ID	Site Address	Site City State Zip	Ethics Institution Name	Approval Date
Argentina	B48-AR10002	Gallo 1330	City of Buenos Aires, Buenos Aires, 1425	Comité de Ética en Investigación del Hospital General de Niños Ricardo Gutierrez	19-Apr-16
Argentina	B48-AR10003	Montes De Oca 40	City of Buenos Aires, Buenos Aires, C1270AAN	Comité de Ética en Investigaciones del Hospital General de Niños Pedro De Elizalde	29-Sep-15
Argentina	B48-AR10005	Belgrano 1500	Córdoba, Córdoba, X5000JRD	Comité Institucional de Ética de la Investigación en Salud del Adulto - CIEIS del Adulto	10-Dec-15
Argentina	B48-AR10006	Calle Necochea 675	Bahía Blanca, Buenos Aires, B8001HXM	Comité de Ética del Hospital Italiano Regional del Sur	29-Sep-15
Australia	B48-AU10001	PO Box 281, Ryrie Street	Geelong, Victoria, 3220	Monash Health Human Research Ethics Committees	07-Mar-16
Australia	B48-AU10002	Locked Bag 4001, Corner Hawkesbury Road and Hainsworth Street	Westmead, NSW, 2145	Monash Health Human Research Ethics Committees	07-Mar-16
Australia	B48-AU10003	48 Liverpool Street	Hobart, TAS, 7000	Tasmania Health and Medical Human Research Ethics Committee	24-Mar-16
Belgium	B48-BE10001	Laarbeeklaan 101	Bruxelles, 1090	UZ Brussel IRB	25-Nov-15
Belgium	B48-BE10002	Route de Lennik 808	Anderlecht, 1070	ULB Hôpital Erasme IRB	25-Nov-15
Belgium	B48-BE10002	Route de Lennik 808	Anderlecht, 1070	UZ Brussel IRB	25-Nov-15
Belgium	B48-BE10003	Wilrijkstraat 10	Edegem, 2650	Ethisch Comité UZ Antwerpen	25-Nov-15
Belgium	B48-BE10003	Wilrijkstraat 10	Edegem, 2650	UZ Brussel IRB	25-Nov-15
Belgium	B48-BE10004	Avenue J. J. Crocq 15	Bruxelles, 1020	Comité d’Ethique HUDERF	25-Nov-15
Belgium	B48-BE10004	Avenue J. J. Crocq 15	Bruxelles, 1020	UZ Brussel IRB	25-Nov-15
Belgium	B48-BE10005	Avenue Hippocrate 10	Bruxelles, 1200	Comité d’Éthique UCL Saint-Luc	25-Nov-15
Belgium	B48-BE10005	Avenue Hippocrate 10	Bruxelles, 1200	UZ Brussel IRB	25-Nov-15
Belgium	B48-BE10006	Boulevard Zoé Drion, 1	Charleroi, 6000	Comité d’Ethique du CHU Charleroi	25-Nov-15
Belgium	B48-BE10006	Boulevard Zoé Drion, 1	Charleroi, 6000	UZ Brussel IRB	25-Nov-15
Belgium	B48-BE10009	Herestraat 49	Leuven, 3000	Commissie Medische Ethiek UZ Gasthuisberg	14-Sep-16
Belgium	B48-BE10009	Herestraat 49	Leuven, 3000	UZ Brussel IRB	14-Sep-16
Belgium	B48-BE10010	Mechelsestraat 24	Lier, 2500	Ethisch Comité Heilig Hart Ziekenhuis Lier	14-Sep-16
Belgium	B48-BE10010	Mechelsestraat 24	Lier, 2500	UZ Brussel IRB	14-Sep-16
Brazil	B48-BR10001	Campus Universitario, s/n, Bairro Monte Alegre	Ribeirao Preto, SP, 14048-900	Comite de Etica em Pesquisa do HCFMRP - USP	29-Feb-16
Brazil	B48-BR10001	Campus Universitario, s/n, Bairro Monte Alegre	Ribeirao Preto, SP, 14048-900	National Committee of Ethics in Research (CONEP)	29-Feb-16
Brazil	B48-BR10002	Avenida Angélica, 2071, 2º andar, Higienopolis	São Paulo, SP, 01227–200	Comite de Etica em Pesquisa da Fundação Jose Luiz Egydio Setubal	29-Feb-16
Brazil	B48-BR10002	Avenida Angélica, 2071, 2º andar, Higienopolis	São Paulo, SP, 01227–200	National Committee of Ethics in Research (CONEP)	29-Feb-16
Brazil	B48-BR10003	Avenida Independencia, 155, 3º andar	Porto Alegre, RS, 90035-074	Comite de Etica em Pesquisa do Hospital da Crianca Santo Antonio	29-Feb-16
Brazil	B48-BR10003	Avenida Independencia, 155, 3º andar	Porto Alegre, RS, 90035-074	National Committee of Ethics in Research (CONEP)	29-Feb-16
Brazil	B48-BR10004	Av. Ipiranga, 6690, Pos graduacao em pediatria	Porto Alegre, RS, 90610-000	Comite de Etica em Pesquisa da PUCRS	29-Feb-16
Brazil	B48-BR10004	Av. Ipiranga, 6690, Pos graduacao em pediatria	Porto Alegre, RS, 90610-000	National Committee of Ethics in Research (CONEP)	29-Feb-16
Brazil	B48-BR10005	Rua General Carneiro 181	Curitiba, PR, 80060-900	Comitê de Ética em Pesquisa do Hospital de Clínicas da Universidade Federal do Paraná	29-Feb-16
Brazil	B48-BR10005	Rua General Carneiro 181	Curitiba, PR, 80060–900	Comite de Etica em Pesquisa envolvendo seres humanos do Setor de Ciencias da Saude da UFPR (CEP/SD)	29-Feb-16
Brazil	B48-BR10005	Rua General Carneiro 181	Curitiba, PR, 80060–900	National Committee of Ethics in Research (CONEP)	29-Feb-16
Brazil	B48-BR10006	Rua Diniz Cordeiro, 30, Botafogo	Rio de Janeiro, RJ, 22281–100	Comite de Etica em Pesquisa IDOR	29-Feb-16
Brazil	B48-BR10006	Rua Diniz Cordeiro, 30, Botafogo	Rio de Janeiro, RJ, 22281–100	National Committee of Ethics in Research (CONEP)	29-Feb-16
Germany	B48-DE10001	Lindwurmstr. 4, Eingang F/G	München, 80337	Ethikkommission der Medizinischen Fakultät der Albert-Ludwigs-Universität Freiburg	16-Dec-15
Germany	B48-DE10001	Lindwurmstr. 4, Eingang F/G	München, 80337	Ethikkommission der Medizinischen Fakultät der Ludwig-Maximilians-Universität München	16-Dec-15
Germany	B48-DE10002	Mathildenstr. 1	Freiburg, 79106	Ethikkommission der Medizinischen Fakultät der Albert-Ludwigs-Universität Freiburg	16-Dec-15
Germany	B48-DE10007	Im Neuenheimer Feld 430, Zentrum f. Kinder- und Jugendmedizin	Heidelberg, 69120	Ethikkommission der Medizinischen Fakultät der Albert-Ludwigs-Universität Freiburg	03-Nov-16
Germany	B48-DE10007	Im Neuenheimer Feld 430, Zentrum f. Kinder- und Jugendmedizin	Heidelberg, 69120	Ethikkommission der Medizinischen Fakultät der Universität Heidelberg	03-Nov-16
Germany	B48-DE10008	Werler Str. 110, Kinderheilkunde	Hamm, 59063	Ethik-Kommission der Ärztekammer Westfalen-Lippe und der Medizinischen Fakultät der Westfälischen Wilhelms-Universität Münster	16-Jan-17
Germany	B48-DE10008	Werler Str. 110, Kinderheilkunde	Hamm, 59063	Ethikkommission der Medizinischen Fakultät der Albert-Ludwigs-Universität Freiburg	16-Jan-17
Italy	B48-IT10001	Via Giustiniani 3	Padova, 35128	Comitato Etico per la sperimentazione clinica della Provincia di Vicenza (CESC-VI)	28-Jan-16
Italy	B48-IT10002	Piazzale L. A. Scuro 10	Verona, 37134	Comitato etico per la Sperimentazione Clinica (CESC) delle Province di Verona e Rovigo	28-Jan-16
Italy	B48-IT10003	Via GB Grassi 74	Milano, 20157	COMITATO ETICO INTERAZIENDALE MILANO AREA A	12-May-16
Italy	B48-IT10004	Piazzale Golgi 2	Pavia, 27100	Comitato di Bioetica della Fondazione IRCCS Policlinico San Matteo di Pavia	14-Mar-16
Italy	B48-IT10005	via Massarenti 11	Bologna, 40100	Comitato Etico Indipendente Azienda Ospedaliera Universitaria Policlinico S. Orsola-Malpighi di Bologna	16-Feb-16
Italy	B48-IT10007	Pediatric Highly Intensive Care Unit, via Commenda, 9	Milan, Milan, 20122	Comitato Etico Milano Area B	20-Sep-16
Netherlands	B48-NL10001	Heidelberglaan 100	Utrecht, 3584 CX	Raad van Bestuur UMC Utrecht	26-Sep-16
Netherlands	B48-NL10001	Heidelberglaan 100	Utrecht, 3584 CX	Central Committee on Research Involving Human Subjects (CCMO)	26-Sep-16
Netherlands	B48-NL10006	Spaarnepoort 1	Hoofddorp, 2134 TM	Raad van Bestuur Spaarne Ziekenhuis	26-Sep-16
Netherlands	B48-NL10006	Spaarnepoort 1	Hoofddorp, 2134 TM	Central Committee on Research Involving Human Subjects (CCMO)	26-Sep-16
Philippines	B48-PH10002	Taft Avenue Ermita, Brgy 670 Zone 72, Manila, 1000 Metro Manila	Manila City	Philippine General Hospital - UPMREB	02-Jun-16
Philippines	B48-PH10003	Rm 101 Chong Hua Medical Arts Center	Cebu City	Chong Hua Hospital - Institutional Review	18-Apr-16
Spain	B48-ES10001	Avenida de Córdoba s/n	Madrid, Madrid, 28,041	Comité Autonómico de Ética de la Investigación de Galicia	10-Nov-15
Spain	B48-ES10001	Avenida de Córdoba s/n	Madrid, Madrid, 28,041	CEIC Área 11 - Hospital 12 de Octubre	10-Nov-15
Spain	B48-ES10002	Carretera de Toledo, km 12.500	Getafe, Madrid, 28,020	CEIC Hospital Univeritario de Getafe	10-Nov-15
Spain	B48-ES10002	Carretera de Toledo, km 12.500	Getafe, Madrid, 28,020	Comité Autonómico de Ética de la Investigación de Galicia	10-Nov-15
Spain	B48-ES10003	C/ A Choupana s.n	Santiago de Compostela, A Coruña, 15,706	Comité Autonómico de Ética de la Investigación de Galicia	11-Nov-15
Spain	B48-ES10004	Passeig Vall D’Hebron, 119–129	Barcelona, 08035	Comité Autonómico de Ética de la Investigación de Galicia	11-Nov-15
Spain	B48-ES10004	Passeig Vall D’Hebron, 119–129	Barcelona, 08035	Comité Ético de Investigación Clínica del Hospital Vall d’Hebrón	11-Nov-15
Spain	B48-ES10005	Paseo Sant Joan de Deu, 2	Esplugues de Llobregat, Barcelona, 8950	CEIC Fundacio Sant Joan de Deu	11-Nov-15
Spain	B48-ES10005	Paseo Sant Joan de Deu, 2	Esplugues de Llobregat, Barcelona, 8950	Comité Autonómico de Ética de la Investigación de Galicia	11-Nov-15
Spain	B48-ES10007	Avinguda de Fernando Abril Martorell, nº 106	Valencia, Valencia, 46,026	Comité Autonómico de Ética de la Investigación de Galicia	30-Sep-16
Spain	B48-ES10009	Calle Profesor Martín Lagos s/n, 6ª Planta Sur	Madrid, Madrid, 28,040	Comité Ético de Investigación Clínica de Galicia	27-Jul-16
Spain	B48-ES10010	C/ Jardin de la isla, nº 6. Edificio Expolocal	Sevilla, Sevilla, 41,014	Comité Ético de Investigación Clínica de Galicia	27-Jul-16
Spain	B48-ES10011	Avda Arroyo De Los Ángeles s/n	Malaga, 29,011	Comité Ético de Investigación Clínica de Galicia	30-Sep-16
Spain	B48-ES10012	C/ Nueva Musa s/n	Almeria, Almeria, 4007	Comité Ético de Investigación Clínica de Galicia	13-Oct-16
Sweden	B48-SE10001	Smörslottsgatan 1	Göteborg, 41,685	Regionala Etikprövningsnämnden i Göteborg	22-Oct-15
Sweden	B48-SE10002	Universitetssjukhuset Linköping	Linköping, 58,185	Regionala Etikprövningsnämnden i Göteborg	22-Oct-15
Sweden	B48-SE10004	Lasarettsgatan 48, Skånes Universitetssjukhus Lund	Lund, 22,185	Regionala Etikprövningsnämnden i Göteborg	22-Oct-15
Sweden	B48-SE10005	Ruth Lundskogs gata 3, Skånes Universitetssjukhus Malmö	Malmö, 20,502	Regionala Etikprövningsnämnden i Göteborg	22-Oct-15
United States Of America	B48-US10004	1611 S Baltimore Street	Kirksville, Missouri, 63,501	Quorum Review Board	24-Feb-17