Patient Reported Experience in a Pediatric Emergency Department.

OBJECTIVES: To describe patient-reported experience in a pediatric emergency department (ED) and determine: (1) whether there are differences between the experience children report in comparison to their parents; and (2) whether factors such as time of visit (day, evening, night) and ED census are associated with patient experience.
METHODS: We conducted a prospective cross-sectional survey of children ≥8 years of age and the parents/guardians of children 0 to 17 years who visited a pediatric ED using a validated patient experience measure. The proportion of respondents for each question indicating that an aspect of their care could have been improved was calculated as problem scores for each survey item. The primary outcome was the overall problem scores for all respondents combined and for children and parents separately.
RESULTS: A total of 237 parents and 109 children completed surveys. The areas with the highest problem scores identified by both parents and children were: having enough to do while waiting to be seen (53.5; 95% confidence interval [CI]: 48.1, 58.8) and when to restart usual activities (34.7; 95% CI: 29.7, 40.0). There were meaningful differences in problem scores between children and parents including: doctors and nurses explaining what they were doing (parents: 19; 95% CI: 14.3, 24.7, child 40.4; 95% CI: 31.2, 50.2) and privacy when examined and treated (parents: 17.3; 95% CI: 12.8, 22.9, child: 36.7; 95% CI: 27.8, 46.5).
CONCLUSION: There are differences in reported experience between children and their parents. This highlights the importance of including children when assessing patient experience in a pediatric setting.

Introduction

There is a growing emphasis on collecting patient-reported outcomes reflecting experiences and satisfaction with health care in multiple settings including the emergency department (ED) (1). Evidence suggests that there is a positive association between patient experience, clinical effectiveness, and patient safety (2). Prior research has also shown a relationship between satisfaction with ED care and measures of ED crowding, such as prolonged ED boarding time and prolonged treatment time (3). However, information on pediatric patient experience in the ED setting remains limited with a focus on parental perceptions of care (4 –7). Recent work suggests that collecting data from pediatric patients is not only possible, but important in assessing the perceived quality of care in the ED (8 –10). Data from research in the area of pediatric pain indicate that parent and child reporting of the pain experience differs, and that parents underestimate their child’s pain (11), which suggests that there may be differences in patient experience as reported by children and their parents.

The primary aim of this project was to measure patient-reported experience in a pediatric ED and to determine if there is a difference between the experiences children report in comparison to their parents. The secondary aim was to determine whether factors such as time of visit (day, evening, night) and ED census are associated with patient experience. The resulting knowledge about patient experience was used to plan ongoing patient experience measurement, and interventions to improve experience, within our ED.

Methods

Project Design and Survey Tool

We conducted a prospective cross-sectional survey of children and their parents who visited a pediatric ED using a validated measure, The patient-reported experience measure (PREM) for children in urgent and emergency care. The PREM was developed and validated by the Royal College of Paediatric and Child Health and the Picker Institute Europe for use in the pediatric ED setting (12). The PREM was developed using rigorous methods including: (a) a review of the literature on patient experience surveys; (b) focus groups with children and parents to determine which aspects of emergency care were most important; (c) cognitive testing of the draft survey and; (d) piloting of the survey in multiple ED settings. The measure consists of 21 questions about the parent/patient experience from ED arrival, including care by EMS if applicable, to discharge. It also includes 4 additional questions, 2 demographic, and 2 open-ended. The PREM tool reports problem scores for each question (excluding demographic questions). The problem score represents the overall proportion of patients/parents, who, according to their response, have indicated that a particular aspect of their care can be improved. For example, for the following question “Did staff explain what they were doing in a way you could understand?”, responses “yes, to some extent/sort of” and “No”, are combined together to create a single problem score.

Project Setting and Population

This project was conducted in the ED of the Alberta Children’s Hospital, a tertiary care children’s hospital in Calgary, Alberta which has an annual census of over 70 000 patients. The project population included all patients aged 0 to 17 years and their parent/guardians that presented to the ED during project shifts. We excluded patients and their parent/guardian if (a) the patient required resuscitation; (b) the parent/legal guardian was not present; (c) parents/guardians were unable to understand the English language consent form/survey; (d) the patient presented with suspected non-accidental trauma; (e) they were eligible for other concurrent research studies in the ED.

This project was reviewed according to local policies as a Quality Improvement (QI) initiative and was deemed by the Research Ethics Board to be outside of the review mandate (13).

Project Protocol

Potentially eligible parent/guardian participants were identified, screened, and enrolled by trained project assistants (PAs). Project assistants approached triaged patients over 21 randomly selected day (0800-1600), evening (1600-0000), and night (0000-0800) shifts (7 each). In order to reduce the impact of confounding factors such as changes in staffing and ED processes, we completed the surveys over a limited time period from December 2016 to January 2017. After screening and obtaining consent from each parent, and assent for all children older than 7 years of age, the PAs provided participating parents and patients with paper surveys to complete over the course of their hospital visit. Children ≥8 years old were given their own child/adolescent version of the PREM survey to complete, separate from their parent, whereas children younger than 8 years of age filled out the survey with their parent. Patients/parents who did not complete the survey in the ED were emailed a copy of the survey(s). Those who did not complete the survey by the evening after their ED visit received a call and had the opportunity to complete the survey over the phone with a PA or by e-mail. The PA’s entered all paper, email and phone responses into REDCap, a secure online database (14).

Demographic data, including patient age, acuity (Canadian Triage Acuity Scale), and number of patients in ED at time of index patient triage were collected from administrative data using the patient’s PHN. The data were then combined with the deidentified REDCap (https://projectredcap.org/resources/citations/) database using the patient study ID.

Outcome Measures

The primary outcome measures were the overall problem scores for all respondents combined and for child and parents separately. The secondary outcome measures were the problem scores reported by: (a) time of arrival (day [triage time 8 am to 16 pm], evening [triage time 16 to midnight], and night [triage time midnight to 8 am] and (b) ED census at time of enrollment (high census [median number of patients in ED at time of enrollment ≥50%] vs low census [median number of patients in ED at time of enrollment <50%]).

Sample Size and Analysis

The survey developer recommends at least 50 to 100 complete responses to achieve meaningful data for service improvement. (12) In order to compare responses between children and parents, we aimed for 150 surveys completed by a parent and 150 surveys completed by a child for a total sample size of approximately 300. This sample size would enable us to detect a difference of 15% in problem scores for questions between parents and children (with a power of 80% and an α of .05). There is no established clinically meaningful difference in problem scores for the PREM measure, but in consultation with the local QI team we determined that identifying problem scores with a difference of 15% between children and parents would be useful in targeting improvement efforts.

Continuous variables were described using medians and interquartile ranges (IQRs) and dichotomous variable using proportions with a 95% confidence intervals (CIs).

Results

A total of 432 families were approached (Figure 1). Two hundred ninety families, including 123 children above the age of 8, agreed to participate. Overall 237 (82%) parents and 109 (89%) children completed the surveys.

Figure 1.

Flow chart of participant enrollment.

Primary Outcomes

Combined

The areas with the highest problem scores identified by both parents and children were: having enough to do while waiting to be seen (53.5; 95% CI: 48.1, 58.8), when to restart usual activities (34.7; 95% CI: 29.7, 40.0), having everything needed while waiting (34.4; 95% CI: 29.4, 39.7), the main reason of the visit being dealt with well (30.9; 95% CI: 26.2, 36.1), doctors and nurses explaining what was wrong (29.8; 95% CI: 25.1, 34.9), and length of wait to be seen (28.3; 95% CI: 23.7, 33.4; Table 1).

Table 1.

Problem Scores for Children and Parents Combined.

Survey Questions	Parents and Child Problem Score (95% CI)	Parents Only Problem Score (95% CI)	Child Only Problem Score (95% CI)
1. If an ambulance was called for your child, did the ambulance staff/paramedics explain what was happening in a way you could understand?	28.2 (15.5-45.1)	32 (16-55)	21.4 (5.7-51)
2. Overall, how well do you think the ambulance staff/paramedics looked after your child?	0.0 (0.0-11.2)	0.0 (0.0-16.6)	0.0 (0.0-26.8)
3. How did you feel about how long you had to wait to be seen?	28.3 (23.7-33.4)	26.6 (21-33)	32.1 (24-42)
4. While you were waiting, did someone keep you informed about what was happening?	19.7 (15.7-24.3)	27.1 (16.6-27.4)	15.6 (9.6-24.1)
5. Was there enough for your child to do when you were waiting to be seen (such as toys, games, and books)?	53.5 (48.1-58.8)	54 (47.4-60.5)	52.3 (42.6-61.9)
6. Was there everything you needed while you waited (such as food and drink, toilets, baby changing facilities, etc)?	34.4 (29.4-39.7)	36.7 (30.6-43.2)	29.4 (21.2- 39.0)
7. Was your child looked after while you waited (eg, were they given pain relief, blankets or sick bowls, etc if needed)?	22 (17.8-26.8)	22.8 (17.7-28.8)	20.2 (13.3-29.2)
8. In your opinion how clean was the waiting area?	8.7 (6.0-12.3)	7.2 (4.4-11.4)	11.9 (6.8-19.9)
9. Did the doctors and nurses that you saw explain what they were doing in a way you could understand?	25.7 (21.3-30.7)	19 (14.3-24.7)	40.4 (31.2-50.2)
10. Did the doctors and nurses that you saw explain what was wrong with your child in a way you could understand?	29.8 (25.1-34.9)	24.5 (19.2-30.5)	41.3 (32.1-51.1)
11. Do you think that the doctors and nurses did everything they could to calm and comfort you and your child?	22 (17.8-26.8)	21.9 (17.0-27.9)	22 (14.9-31.2)
12. If your child was in pain, do you think the doctors and nurses did everything they could to help with your child’s pain?	24.3 (19.9-29.2)	21.1 (16.2-27.0)	31.2 (22.9-40.9)
13. Were you and your child given enough privacy when they were being examined and treated?	23.4 (19.1-28.3)	17.3 (12.8-22.9)	36.7 (27.8-46.5)
15. Did a member of staff tell you when your child could re-start their usual activities, such as playing sports or returning to school?	34.7 (29.7-40.0)	28.7 (23.1-35.0)	47.7 (38.1-57.5)
16. Did staff tell you what you should watch out for at home after your child’s visit?	26.6 (22.1-31.6)	24.1 (18.9-30.1)	32.1 (23.7-41.8)
17. Did staff tell you what to do or who to contact if you were worried about anything after your emergency visit?	20.8 (16.7-25.6)	13.1 (9.2-18.2)	17.4 (11.1-26.1)
18. Overall, did you receive enough information about your child’s condition and treatment?	14.5 (11.0-18.7)	15.2 (11-20.5)	12.9 (7.5-20.9)
19. Overall, how well do you think your child was looked after during their visit?	3.5 (1.9-6.1)	3.8 (1.9-7.3)	2.6 (0.7-8.4)
20. Was the main reason for your emergency visit dealt with well?	30.9 (26.2-36.1)	27.7 (22.3-34.1)	37.6 (28.7-47.4)

Child and parent separately

When survey results were separated by parent versus child responses the 2 groups were similar with respect to patient acuity (median Canadian Triage and Acuity Scale [CTAS] score of 3 for both parent survey [IQR = 1] and child survey [IQR = 0], and median ED census 41 [IQR = 20] for child surveys, and 39 [IQR = 20] for parents). There was, as expected, a difference in the median age of the child visiting the ED, with a median age of 11 for the child survey (IQR = 5) and 8 for parent survey (IQR = 9).

The problem scores highlighted a number of discrepancies between the parent and child surveys (Table 1). There were meaningful differences in problem scores for the questions assessing: doctors and nurses explaining what they were doing (parents: 19; 95% CI: 14.3, 24.7, child 40.4; 95% CI: 31.2, 50.2); doctors and nurses explaining what was wrong (parents 24.5; 95% CI: 19.2, 30.5, child 41.3; 95% CI: 32.1, 51.1); privacy when examined and treated (parents: 17.3; 95% CI: 12.8, 22.9, child: 36.7; 95% CI: 27.8, 46.5) and; when to restart usual activities (parents: 28.7; 95% CI: 23.1, 35.1, child: 47.7; 95% CI: 38.1, 57.5).

Time of day

Of the 346 survey respondents, 153 (44%) arrived during the day shift (0800-1600), 144 (42%) during the evening shift (1600-2400), and 49 (14%) during the night shift (2400-0800). The median CTAS score was 3 for the 3 different shifts (day IQR = 1, evening IQR = 0, and night IQR = 0). The median ED census was 39 (IQR = 16), 43.5 (IQR = 17), and 23 (IQR = 11), for day, evening, and night shifts, respectively. The median age in years of the children was 10 (IQR = 5) for day, 9.5 (IQR = 8.25) for evening, and 6 (IQR = 7) for night.

The proportion of respondents identifying wait times as a problem was higher during the night and evening shifts compared to the day (day: 19.6; 95% CI: 14, 27; evening: 35.4; 95% CI: 28, 44; night: 34.7; 95% CI: 22, 50; Table 2). In contrast, there were 5 survey areas in which the problem scores for respondents presenting during the night shift were notably lower than those presenting during the day and evening shift including: having everything needed while waiting; doctors and nurses explaining what was wrong; privacy when being examined and treated, being told what to watch out for at home and having the main reason for the visit dealt with well.

Table 2.

Problem Scores by Time of Day.

Survey Questions	Day Shift Problem Score (95% CI)	Evening Shift Problem Score (95% CI)	Night Shift Problem Score (95% CI)
1. If an ambulance was called for your child, did the ambulance staff/paramedics explain what was happening in a way you could understand?	25 (6.7-57.1)	29.2 (13.4-51.3)	33.3 (1.8-87.5)
2. Overall, how well do you think the ambulance staff/paramedics looked after your child?	0 (0.0-30.1)	0 (0.0-17.2)	0 (0.0-69.0)
3. How did you feel about how long you had to wait to be seen?	19.6 (14-27)	35.4 (28-44)	34.7 (22-50)
4. While you were waiting, did someone keep you informed about what was happening?	13.1 (8.4-19.7)	22.9 (16.8-30.4)	30.6 (18.7-45.6)
5. Was there enough for your child to do when you were waiting to be seen (such as toys, games, and books)?	48.7 (41.5-57.8)	59 (50.5-67.1)	49 (34.6-63.5)
6. Was there everything you needed while you waited (such as food and drink, toilets, baby changing facilities, etc)?	30.7 (23.7-38.8)	44.4 (36.2-53)	16.3 (7.8-30.2)
7. Was your child looked after while you waited (for example, were they given pain relief, blankets or sick bowls, etc if needed)?	18.3 (12.7-25.5)	27.8 (20.8-36.0)	16.3 (7.8-30.2)
8. In your opinion how clean was the waiting area?	2.6 (0.8-7.0)	14.6 (9.5-21.7)	10.2 (3.8-23.0)
9. Did the doctors and nurses that you saw explain what they were doing in a way you could understand?	23.5 (17.2-31.2)	30.6 (23.3-38.9)	18.4 (9.2-32.5)
10. Did the doctors and nurses that you saw explain what was wrong with your child in a way you could understand?	30.1 (23.1-38.1)	34 (26.5-42.5)	16.3 (7.8-30.2)
11. Do you think that the doctors and nurses did everything they could to calm and comfort you and your child?	19 (13.3-26.3)	27.1 (20.2-35.2)	16.3 (7.8-30.2)
12. If your child was in pain, do you think the doctors and nurses did everything they could to help with your child’s pain?	22.2 (16.1-29.8)	28.5 (21.4-36.7)	18.4 (9.2-32.5)
13. Were you and your child given enough privacy when they were being examined and treated?	26.8 (20.1-34.7)	25.7 (18.9-33.8)	6.1 (1.6-17.9)
15. Did a member of staff tell you when your child could re-start their usual activities, such as playing sports or returning to school?	35.3 (27.9-43.5)	36.1 (28.4-44.6)	28.6 (17.0-43.5)
16. Did staff tell you what you should watch out for at home after your child’s visit?	29.4 (22.5-37.4)	29.9 (22.7-38.1)	8.2 (2.7-20.5)
17. Did staff tell you what to do or who to contact if you were worried about anything after your emergency visit?	10.5 (6.3-16.7)	22.2 (15.9-30.1)	4.1 (0.7-15.1)
18. Overall, did you receive enough information about your child’s condition and treatment?	11.8 (7.3-18.2)	18.1 (12.3-25.5)	12.2 (5.1-25.5)
19. Overall, how well do you think your child was looked after during their visit?	1.3 (0.2-5.1)	6.9 (3.6-12.7)	0 (0-9.1)
20. Was the main reason for your emergency visit dealt with well?	32.7 (25.5-40.8)	34.7 (27.1-43.2)	14.3 (6.4-27.9)

Census

The problem scores (for parent and child combined) were similar when we compared the results for participants enrolled at a time of “high” (≥median census) versus “low” census (

Table 3.

Problem Scores at “High” (>= Median) versus “Low” (

Survey Questions	Above Median Problem Score (95% CI)	Below Median Problem Score (95% CI)
1. If an ambulance was called for your child, did the ambulance staff/paramedics explain what was happening in a way you could understand?	28.6 (12.2-52.3)	27.8 (10.7-53.6)
2. Overall, how well do you think the ambulance staff/paramedics looked after your child?	0.0 (0.0-19.2)	0.0 (0.0-21.9)
3. How did you feel about how long you had to wait to be seen?	32.2 (25.6-39.6)	24.1 (18.0-31.5)
4. While you were waiting, did someone keep you informed about what was happening?	17.8 (12.7-24.3)	21.7 (15.8-28.9)
5. Was there enough for your child to do when you were waiting to be seen (such as toys, games, and books)?	48.9 (41.4-56.4)	58.4 (50.5-65.9)
6. Was there everything you needed while you waited (such as food and drink, toilets, baby changing facilities, etc)?	38.3 (31.3-45.9)	30.1 (23.4-37.8)
7. Was your child looked after while you waited (for example, were they given pain relief, blankets or sick bowls, etc. if needed)?	21.1 (15.5-27.9)	22.9 (16.9-30.2)
8. In your opinion how clean was the waiting area?	9.4 (5.8-14.9)	7.8 (4.4-13.3)
9. Did the doctors and nurses that you saw explain what they were doing in a way you could understand?	25.0 (19.0-32.1)	26.5 (20.1-34.0)
10. Did the doctors and nurses that you saw explain what was wrong with your child in a way you could understand?	31.1 (24.6-38.5)	28.3 (21.7-35.9)
11. Do you think that the doctors and nurses did everything they could to calm and comfort you and your child?	24.4 (18.5-31.5)	19.3 (13.7-26.3)
12. If your child was in pain, do you think the doctors and nurses did everything they could to help with your child’s pain?	25.0 (19.0-32.1)	23.5 (17.4-30.8)
13. Were you and your child given enough privacy when they were being examined and treated?	25.6 (19.5-32.7)	21.1 (15.3-28.2)
15. Did a member of staff tell you when your child could re-start their usual activities, such as playing sports or returning to school?	32.2 (25.6-39.6)	37.4 (30.1-45.2)
16. Did staff tell you what you should watch out for at home after your child’s visit?	30.0 (23.5-37.3)	22.9 (16.9-30.2)
17. Did staff tell you what to do or who to contact if you were worried about anything after your emergency visit?	20.0 (14.6-26.7)	8.4 (4.9-14.0)
18. Overall, did you receive enough information about your child’s condition and treatment?	15.0 (10.3-21.3)	13.9 (9.2-20.3)
19. Overall, how well do you think your child was looked after during their visit?	4.4 (2.1-8.9)	2.4 (0.8-6.5)
20. Was the main reason for your emergency visit dealt with well?	31.1 (24.6-38.5)	30.7 (23.9-38.4)

Discussion

The primary aim of this project was to understand patient and parent experience in a pediatric ED. Similar to what has been reported in the literature, our study reaffirmed that there are identifiable factors that influence patient experience (1,4 –7,10,15 –18). Areas identified as having room for improvement included: having enough to do while waiting to be seen, instructions on when patients could restart normal activities, availability of necessities while waiting, the main reason for the visit being dealt with well, receiving an explanation on what is wrong and wait time before being seen.

When asked about the overall experience, (“Overall, how well do you think your child was looked after during their visit) the problem score was low for both parents and children. However, when asked about specific aspects of care 20% or more of respondents indicated that there was room for improvement for 15 out of 20 questions. Similar to prior work in patient satisfaction (19), these results highlight the importance of assessing specific aspects of care and not just an overall or summary experience measure.

Also in keeping with prior work, we found that there were differences in reported experience between children and their parents. For example, a recently published survey of children and parents in a pediatric ED reported differences between children and their parents or with respect to wait longer than expected, explanations of the procedures carried out, treatment for pain and privacy during the visit (16). Our findings were similar, with children having higher problem scores than their parents for doctors and nurses explaining what they are doing, explaining what is wrong, telling them when to restart their usual activities and being given enough privacy when examined and treated. The previous study used a modified, unvalidated version of the Picker PREM, but the similarity in results confirms that there are differences in experience reported by children as compared to their parents during an ED visit. A literature review on patient satisfaction investigations in the ED setting recommended that future work should measure the satisfaction of children themselves and not just the adult patient population or adults who accompany children to the ED (19). Our work, combined with others, indicate that the same is true in patient experience. Future work measuring and improving patient experience in pediatric settings should include children as well as their parents.

The discrepancy between the problem score for children and parents in our project was particularly high for the question assessing whether children were given enough privacy when examined and treated. The study ED has a mix of single patient rooms as well as treatment spaces separated by curtains and chairs with no separation. We did not collect data on where the respondents were examined, nor did we collect data on the specific aspects of privacy they felt could be improved, including whether or not they were examined without their parent in the room. However, our findings are aligned with prior work done in adult ED settings which demonstrated that ED patients experience breaches of privacy and confidentiality during their ED stay and perceive significantly less auditory and visual privacy in ED treatment areas with curtains compared with a room with solid walls (20,21). Interventions to improve ED patient privacy, including redesigning the ED environment, process management, access control and staff education and training, have been shown to be effective in improving patient’s perception of privacy and satisfaction (20). The discrepant rating between parents and children regarding privacy during examination highlight that further work is required to understand patient privacy needs in the pediatric setting.

With respect to time of visit, combined parent and child scores were lower during the night, compared to day or evening with the exception of questions pertaining to wait time. The reason for this finding is unclear. The patient census was lower at night; however, with the exception of one question, we did not see a difference in experience based on census. Most of the surveys at night were completed by parents, and parents reported overall lower problem scores so this may account for some of the difference between night and day/evening visits. There were only 49 surveys from participants who visited at night which limits further analysis on this finding. Further work with a larger sample size is required to better delineate the impact of time of presentation on patient experience in the pediatric setting.

Although prior work in the adult ED setting identified a link between crowding and patient satisfaction (3), with the exception of one question, we did not identify a difference in patient experience based on ED census. However, the item where we did find a difference related to discharge instructions, which in a pediatric ED where the majority of patients are discharged is particularly important. Although crowding measures have been linked to quality of care (22) there is no consensus on which crowding measures to use, particularly in the pediatric ED setting. Our ability to measure the impact of crowding was also hampered by the fact that we enrolled patients over one month period which likely decreased the variability and potential impact of census. Further exploration with a larger sample size, and various census cut points is required to further understand the impact of census on patient experience.

Limitations

One of the limitations of this project is the limited sample size, particularly for patients who arrived during the night. Our decision to enroll parents/children over a limited time period, although it reduced the impact of confounding factors, also limited our ability to measure the impact of census, and seasonality. Finally, this project took place in a single center as a quality improvement initiative, and therefore the results are not intended to be generalizable to all pediatric centers.

Conclusion

The results of this project provide insight into patient reported experience in the pediatric ED. We identified several problem areas where there is the opportunity to improve patient experience. We also identified areas where children reported higher problem scores than their parents. This highlights the importance of continuing to include children when assessing patient experience in a pediatric setting.

With respect to knowledge generation future work should include assessing patient experience across multiple ED settings to build a better understanding of the factors, including volume, seasonality and time of visit, that impact patient experience in the pediatric ED setting. The next steps in our quality improvement efforts will include designing interventions to improve experience, particularly with respect to privacy during treatment and examination and providing child-focused communication.