How to implement a PEWS in a resource-limited setting: A quantitative analysis of the bedside-PEWS implementation in a hospital in northeast Brazil.

OBJECTIVES: Quantitative analysis of the implementation of the bedside paediatric early warning system (B-PEWS) in a resource-limited setting. The B-PEWS serves to pre-emptively identify hospitalised children who are at risk for cardiopulmonary arrest and subsequently to provide critical care in time.
METHODS: We performed a retrospective review through the medical data records of patients after discharge from the paediatric ward of a philanthropic hospital in Brazil. Nurses' performance using the system was measured with various parameters.
RESULTS: A total of 499 patients were included, and a total of 8024 scores were checked. During the 21-week research period, the implementation rate increased significantly from 66.5% (SD 26.0) in Period 1 to 93.1% (SD 16.6) in Period 2. The number of scores that resulted in a correct total score went from 7.5% in Period 1 to 32.2% in Period 2, p < 0.001. There was an improvement in the correct choice of age group between the two periods (from 32.2% to 53.4%). There was no difference in the mean admission time of patients in the two periods: in the first period 4.8 days (SD 2.9) and in the second period 4.8 days (SD 4.1).
CONCLUSIONS: It is possible to implement a PEWS in resource-limited settings while achieving high implementation rates. However, this is a time- and energy-consuming process. Having an active and involved team that is responsible for implementation is key for a successful implementation. Factors that likely hindered implementation were a large change in workflow for the nursing staff, non-native speakers as main investigators.

INTRODUCTION

Children deteriorate rapidly once their clinical conditions start to worsen. Thus, monitoring children's health status plays an important part role in reducing child mortality [1]. A systematic review showed that paediatric alert criteria are often not validated, inconsistent and subjective and do not consider timely signalling of serious adverse events [2]. Furthermore, in many cases, cardiopulmonary arrest in children can be avoided [3, 4, 5, 6].

Previous research has shown that patient safety in Brazilian hospitals has room for improvement, especially regarding communication [7]. It has been suggested that standardisation of communicating patient information would be an effective method for improving communication in health care [7, 8]. Paediatric early warning systems (PEWS) are clinical tools used to detect early signs of clinical deterioration in hospitalised children in a systematic manner. There are numerous validated PEWS with varying accuracy in predicting deterioration [2, 9, 10, 11]. The validation and research on PEWS are mostly done in high‐resource settings [2, 4, 9, 10, 11, 12], while the literature on PEWS in resource‐limited settings is scarce [13]. Implementing PEWS in high‐resource settings has not shown to result in significant reduction in child mortality, presumably because the baseline mortality is too low to measure the impact of a PEWS, but it results in better child attention [14, 15].

In 2009, Parshuram et al. developed a PEWS called bedside paediatric early warning system (B‐PEWS). When B‐PEWS was developed, the aim was a not too complex, easy‐to‐use system, requiring little equipment. Also, bedside‐PEWS is a relatively objective system when compared to other validated PEWS systems [16]. B‐PEWS is validated to detect deterioration in hospitalised children early on [10, 17]. The validation of B‐PEWS was done in HRS [10]. Thus far, no validation has been done of B‐PEWS in resource‐limited settings. This is an exploratory study aiming to investigate the process of implementation and the obstacles encountered during implementation of the B‐PEWS system in a resource‐limited setting.

METHODS

This is a retrospective chart review of patients after discharge from the paediatric ward of a philanthropic hospital in Brazil.

Setting

The population is composed of children admitted to clinical treatment by SUS (Unique System of Health in Brazil). The hospital is located in Aracaju, the capital of Sergipe, a small federal unit in Brazil (21,910 km2). Its Human Development Index is 0.681, life expectancy at birth is 72.1 years and the infant mortality rate is 18 per 1000 live births. Approximately 50% of the population lives below the poverty index [18].

The paediatric ward at this hospital focuses on the treatment of infectious diseases and acute conditions. It receives patients in the age range of 29 days to 13 years of age. There are 22 beds, one in an isolation room. For escalation of care in this hospital, there was a stabilisation ward and a paediatric intensive care unit available. This comprised seven beds, including an isolation room. The nursing staff on the paediatric ward consisted of four auxiliary nurses (técnicos de enfermagem), one nurse and three‐floor physicians. Typically, one auxiliary nurse would be responsible for five patients. Documentation in this hospital was done on paper forms and accumulated on clipboards. After the visits are over, the floor physicians leave the hospital. One oximeter and one aneroid sphygmomanometer were present in the ward of 22 patients.

Prior to this study, there was no protocol for escalation of care for hospitalised patients. The decision to escalate care was based on clinical assessment. No formalised previous paediatric intensive care unit consultation, rapid response team or code blue team were in place within the hospital. Auxiliary nurses evaluated patients’ heart rate and temperature every six hours, and if a patient was admitted with breathing issues, oxygen saturation was measured each six hours also.

For this research, we chose to implement the B‐PEWS, as proposed by Pasrhuram et al. [17] To calculate, a single B‐PEWS score requires a patient's heart rate, systolic blood pressure, capillary refill time, respiratory frequency, respiratory effort, oxygen saturation and the oxygen therapy that the patient receives. Respiratory effort comprises four categories: normal, mild increase, moderate increase and severe increase/any apnoea. Oxygen therapy comprises three categories: room air, any – <4 L/min or <50% and ≥4 L/min or ≥50%. The following B‐PEWS scores were calculated following the response algorithm. The response algorithm used in this study is shown in Table 1a and b. The first time a score was calculated, and the ‘initial score’ table needed to be used. If the next score that was calculated fell in the same row as the previous score, the ‘subsequent score’ table needed to be used. If a newly calculated score did not fall into the same row as the most recent score, the ‘initial score’ table needed to be used again.

TABLE 1

Recommendations of care, (a) initial score table (b) subsequent score table

(a)
Initial score	Recommendations
0–2	Assess/rescore: 4 h
3–4	Assess/rescore: 2–4 h	Charge nurse review: <8 h	Provider team review: 8 h
5–6	Assess/rescore: 1–2 h	Write nursing Progress Note	Charge nurse review: <4 h	Provider team review: <4 h	Continuous monitoring: consider
7–8	Assess/rescore:15–60 min	Write nursing Progress Note	Charge nurse review: <2 h	Provider team review: <2 h	Continuous monitoring: consider	Consider escalation of care
>8	Assess/rescore: 15–30 min	Write nursing Progress Note	Charge nurse review: <15 min	Provider team review: <15 min	Continuous monitoring: continuous	Consider escalation of care

Implementation

Implementation was mostly carried out by the two main investigators, who were two Dutch undergraduate medical students (non‐native for Portuguese). The implementation team further consisted of a professor of paediatrics who worked in another hospital in the same city, a neonatologist who worked in the same hospital, but not on the paediatric ward and the head nurse of the paediatric ward.

The implementation consisted of many interactive presentations given to the auxiliary nurses. Presentations were given explaining the use of B‐PEWS, the importance of B‐PEWS and detailed instructions on measuring the vital signs required for B‐PEWS.

The main investigators visited the hospital two to three times a week to collect data and to check on the implementation in the paediatric ward. The main investigators would review the calculated B‐PEWS scores that the present auxiliary nurses had calculated. Correct scores would be rewarded with candy, and incorrect scores were explained.

To motivate the auxiliary nurses, a list of the five auxiliary nurses that made the least amount of errors, as well as a sheet with the weekly implementation rate was kept on the wall in the ward. When necessary, small adjustments to the system were made to make it better suited for the ward. After every adjustment to the system, the main investigators made sure to give presentations explaining all the changes to all of the auxiliary nurses.

Because of the limited time span of the study, it was decided to divide the implementation period, which lasted from November 2018 until and including April 2019, into two periods: the first period from December 2018 until and including January 2019 and the second period from February 2019 until and including April 2019. This allowed for comparison between the initial, start‐up phase and the phase in which the nursing staff had gotten used to the system. This in turn allowed us to analyse whether there was any progression over time in a broad sense, without the results being hazed by small, random variances between weekly and monthly data. The month of November was used for schooling the nurses. All documents regarding B‐PEWS that were directed at the nursing staff were in Portuguese.

Data collection

All patients admitted from November 2018 until and including April 2019 in the paediatric ward participated in the study. Data were gathered after discharge or transfer to another ward or another hospital. Since no patients who participated in this study died during the research period, no data were collected from patients who died. The same applied for resuscitations. The month when the patient was dismissed was the month in which his/her data were included. Information obtained was the patient's ID number, whether the patient was admitted to higher‐level care (stabilisation ward, paediatric intensive care unit or another hospital) or discharged from hospital care, whether a patient required resuscitation during admission, as well as date and time the patient was admitted and discharged. A sheet was made on which the patient B‐PEWS scores were collected. This sheet was consulted for performed B‐PEWS scores and was transcribed into an Excel tool which was developed to calculate the amount of errors made in scoring and adherence to the response algorithm. The types of errors that were tracked were mistakes in appointing the correct score to a measured vital parameter, mistakes in adding up sub‐scores, mistakes that led to a final B‐PEWS score that corresponded to a different response algorithm, mistakes using the wrong B‐PEWS scoring table in regard to the patient's age, the amount parameters and scores that were left blank and the amount of untraceable mistakes, for example an incorrect total score with missing sub‐scores. Also, the number of needed B‐PEWS scores was calculated using the admission time and the response algorithm. The response algorithm provides the time frame in which a new B‐PEWS score needs to be calculated. Therefore, the number of needed B‐PEWS could be calculated using the B‐PEWS scoring forms and the date and time of admission and of discharge. Every month, the patients’ data of the hospital were checked, to see if the number of patients in the data collection was consistent with the number of discharges in the paediatric ward.

​Due to the retrospective nature of this study, consent was not obtained from patients whose data were collected for this study.

After the implementation period, the ‘B‐PEWS difference’ was calculated. This was done by subtracting the number of calculated B‐PEWS scores of the number of needed B‐PEWS scores. The B‐PEWS difference gives insight into the adherence of the paediatric ward staff to calculating B‐PEWS scores as often as the response algorithm dictates. The implementation rate was calculated by dividing the number of calculated B‐PEWS scores by the number of needed B‐PEWS scores.

When used properly, the number of calculated B‐PEWS scores was equal to the number of needed B‐PEWS scores. If every score that was calculated for a patient, was calculated within the recommended time frame, the number of calculated B‐PEWS scores would be equal to the number of adherence to ‘assess/rescore’ plus one. If both of the above‐mentioned scenarios occurred for the same patient and no incorrect scores were registered for this patient, this was considered as the best scenario. The number of times that the best scenario occurred was registered as ‘number of best scenario’.

For analysis purposes, patients were categorised as having low risk, moderate risk or high risk after having collected the patients’ data. Low‐risk patients never had a B‐PEWS score higher than two and were not admitted to the stabilisation ward or the paediatric intensive care unit. Moderate‐risk patients at least once had a score higher than two or admitted to the stabilisation ward or the paediatric intensive care unit. High‐risk patients at least once had a score higher than two and were admitted to either the stabilisation ward or the paediatric intensive care unit.

We used frequencies and percentages to describe categorical variables as well as mean and standard deviation to describe continuous variables. We used chi‐square and Exact Fisher test to evaluate the hypothesis of independence between categorical variables. We used Shapiro–Wilks test to evaluate the hypothesis of adherence of continuous variables to normal distribution. Once all tested variables rejected this hypothesis, we used the Mann–Whitney test to evaluate the hypothesis of equality of central tendencies measurements. We adopted a 5% significance level in all tests, and we used software R core team 2021 in all analyses.

The study was approved by the Research Ethics Committee of the Federal University of Sergipe (Opinion nº 2,897,526, CAAE nº 97951218.6.0000.5546).

RESULTS

In the first period, 205 children were discharged from the paediatric ward of HSI, and in the second period, 294 children were discharged. The implementation rate of the B‐PEWS in the paediatric ward during the 21‐week period increased from 56% to over 90% from February 2019 on (Figure 1). The rate increased significantly from 66.5% (SD 26.0) in Period 1 to 93.1% (SD 16.6) in Period 2 (Table 2A,B). The B‐PEWS difference was significantly lower in Period 2 compared with Period 1 (p < 0.001) (Table 2A,B).

FIGURE 1

Implementation rate per month.

TABLE 2A

Comparison of the use of the system between the 1st and 2nd assessment period of the B‐PEWS score

	Period 1	Period 2	p‐value
No. of B‐PEWS calculations, mean (SD)	14.7 (12.0)	18.1 (14.1)	0.002W
No. needed B‐PEWS calculated, mean (SD)	22.2 (15.5)	19.6 (15.1)	0.028W
Implementation rate, mean (SD)	66.5 (26.0)	93.1 (16.6)	<0.001W
No. of sum >2, n (%)	52 (26.1)	70 (24.0)	0.596C

TABLE 2B

Comparison of accuracy of use between the 1st and 2nd assessment period of the B‐PEWS score

	Period 1, n (%)	Period 2, n (%)	p‐value
Heart rate incorrect	130 (65.3)	106 (36.3)	<0.001C
Systolic blood pressure incorrect	163 (81.9)	137 (46.9)	<0.001C
Capillary refill time incorrect	28 (14.1)	9 (2.1)	<0.001C
Respiratory rate incorrect	102 (51.3)	86 (29.5)	<0.001C
Respiratory effort incorrect	12 (6.0)	7 (2.4)	0.055F
Oxygen saturation incorrect	24 (12.1)	15 (5.1)	0.006F
Supplemental oxygen incorrect	13 (6.5)	15 (5.1)	0.555F
Wrong recommendations used because of incorrect score	86 (43.2)	58 (19.9)	<0.001F
No. of total score adequately calculated	14 (7.5)	94 (32.2)	<0.001C
Use of correct age group	64 (32.2)	156 (53.4)	<0.001C
No. blanks in parameters	77 (38.7)	35 (12.0)	<0.001C
No. untraceable mistakes	55 (27.6)	1 (0.3)	<0.001F
Adherence to ‘Assess/rescore’ in no., percentage mean (SD)	65.2 (21.1)	79.3 (16.4)	<0.001W
No. of best scenario	1 (0.5)	27 (9.2)	<0.001F
Average admission time in days, mean (SD)	4.8 (2.9)	4.8 (4.1)	0.319W

Abbreviations: SD, standard deviation; n, absolute frequency; %, relative per cent frequency; W, Mann–Whitney test; C, chi‐square test; F, Fisher exact test; B‐PEWS difference = nr. needed BPEWS – nr. of B‐PEWS calculations.

Among the parameters evaluated with the B‐PEWS score, heart rate, systolic blood pressure, capillary refill time (CRT) and respiratory rate showed a reduction in the number of incorrect scores in the second period (p < 0.001) (Table 2A,B). Respiratory effort, oxygen saturation and the need for supplemental oxygen did not show a statistically significant change in the number of incorrect scores in Period 2 vs. Period 1.

The numbers of parameters that were left blank in B‐PEWS score forms and the number of ‘untraceable mistakes’, were significantly smaller in Period 2 compared with Period 1 (p < 0.001) (Table 2A,B).

Adherence to the ‘assess/rescore’ recommendation increased significantly (p < 0.001) in the second period, going from 65.2% (SD 21.1) to 79.3% (SD 16.4). The number of performed scores that resulted in a correct total score went from 7.5% in Period 1 to 32.2% in Period 2, p < 0.001. There was an improvement in the correct choice of age group between the two periods (from 32.2% to 53.4%). There was no difference in the mean admission time of patients in the two periods: in the first period 4.8 days (SD 2.9) and in the second period 4.8 days (SD 4.1) (Table 2A,B).

In Table 3, we compare the use of the B‐PEWS scores, stratified by the clinical risk of patients, during the two separate assessment periods. There was a significant decrease between the first and the second period in the B‐PEWS difference, the amount of wrong recommendations used because of an incorrect score. There was a significant increase in the percentage of adherence to ‘assess/rescore’, in patients with low and moderate risk (p < 0.001). Further, the number of adequately calculated total scores and the best scenario increased significantly between the two periods for patients with a low risk. The number of adequately calculated total scores and best scenarios did not differ significantly between Period 1 and Period 2 for patients of moderate risk. For patients of high risk, the B‐PEWS difference, the wrong recommendations used because of incorrect score, the number of correctly calculated total scores and the adherence to the ‘assess/rescore’ recommendation did not differ significantly between Period 1 and Period 2. In Periods 1 and 2, there were no high‐risk patients who experienced the best B‐PEWS scenario.

TABLE 3

Comparison of the use of B‐PEWS score stratified by clinical risk between the 1st and 2nd assessment period

	Period 1	Period 2	p‐value
Low‐risk children
Total number, n	144	224	–
B‐PEWS difference, mean (SD)	6.7 (7.2)	1.0 (1.9)	<0.001W
Wrong recommendations used because of incorrect score, n (%)	43 (29.3)	21 (9.5)	<0.001C
No. of total score adequately calculated n (%)	13 (9.2)	84 (38.2)	<0.001F
Adherence to ‘Assess/rescore’ in no., mean (SD)	8.6 (6.4)	13.4 (9.9)	<0.001W
No. of best scenario, n (%)	1 (0.7)	25 (11.4)	<0.001C
Moderate‐risk children
Total number, n	53	71	–
B‐PEWS difference, mean (SD)	9.8 (9.5)	3.0 (6.0)	<0.001W
Wrong recommendations used because of incorrect score, n (%)	40 (83.3)	34 (51.5)	<0.001C
No. of total score adequately calculated n (%)	1 (2.4)	8 (12.1)	0.149F
Adherence to ‘Assess/rescore’ in no., mean (SD)	14.6 (10.4)	21.9 (17.7)	0.014W
No. of best scenario, n (%)	0 (0.0)	2 (3.0)	0.508F
High‐risk children
Total number, n	2	5	–
B‐PEWS difference, mean (SD)	6.0 (5.9)	0.3 (1.9)	0.114W
Wrong recommendations used because of incorrect score, n (%)	3 (75.0)	3 (50.0)	0.571F
No. of total score adequately calculated, n (%)	4 (100)	4 (66.7)	0.467F
Adherence to ‘Assess/rescore’ in no., mean (SD)	26.0 (31.4)	7.0 (8.4)	0.067W
No. of best scenario, n (%)	0 (0.0)	0 (0.0)	–

Abbreviations: SD, standard deviation; n, absolute frequency; %, relative per cent frequency; W, Mann–Whitney test; C, chi‐square test; F, Fisher exact test.

DISCUSSION

B‐PEWS implementation in a hospital working with the SUS resulted in a high implementation rate near the end of the six‐month implementation period. Nonetheless, the implementation showed some limitations regarding the fraction of erroneous scores that were being calculated. At the end of the 21‐week implementation period, the implementation rate was high; however, the fraction of erroneous scores shows that the learning curve for the nursing team was still ongoing. This study hopes to give an insight into the different aspects of the implementation process and thereby aid in the future implementations of B‐PEWS and similar systems.

In this study, the implementation rate was 90% and higher during ten consecutive weeks and reached 100% in the 18th week. All types of mistakes declined significantly during implementation, except for mistakes in appointing scores to respiratory effort and supplemental oxygen, which had the lowest percentage of errors to begin with. However, the number of adequately calculated total scores accounted for only 32.2% of all scores calculated in Period 2, which resulted in a score with a different response algorithm in 19.9% of all scores in Period 2. Also, the number of scores that was calculated within the time frame recommended by the response algorithm remained low at 16.4% in Period 2. In moderate‐risk children, the performance is less compared with low‐risk children. For the category low‐risk children, the auxiliary nurses do not need to change their routine, but moderate‐risk children require more frequent check‐ups. Presumably, this is the reason that the performance in moderate‐risk children is lower. The high‐risk category contained an insufficient number of patients to draw any conclusions.

The amount of erroneous B‐PEWS scores during Period 2 could be attributed to the short duration of the study. Although there were, at the time of writing this article, no data available on the amount of errors made with any PEWS, following the trend for longer could lead more correctly calculated scores. Furthermore, in high‐resource settings, PEWS are used on computers, which reduces human errors; however, in the paediatric ward, there were no computers available. Optional in resource‐limited settings could be a calculating application for the scores which can be opened on smartphones or tablets. During this study, this was not feasible, because electronical devices were scarce.

Limitations of this study included lack of equipment needed to perform the clinical assessments. Another issue was the language proficiency of the main investigators during implementation which created a barrier for this study. It was mostly overcome by interference of the head nurse. Further, there is a possibility that the Hawthorne effect played a role in achieving the high implementation rate, which could make the implementation rate of this study not representative for other implementation scenarios. The same applies for any confounding factors. No patient demographics were obtained in this study, so there is no way of adjusting for any confounding patient demographics.

Earlier research shows that the success of implementing clinical guidelines and systems can depend greatly on how much the new practitioner workflow differs from the prior workflow [12, 19]. The results from this study comply with these findings, as the implementation rate in the paediatric ward drastically improved after adjusting it to resemble the previous manner of monitoring the patients more closely. We would recommend examining the setting in which a PEWS will be implemented so as to better fit the existing system(s).

Different methods were used to enhance implementation, and many of which were based on increasing the nurses’ enthusiasm for the system. Although not scientific, we would like to share what worked and what did not. First of all, the importance of an authority figure who is invested in the implementation and who is often present in the ward is necessary. In our study, this was the head nurse, who visited every shift, and watched over the system and the investigators. Including floor staff to the research, party helped to encourage the nurses to execute the system. Furthermore, constructive feedback and rewards (the candy) helped to maintain the affinity with the system. By keeping track of individual scores and overall performance (the top 5), the nurses were given insight on the effect of their efforts. The nurses were also encouraged to help the investigators to improve the system: the score tables were changed to graphs. Visuality helped to reduce mistakes, seen as the decrease in mistakes made in the second period. In this study, the doctors did receive a presentation to inform them, but there was lack of follow‐up. This led to unawareness of the investigators whether the doctors played an active part in the system and what kind of part this was. The investigators tried to get insight into the recommendations’ follow‐up informing the doctor in case of a higher score, but this proved to be difficult. In hindsight, including the doctors more in the follow‐up would probably have led to better adherence.

The head nurse and nursing staff would have to see the maintenance of the system through. To ensure longevity and replace the function of the investigators, nursing students were tasked to overlook the system under the supervision of the head nurse.

Other benefits that PEWS bring should not be overlooked, such as a universal language to quickly communicate the health status of patients between medical personnel within a paediatric ward, and between wards when transferred.

Hopefully, this study has broadened the scope for research on the implementation of PEWS’ in resource‐limited settings. By any degree, a system for monitoring is verifiable, whereas the effectiveness of any monitoring without a system is not easily verifiable. In that sense, the implementation of B‐PEWS in this setting, and probably many other resource‐limited settings, was an improvement on its own. Further results to be expected in the future are a more noticeable reduction in child mortality in resource‐limited vs high‐resource settings [20], the implementation of PEWS in resource‐limited settings has the potential of reducing overall personnel and equipment costs of hospital care through reduction in clinical deterioration events [14, 21]. The potential of PEWS in resource‐limited settings is yet to be explored.

CONCLUSION

Improving healthcare outcomes in resource‐limited settings by using PEWS starts with knowing where the hardships lie. To investigate fully, what the impact is on health care in the broad sense, for example adverse events in patients and healthcare team assessment, it is necessary to assure that the level of implementation is sufficiently high. Which factors play a role in achieving high levels of implementation is a key question. This research has predominantly shown certain factors which can aid in the implementation, for example more closely resemble a prior system of monitoring, having a fixed team working on the implementation, but also which factors to anticipate on which counterwork, for example lack of back up by higher management, language barriers and lack of funds. Ultimately, a high implementation rate was achieved, but further research is necessary to substantiate the potential positive impact of PEWS in resource‐limited settings, for example longitudinal healthcare improvements, cost reductions and nurse vigilance.