Development and Implementation of a Pediatric Nursing-Clinical Decision Support System for Hyperthermia: A Pre- and Post-test.

This article describes the development process and application of the Pediatric Nursing-Clinical Decision Support System for Hyperthermia. Firstly, we formed the Pediatric Nursing-Knowledge Base for Hyperthermia, which combines publicly available clinical practice guidelines and nursing routines of hyperthermia management. Then, following the nursing process framework, the system was developed using clinical decision support technology. Finally, a pre- and post-test were adopted to examine the effectiveness, usability, and feasibility before (1st to 31st of August 2018) and after (1st to 31st of December 2019) using the system. Its effectiveness was examined by analysis of nursing records' quality, including completeness of nursing assessment, timeliness of nursing diagnosis, individualization of nursing interventions, and timeliness of nursing evaluation. Its usability and feasibility were assessed using the Clinical Nursing Information System Effectiveness Evaluation Scale. There was a significant difference between the two groups in effectiveness, usability, and feasibility. Although the system was developed specifically for our hospital workflow and processes, the Pediatric Nursing-Knowledge Base for Hyperthermia and workflow for hyperthermia management in this study can be used as a reference to other hospitals.

Fever is one of the most common presenting complaints of children admitted to the hospital. Prolonged fever will increase basal metabolic rate and oxygen consumption, which will affect many organ systems such as cardiopulmonary, digestive, and nervous systems. Children's temperature rarely rises above 42°C without influencing a healthy nervous system.[1] Evidence suggests that fever is not dangerous and may be beneficial for the immune response to infection.[2] However, parents and even physicians and nurses who have a misconception about fever may worry about fever and believe it is a disease rather than a symptom.[3-5] Nurses, as the healthcare professionals who interact most with children and parents during hospitalization, are the main disseminators of hyperthermia nursing knowledge. However, in clinical practice, the evidence or knowledge of hyperthermia nursing management is scarce. Some nurses exhibit a corresponding lack of knowledge concerning fever. Inconsistent treatment approaches, along with fever phobia and a lack of knowledge, were exhibited in some studies.[4,6,7] Research on the application of nursing process to fever is rarely presented, so the problems that nurses encounter in the nursing process for fever are not clear. In our hospital, nurses focus mainly on cooling and pay less attention to the comfort and psychological state of children. Caregivers' behavior in the nursing process of fever may also be ignored. Moreover, nursing assessment, diagnosis, intervention, and evaluation were independent from each other and lack restriction in our original nursing information system; the nursing process for fever was easily interrupted.

A clinical decision support system (CDSS) is a knowledge-based information system that utilizes and integrates the characteristics of individual patients with a computerized knowledge base to generate patient-specific assessments and enhanced clinical decisions.[8] Basic actions of CDSSs include alerting, reminding, criticizing, interpreting, predicting, diagnosing, assisting, and suggesting.[9] Based on studies conducted from 1998 to 2004, Garg et al[10] demonstrated that 92% of CDSSs were used by physicians and not as much by nurses, similar to what was concluded by Piscotty and Kalisch[11] from studies between 1990 and 2013. Reasons for this include lack of evidence-based practice, not much availability of CDSS-related technology, and inability to store and reuse clinical data.[12,13]

There are some studies that apply CDSS to manage patients with fever in the emergency department.[14] Few studies focus on hospitalized patients, and fewer on hospitalized children. Fever is a medical diagnosis, and its corresponding nursing diagnosis is hyperthermia. We developed a new system called Pediatric Nursing-CDSS for Hyperthermia (PedN-CDSS-Hyperthermia), which combines evidence-based knowledge of pediatric hyperthermia care and uses the actions of CDSS to manage pediatric hyperthermia scientifically and systematically.

METHODS

This study consisted of two stages. The first stage was PedN-CDSS-Hyperthermia development, which was then continued by its preliminary examination of the implementation before (1st to 31st of August 2018) and after (1st to 31st of December 2019) using the PedN-CDSS-Hyperthermia. A pre-and-post study was used.

Setting Up a Multi-disciplinary Team

To address the needs of PedN-CDSS-Hyperthermia development, a collaborative multi-disciplinary team was set up, which consisted of several departments, including nursing, information and technology, quality control, Ewell Technology Company, and head and senior nurses as the users. Acting as the project leader, the nursing department put forward the requirements as well as the ideas for construction. The information and technology department was responsible for proposing specific operational requirements and providing hardware and software support. The Ewell Technology Company was responsible for optimizing the system construction and function. The quality control department was responsible for overseeing the process of system development to protect data security. The head and senior nurses were in charge of providing feedback to the user interface as well as specific suggestions based on the actual clinical needs. Weekly meetings were established for regular discussion and expedient problem solving to ensure that the PedN-CDSS-Hyperthermia can be launched online as scheduled.

Pediatric Nursing-Clinical Decision Support System for Hyperthermia Development

The PedN-CDSS-Hyperthermia combined hyperthermia management with CDSS. Since the beginning of its development, this system integrated all steps of the nursing process, the conduction of medical advice, and the nursing task list. The system was designed based on several ideas. First, we formed the Pediatric Nursing-Knowledge Base for Hyperthermia (Ped-NKB-Hyperthermia), which combined publicly clinical practice guidelines and nursing routines for hyperthermia management. Second, as the nursing process is similar to a journey, CDSS should be multi-functional along the way, including reminder, notification alert, decision-making tool, and information storage, all of which were used to control all aspects of hyperthermia nursing management to ensure standardized and safe nursing practice. Next, this system was interconnected to the Hospital Information System (HIS) to break any information barriers between doctors and nurses. Besides that, it was built to prescribe prn medical orders to ensure the timeliness of cooling. In addition, this application should be able to create a nursing task list by listing down the nursing to-do items and setting reminders/alerts to ensure timely nursing interventions. Finally, it could generate automated and structured electronic nursing documents to achieve uniform and standardized nursing records.

According to this plan, the Ped-NKB-Hyperthermia should be developed first, and thereafter, the PedN-CDSS-Hyperthermia would follow.

Development of Pediatric Nursing-Knowledge Base for Hyperthermia

The development of the Ped-NKB-Hyperthermia required for building the application was conducted by literature searching from Joanna Briggs Institutions, Cochrane Library, Ovid, China Guide Library, USA Guide Web site, Ontario RN Guide Web site, New Zealand Guide Collaboration, and World Health Organization Guide Web site. The keywords used were as follows: (temperature OR fever OR pyrexia OR hyperthermia) AND (child OR children OR pediatric OR pediatric OR kids OR newborn OR neonatal OR infant OR teenager OR adolescent). Guidelines, systematic reviews, evidence summary, and best practice recommendations in hyperthermia among hospitalized pediatric patients were involved to develop evidence-based and actionable clinical nursing interventions based on the professional judgment of nurses, the needs of children, and the hospital environment. Two rounds of expert symposium were held to discuss the completeness of Ped-NKB-Hyperthermia content and the rationality of language expression. The expert should be a nurse who has worked for more than 10 years, with intermediate title or above, and mastered the management methods of hyperthermia. According to the expert opinions, Ped-NKB-Hyperthermia was formed, including one nursing diagnosis, 19 nursing interventions, and four nursing evaluations. Ped-NKB-Hyperthermia was integrated into the PedN-CDSS-Hyperthermia, and various nursing interventions were presented in a list for nurse to choose. Nurses do not need to edit by themselves to ensure that their documents were standardized.

Introduction of the Pediatric Nursing-Clinical Decision Support System for Hyperthermia Modules

This system adopted Browser/Server architecture, its back end used Java language, and its front end uses C# language. It treated computer and personal digital assistant (PDA) as hardware, and wired and wireless LAN as networks, supporting the joint operation of the computer and PDA. This system included four modules, including nursing assessment, nursing plan, prn medication orders, and nursing task list.

Nursing Assessment Module

The children's temperature needed to be recorded in the temperature sheet, part of the nursing assessment module, which could automatically estimate and inform the schedule/time of temperature reassessment according to the degree of hyperthermia or medication information.

Nursing Plan Module

This module could automatically create the nursing diagnosis, recommend the nursing interventions, remind about the nursing evaluation, and format the structured nursing document. When the axillary temperature was 37.5°C or higher, the system would automatically switch to the nursing plan module and automatically make a nursing diagnosis as hyperthermia. Afterwards, all nursing interventions would be recommended automatically, and the nurse could decide to choose which nursing interventions to apply according to the temperature being measured, the illness, and if patients or their families would like to make a personalized nursing plan. Additionally, it could add nursing interventions when the reassessment temperature was unsatisfactory. Each nursing intervention making time could be recorded and tracked in the system. The nurse needed to scan the patient's identification wristband to perform the nursing intervention through PDA. If the patient's information in the identification wristband did not match the PDA, the system would alert the nurse to recheck. If the patient's axillary temperature had finally gone below 37.5°C for 3 consecutive days or the when the doctor had issued a discharge order, the system would automatically remind the nurse to terminate the nursing diagnosis in time and then would record the termination time. Establishing this system based on the principle of nursing record Problem-Intervention-Outcome, and combining with the clinical situation of our hospital, a structured nursing record sheet was designed and integrated into the PedN-CDSS-Hyperthermia, including nursing diagnosis and time, nursing interventions, nursing outcome and time, and signature of responsible nurses. When the nursing diagnosis of “hyperthermia” was made, the system automatically filled the nursing record sheet according to the nursing methods and results until the nursing diagnosis was terminated.

Medical Order Module

When the axillary temperature was 37.5°C or higher, the temperature value would be automatically pushed to HIS in the form of emergency value, and the floating window would be used to remind the doctor to check the temperature, so as to ensure the medical information was disseminated and hyperthermia is treated timely. The doctor could prescribe a prn medical order that included the drug's name, dosage, and administration at HIS with axillary temperature as the trigger condition. When the patient's temperature hit the trigger condition for the first time, a prn medical order was activated and the red word “order” appeared on the patient's basic information as a reminder, so that the nurse could notice this real-time and submit the medical order. The prn medical order was valid for 24 hours, and it was not allowed to be triggered repeatedly within 6 hours, which not only could ensure drug administration in time but also could minimize the risks of a drug overdose.

Nursing Task List Module

This module summarized each patient's nursing task information, including daily nursing routine, medical orders, and other steps involved. The contents could only disappear after being executed by nurses; otherwise, they would always remain in the task list as a reminder to the nurses. The system could automatically transfer the nursing interventions to the nursing task list once the nurse had completed the nursing plan. The system could also automatically calculate the time to reassess the temperature according to the degree of hyperthermia or medication information, and fed this information into the “nursing task list.” In addition, it could add a prn medical order to the task list to be performed when it was triggered.

Pediatric Nursing-Clinical Decision Support System for Hyperthermia Pilot Implementation

From September 2018 to August 2019, the general surgery ward and immunology ward were selected for the sites of PedN-CDSS-Hyperthermia pilot implementation. Nurses were trained about the whole system process, the operation method, and the function of each module in details through operation demo and system operation helping guide. The scheme or the expression forms of the system (eg, system function's language, color, and layout) were adjusted according to the barriers that nurses encountered along with their feedbacks during the pilot. A WeChat group was also created to include all nurses participating in the pilot wards and all members of the multi-disciplinary team.

Usability and Feasibility Test to Pediatric Nursing-Clinical Decision Support System for Hyperthermia

Before (1st to 31st of August 2018) and after (1st to 31st of December 2019) steady using the PedN-CDSS-Hyperthermia, we used the Clinical Nursing Information System Effectiveness Evaluation Scale,[15] which was based on the framework of the DeLone's and McLean's revised framework (the new D&M),[16] and combined with the actual situation of clinical nursing information system in our hospital to test the system usability and feasibility by nurses' system use experience. The new D&M not only reflected the performance and function of the system but also reflected the effect of the interaction between the information system and users from the perspective of users, and it also reflects the users' cognition of the usability and feasibility to information system.[17] The Clinical Nursing Information System Effectiveness Evaluation Scale included five dimensions, namely, system quality, information quality, service quality, user satisfaction, and net income, with 23 items in total and Cronbach's α = 0.768.

Only certified nurses who had engaged in clinical practice for more than 1 year were included in the study. The exclusion criteria included emergency nurses, office nurses, head nurses, and rotational nurses. After explaining the purpose and significance of this study, the Clinical Nursing Information System Effectiveness Evaluation Scale was informed and assessed on the spot to avoid the nurses' working hours or busy hours by members of the research team who explained the significance of each item to the nurse. The researcher checked the quality of the scale on the spot and invited the nurse to complete it in time once any items were found missing. Two hundred forty-two nurses were included in the control group, and 258 nurses were included in the experimental group. There was no significant baseline difference between these two groups in age, working time, and education degree (P > .05).

Effectiveness Test to Pediatric Nursing-Clinical Decision Support System for Hyperthermia

The quality of nursing records was measured by the completeness of nursing assessment, the timeliness of nursing diagnosis, the individualization of nursing interventions, and the timeliness of nursing evaluation. Specifically, completeness of nursing assessment means actual evaluation times are equal to the theoretical evaluation times according to the degree of fever and whether to take medication. Timeliness of nursing diagnosis means nursing diagnosis is made within 30 minutes of the first axillary temperature of 37.5°C or higher. Individualization of nursing intervention means the contents of nursing intervention are consistent with the patient's condition and actual treatment methods. This index was evaluated back-to-back by the head nurse of the two study wards; when the results were in dispute, the head nurse in another ward was invited to make a decision. Timeliness of nursing evaluation means the nurse evaluates the nursing outcome within 72 to 86 hours of continuous normal axillary temperature, and children with normal axillary temperature less than 72 hours or still in fever but discharged from the hospital are also considered to be timely. The inclusion criteria were nursing records in the immunology and general surgery wards that had patients with body temperature ≥ 37.5°C who had stayed in the hospital for more than 3 days with no transfer history to the other wards. A total of 116 nursing records were included in the control group according to the inclusion criteria, but 26 of them were excluded as lacking nursing diagnosis and being unable to be further analyzed. Finally, 90 nursing records were included in the control group, and 94 nursing records were included in the experimental group. There was no significant difference in baseline characteristics between the two groups in the patients' age, length of hospital stay, and disease (P > .05).

Data Collection and Analysis

The research objects that meet the inclusion and exclusion criteria were selected, and the original data were extracted and recorded strictly according to the definition of outcome indicators. Data collection and input were completed by two graduate students, one of whom was responsible for original data extraction, and the other one was responsible for data verification. Data input and analyses were conducted using IBM SPSS Statistics version 22.0 (IBM Inc, Armonk, NY, USA). The independent sample t test or chi-square test was used to compare the differences between the control group and the experimental group. All hypothesis tests were performed at a bilateral test level, and P < .05 was considered statistically significant.

Ethical Considerations

This research was approved by the Pediatric Research Ethics Board of the Children's Hospital of Fudan University (approval number: IRB105-CCH-IRP-151).

RESULTS

Usability and Feasibility Test Results

The total score of Clinical Nursing Information System Effectiveness Evaluation Scale was 67.52 ± 6.36 vs 93.88 ± 3.71 in the control group and experimental group, respectively (t = −33.073, P = .000). There was a significant difference between the two groups in the dimensions of “information quality,” “service quality,” “customer satisfaction,” and “net income” (P < .01), but there was no significant difference between the two groups in the dimension of “system quality” (P = .19). The results of specific items showed “response time,” “safety,” and “accuracy” between the two groups had no significant difference (P > .05) (Table 1).

Table 1

The Results of Specific Items in the Effectiveness Evaluation Scale of Clinical Nursing Information System Between the Two Groups

Dimensions	Items	Control Group (n = 242)		Experimental Group (n = 258)		t	P
		M	SD	M	SD
System quality	Response time	4.06	±0.76	4.02	±0.95	0.60	.548a
	Safety	4.21	±0.68	4.31	±0.69	−1.61	.110a
	Flexibility	4.17	±0.77	4.33	±0.74	−2.31	.021
	Stability	3.95	±0.83	3.47	±1.02	5.78	.000
Information quality	Integrity	3.53	±0.83	3.74	±0.85	−2.94	.003
	Accuracy	3.46	±0.52	3.45	±0.80	0.21	.832a
	Timeliness	3.48	±0.65	4.30	±0.68	−13.69	.000
	Continuity	3.55	±0.61	4.26	±0.73	−11.81	.000
	Availability	3.52	±0.65	4.42	±0.71	−14.60	.000
Service quality	Tangibility	4.06	±0.69	4.33	±0.70	−4.30	.000
	Reliability	3.91	±0.68	4.68	±0.57	−13.76	.000
	Assurance	4.04	±0.75	4.50	±0.67	−7.27	.000
	Empathy	3.81	±0.76	4.15	±0.77	−4.87	.000
Customer satisfaction	Overall performance	3.16	±0.68	3.85	±0.73	−11.01	.000
	System operation	3.18	±0.72	3.79	±0.72	−9.45	.000
	System function	3.27	±0.72	4.10	±0.72	−12.96	.000
	Decision support	1.71	±0.67	3.81	±0.68	−34.64	.000
	System efficiency	3.33	±0.79	4.15	±0.73	−11.93	.000
Net income	Improve service quality	3.29	±1.08	3.98	±0.75	−8.28	.000
	Improve work efficiency	3.14	±0.99	3.85	±0.73	−9.07	.000
	Reduce costs	2.65	±0.83	3.48	±0.77	−11.70	.000
	Provide decision support	2.14	±0.70	4.01	±0.73	−29.28	.000
	Improve service process	3.37	±0.97	4.03	±0.72	−8.68	.000

aP > .05.

Effectiveness Test Results

In terms of the completeness of nursing assessment, 32 nursing records were incomplete among controls after taking cooling measures but no incomplete assessment in the experimental group (χ2 = 40.46, P < .001). In terms of the timeliness of nursing diagnosis, 12 nursing records were delayed for 3 days or more, with the longest delay period being 14.52 days among controls. In the experimental group, seven nursing records were delayed up to 13.55 hours (χ2 = 51.60, P < .001). In terms of the timeliness of nursing evaluation, 62 nursing records were delayed with a maximum delay of 14 days in the control group, but no delay was observed in the experimental group (χ2 = 97.67, P < .001).

Moreover, before using the PedN-CDSS-Hyperthermia, there were two terminologies used in the nursing diagnosis for “axillary temperature ≥ 37.5°C”: “hyperthermia” and “fever.” The nursing interventions were also already fixed and cannot be changed, with differences only in the “wards” or “nursing diagnosis.” After using the PedN-CDSS-Hyperthermia, there was only one nursing diagnosis, “hyperthermia,” and the nursing interventions could vary from child to child.

DISCUSSION

Pediatric Nursing-Clinical Decision Support System for Hyperthermia Is Useful and Easy to Use, Which Improves the System Experience of Nurses

The key factors influencing the application of nursing information system include nurses' view of the system, acceptance degree, and satisfaction degree.[18] In our study, we found nurses' overall experience of the PedN-CDSS-Hyperthermia was better than that of original nursing information system, with experience in “information quality,” “service quality,” “user satisfaction,” and “net income” superior to the original system, but not in “system quality.” The PedN-CDSS-Hyperthermia had no significant improvement. As the change of information system requires a transitional period generally lasting for 12 to 18 months to adapt to and learn, if there are problems with equipment or software, nurses may have negative experiences during this period.[19,20] PedN-CDSS-Hyperthermia is still in transition period now. In this pilot implementation stage, several technical problems had occurred, such as system obstruction, data loss, and blank screen of computer or PDA, which led to the impression of the system's underperformance among the nurses. However, it is important to note that the original nursing information system had been used in this hospital for many years, and nurses had been accustomed to its mode and function, so the “system quality” of the new system had not been significantly improved. The nursing process for patients requires the nurses to be able to obtain the best information, think about the information, filter the information, and make clinical decision.[21] High satisfaction related to a number of factors, such as usability, reliability, and support provided when problems occurred. The PedN-CDSS-Hyperthermia was able to trigger nursing diagnosis, recommend nursing interventions, remind nursing evaluation, estimate the reassessment time, fill out the structured nursing documents, and inform the doctor automatically, which provides reliable care information and ensures the quality of collected data. In the process of building the PedN-CDSS-Hyperthermia, members of multi-disciplinary team communicated with the clinical nurses timely through the WeChat or face-to-face to solve problems encountered in system operation, which was the main reason for better a better experience to “service quality “feature.

Pediatric Nursing-Clinical Decision Support System for Hyperthermia Could Improve the Quality of Nursing Records

Nurses spend 15%-50% of their time and energy on nursing records, which not only affects the effectiveness of nursing but also reflects the level of hospital management.[22,23] As many as 26 nursing records that had initially met the inclusion criteria in the control group were excluded due to unavailable nursing diagnosis, reflecting that each step in the nursing process was unintegrated and with no proper control before using the PedN-CDSS-Hyperthermia. As the first stage of the nursing process, any errors or omissions in nursing assessment may affect the process or accuracy of the subsequent steps. Before using the PedN-CDSS-Hyperthermia, there were 32 nursing records with no assessment after applying any cooling measures, which led to the interruption of the nursing process. Some nurses were also discovered to be able to properly and timely measure the temperature but could not make a nursing diagnosis or take measures timely when the temperature is abnormally high, leading to the delayed care for children before using the new system. Zega et al[24] also found that although nursing assessment was accurate and complete in most nursing information systems, nursing diagnosis and subsequent steps were insufficient and incoherent. The reason was that nurses were easy to be distracted by other work or by other staffs when caring for children with hyperthermia, and the original system lacked the reminder function; thus, it was unable to prevent delay in assessment, diagnosis, management, and evaluation. As the PedN-CDSS-Hyperthermia could automatically extract each step of the nursing process and fill them into the standardized, structured, and formatted nursing record sheet, it could prevent missing contents in the records effectively and improve the quality of nursing records. Xia et al[25] combined CDSS with the nursing process and applied it to pressure injury; the nursing record defect rate decreased from 16.42% to 8.70%, which improved the quality of nursing records. Chen et al,[26] using CDSS, designed an electronic nursing record reminder system, which includes a total of 12 electronic nursing records, and its accuracy validation was 100%; it can also support the nursing recording process and prepare it for implementation in the following phase.

LIMITATIONS

As the data of the control group were analyzed and extracted retroactively, neither the influence of the information system on nurses' behavior nor the influence of patients' comfort in the cooling process could be tracked. Therefore, the evaluation indexes of the system in this study were only limited to the quality of nursing records and nurses' experience to the system, without clinical outcome indexes. Because the PedN-CDSS-Hyperthermia took a long time to build and adjust, the data of the experimental group began to be collected only a month after the stable operation. That stage still belongs to the “transitional stage” of system construction; therefore, it is necessary to observe and track the effect of the system after the system operating more stably.

The function of prn medical order was originally designed to ensure timely treatment of hyperthermia. However, drugs cannot be charged due to the lack of interconnection between HIS and pharmacy information system; the function of prn medical order was still blocked even though the PedN-CDSS-Hyperthermia realized its interconnection with HIS. Therefore, we need to continue to communicate with the hospital pharmacy to achieve barrier-free communication and information sharing of nursing, medical advice, pharmacy, and patients.

Because the system was specifically developed according to our hospital's hyperthermia management workflow and procedures, its generalizability may be limited to other hospitals. However, the workflow is based on nursing process, and the knowledge base is based on evidence-based nursing for hyperthermia. We believe that the knowledge base and workflow for hyperthermia in this study can be used as a reference to other hospitals.

CONCLUSION

Based on the framework of nursing process, the PedN-CDSS-Hyperthermia combined evidence-based nursing and clinical decision support technology, which included four modules, namely, nursing assessment, nursing plan, medical order, and nursing task list. The PedN-CDSS-Hyperthermia generally enhanced the nursing records' quality and nurses' system use experience, standardized the hyperthermia nursing process, and improved the timeliness, continuity, and integrity of hyperthermia care for hospitalized children.