Developing a tool for nurses to assess risk of infection in pediatric oncology patients in China: a modified Delphi study.

Infections are identified as the most common preventable cause of death in pediatric oncology patients. Assessing and stratifying risk of infections are essential to prevent infection in these patients. To date, no tool can fulfill this demand in China. This study aimed to develop a nursing work-based and Chinese-specific tool for pediatric nurses to assess risk of infection in oncology patients. This research was a modified Delphi study. Based on a literature review, a 37-item questionnaire rating on a 0-5 scale was developed. Twenty-four experts from 8 hospitals in 6 provinces of China were consulted for three rounds. Consensus for each item in the first round was defined as: the rating mean was>3 and the coefficient of variation (CV) was<0.5. Consensus for each item in the second round was defined as CV<0.3. Consensus among experts was defined as: P value of Kendall's coefficient of concordance (W)<0.05. After three rounds of consultation, a two-part tool was developed: the Immune Status Scale (ISS) and the Checklist of Risk Factors of Infection (CRFI). There were 5 items in the ISS and 14 in the CRFI. Based on the ISS score, nurses could stratify children into the low-risk and high-risk groups. For high-risk children, nurses should screen risk factors of infection every day by the CRFI, and twice weekly for low-risk children. Further study is needed to verify this tool's efficacy.

INTRODUCTION

According to data on the GLOBOCAN 2012 website, the incidence of childhood cancers in developed countries such as Europe (13.1/100,000) was higher than in developing countries such as South-East Asia (6.8/100,000)[. However, the mortality rate of childhood cancers in South-East Asia (3.4/100,000) was close to that in Europe (3.9/100,000)[. These data demonstrated that fatality the rates of childhood cancers in developing countries were higher than in developed countries[. Calaminus et al. stated that it was important to address current problems in pediatric oncology in developing countries[. Infections are identified as the most common preventable cause of death in pediatric oncology patients[. The infection rate of Chinese pediatric oncology patients was 45.42% and the mortality rate was 31.78%[. Infection not only threatens children's lives, but also affects the psychological status of children and their parents[. Thus, infection control is essential for pediatric cancer patients and their families.

Nurses always play an important role in infection control[, and the newest guidelines emphasize the importance of the role of nurses in managing infection risks in oncology patients[. However, in China, there are some obstacles for nurses to fulfill this significant task. The first one is the shortage of nurses in China. By 2012, China had 1.85 nurses per 1,000 population[, which was less than the global average[. The other obstacle is the lack of an established process for nurses to follow to prevent infection of oncology children. One widely accepted consensus is that assessing and stratifying risk of infections in oncology patients is essential to prevent infection in these patients[. Based on this consensus, a tool for assessing and stratifying risk of infection may be a solution for the above two obstacles[.

Some tools have been developed to assess risk of infection[; however, the development of these tools was mainly led by pediatric oncologists and pharmacists and they mainly focused on physical factors such as clinical symptoms and laboratory tests[. Thus, most of them were treatment- rather than prevention-oriented[. So a nursing work-based and prevention-oriented tool for oncology children is needed[. Due to historical, political, social and economical conditions, health systems differ among countries. This suggests that tools for assessing and stratifying infection risks should be developed according to local clinical practice[. Local tools for assessing risks of infection in oncology patients were developed in some countries, such as Brazil and Jordan[. Currently, there is no Chinese-specific tool.

Therefore, the present study aimed to develop a nursing work-based and Chinese-specific instrument for assessing risks of infection in pediatric oncology patients. A modified Delphi method was used and a tool involving 19 items was developed.

Materials and methods

Methods

A modified Delphi method was used in this study. The Delphi approach is thought to be especially effective in difficult areas that can benefit from subjective judgments on a collective basis[. It is widely used in nursing fields, such as education, management, and clinical nursing. It normally involves the presentation of a questionnaire to a panel of experts in pertinent fields in order to seek their opinions and reach a consensus on a particular issue[.

Based on a summary of literature, a questionnaire was developed in a face-to-face workshop. Then, three rounds of expert consultation were implemented from 2012 to 2013. This research was approved by the Ethics Committee of Nanjing Medical University, and informed consent of all participants was obtained.

Development of a questionnaire for expert consultation

Literature in English and Chinese was searched. English publications from 1992 to 2012 were searched in Medline and Embase. Chinese publications from 1992 to 2012 were searched in China National Knowledge Infrastructure (CNKI), Wan-Fang Database, and VIP Database. The search keywords were: infection control, risk assessment and neoplasm. In total, 1439 articles in English and 1159 articles in Chinese were retrieved. Articles that involved risk factors of infection in oncology patients were located by reviewing the title and abstract. These factors were summarized systematically. The summary of risk factors of infection were discussed and modified by six researchers (1 pediatric oncologist, 1 infection-control expert, and 4 pediatric nurses). A 37-item questionnaire rating on a 0–5 scale (0 = total disagreement and 5 = total agreement) was developed.

Three rounds of expert consultation

Settings and participants: In this study, experts were from eight tertiary hospitals distributed in six provinces of China (Shandong, Jiangsu, Anhui, Shanghai, Zhejiang and Guangdong). One pediatric oncologist, one pediatric nurse, and one infection-control professional were selected from each hospital. The oncologists had to be physicians with a senior professional title; the pediatric nurses had to have a bachelor's or higher degree with a middle or higher professional title; and infection-control professionals had to have a bachelor's or higher degree with a senior professional title. All experts had to have at least 10 years of related work experience. The participants' demographics are shown in .

Table 1

Participant demographics

Demographics	First-round (n=24)	Second and third-rounds (n=23)
Gender
Male	9	8
Female	15	15
Age(years)
Mean	42.92	40.91
Range	33-56	33-56
Working experience(years)
Mean	19.86	20.04
Range	10-35	10-35
Educational Level
Doctor	5	4
Master	4	4
Bachelor	15	15
Professional title
Senior	17	16
Middle	7	7

The first-round consultation was implemented from September to November 2012 among 24 experts. Experts in Nanjing were consulted by paper-based questionnaires and experts in other cities by electronic questionnaires through e-mail. An introduction to this research and a demographic investigation tool were attached to the 37-item questionnaire.

In the second round consultation, a 32-item questionnaire was developed according to the results of the first-round consultation and implemented in the second-round consultation from January to March, 2013. Twenty-three experts finished this round, and one pediatric oncologist from Guangdong Province withdrew from this study due to personal reasons. In this round, there was no demographic investigation and a summary of the results of the first-round consultation replaced the introduction of this research.

The third round consultation aimed to figure out how to assess risk factors selected by experts in the first two rounds. Three steps were followed. First, a face-to-face workshop was implemented again among the same six researchers to figure out a draft about how to assess these factors according to references. Second, the draft was sent to 23 experts to collect suggestions. Lastly, the suggestions were discussed again in a face-to-face workshop and a tool comprised of two-parts was developed.

Statistical analysis and consensus criteria

Consensus for items in questionnaires and among experts was evaluated in a Delphi study. Rating mean and coefficient of variation (CV) of items are widely used to evaluate consensus for items in questionnaires, and consensus among experts can be evaluated by Kendall's coefficient of concordance (W)[. In this study, consensus for each item in the first round was defined as: rating mean > 3 and CV < 0.5. Consensus for each item in the second round was defined as: CV < 0.3. Consensus among experts was defined as: P value of Kendall's W < 0.05. All the data were analyzed using SPSS version 17 (SPSS Inc, Chicago, IL, USA).

Results

Results of first-round consultation

Kendall's W of the first-round consultation was 0.22 (P < 0.01). The rating means and CVs of items are shown in . According to the criteria of consensus, nine items with rating mean < 3 or CV > 0.5 were deleted from the questionnaire for the second-round consultation. Apart from rating items, 11 experts also wrote some comments, questions and suggestions on the questionnaire. These issues were discussed among researchers. Four new items were developed, and the descriptions 3 items were modified.

Table 2

Rating means and CVs of item in the first-round consultation

Risk factors of infection	Rating mean (X̄)	Standard Deviation(SD)	Coefficient of variation(CV)
1. Diagnosis of disease	3.96	1.22	0.31
2. Chemotherapy regimen	4.67	0.76	0.16
3. Days after beginning chemotherapy	3.85	1.08	0.28
4. Days of hospitalization	2.96*	1.81	0.61**
5. Routine blood test	4.96	0.2	0.04
6. Body mass index	3.88	1.54	0.4
7. History of infection after diagnosis of malignant tumors	3.83	1.53	0.4
8. Level of education of children and guardians	3.11	1.76	0.57**
9. Compliance with treatment	4.4	1.22	0.28
10. Family economy state	3.29	1.71	0.52**
11. Health conditions of guardians	3.67	1.46	0.4
12. Number of persons with access to the ward	3.33	1.55	0.47
13. Protective isolation	4.58	0.83	0.18
14. Children and their guardians' compliance with wearing masks	4.21	1.02	0.24
15. Children and their guardians' compliance with hand hygiene	4.29	1.43	0.33
16. Ventilation of the ward	4.5	0.83	0.19
17. Frequency of going out of the ward	3.35	1.55	0.46
18. Children and their guardians' compliance with oral care	4.17	1.17	0.28
19. Frequency of special oral care	4.08	1.32	0.32
20. Eating habits	3.58	1.47	0.41
21. Disinfection of tableware	3.98	1.28	0.32
22. Condition of oral mucosa and gingiva	4.46	0.88	0.2
23. Eruption of teeth	2.63*	1.76	0.67**
24. Constipation	3.44	1.69	0.49
25. Diarrhea	3.38	1.74	0.52**
26. Children and their guardians' compliance with perianal care	4.33	0.92	0.21
27. Frequency of urination	2.25*	1.75	0.78**
28. Compliance with perineal care	4.39	0.94	0.21
29. Condition of prepuce	3.18	1.5	0.47
30. Menstrual period	2.39*	1.62	0.68**
31. Frequency of bathing	2.43*	1.8	0.74**
32. Frequency of changing clothes	2.52*	1.75	0.70**
33. Condition of skin (such as itchiness, redness, wetness and edema)	4.08	1.41	0.35
34.Condition of wounds	4.5	0.83	0.19
35.Wound dressing	4.17	0.96	0.23
36. Insertion and management of peripheral venous access devices	4.43	0.95	0.21
37. Insertion and management of PICC, CVC or port	4.52	0.9	0.2

Note. *Rating mean, <3.0, **CV>0.50.

Results of second-round consultation

Kendall's W of the second-round consultation was 0.34 (P < 0.01). The rating means and CVs in this round are shown in . Nine items were deleted. There were neither questions nor suggestions from experts in this round.

Table 3

Rating means and CVs of item in the second-round consultation

Risk factors of infection	Rating means (X̄)	Standard Deviation(SD)	Coefficient of variation(CV)
1. Diagnosis of disease	4.24	0.97	0.23
2. Chemotherapy regimen	4.14	0.91	0.22
3. History of infection after diagnosis of oncology	4.11	1.10	0.27
4. Days of hospitalization	4.77	0.58	0.12
5. Routine blood test	3.93	1.05	0.27
6. Body mass index	3.71	1.32	0.36*
7. Compliance with treatment	4.58	0.93	0.20
8. Health conditions of guardians	4.00	1.28	0.32*
9. Family living conditions	3.66	1.17	0.32*
10. Protective isolation	4.67	0.82	0.17
11. Children and their guardians' compliance with wearing masks	4.26	1.06	0.25
12. Children and their guardians' compliance with hand hygiene	4.53	0.85	0.19
13. Disinfection of the wards	4.39	0.80	0.18
14. Number of persons with access to the ward	3.97	1.07	0.27
15. Frequency of going out of the ward	3.65	1.15	0.32*
16. Children and their guardians' compliance with oral care	4.52	0.84	0.19
17. Frequency of special oral care	4.09	1.02	0.25
18. Eating habits	3.74	0.96	0.26
19. Disinfection of tableware	3.75	1.34	0.36*
20. Condition of mouth mucosa and gingiva	4.26	1.07	0.25
21. Dental caries	3.39	1.40	0.41*
22. Children and their guardians' compliance with perianal care	4.22	1.12	0.26
23. Conditions of defecation	3.58	1.22	0.34*
24. Insertion and management of peripheral venous access devices	4.12	1.07	0.26
25. Insertion and management of PICC, CVC or port	4.30	1.02	0.24
26. Children and their guardians' compliance with sustaining care of PICC, CVC, port during discharge period	4.26	1.16	0.27
27. Conditions of skin (such as itchiness, redness, wetness and edema)	3.71	1.18	0.32
28. Condition of wounds	4.44	0.86	0.19
29. Wound dressing	4.43	0.85	0.19
30. Routine urine test	3.81	0.99	0.26
31. Compliance of perineal care	4.21	1.26	0.30
32. Condition of prepuce (only for boys)	3.51	1.23	0.35*

Note. * CV>0.30.

Results of third-round consultation

Twenty-three items remained after two rounds of expert consultation. A draft on how to assess these items was developed. The draft was modified depending on the 15 experts' suggestions. A tool including 19 items was developed, with five items in the Immune Status Scale (ISS) and 14 in the Checklist of Risk Factors of Infection (CRFI) ( materials available online). The process of infection prevention based on the tool is shown in .

Fig. 1

The process of infection prevention based on ISS and CRFI.

Discussion

A panel of experts was essential for a Delphi study[. Experts involved in the present study came from 8 hospitals in 6 provinces. Their opinions can be considered to be geographically representative. The panel consisted of pediatric oncologists, pediatric nurses and infection-control professionals. The different professional backgrounds of these experts helped ensure the validity of the study. Kendall's W in the first two rounds was 0.22 and 0.34, respectively, with P < 0.01. This means that consensus among experts was highly significant.

The first two rounds of consultations aimed to refine the questionnaire items according to experts' rating: 18 items were deleted and four were developed during the process. The third-round consultation was to decide how to assess the 23 items retained after two rounds of consultation. These items were translated into 19 assessment questions, and multiple choices for these questions were developed according to references and experts' suggestions[. The tool developed by this study comprised two parts. The first part was the ISS, which was designed to assess and stratify risk of infection in children. It comprised 5 items and each had 3 choices at different levels. A higher score means a greater risk of infection. Based on the weekly summary ISS score, nurses can divide children into high- and low-risk groups. The second part of this tool was the CRFI. This part included 14 questions that may help nurses screen out risk factors of infection. The factors screened out are targets of infection prevention. Combining both parts, nurses are recommended to assess and stratify risk of infection in individual children every week by ISS. Children stratified into the high-risk group should be screened for risk factors of infection every day by CRFI, compared with twice weekly in the low-risk group. Based on the CRFI, risk-factor-focused and individualized nursing interventions can be developed[. Re-assessment is needed to make sure the risk factor of infection is eliminated by nursing interventions, and a new cycle of assessment-intervention begins. The process of infection prevention based on the tool is shown as .

The aim of this study was to develop a nursing work-based and Chinese characteristic tool to assess risk of infection in pediatric oncology patients. Was the aim ultimately fulfilled? First, most factors in the CRFI, such as hand hygiene and oral care, can be evaluated and intervened by nurses independently[. So, this tool was considered to be nursing work-based.

The tool fully considered Chinese culture and history. The participants in this study had worked in pertinent fields for at least 10 years and their work experience acquainted them with Chinese local clinical practice. Compared to existing tools in other countries[, the tool developed in the present study contained the same items such as absolute neutrophil count. However, it had a few different items and even different choices for the same item. A good example was the sixth item in CRFI: IV settings ( materials available online). Although disposable peripheral vein needles and peripheral venous catheters have almost been abandoned in oncology patients in other countries[, they were still widely used in China because of their low cost. Their use means repeat punctures within a short time, so they may cause infection more easily than other venous access devices[. There were also some other items that were seen as characteristic Chinese risk factors of infection. So, this tool conformed to Chinese local clinical practice. To summarize, the tool developed here was nursing work-based, and Chinese-specific.

Our study had some limitations. Experts in this study all came from southeastern coastal areas of China, which are the most developed parts of the country. There may have been a geographic bias and the tool should be tailored before being used in other areas of China. Further research is underway to establish if this process, based on the tool, can decrease rates of infections in children with malignant tumors.