Validity and reliability of the proposed core competency for infection control nurses of hospitals in Hong Kong.

Literature review and the Delphi approach were used to draft the core competency items of hospital infection control nurses in Hong Kong. Content validity, internal consistency, and test-retest reliability of the proposed core competency were ensured. The result serves as the foundation of developing training and assessment tools for infection control nurses in Hong Kong.

There is only half a century of history of infection control nurses (ICNs) in the world with the first ICN appointed in the United Kingdom in 1959. Infection control programs were introduced to Hong Kong in the 1980s with limited resources.2, 3 After the severe acute respiratory syndrome epidemic in Hong Kong in 2003, infection control became popular. The most prominent change was that more and more ICNs were deployed. Hence, the competency level of ICNs became the concern of hospital management and clients.

Infection control professional groups in the United Kingdom and North America had developed their own competency standards.4, 5 However, these publications from overseas standards cannot be employed directly in the local setting because of differences in health care structure, philosophy and management approach, setting and resources, disease epidemiology, and education background. Therefore, we propose a core competency for specialist ICNs in Hong Kong. This paper describes the process of development of the proposed core competency for ICNs in Hong Kong.

Methods

Design

The study was divided into 3 stages. First, the Delphi approach was used to draft the core competency. Next, both qualitative and quantitative approaches were employed to establish the content validity. Last, the reliability was developed based on traditional statistical methods and the Rasch model. This study was approved by the Human Subjects Ethics Subcommittee of The Hong Kong Polytechnic University, Hong Kong, SAR.

Stage 1: drafting the core competency using the Delphi approach

E-mail was used as the main method of communication for the Delphi approach. At the beginning, the investigator drafted the competency categories of ICNs based on a literature review and transformed them into a questionnaire for collecting opinions. The experts were asked whether they agreed or disagreed with the categories on the questionnaire. The investigator acted as the study coordinator and decision maker. She reviewed the returned comments and modified the items when necessary. Direct discussions with experts were used when clarification of returned information was needed. A few rounds were held until the consensus was achieved. Next, the investigator drafted the competency items from the literature based on the agreed competency categories. The drafted competency items were transformed into a questionnaire. The experts were then asked whether they agreed or disagreed with the listed items. Each questionnaire had a free-text area to allow the experts to fill in additional categories, items, or other comments. Several rounds were conducted until consensus was achieved.

Sample

Purposive sampling was used. A panel of 6 subject matter experts consisted of 2 infection control officers (ICOs) and 4 ICNs who were the leaders of large infection control teams in public hospitals was established.

Data analysis

Only categories or items on which 80% or more of the members agreed were included.6, 7 The κ statistics were worked out to quantify the quality of the content validity by the online calculator of the University of Joensuu, Finland.8, 9, 10

Results of stage 1

A total of 4 rounds were conducted. The first 2 rounds were used to develop the competency categories and the latter 2 to develop the competency items. We finally obtained 10 categories and 51 items. The categories included (1) surveillance, (2) program management and evaluation, (3) evidence-based practice, (4) education, (5) team and service management, (6) collaboration and partnership, (7) outbreak investigation and control, (8) research and development, (9) expert knowledge and continuing education, and (10) professional development. The free-marginal multirater κ for 51 items among six panel experts was 0.84 showing excellent agreement among experts. In addition, some of the draft competency items consisted of double-barreled sentences. We split these items to obtain 64 competency items.

Stage 2: establishing content validity

A questionnaire consisting of the 64-item draft core competency of ICNs that was developed in stage 1 was employed. The experts were asked to rate these items in terms of relevance to the core competency of ICNs. A 4-point ordinal rating scale from Lynn’s study was used. At the end of the questionnaire, the experts were asked whether all the content domains were included in the draft core competency list. They were requested to specify the missing content in the case of a negative answer.

An expert panel that consisted of 3 subject matter experts who were the ICOs from different major public hospitals was established.11, 12

The content validity index (CVI) and inter-rater agreement was calculated. Free-marginal multirater κ statistics were worked out using the online calculator of University of Joensuu, Finland.9, 10

Results of stage 2

The content experts commented that all the dimensions of the core competency were available in the questionnaire. CVI was 0.75, which was fairly satisfactory. The inter-rater agreement was 75% (48/64 items). The free-marginal multirater κ was 0.67 representing a good level of agreement.

Stage 3: establishing reliability

The 64 items developed in stage 1 were transformed into a questionnaire. Subjects were invited to answer 2 identical questionnaires 2 weeks apart.11, 14, 15 To minimize the memory effect, the sequence of items in each questionnaire was randomly assigned. Subjects were asked to rate each competency item as very important, important, undecided, not important or not very important. Their demographic information was also requested.

Because of the limited number of ICNs in Hong Kong, their input was reserved for the follow-up studies. Nurses who had previously worked as ICNs (ie, ex-ICNs) were invited to participate in this study. Eighteen ex-ICNs were invited through snowball sampling.

Demographic data and test-retest reliability were analyzed by SPSS version 15.0 (SPSS, Inc, Chicago, IL). Rating scale diagnositics were performed using Winsteps (Lawrence Erlbaum Associates, Mahwah, NJ) version 1.0.0, which was a Rasch analysis software package. Coefficient α was worked out by both SPSS and Winsteps.

Results of stage 3

Response rate and demographics

In total, 17 (94.4%) of 18 ex-ICNs completed the second questionnaire. Among the 18 subjects, most of them (77.8%) were female. The majority (50.0%) fell into the 31- to 40-year-old age group. Their work experience in the infection control field ranged from 1 to 10 years with the mean of 5.5. Most of them (94.4%) worked as full-time ICNs, and 83.3% worked in an acute setting. All of them were working in public hospitals, and 50.0% of them worked in a large hospital that had more than 1,000 patient beds.

Reliability

The internal consistency for the first returns of 18 ex-ICNs for 64 items was high, with a coefficient α of .98. Seventeen pairs of data were used to check the test-retest reliability. The means of each item on the first and second tests were calculated. The correlation between 2 respective means was then compared and Spearman correlation coefficient was 0.839 (P < .000). This showed that there was no significant difference between the means of 2 returns.

Rating scale diagnostics

The category structure is summarized in Table 1 . Among the 5 rating categories, only 4 were used. No item was rated as “not very important.” Only 2 frequencies rated as “not important.” This illustrated that the ex-ICNs generally agreed with the competency items that were drafted by the field experts, which matched the validity content. The Rasch-Andrich thresholds increased monotonically from −2.97 to 3.95 showing that the scale was ordered. The thresholds between categories were within the optimal distance, which should be between 1.4 and 5 logits. This illustrated that each category defined a distinct position on the scale. Results showed that rating scale of the questionnaire was functional. The probability graph of the rating scale is visualized as Fig 1 . Each category has a distinct peak in the graph. The thresholds were presented at the intersections of rating scale categories.

Table 1

Summary of category structure

Category		Observed
Rating	Label	Count	%	Rasch-Andrich threshold
1	Not very important	0	0	-
2	Not important	2	0	None
3	Undecided	46	4	−2.97
4	Important	585	51	−0.98
5	Very important	519	45	3.95

Fig 1

Probability graph of rating scale.

Conclusion

The core competency of ICNs drafted by panel experts using Delphi approach was valid and reliable. The information on content validity was enhanced by free-marginal multirater κ in addition to CVI. The functional rating scale of the questionnaire ensured the quality of data collected. To promote the quality of infection control service, it is important to equip ICNs in Hong Kong with reliable and valid training and assessment. Core competency is the major component for the development. The result of this study builds the foundation on this future direction.