Iranian Effective Clinical Nurse Instructor evaluation tool: Development and psychometric testing.

BACKGROUND: Clinical education is the heart of the nursing education program. Effective nursing clinical instructors are needed for graduating the future qualified nurses. There is a well-developed body of knowledge about the effectiveness of clinical teaching and the instructors. However, translating this knowledge into a context-based evaluation tool for measuring the effectiveness of Iranian clinical nursing instructors remains a deficiency. The purpose of this study is to describe the development and psychometric testing process of an instrument to evaluate the characteristics of Iranian effective clinical nurse instructor.
MATERIALS AND METHODS: Following a precise review of Iranian literatures and expert consultation, 83 statements about the characteristics that make clinical nurse instructors effective were extracted. In the next phase, the psychometric properties of the instrument were established by looking at the content validity, face validity, and internal consistency. Content validity of the instrument was assessed based on the comments of an expert panel including 10 nursing faculty members. During this phase, 30 items of the instrument were omitted or merged. Face validity of the instrument was assured based on the advices of 10 nursing students and 10 nursing faculty members. Finally, in the pilot test, the data of 168 filled questionnaires were gathered and analyzed by an exploratory factor analysis to reduce the items and identify the factor structure of the instrument.
RESULTS: Through subsequent analyses, of the 83 items, 31 items were merged or omitted. At last, 52 retained items were divided into four subscales including student-centric behaviors, clinical performances, planning ability, and personality traits. The Cronbach's alpha level of the inventory was 0.96, with the value for each domain ranging from 0.87 to 0.94.
CONCLUSIONS: Iranian Effective Clinical Nurse Instructor evaluation tool has acceptable psychometric properties and can be used in evaluating the effectiveness of clinical nursing instructors.

INTRODUCTION

Nursing is a practice-based discipline,[1] and clinical experiences are vital in the nursing education program.[23] These experiences acts as a mediator to bridge the gap between theory and practice,[4] and make nursing students prepare for real-world situations.[5] A well-directed clinical education promotes students’ critical thinking, clinical judgments, decision making, clinical skills, clinical knowledge, and attitudes. It also influences the students’ socialization, professionalization, satisfaction, competency, and interpersonal relationships.[26]

Clinical environments are complex which encompass different intertwined factors[3] and are often perceived by nursing students as threatening situations. In other words, clinical education is considered to be a stressful period for students and instructors.[7]

Clinical education is defined as the integration of knowledge and skills associated with patient care.[8] In this respect, clinical instructors are the most important and influential agents in clinical education.[9] Hence, the success of a clinical education program depends on effective clinical instructors[3] because they are in the best position to assess students’ needs, identify proper learning opportunities, perform instruction and guidance, and conduct fair evaluations.[5]

Nurse instructors are directly responsible for developing students’ abilities in clinical reasoning, decision making, critical thinking, and developing successful interpersonal relationships during clinical education.[10] Therefore, instructors need to be aware of effective teaching practices and be equipped with the necessary competencies to play their crucial roles.[5] The instructor effectiveness is more difficult to be evaluated in complex clinical situations.[11] Nevertheless, this evaluation helps instructors and administrators to improve the quality of their teaching and practice continuously.[12]

Effective clinical instructor

Effectiveness is defined in the Merriam-Webster Online Dictionary as a capability of producing a decided, desired, or intended effect.[13] So, an effective nursing instructor is the person who has the ability to direct the students’ clinical goal achievement. But in some cases, the nursing profession is faced with the theory–practice gap and produces graduates who are not prepared enough to practice competently.[14] It is stated that clinical nurse instructors are not quite effective. Therefore, it is important to understand what sort of behaviors or characteristics with an instructor, as perceived by students or instructors, makes him/her effective.[11] A standard measure for assessing the characteristics of effective clinical nursing instructors can be used to identify common clinical teaching behaviors and discover them from the perceptions of students and instructors.[15] Although some methods are available to assess the effectiveness of clinical nursing instructors, a proper assessment tool should provide accurate and useful information.[16] Therefore, for fulfilling this reason, an efficient and effective instrument is needed.[17] As defined, an instrument is a mechanism for measuring the phenomena, which is used to gather and record information for assessment, decision making, and ultimately understanding.[16] Therefore, the purpose of instrumentation is to generate measures that reduce error in assessing a construct.[18] Using of an inappropriate assessing method or instrument could result in obtaining information that does not answer the question or does not provide useful and accurate information for decision making.[16]

To measure the effectiveness of a clinical nursing instructor, some instruments have been developed. Nursing Clinical Teaching Effectiveness Inventory (NCTEI), developed by Mogan and Knox, is one of the best known such instruments.[19] The NCTEI is reliable and valid and has been extensively used in different sociocultural contexts.[11] It is a 48-item, Likert scale addressing students’ perceptions of effective clinical teaching behaviors. It can be used by either students or instructors, and contains subscales addressing five specific areas: “teaching ability,” “interpersonal relationships,” “personality traits,” “nursing competence,” and “evaluation.”[19] Another instrument is the Effective Clinical Teaching Behaviors (ECTB) inventory developed by Zimmerman and Westfall. It is a 43-item, 5-point Likert scale.[20] There are other instruments proposed by Haag and Schoeps,[21] Reeve,[22] Raingruber and Bowles,[23] and Hou et al.[24]

Most of these instruments have not been used in other sociocultural contexts. Lee believes that despite establishing the reliability and validity of an instrument developed for measuring instructor effectiveness in a specific context, international findings of using this instrument are often inconsistent and vary across cultures.[15]

Clinical nursing education in the Iranian context

In recent years, many studies focusing on clinical nursing education have been performed in the Iranian context. According to these studies, although nursing discipline has been growing in the academic domain, it has faced many challenges such as the theory–practice gap in clinical practice.[25] Cheraghi et al. stressed that the current Iranian clinical environment is not conducive for students’ learning.[26] Iranian nurse students believe that some nursing instructors are ineffective.[27] In addition, most Iranian nurse students have negative attitudes toward clinical learning environments.[28]

According to the qualitative study conducted by Heshmati-Nabavi and Vanaki on the characteristics of effective clinical nursing instructors, such instructors are in harmony with the spirit of nursing, adopt a reflective approach, make clinical learning enjoyable, and provide a patient-centered care.[29] Overall, some characteristics and factors relevant to the effectiveness of clinical nursing instructors reflected in Iranian studies are described as “good clinical supervising,” “having concern on student problems in clinical situations,” “encouraging active participation of students,”[30] “choosing an appropriate teaching style,”[31] “being up-to-date in nursing theory,” “having commitment,” “being practically skillful,” “being self-confident,” “having self-determination,” “making effective interpersonal relationships,”[32] “being interested in clinical education,” “having a clear evaluation process,” “being able to plan for proper course works,”[27] and “supporting of students.”[33]

Although the matter of effectiveness of a clinical nurse instructor has been reflected in some Iranian previous studies, there is no specific, context-based instrument developed for evaluating the effectiveness of the clinical nursing instructor. With regard to the fact that the evaluation of the characteristics of a clinical instructor must be conducted contextually,[34] developing and testing an Iranian context-based instrument is necessary.

Aims

The purpose of this study was to explore the development and psychometric testing of an instrument that can evaluate the characteristics of an effective clinical nurse instructor in the Iranian context.

MATERIALS AND METHODS

This study is a methodological research. Studies of recruiting and retaining subjects and instrumentation are named methodological research.[35] Developing and psychometric testing of this instrument was done through three sequential phases. Phase one included the item generation in which primary items of the instrument were identified through a broad literature review. In phase two, the validity of the primary instrument was assessed and modified. Finally, in phase three, the factor analysis was used to reduce items and identify basic dimensions and reliability of the instrument.

Phase 1: Items generation

According to Hunt (1991), there are two primary approaches to generate items, including deductive and inductive approaches. The deductive approach is used when items are generated on the basis of the theoretical definition of the construct resulting from a thorough review of the literature. Alternatively, the inductive approach involves obtaining responses of participants through an interview to identify the construct.[36] In this study, for generating items, deductive approach was used. In this way, based on key words related to the nursing clinical education, including clinical instructor, and similar terms such as educator, teacher, faculty, mentor, preceptor, and clinical education and learning, and other similar terms including setting, context, and practice, the main nursing studies’ databases were searched. To develop a context-based instrument, the initial focus was placed on Iranian-related studies. Therefore, Iranian databases including SID, Magiran, and IRANMEDEX, alongside the international medical and nursing databases such as MEDLINE, EMBASE, and CINAHL were searched. Language was limited to English and Persian, and no time limit was applied. The abstracts of all founded articles were read by two researchers, and relevant studies, both qualitative and quantitative studies, were included. Full texts of all selected articles were reviewed, and statements that seemed to be directly or indirectly related to effective clinical nursing instructor were extracted. Regarding the selection of indirect statements, researchers’ consensus was considered as the criterion of decision making. Finally, these statements in the form of questions related to the effectiveness of a clinical nursing instructor were arranged.

Phase 2: Validity

The aim of this phase was to assess the validity of the initial instrument. During item generation phase, content and face validity as the two primary concerns need to be considered.[37] In this process, the fundamental main concern is content validity, which may be viewed as the minimum psychometric requirement for measurement adequacy and is the first step in construct validation of a new measure.[36]

Content validity

Content validity is about the degree to which an instrument is representative of the topic and the process being investigated.[38] To assess content validity, we should identify all possible factors that operationalize the construct. But it may be difficult or we may have so many factors that it is not possible to include all of them in the instrument. Therefore, content experts should rate the importance of these factors to determine which are most relevant to the subject of the instrument.[16]

In this study, in order to assess this type of validity, Waltz and Bausell method was used.[39] Hence, 10 faculty members of nursing were recruited as an expert panel. Five of them held a PhD degree and the remaining held the master degree. They have 10 years of experience in nursing education averagely. The initial instrument which had been developed in the first phase was sent to them and they were asked to evaluate about each item regarding item relevancy using a 4-point ordinal rating scale (1: Irrelevant; 2: Somewhat relevant; 3: Quite relevant; 4: Highly relevant).[40] The item-level content validity index (I-CVI) was applied to evaluate to what extent expert panel members agreed on the item relevancy. The I-CVI reveals the proportion of the agreement on each item, and is determined as a function of the total number of included experts. Based on the Lynn guidelines[41] to acceptable I-CVI, regarding the participation of 10 experts in this study, 0.70 was the cut-off point of deciding on removing or preserving each item. After agreement on the relevancy of items, similar to the process of item relevancy, two other criteria including the item clarity and item simplicity were assessed.

Face validity

Face validity can refer to one or all items of a test, and it indicates how well the item reveals the purpose or meaning of the test item or the test itself.[42] To assess the face validity, all items of the instrument were inspected with 10 nurse students and 10 nursing faculty members.

Phase 3: Pilot test

Construct validity

The aim of this phase was to: 1) reduce the instrument's items, 2) determine the underlying structure and dimensionality of items, and 3) identify the reliability of the developed instrument. To this end, an exploratory factor analysis (FA) was used. FA is a technique designed to reduce a set of observed variables (i.e. items) to a smaller set of variables, which reflects the interrelationships among the observed variables. This multivariate technique is also able to determine the underlying structure and dimensionality of a set of variables. By analyzing the intercorrelations among variables, FA shows which variables cluster together to form unidimensional constructs. FA is called exploratory FA when it is used early in developing a scale to identify the number of factors, the correspondence between items and factors, and the quality of items.[43]

Sample and procedure

After getting approval for the fieldwork phase of the research from the research committees of the Nursing and Midwifery Faculty of Tehran University of Medical Sciences and Guilan University of Medical Sciences, through a convenience sampling, 200 undergraduate nurse students of two educational semesters were selected as the research sample in 2011. At first, they were informed about the purpose of the study and the state of their participation. And then, they filled the administered questionnaires. Finally, 168 filled questionnaires were returned to the researchers for data analysis.

Before extracting the factors, some tests must be used to appraise the appropriateness of data for FA. Two main tests are Kaiser–Meyer–Olkin (KMO) Measure of Sampling Adequacy[44] and Bartlett's Test of Sphericity.[45] The KMO measure ranges from 0 to 1, with 0.50 considered as appropriate for the FA. The Bartlett's Test of Sphericity must be significant (P < 0.05) for appropriate FA.[46]

Scree plot can be used to determine the number of factors to retain.[38] In this study, the researchers applied the scree plot for determining of the number of factors which were the best representation of the dimensionality of the instrument. These determined factors were used as foundations for further analysis..

A principal FA with a varimax rotation was applied in all analyses. Criteria for retaining individual items were an absolute factor loading value of ≥0.40. In retaining an individual item with significant cross-loading on a different factor, the consensus among researchers was considered.

Reliability analysis

The internal consistency reliability of subscales was assessed through Cronbach's alpha coefficient [Table 1]. Reliability is the extent to which an instrument produces the same information at a given time or over a period. Assessing the internal consistency as a vital criterion of an instrument's reliability was well documented.[16] The most common method of concluding internal consistency is the Cronbach's alpha.[47]

Table 1

Principal factor analysis with varimax rotation and Cronbach's alpha coefficients

RESULTS

Throughout the item generation phase, as a result of searches in the database, 53 articles related to Iranian context were retrieved. After reviewing these articles, 83 statements that seemed to be related to effective clinical nursing instructors were extracted.

During the step of content validity, 26 items of the instrument were omitted and 3 items were merged in other items. For example, the items “be a member of the clinical team of the ward,” “encourage students’ collaboration,” and “be adopted by members of other clinical disciplines” were omitted. Also, the two items “being present full time in the ward” and “being with students in the clinical situation” were merged as “always be available for students.” After that, the modified instrument was structured in the form of a 54-item, 5-point (1 = strongly disagree, 5 = strongly agree) Likert scale.

Through the step of face validity, based on the advices of students and instructors, some items were modified. For example, the item “has enough knowledge” was replaced with “has enough required nursing knowledge.” After this phase, the instrument with 54 items was ready for pilot test.

In order to study samples for doing pilot study and FA, 200 nursing students were recruited. The participants’ age range was between 19 and 26 years (M = 21.06 years). Most participants were women (92.2%). All of them were undergraduate nursing students from different educational semesters.

These students filled 168 questionnaires. Gathered data from these returned questionnaires were provided for FA. Based on the results of the statistical test, the KMO measure was 0.885 which revealed that the selected samples were adequate to perform the FA. Also, Bartlett's Test of Sphericity was significant (P = 0.00), which indicated the data were factorable.

According to the scree plot, four-factor solutions provided the best representation of the dimensionality of this instrument. After that, the analysis was performed with these four factors as the foundation. The four-factor solution explained 53.58% of the total variance.

The four defined factors as the instrument's subscales were named according to the shared meaning of their related items as “student-centric behaviors,” “clinical performances,” “planning ability,” and “personality traits” [Table 1].

Of the 54 items in the original instrument, 43 items clearly met the criteria for retaining an individual factor. Of the remaining 11 items, two did not meet the 0.40 threshold defined for factor loading and the other 9 items had cross-loading on different factors. Overall, to arrange subscales in the best manner, making some modifications was necessary. Based on this fact, because of closeness in the meaning, item number 19, “encourage students to further learning,” despite having the highest factor loading, was transferred from the second factor to the first factor. In return, item 33, “having a positive attitude toward nursing,” was transferred from factor one to factor two. Similarly, items 12, 18, and 53, which were “allow the students to make a decision independently,” “give a critical view to students,” and “allow self-evaluation opportunity to students,” respectively, were moved from factor three to the first factor. Also, item 43, “introduce valid scientific resources to students,” was moved from factor two to factor three.

Regarding cases with cross-load, nine items including 7, 11, 20, 27, 28, 32, 42, 44, and 48 had equal factor loading in two factors and based on the meaning closeness were allocated to the proper factor. In addition, two items, 35 and 49, which were “place the practice on the patient's needs” and “allocate enough time for students’ evaluation,” were omitted because of a low factor loading.

Based on the reliability analysis, the results show that the instrument's internal consistency was high (α = 0.96, ranging from 0.87 to 0.94 for the subscales).

DISCUSSION

This instrument finally had 52 items grouped into four subscales. These subscales, including student-centric behaviors, clinical performances, planning ability, and personality traits, encompass 54 characteristics/behaviors of an effective nursing clinical instructor.

The subscale student-centric behaviors includes 18 items that represent the humanistic approach of instructor in the clinical education process. According to the Iranian cultural context originated in the Islamic and Persian perspectives, treating others in a respectful way is an essential sociocultural value. These behaviors are also stressed by the current paradigm in education that has a humanistic and student-centric perspective. This concern is in line with what is stressed as caring behaviors of instructors in the global nursing educational context. On the basis of Watson's theory, some instruments with the focus on caring behaviors of an instructor were developed. Cronin and Harrison developed the Caring Behaviors Assessment (CBA) tool. The main subscales of their instrument were: humanism/faith-hope-sensitivity, helping/trust, expression of positive/negative feelings, teaching/learning, supportive/protective/corrective, human need/assistance, and existential/phenomenological.[48] Similarly, Wade and Kasper developed an instrument with subscales of instills confidence through caring, supportive learning climate, appreciation of life's meanings, control versus flexibility, and respectful sharing.[49]

The other main factor related to the effectiveness of the clinical nursing instructor is the clinical performance of the nursing instructors, consisting of 14 items. Since nursing is a practical discipline, practice has an essential role in nursing education. Therefore, it is not surprising that clinical skills of an instructor are regarded as a main expected factor of the instructor's effectiveness. Other similar instruments already have stressed on this factor. In the NCTEI instrument developed by Knox and Mogan, the subscale of nursing competence is similar to what appears in the present instrument as clinical performance.[19] In Clinical Instructor Evaluation Instrument (CIEI) developed by Haag and Schoeps, this factor is reflected in terms of professional competence.[21] Also, in the Clinical Nursing Faculty Competence Inventory developed by Hou et al., clinical nursing skills of instructors appeared as a main subscale.[24]

The third factor, planning ability, with 14 items has a particular focus on the teaching abilities of instructors. This factor was also mentioned by Knox and Mogan in the NCTEI instrument in terms of teaching ability and by Haag and Schoeps in the CIEI in terms of teaching practices. Similarly, Hou et al., in their instrument Clinical Nursing Faculty Competence Inventory, emphasized on general teaching abilities as one of their instrument's subscales. It is important to note that although the instructors are expected to have a wide range of educational management abilities, but in this study, the planning ability was more emphasized than other abilities. One reason of this concern may be the complexity of clinical settings and the importance of a proper plan to overcome this complexity.

The final factor, personality traits of instructors, reflects the personality characteristics necessary for a person in the instructor role. This factor includes six items. Similarly, one of the factors reflected in the NCTEI instrument[19] and also in the CIEI instrument[21] is personality traits.

CONCLUSION

The results of the psychometric testing presented in this study reveal that the Iranian Effective Clinical Nurse Instructor (IECNI) evaluation tool contains four subscales and is internally consistent (α = 0.96). The psychometric properties of this instrument indicate that it is a valid and reliable measure of the clinical nurse instructor's effectiveness in the Iranian context.

Regarding the lack of a trustful and objective method and also a context-sensitive, valid, and reliable instrument to measure the effectiveness of nursing instructors, development of this IECNI evaluation tool is enormously useful. This instrument, because of having Iranian conceptual frameworks – derived from Iranian researches – for generating initial items, can be viewed as a context-based instrument to be applied in the Iranian context. This instrument can be used by instructors as a self-reflecting instrument to increase effectiveness. In addition, it can be applied by nursing administrators in evaluating the educational performances of nursing instructors for different purposes such as internal and external accreditation of nursing schools. Also, by means of this instrument, nursing students can become capable of providing an objective feedback on their instructors. This instrument may be useful for nursing and other health professional researchers to consider clinical instructor's effectiveness as a critical educational measure. In addition, this instrument can be used to measure students’ and instructors’ perceptions of clinical nursing instructor effectiveness. Overall, the IECNI evaluation tool is able to give an objective reflection of the status of nurse instructors’ effectiveness that acts as an essential foundation for further quality promotion.