Psychometric evaluation of the perceived nursing student's Incivility questionnaire.

AIM: Student's uncivil behaviour is one of the most common problems in the educational setting, including nursing schools. It is essential to develop tools for measuring the uncivil behaviour of nursing students to solve this problem. Therefore, this study aimed to evaluate psychometric properties of perceived nursing student's incivility questionnaire among the Iranian community.
DESIGN: In this methodological study, perceived nursing student's incivility questionnaire was completed by 360 nursing students and 121 nursing faculty members.
METHODS: Sampling was done from October 2019-November 2019. Content and construct validity of the questionnaire were evaluated. Reliability was estimated using Cronbach's alpha and McDonald's omega coefficients and composite reliability. The construct validity of nursing student's perceived incivility was investigated by exploratory and confirmatory factor analyses.
RESULTS: Content validity index 0.88 for the whole instrument. The three factors of violent behaviours, irresponsible behaviours and unsound behaviours explained more than 51.485% of the variance. Factor structure extracted using model fit indices (PCFI = 0.763, PNFI = 0.732, CMIN/DF = 2.501, RMSEA = 0.056, GFI = 0.941, AGFI = 0.918, CFI = 0.935) and convergent validity were also confirmed. Internal consistency and composite reliability were estimated to be more than 0.7. The results showed that Iranian perceived nursing student's incivility questionnaire is a three-dimensional construct with good validity and reliability.

INTRODUCTION

Uncivil behaviours include violent, rude and disrespectful behaviours that occur in the classroom as inappropriate interactions between students and faculty (Rawlins, 2017). There is a range of non‐verbal stimulating behaviours such as eye rolling, anger, threats and violence (Abedini & Parvizy, 2019). Such relationships are not accepted in the nursing education environment (Ziefle, 2018). Many students have the knowledge and skills to care, but they are unable to communicate properly, interact effectively and demonstrate professional behaviour. (Sprunk et al., 2014). Understanding the ethical codes and the respectful relationship between colleagues and patients is an integral part of the nursing profession and an important factor in promoting nursing culture (Milesky et al., 2015).

In previous studies, most students and faculties reported incivility among nursing students (Burke et al., 2014). Ibrahim showed that 60.2% of students had irresponsible behaviour and 47.8% of students had aggressive behaviour in the educational environment. Two‐thirds of students stated that they had witnessed malicious behaviour at least once (Ibrahim & Qalawa, 2016). In a similar study, 68% of nursing professors reported high incidence of incivility among nursing students (Rawlins, 2017). More than 50% of beginner nurses also find themselves involved in students' destructive behaviours, and 90% of them have witnessed such behaviours in the clinical setting (Clark, 2017).

Assessing the problem of incivility in nursing education can be the first step in recognizing the problem and improving students' performance in the nursing school (Abedini & Parvizi, 2019). In this regard, Masoumpoor (2015) designed and investigated the psychometric properties of incivility in nursing education's tool. Data analysis led to the design of questionnaires for teachers and students. Content validity was confirmed and after exploratory factor analysis, 3 factors were identified with 37 items that were scored on a 5‐point Likert scale (Masoumpoor, 2015). This tool only considers the teacher's view of nursing student's incivility, and its validity and reliability have not been investigated in any study, so it seems that the Clark questionnaire is more appropriate.

A critical review of incivility assessment tools using the systematic review method indicated that all of the tools were designed for the target population and were not specific to the nursing field. Tools were completed by students or professors. Clark's incivility questionnaire examines the behaviours of students and professors from both perspectives. Its benefits were fewer questions than similar tools, specificity for nursing students compared to general tools for measuring civil behaviour, validation in previous studies and revision in consecutive years by the instrument designer (Masoumpoor et al., 2017). The Clark questionnaire is a comprehensive tool that covers physical, verbal, non‐verbal behaviours and technology use in the classroom. The questionnaire has been translated and used in more than 10 languages in different countries (Al‐Jubouri et al., 2019). Simplicity has made it a suitable tool for evaluation (De Gagne et al., 2016).

The new version of the incivility questionnaire is an enhanced version of the original questionnaire, and a self‐report tool that was reviewed by Clark in 2014. The revised version of the tool contains demographic information and 24 cases of uncivil student's behaviour in the past 12 months, which is assessed on a four‐point Likert scale (not uncivil, somewhat uncivil, moderately uncivil and highly uncivil). The scores range from 24–96, and the overall score is calculated as the mean (Clark et al., 2015). Considering that performance assessment, prevention and control of uncivil behaviour require a valid and comprehensive tool and psychometric properties. The Clark questionnaire is unknown in the Iranian population, so it is necessary to examine the appropriateness of the tool with respect to the Iranian culture. The aim of this study was to evaluate the psychometric properties of perceived nursing student's incivility questionnaire among Iranian community. It is the student's incivility section of Incivility in Nursing Education‐Revised (INE‐R). According to Clark et al. (2015) INE‐R may be separated into students and faculty incivility sections. Student section is used to measure perceived students incivility from both nursing faculty and student point of views (Clark et al., 2015). Past psychometric evaluations of INE‐R were conducted on student section of this questionnaire (De Gagne et al., 2016).

METHODS

The present study is a methodological research that was conducted to evaluate the psychometric properties of the Persian version of perceived student's incivility questionnaire in the Iranian community. The minimum number of samples for factor analysis and construct validity is 5–10 samples per item (Kellar & Kelvin, 2013). The sample size was 481 people. Sampling was done from October 2019–November 2019. Inclusion criteria for this study included students of Iranian nationality, at least two semesters in college and willingness to participate in the study. After obtaining the permission of the tool designer, the questionnaire was translated into a forward‐backward method. Initially, two English‐language nursing specialists translated the questionnaire into Persian. The translated version was then translated into English by two other nurses (PhDs and English specialists), and a final copy was prepared. The final version was sent to the tool designer for approval.

Quantitative and qualitative methods were used to confirm face validity. In the qualitative section, 10 students were interviewed. The difficulty level, proportion and ambiguity of questions were evaluated. All of items were confirmed in this stage. Quantitative face validity of the questionnaire was assessed based on two options scale including “Yes” and “No” that indicated desirable and undesirable items. The data were analysed using Cohen's Kappa Index (CKI). The content validity of the tool was evaluated by both qualitative and quantitative methods. In the qualitative evaluation, the Persian version of the questionnaire was given to 15 nursing faculty members and 5 nursing students to state their opinion based on grammar criteria, usage of appropriate words, necessity and putting the phrases in the right place. The questionnaire was then modified based on brief grammatical suggestions. Quantitative content validity of the questionnaire was assessed based on two content validity ratios (CVR) and content validity index (CVI). To calculate the CVR, 15 experts were asked to assess the necessity of each item in the questionnaire. To determine the value of the content validity ratio index, the expression for which the calculated value was greater than 0.42 on the basis of Lawshe table was considered statistically significant (1 = unrelated, 2 = somewhat relevant, 3 = related, 4 = fully relevant; Lawshe, 1975). The CVI was calculated by dividing the number of experts giving each item a score of 3 and 4 on the panel's overall relevance, with a score of 0.88 and above considered acceptable. The mean content validity index scores of all items were used to calculate the average content validity index.

Factor analysis and comparison of well‐known groups were used to confirm the validity. In the first step, the factors were extracted by exploratory factor analysis. Sampling adequacy index (Kaiser‐Meyer‐Olkin) and Bartlett test were calculated. Then, the extraction of latent factors was performed by SPSS25 software. The presence of an item in the factor was determined to be approximately 0.3 based on the formula CV = 5.152√(n−2) (CV was the number of extractable factors and n was the sample size of the study). At least three items per latent variable should be in each factor (Soleimani et al., 2017). The extracted factors were verified by confirmatory factor analysis and maximum likelihood calculation. GFI was calculated using AMOS5 software.

To evaluate convergent validity, divergent validity and structural reliability based on the Fornell and Larker approach, and composite reliability (CR) mean variance extracted (AVE), mean shared square variance (ASV), maximum shared squared variance (MSV) were measured (Hair et al., 1998).

To assess the relative reliability (test‐retest), a questionnaire was sent to 30 nursing students and faculty members in 2 weeks. Intra‐cluster correlation coefficient (ICC) was calculated using two‐way mixed effects model. For evaluation of internal consistency, perceived incivility questionnaire, Cronbach's alpha and McDonald's omega coefficients of reliability were calculated, and CR values were considered as good reliability (36). Absolute reliability was estimated using standard error of measurement (SEM) and formula SEM = SD√ (1−ICC; Javali et al., 2011).

RESULTS

The study sample included 360 students and 121 faculty members. The mean age of students was 23.21 ± 9.22 years. Most of the students were single (87.5%) and female (65.7%). The mean age of faculty members was 45.13 ± 3 3.15 years and most of them were female (94.5%) and married (86.2%). The correlation coefficient of students and faculties responses was calculated, and after confirming the inter‐rater agreement between their point of views (Pearson correlation coefficient = 0.83), the validity and reliability of the instrument were evaluated.

Results confirm the face validity in terms of readability, feasibility and consistency. Cohen's Kappa Index was calculated 0.86. The values above 0.6 are acceptable (Taherdoost, 2016). The content validity was verified after the tool modification by specialists and the necessary grammar modifications. The results were also calculated from content validity index and content validity ratio. These were at the acceptable level. CVI and CVR values for items was 0.88 and 0.93, respectively, and none of the questions were omitted.

According to the results of exploratory factor analysis, sampling adequacy index (KMO) was 0.874 and Bartlett's test 2,625.511 (p < .001). The three extracted factors including violent behaviours, irresponsible behaviours and unsound behaviours explained 51.485% of the total variance of uncivil behaviour variables (Table 1).

TABLE 1

Exploratory factors extracted from Nursing Students' Perceived incivility Questionnaire

Factor	Item	Factor loading	h2	ʎ	%Variance
Violent behaviours	Q21. Using profanity directed toward others	0.767	0.569	3.330	27.179
	Q22. Threats of physical harm against others (implied or actual)	0.707	0.479
	Q23. Property damage	0.699	.458
	Q20. Making discriminating comments(racial, ethnic, gender, etc.) directed toward others	0.686	0.484
	Q19. Sending inappropriate or rude e‐ mails to others	0.554	0.410
	Q14. Making condescending or rude remarks toward others	0.522	0.316
	Q2. Making rude gestures or non‐verbal behaviours towards others	0.438	0.234
Irresponsible behaviours	Q6. Arriving late for class or other scheduled activities	0.780	0.575	3.537	18.553
	Q7. Leaving class or other scheduled activities early	0.757	0.537
	Q8. Being unprepared for class or other scheduled activities	0.643	0.495
	Q5. Using a computer, phone, or other media device during class, meetings, activities for unrelated purposes	0.580	0.393
	Q4. Refusing or reluctant to answer direct questions	0.460	0.236
	Q9. Skipping class or other scheduled activities	0.366	0.241
	Q3. Sleeping or not paying attention in class (doing work for other classes not taking notes, etc.)	0.338	0.295
Unsound behaviours	Q17. Demanding a passing grade when a passing grade has not been earned	0.530	0.443	2.650	5.753
	Q16. Ignoring, failing to address, or encouraging disruptive behaviours by classmates	0.492	0.356
	Q15. Demanding make‐up exams, extensions, or other special favours	0.427	0.230
	Q13. Cheating on exams or quizzes	0.350	0.356

Abbreviations: h2, communality; ʎ, Eigenvalue.

In confirmatory factor analysis, after correcting the model, goodness of fit of chi‐square was obtained, p < .001, x2 = 245.083. Then, other fitting indices of the model were investigated. All indices PCFI = 0.763, PNFI = 0.732 (acceptable above 0.9), RMSEA = 0.056 (less than 0.05), GFI = 0.941, AGFI = 0.918, CFI = 0.935 (acceptable above 0.9), x 2/  = 2.5 (less than 3 acceptable; Table 2), confirming the suitability of the final model (Figure 1). Factor load values were more than 0.3 and significant (Table 1). Internal consistency and composite reliability of the questionnaire were higher than 0.7 also, the standard error of measurement was estimated (Table 3). ICC was 0.8.

TABLE 2

Fit indices of the first‐ and second‐order confirmatory factor analysis

CFA Index	PGFI	CFI	GFI	AGFI	PNFI	PCFI	RMSEA	CMIN/DF	p‐value	df	X2
Indices	0.678	0.935	0.941	0.918	0.722	0.763	0.056	2.501	.00	98	245.83

Abbreviations: AGFI, adjusted goodness‐of‐fit index; CFA, confirmatory factor analysis; CFI, comparative fit index; CMIN/DF, chi‐square/degree‐of‐freedom ratio; IFI, incremental fit index; PCFI, parsimonious comparative fit index; PNFI, parsimonious normed fit index; RMSEA, root mean square error of approximation.

Fit indices: PNFI, PCFI, AGFI (>0.5), CFI, IFI (>0.9), RMSEA (<0.08), CMIN/DF (<3 good, <5 acceptable).

FIGURE 1

Construct: modified model of first‐order confirmation factor analysis

TABLE 3

Convergent and divergent validity and internal consistency

Factor	Ω	α	CR	MaxR(H)	AVE	MSV	ASV
Violent behaviours	0.787	0.787	0.828	0.835	0.447	0.226	0.832
Irresponsible behaviours	0.809	0.806	0.821	0.838	0.401	0.587	0.804
Unsound behaviours	0.640	0.633	0.606	0.613	0.340	0.587	0.802

Abbreviations: AVE, average variance extracted; ASV: average shared squared variance; CR, construct reliability; MSV, maximum shared squared variance;α, Cronbach's alpha coefficients; Ω, McDonald omega coefficient.

DISCUSSION

The purpose of this study was to assess the psychometric properties of the Persian perceived nursing student's incivility questionnaire. Individuals' perceptions of incivility are culturally dependent and vary across societies. It is therefore necessary to evaluate the characteristics of the tools in each context separately. The results showed that the three factors extracted from the questionnaire accounted for more than 51.485% of the variance. The variance more than 50% indicates the appropriateness of factors (Al‐Jubouri et al., 2019). The three factors of violent behaviours, irresponsible behaviours and unsound behaviours and the explanation of the total variance with the instrument designer are in agreement. In Clark's questionnaire, three levels of threatening, irresponsible and inappropriate behaviour were identified (Clark et al., 2009). Ibrahim and Qawala also assessed students' incivility at three levels of aggressive, irresponsible and inappropriate (Ibrahim & Qalawa, 2016). Arabic version of this questionnaire including five factor model (Al‐Jubouri et al., 2019).

The first factor identified in the questionnaire was violent behaviour. All 7 items were highly correlated with their factor and expressed Iranian students' perceptions of perceived incivility. Questions indicate understanding of violent and uncivil behaviour that is consistent with the construct of threatening behaviours in the continuum model of incivility in nursing education and shows the high level of incivility, meaning that students' violent behaviour is incivility. In the similar study, this factor has been identified as high‐level uncivil behaviour (De Gagne et al., 2016). These items identified as the higher level of uncivil behaviours in the Arabic version of questionnaire (Al‐Jubouri et al., 2019). These actions were marked as the most incivility regardless of the culture of students.

The second factor in this questionnaire was irresponsible behaviour. This factor measures student's perceptions by 7 items. Elements of this factor include irresponsibility and disorder. Irresponsible behaviours in Clark questionnaire are also introduced as one of the dimensions of incivility (Clark et al., 2009). In another study, this factor was assessed as a low level of incivility (De Gagne et al., 2016).

The third factor extracted from the questionnaire was unsound behaviours, which is equivalent to inappropriate behaviours in the main instrument (Clark et al., 2009). An evaluation also showed uncivil behaviours as high‐level, low‐level and very low‐level behaviours, which is consistent with the three factors of the present study (De Gagne et al., 2016). Irresponsible behaviours in the continuum model of Clark's incivility have been perceived as low‐level behaviours (Clark et al., 2015).

The first and second factors in this study included violent and irresponsible behaviours that are similar to the high level of incivility in Clark's study (Clark et al., 2015). Thus, the first factor items or violent behaviours (21, 22, 23, 19, 14 and 2) and the third factor items or unsound behaviours (13, 15, 16 and 17) are high perceived levels. Gagne identified these items as high levels of behaviour. Second factor items (3, 9, 4, 5, 8, 7 and 6) or irresponsible behaviours that are less severe than violent or unsound behaviours have been identified as low levels of uncivil behaviour in past studies (De Gagne et al., 2016).

Cronbach's alpha and McDonald's omega coefficients of CR were higher than 0.7 indicating optimal reliability for the first and second factors. The alpha coefficient for the whole instrument and the three known factors indicated its appropriate reliability. The coefficient alpha in the original version of the tool was 0.889 between items and values (Clark et al., 2015). Based on the findings, this tool is reliable and repeatable. Reliability assessment in the Korean student population was also 0.94. These values were 0.95 for high‐level incivility and higher than 0.75 for low‐level incivility (De Gagne et al., 2016). The Cronbach's alpha for all items was 0.87 in the Arabic version of instrument (Al‐Jubouri et al., 2019).

In the present study, convergent validity and divergent validity showed that all factors had good convergent validity and divergent validity of the first and the second factors were confirmed. There is a convergent validity when the structures in question are close together and share a great deal of variance, and for divergent validity, the structural elements or latent factors must be completely separated. Relative reliability of the questionnaire and optimal CR were detected. Convergent validity must be AVE < 0.5 and CR > AVE, and divergent validity must be >MSV AVE. The obtained values indicate convergent validity and closeness of the item (Hair et al., 1998).

Questions omitted based on factor analysis results include (1) disinterest or indifference to the lessons or content of the syllabus; (12) outline side effects that distract oneself or others, 18) neglect in response to email or other communications, and (24) threatening statements about weapons. These items were not the three factors of violent, irresponsible and abusive behaviours. Disinterest in the lesson, not being intimate and overcoming the class and outlining the topics were not incivility, according to study participants. These behaviours do not appear to be detrimental to the educational process and are not understood as a problem. Threatening statements about weapons also do not fit into the native culture of Iran and did not exist in the intended educational setting. In similar studies, no items have been omitted indicating differences in their culture and social conditions with Iranian student (Clark et al., 2015).

The alpha and CR values for the two factors for violent and irresponsible behaviours were above 0.7 and for the third factor were close to 0.7. The third factor or misbehaviour in previous studies was the high level of incivility that has been identified as a distinct factor in the present findings. The difference in alpha values and invalidity of divergent validity in the third factor may be due to the similarities of the factors with the first factor (violent behaviours). In past studies, this factor has been combined with violent behaviour (De Gagne et al., 2016), whereas in the present study it is identified as a separate factor. It should be noted that the desired values are close to the expected value.

The discrepancy in the findings of the study could also be related to the number of students and professors participating. The study included 358 students and 122 faculty members. In the Gagne study, there were 284 students and no faculty (De Gagne et al., 2016). The sample in the Clark survey was 310 students and 182 faculty members (Clark et al., 2015). Three hundred eighty five nursing students participated in a similar survey (Al‐Jubouri et al., 2019). It is possible that different understanding of students and faculty members has effects on findings. A trait of this tool is that the same questionnaire can be completed by both faculties and students. Researchers that assessed either faculties or students viewpoints do not acquire clear depiction of uncivil behaviours (Clark et al., 2015).

This questionnaire is a self‐report tool, and the results are influenced by the accuracy of the students and professors in completing the answers. The data also included the responses of professors and students. The number of professors and students participating and the differences in views between the two groups can be biased. On the contrary, cultural and class differences between samples may effect on the results.

CONCLUSION

The structure of students' perceived incivility questionnaire has an acceptable factor structure and its internal consistency is confirmed. This questionnaire can be used to measure the level of perceived incivility of Iranian students and to take appropriate measures to promote civility.

CONFLICT OF INTEREST

None.

ETHICAL STATEMENT

This study was approved by the Ethics Committee of Qom University of Medical Sciences in Iran. All participants were told about the goals and their consent to participate in the study was obtained. (Code of Ethics: IR.MUQ.REC.1396.46).