A Study on the Psychometric Properties of Revised-nonverbal Pain Scale and Original-nonverbal Pain Scale in Iranian Nonverbal-ventilated Patients.

BACKGROUND AND AIMS: The nonverbal pain scale is one of the instruments which study pain in nonverbal-ventilated patients with regard to the changes of behavioral and physiological indices. The purpose of the study is to survey the psychometric properties of revised-nonverbal pain scale (R-NVPS) and original-nonverbal pain scale (O-NVPS) in ventilated patients hospitalized in critical care units.
MATERIALS AND METHODS: Four nurses studied pain in sixty patients hospitalized in trauma, medical, neurology, and surgical critical care units using R-NVPS and O-NVPS at six times (before, during, and after nociceptive and nonnociceptive procedures). The test was repeated in 37 patients after 8-12 h.
RESULTS: Cronbach's alpha coefficient for R-NVPS and O-NVPS was 0.8 and 0.76, respectively. The inter-rater correlation coefficient during different times was r = 0.89-0.96 for R-NVPS and r = 0.80-0.87 for O-NVPS. Test-retest correlation coefficient for R-NVPS and O-NVPS was r = 0.55-0.86 and r = 0.51-0.75, respectively. The meaningful difference in pain score between nociceptive and nonnociceptive procedures (P < 0.001) and a higher pain score in patients who confirmed pain (P < 0.001) showed a discriminant and criterion validity for both scales of NVPS, respectively.
CONCLUSIONS: R-NVPS and O-NVPS can both be used as valid and reliable scales in studying pain in ventilated patient. However, in comparing the items, "respiration" (R-NVPS) had a higher sensitivity than "physiology II" (O-NVPS) in assessing pain.

INTRODUCTION

Patients who are hospitalized in critical care units continuously experience pain and discomfort due to being subjected to painful therapeutic and diagnostic methods.[1] Assessment is the first step in pain sedation and is one of the most important goals in patient care.[2] Since pain is a subjective phenomenon, the most authentic instrument for its assessment is the patient's own report.[34] However, due to the effect of sedatives, ventilators, changes in the level of consciousness,[5] or cognitive disorders,[6] the majority of the patients in the critical care unit are unable to communicate or report the pain orally.[78] Therefore, the process of assessing pain in these patients is very difficult.[910] Studies have shown that in the absence of the patient's own report, changes in the behavioral and physiological indicators are important criteria for the assessment of pain.[111213] Accordingly, pain assessment scales for critical unit patients have improved in the last decades. One of these scales is nonverbal pain scale (NVPS) which was used for the first time in 2003 by Margaret Odhner et al. for the assessment of pain in critical care burn unit patients.[14] The scale designed by Odhner had five dimensions: face, activity, guarding, physiological I (including blood pressure, heart rate, and respiratory rate), and physiological II (including dilation of pupils, diaphoretic, flushing, or pallor).[141516] Later studies have shown that the autonomic indicator does not have a good correlation with the other dimensions and with the scale as a whole.[1617] For this reason, NVPS was revised by Wegman in 2005. In the revised version, the autonomic indicators were replaced by “respiratory” assessment.[17] In the study done by Kabes et al., the superiority of revised-NVPS (R-NVPS) over original-NVPS (O-NVPS) was emphasized.[16]

From the time of its design to the present, the efficiency of NVPS in different societies and on different samples has been evaluated, and in most cases, its validity and reliability have been confirmed.[15161819] However, few studies have compared the two versions of NVPS. Moreover, some of the studies have pointed to the ambiguities in the other dimensions of NVPS, such as physiological I in such a way that the reliability of vital signs as an indicator of pain assessment has been put under doubt.[91520] On the other hand, regard to the increase in the score of physiological items in nonnociceptive situations,[1202122] little attention has been paid to the pain cutoff point in the majority of the studies. In this research, we will try to survey the psychometric characteristics of the two NVPS versions and to determine the pain cutoff point for both of the scales.

MATERIALS AND METHODS

The present study is a methodological[23] research which has been done to translate and determine psychometric properties of NVPS. The translation and the back translation of the instrument were carried out after getting permission from an NVPS designer. In this study, NVPS had six items which were designed in two models, four of which, including activities, facial expressions, guarding, and physiology I, were similar in both models. The 5th item, however, differed in the two models. The first model O-NVPS included physiology II, while the second model R-NVPS included ventilation/respiration capacity [Table 1].

Table 1

Adult nonverbal pain scale

The study population included all patients admitted to medical, surgical, trauma and neurological ICUs in three hospitals affiliated with Ardabil University of Medical Sciences (consisting of 35 beds). The inclusion criteria of the research were age restrictions (a minimum of 18 years of age was required); being under ventilation for >24 h; the ability to hear and respond with the head, eyes, or eyebrows; getting a score between −3 and +1 on the basis of Richmond Scale; and getting an consciousness score of 8 or higher on the basis of Glasgow coma scale. According to the exclusion criteria, all of the patients who suffered from quadriplegia, extensive damage to the face and arms, and muscular functional disorders, together with those who received neuromuscular blocking drugs and those who were addicted to narcotics, were excluded from the research. In the period of 4 months, of 86 patients who met the inclusion criteria, 26 were excluded from the study (9 patients due to extubation, 10 due to the sudden drop of consciousness, and 7 due to their families’ dissatisfaction were removed from the section before collecting data).

To carry out observation and to collect data, two nurses, after receiving 6 h of theoretical and practical education on the goals of the research and on how to fill out the questionnaire, began collecting samples as the raters of the research. First rater as the main rater observed patients in both the pretest and retest, but the second rater observed the patients only in pretest for collecting data to determine the inter-rater reliability. NVPS was completed six times on each patient and each time by two raters. Each patient underwent a nociceptive (turning) as well as a nonnociceptive (washing the eyes with normal saline) procedure. Two nurses stood on the two sides of the beds and looked at the patients. They were simultaneously, although independently, observed the patient, first during three stages of nonnociceptive procedure (15 min before [time 1], during [time 2], and 15 min after [time 3] the procedure) and then after 20 min, during three stages of nociceptive procedure (15 min before [time 4], during [time 5], and 15 min after [time 6] the procedure). Before finishing the observations, the raters did not inform about the scoring of each other. To determine the stability of the test, the first rater repeated pain assessment for each patient 8–12 h later. Given the complexity of patients’ conditions in the Intensive Care Unit (ICU), it is difficult to equalize the conditions of pretest and retests in these wards. Despite the short time interval between test and retest, some participants were excluded from the study because of change in consciousness level and Richmond Agitation-Sedation Scale score and/or extubation. Finally, the scales were refilled in 37 participants 8–12 h later by the first rater. According to the previous studies,[9112425] short time after pretest is the acceptable period for the retest because of minimum changes in the conditions of the patients in ICUS.

In each case, the data were analyzed by proper test. In this regard, descriptive statistics and inferential statistics were used. Cutoff point for specifying pain in both versions was determined using the receiver operating characteristic curve and the highest sensitivity and specificity of the instrument.

This study is a part of an MS dissertation in critical care nursing, and permission was obtained from the Ethics Committee of Ardabil University of Medical Sciences. Due to the fact that the patients did not have the ability to express themselves, written satisfaction was taken from those who accompanied them.

RESULTS

In this study, the information gathered from sixty contributors was analyzed. 360 pairs (720) of observations were done to assess discriminant validity, criterion validity, and inter-rater reliability, while 222 observations were done for retest, and in total, 942 pain observations were done on the contributors. The mean age of the patients, of whom 40 (66.7%) were men, was 61 ± 21.19 years. Twenty-five patients contributing to the study were hospitalized in medical units (41.6%), 16 in multiple trauma unit (26.6%), 12 in surgical unit (20%), and 7 in neurology unit (11.6%). The patients’ level of consciousness was 8, 9, and 10 on Glasgow scale. The average of Richmond scale was between +1 and −3. The sedatives received by the patients included midazolam, fentanyl, morphine, and pethidine, while 33 patients (55%) did not receive any sedatives.

In assessing discriminant validity, the research hypothesis was that the pain score would increase during the nociceptive procedure compared with the resting time, and it would remain unchanged during the nonnociceptive procedure. At this stage, there was a meaningful difference in score for both scales in different situations. The Wilcoxon test showed that in both versions of NVPS, the pain score in nociceptive condition had a meaningful increase compared with resting (P < 0.001 and P < 0.001) and nonnociceptive (P < 0.001 and P < 0.001) situations. Moreover, it was shown that the pain score in nonnociceptive condition increased when compared with the resting time (P < 0.001 and P < 0.001). Table 2 shows the score difference between the items of both versions in different situations.

Table 2

Discriminant validity of nonverbal pain scale items during resting (T1), during nonnociceptive procedure (T2), during nociceptive procedure (T5)

To assess the criterion validity, reports were used in which patients confirmed or denied the existence of pain through head and eyebrow gestures. In total, the patients denied the existence of pain in 269 different situations, while they confirmed it in 91 situations, and there was a meaningful difference in the overall pain score and the items of both NVPS versions [Table 3].

Table 3

Criterion validity nonverbal pain scale items (the patient's self-reported pain was used as the gold standard)

Table 4 shows the score changes of each item in situations of the existence and nonexistence of pain. In analyzing the item scores in different situations, it was shown that the “vital signs” item in R-NVPS and “physiology II” in O-NVPS have the weakest correlation with the overall score [Tables 5 and 6].

Table 4

Changes of pain score in situations of pain report and no pain report by patients

Table 5

Average of items and its relation to the total score in different situations by using Spearman rho test in revised nonverbal pain scale

Table 6

Average of items and its relation to the total score in different situations by using Spearman rho test in original nonverbal pain scale

Cronbach's alpha coefficient for R-NVPS and O-NVPS in the overall six times (n = 360) was 0.80 and 0.76, respectively. The correlation coefficient between the two raters in six observations was assessed through intraclass correlation coefficient test, by which the range of 0.89–0.96 and 0.87–0.97 was acquired for R-NVPS and O-NVPS, respectively. In doing the test and retest, the Spearman rho test was used in different situations and they were at the range of 0.55–0.86 and 0.51–0.75 for R-NVPS and O-NVPS, respectively.

In determining the pain cutoff point, it was shown that the pain cutoff point for O-NVPS, with the sensitivity and specificity of 95.6 and 97.4, was 1.5. Furthermore, with the sensitivity and specificity of 95.6 and 96.3, the pain cutoff point for R-NVPS was 1.5.

DISCUSSION

In this study, to achieve the purpose of the research, “determining the psychometric characteristics of O-NVPS and R-NVPS in patients under ventilation,” six items of the scales were put under analysis. The results showed that both O-NVPS and R-NVPS had a proper discriminant validity, in such a way that both versions and all their items showed a higher score in nociceptive situations (turning) compared to nonnociceptive (washing the eyes with normal saline) ones. Other studies in the field have shown that the overall score of R-NVPS in the nociceptive procedure is considerably higher than the nonnociceptive one.[14151618] This shows that NVPS has a very good discriminant validity in discriminating a nociceptive procedure from a nonnociceptive one. However, it was shown that the pain score in nonnociceptive procedure had a meaningful increase compared with the resting condition. In studying the items of R-NVPS and O-NVPS, it was made clear that, among all of the items, only “face” had increased in the nonnociceptive condition and was responsible for the increase of both scales’ pain score in the nonnociceptive procedure. Since the item “face” consists of options such as shedding tears, frowning, and shrinking the forehead, and since the nonnociceptive procedure in this study has been eye care, then the patient's response to the nonnociceptive touching of the eyes can be the natural reaction to the touching of the face. Of course, this can be related to the limitation of both of the scales in some of the nursing procedures. However, in this challenge, paying attention to the pain cutoff point gains in significance because, with the increase of some of the items (such as face) in nonnociceptive procedure, it becomes very important to determine the real pain score.

To determine the criterion validity of NVPS, the patients’ own reports regarding the existence or nonexistence of pain during each stage were used as the gold standard. The results of the study showed that the patients who reported the existence of pain got a meaningfully higher pain score in both NVPS versions and in all of their items than those who did not report pain. However, what is notable is that the “physiology II” item in O-NVPS scarcely changed in nociceptive and nonnociceptive situations, in such a way that, although the patients reported pain in 91 situations, this item changed only in 8.8% of the situations [Table 4]. This shows that, in spite of the acceptable validity and reliability of O-NVPS in all of the situations, the “physiological II” item does not have enough sensitivity for discriminating pain in different situations. In this respect, the item “physiology I” also showed little change in nociceptive situations, because only in 19.8% of nociceptive situations the “vital signs” score witnessed an increase. The situation for the item “respiration,” which replaces “physiology II” in R-NVPS, becomes a little better as its score has increased in 40.7% of nociceptive situations. However, “respiration” is not very good item also because it can be influenced by elements other than pain.

To determine the internal consistency of NVPS, Cronbach's alpha coefficient was used, which was acceptable in the overall observations for R-NVPS and O-NVPS. This shows that all of the items of both versions have an acceptable relationship with each other. The previous studies confirm our findings in this respect since, in the studies which assessed the internal consistency of NVPS for all situations with high sample volume, an acceptable Cronbach's alpha coefficient was achieved. For example, in Juarez et al.,[19] Cronbach's alpha coefficient for all situations in R-NVPS was 0.75. Furthermore, in a study by Chanques et al.,[18] the internal consistency of six-item NVPS for all situations was 0.76. The overall alpha for all situations of R-NVPS in Marmo and Fowler's study was 0.89.[26]

To determine the stability of NVPS, test-retest procedure was used. The results showed that the correlation coefficient between test and retest in both scales in nociceptive and nonnociceptive procedures was lower than the acceptable limits. One of the problems always raised in assessing test stability is the sameness of test and retest situations since with time the variable gets under the influence of confounding elements or different situations. This study showed that the patients got a higher agitation score during the evenings (retest) compared with the morning (test) session (due to receiving lower dosage of sedatives). It seems that the emergence of agitation behavior in these patients is not without effect in the increase of pain score. Studies show that the emergence of anxiety and agitation could lead to the increase of pain score (e.g., in item, respiration and alarm),[52627] This shows that NVPS might get under the influence of elements other than pain. No similar studies were found regarding the stability of NVPS to compare the results; therefore, further studies in assessing test stability are necessary.

To assess the inter-rater reliability, the results showed a very good correlation between the raters. Other studies have confirmed our results to some extent. For example, Chanques et al. in determining the psychometric properties of NVPS in patients under ventilation showed a good inter-rater reliability.[18] Furthermore, Kabes et al., in their study on revising and psychometric of O-NVPS, reported good (90.8) inter-rater reliability.[16] Based on the results of the current study, it could be said that the comprehension of pain signs in NVPS is similar and there is no different interpretation of them.

There are several limitations of the study that should be considered. First, given that nociceptive and nonnociceptive procedures were previously defined, being aware of that could influence the raters’ scoring measure. Of course, a good inter-rater reliability could reduce the worry in this case. Because some of the patients received sedatives and this could lead to a weaker reaction to pain and a lower pain score, and because omitting sedatives was not possible due to ethical reasons, it was counted as one of the research limitations. Third, the patients included in the study were calm from the point of view Richmond scale and as agitation has an influence on the pain score; therefore, it is recommended that, in the future studies, patients with higher restlessness score should be included as well to find out whether this scale can discriminate between pain score and restlessness. Fourth, using eye washing as a nonpainful procedure led to a higher pain score; therefore, it is suggested that in the future studies, other nonnociceptive procedures (such as taking the blood pressure) should be used.

CONCLUSION

The results of this study showed that O-NVPS and R-NVPS have acceptable psychometric characteristics for the purpose of assessing pain in patients who are hospitalized in critical care units and do not have the ability of communication. However, in using these scales, attention should be paid to a number of important issues and especial caution should be taken: First, sometimes, during the nonnociceptive procedures, touching could change the face item on both scales, which does not necessarily indicate the existence of pain. In such situations, attention should be paid to the change in other items and determining the minimum pain score. Another issue to which attention should be paid while using both of the NVPS versions is the agitation condition of the patients. Both scales have an increase of pain in the time of agitation, and this makes the process of distinguishing them from the emergence of the real pain quite difficult. Therefore, in using both of the versions in restless patients, it is necessary for the nurses to pay attention to other signs as well. Third, although vital signs are considered to be an index for assessing pain, the score of this item in different procedures, especially in cases in which pain is reported, does not move in the same direction with the other items and shows fewer changes. Finally, taking all of the situations into consideration, although all of the physiological items (respiration, vital signs, and physiology II) show lower sensitivity and reliability in comparison with the behavioral items, in comparing the items of O-NVPS and R-NVPS, respiration is better than physiology II.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.