What are the validity and reliability of the modified Yale Preoperative Anxiety Scale-Short Form in children less than 2 years old?

BACKGROUND: Accurate measurement of preoperative anxiety is important for pediatric surgical patients' care as well as for monitoring anxiety-reducing interventions. The modified Yale Preoperative Anxiety Scale-short form is well validated for this purpose in children aged 2 years and above, but not in younger children. AIMS: We aimed to validate the Dutch version of the modified Yale Preoperative Anxiety Scale-short form for measuring preoperative anxiety in children less than 2 years old.
METHODS: Two investigators independently assessed infants' anxiety at the holding area and during induction of anesthesia with the modified Yale Preoperative Anxiety Scale-short form and the COMFORT-Behavior scale-live and from video observations. Construct validity and responsiveness of both scales were tested with Pearson correlation coefficient. Internal consistency of the modified Yale Preoperative Anxiety Scale-short form was assessed using Cronbach's α, and inter-rater reliability and intra-rater reliability were tested using the intraclass correlation coefficient and Cohen's linearly weighted kappa. Hypotheses for sufficient inter-rater reliability (r > 0.60) and validity (r > 0.65) had been formulated a priori in line with the COSMIN guidelines.
RESULTS: Behavior of 129 infants (89.1% male) with a median age of 6.5 months (range 0.9-16.5 months) was observed. The correlations between the modified Yale Preoperative Anxiety Scale-short form and COMFORT-Behavioral scale were strong at the holding area and at induction of anesthesia, as were the correlation of change scores between the holding area and induction. Internal consistency of the modified Yale Preoperative Anxiety Scale-short form was excellent at both the holding area and at induction of anesthesia. Inter-rater reliability was good to excellent on scale level and moderate to good on item level.
CONCLUSION: These findings support the validity and reliability of the Dutch version of the modified Yale Preoperative Anxiety Scale-short form in children less than 2-years-old.

What is already known

The modified Yale Preoperative Anxiety Scale is appropriate to assess preoperative anxiety in children aged 2 years and above.

What this article adds

This study supports validity and reliability of this scale to assess preoperative anxiety in infants as well.

BACKGROUND

Preoperative anxiety and distress can affect children before, during and after surgery,1 and lead to negative behavioral changes even 6 months after discharge.2 Children, also young children, who are anxious during induction of anesthesia are more prone to develop postoperative negative behavioral changes, such as nightmares, separation anxiety, and aggression toward authority.3 While older children tend to be more anxious about the anesthetic and surgical processes, younger children may suffer from separation anxiety from parents4 or from preoperative fasting (as children are too young to explain).

Evidence is increasing on the impact of early‐life anxiety and distress. Early‐life stress can negatively affect the sympathetic nervous systems and hypothalamic‐pituitary‐adrenal axis (effects arising before the age of 18 months) and might alter the stress system development.5 Infants may be highly vulnerable to preoperative anxiety due to their age‐related cognitive immaturity.6 They can show suspicious behavior in relation to unfamiliar adults from 7 months of age,7 and thus reflect a subjective sense of unease. Anxiety is a subjective sense of unease, dread, or foreboding. Anxiety and pain behaviors can often not be distinguished, especially in infants, and distress is often the combination of both.8

To improve perioperative care and to monitor anxiety‐reducing interventions, the Yale Preoperative Anxiety Scale (YPAS) has been developed for children aged 2 years and above.4 This scale has been modified6 and shortened in the past years,9 and remains the ‘gold standard’ to evaluate preoperative anxiety in children. Nevertheless, many common procedures in children are performed at the infantile age or even at neonatal age, such as pyloromyotomy and pediatric inguinal hernia repair.10 Thus, the accurate measurement of preoperative anxiety in our youngest patient population is important as well.

Aim and hypotheses

The use of validated health care instruments simplifies measuring the effect of interventions and the interpretation thereof. We aimed to test validity and reliability of the modified Yale Preoperative Anxiety Scale‐ Short Form (mYPAS‐SF) for measuring preoperative anxiety in children less than 2 years old.

A priori hypothesis was formulated considering the expected relation between the mYPAS‐SF and the COMFORT‐B. We hypothesized a moderate positive correlation of at least r > 0.60 between the mYPAS‐SF and the COMFORT‐B at the holding area, and of r > 0.65 at induction of anesthesia. Furthermore, we expected a responsiveness (the correlation of the change values between the holding area and induction of anesthesia) of at least r > 0.70.

MATERIALS AND METHODS

The guidelines of the Consensus‐based Standards for the Selection of health Measurement INstruments (COSMIN) were applied in this clinimetric study (www.cosmin.nl; accessed last on November 30, 2017).11 The data were collected within the framework of a large prospective perioperative trial and the study protocol was approved by the local Medical Ethical Committee (MEC 2015‐264) at Erasmus University Medical Center, The Netherlands. The study has been performed according to the Declaration of Helsinki. Written informed consent was sought from the children's parents or legal representatives.

Participants

The study sample of the prospective perioperative trial consisted of 0‐ to 3‐year‐old infants admitted to the Erasmus MC‐ Sophia Children's Hospital in Rotterdam, the Netherlands, in the period September 2015‐October 2016. Subjects had elective surgery for inguinal hernia, undescended testicles, or hypospadias, performed under general anesthesia with caudal block. Eligible for participation were infants 0‐2 years old. Subjects for whom informed consent from parents or legal representatives was missing were excluded from the analysis.

Instruments

mYPAS‐SF

The mYPAS‐SF is an observational checklist9 with four response categories, each consisting of four to six distinct behavioral descriptions (Data S1). Four categories of behavior are assessed: activity, vocalizations, emotional expressivity, and state of apparent arousal. Partial weights are used to calculate a total score ranging from 23 (low anxiety) to 100 (high anxiety). Previous research has shown good to excellent inter‐ and intra‐observer reliability and validity.6, 12 Previously translated Dutch versions of the mYPAS‐SF were used in this study.13

COMFORT‐B scale

The COMFORT scale was originally designed to assess ventilated children's distress.14 It has been shortened since, and the resulting observational COMFORT‐B scale has shown good validity and reliability to score distress and postoperative pain in 0‐ to 3‐year‐old infants.15, 16, 17 It consists of the six items alertness, calmness, muscle tone, movement, facial tension, and crying (in spontaneous breathing children) or respiratory response (in ventilated children). Each item has five response categories, and the total score is calculated from counting the scores on individual items, ranging from 6 (calm) to 30 (distressed) (Data S2).

Procedure

Parents of candidate subjects were invited to participate at preoperative consultation. At the day of surgery, the child's baseline characteristics and vital signs were recorded at the ward. The child was then accompanied by one parent and one investigator (observer 1) during transfer to the holding area and operation room (OR). The total duration of the transfer was approximately 15 minutes. At arrival in the holding area, observer 1 assessed live behavior with the use of the mYPAS‐SF, while making 2‐minute video recordings. These recordings were afterwards assessed by observer 2 for mYPAS‐SF as well as COMFORT‐B. Video recordings were made again in the OR during 2 minutes before induction of anesthesia (from presentation of mask to induction in case of inhalation induction, or from just before infusion of anesthetic to induction in case of intravenous induction). Live behavior was assessed at the same time. For all video recordings, a computer‐generated randomized list determined the order in which the videos were assessed (holding area first, or induction of anesthesia first) as well as whether first the COMFORT‐B scale or first the mYPAS‐SF would be applied.

Training for outcome assessment

An experienced colleague trained the outcome assessor for both COMFORT‐B assessment and mYPAS‐SF assessment, first from video footage and thereafter by live observations of infants at the ward and OR. The training was completed with 10 live assessments by both the experienced colleague and the assessor simultaneously. Interobserver agreement was calculated with linear weighted Cohen's kappa; a κ ≥ 0.65 was considered sufficient to reliably perform outcome assessment. The kappa for the results of ten paired assessments for the COMFORT‐B scale was 0.77, and that for the mYPAS‐SF was 0.82, both reflecting sufficient inter‐rater reliability.

Anesthetic treatment

Induction and maintenance of anesthesia was standardized. At the ward, EMLA cream® was applied at potential sites of injection (usually both hands). After arrival in the OR, the anesthetist decided on either intravenous or inhalational induction of anesthesia. Anesthesia was induced intravenously with propofol IV (2‐4 mg/kg), or by inhalation of sevoflurane in a mixture of oxygen and air. After induction of anesthesia, a laryngeal mask was placed, and a caudal block with ropivacaine 0.2% was given. Anesthesia was maintained with sevoflurane (0.6‐1.0 MAC) in a mixture of oxygen and air.

Statistical analysis

All data but linearly weighted Cohen's kappa were analyzed with SPSS (IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp). Linear weighted Cohen's kappa was calculated at the Vassarstats website, www.vassarstats.net; assessed at October 16, 2017. Normally distributed variables are summarized using means and standard deviations; continuous variables that were not normally distributed are summarized using the median and the interquartile ranges (IQRs); and categorical variables are summarized using percentages. Comparisons were made in distress and anxiety scores between infants <1 year of age, and infants ≥1 year of age using Mann‐Whitney U tests for not normally distributed values.

Construct validity reflects the degree to which the scores of a measurement instrument are consistent to relational scores with other instruments.18, 19 Responsiveness reflects the ability of an instrument to detect change over time, and reflects the validity of change in multiple scores.18, 19 The correlation between mYPAS‐SF and COMFORT‐B scores reflected the level of construct validity. The correlation between the change scores of the two scales (difference between holding and induction assessment) represented level of responsiveness. Results were compared to the a priori formulated hypotheses (see Aim and Hypotheses).

Reliability reflects the extent to which scores of patients who have not changed, are the same for repeated measurements under several conditions.19 First, internal consistency—reflecting the degree of interrelatedness among items—of the mYPAS‐SF was calculated using Cronbach's α and the result was interpreted as follows: <0.50 unacceptable; 0.51‐0.6 acceptable; 0.61‐0.7 questionable; 0.71‐0.8 moderate; 0.81‐0.90 good; >0.91 excellent. Next, regarding the reliability of the mYPAS‐SF we calculated the inter‐rater reliability and intra‐rater reliability. The inter‐rater reliability on scale level was calculated with the intraclass correlation coefficient (ICC) using a two‐way random model, based on absolute agreement in single measures. The measure of reliability was interpreted as follows: ICC < 0.50 poor reliability; 0.50‐0.75 moderate reliability; 0.76‐0.90 good reliability; 0‐91‐1.00 excellent reliability.20

The inter‐rater reliability on item level was then tested with linear weighted Cohen's kappa over simultaneously observed video recordings. Lastly, the intra‐rater reliability for one observer was calculated from the results of the same videos assessed twice at a 2‐month interval. Strength of agreement on item level was interpreted as follows: <0.20, poor agreement; 0.21‐0.40, fair agreement; 0.41‐0.60, moderate agreement; 0.61‐0.80, good agreement; and 0.81‐1.00 very good agreement.

Cutoff scores were used to identify the anxious versus non‐anxious patient at both the holding area and at induction of anesthesia. A cutoff value of 17 on the COMFORT‐B was found in previous research.16 Receiving operating characteristic curves were used to determine cutoff values on the mYPAS‐SF, with a cutoff value of ≥17 on the COMFORT‐B scale interpreted as anxious (value 1) and values below 17 as non‐anxious (value 0). The mYPAS‐SF value with the optimal combination of sensitivity and specificity was selected as cutoff score for preoperative anxiety.

Two‐sided statistical significance was defined as P < 0.05.

RESULTS

Behavior of 129 patients was assessed (see Figure 1 flowchart and Table 1 patient characteristics). Video footage was missing for four subjects at the holding area (in two cases due to technical problems and in two cases due to lack of video registration) and for two subjects during induction of anesthesia (in one case due to technical problems and in one case due to lack of video registration)). There was a male predominance (89.1%) and the median age was 6.5 months (IQR 3.3‐9.9 months). Mean values of mYPAS‐SF scores as well as from COMFORT‐B scores at the holding area and induction of anesthesia, as well as the mean change scores are represented in Table 2. A statistically significant difference in anxiety and distress scores was found between infants <1 year of age and infants >1 year of age at the induction of anesthesia and at the change in scores between the holding area and induction of anesthesia.

Figure 1

Flowchart on validity and reliability assessment

Table 1

Patient characteristics (n = 129)

	Total
Sex n(%)
Male	115 (89.1)
Female	14 (10.9)
Age in months median(range)	6.5 (0.9‐16.5)
Type of surgery n (%)
Inguinal hernia (m/f)	59/14 (46/11)
Undescended testis	25 (19)
Hypospadias	31 (24)
Type of induction n (%)
Inhalation	110 (85)
Intravenous	19 (15)
Parental presence at induction n (%)	129 (100)

Table 2

Median values (IQR) of video‐assessed modified Yale Preoperative Anxiety Scale‐Short Form scores together with cutoff values

	n	mYPAS‐SF median (IQR)	COMFORT‐B median (IQR)	P‐valuea	Cutoff (sensitivity/specificity)
Holding area	123	23 (23‐40)	14 (14‐15)		37 (0.91/0.86)
<1 yr of age	105	23 (23‐40)	14 (14‐15)	0.657
≥1 yr of age	18	26 (23‐41)	14 (14‐14)
Induction of anesthesia	127	73 (46‐94)	18 (15‐22)		57 (0.92/0.95)
<1 yr of age	108	67 (44‐90)	17 (15‐22)	0.001
≥1 yr of age	19	90 (79‐94)	23 (19‐24)
Change	121	37 (9‐60)	4 (2‐8)
<1 yr of age	104	34 (6‐56)	4 (2‐8)	0.008
≥1 yr of age	17	56 (38‐69)	8 (4‐10)

Cutoff values indicate the non‐anxious versus anxious patient. A statistically significant difference was found in scores between infants <1 yr and ≥1 yr.

P‐value indicates the statistical difference in mYPAS‐SF scores between <1 year and ≥1 year of age.

Construct validity and responsiveness

Validity was tested over n = 123 video observations at the holding area and n = 127 video observations at induction of anesthesia. The correlations between mYPAS‐SF and COMFORT‐B were strong both at the holding area; r = 0.72 (95% confidence interval (CI) 0.62‐0.81); P < 0.001, and at induction of anesthesia; r = 0.92 (0.89‐0.94); P < 0.001. Responsiveness was tested over n = 121 video observations, a strong correlation of r = 0.82 (0.74‐0.88); P < 0.001 was found for the change scores of the mYPAS‐SF and COMFORT‐B between the holding area and at induction of anesthesia (see Figure 2).

Figure 2

The correlation of the change values of the modified Yale Preoperative Anxiety Scale‐short form and the COMFORT‐Behavioral scale with its 95% confidence interval

Reliability

Internal consistency was excellent for mYPAS‐SF (Cronbach's alpha 0.93 at the holding area and 0.93 at induction of anesthesia) and moderate to good for COMFORT‐B (Cronbach's alpha 0.79 at the holding area and 0.87 at induction of anesthesia). Inter‐rater reliability on scale levels was tested over n = 90 observations and showed moderate reliability at the holding area (ICC (95% CI) = 0.57(0.42‐0.70) and good reliability at the induction of anesthesia (ICC = 0.81(0.71‐0.87)), Reliability on item level showed moderate to good agreement on inter‐rater reliability over n = 39 videos and good to excellent agreement on intra‐rater reliability over n = 19 videos (see Table 3).

Table 3

Reliability on item level for the modified Yale Preoperative Anxiety Scale‐Short Form

Reliability	Item	κ (95% CI)
Inter‐rater	Activity	0.41 (0.20‐0.62)
	Vocalization	0.68 (0.52‐0.85)
	Emotion	0.60 (0.41‐0.79)
	Apparent arousal	0.60 (0.40‐0.80)
Intra‐rater	Activity	0.85 (0.67‐1)
	Vocalization	0.95 (0.89‐1)
	Emotion	0.88 (0.75‐1)
	Apparent arousal	0.93 (0.82‐1)

κ = linear weighted Cohen's kappa.

Cutoff scores

Separate cutoff scores were defined for results obtained in the holding area and at induction of anesthesia. A clinical cutoff score of 37 at the holding area presented with excellent sensitivity (0.91) and good specificity (0.89); a clinical cutoff score of 57 at induction of anesthesia presented with good sensitivity (0.92) and excellent specificity (0.95) (Table 2).

DISCUSSION

Our results confirm our hypotheses that the mYPAS‐SF has sufficient validity and reliability to support the use of this scale for evaluating preoperative anxiety children less than 2 years old. The original mYPAS 6 has proven its validity for over 20 years. It has been translated into other languages and tested with good results12, 21, 22, 23 in many different populations. As the mYPAS‐SF remains the mostly used scale for assessing preoperative anxiety in children aged 2 years and above, a logical step was to validate this scale in the younger population.

One could argue whether the term distress would be more appropriate to describe feelings of preoperative anxiety in infants. The concepts of psychological and behavioral distress have been defined to encompass all behaviors of negative affect and responses to aversive internal and external stimuli, associated with pain, anxiety, and fear.14 As written in the introduction, distress is often used to indicate a combination of anxiety and pain.8 As the preoperative situation is mostly not associated with pain, the term anxiety seems suitable for the use in infants as well.

Our results show good reliability at induction of anesthesia, and moderate to good reliability at the holding area. Previous validation studies have reported lower inter‐rater reliability at the holding area as well.4, 12, 23 The decreased reliability can in part be explained by the low variance in scores, as 75% of the infants had low scores on both the mYPAS‐SF and the COMFORT‐B scale at the holding area.

Several other possible reasons spring to mind. Behaviors at the holding area were sometimes difficult to assess because very young infants do not display behaviors such as talking, or were asleep (n = 13, 10.6%). As a next step to make the mYPAS‐SF more suitable for infants, selected items could be deleted and new items added to more specifically cover behavioral aspects for this age group.

The difference in anxiety levels between infants <1 and ≥1 year of age also gives room for thought. Developmental age affects how children express their anxiety. Young children are less likely to experience separation anxiety than older children, and therefore may be more easily comforted by healthcare providers.24 Even though all infants in our sample were accompanied by one parent during induction of anesthesia, still, the older infants in the sample experienced high levels of anxiety. The high percentages of anxious infants at the holding (25%) and at induction of anesthesia (65%), and the higher levels of anxiety in the older study population, indicate the need for development of anxiety‐reducing interventions in the OR.

An additional aspect contributing to high levels of distress and anxiety in infants could be mandatory preoperative fasting. This cannot be explained to very young infants and their feelings of hunger could contribute to discomfort and consequently higher scores on the mYPAS‐SF. Currently more attention is being paid to postoperative consequences of preoperative fasting and possibilities to shorten the fasting time.25

Clinical relevance

The use of validated healthcare instruments is important to accurately measure the effect of interventions. Over 200 000 inpatient operative procedures have been done in children in the United States in 2009.10 Many common procedures in children are performed at the infantile age or even at neonatal age, such as pyloromyotomy, pediatric inguinal hernia repair, and gastroschisis or omphalocele correction (together almost 20 000 procedures in 2009).10 In addition, there is a rapidly increase in the number of outpatient procedures, including those in infants. It therefore seems important to have a valid instrument to measure preoperative anxiety in regular infant patient care and to evaluate the effects of anxiety‐reducing interventions. With the validation of the mYPAS‐SF for children less than 2 years old, this is now possible.

Strengths and limitations

Strengths of the study are the large sample size and specific age range. Furthermore, we addressed construct validity and responsiveness as well as various types of reliability (internal consistency, inter‐rater reliability and intra‐rater reliability). Responsiveness had not been tested before. Video assessment was randomized to prevent structurally moderation of scores as a consequence of repeated observation. Some limitations need to be addressed. First, COMFORT‐B assessment by two observers, video and live, would have strengthened our validity results. Second, a COMFORT‐B cutoff score for pain was used to identify a cutoff score for anxiety. Although anxiety and pain show interrelation in terms of distress, they are not interchangeable and this limits the interpretation of the results. Third, the patient population was predominantly male. The low number of girls prevented valid evaluation of gender differences in assessment of anxiety. Although this does not interfere with the validity and reliability assessment of the mYPAS‐SF, the generalizability of our results to both boys and girls is limited.

CONCLUSION

The findings of this study support the validity and reliability of the mYPAS‐SF to assess levels of preoperative anxiety in children less than 2 years old. These results support the use of this scale in clinical circumstances, and for evaluating preoperative anxiety‐reducing interventions.

CONFLICT OF INTEREST

No conflicts of interest declared.

Click here for additional data file.

Click here for additional data file.