The Utilization of a Rapid Agitation Scale and Treatment Protocol for Patient and Staff Safety in an Inpatient Psychiatric Setting.

Agitation is a common and potentially dangerous condition requiring rapid recognition and treatment in acute psychiatric units. Prompt intervention can prevent a patient with agitation from harming themselves, harming others, or needing restraints or seclusion. After the review of numerous guidelines, the Modified Agitation Severity Scale (MASS) agitation treatment protocol was developed to identify and manage agitation in an inpatient adult psychiatric setting. This protocol involved modifying an existing agitation scale and pairing scores with a treatment algorithm to indicate which behavioral and medication interventions would be most appropriate. All scoring and interventions were recorded in the electronic medical record (EMR). Three months of data were collected before and after the protocol was implemented. The new, modified scale had high reliability and correlated well with another validated agitation scale. Perceived patient safety was high during both study phases. Nurses' perceptions of safety trended upward after the protocol was implemented, though these differences were not significant, likely due to insufficient power. Although there was no decrease in seclusion events after implementation of the treatment protocol, there was a 44% decrease in restraint events and average restraint minutes per incident. Despite a potential increase in workload for nursing staff, implementation of the protocol did not increase burnout scores. Physicians continued to order the protocol for 55% of patients after the study period ended. These findings suggest that including a rapid agitation assessment and protocol within the EMR potentially improves nurses' perceptions of unit safety, helps assess treatment response, reduces time patients spend restrained, and supports decision making for nurses.

Introduction

Patients with acute agitation are common in the inpatient psychiatric setting [1-4]. Agitation, which is characterized by excessive or inappropriate motor or verbal activity, signals the beginning of a behavioral emergency and is associated with several psychiatric conditions [3]. When unmanaged, agitation can progress quickly from mild symptoms, such as restlessness and anxiety, to severe symptoms, such as assault of staff or other patients. This progression not only decreases feelings of safety, but also increases staff burnout [5]. Early recognition, appropriate assessment, and effective treatment of agitation can reduce the risk for escalation to violence and injuries [6, 7].

Higher rates of patient agitation affect healthcare in terms of longer lengths of stay, more readmissions, higher medication use, higher workers’ compensation costs, staff time off from work due to injuries, and increased administration costs [6, 7]. In most circumstances, non-pharmacological methods such as verbal de-escalation are helpful to decrease agitation [1, 8–10]. However, when initial measures fail, medication has been shown to effectively calm agitated behaviors and treat the underlying causes of agitation, such as anxiety, mania, and psychosis [1, 9, 11–13]. In cases where agitation leads to violent behavior, more coercive responses, including involuntary medication administration, seclusion, and restraints, could be necessary [13]. However, these methods should generally be avoided if possible, given evidence they can lead to physical injury and psychological trauma [14]. For this reason, efforts to improve early identification and reduction of agitation in hospitals have become an integral component of quality assessment surveys [15, 16].

Multiple guidelines have been written to direct the assessment and treatment of agitation in patients with acute psychiatric disorders [1, 4, 13, 17, 18]. These guidelines generally recommend that the physician evaluate the patient and attempt to determine the etiology of agitation, that symptoms be assessed with the use of a standardized rating scale, and that treatment for agitation be specific to the underlying cause [13, 19, 20]. If there is no routine procedure for assessment of agitation, there could be delays in identification and treatment, leading to escalation of more coercive management responses or resulting in staff and patient injuries [13]. Having a standard agitation treatment protocol could prevent these delays and provide more direction about when certain interventions are appropriate.

While several evidence-based assessment scales for agitation exist, many do not have strong enough clinical utility to guide rapid treatment decisions. A recent review summarized rating scales of psychomotor agitation for use in various treatment settings [21]. In this review, the scales that best fit inpatient psychiatric settings and had strong psychometric properties took too long to administer to be used practically. The shorter scales with strong psychometric properties were either not intended for or not appropriate for in-the-moment assessment of an inpatient psychiatric population, especially because they generally did not describe specific problematic behaviors [21].

In the present study, we modified an existing descriptive agitation scale, the Agitation Severity Scale [22], for rapid clinical assessment. Though the original scale assesses for various, specific agitated behaviors and demonstrates strong content validity and strong interrater reliability, administration of the scale takes an average of 10 min. After modification (described in the methods section, below), the scale could be administered in one minute. This modified scale was then linked to a treatment protocol to guide nursing staff decisions regarding both behavioral interventions and medication distribution. The MASS agitation treatment protocol was implemented in an adult inpatient psychiatric setting for three months. Study results were compared with three months of baseline data before implementation. Outcomes measured included patient perceptions of safety, nurse perceptions of safety, nurse burnout, and scale convergence validity with another commonly used agitation measure. Additional information regarding seclusion and restraint rates was also collected for 18 months before and after protocol implementation.

Methods

This study was conducted at a psychiatric hospital in the midwestern United States on a high-acuity, adult psychiatric unit following approval by the hospital system’s Institutional Review Board. A workgroup of experienced professionals, including psychiatric physicians and nurses, pharmacists, research specialists, and information technology professionals, convened to design and carry out this study. There were five phases of study design: selection and modification of an agitation measurement scale, development of a treatment protocol including behavioral and medication interventions, development of nursing and patient safety surveys, training of physician and nursing staff, and data collection upon implementation into the electronic medical record (EMR).

Development of a Modified Version of the Agitation Severity Scale (MASS)

With permission from the author of the Agitation Severity Scale, a team of psychiatrists collaborated with a PhD-level biostatistician and an experimental psychologist to revise the scale. The original version of the Agitation Severity Scale [22] measured five domains of agitation: nonverbal facial expressions, verbal behaviors, purposeful motor behaviors, non-purposeful motor behaviors, and interpersonal behaviors. For each of these five domains, 4 to 6 specific behaviors were listed, for a total of 25 items. These behaviors were ranked on Likert scales with scores ranging from 0 to 75 (see supplementary materials for original Agitation Severity Scale).

In creating the Modified Agitation Severity Scale (MASS) (Table 1), the following changes were made: 1) items were organized by severity level rather than domain; 2) each item was ranked as either present or absent, rather than assessed by Likert scale; and 3) scores on items were weighted. These modifications decreased the time for completion from ten minutes to about one minute. Weighting scale items was the most significant modification. On the original scale, a behavior like pacing held the same weight as did a behavior like screaming. In the modified scale, psychiatrists on the research team relied on clinical experience to categorize behaviors into severity weights (very mild behaviors = 1, mild behaviors = 2, moderate behaviors = 3, and severe behaviors = 10) for a total score ranging from 0–83.

Table 1

Modified Agitation Severity Scale

Weighted Value	Agitation Symptom	Point Value if Symptom Present
1	Red in the Face	1
	Grinding Teeth	1
	Tapping, Clenching, Involuntary Movements of Hands	1
	Disorganized Thinking	1
	Emotionally Labile, Anxious, Nervous	1
	Unable to Reason	1
2	Spitting	2
	Getting Attention with Voice	2
	Darting Eyes	2
	Demanding	2
	Speaking more Quickly than Baseline	2
	Angry Tone of Voice	2
	Persistent Disruptive Verbalizations	2
	Restless	2
3	Yelling, Audibly Louder than Baseline	3
	Unable to be Calmed	3
	Inappropriate Behavior	3
	“In Your Face”	3
	Decreased Self-Control, Impulsiveness	3
	“Puffed up”, Chest Out, Threatening Posture	3
	Confrontational	3
10	Biting	10
	Throwing Objects	10
	Violating Self or Others	10
	Physical Violence to Self or Others	10
	TOTAL SCORE (Maximum score 83)
Agitation Severity: 1 to 4: Very mild agitation                                 4 to 6: Mild agitation                                 7 to 9: Moderate agitation                                 Greater than 9: Severe agitation and/or violent

Total score is a sum of each present symptom multiplied by the weighted value for the section

Development of the MASS Agitation Treatment Protocol

Multiple guidelines for the behavioral and pharmacological treatment of acute agitation were thoroughly reviewed by the research team and used to develop the study’s agitation treatment protocol (Table 2) [1, 11–13, 20, 21]. A list of recommended non-pharmacologic interventions was formulated based on these guidelines and with the help of nursing staff. Medication use was encouraged if non-pharmacologic nursing measures were not successful, with the purpose of calming the patient without causing over sedation. Based on the most commonly recommended medications in recent agitation guidelines, the MASS agitation treatment protocol incorporated four different pharmacologic tracks: a benzodiazepine (lorazepam), a first-generation antipsychotic (haloperidol), and a second-generation antipsychotic (olanzapine; with a low-dose option and a standard dose option). The patient’s physician determined which medication track was ordered upon admission, based on the most likely etiology of the patient’s agitation. An oral medication was preferred over the intramuscular route if the patient was willing to accept it. Intramuscular medication was only administered in the event the patient was a danger to themselves or others and was preferred to be utilized before seclusion or restraint, which were only utilized as a last resort and required notification of the physician.

Table 2

MASS Agitation Treatment Protocol

Assess and record MASS now, then at 0900 and 2100 while awake  • Calculate MASS score based on behavior observed within the most recent 1 h  • If medication given, repeat MASS hourly until score is less than 4, then resume at 0900 and 2100 scoring schedule
Agitation Score	Severity	Possible Treatment Interventions
1–3	Very Mild	Behavioral Interventions
4–6	Mild	Behavioral Interventions	Oral Medication
7–9	Moderate	Behavioral Interventions	Oral Medication		Intramuscular Injection
> 9	Severe and/or Violent	Behavioral Interventions	Oral Medication		Intramuscular Injection	Seclusion or Restraint
Behavioral Interventions
Speak with patient about frustration				Encourage patient to channel feelings into activity
Identify wants and feelings				Bring in alternative staff person
Quiet room				Staff directed time out
Encourage use of coping skills patient identifies as helpful				Redirect attention and offer choices
Encourage patient to engage in relaxation techniques				Offer fluids
Encourage self-time out				Offer food
Remove provoking stimuli				Offer nicotine replacement
Medication Interventions
Choose one of the following:
Ο Lorazepam
• 1 mg Oral six times daily prn for MASS score of 4 to 9; (Max 8 mg in 24 h)
• 1 mg Intramuscular six times daily prn for MASS score of 7 to 9 and patient unable/unwilling to accept oral medications (Max 8 mg in 24 h)
• 2 mg Oral four times daily prn for MASS score of 10 or above (Maximum 8 mg in 24 h)
• 2 mg Intramuscular four times daily prn if MASS score 10 or above and patient unable/unwilling to accept oral medications: (Max 8 mg in 24 h)
Ο Haloperidol
• 2 mg Oral six times daily prn for MASS score of 4 to 9; (Max 20 mg in 24 h)
• 2 mg Intramuscular six times daily prn for MASS score of 7 to 9 and patient unable/unwilling to accept oral medications (20 mg in 24 h)
• 5 mg Oral four times daily prn if MASS score 10 or above; (Max 20 mg in 24 h)
• 5 mg Intramuscular four times daily prn if MASS score 10 or above and patient unable/unwilling to accept oral medications (Max 20 mg in 24 h)
Ο Olanzapine (Low Dose): Lower doses should be considered for the elderly, patients with low body weight, dehydration, and no previous exposure to antipsychotic medication
• 2.5 mg Oral three times daily prn for MASS score of 4 to 9; (Max 20 mg in 24 h)
• 2.5 mg Intramuscular three times daily prn for MASS score of 7 to 9 and patient unable/unwilling to accept oral medications (Max 20 mg in 24 h)
• 5 mg Oral three times daily prn for MASS score of 10 or above (Max 20 mg in 24 h)
• 5 mg Intramuscular three times daily prn if MASS score 10 or above and patient unable/unwilling to accept oral medications (Max 20 mg in 24 h)
Ο Olanzapine (Standard Dose)
• 5 mg Oral three times daily prn for MASS score of 4 to 9; (Max 30 mg in 24 h)
• 5 mg Intramuscular three times daily prn for MASS score of 7 to 9 and patient unable/unwilling to accept oral medications (Max 30 mg in 24 h)
• 10 mg Oral three times daily prn for MASS score of 10 or above (Maximum 30 mg in 24 h)
• 10 mg Intramuscular three times daily prn if MASS score 10 or above and patient unable/unwilling to accept oral medications (Max 30 mg in 24 h)
Notify Physician: Nurse to contact on-call physician if
• Maximum daily dose met for agitation medication,
• 3 or more doses of MASS protocol medication given in less than 4 h, or
• Concerns for acute muscle stiffness

The MASS was scored by the nurses at the time of admission and then twice daily as long as the protocol was ordered. The MASS scores were entered directly into a flowsheet within the EMR, with the score ranges serving as a decision tree for selecting various management strategies from the associated agitation treatment protocol (Table 2). For scores of 0 to 3 (very mild), behavioral interventions such as speaking to the patient about their frustrations, taking the patient to a quiet room, or offering nicotine replacement therapy (for patients with a history of nicotine dependence) were recommended. For scores of 4 to 6 (mild), oral medication could be offered in addition to the previous recommendations. For scores 7 to 9 (moderate), the nurse could utilize any of the previous recommendations, as well as intramuscular injection of medication if the patient was acutely dangerous. For scores 9 and above, seclusion or restraint could be administered if needed, with physician notification.

Nursing staff recorded the treatment interventions in the EMR. Data related to any involuntary treatments, such as medication administration, seclusion, and restraint, were carefully documented and administered according to federal and state requirements. Nurses were instructed to contact the on-call physician for acute muscle stiffness, three or more MASS agitation treatment protocol medication doses administered in less than four hours, or if the maximum daily dose was met for agitation medication.

Implementing the Modified Agitation Severity Scale (MASS) Protocol

Study data were collected in two blocks consisting of a baseline phase and an intervention phase, both lasting three months (Fig. 1). At the beginning of the baseline phase, nursing staff were asked to voluntarily and anonymously complete a nursing safety survey and burnout inventory. Next, three months of baseline data, including voluntary patient safety surveys at discharge, were collected. After the baseline phase was complete, the new MASS agitation scale and protocol were implemented as a physician order in the EMR. Per the protocol, the nursing staff would routinely record agitation scores and medication administered until patient discharge. As in the baseline phase, voluntary patient safety surveys were administered at discharge.

Fig. 1

Study Design

After the intervention phase, nurses again completed a safety survey and burnout inventory. Additional outcome measures collected during and after the study period included total number of seclusion and restraint events. Demographic variables, urine drug screen results, length of stay, and discharge psychiatric diagnoses were also collected through the EMR for patients admitted during the study period.

Patient and Nurse Surveys

The patient and nursing safety surveys were developed by the research team and are available in the supplementary materials. All patients were asked if they were willing to voluntarily complete a patient safety survey on the day of discharge during both the baseline and the intervention periods. This survey had eight items that were rated from 1 to 5, with lower scores indicating positive perceptions of safety. Example items included “The unit I stayed on was safe” and “Staff noticed my frustration as it was occurring or soon after.”

The nursing survey given at baseline had six items that were rated from 1 to 5, with lower scores also indicating positive perceptions of safety. Example items included “I feel, as a staff member, safe on the unit” and “I feel agitated patients get appropriately medicated.” The post-intervention phase nursing survey had the original items plus four additional items: “A uniform agitation scale (MASS) is helpful,” “I understand our agitation scale,” “I use the MASS to assess agitation,” and “I communicate MASS scores to the team.”

The Oldenburg Burnout Inventory was used to assess nurse burnout at baseline and post-intervention [23]. This scale has two dimensions, exhaustion and disengagement, as well as a total score. Nurses were not followed over time, but rather were compared independently due to the relative frequency of staff turnover.

Convergence Validity Measure

This study also correlated the MASS scale with another evidence-based agitation assessment, the Clinical Global Impressions-Aggression Scale (CGI-A), which has been used in emergency psychiatric settings [21]. The CGI-A consists of one item, “level of agitation,” and is rated as none, slight, moderate, severe, and aggressive. Although a simple measure, it does not describe individual symptoms or characterize agitation well for communication or historical review. However, its brevity and clinical use made it a suitable scale to use as a comparison measuring convergent validity in the revised MASS scale.

Statistical Analyses

Independent sample t-tests were used to compare nurse and patient survey responses at baseline and following intervention, with one test comparing the total score and additional tests comparing each item. The Oldenburg Burnout Inventory (total score and each dimension separately) was compared at baseline and following the intervention, also using an independent samples t-test. Correlations between the CGI-A and the MASS were used to determine how strong the relationship was between these two scales. These correlations included the initial MASS score versus the initial CGI-A score and the average MASS score versus the average CGI-A score throughout a patient's stay. In addition, the CGI-A was correlated with each individual MASS score given through the duration of each patient’s stay (reported in the supplemental materials of this paper). The MASS scale was correlated with age, gender, and length of stay. In addition, MASS scores were correlated with patient diagnosis and urine drug screen results to measure potential sub-population patterns.

Results

Demographics

This study included EMR data from 742 patients, with 353 in the baseline phase and 389 in the intervention phase (Table 3). Of the patients in the intervention phase, 243 received the MASS agitation treatment protocol, leaving 110 patients for whom the protocol was not ordered. Some patients had a very short length of stay and were discharged before the scale could be utilized. For others, there may have been clinical reasons that the attending physician did not feel the protocol was appropriate, such as for patients who were in active alcohol withdrawal and were being treated with other protocols with similar medications. Other patients were likely missed due to physician oversight.

Table 3

Demographics

Age
Mean = 35.76	SD = 12.43
Gender
Male	50.1% (n = 372)
Female	49.9% (n = 370)
Race
Asian	0.7% (n = 5)
Black or African American	11.1% (n = 82)
Indigenous	8.6% (n = 64)
White	74.2% (n = 551)
Other	3.9% (n = 29)
Unknown	1.5% (n = 11)
Ethnicity
Hispanic	3.8% (n = 28)
Non-Hispanic	96.2% (n = 714)

There were no significant differences in any demographic variables in the baseline and intervention periods of the study. The average length of stay was 6.51 days with a standard deviation of 5.42 days. There were no significant differences in length of stay in the pre- and post-periods of the study.

Nurse Survey

The Nurse Survey contained six individual items that were assessed at baseline and post-intervention, with an additional four individual items added to the post-intervention survey. Twenty-four nursing staff completed the baseline survey and the Oldenburg Burnout Scale. Twenty-two nursing staff completed the post-intervention surveys. Figure 2 shows the percent of nurses who “agreed” or “strongly agreed” with each item. There were no significant differences between study phases on any of the nurse survey questions; however, a higher percentage of nurses “agreed” or “strongly agreed” on each question after the intervention period. Results from the second nurse survey showed that 73% of nurses reported they either “agreed” or “strongly agreed” that “A uniform (MASS) agitation scale is helpful.” There were no significant differences on the Oldenburg Burnout Scale at baseline and following the intervention (baseline mean = 36.17 (SD = 6.74), post intervention mean = 36.11 (SD = 8.29), t(35) = 0.03, p = 0.980).

Fig. 2

Nursing Survey Results

Patient Survey

On the Patient Safety Survey, independent samples t-tests showed no significant pre- and post- differences on the overall scale or any of the items (overall scale mean baseline 12.20 (SD = 6.38), mean post-intervention 13.25 (SD = 8.43), t(402) = -1.29, p = 0.199). Most patients “agreed” or “strongly agreed” with each safety question (82.9% to 92.7% for each question). The scale reliability as measured by internal consistency was high, with a Cronbach’s alpha of 0.96 for the overall scale, 0.94 for the pre-period, and 0.97 for the post-period. 75.4% of patients (266 of 353 patients) completed the survey in the baseline period, while only 36.8% of patients (143 of 389 patients) completed the survey in the intervention period.

MASS (Modified Agitation Severity Scale)

The MASS has a score range from 0–83. Figure 3 shows the percent of patients who scored in each severity category initially on the date of admission. The average initial MASS score was 3.78 with a standard deviation of 7.49, indicating overdispersion in the data due to a high amount of variability. The MASS did not significantly correlate with age, r(241) = 0.02, p = 0.765. An independent samples t-test showed no gender differences on MASS score, mean female 4.26 (SD = 9.00), mean male 3.23 (SD = 5.28) (no other genders were in the EMR), t(241) = -1.07, p = 0.287. The MASS did not significantly correlate with length of stay, r(241) = 0.03, p = 0.598.

Fig. 3

Initial MASS Scores

MASS Correlation with the CGI-A Agitation Scale

Average initial MASS scores were collected from 243 patients. Average initial CGI-A scores were collected from 186 of those patients. Fewer CGI-A scores were collected than MASS scores due to a technical problem with making the CGI-A a required item in the associated flowsheet of the EMR, though nursing staff were strongly encouraged to complete both scales. The average initial MASS score had a strong correlation with the CGI-A score, r(184) = 0.75, p < 0.001. The maximum number of times the MASS was scored for a patient was 65. The minimum correlation between MASS and CGI-A was 0.620, and the maximum was 1.00. All correlations between the MASS score and the CGI-A score except for 4 were significant (see supplementary materials).

MASS Correlations with Psychiatric Diagnoses and Substance Use

Ten psychiatric diagnoses were assessed for correlation with the MASS (see supplementary materials). Anxiety was negatively correlated with MASS scores, r(65) = -1.13, p = 0.044, indicating a small to medium negative correlation. The remaining diagnoses that did not correlate with MASS scores included major depressive disorder, bipolar disorder, psychosis, post-traumatic stress disorder, traumatic brain injury, personality disorder, autism spectrum or neurodevelopmental disorder, substance use disorder, and substance withdrawal.

Seclusions and Restraints

Differences in seclusion and restraint events (both total incidents and minutes) were compared in the 18 months before and after implementation of the MASS agitation treatment protocol (see Fig. 4). Outliers were excluded if they were three standard deviations away from the mean. These outliers included the following: a 145-min restraint and a 239-min restraint before the MASS was implemented, a 143-min restraint after the MASS was implemented, and a 580-min seclusion before the MASS was implemented.

Fig. 4

Average Restraint Minutes per incident

With regard to restraints, there was a 44.1% nonsignificant decrease in total incidents, with 68 incidents before the protocol was implemented and 38 incidents after implementation. There was also a 44.4% significant decrease (t(104) = 2.00, p = 0.047, effect size d = 0.41 [medium]) in average restraint minutes per incident in the 18 months after the MASS was implemented. Before the protocol, restraint incidents lasted an average of 18 min with a standard deviation of 22. After the protocol was implemented, the incidents decreased to an average of 10 min with a standard deviation of 15 min. Figure 4 shows the decreased variability in restraint minutes following MASS implementation.

There were no significant changes in seclusion incidents or minutes after MASS implementation. There were 22 seclusion incidents in the 18 months before protocol implementation and 28 seclusion incidents in the 18 months after protocol implementation. The average minutes for seclusion events in the 18 months before MASS implementation was 132 with a standard deviation of 141 min. After MASS implementation, the average minutes for seclusion events was 137 min with a standard deviation of 97 min.

Discussion

In the inpatient psychiatric setting, most patient behavior throughout the stay will be observed and documented by the nursing staff. This information is then typically reported to the physician and other treatment team members, through either chart notations or interdisciplinary team meetings. However, it is common for these communications to include only vague descriptions of behaviors and even for information to be lost during shift changes, making this typical standard of care less helpful for treatment decision making. A prominent benefit of this protocol was the ability for all team members to see patient MASS scores in the EMR flowsheet, along with the behavioral and medication interventions utilized. These scores and interventions could then be tracked over the course of the patient stay to guide treatment and measure effectiveness. It is also common in the inpatient psychiatric setting for the physician to place standing orders for medication to be given for agitation as needed. However, there is often no further direction given to the nursing staff about when medication administration is appropriate. This protocol offers nursing staff guidance about when these specific interventions should be considered. It is also possible for patients to become unexpectedly agitated on the unit after hours when the physician has left the hospital. In these cases, the nurses may have to contact the physician and wait on additional orders to be placed, causing delays in treatment, which can lead to dangerous consequences. Having a pre-ordered protocol for agitation treatment could prevent these delays.

After modifying the Agitation Severity Scale, the research team sought to determine whether the new, more time efficient version would correlate with another well-validated agitation scale. Results showed that the MASS had a very large correlation, high reliability, and strong convergent validity with the CGI-A scale, which is commonly used in the emergency psychiatric setting. This finding indicates that the MASS score is in agreement with the one-item CGI-A score regarding the severity of agitation being measured. Therefore, the MASS is able to capture level of agitation very similarly to the CGI-A, while providing the detail needed for treatment decisions. Rates of coercive treatments, such as restraint and seclusion events, were also carefully recorded to determine the possible efficacy of the new protocol. Although there was no change in seclusion events, restraint frequency and minutes sizably decreased in the 18 months following MASS agitation protocol implementation. The effect could be due to routine assessments leading to earlier recognition of agitation and needed intervention so that more coercive measures were needed less often. This finding could also reflect that nursing staff had easier access and more direction about when medication intervention was appropriate. This decrease in restraint episodes likely decreased patient suffering and increased feelings of safety for all involved. Restraint minutes were also much less variable after implementation of the MASS agitation treatment protocol. This reduced variability may suggest that patients were being treated more systematically, and that this systematic treatment may have resulted in shorter and less variable restraint time. It is unclear why there was no significant change in seclusion events. However, it could be related to the relatively low frequency of these events at baseline compared with restraint events. Therefore, a larger sample size might be needed to find any differences.

Data were also collected to determine whether MASS scores were correlated with several different psychiatric diagnoses. Results showed that MASS scores were negatively correlated with anxiety, but not with any other psychiatric disorder. This finding could reflect a tendency for patients with high anxiety to be more withdrawn in this setting and less likely to have aggressive behaviors. However, it was surprising that there were no correlations with other disorders, given the large literature base that indicates that mental health disorders, especially acute mania and psychosis, are associated with higher degrees of violence [24, 25]. This lack of correlation may be related to the highly variable nature of the data, with a standard deviation so variable that it was over dispersed. With this level of variability, a larger sample size might be needed to detect potential differences.

This study also specifically assessed nursing staff feelings of safety, as they spend the most face-to-face time with patients on the unit. Caring for patients displaying agitated behaviors can be dangerous, stressful, and demanding, and prevents nurses from attending to the other numerous duties that are expected of them. Although there were no significant differences found on the nursing staff safety survey, perceptions of safety after the protocol was initiated trended in the positive direction on all questions (Fig. 2). In the post-intervention survey, nursing staff reported overwhelmingly that they thought the protocol was helpful. This is likely because the protocol provided more specific directions about how to manage patients with agitation, whereas there was no specific guidance in the baseline phase. Having standing orders for medication also likely improved feelings of safety for nurses. Having to contact a physician after hours for medication orders can be onerous for nursing staff and can delay the ability to provide treatment to a patient with acute agitation, leading to multiple safety concerns.

The burden associated with caring for patients displaying agitated behaviors is well known and includes nursing distress and burnout [5]. Agitated patient behaviors have been linked with increased absenteeism, increased staffing turnover, and negative reactions towards patients [5]. The present study’s findings showed no significant differences in nursing perception of burnout as measured on the Oldenburg Burnout Scale before and after implementation of the protocol. This result is of particular interest given that the nursing staff were asked to perform the additional daily task of scoring the MASS multiple times a day for each patient, effectively increasing their workload. However, this extra work might have been offset by the nurses having to do fewer emergent tasks, such as contacting a physician for medication orders after the patient had already become agitated, with potentially dangerous delays in treatment.

Patients’ perceptions of safety were not significantly different before or during the study intervention period. All patients were asked to complete this questionnaire voluntarily at discharge, but many patients chose not to respond, leaving many potential responses unrecorded, especially in the intervention period. This discrepancy is likely also the result of the stress of the beginning of the COVID-19 pandemic during the last few weeks of the intervention period. The nursing staff may not have offered the survey to every patient on discharge due to an increase in workload related to changing pandemic-related requirements. On average, patients who did complete the survey answered that they either “strongly agreed” or “agreed” on each of the seven items of the survey, indicating that even before the protocol was implemented, they overwhelmingly felt safe. This suggests a potential ceiling effect on patients’ perceptions of safety, although the voluntary nature of the survey presents a risk of selection bias.

Physicians’ perceptions of the utility of the scale and protocol were not directly measured. Many of the physicians who work on the unit were involved in the development of the study, and any survey of physicians would likely introduce additional bias. However, post-study analysis of the EMR showed that in the 20 months after the study period ended, physicians continued to order the MASS agitation treatment protocol for 55% of admitted patients. It is likely that the protocol is being ordered specifically for patients who the physician thinks may become agitated during their hospital stay. However, further research is needed to determine in what cases and for what reasons physicians decide to order the protocol.

Limitations and Future Directions

An important limitation of this study relates to comparing the amount of medicine given for agitation in the baseline period and the intervention period. In the baseline period, physicians did not always use the term “agitation” or similar terms (i.e., aggression, violence) when ordering medicine for these purposes. Instead, they sometimes used terms such as “anxiety” or “mania.” Therefore, through EMR review it was not possible to differentiate when as-needed medication was given for agitation-related reasons in the baseline period. We did record as-needed protocol medications given per day during the intervention period and could use these data in potential follow-up studies of this scale and protocol.

Another limitation was the underpowered nature of the nurse survey. Even though the responses to each question trended in the positive direction, no statistically significant differences were demonstrated. This survey was especially underpowered because individual nurse responses were not tracked over time; instead, independent samples were used.

A final limitation is that data collection was stopped immediately before the COVID-19 pandemic began in March of 2020. This means that the post-intervention surveys were given during the start of the COVID pandemic in early 2020. It is possible that perceptions of safety on the unit were negatively affected by the threats of the virus, rather than patient agitation. Additional stressors during this time included the institution of COVID screening and testing, mask availability, new isolation protocols, general fear of contracting the virus, concerns about loss of income due to need to quarantine, and patient and staff refusal to wear masks. Interestingly, the only study measure collected during the COVID-19 pandemic was the nurse survey, and responses on every question in the nurse safety survey directionally improved. However, staff turnover on this unit was relatively high during this period. This turnover could be an indicator of burnout that was not captured in our survey.

Future studies of this protocol could include the evaluation of the response of underlying psychiatric symptoms, such as the speed of resolution of mania, psychosis, or anxiety. Items in the scale that were used less often could be removed in future iterations, streamlining this measure. In addition, future versions of this study could have an experimental design allowing for accurate comparisons of the amount of medication given with the protocol. It could also be useful to pilot the protocol with different and more varied populations, including geriatric populations and emergency department populations, to examine its utility in multiple medical settings.

After approximately two years of utilizing the MASS agitation treatment protocol, the state health care authority asserted that intramuscular (IM) medication should be removed from the protocol after a routine audit. A further discussion and revised protocol are presented in the supplementary materials.

Conclusions

The MASS agitation scale and corresponding treatment protocol were successfully implemented over a three-month period in an adult inpatient psychiatric population. The scale had high internal consistency and correlated well with another validated agitation scale. This rapid assessment measure resulted in nurses regularly and systematically measuring agitation symptoms in patients for the purpose of initiating behavioral and medication treatments as early as possible, with the goal of avoiding more severe symptoms and coercive treatments. The protocol provided clear instructions to nursing staff about what treatments could be administered and when they should be administered in the absence of direct physician oversight, potentially preventing delays in treatment. Nurses did not feel more burned out when they were asked to do the additional task of using the scale, and the post intervention nurse survey indicated that the nurses found the protocol to be helpful. Following the study period, physicians continued to order the protocol for a majority of patients, indicating they found it useful for treatment of patients at risk for agitated behavior. Furthermore, there has been a clear trend of decreasing frequency and length of restraint events since the protocol was implemented, thereby increasing unit safety for patients and staff. These findings suggest that including a rapid agitation assessment such as the MASS and a corresponding treatment protocol within the EMR allows for agitation to be quickly detected and addressed, reduces restraint events, increases nurse feelings of safety, and simplifies individual decision making for nurses with regard to managing agitated patients.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 40 KB)