Interobserver agreement between emergency clinicians and nurses for Clinical Opiate Withdrawal Scale.

OBJECTIVES: The Clinical Opiate Withdrawal Scale (COWS) is a validated, commonly used tool to objectively quantify withdrawal symptoms, often in anticipation of treatment with buprenorphine. Our primary aim was to determine the agreement between emergency department (ED) nurses compared with emergency physicians in determining this score in ED patients who presented for opioid withdrawal treatment. Secondarily, we wanted to investigate the safety of buprenorphine induction in the ED setting.
METHODS: Scoring for opioid withdrawal using the COWS was performed by ED clinicians and ED nurses independently on 120 patients. In addition to overall concordance, agreement (weighted kappa) was calculated between the 2 scores by various cutoffs: overall severity, COWS ≥ 5, and the 11 different individual measures. Patient documents also were reviewed for complications that could be possibly linked to buprenorphine induction.
RESULTS: Our study sample of 120 subjects was 77% Hispanic and 78.3% male. The clinicians assigned a median interquartile range overall COWS score of 6 (2-12), which categorizes as mild withdrawal. Seventy-eight (65%) subjects met the criteria of withdrawal (≥ 5 COWS) and 69 (58%) received an induction dose of buprenorphine (range 2 mg-24 mg) during the ED visit. No adverse effects or worsening withdrawal were reported. The overall observed concordance, based on severity withdrawal categorization, for all clinician pairs, was 67.5% (81/120) (95% confidence interval [CI], 58.7-75.2%). The weighted kappa for that concordance was 0.55 (95% CI, 0.43-0.67), giving a moderate strength of agreement. When data are dichotomized by COWS score ≥5, concordance was 82.5% (99/120) (95% CI, 74.7%-88.3%) and the weighted kappa was 0.65 (95% CI, 0.51-0.78), indicating substantial agreement. The breakdown by the 11 factors that constitute COWS showed only substantial agreement for pulse measurement.
CONCLUSION: The agreement between ED clinicians and nurses for the overall COWS scoring in patients presenting for opioid withdrawal treatment was substantial. COWS scoring by ED nurses may help expedite treatment with buprenorphine on presentation.

INTRODUCTION

Opioid use disorder (OUD) is a significant public health crisis in the United States with drug overdose deaths increasing in 2019 after a slight decrease in 2018. , The current increase in mortality is primarily driven by illicitly manufactured fentanyl. Randomized clinical trials have demonstrated the effectiveness of medication for addiction treatment (MAT). , , , There is increasing evidence demonstrating the overall benefit of OUD opioid agonist treatment for OUD patients. Current research shows a reduction in mortality, especially in overdose deaths in patients given OUD opioid agonist therapy with methadone and buprenorphine; , , , decreased illicit opioid use, and involvement with the criminal justice system; , , reduction in transmission of infectious diseases such as HIV and hepatitis C; , and increased retention in OUD treatment , , as well as reduced health care costs. , ,

Background

The emergency department is often a 24/7/365 focal point for patients suffering from complications of OUD. It is common for ED clinicians to encounter the same patient multiple times over any given time period. Therefore, the ED is a logical and appropriate place to start MAT.

Major regulatory groups, including the Joint Commission and Substance Abuse and Mental Health Services Administration, have recommended improvements in the diagnosis and treatment of OUD. An accurate and rapid assessment of opioid withdrawal is important in the clinical management of opioid‐dependent patients. There have been multiple tools used in the past to accurately identify and treat opiate withdrawal patients; however, these tools have not been well‐validated. The Clinical Opioid Withdrawal Scale (COWS) is a commonly used 11‐item validated tool to identify patients with possible opioid withdrawal. , It provides a way for emergency clinicians to objectively assess opioid withdrawal.

Importance

With the expanded use of buprenorphine and the potential for precipitating withdrawal, clinicians have developed algorithms for the initiation of buprenorphine in the ED. These algorithms include a minimum COWS score to appropriately initiate treatment in the ED or clinic while avoiding unnecessarily precipitating opioid withdrawal. ED nurses are usually the first line in engaging patients as they make initial contact with them. Based on the expeditious evaluation by ED nursing staff, there is potential to quickly score patients in triage and start treatment if needed, thereby avoiding escalation of symptoms or, even worse, elopement.

Goals of this investigation

Currently, there is no study that has compared nursing to physician COWS scoring of patients with a potential opioid withdrawal syndrome. We analyzed our first‐year data in our ED buprenorphine program to see how much concordance exists between the nurses and ED clinicians using COWS.

METHODS

Study design and setting

The study is a chart review involving a combination of both prospective and retrospective elements. Because this research study involved minimal risk and involved materials (data, documents, and records) that have been or were to be collected solely for non‐research purposes, it qualified for both exempt criteria and expedited review of Title 45, Part 46 of the Code of Federal Regulations (CFR), specifically 45 CFR 46.101(b)(4) and 45 CFR 46.110 category 5. The research study was approved by meeting a waiver of consent by the Sierra Foundation Human Research Protections Program Institutional Review Board as well as the institution's Human Study Ethics Board.

The study took place between July 1, 2019, and July 31, 2020 in a single‐center community ED with an annual volume of 55,000 patients and located along the Southern California and Mexican international border region. This was a sample of all adults who were identified as having a COWS score assessed independently by both the patient's nurse and ED clinician. The study period extended to a 13‐month period rather than a traditional convenient 12 months to allow more patients for statistical validity.

ED clinicians as described in our study consist of ED physicians, physician assistants (PAs), or nurse practitioners (NPs). ED physicians assumed a supervisory role for PAs and NPs. Our program uses only registered nurses (RN) to provide direct care to patients and not licensed practical nurses (LPN). Both ED nurses and ED clinicians had similar training sessions on opiate withdrawal symptoms and introducing them to scoring COWS based on physical examination. All patients were offered either buprenorphine induction, a prescription, or both. Within an hour of each other, but independently, both the nurse and clinician assigned to that patient independently assigned a COWS score. Because of separate charting within the electronic medical record (MedHost), the ED nurse and ED clinician were blinded to the score attribution by the other. Repeat COWS after buprenorphine induction was not mandated by the study protocol.

The hospital is located 12 miles north of the United States ‐Mexico border where 87.0% of the total population identify as LatinX/Hispanic. Additionally, the hospital site nursing staff also identify as LatinX/Hispanic and are bilingual. Most patients spoke English. Those patients who were Spanish speaking only communicated through certified Spanish interpreters who are also members of the ED staff.

For our study, we adopted the opioid withdrawal classification categories, which were initially introduced by Wesson and Ling in 2003 and based on the author's clinical expertise. However, it was left to each individual clinician to decide at what withdrawal classification they would start buprenorphine induction.

The chart review methods and the data abstraction process were standardized. A database registry and tracking log of all patients seen in the ED for opiate use disorder was established and information entered retrospectively during the study period. One of the principal investigators certified in human subject research by the Collaborative Institutional Training Initiative and with experience in data abstraction and knowledgeable about the medical record was primarily responsible for data abstraction. For monitoring data validity and quality assurance, the study data at multiple intervals were cross‐referenced with the original patient documents.

For each patient, baseline data including age, gender, ethnicity, and whether the patient received buprenorphine induction in the ED were collected. Data were also collected on the total dose of buprenorphine administered and the overall condition of the patient after induction. Post induction vital signs were noted including documentation of buprenorphine‐precipitated withdrawal, respiratory depression, excessive sedation, or other unpleasant side effects such as feeling worse, itching, headache, nausea, or vomiting that could be attributed to buprenorphine. Additional clinical data collected at each COWS evaluation included date and time of assessment and clinician type (clinician versus nurse).

Selection of participants

Patients were eligible for this study if they presented to the ED primarily for the treatment of opioid withdrawal. There were 142 patients identified as having a COWS performed independently by both the nurse and attending clinician. Eighteen observation pairs were removed for having more than 60 minutes between the nurse and clinician observations. An additional 2 pairs were removed for having incomplete subfactor information (pulse, sweating, restlessness, etc). In total, 120 pairs were included and analyzed.

Measurements

We estimated the frequency of concordance between the overall COWS scores obtained by the 2 observers, clinician versus nurse. Their agreement was assessed categorically, first, based on overall degree of withdrawal: none 0–4; mild 5–12; moderate 13–24; moderate‐severe 25–36; severe > 36. Second, the data were dichotomized by COWS score ≥ 5, which was the cutoff used for initiating buprenorphine in the ED. Finally, each COWS score was deconstructed into 11 categorical indices that contributed to the overall COWS score to examine agreement between clinician and nurse pairs.

The Bottom Line

The Clinical Opiate Withdrawal Scale (COWS) is a validated, commonly used tool to objectively quantify withdrawal symptoms. This study found that the agreement between emergency department clinicians and nurses for the overall COWS scoring in patients presenting for opioid withdrawal treatment was substantial. COWS scoring by ED nurses may help expedite treatment with buprenorphine on presentation.

Statistical analysis

Simple concordance between clinician and scoring was calculated, along with 95% confidence intervals (CIs), and compared with a Bland‐Altman plot. For time differences and overall scoring, due to non‐normally distributed data, we calculated medians with interquartile range (IQR) 1–3 range. To measure interobserver agreement, a weighted kappa statistic was used to measure concordance beyond chance. An agreement was considered perfect if kappa > 0.8, substantial if 0.61 to 0.8, moderate 0.41 to 0.6, fair 0.21 to 0.4, and slight if below 0.2. Sample size calculation was based on a priori assumption that raters would classify 50% of subjects as positive (COWS scores ≥5) with an alpha of 0.05 and power of 0.80. Based on that calculation, we required a minimum of 96 subjects.

RESULTS

Characteristics of study subjects

We collected data on 120 subjects, 94 (78.3%) of whom were male with an average age of 38 years (range 19–70). Ninety‐two (77%) of the 120 subjects identified primarily as Hispanic or Latino, 22 (18%) White/Non‐Hispanic, 3 (2.5%) African American, and 3 (2.5%) Other. The median (IQR) time interval between nurse and clinician COWS assessment was 15 (6–30) minutes. Buprenorphine dosing for induction was done during the index ED visit for 68 (57%) of the patients (Table 1). No adverse effects or worsening withdrawal was reported.

TABLE 1

Buprenorphine dosing

Induction in ED	Dose (mg)	Number (n = 120)	Percentage
No	0	52	43
Yes	2	7	5.8
	4	1	0.8
	6	1	0.8
	8	38	32
	16	19	16
	24	2	1.6

Abbreviation: ED, emergency department.

The median (IQR) overall COWS score, based on clinician scoring was a mild 6 (2–12), ranging from 0 (none) to 31 (moderate‐severe), median based on nurse scoring, mild 5 (2–8.3) with range 0 (none)–27 (moderate‐severe). Breakdown by category of withdrawal severity (based on clinician categorization), revealed that all patients with moderate to moderately severe withdrawal received buprenorphine (Table 2).

TABLE 2

Buprenorphine ED induction based on withdrawal physician categorization

Withdrawal severity	Number of patients	Number induced	Percentage induced (%)
None (0–4)	51	9	17.6
Mild (5–12)	53	34	64.2
Moderate (13–24)	22	22	100
Moderate severe (25–36)	3	3	100
Severe (> 37)	0	0	N/A
TOTAL	120	70	58.3

Abbreviation: ED, emergency department.

Main results

The Bland‐Altman plot (Figure 1) shows that agreement generally good at lower mean overall COWS scores increasingly diverges at higher scores, that is, patients with worse withdrawal.

FIGURE 1

Bland‐Altman plot showing comparison of Clinical Opioid Withdrawal Scale (COWS) scores between MD and registered nurse (RN) (mean of differences ± 95% limits of agreement)

The overall observed concordance, based on severity withdrawal categorization, for all clinician pairs, was 67.5% (81/120) (95% CI, 58.7–75.2%). The weighted kappa for that concordance was 0.55 (95% CI, 0.43–0.66), giving a moderate strength of agreement. When data were dichotomized by COWS score ≥ 5, concordance was 82.5% (99/120) (95% CI, 74.7%–88.3%) and the weighted kappa was 0.65 (95% CI, 0.51–0.78), indicating substantial agreement. Further analysis by the 11 factors measured in calculating the COWS score, showed only substantial strength of agreement for pulse measurement; all others ranged from fair to moderate (Table 3). The breakdown by individual measures showed the highest agreement for observed (clinician‐rated) measures of pulse and pupil size with less agreement among the rest of the measures, especially the subjective (patient‐rated) items.

TABLE 3

Observed concordance along with weighted kappas for agreements by individual contributor to COWS

Individual measurements	Observed concordance	Weighted kappa	95% CI LL	95% CI UL	Strength of agreement
Heart rate	84.1%	0.78	0.68	0.88	Substantial
Sweating	65.8%	0.45	0.31	0.59	Moderate
Restlessness	64.2%	0.50	0.28	0.56	Moderate
Pupil size	70.0%	0.18	0	0.38	Fair
Bone/joint aches	60.0%	0.42	0.29	0.54	Moderate
Runny nose/tears	70.8%	0.33	0.15	0.51	Fair
Gastrointestinal upset	65.0%	0.48	0.35	0.61	Moderate
Tremor	65.0%	0.32	0.15	0.49	Fair
Yawning	81.7%	0.29	0.10	0.48	Fair
Anxiety/irritability	58.3%	0.46	0.35	0.58	Moderate
Gooseflesh	83.3%	0.44	0.21	0.66	Moderate

Abbreviations: CI, confidence interval; COWS, Clinical Opiate Withdrawal Scale; LL, lower limit; UL, upper limit.

Repeat COWS scores were not a study protocol. However, a few clinicians and nurses documented repeat scores post buprenorphine induction and discharge. Although these scores were similar for the 2 study groups, the numbers were insufficient for statistical analysis.

Limitations

Brief training sessions with the nurses as well as clinicians in COWS scoring were performed before the study. Nursing staff who undergoes simulation and debriefing on opioid withdrawal cases had significant improvement in scoring outcome reliability but no change in confidence in correct scoring. Timing of assessment could have influenced our results, because nurses usually were the first to perform the COWS, followed by the clinicians. In addition to the sequence effect, the time gap, although minimal, could have decreased the agreement seen in this study. There is always the possibility that the clinicians could have previewed the nurse's COWS score rather than performing an indirect COWS score independently through direct observation on the same patient. However, this was unlikely given the compartmentalization of the electronic record. In this chart review, data abstraction was not blinded to the purpose of the study and there is always a possibility that errors of intrarater reliability could arise by the possibility of contamination by knowledge of previous data coding because the same abstractor recorded the same set of variables.

The patient population was largely Hispanic (77%) and male (78.8%). This could raise questions about how generalizable this is to other patient populations, and whether language, culture, or bias affected patient's COWS scores. Finally, obviously, there is no way possible to have a true gold standard for the COWS in this study because of the subjective nature of portions of the test.

DISCUSSION

OUD is currently a critical public health crisis. Although there are 3 Food and Drug Administration (FDA)‐approved drugs that can be used for effective MAT, there are many obstacles standing in the gap. These roadblocks include lack of linkage to treatment facilities, federal restrictions limiting the availability of healthcare clinicians that can prescribe MAT, getting buy‐in from ED clinicians that MAT is an appropriate use of the ED, and alleviating and overcoming their understandable fear of buprenorphine induced precipitated opiate withdrawal. COWS is the most used tool for assessing the severity of opioid withdrawal in the ED and is very simple to administer.

Recently, several treatment pathways have been employed for OUD, a chronic disease that can and should be treated. These therapeutic modalities usually involve opioid maintenance treatment, detoxification (detox), , and MAT (formerly known as medication‐assisted treatment). , MAT is an evidence‐based treatment for OUD that combines behavioral therapy and FDA‐approved medications including methadone, naltrexone, and buprenorphine. Opioid agonists like methadone and partial agonists like buprenorphine, substances that fully bind to and stimulate opioid receptors, are frequently used for both maintenance and detox purposes. There is evidence suggesting that medication‐assisted treatment is more effective than opioid maintenance treatment and detox. In a recent comparative effectiveness research study of patients with OUD that compared 6 different treatment pathways, only treatment with buprenorphine or methadone was associated with reduced risk of overdose and serious opioid‐related acute care use compared with no treatment during 3 and 12 months of follow‐up.

Effective opioid withdrawal management requires early identification and assessment of withdrawal symptoms. The main method of assessing and quantifying opioid withdrawal intensity has quintessentially involved withdrawal scales of some form with at least 18 different opioid withdrawal scales used within the last 80 years. Such scales tend to employ a combination of patients’ self‐reports (symptoms), observable behaviors (signs), and/or measured physiological parameters. The scores for the 11‐item COWS, the current most popular opioid withdrawal scale, range from 0 to 47 with specific ranges for the level of severity. Scores from 5 to 12 are considered mild, scores from 13 to 24 are considered moderate, scores from 25 to 26 are considered moderately severe, and scores more than 36 are considered severe withdrawal.

Although it has been recognized in clinical practice that there may be substantial interrater variability with COWS there is also a lack of consensus regarding the appropriate diagnostic thresholds to start ED buprenorphine induction. Different clinicians have adopted different levels of initial COWS levels on arrival to start buprenorphine induction. With the increasing availability of MAT in the controlled setting of the ED a target score of 5–12 is increasingly adopted. Even then a few clinicians are still comfortable to base their inductions purely on patient history and clinical assessment of presumed opiate withdrawal without a calculated and quantifiable scale.

The primary aim of our study was to determine if there was significant agreement between ED nurses compared to emergency physicians in determining this score in ED patients who presented for opioid withdrawal treatment. This study showed relatively good agreement between nurses and clinicians using COWS in patients presenting to the ED either in opioid withdrawal or in seeking help with opioid withdrawal. The best agreement was observed in using the treatment cutoff of ≥5.

To our knowledge, there are no prior clinical studies of interrater agreement between ED clinicians and nurses. Although the concordance was good in our study, it was not excellent. There are possible explanations for this. The COWS consists of observed (clinician‐rated) as well as subjective (patient‐rated) components of opiate withdrawal signs and symptoms. The COWS items can be identified as 6 objective items (resting heart rate, tremor, yawning, pupil size, gooseflesh skin, and runny nose or tearing), 1 subjective item (anxiety or irritability), and 4 items that have both objective and subject components (gastrointestinal upset, sweating, restlessness, and bone or joint aches). The subjective components can be interpreted and rated differently by different observers leading to interobserver variability. In our study, this was the first time both groups were calculating COWS scores. It would be the expectation that with practice and experience in time there would be a further boost to the agreement between the 2 groups. The clinical implications for patient care would be that any ED instituting a COWS protocol should have a concurrent audit initially.

The underlying premise behind clinical guidelines for therapeutic intervention in opiate withdrawal is that buprenorphine induction should occur in a patient who is physically dependent on opioids and is in mild to moderate withdrawal or waiting for that patient to be in in moderate to severe opiate withdrawal. Different protocols suggest dosing at different COWS scores for buprenorphine induction for patients in mild to moderate withdrawal (COWS 5–24) in order to avoid precipitated withdrawal. In our study the best agreement was in using the treatment cutoff of ≥5 on COWS. At this level, we did not experience any precipitated withdrawal. Looking at this cutoff is important because it has the most implications for treatment. This is the level at which the first dose of buprenorphine is offered in real‐time as opposed to discharge with a prescription, which we offered to all patients who accepted MAT.

Many clinicians still shy away from initiating buprenorphine induction in patients with mild withdrawal symptoms because of a fear of precipitated withdrawal. As evidenced in our study, buprenorphine precipitated withdrawal, although very real, may not be as common as clinicians think. However, it has recently been shown that about 1 in 20 patients treated for a non‐fatal opioid overdose in an ED died within 1 year of their visit, many within 2 days following the visit. Two‐thirds of these deaths were directly attributed to subsequent opioid‐related overdoses. Immediate buprenorphine induction for many more patients with substance use disorder in the ED during the index visit and continued treatment after would save lives by reducing opioid‐related deaths.

Based on prior studies, we expected that the ED nurses and emergency physicians to have an adequate amount of agreement using the COWS. Nurses, with brief training, can identify a COWS score to make appropriate changes in treatment plans of patients suffering from opioid withdrawal. Although our program does not use LPNs, we believe that even they too could be trained to score for COWS. Clinicians did tend to score patients higher for withdrawal, which may be a function of the time gap between nursing and physician scoring because in most practices nurses are at the bedside earlier than clinicians. An alternative explanation could be differences in the assessment approach. We expected the best agreement would be for pulse rate, one of the most objective measurements. Surprisingly, some of the more subjective measures, such as the presence of aches, restlessness, or anxiety, scored better than more objective measures, such as pupil size and tremor.

Although the patient population was largely Hispanic (77%) and male (78.8%), we believe that language, gender, culture, or implicit bias did not affect patient's COWS scores. We feel that our findings could be generalizable to other patient populations.

In conclusion, we found that there is an adequate amount of agreement between nurses and emergency clinicians using the COWS. Future studies should be aimed at allowing expanded use of COWS for ED nurses in triaging opioid withdrawal patients, which can expedite proper identification and treatment of ED patients with buprenorphine.

AUTHOR CONTRIBUTIONS

CAT conceived the original study concept. CAT and LM refined the study design and obtained research funding. LM was responsible for the quality and accuracy of the data acquisition and handling. CAT and FQ ran the statistical analysis of the data. Christian A. TomaszewskiCAT drafted the manuscript, and all authors contributed substantially to its data analysis and revision. CAT and LM take responsibility for the paper as a whole.

All authors attest to meeting the 4 ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

CONFLICTS OF INTEREST

None.