Application of risk assessment tools to predict opioid usage after shoulder surgery.

Background: Currently 128 people die daily from opioid-related overdoses in the United States. This burden has instigated a search for viable means to guide postoperative prescription decision-making. The Opioid Risk Tool (ORT) and the Screener and Opioid Assessment for Patient with Pain (SOAPP) are validated risk assessment tools to predict opioid usage in high-risk populations. The purpose of this study was to evaluate the accuracy of these opioid risk assessments and pain intensity scores, including the Patient-Reported Outcomes Measurement Information System (PROMIS), to predict postoperative opioid use and dependence in shoulder surgery.
Methods: A retrospective review of 81 patients who underwent shoulder surgery and completed 3 preoperative risk and pain assessments within a single hospital system from 2018 to 2020 was performed. Demographic variables and ORT-O, SOAPP-R (the revised version of the SOAPP assessment), and PROMIS 3a scores were recorded from preoperative assessments. Opioid prescriptions were recorded from Electronic-Florida Online Reporting of Controlled Substances Evaluation. Dependence was defined as opioid prescriptions at or greater than 3 months after surgery. Risk assessment scores were compared and tested against postoperative opioid prescriptions using statistical analyses and logistic regression modeling.
Results: In the cohort, there were 36 female and 45 male patients with an average age of 64.5 years and body mass index of 28.0. Preoperatively, the average pain score was 6.2, and 7.8% of patients reported prolonged preoperative narcotics use. The average ORT-O score was 3.0, with 35.8% of patients defined as either medium or high risk, and the average PROMIS pain intensity preoperatively was 10.8. Neither the ORT-O nor the PROMIS pain score were good predictors of postoperative opioid dependence (area under curve = 0.39 and 0.43, respectively). The SOAPP-R performed slightly better (area under curve = 0.70) and was the only assessment with significantly different mean scores between patients with postoperative opioid dependence and those without (33.4 and 24.5, respectively, P = .049) and a moderate correlation to postoperative total morphine equivalents (R = 0.46, P = .007).
Conclusion: With recent focus on preoperative risk assessments to predict postoperative opioid use and dependence, it is important to understand how well these tools work when applied to orthopedic patients. While the ORT may be helpful in other fields, it does not seem to be a strong predictor of postoperative opioid use or dependence in patients undergoing various types of shoulder surgery. Future studies are needed to explore the utility of the SOAPP-R in a larger sample and identify tools applicable to the orthopedic population to assist surgeons in screening at-risk patients.

The opioid epidemic continues to pose a significant threat in the United States with roughly 128 people dying each day from opioid-related overdoses in the United States in 2018., While the United States comprises just 5% of the global population, we account for 80% of global opioid consumption. Among specialties, orthopedic surgeons are one of the highest prescribers of opioids in the United States, and it has been reported that as much as 75% of prescribed postoperative opioids go unused.,,,,, Most orthopedic surgeons agree that overprescription is a problem, and significant efforts have been made to reduce opioid prescribing practices including the investigation and development of multimodal pain-management protocols and patient education tools.,,,,, However, the use of narcotics for postoperative pain control still remains frequent, particularly following shoulder surgeries.,, Given the persistent challenge of the opioid crisis and the rapidly increasing utilization of shoulder surgery, there is a growing need for tools to identify patients at risk of chronic use, abuse, and dependence.,

Several studies have identified specific patient demographics and preoperative factors predictive of increased and prolonged use of opioids after an upper extremity surgery.,,,, In the present literature, considerable evidence suggests that preoperative opioid consumption, abuse, or dependence is predictive of increased opioid consumption, prolonged use, and dependence perioperatively and postoperatively (Table I).,,,,,41, 42, 43,,,,, Preoperative opioid use has also been linked to worse clinical outcomes and lower rates of patient satisfaction.,, Sabesan et al found that, compared to other pathologies, use of a reverse shoulder arthroplasty after a proximal humerus fracture was associated with an increased postoperative opioid dependence, and Jildeh et al found specific surgical factors including presence of biceps tenodesis and number of concomitant procedures significantly increased postoperative opioid demand., Additionally, social and medical history factors including current smoking status, income bracket, young age, history of psychiatric and mood disorders, fibromyalgia, and low back pain have been found to contribute to increased risk.,,,,,,,, Despite significant interest and investigation into individual factors to predict chronic opioid use and dependence, there is a paucity of data in the orthopedic literature on simple, effective screening tools for shoulder surgery patients that are easily implemented in the clinical environment.

Table I

Opioid consumption after surgery studies.

Author	Name of study	Type of study	Type of surgery	Comments
Berglund et al, 20184	Effect of opioid dependence or abuse on opioid utilization after shoulder arthroplasty	Retrospective chart review	Shoulder arthroplasty	Opioid use is similar within the first postoperative month but is greater among opioid-dependent patients from 2 to 12 mo.
Chatha et al, 202013	How orthopedic surgeons can impact opioid use and dependence in shoulder arthroplasty	Retrospective case-control	Shoulder arthroplasty	Total shoulder arthroplasty patients have 3.5× increased risk for postoperative opioids if they use opioids preoperatively.
Cheah et al, 201714	The perioperative effects of chronic preoperative opioid use on shoulder arthroplasty outcomes	Retrospective cohort design	Total shoulder arthroplastyReverse total shoulder arthroplasty	Preoperative opioid use was associated with significantly higher perioperative opioid consumption.
Curtis et al, 201919	Effect of preoperative opioid usage on pain after total shoulder arthroplasty	Retrospective design	Total shoulder arthroplasty	Preoperative opioid use strongly predicted postoperative opioid use in patients that underwent total shoulder arthroplasty.
Jildeh et al, 202039	Risk factors for postoperative opioid use in arthroscopic shoulder labral surgery	Retrospective chart review	Shoulder arthroscopy	Preoperative opioid use, number of procedures at the time of initial surgery, and presence of biceps tenodesis were found to significantly increase the demand for opioids postoperatively.
Jones et al, 201940	Opioid use following shoulder stabilization surgery: risk factors for prolonged use	Retrospective design	Shoulder arthroscopy	Preoperative opioid use in patients undergoing shoulder arthroscopy had the highest risk of prolonged opioid use after surgery.
Khazi et al, 202041	Risk factors for opioid use after total shoulder arthroplasty	Retrospective cohort comparison	Total shoulder arthroplasty	Chronic preoperative opioid use had the highest risk of prolonged postoperative opioid use in patients undergoing total shoulder arthroplasty.
Kolade et al, 202043	Study of variations in inpatient opioid consumption after total shoulder arthroplasty: influence of patient- and surgeon-related factors	Cross-sectional design	Total shoulder arthroplasty	Preexisting psychiatric disorders, preoperative opioid use, highest quartile of median household income, current-smoker status, age <60 years were associated with significant inpatient opioid consumption after total shoulder arthroplasty.
Martusiewicz et al, 202047	Outpatient narcotic consumption following total shoulder arthroplasty	Prospective cohort	Total shoulder arthroplasty	Preoperative opioid use was associated with increased opioid consumption.
Okoroha et al, 201950	Pain after anatomic total shoulder arthroplasty vs. reverse total shoulder arthroplasty	Prospective cohort	Total shoulder arthroplastyReverse total shoulder arthroplasty	Patients with preoperative opioid use were more likely to continue to require opioids postoperatively.
Sabesan et al, 201956	Diagnosis can predict opioid usage and dependence in reverse shoulder arthroplasty	Retrospective chart review	Total shoulder arthroplasty	Preoperative opioid-dependent patients had 8× higher risk to remain dependent after surgery.
Westermann et al, 201768	Opioid consumption after rotator cuff repair	Retrospective chart review	Shoulder arthroscopy	Patients with psychiatric conditions, lower back pain, myalgia, and preoperative opioid use have increased risk of postoperative opioid use.

Two popular risk assessment tools recommended by the American Academy of Orthopaedic Surgeons and other medical specialties that are widely used in practice are The Opioid Risk Tool (ORT) and the Screener and Opioid Assessment for Patient with Pain (SOAPP)., The ORT is a self-reported instrument that can be administered by a clinician or independently by the patient. It was originally validated for prediction of aberrant drug-related behaviors (ADRBs) in a chronic pain population by Lynn Webster and Rebecca Webster (2005) and considers multiple factors generating a total weighted numeric score that is then categorized by risk (low, moderate, or high). Subsequent investigators experimented with modified editions for broader applicability, including the ORT-O which includes additional orthopedic and upper extremity-specific questions., The SOAPP-R, the revised version of the SOAPP assessment, is a concise, validated patient-reported outcome measure designed to predict ADRBs in chronic pain patients. Using answers from Never (0) to Very Likely (4) on 24 questions, patients are determined as high or low risk for ADRBs using cutoff scores. While these tools have been used with some success to predict opioid use in other patient populations, they have not yet been validated for use in patients undergoing shoulder surgery.,,,,,,, In addition to risk-assessment tools, a variety of easily administered pain scales have been explored to determine if preoperative pain severity can predict postoperative opioid use., The Patient-Reported Outcomes Measurement Information System (PROMIS), created by the National Institute of Health, includes a number of questions including pain assessment that have been validated against traditional pain scales in several areas of medicine, including orthopedics and upper extremity surgery specifically., However, the simplest of the PROMIS pain scores, the 3-item PROMIS 3a Pain Intensity survey, has not yet been investigated for its potential to predict postoperative opioid dependence following shoulder surgery.

Considering the familiarity and ease of administration of the ORT, SOAPP-R, and PROMIS 3a, the applicability of these tools in prediction of opioid consumption and outcomes following shoulder surgery is important. The purpose of this study was to validate and evaluate the accuracy of these 3 scores to predict postoperative opioid use after shoulder surgery. We hypothesize that if these tools have been validated in other surgical populations, then they should be able to sufficiently predict opioid dependence in shoulder surgery patients.

Materials and methods

This was an institutional review board-approved retrospective review of prospectively collected data of 400 patients in a shoulder arthroplasty registry. There were 81 patients identified who completed risk assessment tools preoperatively and underwent shoulder surgery performed by 2 fellowship-trained shoulder surgeons between 2018 and 2020 within a single hospital system. All 3 risk assessment tools were administered to patients preoperatively, and data on narcotic use after surgery were collected. All types of shoulder surgery were included in the study (total, reverse shoulder arthroplasty, rotator cuff repair). Demographic variables including age, sex, American Society of Anesthesiologists class, body mass index, and surgery type were obtained from chart review. Data were collected using preoperative and postoperative assessments. Preoperative assessments included self-reported patient responses for pain visual analog score (VAS), narcotics use in the 3-month and 1-month time periods before surgery, ORT-O, SOAPP-R, and PROMIS 3a questionnaires (Supplementary Appendices S1-S3). Preoperative and postoperative opioid use data were obtained from the Electronic-Florida Online Reporting of Controlled Substances Evaluation.

The American Shoulder and Elbow Surgeons (ASES) pain score was calculated by subtracting the VAS from 10 and multiplying by 5. ORT-O scores were calculated based on scored patient responses (Supplementary Appendix S1), and patients were then categorized as either low risk (0-3), medium risk (4-7), or high risk (≥8). Positive scores were defined as those in either the medium- or high-risk category. SOAPP-R scores were calculated based on the SOAPP-R 24-question patient questionnaire with answer choices of Never (0), Seldom (1), Sometimes (2), Often (3), and Very Often (4) (Supplementary Appendix S2). Scores totaling greater than 18 were considered positive, while scores less than 18 were considered negative. PROMIS scores were calculated by summing the points received on each of the 3 questions included in the PROMIS pain intensity scale (Supplementary Appendix S3). Patients were categorized as low risk (3-8), medium risk (9-11), or high risk (12-15). Positive scores were defined as those in either the medium- or high-risk category. Postoperative total morphine equivalents (TMEs) were calculated for each patient's postoperative opioid usage. Postoperative dependence was defined as opioid prescriptions for ≥3 months after surgery.

Statistical analyses

Descriptive statistics were used to summarize opioid consumption patterns, whereby baseline characteristics of the sample were summarized using means and standard deviations for continuous variables and percentages for categorical variables. The accuracy of VAS, pain score of ASES, PROMIS 3a, and each risk tool (ORT-O and SOAPP-R) to predict the use of opioids at 1 and 3 months after surgery was evaluated obtaining the area under receiver operating characteristic (ROC) curves. ROC curves were generated by plotting sensitivity on the y-axis as a function of [1-specificity] on the x-axis for a continuum of diagnostic criteria. This was used to visualize analysis of the trade-offs between the sensitivity and specificity of each risk tool regarding the predictive accuracy of each scale. Specifically, it was used to assess the predicative accuracy of each scale to classify patients as high or low risk for postoperative opioid use by obtaining sensitivities, specificities, predictive values, and likelihood ratios, employing the observed use of opioids at 1 and 3 months after surgery as the reference standard. This type of analyses using ROC curves has been previously used to assess the validity and reliability of the QuickDASH score. A shift to the right on an ROC curve is equivalent to increasing sensitivity, whereas a shift toward the left signifies an increase in specificity, with decrease in sensitivity of the diagnostic test being assessed. A “perfect” ROC curve would be a right angle along the positive y and x axes proving sensitivity and specificity equal to 100% for the risk tool assessment. T-tests were used to compare the mean scores obtained in each scale between patients who used and did not use opioids at 1 and 3 months after surgery. The difference between means along with 95% confidence intervals (95% CI) was also obtained. Lastly, the Pearson correlation coefficient was used to correlate the association between the score provided by each scale and total TME after surgery. All analyses were conducted using IBM SPSS Statistics version 26 (Armonk, NY, USA).

Results

Demographics and characteristics of the patients included in the study are presented in Table II. There were slightly more males (55.6%) than females, and the average age was 64.5 years (±12.3) with an average body mass index of 28.0 (±3.6). Of the 81 patients in the cohort, ORT-O scores were available for all (100%), PROMIS 3a Pain Intensity and VAS pain scores were available for most (95% and 85%, respectively), and SOAPP-R scores were available for 33 patients (41%). Preoperatively, the mean VAS pain score was 6.2 (±2.7), mean ASES pain score was 18.8 (±13.6), and mean PROMIS 3a pain intensity score was 10.8 (±2.9) with 60 patients (77.9%) in the medium- or high-risk category. The average ORT-O score was 3.0 (±3.3), with 52 patients (64.2%) in the low-risk category, 25 patients (30.8%) in the medium-risk category, and 4 (4.9%) patients in the high-risk category. Of the patients with SOAPP-R assessments available, the average score was 27.2 (±12.0), and 31 patients (93.9%) had positive scores. Six (7.8%) patients reported prolonged preoperative narcotics use at least 3 months before surgery, and 13 (16.9%) reported use within the month before surgery. Postoperatively, 61 patients (75.3%) were prescribed opioids in the month after surgery, and 16 patients (19.8%) were prescribed opioids at or after 3 months postoperatively. On average, there were 0.8 (±1.6) postoperative opioid prescriptions per patient, and 29 patients (35.8%) refilled their postoperative prescription. The mean postoperative TME was 58.8 (±75.1). (Table II)

Table II

Patient demographics and characteristics.

Preoperative	N	Value
Gender: male, N (%)	81	45 (55.6)
Age (yr), mean (SD)	81	64.5 (12.3)
BMI, mean (SD)	12	28.0 (3.6)
VAS pain score, mean (SD)	69	6.2 (2.7)
ASES pain score, mean (SD)	69	18.8 (13.6)
PROMIS pain intensity, mean (SD)	77	10.8 (2.9)
Medium and high risk by PROMIS, N (%)	77	60 (77.9)
ORT-O score, mean (SD)	81	3.0 (3.3)
Medium and high risk by ORT-O, N (%)	81	29 (35.8)
SOAPP-R score, mean (SD)	33	27.2 (12.0)
Positive by SOAPP-R, N (%)	33	31 (93.9)
Narcotics use reported at 3 mo preop, N (%)	77	6 (7.8)
Narcotics use reported at 1 mo preop, N (%)	77	13 (16.9)
Postoperative
Used opioids at 1 mo postop, N (%)	81	61 (75.3)
Used opioids at 3 mo postop, N (%)	81	16 (19.8)
Number of postop prescriptions, mean (SD)	81	0.8 (1.6)
Postop prescription refills, N (%)	81	29 (35.8)
Postop TME, mean (SD)	81	58.8 (75.1)

SD, standard deviation; BMI, body mass index; VAS, visual analog score; ASES, American Shoulder and Elbow Surgeons; PROMIS, Patient-Reported Outcomes Measurement Information System; ORT-O, Opioid Risk Tool with orthopedic questions; SOAPP-R, revised Screener and Opioid Assessment for Patient with Pain; TME, total morphine equivalent.

The discrimination of the ORT-O was no better than chance for predicting opioid use at 1 month (Fig. 1, A) and 3 months (Fig. 1, B) postoperatively (area under curve [AUC] = 0.41, 95% CI = 0.26-0.55, and AUC = 0.39, 95% CI = 0.22-0.57, respectively). The sensitivity and specificity values of the ORT-O for the identification of opioid use 1 month after surgery were 32.8% (95% CI = 22.3-45.3) and 45.0% (95% CI = 25.8-65.8), respectively, and those at 3 months after surgery were 25% (95% CI = 10.2-49.5) and 58.5% (95% CI = 46.3-69.6), respectively, (Table III). The mean ORT-O scores of patients who used and did not use opioids at 1 month and 3 months after surgery were not significantly different (P = .48 and P = .80, respectively) (Table IV). Despite being statistically significant, the correlation between ORT scores and TME is considered to be weak by a conventional approach t (R = 0.31, P = .005) such that the R value is between 0.10 and 0.39 (Table V).

Figure 1

ROC curve for ORT-O as a predictor of use of opioids at 1 month (A) and 3 months after surgery (B). ROC, receiver operating characteristic; ORT-O, Opioid Risk Tool with orthopedic questions.

Table III

Diagnostic accuracy of PROMIS, ORT-O, and SOAPP-R scores for the identification of opioid use after surgery.

Parameter	Use at 1 mo postop	95% CI	Use at 3 mo postop	95% CI
PROMIS pain intensity (N = 77)
Sensitivity	77.2%	64.8-86.2	78.57%	52.41-92.43
Specificity	20.0%	8.1-41.6	22.22%	13.72-33.91
Positive predictive value	73.3%	61.0-82.9	18.33%	10.56-29.92
Negative predictive value	23.5%	9.555-47.26	82.35%	58.97-93.81
Diagnostic accuracy	62.3%	51.2-72.3	32.47%	23.06-43.54
Likelihood ratio of a positive test	1.0	0.8-1.1	1.01	0.92-1.10
Likelihood ratio of a negative test	1.1	0.1-9.4	0.96	0.31-3.03
Diagnostic odds	0.8	0.2-3.0	1.05	0.26-4.28
ORT-O (N = 81)
Sensitivity	32.8%	22.3-45.3	25%	10.18-49.5
Specificity	45.0%	25.8-65.8	58.46%	46.34-69.64
Positive predictive value	64.5%	47.0-78.91	12.9%	5.13-28.85
Negative predictive value	18.0%	9.8-30.8	76%	62.59 - 85.70
Diagnostic accuracy	35.8%	26.2-46.7	51.85%	41.14-62.40
Likelihood ratio of a positive test	0.6	0.4-0.9	0.60	0.13-2.80
Likelihood ratio of a negative test	1.5	1.1-2.0	1.28	1.05-1.57
Diagnostic odds	0.4	0.1-1.1	0.47	0.14-1.61
SOAPP-R (N = 33)
Sensitivity	92.3%	75.9-97.9	100%	72.25-100
Specificity	0.0%	0.0-35.4	8.70%	2.42-26.80
Positive predictive value	77.4%	60.2-88.6	32.26%	18.57-49.86
Negative predictive value	0.0%	0.0-65.8	100%	34.24-100
Diagnostic accuracy	72.7%	55.8-84.9	36.36%	22.19-53.38
Likelihood ratio of a positive test	0.9231	NA∗	1.095	0.99-1.20
Likelihood ratio of a negative test	NA∗	NA∗	0.0	NA∗
Diagnostic odds	NA∗	NA∗	NA∗	NA∗

PROMIS, Patient-Reported Outcomes Measurement Information System; ORT-O, Opioid Risk Tool with orthopedic questions; SOAPP-R, revised Screener and Opioid Assessment for Patient with Pain; CI, confidence interval.

NA, given that the value for false negatives was 0.

Table IV

Differences of mean PROMIS pain intensity, ORT-O, and SOAPP-R scores between patients who used and did not use opioids at 1 mo and 3 mo after surgery.

	Opioid use at 1 mo postop				Mean difference (95% CI)	P value
	Yes		No
	N	Mean (SD)	N	Mean (SD)
PROMIS pain	57	10.9 (3.1)	20	10.6 (2.4)	0.3 (−1.2 to 1.8)	.69
ORT-O	61	2.8 (3.5)	20	3.4 (2.6)	−0.6 (−2.3 to 1.1)	.48
SOAPP-R	26	27.6 (13.4)	7	25.9 (3.8)	1.7 (−8.8 to 12.3)	.74

PROMIS, Patient-Reported Outcomes Measurement Information System; ORT-O, Opioid Risk Tool with orthopedic questions; SOAPP-R, revised Screener and Opioid Assessment for Patient with Pain; CI, confidence interval; SD, standard deviation.

Statistically significant P < .05.

Table V

Correlation coefficients for the association between PROMIS pain intensity, ORT-O, and SOAPP-R scores and TME after surgery.

Scale	Pearson coefficient (R)	P value
PROMIS pain	0.04	.708
ORT-O	0.31	.005∗
SOAPP-R	0.46	.007∗

PROMIS, Patient-Reported Outcomes Measurement Information System; ORT-O, Opioid Risk Tool with orthopedic questions; SOAPP-R, revised Screener and Opioid Assessment for Patient with Pain; TME, total morphine equivalent.

Statistically significant P < .05.

While the SOAPP-R score performed slightly better in predicting opioid use postoperatively, the smaller number of patients assessed resulted in poor precision for estimates at 1 month (Fig. 2, A) (AUC = 0.67, 95% CI = 0.33-0.77) and 3 months (Fig. 2, B) after surgery (AUC = 0.70, 95% CI = 0.50-0.91). The sensitivity and specificity values of the SOAPP-R for the identification of opioid use 1 month after surgery were 92.3% (95% CI = 75.9-97.9) and 0.0% (95% CI = 0.0-35.4), respectively, and those at 3 months after surgery were 100% (95% CI = 72.25-100) and 8.7% (95% CI = 2.4-26.8), respectively, (Table III). A statistically significant difference was observed between the mean SOAPP-R scores of patients who did and did not use opioids at 3 months postoperatively (8.9, 95% CI = 0.06-17.7, P < .05), but not at 1 month (P = .74) (Table IV). The SOAPP-R score showed a moderate correlation with postoperative TME that was statistically significant (R = 0.46, P = .007) (Table V).

Figure 2

ROC curve for SOAPP-R as a predictor of use of opioids at 1 month (A) and 3 months after surgery (B). ROC, receiver operating characteristic; SOAPP-R, revised Screener and Opioid Assessment for Patient with Pain.

The PROMIS 3a pain intensity score, VAS pain score, and ASES pain score did not show good predictive accuracy for opioid use at either 1 month or 3 months postoperatively (Figure 3, Figure 4, Figure 5). The ROC curve for PROMIS 3a showed an AUC of 0.53 and 0.43 at 1 month and 3 months, respectively. The sensitivity and specificity of the PROMIS 3a Pain Intensity for the identification of opioid use 1 month after surgery were 77.2% (95% CI = 64.8-86.2) and 20.0% (95% CI = 8.1-41.6), respectively, and those at 3 months after surgery were 78.6% and 22.2%, respectively, (Table III). In comparing the mean PROMIS 3a pain intensity scores of patients who used and did not use opioids at 1 month and 3 months after surgery, there was no significant difference at either time point (P = .69 and P = .55, respectively) (Table IV). The PROMIS pain intensity score also showed a negligible correlation to postoperative TME (R = 0.04, P = .71) (Table V). Finally, surveys of patients for opioid use both 3 months and 1 month prior to surgery were assessed for diagnostic accuracy of opioid use at both postoperative time intervals, but neither parameter yielded sensitivities and specificities that were simultaneously acceptable (Table VI).

Figure 3

ROC curve for PROMIS 3a as a predictor of use of opioids at 1 month (A) and 3 months after surgery (B). ROC, receiver operating characteristic; PROMIS 3a, Patient-Reported Outcomes Measurement Information System.

Figure 4

ROC curve for VAS pain as a predictor of use of opioids at 1 month (A) and 3 months after surgery (B). ROC, receiver operating characteristic; VAS, visual analog score.

Figure 5

ROC curve for ASES pain as a predictor of use of opioids at 1 month (A) and 3 months after surgery (B). ROC, receiver operating characteristic; ASES, American Shoulder and Elbow Surgeons.

Table VI

Diagnostic accuracy of preoperative opioid use for the identification of opioid use after surgery.

Parameter	Use at 1 mo postop	95% CI	Use at 3 mo postop	95% CI
Opioid use 3 mo before surgery (N = 77)
Sensitivity	8.7%	3.8-19.0	28.6	11.7-54.7
Specificity	95.0%	76.4-99.1	96.8	89.1-99.1
Positive predictive value	83.3%	43.7-97.0	66.7	30.0-90.3
Negative predictive value	26.8%	17.9-38.1	85.9	76.0-92.2
Diagnostic accuracy	31.2%	21.9-42.2	84.4	74.7-90.9
Likelihood ratio of a positive test	1.8	0.004-734.3	9.0	1.0-81.6
Likelihood ratio of a negative test	1.0	0.9-1.0	0.7	0.6-0.9
Diagnostic odds	1.8	0.2-16.7	12.2	2.0-75.6
Opioid use 1 mo before surgery (N = 77)
Sensitivity	17.5%	9.8-29.4	25.0	10.2-49.5
Specificity	85.0%	64.0-94.8	81.5	70.5-89.1
Positive predictive value	76.9%	49.7-91.8	25.0	10.2-49.5
Negative predictive value	26.6%	17.3-38.5	81.4	70.5-89.1
Diagnostic accuracy	35.1%	25.4-46.2	70.4	59.7-79.2
Likelihood ratio of a positive test	1.2	0.2-5.6	1.4	0.3-6.9
Likelihood ratio of a negative test	1.0	0.9-1.03	0.9	0.8-1.1
Diagnostic odds	1.2	0.3-4.9	1.5	0.4-5.4

CI, confidence interval.

Discussion

Opioid prescribing guidelines and standardized protocols have emerged to promote safer practices, reduce risk of dependence, and curb the opioid crisis. The American Academy of Orthopedic Surgeons recommends such protocols including the use of predictive screening tools, specifically the ORT and SOAPP, to identify patients at risk of opioid dependence after surgery., Despite these tools being endorsed for use in orthopedic populations, to our knowledge, none of these scores have been validated for effectiveness in predicting postoperative opioid use or dependence following upper extremity surgery.

The ORT was originally introduced in 2005 having demonstrated excellent c-statistics for predicting ADRBs (AUC = 0.82 for males and AUC = 0.85 for females) in chronic noncancer pain patients but has since been widely applied to other populations. In this study, the ORT-O score, a version of the ORT with additional orthopedic questions, did not perform any better than expected by chance for predicting opioid use or dependence (AUC = 0.41 for 1 month, AUC = 0.39 for 3 months), and the correlation between the ORT-O and TME after surgery was also considered weak (R = 0.31, P = .005). Previous reassessments of the ORT's validity and applicability to patient populations outside the original sample, a cohort of patients in Salt Lake City from 2001 to 2002, have indicated mixed results, leading to concerns with its broad utility.,,, In a cross-sectional study of chronic non-cancer-pain patients, Lakha et al observed discrepancies in the relevance of certain ORT risk factors related to gender and their relationship with the ORT risk classifications compared to the original study. In another pain-management sample, the self-report ORT was unable to reliably predict ADRBs, and there were significant differences between self-report and clinician-administered ORT scores., Our findings add to these concerns given that even the ORT-O with orthopedic questions did not show acceptable predictability of postoperative opioid use or dependence following surgery.

The SOAPP-R, a revised edition of the SOAPP introduced by Butler et al, differs from PROMIS and ORT in that it has been investigated as a useful tool to evaluate opioid use in orthopedic surgery. A 2015 study of total hip replacement, total knee replacement, thoracotomy, mastectomy, and lumpectomy patients found that a 9-point increase in SOAPP-R scores correlated with a 2.37 odds increase in preoperative opioid use and a 3.02 odds increase in illicit preoperative opioid use. Our analysis, aimed at predicting postoperative rather than preoperative opioid use, found that the SOAPP-R performed better than the other scores at predicting opioid use at 1 month and 3 months after surgery (AUC = 0.67 and 0.70, respectively), but the small number of patients assessed with this scale resulted in poor precision of the c-statistic and low specificity for postoperative opioid use and dependence (0.0% and 8.70%, respectively). Still, the SOAPP-R was the only scale with a statistically significant difference between the mean scores of patients who did and did not use opioids at 3 months after surgery (8.9, 95% CI = 0.06-17.7, P = .049) and was also the only score with moderate correlation to postoperative TME (R = 0.46, P = .007). Compared to the other screening tools assessed in the present study, the SOAPP-R is much more extensive with 24 questions that cover several psychosocial factors and ADRBs. One criticism of the applicability of the SOAPP-R to the clinical environment has been its length. However, in a study of emergency department patients who were administered the SOAPP-R on a tablet device, the mean time spent on the questionnaire was 164 seconds, with 75% of the patients completing it in less than 3 minutes. Still, studies validating shorter versions of the tool are ongoing.,,, Since our results indicated significance and with studies concluding the incorporation of this screening tool in other settings, SOAPP-R may warrant further investigation in the shoulder surgery population.

The presence and degree of preoperative pain and use of preoperative opioids have each been proposed as possible predictors of postoperative opioid use. We assessed the PROMIS 3a pain intensity score in addition to the widely used VAS pain score and calculated ASES pain score and found that none of these parameters exceeded chance discrimination for predicting opioid use at 1 month (AUC = 0.53, 0.5, and 0.49) or dependence after surgery (AUC = 0.43, 0.41, and 0.60, respectively). Furthermore, the PROMIS pain intensity score displayed weak diagnostic accuracy for postoperative opioid use (62.3% at 1 month, 32.47% at 3 months), and when assessed against the secondary outcome of postoperative TME, there was no correlation between the 2 (R = 0.04, P = .708). Overall, our findings did not support good predictability of the PROMIS pain intensity score, VAS, or ASES pain scores for opioid dependence after surgery.

Few studies have focused on a direct link between preoperative pain intensity and prolonged postoperative opioid use after orthopedic surgery. Abrecht et al identified pain score in the Postanesthesia Care Unit to be an independent predictor of perioperative opioid use after total knee arthroplasty, but patients were only followed up to postoperative day 2. Goesling et al found that patients still taking opioids 6 months after total knee and hip arthroplasty had statistically significant higher preoperative pain although the difference reported was small (1.11 on a scale of 0 to 10), and preoperative pain severity as an independent predictor for postoperative opioid use was not evaluated. The authors did, however, identify preoperative opioid use to be independently associated with persistent postoperative opioid use, consistent with a considerable and growing body of evidence in the orthopedic literature.,,,,,,,,, It is worth highlighting that only 16.9% of patients in our cohort reported preoperative narcotics use and that only 30.9% had opioid prescriptions in the 3 months before surgery according to Electronic-Florida Online Reporting of Controlled Substances Evaluation, which is slightly lower than the 36%-52% rate of preoperative opioid use cited in most studies,,,,, and may explain the parameter's low sensitivity for predicting postoperative opioid use in the present study.

Limitations

Our study is not without limitations. Given the retrospective nature, we had a relatively small sample size after excluding patients who had not been assessed at the decided preoperative and postoperative time points. This, however, does not create a concern for power analysis as the issue of power analysis and sample size to refute a null hypothesis does not apply to most analyses reported in this study, and for those in which it applies, the findings show that the power was enough. The analysis for the diagnostic accuracy of the 3 instruments used is not based on hypothesis testing, and therefore, such analyses did not entail any comparisons. They just provide estimates for different indicators of diagnostic accuracy. Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, as well as values presented in Table III, provide 95% CI for each indicator. The width of the CIs allows for the assessment of precision of each estimate, which does suggest that the sample size is limited. The purpose of our study was not to compare the applicability of the 3 risk tools but rather to assess the accuracy of each tool.

Hypothesis testing which does concern power analysis may apply for the results demonstrated in Table IV, which shows the differences in the mean of the scores for each risk tool obtained between subjects who were and were not using opioids at 1 and 3 months postoperatively. The null hypotheses were tested here, in which the majority of the P values are >0.05. However, because 95% CI for the differences between means observed is not wide, the statistical power for such comparisons was acceptable.

Other limiting factors within our cohort was that an even smaller number of patients had completed all 3 risks assessment tools especially the SOAPP-R, likely attributable to its length relative to the other screening tools. Further research should assess the predictability of the SOAPP-R in a larger population of upper extremity surgery patients or in specific populations such as rotator cuff repair patients. Additionally, as noted, a relatively small percentage of the patients in the present study were reported to have been using preoperative opioids when compared to available literature, which may reflect specific clinical recommendations or practice differences of the 2 surgeons included. The broad use of these screening tools with minimal evidence of predictability for postoperative opioid dependence after surgery highlights the need for further studies in larger populations.

It should also be noted that the PROMIS assessments include several item banks of different purposes including the PROMIS-Rx Misuse tool which is intended to measure misuse of prescription pain medication. In the present retrospective study, we only had data available for the PROMIS pain intensity score, a different item bank designed to quantify pain which we assessed for its ability to predict postoperative opioid use. We still believed that this risk tool required scrutiny for validity in our population, as the purpose of PROMIS is to provide clinicians with important patient-reported information to understand how various treatments may affect what patients do (consume opioids) with the symptoms (pain) that they experience.

Since our study design include various types of shoulder surgery, specificity of our analysis may be variable where literature reports that some procedures or diagnoses are more painful than others. Therefore, patients may demand more narcotics depending on pathology and surgery type., Although our patient cohort did receive preoperative, intraoperative, and postoperative patient education along with postoperative physical therapy, this was not directly assessed as a variable and may have directly influenced analysis of the risk tools. There have been many studies that have validated the role of patient education, preoperative, and postoperative physical therapy in reducing postoperative opioid consumption. Among other studies Sabesan et al found an 85% reduction in the number of patients who used opioids in the 48-hour period after surgery. In addition, the study showed that among patients undergoing shoulder arthroplasty 100% of those who received preoperative, intraoperative, and postoperative patient education in addition to multimodal pain management were opioid free by 2 weeks., Brown-Taylor et al reported that 8 studies concluded a relationship between early PT and reduced subsequent opioid use in orthopedic procedures.,, Therefore, these variables may be valuable areas to examine in future studies. Future analysis may also investigate these tools on a more nuanced level to stratify risk assessments by surgery type to see if risk tools may be more valuable for specific shoulder procedures as opposed to others.

Conclusion

None of the 3 common screening tools assessed in the present study, the ORT-O, PROMIS pain intensity, and SOAPP-R, were found to be optimal for predicting postoperative opioid dependence in shoulder surgery patients. However, in a small subset of patients, the SOAPP-R was moderately correlated to postoperative TME and significantly different between patients who did and did not use postoperative opioids. Further studies are needed to validate opioid risk screening tools in upper extremity surgery patients and other populations to effectively guide postoperative prescription practices. Having such a tool would enable surgeons to make advanced referrals to pain management specialists, inform prescribing practices, create opportunities for focused patient education and customized pain protocols, and ultimately aid in reducing chronic postoperative opioid use.

Disclaimers

Funding: No funding was disclosed by the authors.

Conflicts of interest: The authors, their immediate families, and any research foundation with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.