Predicting the need for surgical intervention prior to first encounter for individuals with shoulder complaints: a unique approach.

BACKGROUND: Increasing demand for musculoskeletal care necessitates efficient scheduling and matching of patients with the appropriate provider. However, up to 47% to 60% of orthopedic visits are made without formal triage. The purpose of this study was to develop a method to identify, prior to the initial office visit, the probability that a patient with shoulder symptoms will need surgery so that he or she can be appropriately matched with an operative or nonoperative provider. We hypothesized that patients who had an injury, previously saw an orthopedic provider, or previously underwent magnetic resonance imaging on the affected shoulder would be more likely to undergo surgery.
METHODS: Drawing from expert opinion on potential risk factors (which could be identified prior to the initial office visit) for requiring operative intervention for a chief complaint of shoulder symptoms, we developed a branching-logic questionnaire that required a maximum of 7 responses from the patient during the scheduling process. We administered the questionnaire to patients calling with a chief complaint of shoulder symptoms at the time of initial appointment scheduling in a sports health network. A chart review was later completed to determine the ultimate treatment (operative vs. nonoperative) of each patient's complaint. A multivariate predictive model was then developed to determine the characteristics of patients with a higher surgical risk.
RESULTS: We successfully developed a model capable of determining surgical risk from 7% to 90% based on patient sex, previous magnetic resonance imaging status, and injury status.
CONCLUSIONS: Our predictive model can aid in patient clinical scheduling and ensure optimal matching of a patient with the best provider for the patient's care. Decreased wait times and appropriate matching may lead to increased patient satisfaction, superior outcomes, and more efficient use of health care resources.

Musculoskeletal disorders are major drivers of disability globally, and they lead millions of Americans to engage the health care system every year. From 2005 to 2010, a musculoskeletal complaint was the primary reason that patients sought health care in over 57% of cases. Because of the large volume of these complaints, it is unsurprising that orthopedic visits ranked fifth among all types of visits with almost 50 million patient encounters in 2012. Within non-spine orthopedic surgery visits, the second most common reason patients sought care from an orthopedic surgeon was “shoulder symptoms,” accounting for over 11 million primary visits to physicians' offices in 2010.,

In addition to being common reasons for clinic visits, musculoskeletal problems are common reasons for surgical intervention. In 2012, surgical procedures related to orthopedics (excluding spine operations) accounted for 17.8% of all ambulatory surgical procedures, and procedures involving muscles and/or tendons and procedures involving joints were the second and third most common outpatient procedures, respectively, performed in community hospitals. Shoulder operations in particular are becoming more common, with the volume of rotator cuff repair procedures increasing by 115% from 1996 to 2006 and the volume of shoulder arthroplasties increasing by 250% from 2000 to 2008.

Given the demand for the clinical and operative services of orthopedic surgeons, both in general and for shoulder complaints specifically, the health care system should function to optimize the efficiency of treatment. Despite increasing surgical volumes and interventions, a majority of the patients seen for musculoskeletal problems are treated nonoperatively. Fellowship-trained primary care sports medicine providers are well positioned to manage the nonoperative care of many musculoskeletal problems while simultaneously determining the threshold for surgeon referral and consultation. In 2012, however, 47% to 60% of the 50 million orthopedic office visits occurred without a referral, which means that many patients are left to make the determination of whether their condition best matches the services provided by an operative or nonoperative provider.

The lack of a formal method of triage often leads to the patient seeing a provider who does not subspecialize in the area of the patient's chief complaint. Even more significantly, the current system lacks any method to pair individuals who are likely to require surgery with surgeons and to pair individuals with a low likelihood of needing surgery with nonoperative physicians. Thus, individuals with a high likelihood of needing surgery are just as likely to see a nonoperative provider as those with a very low likelihood of needing surgery. This leads to substantial inefficiencies, in which providers are not seeing the optimal populations for their practice. In addition, it can contribute to less-than-optimal patient care, as a patient who requires surgery may be unable to access a surgeon on account of the surgeon's schedule being filled with patients who do not require surgery, leading to increased cost and longer wait times that may result in decreased outcomes.

Therefore, in this study, we developed and administered a short branching-logic questionnaire to patients with shoulder symptoms during their initial scheduling encounter and then developed a multivariate logistic regression model allowing us to predict a patient's likelihood of receiving a recommendation for surgery. Our hypothesis was that patient responses (specifically having incurred an injury, having previously seen an orthopedic provider, and having previously undergone magnetic resonance imaging [MRI] of the affected shoulder) to the questionnaire would be predictive of the recommendation for future surgical intervention.

Methods

Questionnaire

A branching-logic questionnaire was developed based on expert opinion by two of the authors (J.E.K. and K.P.S.) (Fig. 1). The questionnaire was prospectively administered to all patients calling to request an appointment with the Cleveland Clinic Sports Health network with a primary complaint of shoulder symptoms from January 2015 to June 2016. The questionnaire was administered by the appointment scheduling staff, with a maximum number of 7 questions asked per patient. Regardless of questionnaire responses, patients were scheduled to see either a surgeon or primary care sports physician according to their preference unless their responses prompted a referral to the emergency department (ED) (Fig. 2). Providers remained blinded to the questionnaire for the entirety of patient care.

Figure 1

Branching-logic questionnaire administered to patients with shoulder complaints requesting an appointment. ER, emergency room; eval, evaluation; MRI, magnetic resonance imaging; Ortho, orthopedic.

Figure 2

Exclusion flowchart.

The questionnaire consisted of scripted binary questions (with the exception of chief complaint) following a tree-like branching logic (Fig. 1). During each patient's scheduling call, he or she followed a complete “answer path” as a result of his or her specific responses to the questions encountered. The questionnaire has 22 possible complete answer paths of variable question length. In addition, 20 truncated answer paths exist, which include only the first element or more consecutive elements in an answer path but not the complete answer path. For example, a patient could have answered “Pain—No—No—Yes—No—Yes” as a complete answer path but also be considered to have followed the truncated answer paths of “Pain—No,” “Pain—No—No,” “Pain—No—No—Yes,” and so forth. In total, there are 42 possible complete and truncated answer paths that could be considered for modeling and analysis. Although more complete answer paths offer more specific information about a given record, some truncated and complete answer paths were so rare that they were not generalizable outside of the given sample. Because of the small sample size, any truncated or complete answer path with fewer than 10 corresponding records was excluded from analysis; 23 answer paths were available for analysis and as candidates for inclusion in a predictive model.

Chart review

The questionnaire was administered a total of 1986 times between January 2015 and June 2016. One questionnaire was incomplete and had to be excluded, leaving 1985 records available for analysis (Fig. 2 shows the STROBE [Strengthening the Reporting of Observational Studies in Epidemiology] diagram). Of these, 1239 were randomly selected for manual chart review by 3 authors (S.G., W.A.C., and J.A.M.). This number was chosen because it would provide an ample sample size for modeling calculations while maximizing time efficiency for chart review. Patients must have been aged between 13 and 75 years at the time of the initial visit to be included for analysis. Any patients with initial visits on or after April 1, 2016, were excluded to allow at least 3 months between the initial visit and chart review for workup and treatment. In addition, patients were excluded if treatment had yet to be determined by the provider at the time of chart review, if the presenting problem was not shoulder related, if the patient was referred to the ED per the questionnaire, or if the questionnaire had been administered in a previous patient call. Among the 1239 records reviewed, 760 patients met the inclusion and exclusion criteria (Fig. 2). Relevant data extracted from each chart included laterality of shoulder complaint, whether the patient was referred by another physician, previous MRI orders from referring physicians, diagnoses, and recommended or completed intervention. Intervention outcome was split into surgical and nonsurgical based on provider-recommended treatment. In addition, the physicians who developed the algorithm were not included in the group of physicians who made decisions regarding the treatment of the patients.

Statistical analysis

Logistic regression models were built from all candidate predictors known at the time of scheduling (answer paths, patient age, patient sex, and prior surgery indicated) and additional variables extracted from chart review that could be added to the scheduling process in the future (“referred by other physician,” “MRI ordered by other provider,” “shoulder laterality,” and “bilateral involvement”). Automated model selection was used to suggest 4 candidate models. These models were selected by all 4 combinations of selection logic (stepwise and backward) and selection criteria (Akaika and Bayesian). After the models were fit, the ability of the models to discriminate between patients who were recommended to undergo surgery and those who were not recommended to undergo surgery was measured using a concordance index. Bootstrap resampling was performed to bias correct this estimate. Similar methods were used to evaluate the calibration of the model (agreement between predicted and actual risk). From these 4 candidate models, the final model was selected based on clinical relevance, simplicity, and predictive power. Analyses were performed using R software (version 3.1; R Foundation for Statistical Computing, Vienna, Austria).

Results

Table I shows the comparisons of patients who were included in the final data set vs. those who were excluded. The 2 groups were similar in all respects with the exception of the distribution of their primary complaints (P = .038). Patients expressing “looseness” and “weakness” as their primary complaint were somewhat more represented in the excluded data set, whereas patients expressing “stiffness” were somewhat more represented in the final data set.

Table I

Comparison of patients included and excluded from final data set

Factor	Overall (N = 1985)		Excluded (n = 1225)		Included (n = 760)		P value
	n	Statistic	n	Statistic	n	Statistic
Age	1985	39.4 ± 19.5 yr	1225	39.3 ± 20.2 yr	760	39.6 ± 18.3 yr	.73∗
Sex	1947		1191		756		.78†
Male		1283 (65.9)		782 (65.7)		501 (66.3)
Female		664 (34.1)		409 (34.3)		255 (33.7)
MRI ordered by other provider	1075		316		759		.61†
No		924 (86.0)		269 (85.1)		655 (86.3)
Yes		151 (14.0)		47 (14.9)		104 (13.7)
Indicated injury	1985		1225		760		.10†
No		1078 (54.3)		645 (52.7)		433 (57.0)
Yes		721 (36.3)		467 (38.1)		254 (33.4)
Unknown		186 (9.4)		113 (9.2)		73 (9.6)
Primary complaint	1985		1225		760		.038†,‡
Pain		1724 (86.9)		1065 (86.9)		659 (86.7)
Stiffness		154 (7.8)		84 (6.9)		70 (9.2)
Looseness		67 (3.4)		45 (3.7)		22 (2.9)
Weakness		40 (2.0)		31 (2.5)		9 (1.2)

MRI, magnetic resonance imaging.

Statistics are presented as mean ± standard deviation or number (column percentage).

Analysis of variance.

Pearson χ2 test.

Statistically significant.

After application of the inclusion and exclusion criteria, patients who underwent or were advised to undergo surgery were compared with those who were prescribed a nonsurgical treatment plan by the demographic and descriptive variables presented in Table II. Compared with those who were not advised to undergo surgery, patients who underwent or were advised to undergo surgery tended to be male patients, have previous MRI ordered, and indicate they had an injury among other characteristics.

Table II

Comparison of patient characteristics, questionnaire responses, and surgical risk

Factor	Overall (n = 760)		No surgery indicated (n = 611)		Surgery indicated (n = 149)		P value
	n	Statistic	n	Statistic	n	Statistic
Age	760	39.6 ± 18.3 yr	611	39.6 ± 18.4 yr	149	39.4 ± 18.1 yr	.86∗
Sex	756		608		148		.012†,‡
Male		501 (66.3)		390 (64.1)		111 (75.0)
Female		255 (33.7)		218 (35.9)		37 (25.0)
Prior surgery indicated	760		611		149		.20†
No		707 (93.0)		572 (93.6)		135 (90.6)
Yes		53 (7.0)		39 (6.4)		14 (9.4)
Referred by other physician	759		611		148		.78†
No		418 (55.1)		335 (54.8)		83 (56.1)
Yes		341 (44.9)		276 (45.2)		65 (43.9)
Shoulder involved	760		611		149		.38†
Left		281 (37.0)		229 (37.5)		52 (34.9)
Right		422 (55.5)		333 (54.5)		89 (59.7)
Both		57 (7.5)		49 (8.0)		8 (5.4)
Bilateral involvement	760		611		149		.27†
No		703 (92.5)		562 (92.0)		141 (94.6)
Yes		57 (7.5)		49 (8.0)		8 (5.4)
MRI ordered by other provider	759		610		149		<.001†,‡
No		655 (86.3)		555 (91.0)		100 (67.1)
Yes		104 (13.7)		55 (9.0)		49 (32.9)
Indicated injury	760		611		149		<.001†,‡
No		433 (57.0)		369 (60.4)		64 (43.0)
Yes		254 (33.4)		186 (30.4)		68 (45.6)
Unknown		73 (9.6)		56 (9.2)		17 (11.4)
Indicated dislocation or fracture	760		611		149		.004†,‡
No		57 (7.5)		38 (6.2)		19 (12.8)
Yes		16 (2.1)		10 (1.6)		6 (4.0)
Unknown		687 (90.4)		563 (92.1)		124 (83.2)

MRI, magnetic resonance imaging.

Statistics are presented as mean ± standard deviation or number (column percentage).

Analysis of variance.

Pearson χ2 test.

Statistically significant.

We developed a multivariate model to predict the probability a patient would require surgical intervention, as described in the “Methods” section. The model is described in Table III and has a bias-corrected concordance index of 0.688. The concordance index represents the rate at which a model correctly assigns a higher probability of needing surgery to a patient who ultimately underwent or was advised to undergo surgery compared with a patient who was prescribed a nonoperative treatment plan. A concordance index of 0.5 reflects assignment by chance. The calibration plot for the predictive model is depicted in Figure 3.

Table III

Predictive model for required surgical intervention and relevant ORs

Factor	OR	95% CI of OR	P value	Coefficient	SE
Intercept	—	—	.733	0.247	0.726
MRI ordered by other provider	4.45	2.79-7.10	<.001	1.49	0.237
Male (vs. female)	1.6	1.05-2.49	.031	0.473	0.219
Injury status
Indicated injury on weakness or instability branch	1	Ref	Ref	Ref	Ref
Indicated injury on AP “Pain—Not Crushing Pain—Injury” (excluding AP below)	0.167	0.033-0.659	.016	−1.79	0.74
Did not encounter injury question	0.129	0.0243-0.544	.008	−2.05	0.773
Indicated no injury	0.0797	0.0161-0.308	<.001	−2.53	0.729
Indicated injury on AP “Pain—Not Crushing Pain—Injury—No ED Visit—No Pain Raising Arm”	0.0603	0.011-0.264	<.001	−2.81	0.792

OR, odds ratio; CI, confidence interval; SE, standard error; MRI, magnetic resonance imaging; Ref, reference; AP, answer path; ED, emergency department.

Figure 3

Calibration plot of predictive model. CI, concordance index.

The predictive model included sex, MRI ordered by another provider (yes or no), and several items relating to the injury status of the patient. This “injury status” factor included 5 mutually exclusive levels: (1) the patient did not encounter an injury question on the questionnaire, (2) the patient directly indicated “no injury” on the questionnaire, (3) the patient indicated injury on the weakness or instability branch of the questionnaire, (4) the patient indicated injury via the “Pain (Yes)—Crushing Pain (No)—Injury (Yes)” answer pattern, or (5) the patient indicated injury via the “Pain (Yes)—Crushing Pain (No)—Injury (Yes)—ED Visit (No)—Pain Raising Arm (No)” answer pattern.

The model-adjusted odds ratios (ORs) corresponding to the variables included in the model (injury status, sex, and MRI status) are listed in Table III. The odds of requiring surgical intervention were greater in patients with a previous MRI order from another provider than in those with no previous MRI order (OR, 4.45; 95% confidence interval [CI], 2.79-7.10; P < .001). Male patients were also more likely to require surgical intervention than were female patients (OR, 1.6; 95% CI, 1.05-2.49; P = .031).

The odds of requiring surgical intervention based on injury status depended on where (and if) injury status was reported on the branching-logic questionnaire. Patients who indicated injury after reporting weakness or instability in the first level of the questionnaire were at the highest risk of surgery (OR, 1 [reference]). The odds of surgical intervention decreased in patients who indicated injury through the answer path of “Pain—Not Crushing Pain—Injury” (OR, 0.167; 95% CI, 0.033-0.0659; P = .016), and the odds of surgery further decreased if patients continued down that same answer path but also indicated that they did not visit the ED and had no pain when raising their arm (OR, 0.0603; 95% CI, 0.011-0.264; P < .001). In addition, the odds of surgical intervention were also relatively low if the patient directly indicated no injury (OR, 0.0797; 95% CI, 0.0161-0.308; P < .001) or did not encounter an injury question (OR, 0.129; 95% CI, 0.0243-0.544; P = .008).

Table IV is a tabulated version of the nomogram corresponding to the predictive model. Each risk factor that is included in the predictive algorithm is displayed with its corresponding point value for a given response. In our sample, 19.6% of patients seen (149 of 760) either underwent or were recommended to undergo surgery. Using our model, we are capable of determining surgical risk ranging from 7% to 90% based on patient variables and questionnaire response patterns used in our model. The risk of requiring a surgical intervention as calculated by the model is shown with the corresponding total point value. For example, a male patient who previously underwent MRI and who indicated a shoulder injury on the weakness or instability branch of the questionnaire would accumulate 170 points by the nomogram and would have a risk of being recommended for surgery near 90%. Similarly, we can also profile a patient with the lowest possible risk of being recommended for surgery. Such a patient would be a female patient, would not have undergone MRI ordered by another provider, and would have followed the answer path “Pain—Not Crushing Pain—Injury—No ED Visit—No Pain Raising Arm.” Such a patient would have 0 points by the nomogram and a risk of being recommended for surgery near 7%. The traditional depiction of the predictive model as a nomogram is shown in Figure 4.

Table IV

Summary of point distribution for the predictive model

Variable	Level	Points	Surgical risk
MRI ordered by other provider	Yes	53
Male (vs. female) points	Yes	17
Injury status	Yes, with weakness or instability	100
	Yes, from 3-level AP	36
	Not questioned	27
	Indicated no injury	10
	Yes, from 5-level AP	0
Total points		0	0.07
		42	0.2
		91	0.5
		141	0.8
		170	0.9

MRI, magnetic resonance imaging; AP, answer path.

The presented model has a corresponding concordance index of 0.688.

Figure 4

Nomogram of predictive model. MRI, magnetic resonance imaging; AP, answer path.

Discussion

In this study, we built a branching-logic questionnaire based on expert opinion to be administered to patients presenting with the chief complaint of shoulder symptoms at the time of initial office visit scheduling. After administering it to patients prospectively and later completing a chart review to determine what each patient's definitive treatment outcome was (surgical vs. nonoperative), we constructed a multivariate logistic regression model to facilitate prediction of a patient's likely treatment outcome based on his or her responses. The model with the best fit to the data (bias-corrected concordance index of 0.688) showed that by determining whether MRI was ordered by another provider, the sex of the patient, and the injury status of the patient, the patient's likelihood of being recommended for surgery could be predicted in a range of 7% to 90% from a population in which the incidence of being recommended for surgery was 19.6%.

The methods used in this study have been used in the literature to address multiple problems in treating shoulder pathology. Modeling has been used to examine the epidemiology of shoulder dislocations and identify populations that are at higher risk, to study the epidemiology of musculoskeletal upper-extremity ambulatory surgery, and to relate preoperative factors to the likelihood of postoperative disability. In the field of shoulder arthroplasty, preoperative patient-reported outcome measures have been shown to be predictive of postoperative outcomes. However, to our knowledge, this is the first study using these methods to identify preoperative factors that are predictive of a patient with shoulder symptoms receiving a recommendation for surgery from a sports medicine provider.

As demand for nontraumatic orthopedic services increases, there is pressure to more efficiently see and treat those patients. For example, developed nations with nationalized health care systems have investigated additional triage methods, such as initial physiotherapist triage, to act as a gatekeeper to orthopedic services to reduce initial orthopedic visit and surgical wait times. These systems, however, rely on the clinical evaluation of trained practitioners that increases patient burden and costs through additional clinical visits and imposes additional barriers to provider access. Our unique questionnaire allows optimal matching through simple questions administered over the phone at the time of appointment scheduling and determination of the likely need for surgery even before the patient sees a provider.

Accurate pre-visit information, such as the surgical risk determined by our model, aims to improve patient satisfaction by allowing for more informed patient choices while simultaneously improving the utilization of health care resources. Current systems of referral and surgical provider triage vary widely in the United States, with primary care physician referral common in health management organization settings. Outside of these formalized referral systems, many patients are left to determine which type of provider to seek based on a cursory understanding of each provider's specialties or capabilities. With the model developed through this study, we sought to further inform both patients and providers with accurate information on the surgical risk of each patient presentation. With this information, patients could make an informed decision to see an operative or nonoperative provider. Surgical and nonsurgical providers, likewise, could reserve some of their clinical appointment slots for those patients with the highest probability of needing the providers' services according to the questionnaire. Such a system would permit all patients to be able to see the provider they want, but it would also facilitate quick access to the optimal provider for those patients who are most likely to need his or her services (according to the model). The benefits of faster and more accurate patient-provider matching leads to preventing delays in care and improving patient satisfaction by decreasing wait times and visits to multiple providers. In addition, both operative and nonoperative clinicians would see a higher proportion of patients during clinical hours appropriate for the clinicians' specific skill sets and services.

Finally, our model can easily be incorporated into a patient-scheduling workflow. Schedulers can implement our questionnaire over the phone or online to determine each patient's individualized model score. This score and the associated surgical risk can be presented to both the patient and the scheduler to determine the appropriate provider who can best initially manage the patient's shoulder problem. Implementation of our model into a scheduling workflow would only add minutes to current scheduling practices at little financial cost. Future work and other possible implementations could include the development of a smartphone application that could be used by primary care providers to determine surgical risk and the optimal referral physician. Improved patient-provider matching and elimination of redundant visits to inappropriate providers would both decrease wasteful health care spending and increase patient satisfaction through more personalized and streamlined clinical care.

There are some limitations to our model and study methods. Surgical recommendations were not verified against a standard set of criteria to determine the true necessity of intervention. In the current pay-for-service model of American health care, providers may be biased toward intervention, but all surgical recommendations reviewed in this study were made by hospital-employed, salaried physicians, mitigating the risk of a direct financial conflict of interest and bias toward intervention. Another limitation is that our follow-up was exclusively based on chart review; thus, if a patient pursued surgical treatment elsewhere, this would not have been reflected in this study. Our model has not been validated outside of the Cleveland Clinic Sports Health network or outside of the northeast Ohio region, so broader geographic studies would improve the validity of the model. Future studies could also seek to refine the questionnaire based on the findings of this study and the input of additional experts to further improve its predictive ability. Despite these potential limiting factors and areas for future study, our approach to optimizing the appropriate scheduling of patients with shoulder symptoms using a branching-logic questionnaire can provide great benefits in efficiency and patient satisfaction by matching the patient with the provider best suited to treat the patient.

Conclusion

In this study, we developed a predictive surgical risk model from a branched-logic questionnaire implemented at the time of patient clinical visit scheduling. Our model could predict surgical risk ranging from 7% to 90% based on information that can be easily gleaned from yes or no questions. This predictive model can aid in patient clinical scheduling and optimize matching of patients to the appropriate provider, thereby reducing wasteful health care spending, decreasing wait times for patients to see the optimal provider, and increasing patient satisfaction.

Acknowledgments

The authors thank the Cleveland Clinic orthopaedic patients, staff, and research personnel, including William Messner. They also thank Brittany Stojsavljevic, editorial assistant at the Cleveland Clinic Foundation, for editorial management.

Disclaimer

The research reported in this publication was partially supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under award number R01AR053684 (K.P.S.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Gregory J. Strnad discloses intellectual property royalties on a patent for the nPhase commercial outcome measurement evaluation system.

Kurt P. Spindler discloses the following relevant financial conflicts of interest outside the submitted work: NIH grants R01 AR053684-10 and U34 AR066631. This author also discloses intellectual property royalties on a patent for the nPhase commercial outcome measurement evaluation system and support from Smith & Nephew Endoscopy and DonJoy Orthopaedics.

The other authors, their immediate families, and any research foundations with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.