Characterizing negative reviews of orthopedic spine surgeons and practices.

Background: Recent evidence suggests that patients prefer subjective and crowd-sourced information over data-driven or quality-based outcomes when choosing a surgeon. Online physician rating and review websites continue to increase in popularity, and over half of patients use them to research physicians. Specifically, Yelp.com is the most frequently utilized online resource by patients. Data regarding the characteristics of negative reviews for spine surgeons and practices is lacking.
Methods: Orthopedic Spine surgeons and practices in 8 major US metropolitan regions were surveyed for one-star reviews on Yelp.com. The factors noted in the reviews were recorded and they were classified according to their clinical or nonclinical nature. Reviews were also subclassified into nonsurgical or surgical episodes of care.
Results: A total of 6,286 Yelp reviews were discovered, 671 (10.6%) of which were rated one-star. The majority of negative reviews (76.4%) were from patients who did not report surgery by the surgeon or practice. Of all comments, 491 (77.6%) related to nonclinical complaints. The most common factors noted in negative reviews were related to bedside manner, rude or unprofessional staff, and wait time.
Conclusion: Choosing a surgeon is a complex process for patients. The large majority of negative reviews were related to nonclinical issues such as poor bedside manner or rude staff and most of these were written by patients that did not undergo a surgical procedure. This may explain the large discrepancy that has been observed between quality metrics and online crowd-sourced reviews. Paying attention to these nonclinical factors may represent the most feasible and valuable targets to improve a surgeon's practice and attract future patients.

Introduction

Low back pain is the leading cause of disability worldwide [1]. It is estimated that 1-2% of the population is disabled because of low back pain and up to 80% of the population will be affected by it [2]. Pain originating from the cervical or thoracic regions can also be disabling and present significant burdens to the individual [3], [4], [5]. Not surprisingly, much attention has been focused on the appropriate management of this pain.

Although surgical intervention is not a first-line treatment, it remains an option after failure of conservative measures such as anti-inflammatory medications, activity modification, physical therapy, and steroid injections. The role of surgical management in these patients continues to significantly increase in the United States. The rate of spinal fusions alone have showed a 137% increase since 1998, a rate that surpasses that of other common orthopedic procedures such as hip and knee arthroplasty [6,7] Other spinal procedures, including discectomies and laminectomies, have increased during this period as well [8]. Total health expenditures for all adults with chronic back pain have been estimated to be as high as $102 billion annually [9].

Considering the rising cost associated with spine surgery as well as healthcare delivery as a whole, it is no surprise that there has been a heightened interest in maximizing the value of health care in recent years. Although provider quality data had been publicly available for years, the Affordable Care Act prompted more transparency of previously confidential Medicare utilization and payment data. In doing so, it has increased the transparency of provider comparison and patient-centered outcomes. This was followed in 2014 by the Centers for Medicare and Medicaid Services (CMS) release of detailed utilization information for over $77 billion in payments from over 880,000 physicians [10]. As part of this collaborative initiative, the Consensus Core Set of Orthopedic Measures was released in 2016 in order to simplify the reporting of value-based payment and inform decision-making by patients [11]. This release has caused an increased reliance by payers and patients on online reviews and ratings related to comparing providers.

In addition to the quantifiable disease specific and data-driven outcomes available to compare providers, there is now also an additional focus on the subjective patient experience. The Consensus Core Set of Orthopedic Measures includes “patient experience” as one of the core 7 domains. This is also demonstrated by the Hospital Consumer Assessment of Healthcare Providers and Systems measurement of patient perceptions of their hospital experience [12]. In addition to these formal methods, there are many public and private websites that serve as resources to document patient perceptions and opinions of the their care. Specifically, it has been demonstrated that Yelp.com is the most frequently used online resource for patients to evaluate providers [13].

Online physician ratings and reviews continue to increase in number, and up to 60% of patients utilize them in choosing a physician [14,15]. There have been several previous reports characterizing online reviews of orthopedic surgeons. One such study demonstrated that although the majority of reviews were positive, over 30% were either negative or extremely negative [16]. Several other studies have also characterized the reviews or evaluated the factors associated with positive reviews [17], [18], [19]. To the authors’ knowledge, there has not been an investigation focused on spine surgeon reviews and only one study analyzing very negative reviews, which was limited to arthroplasty surgeons [20]. The purpose of this study was to characterize negative reviews for spine surgeons and practices.

Materials and methods

As all data utilized in the study was publicly available, the study was deemed IRB exempt. A search was performed using the keywords “orthopedic spine surgery” on yelp.com for 8 major metropolitan areas including New York, Boston, San Francisco, Los Angeles, Dallas, Phoenix, Seattle, and Washington DC. Although there exist several online physician review websites to review surgeons, Yelp was chosen as it has been reported as the most frequently visited by patients [13]. The practice or surgeon named in the review was then confirmed to be an accredited orthopedic surgeon that performs spine surgery through a secondary internet search using physician websites, hospital group websites, Angie's List, Doximity, and vitals.com. Reviews that did not name a particular surgeon of a practice that performs multiple specialties were excluded in order to focus particularly on confirmed orthopedic spine surgery reviews. All reviews other than single-star (out of a possible 5 stars) were excluded from the study (Fig. 1).

Fig. 1

Online review selection.

Flowchart analysis of orthopedic spine surgery reviews analyzed from yelp.com.

One-star reviews were classified as directed toward a physician, practice, or both. Reviews were categorized as clinical if comments included a reference to treatment complications, reoperation, pain, perceived misdiagnosis, readmissions, delay in communication, unclear treatment plan, or delay in care. Reviews that acknowledged bedside manner, professionalism, wait time, cost issues, limited time with provider, insurance issues, facilities, and scheduling were categorized as nonclinical. Reviews were considered surgical or nonsurgical on the basis of whether or not a surgical episode of care was explicitly referenced. Comments referencing hospitalization were not considered surgical unless surgery was directly referenced. Reviews that were unable to be categorized per the above criteria were excluded. Reviews were initially classified and categorized as mentioned above by two authors separately (JS and KL). Inter-rater reliability was calculated (83.3% agreement and Kohens Kappa statistic of 0.667) and a third author (JP) resolved conflict when classifications and categories did not coincide between authors JS and KL.

Univariate analysis was performed to determine means and 95% confidence intervals. Continuous variables were compared using the student's t-test. Categorical variables were analyzed using the chi-square test with alpha set to 0.05. The rate ratio (the ratio of the rate for nonsurgical divided by surgical reviews) was determined for each category. Statistical analysis was performed using a commercially available software package (JMP Pro v.13; SAS Institute, INC.).

Results

A total of 6,286 Yelp reviews were discovered. Of these, 671 (10.6%) were identified as single-star reviews. These reviews were recorded on a total of 305 surgeons, averaging 2.2 negative reviews per surgeon included. A total of 374 reviews were excluded due to an inability to classify them or an unclear relation to spine surgery, and the remaining 297 reviews were included for analysis (Fig. 1). Nonsurgical patients were associated with 227 reviews and 70 were linked to surgical patients (Table 2). Of all comments within reviews, there were 491 comments relating to non-clinical complaints and 142 relating to clinical issues (Table 1).

Table 2

Clinical focus of surgical vs. non-surgical patients.

	Surgical (n = 70)	Non-surgical (n = 227)	p-value
Clinical	99 (54.7%)	43 (9.5%)	<0.001
Non-clinical	82 (45.3%)	409 (90.5%)	<0.001

Table 1

Negative online review clinical and surgical characterization.

Focus of the Review	Number (percent) of complaints	Surgical Patients (n = 70)	Non-Surgical Patients (n = 227)	p-value
		Total complaints, N (%)	Total complaints, N (%)
Clinical Complaints (N = 142)
Complication	48 (16%)	45 (64%)	3 (1%)	<0.001
Readmission	4 (1%)	3 (4%)	1 (0%)	<0.001
Reoperation	9 (3%)	9 (13%)	0 (0%)	N/A
Uncontrolled pain	37 (12%)	27 (39%)	10 (4%)	<0.001
Misdiagnosis	18 (6%)	6 (9%)	12 (5%)	0.232
Unclear treatment plan	8 (3%)	1 (1%)	7 (3%)	0.430
Delay in care	13 (4%)	7 (10%)	6 (3%)	<0.001
Clinical Other	5 (2%)	1 (1%)	4 (2%)	0.833
Non-Clinical Complaints (N = 491)		99	43
Bedside Manner Doctor/Unprofessional	146 (49%)	32 (46%)	114 (50%)	0.595
Bedside Manner Midlevel/Unprofessional	21 (7%)	5 (7%)	16 (7%)	0.976
Rude/unprofessional Staff	92 (31%)	12 (17%)	80 (35%)	0.011
Wait Time	75 (25%)	9 (13%)	66 (29%)	0.012
Not enough time spent with provider	49 (16%)	4 (6%)	45 (20%)	0.008
Cost/billing/insurance	66 (22%)	18 (26%)	48 (21%)	0.406
Facilities	10 (3%)	1 (1%)	9 (4%)	0.287
Scheduling issues	30 (10%)	1 (1%)	29 (13%)	0.008
Non-clinical Other	2 (1%)	0 (0%)	2 (1%)	0.432
		82	409

Table 1. Clinical and non-clinical review comments categorized by surgical and non-surgical patients. Bold denotes statistical significance (p<0.05) rate of complaint between surgical and non-surgical patients.

Clinical factors most commonly addressed included complication (48 reviews, 16%), uncontrolled pain (37 reviews, 12%), perceived misdiagnosis (18 reviews, 6%), delay in care (13 reviews, 4%), reoperation (9 reviews, 3%), and unclear treatment plan (8 reviews, 3%). The most common non-clinical complaints referenced bedside manner (146 reviews, 49%), rude or unprofessional misconduct (92 reviews, 31%), wait time (75 reviews, 25%), cost or insurance issues (66 reviews, 22%), limited time with provider (49 reviews, 16%), and scheduling issues (30 reviews, 10%).

A greater number of reviews were derived from patients that did not explicitly cite a surgical episode (227) when compared to those that reported surgery (70). When compared, nonsurgical reviews were more likely to contain nonclinical rather than clinical complaints (p<0.001). Of these, scheduling issues (p=0.008), limited time with provider (p=0.008), rude or unprofessional misconduct (p=0.011), and wait time (p=0.012) were more common for nonsurgical versus surgical reviews. Conversely, clinical complaints were proportionately more common in patients that reported a surgical episode (p<0.001) (Table 2). When compared to non-surgical patients, surgical patients demonstrated higher rates of clinical complaints related to uncontrolled pain (p<0.001), delayed care (p<0.001), complication (p<0.001) and readmission (p<0.001).

Discussion

Spine surgeons are experiencing an increased demand for their work as well as heightened use of data-driven metrics and evaluation measures. The accessibility of quality data has generated the rise in popularity of online evaluation and review websites, and it is predicted that patient utilization of these resources will continue to rise in orthopedic surgery [14,21,22]. Accordingly, it is important to understand how patients interpret and apply this data in choosing a surgeon. The purpose of this study was to characterize the extremely negative reviews of spine surgeons and practices. Single-star reviews comprised 10.6% of all reviews analyzed. Our results demonstrated that non-clinical factors such as bedside manner, rude or unprofessional staff, wait time, and cost or insurance issues were more likely to be associated with a negative review than pain, complications, or reoperation. A minority of these patients reported a surgical episode of care.

Crowd-sourced information relating to patient satisfaction of their healthcare experience is available through several public websites. It remains unclear whether the information on these websites accurately portrays the quality of care received. A study investigating the five busiest physician-rating websites and user-generated data found that none contained the 7 Consensus Core Set of Orthopedics Measures [17]. Additionally, numerous investigations have assessed the correlation between online reviews and various outcomes including malpractice claims, mortality, infection, and readmission. Results have revealed weak, if any, correlation between these outcomes and online ratings [15,23,24]. Similar findings have been reported for nursing home care, as Yelp.com reviews and a CMS-sponsored site had poor correlation in terms of overall star ratings and quality measures [25]. In our study, we found that 77.6% (491/633) of all one-star reviews reflected nonclinical rather than clinical issues. This suggests the large majority of negative reviews are exclusive of factors that reflect healthcare quality and coincides with previous investigations that report a poor correlation between online ratings and patient outcomes.

Several studies have evaluated the psychology and behavior of online physician reviews. It has been demonstrated that patients have higher comprehension when information is presented with low cognitive burden and prefer patient comments over quality metrics [13,26]. This may explain why, when compared to other online patient resources, CMS data ranks 8 out of 13 in terms of overall patient use [13]. Patients also demonstrate a preference toward more succinct reviews, which may promote user-generated data to trend toward either extremely positive or extremely negative [26]. The rate of such extreme reviews has been shown to be as high as 64% [16].

The motivation of a dissatisfied customer to retaliate against the business along with the idea that satisfied patients are less likely to leave reviews may lead one to expect a higher proportion of strongly negative reviews [27]. However, this was not borne out in our study as one-star reviews comprised only 10.6% (671/6286) of all reviews. It is difficult to discern why this was the case in our study but may reflect an increased awareness of the impact of online reviews and efforts by physicians to combat them. Investigations have reported that some physicians directly ask for online reviews as well as hand out comment cards in order to improve their online rating [28]. Further, there now exist companies that help improve physician ratings as well as prevent damaging online rating from being placed [29,30].

The majority of patients utilize online reviews to choose a physician, and the number of online ratings continue to increase [14]. Accordingly, it is beneficial for physicians to understand the nature of online reviews and how to maintain a favorable rating. In the presented study, the most common reasons for negative reviews were related to negative interactions with healthcare providers, whether it was poor bedside manner of the physician (49%) or rude/unprofessional staff (31%) (Table 1). Logistical issues such as scheduling issues (10%), insufficient time with provider (16%), and cost or insurance problems (22%) were less common (Table 1).

Non-surgical patients were found to be responsible for the vast majority of negative reviews and were also more likely to report non-clinical factors. This is consistent with the high degree of nonclinical factors cited in negative reviews for dermatologists and arthroplasty surgeons [20,31]. Contrary to what a surgeon commonly defines as successful practice (e.g., outcomes, quality), the public considers friendly demeanor and amicable staff interactions as most important in choosing a physician. Paying attention to these factors may represent the most feasible and valuable targets to improve a surgeon's clinical practice, maintain a favorable online presence, and attract future patients.

This study has several limitations. Our search was limited to 8 metropolitan areas which may limit the applicability of our findings, especially to smaller or more rural areas. Although we used multiple reviewers to mitigate error, it is possible that reviews were erroneously categorized. Another limitation is that only Yelp.com reviews were considered, which may not reflect all online review or rating sites.

Choosing a surgeon is a complex process for patients. To the authors’ knowledge, this is the first investigation to focus on negative reviews for spine surgeons. We found that the large majority of negative reviews were related to nonclinical issues such as poor bedside manner or rude staff and that most of these were written by patients that did not undergo a surgical procedure. These nonsurgical patients were more likely to complain about wait times, scheduling issues, and limited time with the provider when compared to surgical patients. Patients that underwent surgery were also more likely to complain about pain, delayed care, and readmission.

Source of Funding

None.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.