Cross-sectional analysis of online patient reviews of infertility care providers.

OBJECTIVE: To observe the effects of practice type, location, and mandated insurance coverage on infertility physician online reviews by patients.
DESIGN: Retrospective cohort study.
SETTING: Not applicable. PATIENTS: Patient online reviews of fertility specialists from 2016 to 2019. INTERVENTIONSS: None. MAIN OUTCOME MEASURES: The analysis consisted of the average rating out of 5 for each physician published on Vitals, RateMD, and Healthgrades.
RESULTS: Data were collected on 1,097 specialists. Physicians practicing in states with versus without mandated insurance coverage received an average rating of 4.093 versus 4.076, respectively. The average rating was 3.964 for physicians affiliated with a university or hospital versus 4.128 for those working in a private practice. Significant differences were found in physician ratings from the four regions. It was revealed that physicians who practiced in the South (n = 354) received significantly higher mean average ratings than those in the Northeast (n = 327) and Midwest (n = 175). Physicians practicing in the West (n = 241) received significantly higher ratings than those in the Midwest (n = 175).
CONCLUSIONS: The average online patient rating of infertility specialists was found to be significantly higher for physicians working in a private practice compared with those affiliated with a university or hospital system. No significant difference was found between the average rating in states with versus without mandated insurance coverage for infertility treatment. We propose that qualities other than patient financial responsibility are implicated in the factors used to rate physicians.

Discuss: You can discuss this article with its authors and other readers at

In 2010, 88% of Americans used the internet to search for information related to health, including diseases, medications, hospitals, and physicians (1). Although 65% of patients were aware of physician rating websites, only 23% used this resource (2). Physician ratings revealed a patient’s experience from their clinical encounter. Today, many businesses survey their customers to improve the quality of clinical service. Consequently, in recent years, health professionals have seen a drastic increase in the use and marketing of rating sites as they offer forums for peer-to-peer communication regarding basic physician information such as location and hours and subjective insights such as overall patient experience and physician interaction.

Rating websites are structurally similar to social networking sites, travel websites, and restaurant websites (3). They allow patients to make a more informed choice regarding their providers and practices from a consumer perspective as opposed to an organized perspective (4). In a study of 1,000 Amazon Mechanical Turk users, it was found that individuals perceive commercial website ratings equally comparable to ratings provided by government websites when choosing a primary care physician (5). In a survey of patients in Germany, 65% of respondents had chosen their physician because of positive online reviews and 52% decided against seeing a physician based on negative reviews (12).

From 2005 to 2010, on the patient review site RateMD.com, the number of physicians with five or more ratings rose from <1% to 12.5%. Although the field of Obstetrics and Gynecology (Ob/Gyn) was the most likely specialty to be rated, only 32% of all Ob/Gyn physicians were reviewed (6). Physicians specializing in areas with less patient interaction (radiology, pathology, anesthesiology) have been found to have fewer ratings (7). Rating websites are also very prevalent in many other specialties. For example, in a study in the field of urology, the majority (79%) of 500 randomly selected urologists were found to have at least one rating on ten different rating sites (8).

In the field of reproductive endocrinology and infertility (REI), physician online presence plays a dual role of advertising not only for their services offered but also for success rates as reported to the Society for Assisted Reproductive Technology (SART) or the Centers for Disease Control and Prevention (CDC) (6). Patient online reviews are equally as important to REI physicians to help maximize patient-centered care, treatment compliance and patient well-being, all helping to minimize patient anxiety and depression during an already stressful period (9).

Minimal data exist to correlate online patient reviews with insurance coverage. Patient expense has been shown to be a contributing factor to reviews in a study of primary care physicians. No significant difference has been shown when comparing region of the country to online reviews (10).

The objective of the present study was to determine the impact, if any, on physician online ratings based on practice type (private vs. university or hospital based), practice geographic location, and infertility insurance coverage.

Materials and Methods

Study Summary

This was a retrospective cohort study analyzing patient online reviews of infertility specialists across the United States obtained from three popular rating websites: Vitals (https://www.vitals.com), RateMD (https://www.ratemds.com), and Healthgrades (https://www.healthgrades.com). The results consist of the average rating of infertility physicians based on the geographic location of their clinic, whether the state in which the physician practices has mandated versus nonmandated infertility insurance, and the type of clinic practice setting, i.e., university or hospital based vs. private practice. The clinic practice designation was determined based on location, i.e., freestanding private clinic or on a hospital or university campus.

The ratings for each physician were averaged to a score based on a 5-point scale. Physician ratings from the three websites were collected and a weighted mean formula was used in Excel to obtain the final composite rating. With the use of this formula, the rated physicians were weighted according to the number of ratings each had received. Physicians were omitted if they did not receive any ratings on all three websites.

The time period of analysis was limited to January 2016 to April 2019. Exclusion criteria included not being registered on the SART or CDC public database across the three websites within the allotted time frame. A total of 185 physicians were excluded owing to having no reviews on all three websites. Of those, 50 were affiliated with hospitals or universities and 135 were in private practice. There was no prevalent region where a physician had no reviews.

Sample Size

The names of 1,282 physicians were initially derived from clinics listed on the SART and CDC websites. After excluding the 185 with no reviews, there was a final sample size of 1,097 physicians across the United States analyzed from January 2016 to April 2019. The average rating for all physicians included in this study was 4.09 ± 0.93. Average ratings per state are shown in Figure 1.

Figure 1

Average ratings, on a 5-point scale, of all physicians within each state. The lower limit of the scale is 2.5 because there was no average rating below this number.

Collection of Physicians

The physicians’ names and clinic locations were obtained through the SART website and compared with the CDC’s reporting clinics of assisted reproduction to verify fertility specialists and clinics. The type of clinic (private vs. hospital) was collected from the latter database and, if unavailable, the information was determined from further online analysis.

Collection of Ratings

The ratings were collected from three popular physician rating websites: Vitals, RateMD, and Healthgrades. Physicians were divided based on region and practice setting. A state was determined as having mandated infertility insurance if it had passed laws that require insurers to either cover or offer coverage for infertility diagnosis and treatment as of 2 years before initiation of the analysis (i.e., by January 2014). A subanalysis was performed for states that mandated insurance providers to offer infertility coverage but not requiring it. Clinic location was grouped by the four regions of the United States: West (Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, Wyoming, Alaska, California, Hawaii, Oregon, and Washington), Midwest (Illinois, Indiana, Michigan, Ohio, Wisconsin, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota), South (Delaware, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, District of Columbia, West Virginia, Alabama, Kentucky, Mississippi, Tennessee, Arkansas, Louisiana, Oklahoma, and Texas), and Northeast (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont, New Jersey, New York, and Pennsylvania).

Statistical Analysis

Independent-sample t tests for equality of means were used to determine differences in means. An analysis of variance (ANOVA) and a Tukey honestly significant difference test were used to determine differences in means between the geographic regions. Institutional Review Board (IRB) exemption was granted by Osceola Regional Medical Center IRB and the University of Central Florida IRB in Orlando, Florida.

Results

Website Ratings and Reviews

The Vitals website (n = 952) was more popular than the RateMD (n = 574) and Healthgrades (n = 786) websites (P<.001 vs. RateMD; P<.001 vs. Healthgrades). The average rating among all three websites was similar with no significant difference, indicating that website did not affect ratings: Vitals 4.08, RateMD 4.09, and Healthgrades 4.11.

Insurance Mandated Coverage

Of the 1,097 physicians, 657 practiced in mandated states and received a mean rating of 4.09 ± 0.93, and 440 practiced in nonmandated states and received a mean rating of 4.07 ± 0.94 (P=.762). The mean number of reviews per physician was 16.5. In a subanalysis of the two states, California and Texas, that only required offering infertility insurance, a lower average rating was given compared with mandated states (P<.05).

Practice Type

Ratings were not significantly different between physicians in private practice (4.13 ± 0.91; n = 817) and those who were university or hospital affiliated (3.96 ± 0.97; n = 280; P=.011).

Geography

A one-way ANOVA revealed significant differences in physician ratings among regions (P<.001). Physicians who practiced in the South (4.22 ± 0.85; n = 354) scored significantly higher mean average ratings (P<.01) than those in the Northeast (3.99 ± 0.93; n = 327) and Midwest (3.91 ± 1.01; n = 175). Physicians practicing in the West (4.14 ± 0.95; n = 241) experienced significantly higher ratings (P<.05) than those practicing in the Midwest (3.91 ± 1.01; n = 175).

Discussion

To our knowledge, this is the first survey of infertility specialist ratings controlled for practice type, geographic location, and insurance coverage. Geographically, patient ratings from the South and West were significantly higher than from those in the Midwest. The average rating for all physicians across the United States was high, at 4.09 out of 5, with no significant difference among states with or without mandated insurance coverage. There was a significantly higher rating for physicians in private practice versus university or hospital based: 4.13 versus 3.96, respectively. An intriguing outcome occurred when a subanalysis revealed a statistically lower rating in the two states that required only offering infertility insurance rather than mandating. This appears to contradict the lack of significance seen when comparing ratings from mandated versus nonmandated states. A mandate “to offer” requires that health insurance companies make available for purchase a policy which offers coverage of infertility treatment. Although two states are a small sample to provide a compelling trend, patients may feel more frustrated knowing their employer had the opportunity to provide infertility coverage and did not.

The volume of fertility specialists with reviews on multiple websites from January 2016 to April 2019 is a display of how commonplace online reviews have become. Compared with a 2013 study in which 17% of physicians in all specialties had at least one review (13), our study showed that 86% of registered fertility specialists had at least ten reviews across three websites. This is indicative of the increase in overall physician ratings and specifically in the field of infertility. In addition, this high number of reviews is consistent with previous research showing that physicians in the subspecialties of Ob/Gyn have a high number of ratings (6).

The average rating of each specialist across the United States was consistent with previous research showing that online physician ratings are improving and that, typically, online patient reviews are favorable and not decreased substantially by displeased patients (6). There are several theories that may explain why physician ratings were higher for private practice compared with hospital- or university-based practice. The most likely explanation could be continuity of care. In a private practice, the patient is more likely to see the same physician for the entirety of their treatment and, potentially, build a better rapport, unlike in a university- or hospital-based practice, where the patient may see several physicians, including residents and fellows, diluting the attending physician-patient relationship.

In addition, private practice fertility clinics are standalone sites with nonhospital “boutique” amenities that may affect the patient’s overall impression of their fertility care. University- or hospital-based physicians are often faced with the difficulty of multiple different demands for their time, e.g., education, research, administrative meetings, which affect the amount of time spent with patients (9).

The publication of online reviews, both numeric and narrative, can be a point of contention for physicians and their wellness. In a survey of 828 physicians, 75% believed that posting narrative comments from online ratings would increase their job stress, and 46% considered that the posting of comments would have a negative effect on the patient-physician relationship. However, 51% of patients supported the publication of narrative comments, likely owing to the perceived ability to make an informed choice of physician (11).

Online reviews may not always reflect successful outcomes. In a study observing the online patient reviews of orthopedic surgeons on popular physician rating websites, it was found that the mean overall ratings per physician were not associated with the surgeon-specific outcomes (12), implying that confounding factors play a role in how the patient rates the physician. The rating may reflect inflation due to the patient feeling grateful for an experience and may not be based whatsoever on the physician’s performance.

There are several limitations in our analysis. We initially, erroneously, hypothesized that physician reviews would be higher in areas with health insurance mandates for fertility coverage. Our results suggest that patient reviews may be influenced by other qualities or circumstances, such as physicians’ personality, medical capabilities, staff availability, and clinic amenities, as opposed to the financial burden of patients. Education and background can influence where a physician practices and, therefore, their patient population. The type of patient population that the physician serves is correlated with the likelihood a patient will write a positive review (11). Physician experience, board certification, and educational background were not included in the quantitative analysis of online ratings and may be a limitation.

Because online patient reviews are subjective, credibility and accuracy cannot be determined. One cannot confirm that the review is written by a verified patient versus a competing physician or the original physician trying to improve their personal ratings. It was not possible to account for practices that encourage patients with better outcomes and better physician-patient relationships to write online reviews.

For these reasons, the American Medical Association recommends against the development of physician-rating websites. In contrast, the National Health Service in the United Kingdom encourages patients to review physicians and hospitals (3).

Further investigation is warranted to determine the impact on reviews, if any, pertaining to group versus solo practice, physician compensation structure, i.e., salary versus productivity, and volume of patients per physician. One could also provide surveys to current patients to investigate which are the most positive and negative parts of their infertility journey.

Although the quality of care received by the patient is subjective, the ratings give an insight into their personal experience and overall satisfaction with their visit. This can aid the decision making of future patients and in addition help physicians to improve their practices, patient interactions, or clinic amenities to appeal to a larger patient demographic (11).

In conclusion, the average online patient rating of infertility specialists was found to be significantly higher for physicians working in the South and West as well as for those in a private practice compared with those in a university or hospital system. No significant difference was found between the average ratings in states with mandated insurance coverage for fertility treatment compared with those without coverage. Qualities other than patient financial responsibility may influence how patients rate their physicians.