The quality and accuracy of direct-to-consumer biologic marketing for shoulder pathology are poor.

Background: The growing role of biologic therapies as adjunct or standalone procedures in orthopedic practice has led to greater levels of direct-to-consumer biologic marketing. The present study aims to assess the quality, accuracy, and readability of online educational resources available to patients regarding biologic therapies for shoulder pathology.
Methods: Eight search terms relevant to shoulder biologic therapies (shoulder + BMAC, Bone Marrow Aspirate Concentrate, PRP, Platelet Rich Plasma, Lipogems, Adipose Tissue, Biologic therapy, and Stem cell therapy) were searched across three separate search engines. The first 25 websites of each search were recorded. Duplicate websites and those not specific to shoulder pathology were excluded. Three evaluators independently assessed quality using an author-derived scoring rubric for a total of 25 possible points and accuracy for a total of 12 possible points. The Flesch-Kincaid readability test was used to quantify reading levels. Websites were further characterized by authorship and the presence of commercial bias.
Results: Of the 600 results from the initial search, 59 met inclusion criteria. The mean quality of the websites was poor, with 7.97 ± 2.3 of 25 points (32%). The mean accuracy was low, with 8.47 ± 1.52 of 12 points (71%). The average reading level was 11.2 ± 1.93, with 32% of websites' reading at greater than 12th grade reading level. The search terms of "shoulder PRP" and "shoulder Platelet Rich Plasma" yielded the highest quality results (mean = 8.14 ± 2.63). "shoulder Lipogems" and "shoulder Adipose tissue" yielded the most accurate results (mean = 9.25 ± 0.96). "shoulder BMAC" and "shoulder bone marrow aspirate concentrate" were most difficult to read (mean = 12.54 ± 3.73). Sixty-four percent of websites were authored by physicians, hospitals, or medical groups. The accuracy of websites authored by health care professionals was significantly higher than the accuracy of those authored by other industry sources (P = .01). Fifteen percent of websites demonstrated commercial bias. Discussion: The online resources available to patients seeking information about biologic therapies for the treatment of shoulder pathologies are of very poor quality, moderately poor accuracy, and advanced readability. Providers should caution patients about the reliability of direct-to-consumer biologic marketing for shoulder pathology.
Conclusion: The information available to patients online regarding the diagnosis, evaluation, and treatment of shoulder pathology with biologic therapies is of poor quality and accuracy and difficult readability.

Biologic agents used to treat musculoskeletal pathology are postulated to have strong tissue healing potential and anti-inflammatory properties. They are increasing in use and include therapies such as platelet-rich plasma (PRP), bone marrow aspirate concentrate (BMAC) adipose-derived stromal cells, and stem cell. Their role in the treatment of common shoulder pathology, such as rotator cuff tendinopathy and glenohumeral osteoarthritis, is expanding. Despite a rapid proliferation in the availability of biologic agents, there continues to be a lack of clinical evidence and expert consensus on the indications for use and efficacy of such therapies. The widening gap between availability and clinical evidence has been driven by a strong increase in patient demand for biologic therapies secondary to increased amounts of direct-to-consumer marketing.

As patients increasingly turn to online resources to understand medical diagnoses and available treatments, it becomes imperative for providers to be cognizant of the informative value of the content available to their patients. Previous investigations on the content of websites about the diagnosis and treatment of various orthopedic conditions have doubted the value of the information provided by these resources.,,,, More recent examinations have specifically examined online resources in regard to biologic therapies in orthopedic practice. Ghodasra et al demonstrated that the low quality, low accuracy, and difficult readability of online patient resources regarding the use of PRP have the potential to greatly misinform patients. Nwachukwu et al reached a similar conclusion in the use of biologic therapies, specifically for knee pathologies, and also demonstrated that the search term used by the patient can significantly impact the quality of the online resources available to them.

The growing role of biologic therapies as adjunct or standalone procedures in orthopedic practice has led to greater levels of direct-to-consumer biologic marketing. This study aims to assess the quality, accuracy, and readability of online educational resources available to patients for biologic therapies in shoulder pathology. We hypothesize that the information available on the diagnosis, evaluation, and treatment of shoulder pathology with biologic therapies will be of poor quality and accuracy and difficult readability.

Methods

Data collection

To collect online resources, eight search terms relevant to shoulder biologic therapies (shoulder + BMAC, Bone Marrow Aspirate Concentrate, PRP, Platelet Rich Plasma, Lipogems, Adipose Tissue, Biologic therapy, and Stem cell therapy) were searched across three separate search engines (Google, Bing, and Yahoo). To create a comprehensive search, the chosen terms also accounted for colloquial abbreviations and catchphrases, such as “stem cell therapy.” This selection process yielded 24 unique searches. All searches were performed on the same day in May 2021. The first 25 results from each search were recorded. Sponsored advertisements at the top of the search results were not counted in the top 25.

Duplicate websites, peer-reviewed articles, and clinical trials were excluded. Further exclusion criteria were broken links, login requirements, information not pertaining to shoulder pathology, video content only, or information solely intended for medical professionals. Websites were characterized by the types of authorship, which included physician website, hospital/clinic/medical group website, medical device or sales industry website, professional organization website, online news article, or other. Websites were also evaluated for the presence of commercial bias, defined as containing advertisements for for-profit products or services.

Online resource assessment

Three evaluators independently assessed each website; the evaluators were trained by the senior author to independently evaluate and grade websites. First, each evaluator studied two peer-reviewed articles regarding biologic therapies for shoulder pathology., Then, the evaluators were independently tested by the senior author on each of the 25 quality criteria developed from the guidelines set by the American Academy of Orthopaedic Surgeons. The senior author would only proceed to the next criteria after the evaluator demonstrated sufficient knowledge of the tested concept and was educated on any deficiencies in their understanding. For example, the evaluator would have to demonstrate sufficient understanding of the types of nonbiologic injections and how they differ from biologic agents (column 2, criteria 10, Table I) for the senior author to proceed to the next listed criteria.

Table I

Scoring rubric for website quality in shoulder biologic therapies.∗

Characteristics

Diagnosis and evaluation

1. Presents more than one biologic option 2. Discusses the definition and mechanism of action of the biologic therapy 3. Does not use the phrase “stem cells” when describing Food and Drug Administration-approved biologic agents 4. Discusses the process of obtaining/harvesting the biologic agent 5. Describes methods of application (injection) 6. Mentions variability in preparation 7. Mentions cost of biologic therapy 8. Mentions risk/side effects of biologic therapy 9. Acknowledges that the evidence base is limited (ie, still investigational) 10. Mentions any basic science evidence 11. Mentions any clinical evidence 12. Mentions need for randomized controlled trials

1. Describes in detail the anatomy of the shoulder 2. Describes in detail the function of the shoulder structures 3. Describes the shoulder conditions that biologic therapies can be used for (osteoarthritis, rotator cuff tendon tear/tendinitis, biceps tendon tear/tenosynovitis, frozen shoulder, subacromial impingement) 4. Physicians may examine your shoulder 5. Physicians may obtain radiographs 6. Physicians may obtain a magnetic resonance image (MRI) to evaluate 7. Describes appropriate patient selection for biologic therapies 8. Describes conservative treatment options—including activity modification, nonsteroidal anti-inflammatory drugs (NSAIDs) 9. Describes utility of physical therapy 10. Describes nonbiologic injections (corticosteroid, viscosupplementation) 11. Describes surgical treatment options 12. Mentions possibility of biologic therapy augmentation for a surgical procedure 13. Discusses any necessary rehabilitation after biologic treatment (ie, rest, physical therapy)

Total: _ / 25 points

Scoring sheet determined by “Optimizing Clinical Use of Biologics AAOS, 2019” and “The role of biologic agents in the management of common shoulder pathologies: current state and future directions.” James B. Carr II, MD, and Scott A. Rodeo, MD, JSES, 2019.

All evaluators were blinded to the search term and engine used to locate each website. Quality was assessed using an author-derived scoring rubric (Table I), with one point assigned for each criterion included in the website, for a total of 25 possible points. The scoring rubric for the assessment of website quality was developed by the senior authors based on a comprehensive literature review of biologic therapies in shoulder pathology and consensus guidelines from the 2018 American Academy of Orthopaedic Surgeons Annual Meeting., The rubric was modified from existing nonvalidated scoring systems, to examine whether direct-to-consumer online resources communicate critical components of the diagnosis, evaluation, and treatment of shoulder pathologies by biologic therapies. Twelve points were attributed to characteristics of biologic therapies and 13 points, to the diagnosis and evaluation of shoulder injuries. The total quality score of each website was defined as the average score of all three evaluators.

For determining accuracy, reviewers assigned 1 point if they perceived less than 25% of a website’s content to be accurate, 2 points for 25%-50% accuracy, 3 points for 51%-75%, and 4 points for 76%-100%. This scoring system was developed from established literature examining online resources for shoulder instability. The total accuracy score of each website is the sum of the individual evaluator scores, for a total of 12 possible points.

The Flesch-Kincaid method was used to quantify reading levels. This method provides a score that corresponds to the U.S. grade level the reader must have completed to understand the passage and is calculated with the following formula: (0.39 x average number of words per sentence) + (11.8 x average number of syllables per word) – 15.59. Microsoft Word (Microsoft, Redmond, WA, USA) was used to calculate Flesch-Kincaid levels. Websites were separated by those that were above a 12th grade reading level and those below, to reflect the level of education required by the reader to comprehend the information contained within the resources. Resources at or above a 12th grade reading level would require at least a high school education, as measured by the U.S. education system. This corresponds to completion of year 13 in the United Kingdom education system.

Data analysis

Descriptive statistics were generated for the entire cohort, as well as by search term, search engine, authorship, presence of commercial bias, and being below or above a 12th grade reading level. Given low sample sizes in certain categories, search terms were analyzed in aggregate (“PRP” and “platelet rich plasma” vs. “BMAC” and “bone marrow aspirate concentrate” vs. “lipogems” and “adipose tissue” vs. “biologic therapy” and “stem cell therapy”). Authorship was likewise aggregated into websites authored by health care professionals and those authored by other industry sources. A correlation analysis with the Pearson correlation coefficient was performed between quality and readability and quality and accuracy for the full sample. Independent sample t-tests and analysis of variance tests were used to determine differences in quality, accuracy, and readability within the cohort. Statistical significance was determined by P < .05. Analyses were performed with JMP (SAS Institute, Cary, NC, USA).

Results

The initial search yielded 600 results. There were 296 duplicate websites and 245 websites that did not meet inclusion criteria. There were 59 websites remaining in the final cohort relevant to the use of shoulder biologic therapies in shoulder pathology that were evaluated (Fig. 1).

Figure 1

Flowchart of the website selection process.

The mean quality of all the websites in the cohort was poor, with an average score of 7.97 ± 2.3 of 25 points (32%) (Table II). The quality ranged from 3.3 to 14.7. There were 12 possible points for the inclusion of the characteristics of biologic therapies, with the mean quality of this section being 4.49 ± 1.31 (37%) for all websites. Only 8 of 59 (14%) websites explicitly stated that biologics are experimental therapy. There were 13 possible points available for the diagnosis and evaluation of shoulder pathology, with the mean quality of this section being 3.48 ± 1.75 (27%) for all websites.

Table II

Descriptive statistics.

Category	Quality		Accuracy		Readability		Number of websites (N)
	Mean ± SD	P value	Mean ± SD	P value	Mean ± SD	P value
Total cohort	7.97 ± 2.30	–	8.47 ± 1.52	–	11.19 ± 1.93	–	59
Search term
BMAC/bone marrow aspirate concentrate	7.07 ± 1.04	.81	8.80 ± 1.10	.69	12.54 ± 3.73	.31	5
PRP/platelet rich plasma	8.14 ± 2.63	.81	8.35 ± 1.74	.69	11.12 ± 1.65	.31	26
Lipogems/adipose tissue	7.67 ± 2.00	.81	9.25 ± 0.96	.69	11.88 ± 2.17	.31	4
Biologic therapy/stem cell therapy	8.03 ± 2.21	.81	8.42 ± 1.44	.69	10.88 ± 1.68	.31	24
Authorship
Health care professional(s)	8.23 ± 2.48	.13	8.82 ± 1.56	.009∗	11.27 ± 1.82	.34	38
Other industry sources	7.51 ± 1.91	.13	7.86 ± 1.28	.009∗	11.05 ± 2.15	.34	21
Readability
Greater than 12th grade level	7.63 ± 2.18	.22	8.21 ± 1.47	.18	13.48 ± 1.19	–	19
Less than 12th grade level	8.13 ± 2.37	.22	8.60 ± 1.55	.18	10.10 ± 1.06	–	40
Commercial bias
Present	7.44 ± 1.15	.22	7.89 ± 1.05	.10	12.07 ± 3.08	.07	9
Not present	8.07 ± 2.45	.22	8.58 ± 1.58	.10	11.03 ± 1.64	.07	50

Health care professionals include physician, hospital, and clinic group websites.

Other industry sources include medical device or sales industry, professional organization, news article, and other websites.

A P value <.05 is considered significant.

The mean accuracy was also low for the cohort, with an average of 8.47 ± 1.52 of 12 possible points (71%). The accuracy ranged from 5.0 to 11.0. The average reading level was 11.2 ± 1.93, with 19 of 59 websites' (32%) reading at greater than a 12th grade level. Overall, the readability ranged from 7.3 to 17.3.

The search terms of “shoulder PRP” and “shoulder Platelet Rich Plasma” yielded the highest quality results, with a mean of 8.14 ± 2.63 (Table II). “shoulder Lipogems” and “shoulder Adipose tissue” yielded the most accurate results, with a mean score of 9.25 ± 0.96 of 12 points. “shoulder BMAC” and “shoulder bone marrow aspirate concentrate” were most difficult to read, with a mean reading level of 12.54 ± 3.73. There was no significant differences between search terms for quality (P = .81), accuracy (P = .69), or readability (P = .31).

Thirty-eight of 59 websites (64%) were authored by physicians, hospitals, or medical groups (health care professionals). Accuracy of websites authored by health care professionals was significantly higher than accuracy of those authored by other industry sources (P = .01). Quality (P = .13) and accuracy (P = .34) did not differ significantly between sources of authorship (Table II).

Nine of 59 websites (15%) demonstrated commercial bias. Quality (P = .22), accuracy (P = .10), and readability (P = .07) did not differ significantly between websites that showed commercial bias and those that did not (Table II). There was a weak negative correlation (R = -0.15, P = .25) between quality and readability and a weak positive correlation (R = 0.21, P = .09) between quality and accuracy.

Discussion

The exponential growth of biologic therapies as adjunct or standalone procedures in orthopedic practice is driven largely by patient demand and is not fully supported by a robust body of clinical evidence., As patients increasingly seek health information online, the onus is on providers to understand the content and informational value of the resources available to their patients. The present study examines the quality, accuracy, and readability of online resources pertaining to biologic therapies in shoulder pathology. Our findings demonstrate that the online resources available to patients are of poor quality and accuracy and difficult readability, which is consistent with our hypothesis. This study contributes to the existing literature by taking a comprehensive approach to identifying biologic agents and specifically examining their application to shoulder pathologies.

The quality of the direct-to-consumer resources available regarding shoulder pathology was alarmingly low at 7.97 ± 2.3 of 25 points, for an overall score of 32%. This outcome is consistent with prior examinations that used similar scoring systems,,, thus further contributing to concerns over the reliability of resources available to patients seeking information on biologic therapies for the treatment of musculoskeletal pathologies. Furthermore, only 14% of websites clearly stated that biologic therapies are experimental. Given the lack of evidence to support the regular use of biological agents in clinical practice, patients are placed at a disadvantage if available online resources fail to clarify the current experimental status of such therapies. As the use of biologic therapies for the management of a variety of shoulder pathologies rapidly increases, the content of online resources will need to become more comprehensive to appropriately inform patients of the applications, benefits, and risks of such agents.,,

Interestingly, the present study did not demonstrate significant differences in the quality of websites based on the search term used to find them, which conflicts with previously published literature. In a 2012 examination of the quality, accuracy, and readability of the information available online regarding lateral epicondylitis, “tennis elbow” and “lateral epicondylitis” yielded higher-quality information than “elbow pain.” Similar examinations of online resources pertaining to hallux valgus and developmental dysplasia of the hip also demonstrated that the use of clinically appropriate search terms can significantly impact the quality of information accessed by patients., More specifically to biologic therapy, Nwachukwu et al examined online resources pertaining to knee biologic therapies and demonstrated differences in the quality of websites based on the search term. There were significant differences between a full-length search term and its abbreviated counterpart, such as “knee PRP” and “knee platelet rich plasma.” Such differences were not observed between “shoulder PRP” and “shoulder platelet rich plasma” and “shoulder BMAC” and “shoulder bone marrow aspirate concentrate” in the present study. One possible explanation for this result is that the terminology of biologic therapies is inconsistent and often misused; it is possible that within shoulder surgery, this has prevented a more established search term from capturing a difference in quality. Further studies examining online resources regarding biologic therapies for specific shoulder pathologies, such as rotator cuff tears or glenohumeral osteoarthritis, are necessary to further delineate if such differences between search terms are applicable to shoulder surgery.

Our study’s analysis of the quality and accuracy of websites based on the type of authorship yielded valuable insights into the role of health care professionals in the growing industry of biologic therapies. A majority of the websites (64%) included in the cohort were authored by health care professionals, including physicians, hospitals, and clinical groups. The accuracy of these websites was significantly higher than the accuracy of those authored by other industry sources, including medical device and sales professionals, professional organizations, and news articles. This finding indicates that the involvement of health care professionals in the production of online resources regarding biologic therapy leads to the delivery of more accurate information to patients. Garcia et al examined the accuracy of websites related to shoulder instability and demonstrated that the accuracy of those authored by physicians was significantly higher than the accuracy of those authored by nonphysicians. However, a more recent examination of online educational resources for PRP demonstrated that the average accuracy of websites authored by health care providers was significantly lower than that of their counterparts, although this study was not specific to shoulder pathology.

Within websites produced by health care professionals, those authored by hospital or clinical groups had the highest quality, with a mean of 8.48 ± 2.66. Of note, there was no significant difference in quality between the two types of authorship, indicating that although websites authored by health care professionals are more accurate, they lack critical information regarding the characteristics of biologic therapies and their role in the diagnosis and evaluation of shoulder pathologies. This finding was inconsistent with current literature. Previous examinations of websites related to biologic therapies for musculoskeletal pathologies consistently demonstrate that websites authored by physicians and hospitals score significantly higher on quality than other websites.,, However, this present study is the first to our knowledge that assessed both the quality and accuracy of online resources for shoulder biologic therapies. Given the overall poor quality of this study cohort, our findings demonstrate a critical gap between the quality and accuracy of websites originating from health care sources. This is further supported by a weak and nonsignificant correlation between website quality and accuracy. When authoring content that is used to directly market biologic therapies to patients, professionals should ensure that the information they provide is not only accurate, but also comprehensively addresses the role of biologic agents in shoulder surgery by including the components listed in Table I.

The mean readability of the websites was at an 11th grade reading level, which is higher than the 8th grade reading level of the average American resident and the 5th grade reading level of the average Medicare beneficiary., Although a majority of the websites read below a 12th grade reading level, only one website (1.7%) was below an 8th grade reading level. The tendency for online resources to be of difficult readability has been long established.,, Readability did not significantly correlate with the quality of websites (R = −0.15, P = .25) in the present study, which was also demonstrated by Goldenberg et al’s examination of the online resources available for rotator cuff repair. Shoulder providers should keep in mind that even websites containing comprehensive information about the role of biologic therapies in treating shoulder conditions might not be entirely comprehensible to their patients. Inappropriately high reading levels render online resources regarding biologic therapies for shoulder pathology inaccessible to the average patient and decrease proper understanding of the role of biologic therapies in orthopedic practice.

This study has several limitations. The quality and accuracy scores were determined through subjective evaluation. Although three independent evaluators were used, this process may introduce bias. Our results are also only generalizable to the search terms and engines used in the study. This was controlled by using eight broad search terms and including abbreviations of full search terms to comprehensively capture ways in which patients may search for information online. However, it is possible that patients use other search terms or combinations to find information. Similarly, we only included the top 25 results from each search in the initial data collection. This approach may have excluded some websites with strong informational value. This limitation may be mitigated by consumer behavior as patients seeking health information online are more likely to change their search term than navigate to the second page of a search engine for further results. Finally, although the scoring rubric used to assess website quality was based on a comprehensive literature search and balanced the inclusion of characteristics of biologic therapies with information specific to shoulder pathology, it is novel to the present study and not a validated system.

Conclusion

The online resources available to patients seeking information about biologic therapies for the treatment of shoulder pathologies are of very poor quality, moderately poor accuracy, and advanced readability. Providers should caution patients about the reliability of direct-to-consumer biologic marketing for shoulder pathology.

Disclaimers

Funding: No funding was disclosed by the authors.

Conflicts of Interest: The authors, their immediate families, and any research foundation with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.