Modern Internet Search Analytics and Total Joint Arthroplasty: What Are Patients Asking and Reading Online?

BACKGROUND: Patients considering total joint arthroplasty often search for information online regarding surgery; however, little is known about the specific topics that patients search for and the nature of the information provided. Google compiles frequently asked questions associated with a search term using machine learning and natural language processing. Links to individual websites are provided to answer each question. Analysis of this data may help improve understanding of patient concerns and inform more effective counseling.
METHODS: Search terms were entered into Google for total hip and total knee arthroplasty. Frequently asked questions and associated websites were extracted to a database using customized software. Questions were categorized by topic; websites were categorized by type. JAMA Benchmark Criteria were used to assess website quality. Pearson's chi-squared and Student's t-tests were performed as appropriate.
RESULTS: A total of 620 questions (305 total knee arthroplasties, 315 total hip arthroplasties) were extracted with 602 associated websites. The most popular question topics were Specific Activities (23.5%), Indications/Management (15.6%), and Restrictions (13.4%). Questions related to Pain were more common in the TKA group (23.0% vs 2.5%, P < .001) compared to THA. The most common website types were Academic (31.1%), Commercial (29.2%), and Social Media (17.1%). JAMA scores (0-4) were highest for Government websites (mean 3.92, P = .005).
CONCLUSION: The most frequently asked questions on Google related to total joint arthroplasty are related to arthritis management, rehabilitation, and ability to perform specific tasks. A sizable proportion of health information provided originate from non-academic, non-government sources (64.4%), with 17.1% from social media websites.

Machine learning, broadly defined, is a field of computer science that uses algorithms to recognize patterns in data. Recent developments in a subset of machine learning, known as deep learning, now allow for pattern recognition in vast quantities of data that were previously too computationally complex to process [1]. In medicine, deep learning has led to advances such as a tool that predicts diabetic retinopathy using retinal fundus photographs and a test that distinguishes COVID-19 from community-acquired pneumonia using chest computed tomography imaging [[2], [3], [4]]. Machine learning algorithms using patient data from the electronic medical record have been designed to predict acute kidney injury, cancer mortality rate, and prognosis following solid organ transplantation [[5], [6], [7]]. New applications are sure to emerge as this technology matures [8].

Perhaps the most common way people interact with a sophisticated deep learning algorithm is by using Google Web Search, by far the most widely used search engine in the United States [9]. In 2015, Google introduced a machine learning-based system, known as RankBrain, to recognize patterns in individual search queries [10]. By analyzing a large dataset of search queries, this technology allowed Google to predict individual searches and offer suggestions after a search query is entered [10]. In 2018, Google added a natural language processing system into its search analytics platform, a technology called BERT (Bidirectional Encoder Representations from Transformers) [11,12]. Natural language processing significantly expands the capability of deep learning algorithms to identify search patterns more accurately. The technology behind BERT has been adapted to help organize patient data in the electronic health record and recognize health information disseminated on social media [[13], [14], [15]]. Using RankBrain and BERT, the Google search results page now provides an extensive list of questions frequently asked along with the original search query [11]. Additionally, links to websites are provided to “answer” each associated question [11].

Internet usage rates among patients considering elective orthopedic procedures have been reported to be as high as 84% [16,17]. Up to 80% of these patients research their condition online, with one survey indicating that 30% of these patients specifically discuss information found online with their surgeon [[16], [17], [18], [19], [20], [21], [22]]. Multiple studies have evaluated the quality and readability of online resources for orthopedic procedures [20]. However, these studies provide little insight into what specific information patients are trying to obtain when searching their conditions online. Using the modern search analytics system employed by Google, clusters of frequently asked questions associated with specific orthopedic conditions and procedures can be identified and analyzed.

We present an analysis of the questions most frequently associated with total hip arthroplasty (THA) and total knee arthroplasty (TKA) by question type and topic. The websites provided to address each question are also analyzed for source and quality. We hypothesize that there are distinct search patterns for THA compared to TKA. Greater understanding of this data could allow for better understanding of patient concerns as well as inform more effective counseling regarding total joint arthroplasty.

Materials and Methods

Search terms were entered into Google Web Search (www.google.com) using a clean-installed Google Chrome browser for THA (“hip replacement,” “total hip replacement,” “total hip arthroplasty”) and TKA (“knee replacement,” “total knee replacement,” “total knee arthroplasty”). A clean-installed browser was used to minimize the effect of personalized search algorithms employed by Google Search. On each results page, the list of frequently associated questions was refreshed until approximately 100 questions were generated for each of the 6 search terms. Multiple studies of the quality of online health information have included between 50 and 150 websites [20]. Thus, we chose to generate 100 questions per search query to reflect the precedent set in the existing literature. A freely available program (Scraper, version 1.7) was adapted to extract each question and its associated website to a database. Questions and websites were identified on the webpage by its unique XML Path Language (XPath) string.

The questions were first categorized according to Rothwell’s system as previously reported in the literature [23,24]. The definitions for Rothwell’s classification are summarized in Table 1 . This classification was expanded for the purpose of this study into 10 topics relevant for total joint arthroplasty: Specific Activities, Timeline of Recovery, Restrictions, Technical Details, Cost, Indications/Management, Risks/Complications, Pain, Longevity, and Evaluation of Surgery. Descriptions for these topics may be found in Table 1.

Table 1

Rothwell’s Classification of Questions, Question Classification by Topic, and Website Categorization.

Rothwell’s Classification	Description
Fact	Asks whether something is true and to what extent, objective informationExample: How long will I need a walker?
Policy	Asks whether a specific course of action should be taken to solve a problemExample: Should I delay hip replacement surgery?
Value	Asks for evaluation of an idea, object, or eventExample: How successful is a hip replacement?
Question Classification by Topic	Description
Fact
Specific activities	Ability to perform a specific activity or action after TJA
Timeline of recovery	Specific questions regarding length of time for recovery milestones
Restrictions	Restrictions to activity or lifestyle during recovery or indefinitely
Technical details	Surgical procedure, includes specific questions about implants
Cost	Cost of surgery and/or rehabilitation postoperatively
Policy
Indications/management	Surgical indications, arthritis management, timing of surgery
Risks/complications	Management of risks/complications during and after surgery
Value
Pain	Pertains to duration, severity, and management of pain
Longevity	Specific questions regarding longevity of TJA
Evaluation of surgery	Evaluation of TJA, eg, successfulness or invasiveness
Website Categorization	Description
Commercial	Commercial organization that positions itself as a source of health information, includes medical device and pharmaceutical companiesExample: WebMD, Everyday Health
Academic	Institution with a clear academic mandate, including universities, academic medical centers, academic societiesExample: AAOS, Mayo Clinic, HSS
Medical practice	Local hospital or orthopedic practice without an academic affiliationExample: New York Orthopedics
Single surgeon personal	Website built and maintained by individual surgeon. Excludes biography pages on institutional websitesExample: EdwinSu.com, DrRMarx.com
Government	Websites ending in.gov or maintained by a national governmentExample: Medline, PubMed
Social media	Websites maintained by nonmedical organizations primarily designed for information sharing between internet users. Includes health blogs, internet forums, and support groupsExample: fitpro.com, silversneakers.com

TJA, total joint arthroplasty.

In accordance with previous studies, websites were categorized by source into the following groups: Commercial, Academic, Medical Practice, Single Surgeon Personal, Government, and Social Media (Table 1) [25,26]. Commercial websites were defined as ownership by a for-profit entity not involved in direct patient care. Academic websites were maintained by an organization with a clear academic mandate. The Medical Practice category consists of websites maintained by private medical groups without an academic mandate. Single Surgeon Personal websites are maintained by a single surgeon and separate from biography pages on an affiliated institutional website. Government websites included pages directly maintained by a national governmental entity such as the National Institutes of Health (United States) or the National Health Service (United Kingdom). Social Media was defined as websites maintained by non-medical organizations and primarily designed for sharing information between users.

The Journal of the American Medical Association (JAMA) Benchmark Criteria was used as a measure of website quality. The JAMA Benchmark Criteria consists of 4 components (Authorship, Attribution, Currency, Disclosure) with one point assigned for the presence of each component (Table 2 ) [27]. This instrument has been used in multiple previous studies investigating the quality of online health information [20,[28], [29], [30]].

Table 2

JAMA Benchmark Criteria.

Criteria	Description
Authorship	Clearly identifiable author and contributors with affiliations and relevant credentials present
Attribution	References and sources clearly listed with any copyright information disclosed
Currency	Clearly identifiable posting date of any content as well as date of any revisions
Disclosure	Website ownership clearly disclosed along with any sponsorship, advertising, underwriting, and financial support

JAMA, Journal of the American Medical Association.

Classification of questions and websites was performed by 2 independent reviewers (T.S.S., W.I.) after agreeing upon the definitions in Table 1. Each website was evaluated for the 4 JAMA Benchmark Criteria and the resulting scores were recorded. Statistical analysis was performed using Microsoft Excel (Microsoft Corporation, Redmond, WA). Interobserver reliability for question classification and website categorization was assessed using Cohen’s kappa coefficient [31]. In the final analysis, discrepancies between the 2 reviewers were resolved by consensus decision between the 2 reviewers and a third party (P.B.). Pearson’s chi-squared tests were used to analyze nominal data. Student’s t-tests were performed to compare JAMA Benchmark Scores. No funding was necessary to perform this study.

Results

A total of 620 questions (305 TKAs, 315 THAs) and 602 associated websites were extracted and categorized. Interobserver reliability as assessed using Cohen’s kappa coefficient was 0.93 for question classification and 0.91 for website classification. The top 10 most frequently asked questions for THA and TKA are presented in Table 3 .

Table 3

Top Ten Most Popular Questions for THA and TKA.

Total Hip Arthroplasty	Total Knee Arthroplasty
How long does it take for bone to grow into hip replacement?	Can I stay alone after total knee replacement?
What you cannot do after a hip replacement?	How long will I need pain medication after total knee replacement?
Can hip replacement be avoided?	What should I avoid after knee replacement?
Are there permanent restrictions after hip replacement?	Why is a knee replacement so painful?
Can you ever cross your legs after hip replacement?	How do you sit on a toilet after knee surgery?
Can I squat after hip replacement?	How long does it take to bend your knee after surgery?
How do you poop after hip surgery?	Why does knee replacement hurt more at night?
How do you shower after hip surgery?	Can I vacuum after knee replacement?
Is it OK to sit in a recliner after hip replacement surgery?	Can you damage a knee replacement?
What is the time frame for blood clots after hip replacement?	Can you wait too long for a knee replacement?

THA, total hip arthroplasty; TKA, total knee arthroplasty.

The majority questions fell into the Fact category using Rothwell’s system (Fig. 1 A; 56.4% TKA, 57.5% THA). In the TKA group, the most popular question topics were Specific Activities (24.3%), Pain (23.0%), Restrictions (15.1%), and Timeline of Recovery (13.1%) (Table 4 ). In the THA group, the most popular question topics were Specific Activities (22.9%), Indications/Management (19.0%), Restrictions (11.7%), and Technical Details (11.1%) (Table 4). Questions related to Pain were significantly more common in the TKA group (23.0% vs 2.5%, P < .001) compared to THA (Fig. 1B). In the THA group, there were significantly more questions related to Technical Details (11.1% vs 3.9%, P < .001), Indications/Management (19.0% vs 12.1%, P = .02), Risks/Complications (9.8% vs 0.3%, P < .001), and Longevity (3.2% vs 0.0%, P < .001) compared to TKA (Fig. 1B).

Fig. 1

Relative distribution of questions by Rothwell’s classification and by topic for THA and TKA. (A) The percentage of questions in the THA and TKA groups belonging to the Fact, Value, Policy, or Other classifications. (B) The number of questions in each topical category in the THA and TKA groups. Statistically significant differences (P < .05) in the number of questions within a topical category between the THA and TKA groups are indicated with ∗. THA, total hip arthroplasty; TKA, total knee arthroplasty.

Table 4

Distribution of Searched Questions by Rothwell’s Classification and by Topic for TKA and THA.

	TKA	% TKA	THA	% THA	Total	% Total	P-Value
Total questions	305	49.2%	315	51.8%	620	100%
Rothwell’s classification
Fact	172	56.4%	181	57.5%	353	56.9%	.78
Policy	38	12.5%	91	28.9%	129	20.8%	<.001
Value	93	30.5%	39	12.4%	132	21.3%	<.001
Unrelated	2	0.7%	4	1.3%	6	1.0%	.43
Topical classification
Specific activities	74	24.3%	72	22.9%	146	23.5%	.68
Timeline of recovery	40	13.1%	34	10.8%	74	11.9%	.37
Restrictions	46	15.1%	37	11.7%	83	13.4%	.22
Technical details	12	3.9%	35	11.1%	47	7.6%	<.001
Cost	0	0.0%	3	1.0%	3	0.5%	.09
Indications/management	37	12.1%	60	19.0%	97	15.6%	.02
Risks/complications	1	0.3%	31	9.8%	32	5.2%	<.001
Pain	70	23.0%	8	2.5%	78	12.6%	<.001
Longevity	0	0.0%	10	3.2%	10	1.6%	<.001
Evaluation of surgery	23	7.5%	21	6.7%	44	7.1%	.67
Unrelated	2	0.7%	4	1.3%	6	1.0%	.43

THA, total hip arthroplasty; TKA, total knee arthroplasty.

Statistically significant P-values are bolded (P < 0.05).

The most common website types for both the TKA and THA groups were Academic (31.1%), Commercial (29.2%), and Social Media (17.1%) (Fig. 2 A). There was a statistically significantly higher rate of websites in the Other category in the THA group compared to TKA (Table 5 ; 1.3% vs 0.3%, P = .04). Websites in the Other category originated mostly from law firms (4 out of 8) and financial firms (3 out of 8). These websites were associated with questions regarding lawsuits (categorized under Risks/Complications) that were more frequent in the THA group. Otherwise, there were no statistically significant differences in the distribution of website source between the TKA and THA groups.

Fig. 2

Relative distribution of website sources by type of surgery and question topic. (A) The percentage of websites in the THA and TKA groups belonging to each of website sources (commercial, academic, single surgeon personal, medical practice, government, social media, and other). (B) The percentage of website belonging to each source stratified by question topic. Statistically significant (Pearson’s chi-squared test) sections for which P < .05 is designated with “∗”; P < .01 with “∗∗”; and P < .001 with “∗∗∗”

Table 5

Distribution of Website Categories for TKA and THA With JAMA Benchmark Score.

	JAMA Scorea	TKA	% TKA	THA	% THA	Total	% Total	P-Value
Total websites	2.37 (1.22)	294	48.8%	308	51.2%	602	100%
Website categorization
Commercial	3.66 (0.65)	86	29.3%	90	29.2%	176	29.2%	.99
Academic	2.06 (0.94)	84	28.6%	103	33.4%	187	31.1%	.20
Single surgeon	0.81 (1.11)	14	4.8%	14	4.5%	28	4.7%	.90
Medical practice	1.85 (1.15)	35	11.9%	38	12.3%	73	12.1%	.87
Government	3.92 (0.39)	15	5.1%	12	3.9%	27	4.5%	.47
Social media	2.18 (0.84)	59	20.1%	44	14.3%	103	17.1%	.06
Otherb	2.14 (1.21)	1	0.3%	7	2.3%	8	1.3%	.04

JAMA, Journal of the American Medical Association; THA, total hip arthroplasty; TKA, total knee arthroplasty.

Statistically significant P-values are bolded (P < 0.05).

JAMA score reported as mean (standard deviation).

Website sources in this category were law firm (4), financial firm (3), and non-medical device manufacturer (1).

The distribution of website sources stratified by question topics is shown in Figure 2B. Of note, Social Media websites were more commonly linked to questions about Specific Activities compared to other categories (26.9%, P < .001). Government websites were more commonly associated with questions about Restrictions (19.5%, P < .001). Single Surgeon Personal websites were more commonly associated with questions on Risks/Management (12.9%, P = .02) and Pain (15.5%, P < .001). Websites maintained by non-academic Medical Practices were more associated with questions regarding Technical Details (23.2%, P = .03) and Indications/Management (20.0%, P = .008).

The highest JAMA Benchmark scores were found in Government websites (mean 3.92, P = .005) and Commercial websites (mean 3.66, P = .016) compared to all other sources. Single Surgeon Personal websites yielded the lowest JAMA score at 0.81. The mean score for Academic websites was 2.06, this was significantly lower than scores for Government (P < .001) and Academic websites (P < .001) and not significantly different from Social Media (2.18, P = .84). JAMA scores for Social Media websites were higher in the THA group compared to TKA (2.70 vs 1.61, P < .001).

Discussion

By analyzing millions of search queries related to total joint arthroplasty, Google’s search analytics essentially distill the curiosity of countless patients into a list of frequently asked questions. The data originates from the patients themselves and is captured outside of the usual instruments and surveys of standard clinical research. To our knowledge, this is the first study to analyze what patients search for on the internet regarding total joint arthroplasty.

Distribution of Question Topics

The distribution of question topics provides insight into how to best address matters that patients are concerned about. For both TKA and THA, questions regarding the details of rehabilitation and the ability to perform specific tasks after surgery accounted for 48.8% of all questions analyzed. Within the Specific Activities category, commonly asked activities include squatting, sitting, showering, personal hygiene, and household cleaning. The relatively high proportion of questions on these topics may be a result of postoperative patients searching for additional information during their recovery. This could represent an area for improvement in the resources available to patients for guidance through the rehabilitation process. Especially for practices that are shifting toward outpatient arthroplasty, eliminating the postoperative hospital stay reduces the opportunity for patients to ask questions and receive counseling [32]. Smartphone applications and telemedicine have been used to improve patient monitoring and communication with some success [33,34]. In one study, patients who engaged with their surgical team via telemedicine postoperatively were found to have higher satisfaction compared to traditional follow-up [35]. Although our results highlight the importance of providing guidance during the rehabilitation process, ultimately the most effective strategies to improve patient counseling will depend on individual practice settings.

Although the distribution of questions regarding rehabilitation and activities were relatively similar between TKA and THA, there were several notable differences. Questions regarding pain, particularly pain at night, were significantly more common in the TKA group compared to THA (23.0% vs 2.5%, P < .001). The phrasing of many of these questions would suggest that they are asked by postoperative patients (eg, “When will pain get better after a knee replacement?” or “Why does knee replacement hurt more at night?”). Previous studies have shown increased postoperative pain in patients undergoing TKA compared to THA [36]. Particularly with TKA, effective management of postoperative pain is associated with faster rehabilitation and decreased risk of complications [[37], [38], [39]]. Effective pain management also serves to decrease the amount of opiates necessary after surgery [[40], [41], [42], [43]]. Increasingly, multimodal pain management strategies have become standardized and lead to improved postoperative pain control [39,44]. Our finding regarding postoperative pain in TKA highlights the need for continued research in postoperative pain management.

In the THA group, questions regarding the technical details of surgery, surgical indications, complications, and implant longevity were more common compared to the TKA group. These questions appear to be asked by preoperative patients seeking to better understand the surgery. These questions included subjects such as implant fixation, osseointegration, bearing surfaces, surgical approach, deep vein thrombosis, and timing of surgery. It is possible that patients may consider THA to be more difficult to conceptualize than TKA or that there are more publicized examples of THA failure. However, these explanations are speculative at best; it is unclear why there are more questions regarding these topics in the THA group.

Overall, our findings support our anecdotal experience with common patient questions, which supports validity of our methodology to better understand our patients in a more objective manner. We believe this data is useful for surgeons designing preoperative classes or other perioperative counseling programs.

One important limitation to this study is inherent with the dynamic nature of Google’s search analytics system. The questions and websites presented by Google will change as more data are generated by patients researching total joint arthroplasty online. Furthermore, because Google takes individual search patterns into account when presenting search results, questions and websites can differ from person to person. We address this potential variability in the data by using a large sample size of questions and websites. To minimize the effect of individual search history, all data were extracted using a clean-installed web browser. Although we do not believe that these issues significantly affect the validity of our findings, the inherent qualities of Google’s proprietary system still represent a limitation. Our categorization system is also an important limitation of this study. Although the categories were made in accordance with previous studies and tested for interobserver reliability, there remains inherent overlap between some of the categories.

Improving the Quality of Online Health Information

Although the academic sources were the most common type of website in this study, a sizable proportion of websites analyzed originated from non-medical, non-government sources (47.6%). Interestingly, the average JAMA score was significantly higher in non-medical, non-government sources (3.09 vs 2.06, P < .001). There was also no statistically significant difference in JAMA scores between academic websites and social media (2.06 vs 2.18, P = .84). These relatively low scores should be cautiously interpreted. As an instrument, the JAMA Benchmark Criteria is limited in its ability to assess the accuracy and appropriateness of online information. A perfect score can be obtained simply by the presence of an author, date, disclosures, and list of references. As such, the JAMA Benchmark Criteria is a better measure of website transparency than information quality. This limitation is acknowledged by Silberg [27] in the article that first described the JAMA score. Additionally, modern website designs may distribute the components of the score across multiple pages, thus decreasing the score for an individual page. Despite these limitations, the JAMA Benchmark Criteria remains as one of the most well-established instruments available for assessing online health information [20]. Instruments that assess the information quality are difficult to standardize across multiple fields of study. As such, many studies examining the quality of online literature on specific orthopedic topics utilize original instruments designed for that subject [20,45,46]. Certain principles of the JAMA score, such as authorship and citations, may help an otherwise high-quality website appear more transparent to readers, which may indirectly affect the subjective credibility of the website. However, for academic institutions and surgical practices, we ultimately believe that it is more valuable to provide high-quality and up-to-date information rather than focus on the specific components of the JAMA score.

We considered the website source itself as an indirect indicator of information quality. Websites maintained by academic centers, established practices, and professional societies almost certainly provide more accurate and up-to-date information compared to sources such as social media. Non-medical websites can still play a certain role for patients. For example, our study found that social media websites were more commonly linked to questions about performing specific tasks, a topic that may be discussed between patients in groups online. Internet communities that facilitate patient communication can be useful as a supplemental source of information and support [47]. Still, the best information available regarding total joint arthroplasty likely comes from sources that are directly or indirectly maintained by orthopedic surgeons. We believe that the arthroplasty community should take the lead in providing the most up-to-date and accessible information for patients on the internet. There is opportunity to optimize modern orthopedic websites to better integrate with Google’s changing search algorithms to reach more patients. Search engine optimization techniques that have been widely employed by businesses are now also being used by medical practices to increase viewership [22,48,49]. As advances in information technology continue to shape our world, the field of orthopedics will need to adapt appropriately to best take care of our patients.