Information quality and dynamics of patients' interactions on tonsillectomy web resources.

Information technologies have drastically altered the way patients gather health-related information. By analysing web resources on tonsillectomy, we expose information quality and dynamics of patients' interactions in the online continuum. Readability was assessed using Flesch Reading Ease (FRE), Flesch Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), and Gunning Fog Index (GFI). Comprehensibility and actionability were assessed using the Patient Education Materials Assessment Tool (PEMAT). Metrics of forums included author characteristics (level of disclosure, gender, age, avatar image, etc.), posts' motive (community support vs. medical information) and content (word count, emoticon use, number of replies, etc.). Analysis of 6 professional medical websites, of 10 health information portals, and of 3 discussion forums totalizing 1369 posts on 358 threads, from January 1, 2007 to December 31, 2014, reveals that online resources exceed understandability recommendations. Women were more present on online health forums (68.2% of authors disclosing their gender) and invested themselves more in their avatar. Authors replying were significantly older than authors of original posts (39.7 ± 0.8 years vs. 29.2 ± 0.9 years, p < 0.001). The degree of self-disclosure was inversely proportional to the requests for medical information (p < 0.001). Men and women were equally seeking medical information (men: 74.0%, women: 77.0%) and community support (men: 65.7%, women: 70.4%), however women responded more supportively (women 86.2%, men 59.1%, p < 0.001). The dynamics of patients' interactions used to overcome accessibility difficulties encountered is complex. This work outlines the necessity for comprehensible medical information to adequately answer patients' needs.

Introduction

While a few decades ago, the only way for patients to gather health-related information was through direct interactions with health professionals, information technologies have dramatically changed this situation. Indeed, the last decades have witnessed a constant migration of information consumption to online spaces, where information becomes dynamic (Brossard, 2013).

Internet accessibility enables rapid retrieval of formerly inaccessible health-related information through a variety of platforms, such as official resources on health associations' websites, health information portals, or community discussion boards, with up to 72% of Internet users seeking health information online (Fox and Duggan, 2013). Patients can not only share information and advice, but also experiences and peer or community support (Klemm et al., 1999, Owen et al., 2004, Blank and Adams-Blodnieks, 2007, Blank et al., 2010), a possibility that was unconceivable on a large scale before online environments gained such prominence.

From a public health perspective, neglecting this online dimension in population health education has become impossible. Quite logically, both professional medical websites (Eloy et al., 2012, Svider et al., 2013, Kasabwala et al., 2012, Hansberry et al., 2014a, Hansberry et al., 2014b) and patient discussion boards have become the target of scrutiny (Klemm et al., 1999, Blank and Adams-Blodnieks, 2007, Mo et al., 2009, Blank et al., 2010, Attard and Coulson, 2012, Chen, 2012). However, this field is still nascent, and no attempt has been made to analyse simultaneously the information provided on various health information platforms, and the strategies patients use to overcome accessibility difficulties encountered in terms of readability and content understandability.

We aimed to answer this need by analysing the particular case of tonsillectomy, which represents an ideal model for this purpose. First, tonsillectomy affects a large population, accounting for 1/6th of all surgical procedures in children < 15 years old, as well as 26% of all surgical procedures on the nose, mouth and pharynx all ages combined (Cullen et al., 2009). Second, tonsillectomy provides a mixed population both in terms of gender and age, an aspect lacking in the majority of previous studies (Klemm et al., 1999, Blank and Adams-Blodnieks, 2007, Mo et al., 2009, Blank et al., 2010, Attard and Coulson, 2012, Chen, 2012). Third, tonsillectomy patients and caregivers, as with any surgery, need to be supplied with adequate information and support throughout the operative period to better manage complications such as throat pain, dehydration, nausea and vomiting, delayed feeding, voice changes and hemorrhage (Baugh et al., 2011). Finally, due to the well-known occurrence of transient – yet unpleasant – side effects following the surgery, tonsillectomy represents an ideal model to study peer or community support. The aim of this study was to analyse simultaneously and for a same pathology 1) the quality of online information available in terms of readability, understandability, and actionability, 2) information seeking online behavior of patients, and 3) how information quality as defined above impacts users' online behavior.

Our study enlightens that patients' needs are not currently fully addressed, and reveals that the online health information gathering strategies of patients are considerably more complex than originally speculated. Furthermore, our data sheds new light on the mechanisms and gender-specificities of community-based online interactions related to health concerns.

Materials and methods

Website selection and analysis

Six official medical webpages on tonsillectomy were selected from websites of professional otolaryngology or pediatric associations from countries having English as a first and native language. Ten health information portal websites were selected from web-portals centered on health education and communication, non-affiliated to a specific medical association, with English as a primary language, and with an active forum (≥ 20 posts monthly) (Fig. 1). References specifically targeted to health professionals (i.e., medical doctors, otorhinolaryngologists, or nurses and other health care professionals) were excluded. The readability, understandability, and actionability of the material presented on these 16 webpages were evaluated in January 2015.

Fig. 1

Online health information platforms data sources.

Readability was evaluated with a battery of four tests considered as highly reliable for biomedical context (Meade and Smith, 1991, Friedman and Hoffman-Goetz, 2006, Wang et al., 2013), selected among the most commonly used readability analysis tools in the otolaryngology field (Eloy et al., 2012, Kasabwala et al., 2012, Cherla et al., 2012, Cherla et al., 2013, Svider et al., 2013, Alamoudi and Hong, 2015): the Flesch Reading Ease (FRE), the Flesch Kincaid Grade Level (FKGL), the Simple measure of Gobbledygook (SMOG), and the Gunning Fog Index (GFI). This particular battery has been validated in the context of online materials (Guitton, 2015a, Guitton, 2015b), including general online material such as posts (Guitton, 2015a), and health-related online material (Guitton, 2015b). As recommended by the NIH (Wang et al., 2013, Anon., 2013a), all the retrieved texts were formatted for readability assessment purposes.

Understandability (the clarity in terms of message processing of the presented material), and actionability (the possibility for patients to take action following the exposure to the presented material) were evaluated using the Patient Education Materials Assessment Tool (PEMAT) for printable material (Anon., 2013b, Shoemaker et al., 2014), which has already been used to evaluate online health-related material (Guitton, 2015b). Two scores on a 100 scale were generated for understandability, and actionability using the PEMAT by two independent judges.

Forum selection and analysis

Inclusion criteria for web-based medical discussion boards (“forums”) included: being in English language, having a significant ear, nose, and throat community within the forum (≥ 800 total threads), being active throughout 2014 (≥ 20 posts monthly), and having an embedded search engine allowing specific term search. Discussion boards relating strictly to cancer were excluded. Three separate online discussion groups were identified (Fig. 1). Data was collected on all forum threads relating to tonsil removal or tonsillectomy last updated from January 1, 2007 to December 31, 2014.

For each post's author, the following characteristics were noted: self-reported gender and age when disclosed, forum activity (number of posts), number of days between the user's account creation and thread creation, number of days active on forum (between user's account creation and last post), and finally whether the author associated an avatar image (whatever the nature of this image) to his/her account. In order to avoid bias within threads, personal information was noted only once per user on each thread for analysis purposes (including the initial author of the thread). The total word count and the number of replies to each post were also noted.

For each thread, the target of the initial post (whether the author was posting on his own behalf, for somebody else including a child, or specifically for a child), and the intention of the initial post (seeking medical information or advice or looking for community support) were recorded. Medical information seeking was defined as looking for medical advice, solutions, explanations, (e.g., “can I continue doing sports after my surgery?” and “do I have an infection, is the white stuff normal?”). Community support was defined as sharing an experience (e.g., “…, this is my experience, just thought I would share”), looking for support when scared, worried, in pain, etc., or asking for feedback (“has anyone else experienced this?”). These two categories were not mutually exclusive and could be evidenced within a same post. Subsequent replies were also assessed to determine if the author was giving peer support, seeking or providing medical information.

Evaluation of information quality

Although evaluating the quality of information was not the aim of this study, it is nonetheless an important issue for health-related web material. Therefore, the medical information provided was evaluated in terms of adequacy (adequate, erroneous or dangerous) by a trained medical doctor. When information was evaluated as erroneous by the first medical evaluator, this judgment was subsequently validated by an otorhinolaryngologist.

Statistical analysis

Coding was performed by a single observer over a period of one week. Validity was assessed by an intra-judge analysis performed 3 months after the initial coding on 20% of the corpus (intra-judge reliability > 90%). Despite the large size of our samples, given the facts that single individuals could register through several different accounts, non-parametric statistical approaches were favored over parametric approaches in order to avoid the generation of false positive results. Therefore, when appropriate, comparisons were made using the non-parametric Mann-Whitney U test. Proportions were compared using z-tests. All statistical analyses were conducted with the rejection level set at p ≥ 0.05. When applicable, results are presented as means ± SEM.

Results

Webpages' analysis

Patient education web pages had reading levels above the recommended eighth grade reading level (medical association websites: FRE: 54.7 ± 3.5, FKGL: 9.0 ± 0.6, SMOG: 8.6 ± 0.4, GFI: 11.8 ± 0.6; health information portals: FRE: 58.2 ± 2.3, FKGL: 8.8 ± 0.4, SMOG: 8.6 ± 0.3, GFI: 11.7 ± 0.4).

PEMAT evaluations were consistent across judges (no significant differences neither for understandability, p = 0.32; nor for actionability, p = 0.55). Web pages scored poorly for understandability (medical association websites: 66.9 ± 5.0, health portal websites: 58.2 ± 5.9, not significantly different, p = 0.33) as well as actionability (medical association websites: 39.2 ± 6.6, health portal websites: 27.0 ± 7.0, not significantly different, p = 0.26).

Forums' analysis

Forums' content was available publicly. All online health communities were moderated by web community staff or active community members, but not by registered healthcare professionals. A total of 358 threads were analysed, for a total of 1369 posts (including initial posts and replies) (Table 1).

Table 1

Characteristics of forum posts.

Gender and age were self-declared and noted when explicitly stated in the author's message or on their profile. No assumptions were made based on style of writing or on avatar image. Data are presented as means ± SEM.

Characteristic	Total (N = 1369)		Initial posts (N = 358)			Replies (N = 1011)
	Women (N = 732)	Men (N = 341)	Women (N = 151)	Men (N = 72)	Undeclared (N = 135)	Women (N = 581)	Men (N = 269)	Undeclared (N = 161)
Declared gender – %	68.2	31.8	67.7	32.3	37.7	68.4	31.6	15.9
Age – year	–	–	29.0 ± 1.2	29.6 ± 1.5	–	40.1 ± 0.9	38.7 ± 2.1	–
Exchanging community support – %	–	–	70.4	65.7	55.0	86.2	59.1	75.8
Seeking medical information – %	–	–	77.0	74.0	85.5	8.4	7.9	28.6
Providing adequate information – %	–	–	–	–	–	34.4	60.4	14.9
Providing erroneous information – %	–	–	–	–	–	2.5	2.1	1.2
Word count	–	–	253.3 ± 30.2	199.8 ± 21.8	135.9 ± 9.1	130.6 ± 6.2	105.6 ± 7.5	80.2 ± 5.0
Emoticon use – %	–	–	19.87	11.11	1.53	16.57	3.7	0.6

Characteristics of posts' authors

Most authors disclosed their gender (75.1%; 62.3% for initial posts and 84.1% for replies, p < 0.001). More women than men were observed on the forum (total: 68.2% women and 31.8% men; initial post: 67.7% women and 32.3% men; and replies: 68.4% women and 31.6% men). A total of 42.7% of authors disclosed their age (initial posts: 35.9%, replies: 48%, p < 0.001). While the overall age was of 35.2 ± 0.6 years, the age of initial posts' authors (29.2 ± 0.8 years) was significantly different from the authors of the replies (39.7 ± 0.8 years, p < 0.001). However, no major differences were seen when comparing men and women (initial posts: women: 29.0 ± 1.2, men: 29.6 ± 1.5; replies: women: 40.1 ± 0.9, men: 38.7 ± 2.1). Women had more profile images (women: 8.5%, men: 5.6% (no significantly different from women, p = 0.43), undisclosed: 0.8%, p < 0.01 from women, p < 0.05 from men), (Fig. 2) and posted more (women: 32.6 ± 12.9, men: 12.7 ± 4.7 (not significantly different from women, p = 0.91), undisclosed: 4.8 ± 1.1, p < 0.05 from women, p = 0.06 from men).

Fig. 2

Gender differences in self-investment according to profile images, emoticon use and word count.

Gender differences in self-investment on the forum were evidenced by the use of profile images (full bars), the use of emoticons (empty bar), and the number of words in initial posts (circles, mean ± SEM). Significant differences were observed for profile images between undisclosed gender and women (p = 0.003) and men (p = 0.04), for emoticon use between undisclosed gender and women (p < 0.001) and men (p = 0.003). Word count was significantly higher for women (p < 0.001) and men (p = 0.01) compared to undisclosed gender.

Initial posts' response rate was similar between women (3.3 ± 1.1) and men (2.2 ± 0.4, not significantly different, p = 0.46). Women wrote longer messages (158.6 ± 8.6 words; men: 123.2 ± 7.5 words, p < 0.001; undisclosed gender: 104.2 ± 4.9, p < 0.001 from women, not significantly different from men, p = 0.65). This was true both for initial posts (women: 253.3 ± 30.2 words; men: 202.0 ± 21.8 words; undisclosed gender: 133.4 ± 8.4 words, p < 0.001 from women, and p = 0.01 from men), and replies (women: 130.6 ± 6.2 words; men: 105.6 ± 7.5 words; undisclosed gender: 80.2 ± 5.0 words, p < 0.001 from women, not significantly different from men, p = 0.08).

Striking differences were observed with regards to the use of emoticons between authors not disclosing their gender and authors declaring their gender (women: initial posts: 19.9%, replies: 16.6%; male: initial posts: 11.1%, replies: 3.7%; undisclosed gender: initial post: 1.5%, (p < 0.001 from women, p < 0.01 from men), replies: 0.6%, (p < 0.001 from women, p = 0.05 from men), Fig. 2). While women's use of emoticons was stable across initial posts and replies (not significantly different, p = 0.35), the use of emoticons by men considerably dropped for the replies (p < 0.01; significantly lower than emoticons used by women in replies p < 0.001).

Authors' motives

Overall, 81.5% of posts were written “for self”, with no significant differences between women and men (82.2% and 91.8% respectively). The remaining 18.5% were written for others (including 80.3% for a child). More women were posting for children (women: 14.5%, men: 4.1%, p < 0.05). Whatever the gender, initial posts were seeking both medical information (79.6%; women: 77.0%, men: 74.0%, undisclosed gender: 85.5%, p < 0.05 from gendered posts), and community support (63.7%; women: 70.4%, men: 65.7%, undisclosed gender: 55.0%, p < 0.01 from gendered posts, Fig. 3). More posts written for a child were seeking medical information (82.7%) than community support (55.8%, p < 0.01).

Fig. 3

Motives of initial posts as a function of the degree of self-disclosure.

All initial posts were stratified according to the degree of self-disclosure (whether authors declared their gender, their age, or a combination of both). For each category, the motives of the posts (seeking medical information or seeking community support) were identified (discussion board presence). Lines indicate the groups between which statistical significance has been evidenced. Asterisks indicate p values (* for p < 0.05, ** for p < 0.01, *** for p < 0.001).

Initial posts seeking community support were longer (250.2 ± 21.2 words) compared to medical information (171.6 ± 8.9 words, p < 0.001), but elicited the same amount of replies (community support: 3.2 ± 0.8 replies, medical information: 2.6 ± 0.6 replies, not significantly different, p = 0.31).

Replies were shorter than the initial posts (114.3 ± 4.1 words compared to 198.4 ± 14.2 words, p < 0.001) and included peer support (75.6%), adequate recommendations (39.9%), requesting further information (11.5%), and erroneous information (2.2%), (Fig. 4). No replies included dangerous information. Within the replies, women shared more peer support (86.2%; undisclosed gender: 75.8%, p < 0.01 from women; men: 59.1%, p < 0.001 from women, p < 0.001 from undisclosed gender), whereas men were giving more adequate recommendations (60.4%, women: 34.4%, p < 0.001 from men, undisclosed gender: 14.9%, p < 0.001 from women, p < 0.001 from men). Requests for further information were mainly coming from authors not disclosing their gender (women: 8.4%; men: 7.9%, not significantly different from women, p = 0.82; undisclosed gender: 28.6%, p < 0.001 from women, p < 0.001 from men). There were no differences regarding erroneous information (women: 2.5%; men: 2.1%; undisclosed gender: 1.2%).

Fig. 4

Author's motives as a function of posters' gender for initial posts and replies.

Proportions of post authors according to their gender (female, male, or undisclosed) for initial posts (superior area) and replies (inferior area), for each of the identified motives (community support request (A) or medical information request (B) for initial posts) or type of replies (community support (C), further information request (D), adequate information (E), and erroneous information (F) for replies). Asterisks indicate p values (* for p < 0.05, ** for p < 0.01, *** for p < 0.001).

Discussion

In the last decades, high demands on physicians and specialists have led to shorter consultations. As a consequence, many patients leave without properly understanding what they were told (Bodenheimer, 2008). Logically, patients migrate to the Internet to fill this gap and gain a better control over their health status (Bhandari et al., 2014).

Since patient health literacy has critical health consequences (American Medical Association, 1999, Kutner et al., 2006), many medical association web sites have introduced education materials as a way for patients to improve their health outcomes. However, in accordance to previous studies (Hansberry et al., 2014a, Hansberry et al., 2014b, Eloy et al., 2012, Kasabwala et al., 2012, Cherla et al., 2012, Cherla et al., 2013, Svider et al., 2013, Wang et al., 2013, Alamoudi and Hong, 2015, Zellmer et al., 2015), our results show that online available resources exceed anticipated comprehension of the majority population, and score poorly in terms of understandability and especially actionability. This does not necessarily mean that the quality of the information itself is not good (in fact, almost not erroneous and/or dangerous information was detected in our sample). Rather, it means that the way information is provided to users – who are potential patients or patients' relatives – is not adapted, or at the very least not optimized.

Obviously, this discontinuity between the patients' needs and the available information has an impact on users' satisfaction. This discontinuity bears the risk of seeing the patients migrate to uncontrolled, poorly mediated resources such as blogs and forums. While this was suspected, our results demonstrate this phenomenon; indeed, the majority of threads originated from information seeking, while, due to their structures, forums should be more suited for peer support. Moreover, medical information seeking was associated with a lower degree of author's self-disclosure, emphasizing the dichotomy between this behavior and optimal social communication which would require a certain amount of self-disclosure. Studies so far were unable to fully document the behavioral dynamics of online patients' exchanges. Indeed, most studies have focused either on gender specific cancer forms such as prostate and breast cancer, therefore bringing heavy gender-related issues in the analysis (Klemm et al., 1999, Owen et al., 2004, Blank and Adams-Blodnieks, 2007, Gooden and Winefield, 2007, Blank et al., 2010), or on chronic diseases such as Parkinson's, fibromyalgia, and diabetes (Attard and Coulson, 2012, Chen, 2012), and therefore studying patient (virtual) communities structured on a long-term perspective rather than instantaneous or pseudo-synchronous information seeking individual strategies. By choosing tonsillectomy as a model, we were able to eliminate these two potential biases.

Despite the fact that a third of the forum contributors did not disclose their gender, more female presence was noted, consistently with previous studies (Bhandari et al., 2014). Interestingly, women presented more self-investment than men, as seen with the proportion of user having a profile image, the use of non-verbal cues such as emoticons, and by the length of their posts, in line with the hypothesis that women make a heavier use of multi-modality in avatar-based communication (Lomanowska and Guitton, 2012). However, while the utilisation of the Internet is usually associated with younger population, surprising results came when considering the age of the authors. Indeed, the respondents were significantly older than original posters. This was quite unsuspected, and has interesting consequences regarding the potential quality of the interactions as well as of the quality of the information shared. Indeed, while in a classical forum interactions are supposed to take place between peers, our results may suggest that, in the context of health related forum, interactions might occur between individuals with different life experience. This age and experience difference between original posters and forum members answering their queries is likely to contribute to one of the main functions of health-related forums, which is peer-to-peer support. With the observed age difference, the support is coming not only from equivalent peers, but also from other people who already “went through” the experience of this type of surgery.

While it is a common belief that men frequenting online discussion boards are more prone to seek information in comparison to women who are believed in having a stronger tendency toward community support (Klemm et al., 1999, Zakowski et al., 2003), the situation is far more complex (Mo et al., 2009). Although research has found gender differences in styles of communication between male and female cancer forums (Zakowski et al., 2003, Owen et al., 2004, Gooden and Winefield, 2007, Mo et al., 2009), differences within mixed gender communities as well as between informational and emotional support were less evident (Mo et al., 2009). Our results document this phenomenon further, and shed a new light on some of the literature controversies. Our results strongly suggest that “support” is not a monolithic behavioral motive, but that it should be understood has a biphasic entity, with in the one hand “support seeking”, and in the other hand, “support offering”. While both genders (males and females) display support-related behavior, gender differences occur within this biphasic reality. Indeed, while there were no difference between men and women searching for either community support or medical information, women offered more support in the latter replies. Therefore, while the initial needs were identical for both men and women, reactions were differential depending on the gender of the author: men providing more accurate factual information, while women offering more support, echoing conventional gender roles. Whether this was cultural or based on specific cognitive mechanisms remains unknown. Nonetheless, this should be taken into consideration when developing patient-oriented online resources.

Surprisingly, very few erroneous or dangerous information or suggestion were observed in the forums. This was probably due to the implication of both the community and the moderators, with inappropriate posts being potentially deleted before analysis. However, it is important to note here that the fact that none (or almost none) false information was observed cannot be used as a single proxy to predict the quality of the information provided. Indeed, the degree of completion of the information is an important element to consider – particularly when talking about decision-making process between different medical or surgical options. Furthermore, when taking a patient's point of view, actionability of the provided information is a key issue. And clearly, actionability of the observed information was low. Nonetheless, the present data suggest that community-based online health forums might be less harmful than usually considered by the health professionals.

A few factors may still limit our current analysis. Given the fact that data were extracted in a public forum, there was no way to insure that the self-disclosed age and gender, nor that the user name was related in any way to the real identity of the users. However, given the topic of the forums and the fact age and gender were disclosed on a voluntary basis, there is no immediate reason to believe that users disclosing them would provide false information. Although factors such as login and browsing times or login frequency are unlikely to be of importance in the context of forum-based asynchronous communication, elements such as users education, income, or region of origin might potentially represent confounding factors, which should be investigated in future studies.

In conclusion, the use of Internet may contribute to reduce gaps in health knowledge and help patients improve their health literacy (Brossard, 2013). While there is still a clear need for more adapted and controlled resources, online platforms have the potential to effectively convey information to patients. Working toward a global view of the health-related online information continuum may be the first step in recognizing contemporary patient needs. Analysis of patients' interactions and dynamics on health-related forums demonstrates that when taken as a community, patients are less naive and more resourceful than when considered individually. While Internet can be a powerful tool for physicians and patients alike to convey useful health-related information and to empower individuals, much work is still needed to optimize this information gathering process in order to relieve healthcare access barriers.