Improving Teen Girls' Skills for Using Electronic Health Information.

BACKGROUND: In a world increasingly dependent on the Internet for information, it is not surprising that people use the internet to find answers to their health-related questions. Research has shown that teen girls are more likely to search for health information online than boys, but that they do not feel confident in using the information they find.
OBJECTIVE: To address this disparity, teen girls were engaged in the process of developing a teen-friendly, internet-based tool that explains the best way to find, evaluate, and use online health information.
METHODS: Focus groups were held with girls and their parents to inform the design of the tool. After collaborating with information technology, marketing, and video production teams, a tool was developed consisting of a webpage, videos, and an interactive game. The efficacy and acceptability of the tool were tested among our target demographic through a usability trial. KEY
RESULTS: Parent and teen focus groups informed the three-step design of the tool. Teen girls reported significantly higher levels of eHealth literacy after using the tool. Dissemination of the tool through a national targeted ad campaign generated web traffic to the tool.
CONCLUSIONS: An internet-based training tool has been shown to improve teen girls' eHealth literacy. Findings from this developmental study can be used to inform efforts to improve eHealth literacy in adolescent girls. [HLRP: Health Literacy Research and Practice. 2021;5(1):e26-e34.] PLAIN LANGUAGE
SUMMARY: This research study used teen and parent input to help design an internet-based training tool to teach teen girls the best way to use the internet to find health information. Teen girls' ability to find, understand, and evaluate online health information significantly improved after using the tool. The tool was advertised throughout the United States on various social media platforms. ©2021 Zaim, Keedy, Dolce, et al.; licensee SLACK Incorporated.

In the United States, 92% of teens use the internet every day (Lenhart, 2015). Searching for health information is one of the most common online activities; teens often look online for information about specific diseases, treatments, diet and exercise, and mental and sexual health (Bansil et al., 2006; Borzekowski & Rickert, 2001; Lenhart et al., 2005). Beyond information searches, teens also access electronic health (eHealth) information through social media sites, smartphone applications, video sharing platforms, or electronic patient portals. First defined in 2001, eHealth has evolved to encompass health information and services distributed through the internet and other communication technologies (Cunningham, 2016; Eysenbach, 2001; World Health Organization, 2020).

Health literacy is defined as the degree to which people have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health care decisions (Ratzan et al., 2000). A vital component of this, eHealth literacy encompasses the ability to find, understand, and evaluate online health information (Norman & Skinner, 2006). However, only 40% of girls believe they can recognize high-quality eHealth resources and nearly 60% do not feel confident in their ability to use eHealth information to make decisions (Chisolm et al., 2011). Research shows that this lack of confidence extends into adulthood, and that women are more likely than men to search for health information online (Escoffery, 2018). Furthermore, literature suggests that African Americans are less willing to use technology to access health information than their White counterparts, and perceiving websites as secure has been found to increase eHealth use in a population of African American cancer survivors (Gordon & Hornbrook, 2016; Senft et al., 2019; Veinot et al., 2013). Therefore, it is crucial to target consumers of health information from the internet who are from racial and ethnic minority groups through a training tool that addresses the proper way to find eHealth information.

Teens have been found to use search engines as a primary starting point for searching for eHealth information, but they do not feel confident in their search skills (Jiménez-Pernett et al., 2010). Previous research has shown that students exposed to a reliable medical search engine had higher eHealth literacy scores than those without any exposure (Ghaddar et al., 2012). Teens felt more confident searching for health information online when they had been given information about how to do it in a productive manner. In a world increasingly dependent on the internet for information, eHealth literacy is crucial, as some websites that claim to be educational may actually be commercial, misleading, dangerous, or inaccurate.

This developmental study engaged teen girls in the process of designing a teen girl-focused, interactive, internet-based eHealth literacy training tool (Nationwide Children's Hospital, 2018). The tool is meant to improve teens' skills in finding health information online, proficiency in evaluating the quality of health information, and the ability to use health information in conjunction with health care providers to make informed health decisions. The tool was evaluated through a survey-based usability trial to determine if use of the tool improved eHealth literacy among teen girls. In addition, the tool was publicized through a national paid advertising campaign. It was hypothesized that use of the tool would lead to improvements in eHealth literacy 30 days after use.

Methods

Recruitment and Consent

All project participants were recruited from a database of past research participants who had expressed interested in new projects, along with the distribution of flyers throughout clinics at Nationwide Children's Hospital (NCH). For the purpose of project recruitment, anyone who identified as female would be allowed to partake in all research activities. Research activities were approved as human subjects research by the NCH Institutional Review Board, and verbal consent was obtained from all participants.

Focus Groups

Two focus groups, one consisting of girls age 13 to 18 years and the other consisting of their parents, were held in the autumn of 2017 to gain insight into how teens use the internet when seeking health information. A discussion guide was developed consisting of questions asking participants to describe their experiences when searching for health information online. They were also asked to provide input into the design of an internet-based eHealth literacy training tool. A trained facilitator led the focus groups, and a member of the research team (H.Z. or H.K.) was present to take notes. Discussions were audio-recorded. Upon completion of focus groups, two members (H.Z. and H.K.) of the research team listened to recordings individually and compiled summaries in conjunction with written notes taken during the sessions. The two researchers compared session notes, and key themes were identified for each focus group through discussion and the selection of notable excerpts (Table and Table ).

Design of the Internet-Based Tool

Along with NCH marketing and information technology teams, and an outside videography team that specializes in storytelling, the research team used the main themes to inform the design and content of an internet-based tool (Table ). The tool, entitled “eTeen Health,” consists of a webpage, a video, and an interactive game focused on teaching girls how to use the internet to appropriately find health information (Nationwide Children's Hospital, 2018). Two teen girls aided in designing the tool and four teen girls are featured in the video to ensure that it appeals to our target demographic.

Usability Trial

A survey-based usability trial was completed in the summer of 2018 to measure the effectiveness of the tool on increasing eHealth literacy and the acceptability of the tool among teen girls. Thirty healthy girls ranging from age 13 to 18 years were recruited from a database of past research participants and through flyers posted in clinics at NCH. They were emailed a link to the surveys and the internet-based tool. Questionnaires included a pre-training survey before using the tool, a satisfaction survey immediately after, and a post-training survey 30 days later.

The pre-training survey contained questions on demographics, as well as assessments of health literacy and eHealth literacy. The satisfaction survey evaluated the girls' perceptions of the tool, internet usage, and technology fluency. The post-training survey re-evaluated the girls' eHealth literacy to ascertain how well the participants recalled information they learned from using the tool. eHealth literacy was assessed using the eHealth Literacy Scale (eHEALS), an eight-question measure used to evaluate eHealth literacy with two supplementary questions to determine general interest in eHealth (Norman & Skinner, 2006). Responses to each question in the eHEALS are based on a 5-point Likert scale ranging from strongly agree to strongly disagree. eHEALS scores can range from 8–40, and higher scores on the scale indicate better eHealth literacy.

Ad Buy

An ad buy, or a paid advertising campaign, was used to publicize the tool during Health Literacy Month (in October 2018). The videography team filmed four 20- to 30-second pre-roll clips featuring an adolescent medicine clinician. Our two teen girl project aides helped in selecting Twitter, Facebook, Instagram, and YouTube to promote our pre-roll videos; the aides claimed that these outlets were the most popular among their friends and classmates. The NCH marketing team used Google Analytics to measure website traffic, along with built-in metrics tools through Facebook, Twitter, and YouTube to assess performance of the advertising campaign.

Results

Focus Groups and Design of the Internet-Based Tool

The results of the teen and parent focus groups are discussed below, along with how those results were used to inform the design and content of the internet-based tool.

A total of 20 teen girls participated in the focus groups. Most participants were African American (70%) and the average age was 16.1 years (Table ). Four major themes were identified: confusion, convenience, behavior change, and education (Table ). These themes informed the design and content of the internet-based tool (Table ).

A recurring theme through the teen focus group discussions was confusion. Teens reported confusion due to the vast amount of health information posted online. They relayed feeling overwhelmed by this, and even stated incidences of finding contradictory information during an internet search. In line with this, the girls expressed that they wanted the information in our tool presented in a clear and direct way. Accordingly, the tool lays out three easy steps to follow when looking for health information online: “Search It,” “Test It,” and “Use It.” The girls also thought that highlighting other teens' experiences with finding information online would help them cultivate a connection with girls their age and relay the information in a straightforward, teen-friendly way. Therefore, the tool includes a video featuring four teen girls describing the steps and sharing an anecdote about searching for online health information.

Another prominent theme throughout the teen focus groups was behavior change. Girls claimed that at times they change their behaviors based on eHealth information, particularly when it comes to their diet. For example, after reading information online about food nutrition, many teen girls will alter their diets. When asked directly for input on what would best encourage them to use information from the tool, girls claimed that they liked online games or quizzes to reinforce a new skill. Therefore, the tool emphasizes the second step of finding health information, “Test It,” through an interactive game in which girls can practice identifying features on a mock website (e.g., a labeled advertisement, a website with an “.edu” domain name url) to determine if the website is trustworthy. The interactive game reinforces the information relayed on the website and encourages behavior change in safer internet use.

Education was another significant theme pulled from the teens' discussions. Teens use the internet to educate themselves on health information. Many stated that they like the autonomy of using the internet to learn about health on their own without having to rely on their parents. It was important when creating the tool to not only teach teen girls how to find accurate health information online, but to also emphasize the importance of bringing the information to a trusted adult or health care provider. This information is highlighted through the third step for finding eHealth information: “Use It.” The tool directs girls to talk with their parents, school nurse, or other trusted adult about online health information they find. It also teaches girls how to use the hospital's patient portal (“MyChart”), to send a question to their doctor through an online instant messaging platform.

Finally, a frequent theme from the focus groups was convenience. Teens said that the internet is a convenient way to find health information. Although girls agreed on the importance of using trustworthy websites, they concluded that convenience (i.e., selecting the first website listed during a Google search) exceeds accuracy (i.e., taking the time to select a reliable website) while searching for health information. Therefore, the tool highlights the first step to finding health information: “Search It.” This step teaches teens to begin their internet searches with a trusted health website as opposed to using a generic search engine to avoid having to click through several links before finding the most accurate information. The participants primarily used their smart phones as a quick and convenient way to access the internet; therefore, the tool is mobile-friendly.

One parent focus group was held with 13 participants. Most of the participants (61.5%) were between ages 35 and 44 years and had completed some college or a 2-year degree (53.8%) (Table ). Three major themes were identified from the parent focus group: accessibility, trust, and fear (Table ). Again, these themes were used to inform design and content of the internet-based tool (Table ).

Accessibility was a major and recurring theme throughout the parent focus group. Accessibility and convenience when using the internet made it a first step for many parents, and they stated that their children also prefer using the internet for similar reasons. Parents agreed that their children were more likely to access a webpage if its associated with a health system familiar to them. Consequently, the “Search It” step reinforces the importance of starting a search off with a trusted source, and the tool is housed on the Nationwide Children's Hospital website (Nationwide Children's Hospital, 2018). Also, the tool is mobile-friendly.

Another common theme from the parent discussion was trust. Parents stated that it is difficult to determine which websites are trustworthy and accurate; their lack of trust in the internet lead them to use it solely as a starting place when accessing health information. Consequently, they expressed a lack of trust in their children correctly using the internet for health information. This reinforced the need for our tool to teach teens to look for authors, references, dates, and review boards while searching online for health information. This information is relayed through our second step: “Test It.”

Finally, fear was a prominent theme from the parent focus group. Parents discussed how finding health information online can instill fear about having a particular condition. They also experienced fear when information was found after hours and there is no doctor available to call. For this reason, parents agreed that they should remain involved in the conversation with their teens about online health information. To reinforce this, the tool highlights a third step, “Use It,” which directs teens to talk with their parents, doctors, or a trusted adult about all of the health information that they find online.

Twenty-seven of the 30 teen girls completed all three surveys assessing the usability and impact of the designed tool. Participants ranged in age from 13 to 18 years, with an average age of 16.1 years (standard deviation 1.7 years) (Table ). The sample was racially diverse with 40.7% identifying as White and 33.3% as African American. The remaining 26% selected Asian, Native Hawaiian, American Indian, or Multiracial when asked about race. Participants' education levels ranged from eighth grade to having a high school diploma or GED. In the pre-training survey, 85.2% of the teen girls indicated that they use their phone to look up health information online, and the majority look up information about specific diseases or medical problems. Each participant had a cell phone with internet access, and most had at least two or more personal electronic devices (i.e., a laptop computer, a game console, a tablet) with internet capability.

In the satisfaction survey, participants responded that they liked the tool (92.6%), found the tool useful (88.9%), and agreed that they would recommend sharing this tool with a friend (93.5%). They also thought they would use what they learned from the tool in the future (90.4%).

The average eHEALS score prior to piloting the training tool was 27.1 (6.0). The teen girls had a significant increase in average post-training eHEALS score to 32.6 (4.6, p < .00) (Table ). There was a significant increase in the average response score for each individual measure in the eHEALS at the post-training survey as well. Before using the tool, the average response to “I know how to use health information I find online” was 3.4 (1.1) on a scale of 5, but after using the tool the average score was 4.2 (0.7, p = .01). The participants' understanding of eHealth changed according to the supplementary eHEALS questions. Before using the tool, the average score for “how useful is the internet in helping you make health decisions” was 3.4 (0.8), and after using the tool it increased to 3.9 (0.7, p = 0.01). A similar increase was seen in participants' answers to “how important is it for you to be able to access health resources on the internet.” The average score for participants before using the tool was 3.8 (0.9), and the average score increased to 4.2 (0.8) after using the tool.

The paid advertising campaign ran nationally from September 15, 2018 to October 31, 2018. The tool was advertised on four paid channels: Facebook, Instagram, YouTube, and Twitter. In that span of time, there were a total of 12,596 visits to the tool. Ads on Facebook and Instagram were the most successful, driving the most amount of website traffic during our paid advertising campaign. However, of all the paid advertising channels, YouTube drove the highest number of views of the 2-minute video.

Discussion

This study tested the impact of a web-based interactive tool specifically designed for teen girls to improve eHealth literacy. Teen girls regularly use the internet to find information about their health; however, they report difficulty with establishing the reliability of online health information. The internet-based tool taught teens how to search for, evaluate, and use online health information appropriately. The tool included web content, an instructional video with lessons and testimonials from other teen girls, an interactive game, and short video clips from an adolescent medicine physician to direct teens to the web-page during the targeted ad buy period. Teen and parent focus group input was critical in developing the tool and identifying platforms for the ad buy, as relatability to teens was a top priority.

Other websites do exist that educate people on how to appropriately navigate the internet when searching for health information (Medline Plus, 2019; National Institutes of Health, 2018). Although these sites provide valuable insight on the topic, they are not tailored for teen girls. Other health-related sites and interactive online tools created for teens have been shown to positively change behaviors (Mirman et al., 2014). In addition, teen girls have been shown to have positive responses to a web-based tool outlining information about contraceptive use, including the use of videos sharing peer experiences (Gressel et al., 2014). To our knowledge, this is the first online tool focused on improving eHealth literacy specifically tailored for teen girls.

Overall, teens responded positively to the tool and had significant increases in eHealth literacy after using it. In fact, the eHEALS item with the largest improvement from pre-training to post-training was “I can tell high quality health resources from low quality health resources on the internet.” The three steps in the tool outlined skills needed to find and evaluate reliable health information websites. Teens felt more confident applying these skills after using the tool. This demonstrates the ability of an internet-based interactive tool to potentially help teen girls become wiser and more efficient consumers of online health information. This provides them with the ability to improve their engagement with the health care system, especially as health information is increasingly communicated through the internet (Stellefson et al., 2011).

Teens reported the least increase in the eHEALS measure “I know how to use the health information I find on the internet.” The use of health information after evaluating its quality was not the focus of the tool. Teens were encouraged to bring the information to a trusted adult or doctor. This lack of focus may have limited the teen girls' improvement in this particular area of eHealth literacy. Adults have also been found to have the lowest scores for this eHEALS component in previous work (Escoffery, 2018).

Study Limitations

Limitations of this study include relying on a convenience sample to recruit teen girls for the usability trial, which limits generalizability because those who volunteered for the trial may have more motivation or interest in learning how to improve eHealth literacy than those chosen through random assignment. Additionally, although the ad buy resulted in more than 12,000 visits to the tool, we do not have information about the specific audience that was reached; it is possible that girls who already visit reliable websites were inadvertently targeted during the campaign. Another limitation is that feedback on the tool development was primarily obtained through teens and parents from urban areas, which may not reflect the needs and preferences of those in other settings. Nevertheless, this preliminary study shows that developing an internet-based tool with teen and parent input is feasible and can increase teen eHealth literacy.

More research is needed to determine if results are reproducible with a larger group of teen girls and with teen girls from more diverse backgrounds. Findings from this developmental study can be used to inform a larger-scale clinical trial about improving eHealth literacy for adolescent girls. Measuring sustained changes in eHealth literacy past 30 days could also help to improve the impact of similar training tools (McLuckie et al., 2014). Determining more effective ways to disseminate the tool using targeted ad buys to increase the number of teens who are directed to the internet-based tool would also be useful. Behavioral health conditions such as anxiety, depression, and eating disorders disproportionally affect teen girls (Kessler et al., 2005; Merikangas et al., 2010); thus, future research should examine whether a similar internet-based tool could improve eHealth literacy for girls facing these specific health issues.

Table 1

Teen Focus Group Themes

Theme	Quotes
Confusion	“I will sometimes ask the doctor, but when it is something that don't sound right, and I feel like I am going to say it wrong then I am gonna wait [and go online].”
	“You spend a lot of your time doing the things that don't work, thinking that they will. Then you finally get to the things that do work and everything is worse. And so, it takes you a long time to figure out how to take care of yourself.”
	“I think it's really bad, cause it's like, say you're looking up like ‘Oh, why do I got a pimple on my neck?’ and then like the first result is ‘Oh, like you got like lung cancer or something.’ It can be something that is completely off.”
Convenience	“Since my phone is like my best friend, I am usually on my phone and I will go to like the National Institutes of Health because I can actually sorta trust them to be correct and everything.”
	“When I want information…what I find annoying is that I have to put a request in for that.”
	“[Searching for health information] is like easy. [Your phone] is right there in your hand. You know, you get so used to it and you do it without thinking sometimes.”
Behavioral change	“[Websites] say that eating different foods will do this and I won't eat it or I will cut it out completely from my diet.” “I watched a video about how different things are bad for you. It kind of changed my mind about what I was eating.”
Education	“I feel like I need to start knowing all my information and stuff like especially when it comes to my shot regimens and stuff I need to know. Rather than me having to call my mom all the time.”
	“When you get a shot they give you information about it. It has websites and stuff like that, but I never felt like they stressed it, like stressed the importance of [the websites].”

Table 2

Parent Focus Group Themes

Theme	Quotes
Accessibility	“I Google everything.”
	“I think the easiest thing is you can just type in your symptoms and it always gives you some results.”
	“It is so easy now to go online.”
Trust	“I never fully trust the internet. I just use it as a starting point. Like is this something I should be concerned about right now or is this something that can wait.”
	“I think you want to trust your kids with everything but at the same time, they are kids.”
Fear	“I mean it concerns me because it concerns us as adults, some of the stuff we find. So, I am trying to imagine my 15- or 16-year-old looking at this stuff not knowing as much as I do and literally freaking out.”
	“It can be overwhelming. You punch in your symptoms and you have like five or six different possibilities of what it is and if it is after hours you can't really call your doctor.”

Table 3

Themes Informing Tool

Group	Theme	Design and Content of Web-Based Tool
Teen focus group	Confusion	Three clear steps to find eHealth information
		Video featuring other teen girls
	Convenience	“Search It” – teach girls to begin start at a trusted site
		Mobile-friendly
	Behavior change	“Test It” – interactive quiz
	Education	“Use It” – teach teens to take information to a trusted adult and message doctors through patient portal
Parent focus group	Accessibility	“Search It” – housed on Nationwide Childrens' Hospital website
		Mobile-friendly
	Trust	“Test It” – tell teens to look for a website's authors, references, dates, and review board
	Fear	“Use It” – teach teens to talk to parents or message doctors through patient portal

Table 4

Sample Demographics

Demographic	Parent Focus Group, n (%)	Teen Focus Group, n (%)	Usability Trial, n (%)
Sample size, N	13	20	27
Age, years
13–18	-	6.1 (1.7)	16.1 (1.7)
35–44	8 (61.5)	-	-
45–54	5 (38.5)	-	-
Race
Black	6 (46.2)	14 (70)	9 (33.3)
White	5 (38.5)	5 (25)	11 (40.7)
Asian, Native Hawaiian, American Indian, or Multiracial	2 (15.4)	1 (5)	7 (25.9)
Education
8th grade or less	-	4 (22.2)	5 (18.5)
Some high school	2 (15.4)	11 (61.1)	17 (63)
High school diploma or GED	1 (7.7)	3 (16.7)	5 (18.5)
Some college or a 2-year degree	7 (53.8)	-	-
College graduate or higher	3 (23.1)	-	-
School type
Public	-	16 (80)	20 (74.1)
Private	-	1 (5)	6 (22.2)
Homeschool	-	1 (5)	1 (3.7)
Other	-	2 (10)	0 (0)

Table 5

eHEALS Measures

Measure	Pre-Training Score, M (SD)	Post-Training Score, M (SD)	p Valuea
Overall eHEALS (8–40)	27.1 (6.0)	32.6 (4.6)	< .01
Primary eHEALS questions
I know what health resources are available on the internet	3.2 (1.1)	3.8 (0.7)	0.02
I know where to find helpful health resources on the internet	3.3 (1.0)	4.0 (0.9)	0.01
I know how to use the health information I find on the internet	3.4 (1.1)	4.1 (0.9)	0.01
I know how to use the internet to answer questions about my health	3.7 (0.9)	4.2 (0.7)	0.01
I know how to use the health information I find on the internet	3.5 (1.0)	3.9 (0.9)	0.14
I have the skills I need to evaluate the health resources I find on the internet	3.6 (1.0)	4.2 (0.7)	0.01
I can tell high quality health resources from low quality health resources on the internet	3.3 (1.1)	4.5 (0.8)	<0.01
I feel confident in using information form the internet to make health decisions	3.2 (0.9)	3.9 (0.7)	<0.01
Supplementary eHEALS questions
How useful is the internet in helping you make health decisions?	3.4 (0.8)	3.9 (0.7)	0.01
How important is it for you to be able to access health resources on the internet?	3.8 (0.9)	4.2 (0.8)	0.07

Note. eHEALS = eHealth Literacy Scale.

Wilcoxson Signed Rank Test.