The Young Men and Media Study: A Pilot Randomized Controlled Trial of a Community-Informed, Online HIV Prevention Intervention for 14-17-Year-Old Sexual Minority Males.

The Young Men and Media study developed and pilot tested a community-informed, online HIV prevention program for adolescent sexual minority males (ASMM) in the United States. The developed intervention uses nine interactive modules to increase sexual health knowledge, promote critical examination of pornography, and decrease sexual risk among ASMM. Participants (N = 154, age 14-17 years) were recruited online in Spring 2020 and randomized to the intervention (n = 77) or other existing HIV websites (n = 77). Of the 65 intervention participants who logged in to the website, most completed all nine modules and found the content useful (average module score 4.3 out of 5 stars). The intervention also showed improved HIV/STI knowledge, increased pornography knowledge, and reduced beliefs that pornography is an accurate depiction of male-male sex. Results indicate that the Young Men and Media intervention is feasible, acceptable, and may positively impact sexual health outcomes.

Introduction

Adolescent sexual minority males (ASMM) are disproportionately impacted by HIV in the United States (U.S.; [1, 2]). In 2018, 21% of new HIV diagnoses were among 13–24-year-olds; an alarming 1 out of 5 of those diagnoses were among 13–19-year-olds [2]. Among male adolescents, ASMM account for 92% of new HIV infections [3]. Despite concerted efforts to understand factors driving HIV infection among ASMM [4], incidence rates have remained steady for the past decade [5]. Developmentally appropriate prevention interventions designed specifically for ASMM are needed.

Despite this need, few HIV prevention interventions have been designed for sexual minority males younger than 18 years of age. Currently under evaluation, the SMART program is a stepped-care package of increasingly intensive eHealth interventions targeting HIV-related sexual behaviors among sexual minority males 13–18 years old [6]. Queer Sex Ed, a predecessor to the SMART program, is an online sexual health intervention for sexual and gender minority adolescents ages 16–20 years that teaches users about condom use, coming out, sexual identity, healthy relationships, and sexual pleasure while providing users with resources for local HIV testing and birth control [7]. MyPEEPS uses a group-based, in-person format to address sexual health and minority stress among sexual minority males ages 16–20 years [8]. Guy2Guy is a text messaging HIV prevention intervention for ASMM 14–18 years old designed to increase HIV knowledge, motivation for condom use, and behavioral skills for engaging in condom use [9]. Queer Sex Ed, MyPEEPS, and Guy2Guy have all demonstrated feasibility and acceptability as well as preliminary efficacy among young sexual minority males [7-9]. Although these interventions are promising, only Guy2Guy sampled ASMM younger than 16 years old. As health behavior patterns developed during early adolescence predict later health behaviors, including sexual behaviors, early intervention prior to sexual debut is critical [10]. Additionally, these interventions do not directly address online pornography use, which has been shown to be an important influence on ASMM’s sexual behavior [11, 12].

Because ASMM are unlikely to receive sexual health information relevant to male-male sex through traditional sexual health education channels (e.g., schools, parents; [13-15], they often turn to the Internet to learn about sex and sexual health [14, 16]. Online pornography has been cited by ASMM as an important source of sexual “education” and most ASMM view pornography regularly [12, 17–19]. Online pornography encompasses an array of media (e.g., images, video, live streamed content) with a wide range of producers (e.g., professional studios, individual performers, amateur performers; [20]). In addition to using pornography during masturbation, ASMM report using pornography to learn about important topics relevant to their sexuality, including to learn the mechanics of anal intercourse, explore their attractions and what they find arousing, and understand sexual minority culture [12, 16, 17]. However, the use of pornography for sexual health “education” can also be problematic. Pornography use has been associated with negative body image and increased symptoms of depression and anxiety among sexual minority men [21, 22], as well as lower sexual and relationship satisfaction among adolescents and young adult males [23, 24].

Most relevant to sexual health and HIV prevention among ASMM, pornography typically does not address the potential health implications of portrayed sexual behavior [11, 25]. For example, despite a trend toward more frequent depictions of condomless anal sex in male-male pornography, the health risks related to engaging in condomless anal sex [e.g., sexually transmitted infections, HIV in the absence of pre-exposure prophylaxis (PrEP)] are rarely addressed in pornography [26, 27]. Further, multiple studies have found that viewing condomless anal sex in pornography is associated with engaging in condomless anal sex in real life among sexual minority men, including among ASMM [12, 22, 28–31]. Given pornography’s relevance in the sexual development of ASMM and use as a source of sexual health education, helping ASMM develop “pornography literacy” skills (i.e., skills to become informed, active, and discriminating pornography consumers) is critical in HIV prevention programming [11, 12, 16, 17, 32].

The widespread use of the Internet among ASMM for sexual health information also means that it is well-suited as a media channel to reach ASMM for HIV prevention efforts [14, 16]. Online interventions can be an efficient, convenient, and salient platform through which to address ASMM’s HIV prevention needs [33, 34]. Indeed, several of the HIV prevention efforts developed for adolescent and young adult sexual minority men have used the Internet to either recruit participants or deliver HIV prevention intervention content [6, 7, 9, 35–38]. Therefore, we capitalized on the Internet’s suitability to develop and test an HIV prevention intervention for ASMM.

The current research, labeled the “Young Men and Media study,” developed and pilot tested a community-informed, online sexual health program for ASMM. The developed intervention uses interactive online modules to increase sexual health knowledge, promote critical examination of pornography, and decrease sexual risk among ASMM. In this report, we provide feasibility and acceptability data for the Young Men and Media intervention and examine preliminary efficacy on HIV prevention and pornography literacy outcomes (i.e., HIV/STI knowledge, PrEP awareness, condom knowledge, and pornography knowledge and beliefs).1

Methods

Recruitment and Eligibility

Participants (N = 154) were recruited from March 27, 2020 to May 8, 2020 via online advertisements and posts on social media sites (e.g., Facebook, Instagram). Eligibility criteria were: (1) age 14 to 17 years, (2) cisgender male, (3) self-identify as gay/bisexual, report being sexually attracted to males, and/or report having voluntary sexual contact with a male partner (past year), (4) have intentionally viewed pornography (deemed necessary because the intervention included a pornography literacy component), (5) reside in the U.S., (6) have a personal email address, and (7) be new to the study.

Social media advertisements/posts included visual images (e.g., GIFs) that featured young men and emoji. The written text described how participating in the study could help researchers develop and test a more inclusive online sexual health program. To optimize racial/ethnic diversity and increase the potential that advertisements/posts would be seen by males who are interested in male sexual partners, but not necessarily identify as gay/bisexual, advertisements/posts were broadly targeted to 14–17-year-old males in the U.S. who were identified by the Facebook/Instagram algorithms as interested in topics the study’s Youth Advisory Board (N = 4–5, 16–18 years old) and our own research generated as popular among racial/ethnic minority ASMM (e.g., Cardi B, RuPaul's Drag Race, Nicki Minaj, Beyoncé, Todrick Hall, Lizzo, Bad Bunny). To further increase recruitment of racial/ethnic minority ASMM, advertisements were also targeted using multicultural affinities (e.g., African American, Hispanic, Asian American).

Upon clicking on an advertisement or social media post, potential participants were directed to the study website, hosted with REDCap [40]; the website described the study and asked potential participants to respond to a series of eligibility questions. Potential participants who passed the initial screening progressed to consent material. Capacity to consent was confirmed via four questions that evaluated respondents’ ability (1) to name things they would be expected to do during the study, (2) to understand randomization procedures, (3) to explain what they would do if they experienced distress during the study, and (4) to identify potential risks of participation [41-44]. Specifically, participants were asked: (1) “If you agree to be in this study, what are we asking you to do?” (2) “How will it be decided which group of the study you are assigned to?” (3) “What can you do if you experience distress while taking part in this study?” and (4) “What are the potential risks of being in this study?” Respondents unable to answer all four questions after three tries were ineligible. Those who consented received an email containing a unique link to the baseline. To protect against fraudulent or duplicative enrollments, screening and baseline survey responses were cross-referenced using age (age vs. date of birth), location (zip code vs. state of residence), sexual activity (multiple questions across the screener and survey assessing sexual behavior), and email address [45, 46]. All procedures, including a waiver of guardian consent, were approved by The Boston Medical Center and Boston University Medical Campus Institutional Review Board. The study is registered at ClinicalTrials.gov (NCT04109443).

Procedures

Participants were assessed at baseline, randomly assigned to the Young Men and Media intervention (n = 77) or control (n = 77) and re-assessed at post-intervention and 3-month follow-up.

Randomization

After confirming eligibility, participants were randomized to intervention or control based on a permuted block randomization procedure, with small, random-sized blocks. Randomization assignment was given out via REDCap [40].

Young Men and Media Intervention

Participants randomized to the Young Men and Media intervention were sent an email with a link, log-in, and temporary password to the intervention website and asked to complete all intervention modules within three weeks of receiving the email. If a participant had not logged in, they were sent up to three reminder emails in a 10-day period.

A cross-sectional online survey of ASMM from across the U.S. (N = 207, 14–17 years old; [12, 14, 44]) and the Youth Advisory Board informed the design, content, and form of the intervention website. Website programming followed an iterative process from initial development/design to alpha/beta testing in collaboration with the Youth Advisory Board and the website developer. As outlined in Table 1, four main topics were identified by the study team, survey participants [12, 14], and the Youth Advisory Board as important foci for the intervention: (1) male anatomy, including information about how anal sex can be pleasurable and about anal health; (2) HIV/STI prevention information, including transmission risks, accessing testing, and condom use; (3) general sexual health information, including types of male-male sex, consent, dating safety, and partner communication; and (4) pornography literacy skills, including differences between pornography and reality, what’s behind the scenes on a pornography set, and normalization of pornography use among male youth. These topics were covered in nine distinct modules (see Table 2 for detailed module descriptions and screen shots). Intervention modules were interactive (e.g., games, videos, animations). The website had an interface like Netflix (i.e., each module had an icon with a short description of the content when a user hovered over it or clicked on it) and a responsive design (i.e., worked on a mobile device, tablet, or computer). Upon login, users had access to three modules (HIV/STI jeopardy, anal health/anatomy animated video, and the pornography set scavenger hunt game). Once participants completed the three initially unlocked modules, the remaining modules were unlocked. Participants could engage with the modules in any order and as often they chose. This module flow was recommended by the Youth Advisory Board and adopted by the study team to increase the likelihood that participants would, at a minimum, be exposed to the three initial unlocked modules that collectively cover all four areas of important foci identified above.

Table 1

Young men and media intervention module overview

Topics	Intervention modulesa
Male anatomy/anal health
Penis, anus, prostate anatomy	Anal health/anatomy animated video by blue seat studios
Fiber/anal douching	Question and answer video with an anal health expert
How to safely and comfortably have anal sex	Question and answer video with an anal health expert
How to use lubrication or lube	Question and answer video with an anal health expert
HIV/STI prevention
HIV/STI transmission 101	HIV/STI jeopardy game
How to access services (e.g., testing, doctors)	HIV/STI jeopardy game
How to use a condom	Anal health/anatomy animation by blue seat studios
How to talk with sexual partners about HIV/STIs	Choose your own dating adventure game
General sexual health
The types of sex you can have with a male partner	Sex menu: scrolling illustrations of behaviors (holding hands to anal sex)
How to pick partners	Choose your own dating adventure game
Communication with partners	Choose your own dating adventure game
Consent with sexual partners	Tea consent animated video blue seat studios
How to talk with sexual partners about what you would/wound NOT like to do sexually	Choose your own dating adventure game
Sex-seeking website/app etiquette/protection	Choose your own dating adventure game
Pornography literacy
What's omitted	Porn matching game
Behind the scenes	Porn set scavenger hunt game
Normalization of pornography use	Porn by the numbers infographic

aPlease contact the lead author to request access to the intervention website

Table 2

Detailed descriptions and screen shots of the young men and media intervention modules

Module	Description	Screen shot
Initital unlocked content
HIV/STD trivia	This game is based on traditional jeopardy (5 columns with 5 questions in each column). The questions cover the basics of HIV/STI transmission, how to prevent HIV/STIs, and how to access services. Example questions include: “What does HIV stand for?” and “What are ways you can protect yourself from getting HIV and other sexually transmitted infections?”
Porn set game	In this game participants click on items (e.g., the producer, STI test results, the lighting rig) on an illustrated porn set. There is a description for each clickable item which explains why it is there
Male-male anal sex 101	This animated video uses humor to cover basic anatomy of the penis, anus, and prostate as well as how to correctly put a condom on, the importance of talking with partners, and a brief introduction to anal health
Content unlocked after completion of initial unlocked content
Sex menu	This scrolling deck of images defines male-male sex by covering a wide range of behaviors including hand holding, cuddling, licking, masturbation, oral sex, and anal sex
Choose your own dating adventure	This game allows participants to play out different dating scenarios including choices related to where to meet partners, using alcohol or drugs on a date, whether to have sex or not, talking about HIV/STIs with partners, and talking about what you do and do not want to do sexually with partners. Participants are given feedback about their choices and how their choices relate to their HIV/STI risk and safety
Consent: it’s as easy as tea	This animated video uses humor to convey what does and does not constitute consent between sexual partners
Porn match	In this game, participants are given a list of sexual and relationship behaviors and asked to sort them into three buckets: (1) Shown in porn, but not very realistic (e.g., everyone has 6 or 8 pack abs, public sex); (2) Shown in porn and realistic (e.g., mutual masturbation, blow jobs without deep throating); (3) Not often shown in porn, but happens in real life sexual relationships (e.g., talking about relationships, getting tired, putting down a towel to keep bedding clean)
Porn by the numbers	This infographic (using data from the formative cross-sectional survey; [12]) provides basic statistics about how many 14–17 year old ASMM report viewing porn, how often they view, how many are masturbating while they are viewing, and the top three reasons for viewing

Control

Participants randomized to the control condition were automatically sent an email with links to the centers for disease control and prevention (CDC) HIV prevention and the national HIV and STD testing resource websites [47, 48]. They were encouraged to visit at least one of those sites within three weeks of receiving the email.

Post-Intervention and Follow-Up Assessments

Three and 15 weeks after baseline, participants received an email with a unique link to the post-intervention and follow-up assessments.

Retention

REDCap maintained an electronic system that notified participants via email when they were due for an assessment and reminded them up to three times within a 10-day period if they missed an assessment. In the baseline survey, participants also had the option of providing their phone number and indicating if the study team could leave messages. If participants had not completed assessments after their third reminder email, a research assistant called their phone number (if provided) to remind them of the assessment. Participants were compensated via electronic gift cards at the completion of the baseline assessment ($15) and after each of the follow-up assessments ($25 for post-intervention, $35 for 3-month follow-up). Participants were given a $20 bonus if they completed all three assessments.

Measures

Socio-Demographics

Characteristics included recruitment source, census region of the U.S. based on self-reported state residence, age, race/ethnicity, urbanicity based on participant ZIP code [49], and sexual orientation.

Study Feasibility

Measures included the percentage of individuals who completed screening, were eligible, consented, completed the baseline, and were retained for the post-intervention and 3-month follow-ups.

Intervention Feasibility

Measures included the percentage of participants randomized to the intervention who logged in to the intervention website, the number of reminders required for logging in, the number of modules completed, and the percentage of participants who completed each module.

Intervention Acceptability

Immediately upon completion of each module, the intervention website was programmed to ask participants to rate how useful they found the module (1 to 5 “stars”). At post-intervention follow-up, all participants were asked to complete a satisfaction questionnaire. Intervention participants were asked whether they visited the intervention website (yes, no). For those who reported visiting the intervention website, they were asked: (a) to rate the overall website quality (excellent/good, fair/poor); (b) whether they received the kind of sexual health information they wanted (yes, no); (c) how much of their sexual health needs were met (almost all/most, only a few/none); (d) if they would recommend the website to a friend (yes, no); (e) how satisfied they were with the amount of sexual health information they received (very/mostly satisfied, indifferent/quite dissatisfied); (f) whether the website helped them feel more prepared to have a sexual relationship with a male partner (yes, no); (g) how satisfied they were with the website (very/mostly satisfied, indifferent/quite dissatisfied); and (h) whether they would come back to the website if in need of sexual health information (yes, no). Participants in the control group were asked which of the control websites they went to, if any, and for each control website they reported going to they were asked to answer the acceptability questions listed above.

Efficacy Measures

Efficacy measures were asked at baseline, post-intervention follow-up, and 3-month assessments. The 18-item HIV Knowledge Questionnaire (HIV-KQ-18; [50]) asked participants to indicate whether HIV-related statements were true, false, or don’t know. Correct answers were summed, with a higher score indicating more HIV knowledge. Internal consistency was acceptable in this sample (Cronbach’s alphas = 0.65–0.75). STI knowledge was assessed using the 27-item STD-Knowledge Questionnaire (STD-KQ; [51]) which asked participants to indicate whether STI-related statements were true, false, or don’t know. Correct answers were summed, with a higher score indicating more STI knowledge. Internal consistency was acceptable in this sample (Cronbach’s alphas = 0.85–0.86). PrEP awareness was assessed using the following statement: “There is a daily medication called Truvada (commonly known as PrEP) that can be used to decrease the chance of getting HIV” (true = 1, false/don’t know = 0).

Condom knowledge was measured using six true, false, or don’t know items: “It is ok to use the same condom more than once”; “Condoms have an expiration date”; “When putting on a condom, it is important to leave space at the tip”; “It is okay to use petroleum jelly or Vaseline as a lubricant when using latex condoms”; “When using a condom, it is important for the man to pull out right after ejaculation”; and “Wearing two latex condoms will provide extra protection” [52]. Correct answers were summed and higher scores indicated more condom knowledge. Internal consistency was acceptable in this sample (Cronbach’s alphas = 0.61–0.65).

Two questions were used to assess participants’ pornography knowledge: (1) “Most professional porn actors are required to test for HIV and other sexually transmitted infections prior to shooting a condomless scene” (true = 1, false/I don’t know = 0) and (2) “A lot of porn actors use medications to help them maintain strong erections for a long time” (true = 1, false/I don’t know = 0). To assess participants’ beliefs about pornography, participants were asked how much they agreed with the following statements: (1) “Online porn accurately represents what sex is like between male partners” (strongly agree/agree = 1, disagree/strongly disagree = 0) and (2) “Almost all young men who are interested in male partners view online porn” (strongly agree/agree = 1, disagree/strongly disagree = 0).

At baseline and 3-month follow-up participants were also asked three questions assessing their beliefs about the influence of pornography on themselves. Specifically, participants were asked how much they agreed with the following statements: (1) “Online porn influences what I think sex should be like” (strongly agree/agree = 1, disagree/strongly disagree = 0), (2) “Online porn influences what I think my sex partner(s) should look like” (strongly agree/agree = 1, disagree/strongly disagree = 0), and (3) “Online porn influences what I think I should look like” (strongly agree/agree = 1, disagree/strongly disagree = 0).

Data Analyses

Sample characteristics and univariate distributions of the study feasibility as well as the intervention feasibility, acceptability, and efficacy measures, were described using frequencies (%), measures of central tendency (mean/median), and variation. Fisher’s exact tests were used to assess potential between-group differences in socio-demographic characteristics at baseline.

Because this was a pilot study, analyses were not powered to detect intervention effects. Analyses to assess preliminary efficacy were based on the intent-to-treat sample, with all randomized participants included in the analysis. Specifically, we used a series of longitudinal quantile regression models to examine the associations between intervention and continuous efficacy outcome measures. Models regress median outcomes on groups (intervention vs. control), time, and group-by-time, and adjust standard errors for the correlated repeated measures within participant. Quantile regression is preferable over the standard longitudinal mixed effects model when outcomes are skewed, which was the case in our scale scores [53]. To examine effects of the intervention on the binary preliminary efficacy outcomes, a series of longitudinal models implemented with Generalized Estimating Equations (GEEs) with robust standard errors were used, with a specified Bernoulli outcome distribution and logit link function. Interest was in estimating odds ratios and corresponding confidence intervals. Models included main effects of group (intervention vs. control), time, and group-by-time, to allow for possible differences between groups that varied over assessments. Models used likelihood/quasi-likelihood approaches to estimation and thus made use of all available data without directly imputing missing outcomes. Analyses were run in STATA SE 15.0 with significance level set at 0.05 a priori.

Results

Study Feasibility

A total of 1183 potential participants clicked on the survey link, 1144 (97%) agreed to the screener, and 422 (37%) were eligible (see Fig. 1). Approximately one-half of the respondents who passed the initial eligibility screening (n = 208; 49%) completed the consent process, agreed to participate, and were emailed the survey. Of these, 183 (88%) completed the baseline survey. Twenty-nine (16%) participants were excluded due to internal discrepancies suggesting ineligibility, a potential duplicate, or a programming error in the screening process. This left a final sample of 154 ASMM (n = 77 intervention, n = 77 control). Overall retention was 89% (92% intervention, 86% control) at post-intervention and 86% (87% intervention, 84% control) at 3-month follow-up.

Fig. 1

Study flow diagram

The average age of participants was 16 years (SD = 0.9) (see Table 3). The majority (52%) were recruited via Instagram. Participants reported living in 41 states (17% Northeast, 25% Midwest, 27% South, and 31% West). Approximately one-half (48%) identified as racial/ethnic minorities and the majority (87%) lived in a metropolitan area. Most self-identified as gay (53%) or bisexual (39%). The remaining identified as heterosexual (5%), queer (1%), or another sexual orientation (3%). There were no socio-demographic differences between those randomized to the intervention and control (all ps > 0.10).

Table 3

Socio-demographics by study arm among 14–17-year-old sexual minority males in the United States (N = 154)

	Total	Intervention	Control
	N = 154	n = 77	n = 77
Socio-demographics	n (%)	n (%)	n (%)	χ2
Recruitment source				0.8
Instagram	79 (52)	37 (48)	42 (55)
Facebook	58 (38)	31 (48)	27 (36)
From a friend	16 (11)	9 (12)	7 (9)
Region				2.9
Northeast	26 (17)	11 (14)	15 (20)
Midwest	38 (25)	16 (21)	22 (29)
South	42 (27)	24 (31)	18 (24)
West	47 (31)	26 (34)	21 (38)
Age				6.1
14	9 (6)	6 (8)	3 (4)
15	31 (20)	16 (21)	15 (19)
16	71 (46)	40 (52)	31 (40)
17	43 (28)	15 (19)	28 (36)
Race/ethnicity				0.4
White	80 (52)	41 (53)	39 (51)
Latino	40 (26)	20 (26)	20 (26)
Black/African American	17 (11)	9 (12)	8 (11)
Mixed race/other	16 (11)	7 (9)	9 (12)
Metropolitan residence	132 (87)	68 (88)	64 (85)	0.3
Sexual orientation				0.5
Gay	81 (53)	39 (51)	42 (55)
Bisexual	59 (39)	31 (40)	28 (37)
Heterosexual	8 (5)	4 (5)	4 (5)
Queer/another sexual orientation	5 (3)	3 (4)	2 (3)

Intervention Feasibility

Of the 77 participants who were randomized to the intervention, 65 (84%) logged into the intervention website. The majority logged in without a reminder (43/65, 66%). Of those who logged in, 57 (88%) completed at least one module. The median number of modules completed was 9 (M = 7, SD = 2.7). Of those who logged in, the majority (57/65, 88%) completed the pornography set scavenger hunt game, 52/65 (80%) completed HIV/STI jeopardy, and 51/65 (78%) completed the anal health/anatomy animated video. Among the 51 participants who completed all three initially unlocked modules, thus unlocking the remaining modules, participants were most likely to complete the choose your own dating adventure game (47/51, 92%), the pornography by the numbers infographic (47/51, 92%), the pornography matching game (46/51, 90%), and the illustrated sex menu (46/51, 90%). The question and answer about anal health video and the animated video about consent were each viewed by 39/51 (76%) of the participants. At the post-intervention assessment, 58 intervention participants (58/77, 75%) reported going to the intervention website and 48 control participants (48/77, 62%) reported going to at least one of the control websites (22/77, 29% reported only going to the CDC HIV Prevention website; 4/77, 5% reported only going to the National HIV/STD Testing website; 22/77, 29% reported going to both websites).

Intervention Acceptability

Across all modules, on average, intervention participants rated the invention content 4.3 stars out of 5 (SD = 0.8). Average ratings ranged from 3.8 to 4.5 stars, with the choose your own adventure game (M = 4.5, SD = 0.7) and the anal health/anatomy animated video having the highest average star scores (M = 4.5, SD = 0.6).

As shown in Table 4, a greater proportion of participants who visited the intervention website reported positive experiences than the proportion reporting positive experiences among those who visited the control websites. The one exception was whether participants would recommend the website to a friend, where 86% of participants who reported going to the CDC HIV prevention website reported that they would recommend the website to a friend compared to 81% intervention participants who would recommend the intervention website to a friend and 73% of control participants who would recommend the National HIV/STD Testing website to a friend.

Table 4

Acceptability ratings for the intervention and control websites

	Young men and media intervention website	CDC HIV prevention website	National HIV/STD testing website
	n = 58	n = 44	n = 26
	n (%)	n (%)	n (%)
Received the kind of sexual health information they wanted	54 (93)	37 (88)	20 (77)
Helped them feel more prepared to have a sexual relationship with a male partner	52 (90)	28 (67)	19 (73)
Almost all/most sexual health needs met	47 (81)	30 (71)	18 (69)
Would recommend the website to a friend	47 (81)	36 (86)	19 (73)
Very/mostly satisfied with the amount of sexual health information they received	46 (79)	33 (79)	19 (73)
Excellent/good overall website quality	46 (79)	33 (75)	18 (69)
Very/mostly satisfied with the website	46 (79)	33 (79)	16 (62)
Would come back to the website if in need of sexual health information	46 (79)	32 (76)	18 (69)

Intervention Efficacy

Median baseline HIV knowledge scores were 13 for both intervention (M = 12.3, SD = 3.4) and control (M = 12.2, SD = 3.4) groups. At post-intervention median scores were 14 for intervention (M = 13.7, SD = 2.6) and 13 for control (M = 12.8, SD = 3.4). At 3-month follow-up median scores were 14 for both intervention (M = 13.3, SD = 3.0) and control (M = 12.7, SD = 3.6). Although not significant, point estimates for treatment effects on the HIV knowledge score were in the expected direction. Namely, the median score was higher among intervention participants relative to control at post-intervention (b = 0.70, 95% CI − 1.67 to 3.07) and 3-month follow-up (b = 0.56, 95% CI − 1.93 to 3.04), controlling for baseline.

Median baseline STI knowledge scores were 10 for the intervention group (M = 10.0, SD = 5.5) and 11 for the control group (M = 10.3, SD = 5.7). At post-intervention median scores were 15 for intervention (M = 13.6, SD = 5.6) and 11 for control (M = 11.4, SD = 6.0). At 3-month follow-up median scores were 13 for the intervention group (M = 12.7, SD = 5.6) and 14 for control (M = 12.7, SD = 5.8). Results indicate effects favoring intervention for median STI knowledge score at post-intervention (b = 4.00, 95% CI − 1.47, 5.46), controlling for baseline. However, point estimates did not suggest a difference in median scores at 3-month follow-up (b = − 1.00, 95% CI − 3.23 to 1.23).

Approximately three-quarters of participants were aware of PrEP (intervention = 75%; control = 74%) at baseline. At post-intervention, 83% of intervention participants and 90% of control participants reported being aware of PrEP. At 3-month follow-up 83% of intervention participants and 88% of control participants reported PrEP awareness. A longitudinal model of PrEP awareness did not suggest group differences at either time point, with the odds of PrEP awareness 43% lower for the intervention group relative to control at post-intervention (OR 0.57, 95% CI 0.21–1.56) and 29% lower for intervention vs. control (OR 0.71, 95% CI 0.26–1.91) at 3-month follow-up.

Median condom knowledge scores at baseline were 5 for the intervention group (M = 4.5, SD = 1.3) and 4 for the control group (M = 4.2, SD = 1.6). At post-intervention median scores were 5 for the intervention group (M = 4.9, SD = 1.2) and 5 for the control group (M = 4.5, SD = 1.5). At 3-month follow-up median scores were 5 for the intervention group (M = 4.6, SD = 1.5) and 5 for control group (M = 4.9, SD = 1.2). Point estimates for intervention effects on condom knowledge score did not indicate any differences at post-intervention or 3-month follow-up (b = 0.002, 95% CI − 0.39, 0.40 at post-intervention; b = 0.001, 95% CI − 0.54, 0.54 at 3-month follow-up), controlling for baseline.

For pornography knowledge, at baseline, two-thirds of participants (intervention group = 75%, control = 75%) accurately identified that professional porn actors are required to get HIV/STI tests prior to shooting a condomless scene. At post-intervention, 92% of intervention participants and 69% of control participants accurately reported that HIV/STI testing is required for porn actors prior to a condomless scene. At 3-month follow-up, 82% of intervention participants and 80% of control participants accurately noted that HIV/STI testing is required for actors engaging in condomless sex scenes. Significant between-group differences were seen at post-intervention, such that intervention participants had a higher odds of knowing that professional porn actors are required to test for HIV and other STIs prior to shooting a condomless scene compared to control (OR 1.88, 95% CI 1.06–3.29). These effects were not significant at 3-month follow-up, though the pattern of results was similar (OR 1.06, 95% CI 0.60–1.85).

At baseline, 74% of intervention participants and 69% of control participants accurately reported that many porn actors use medications to help them maintain their erections. At post-intervention, 96% of intervention participants and 71% of control participants accurately reported that same knowledge. At 3-month follow-up, 79% of intervention participants and 75% of control participants accurately identified that same information. Longitudinal models showed a significant effect of intervention versus control with intervention participants having a higher odds of knowing that many porn actors use medications to help them maintain their erections compared to control participants at post-intervention (OR 2.39, 95% CI 1.35–4.25). There were no significant effects at 3-month follow-up, though the direction of effects were similar (OR 1.27, 95% CI 0.74–2.16).

For pornography beliefs, at baseline, approximately one-third of participants (intervention group = 34%, control = 34%) agreed that pornography accurately represents sex between two men. At post-intervention, 20% of intervention participants and 35% of control participants agreed that pornography accurately represents male-male sex. At 3-month follow-up, 24% of intervention participants and 31% of control participants agreed that pornography accurately represents sex between two men. A longitudinal model suggested that at both time points the intervention group was less likely than the control group to agree that pornography accurately represents sex between two men; however, this difference did not differ statistically. Namely, at post-intervention, intervention participants had 55% lower odds of believing that pornography accurately represents sex between two men compared to control (OR 0.45, 95% CI 0.09–2.32). Similarly, at 3-month follow-up, intervention participants had 27% lower odds of believing that pornography accurately represents sex between two men compared to control participants (OR 0.73, 95% CI 0.14–3.33).

At baseline, most participants agreed that almost all ASMM view online pornography (intervention group = 86%, control = 91%). At post-intervention, 93% of intervention participants and 85% of control participants agreed that almost all ASMM view online pornography. At 3-month follow-up, 82% of intervention participants and 91% of control participants agreed that almost all ASMM view online pornography. Although there were no significant effects of intervention versus control on whether participants agreed that almost all ASMM view online pornography at post-intervention (OR 1.10, 95% CI 0.53–2.26), models showed some indication that intervention participants were less likely to believe that almost all ASMM view online pornography compared to control participants at 3-month follow-up (OR 0.53, 95% CI 0.26–1.09).

Finally, at baseline, approximately three out of five participants agreed that pornography influences what they think sex should be like (intervention = 58%, control = 60%). At 3-month follow-up, 45% of intervention participants and 61% of control participants agreed that pornography influences what they think sex should be like. Further, at baseline, 38% of intervention participants and 35% of control participants agreed that pornography influences what they think their sexual partners should look like. At 3-month follow-up, 24% of intervention participants and 42% of control participants agreed that pornography influences what they think their sexual partners should look like. Finally, at baseline, approximately half of the participants (intervention = 51%, control = 49%) agreed that pornography influences what they think they should look like. At 3-month follow-up, 42% of intervention participants and 51% of control participants agreed that pornography influences what they think they should look like. Models did not suggest a significant intervention effect on perceptions of whether pornography influences what sex should be like (OR 0.95, 95% CI 0.49–1.80), what sexual partners should look like (OR 1.12, 95% CI 0.58–2.16), or what participants should look like (OR 1.05, 95% CI 0.56–1.98) at 3-month follow-up.

Post hoc analyses of efficacy outcomes were conducted comparing participants who were exposed to at least one module of intervention content (n = 57) to those who viewed no intervention content (n = 97). Larger gains in median STI knowledge scores (baseline: exposed = 11, not-exposed = 10; post-intervention: exposed = 16, not-exposed = 11; 3-month: exposed = 14, not-exposed = 13) and PrEP awareness (baseline: exposed = 75%, not-exposed = 74%; post-intervention: exposed = 91%, not-exposed = 83%; 3-month: exposed = 88%, not-exposed = 83%) compared to the intent-to-treat analyses were observed. Larger differences in knowledge about HIV/STI testing among pornography actors (baseline: exposed = 75%, not-exposed = 75%; post-intervention: exposed = 96%, not-exposed = 70%; 3-month: exposed = 83%, not-exposed = 80%) and porn actors use of medications to help them maintain their erections (baseline: exposed = 67%, not-exposed = 74%; post-intervention: exposed = 96%, not-exposed = 75%; 3-month: exposed = 85%, not-exposed = 72%) were also observed. Further, larger differences were seen in agreement that pornography accurately portrays male-male sex (baseline: exposed = 35%, not-exposed = 33%; post-intervention: exposed = 15%, not-exposed = 35%; 3-month: exposed = 23%, not-exposed = 30%) or influences what participants think sex should look like (baseline: exposed = 54%, not-exposed = 62%; 3-month: exposed = 39%, not-exposed = 63%) compared to what was observed in the intent-to-treat analyses. All other efficacy measures had equivalent scores and frequencies between the intent-to-treat analyses and post hoc analyses. Although there were no significant differences between groups on any of the outcomes in longitudinal analysis, the patterns of results were the same as the intent-to-treat analyses.

Discussion

Despite disproportionate risk for HIV among ASMM in the U.S. [1, 2], few HIV prevention interventions have been developed to meet the sexual health needs of ASMM younger than 18 years old [7, 8], and only two interventions have been developed for ASMM younger than 16 years old [6, 9]. The Young Men and Media study sought to develop and pilot test a community-informed, online sexual health program specifically for 14–17-year-old ASMM. The goal of the program was to increase sexual health knowledge, promote critical examination of pornography, and decrease sexual risk among ASMM. The results indicate that the developed intervention is feasible, acceptable, and may positively impact sexual health knowledge and beliefs.

Similar to previous HIV prevention intervention studies focused on ASMM [7-9], the online methods used to reach, engage, and retain participants in the Young Men and Media study were highly effective. Specifically, a diverse sample of 154 ASMM from across the U.S. were recruited using social media in only six weeks. Additionally, participant retention was high at post-intervention and 3-month follow-ups. Further, 84% of participants randomized to the intervention logged in to the intervention website and 74% completed at least one module. Among those who completed at least one module, most completed all nine modules. In contrast, only 62% of control participants reported going to the CDC HIV Prevention website, the National HIV/STD Testing website, or both. Clearly, online methods are a viable means to engage ASMM in sexual health promotion and HIV prevention intervention research.

Also similar to previous tested interventions with ASMM [7-9], participants in the current study positively responded to the interactive content. Intervention participants rated the entire suite of modules highly, giving the modules 4.3 stars out of 5 on average. The choose your own adventure dating game and the anal health/anatomy animated video received the highest scores, possibly an indication of the participants’ interest in the topics covered. Further, as reflected in Table 4, a greater proportion of intervention participants reported overall positive experiences with the intervention website than the proportion of control participants reporting positive experiences with the control websites. These findings corroborate previous research and indicate that ASMM enjoy online sexual health education that is interactive, explicit, and broadly focused on issues pertinent to male-male sexual relationships and not solely “risk reduction” and prevention [14]. That said, although the intervention website was rated more positively than the control websites, all of the websites were well-liked by the participants who reported accessing them, illustrating the need and desire for well-designed and reputable online sexual health information for ASMM.

Notably, all intervention participants who engaged with intervention content viewed at least one pornography literacy item. Specifically, of the three initially unlocked modules, all participants who viewed intervention content viewed the pornography set scavenger hunt game. Once participants unlocked all modules, the pornography by the numbers infographic and the pornography matching game were the second and third most viewed modules. As participants were allowed to choose which modules they engaged with, the popularity of the three pornography literacy items reinforces previous research supporting ASMM interest in understanding how pornography use relates to their sexual health and how the sexual behaviors portrayed in pornography relate to sexual behaviors in “real life” [11, 12, 14].

In addition to being feasible and acceptable, the intervention shows indications of preliminary efficacy on HIV and STI knowledge outcomes and pornography knowledge and beliefs. Specifically, median scores for HIV and STI knowledge shifted positively for both the intervention and control participants over time, with participants in the intervention group having indications of slightly more knowledge gain compared to controls, although not statistically significant. PrEP awareness also increased over time for both intervention and control participants. Notably, there was a high level of PrEP awareness at baseline (i.e., three-quarters of the participants were already aware of PrEP). Although it is unclear whether the increase in PrEP knowledge can be attributed to visiting the websites, it is promising that awareness in this sample was high comparatively to previous samples of ASMM and increased over the study period [54]. Further, although condom knowledge did not change substantially over time for either group, it is worth noting that condom knowledge among participants was high at baseline. Thus, the lack of change might reflect a ceiling effect.

In terms of pornography knowledge and beliefs, participants in the intervention group had significant gains in pornography knowledge and decreased beliefs in the accuracy of pornography, although not statistically significant, compared to control participants. Notably, at baseline, one-third of participants reported that pornography accurately reflects sex between two men, the majority reported that pornography influences what they think sex should be like, and half report that pornography influences what they think they should look like, speaking to the continued importance of addressing pornography use among this population [11, 12, 32].

Additionally, post hoc analyses comparing participants who were exposed to at least one intervention module versus those who were not showed larger differences in median scores and frequencies on multiple outcomes compared to what was observed in the intent-to-treat analyses. Although statistically significant differences in outcomes by exposure to intervention content were not seen in the post hoc longitudinal analyses, this may be due to decreased power. As the same patterns of results were observed in the post hoc analyses with larger differences between groups on multiple outcomes the intervention appears to show some promise for impacting the sexual health of ASMM. Additional nuanced analyses that assess the impact of exposure to specific intervention content on corresponding outcomes (e.g., exposure to HIV/STI jeopardy and HIV/STI knowledge) is warranted to understand which components of intervention content were most impactful.

Several limitations should be kept in mind when thinking about these results. Although we recruited a diverse sample from across the U.S., the findings may not generalize to all ASMM (e.g., transgender ASMM or those who do not have Internet access). That said, most adolescents in the U.S. have access to a smartphone and use social media [55], decreasing concerns about biased samples from online recruitment and intervention methods. Further, 26% of participants who were randomized to the intervention did not engage with any of the intervention modules. Although we are not able to empirically assess the reasons why these participants did not complete any of the modules, future research with ASMM may want to consider more active methods to encourage full intervention participation (e.g., reminder calls) in addition to reminder emails. Additionally, PrEP intervention content and the related outcome measure focused on PrEP awareness. Given that PrEP awareness was high among intervention and control participants, future research among ASMM should assess and target additional PrEP constructs, including PrEP intentions and self-efficacy. The study was also conducted during the first six months of the COVID-19 pandemic in the U.S. Thus, adolescents who were able to participate during this period may differ in distinct ways from those who were not (e.g., had free time or privacy to participate, could prioritize study participation over other demands). Additionally, the COVID-19 pandemic significantly altered the sexual behaviors of youth across the U.S., including participants in our sample [39]. As such, we were unable to report on how the intervention may have impacted sexual behaviors.

Despite these limitations, the Young Men and Media intervention shows promise. Results indicate that the intervention is feasible, acceptable, and may positively impact sexual health outcomes. Further, to our knowledge, only one other published intervention and one intervention currently under investigation that have attempted to intervene with ASMM as young at 14 years old [6, 9]. As early intervention prior to or around the age of sexual debut is critical to the development and maintenance of healthy sexual behaviors [10], with additional assessment and development the Young Men and Media intervention may be useful in addressing the sexual health needs of these young ASMM. Further, this is the first HIV prevention intervention for ASMM that specifically included a pornography literacy component, which was acceptable and well-liked by participants. Given that pornography is viewed by the majority of ASMM and used as a source of sexual health information [12, 16, 17], pornography literacy skills are essential to meet the needs of ASMM and counter pornography’s potentially harmful modeling of risk behaviors and its incomplete depiction of intimate relationships [11]. Overall, the encouraging results of this pilot study warrant further examination and potentially a fully powered trial, including the collection of sexual behavior data and objective outcomes. If the Young Men and Media intervention indeed increases ASMM sexual health knowledge and decreases sexual risk it could help to decrease HIV transmission and acquisition among ASMM.