The Men's Safer Sex Trial: A feasibility randomised controlled trial of an interactive digital intervention to increase condom use in men.

OBJECTIVES: We aimed to determine the feasibility of an online randomised controlled trial (RCT) of the Men's Safer Sex website, measuring condom use and sexually transmitted infection (STI).
METHODS: For this study 159 men aged ≥16 with female sexual partners and recent condomless sex or suspected STI were recruited from three UK sexual health clinics. Participants were randomised to the intervention website plus usual clinic care (n = 84), or usual clinic care only (n = 75). Online outcome data were solicited at 3, 6, and 12 months.
RESULTS: Men were enrolled via tablet computers in clinic waiting rooms. Software errors and clinic Wi-Fi access presented significant challenges, and online questionnaire response rates were poor (36% at 3 months with a £10 voucher; 50% at 12 months with £30). Clinical records (for STI diagnoses) were located for 94% of participants. Some 37% of the intervention group did not see the intervention website (n = 31/84), and (as expected) there was no detectable difference in condomless sex with female partners (IRR = 1.01, 95% CI 0.52 to 1.96). New acute STI diagnoses were recorded for 8.8% (7/80) of the intervention group, and 13.0% (9/69) of the control group over 12 months (IRR = 0.75, 95% CI 0.29 to 1.90).
CONCLUSION: It is likely to be feasible to conduct a future large-scale RCT to assess the impact of an online intervention using clinic STI diagnoses as a primary outcome. However, practical and technical challenges need to be addressed before the potential of digital media interventions can be realised in sexual health settings.Trial registration number: ISRCTN18649610.

Introduction

Men’s sexual health

Sexually transmitted infections (STI) are a major public health problem, with high social and economic costs. The majority of genital warts, herpes and chlamydia occurs in opposite sex partnerships, with other infections (syphilis, gonorrhoea and HIV) particularly affecting men who have sex with men.[1] Condoms are effective for prevention of STI; however, there are many barriers to successful use, for example decrease in sensation, interruption of sex, incorrect size or fit, use of alcohol/recreational drugs, anxiety affecting sexual performance and stigma associated with carrying condoms.[2,3] Condoms can be a barrier to intimacy and trust,[3] and use is often lower in established relationships.[4]

Prevention efforts are needed to target the obstacles to condom use that men face.[2] While there are a variety of health promotion interventions aimed at improving sexual health for men who have sex with men (MSM), there are fewer interventions specifically for adult men who have sex with women (MSW),[5,6] despite the fact that MSW report more inconsistent condom use than MSM.[7] Men are less likely than women to visit health professionals and generally have shorter clinic appointments,[8,9] so may be less likely to be offered health promotional advice or risk reduction counselling in routine appointments. Men may also be reluctant to discuss their sexual health with health professionals, partners or friends.[10] An online intervention may therefore offer an alternative avenue to reach men.[11]

Digital interventions for sexual health

There has been an explosion of interest in digital health worldwide, since mobile phone and internet interventions can facilitate access to self-care as well as relieving pressure on over-stretched or scarce health services.[12] However, the pace of innovation is outstripping the capacity to evaluate digital interventions rigorously.[13]

Interactive digital interventions (IDIs) are computer-based programs that provide information and one or more of decision support, behaviour-change support or emotional support for health issues, through provision of personally relevant tailored material.[11] IDIs are suitable for sexual health promotion because access can be private, anonymous and self-paced.[14] Interventions can be targeted for specific groups (e.g. by age, gender or sexuality), and content can be tailored for individuals.[15] IDIs can be costly to develop, but offer the advantages of intervention content fidelity[16] and the potential to reach large audiences at relatively low dissemination costs.[17] IDIs can impact upon sexual behaviour (such as condom use)[11,18] as well as increasing knowledge, self-efficacy and safer sex intention.[11,19] More evidence is needed to establish effects on biological outcomes (STI) and cost-effectiveness.

The Men’s Safer Sex website is an IDI designed to increase condom use and reduce STI. The website addresses men’s barriers to condom use, providing tailored advice for individuals.[20,21]

National Health Service sexual health clinics

Sexual health clinic provision varies across the UK.[17] National Health Service (NHS) provision is free and confidential, and services can be accessed using a pseudonym if wanted, since proof of identity is not required. Waiting times can be long, especially for drop-in services in areas of high demand. Clinics often have electronic medical records that are not integrated with other hospital or family doctor records, and facilities such as patient access to Wi-Fi are not routinely available. Since it is usual to wait some period of time before seeing a sexual health clinician, we chose to offer the Men's Safer Sex website in sexual health clinic waiting rooms, to take advantage of a potential ‘teachable moment’.[13]

Digital interventions can be particularly appropriate for sexual health,[17] and online trials have the potential to be an efficient and cost-effective way to conduct research.[16] However, it can be difficult to recruit people to trials in clinical settings, and it can be difficult to retain participants in online trials.[16] This feasibility trial was designed to evaluate recruitment and retention rates and methods of sexual health outcome measurement, to add to evidence about how to conduct rigorous eHealth research in healthcare settings.

Aim

The aim of this study was to establish the feasibility and optimal design of a full-scale randomised controlled trial (RCT) to evaluate the effect of the Men's Safer Sex website on condom use and STI acquisition among men attending sexual health clinics.

Research questions

Is it feasible to recruit MSW to an online trial (in sexual health clinic settings)?

What proportion of men can be retained in an online trial at 3, 6 and 12 months?

What proportion of clinic records can be located (for STI diagnoses over 12 months)?

Is it feasible to collect data on the main outcomes of interest at 3 and 12 months (condom use and STI acquisition)?

What proportion of the intervention group will access the intervention website?

Methods

Design

This study was a feasibility RCT,[22] with ethical committee approval from City and East NHS Research Ethics Committee (reference number 13 LO 1801).

Participants and setting

Participants were recruited from three busy English sexual health clinics, two in central London and one in the West Midlands.[23] These clinics served a diverse range of patients in terms of age, socio-economic status, ethnicity and sexuality.

Inclusion criteria

Inclusion criteria were: men aged 16 years and over (with no upper age limit); able to read English; with access to the internet; and at high risk of future STI (i.e. two or more sexual partners in the past year and some non-condom use in the last 3 months; or symptoms of acute STI; or seeking treatment for an STI); and for whom at least half of their sexual partners are female. HIV-positive men and men with hepatitis B or C were excluded, since patients with these diagnoses are likely to receive health promotion in the course of routine clinical care. Men who have had sexual experience only ever with males, predominately with males, or no sexual experience at all were also excluded.

Intervention

The Men's Safer Sex website content and design was based on evidence from the sexual health literature, theories of behaviour change, qualitative interviews with men in sexual health clinics and discussions with clinical and academic experts in sexual health and digital technologies.[20] The development process was iterative, with a high level of user involvement.[20] The website addresses men’s barriers to condom use, particularly condom use skills, impact upon pleasure and judgement about potential risk of STI. The website content incorporates behaviour-change techniques, and provides tailored feedback for individual users to address their barriers to condom use (See Appendix 1, Men's Safer Sex website screenshots). The website is designed to be delivered initially in clinic, to make use of the time when patients are waiting to be seen, but also provides online access after patients have left the clinic. The intervention website content and design are reported in detail elsewhere.[20,21] The website is available at www.menss.co.uk.

Procedures

Participants were screened for eligibility, gave informed consent and completed baseline questionnaires using a tablet computer positioned in the waiting room or a side room of the sexual health clinics. The study was advertised in leaflets and on posters, and research and clinic staff also directed participants to the tablet computer. Enrolment was self-directed, with staff available to help with technical problems only. Eligibility was checked with a short set of screening questions online, and eligible participants provided informed consent online. Baseline data were collected online, and then participants received a message informing them of whether they had been randomly allocated to the control group or the intervention group. Those allocated to the intervention group were asked to engage with the MenSS website for as long as they wished (usually until they were called in to see health professionals). They also had access to the website after leaving the clinic, by logging in.

At 3, 6, and 12 months after their initial clinic visit, participants were invited via email to complete an online follow-up questionnaire. If they did not initially complete questionnaires, they received three further email prompts at 1-week intervals, as well as two text messages to their mobile phone. If participants still did not respond, a researcher telephoned them 1 week after the final email and text message. Self-reported STI diagnoses were collected at all time points, and clinical diagnoses were collected at 12 months by recording diagnoses or suspected diagnoses noted in clinic records at the sexual health clinics that participants were recruited from.

Outcomes

We measured response rates for the online questionnaire at 3, 6 and 12 months, and proportion of clinic records which could be located using details given by participants. For sexual health outcomes, we adapted the Sexunzipped online sexual health questionnaire,[24] selecting items for inclusion based on a literature search for established measures and consultation with experts. Interviews were conducted with men in sexual health clinics (N = 11) to gain feedback on successive versions of the outcome questionnaire, and we modified the structure and content of the outcome questionnaire based on men’s feedback.[22]

In the light of evidence that measurement alone may prompt behaviour change,[25,26] a limited number of outcomes was measured at baseline (condom use and hypothesised main mediators, and self-reported STI diagnoses), and a full range of outcomes was assessed at 3, 6 and 12 months (see Table 1 and Appendix 2). Important reasons for condomless sex are being in a regular relationship, and not relying on condoms for contraception, so we asked about these factors to allow a more nuanced interpretation of ‘condomless sex’. We asked about motivation, intention, beliefs about pleasure and non-condom use at baseline, since we felt these to be the strongest predictors of condom use. All assessments had a recall period of the previous three months.

Table 1.

Variables assessed at each time point.

Baseline measures	3, 6, and 12-month follow-up assessments
Demographic details (age, employment status, ethnicity)
Sexual health outcomes
Number of sexual partners	Number of sexual partners
Condom use – number of episodes of condomless sex Numbers of sex partners (without condoms)	Condom use – number of episodes of condomless sex Numbers of sex partners (without condoms)
Self-reported STI diagnoses	Self-reported STI diagnoses
Contraception use and pregnancy in female partners	Contraception use and pregnancy in female partners
Health-Related Quality of Life	Health-Related Quality of Life
	Health service use
Possible mediators of condom use
Motivation to use condoms	Motivation to use condoms
Intentions to use condoms	Intentions to use condoms
Beliefs about pleasure	Beliefs about pleasure
Non-condom use due to intoxication	Non-condom use due to intoxication
	Evaluation of condom use
	Communication
	Identity
	Self-efficacy
	Condom problems
	Knowledge

Sexual health outcome measures

Condom use

The objective of the study was to promote condom use for penetrative sex,[22] and so the main sexual health outcome of interest was the number of episodes of condomless vaginal or anal sex with female partner/s over the previous 3 months, assessed at the 3-month follow-up. We anticipated that the main opportunity for engagement with intervention website would be at the initial clinic visit, and that any change in condom use behaviour would occur over the subsequent 3 months.

Sexual partners

Participants were asked to report the number of sexual partners over the last 3 months (both female and male). The number of partners with whom participants had condomless sex over the previous 3 months was also assessed: female (vaginal and anal sex) and male (anal sex).

Contraception use and pregnancy

Participants were asked which types of contraception (if any) female partners were using. Participants were asked whether a female partner had been pregnant in the last 3 months, and the outcome of that pregnancy if known.

STI diagnoses

Participants were asked to report STI diagnoses over the past 3 months at every follow-up point. Participants were also asked whether they had received treatment due to a partner being diagnosed with an STI. In order to assess laboratory diagnoses, all STI diagnoses recorded in sexual health clinic records over the 12-month study period (at the participating sites) were collected at the end of the study.

Engagement with the intervention (patterns of website use)

Customised Drupal™ web analytics software was used to record website usage (times the website was visited, pages visited), to assess engagement with the Men's Safer Sex intervention.

Adverse effects

Adverse impacts on sexual health outcomes were recorded at 3, 6 and 12 months. Beyond noting any deterioration in outcome measures, participants were also asked to report whether they had experienced any adverse impacts as a result of the study, recording this in a free text box on each of the follow-up questionnaires.

Hypothesised mediators of condom use

We measured hypothesised mediators of condom use behaviour change which were specifically targeted within the intervention, to explore the mechanisms by which behaviour might have changed (see Appendix 2, outcome questionnaire). Determinants of condom use were identified using the theoretical frameworks of the COM-B model[27] and the PRIME theory of motivation.[28] Mediators included condom use errors and problems;[29] knowledge of risk of STIs and condom sizes; communication with partners;[30] perceptions of identity and condom use;[28] beliefs about pleasure and condom use;[31] self-efficacy;[32] motivation; and condomless sex due to intoxication. Full details of all trial outcome variables are described in the Men's Safer Sex RCT protocol.[22] Online questions were compulsory, so completeness of data for participants who submitted outcome questionnaires was excellent.

Randomisation

Once participants had been checked for eligibility, given informed consent and submitted baseline data, they were randomly allocated 1:1 using an automated computer algorithm to either the intervention or control group. This allocation was unalterable.

Blinding

It was not possible to blind participants to their allocation, as those in the control condition received ‘usual care’, with no access to an intervention website. Research staff were blind to allocation condition, as allocation occurred automatically on the tablet computer; however, allocation group could sometimes be inferred by the length of time that participants spent on the tablet computer. Data were analysed by statisticians who were blinded to group identification during analysis.

Statistical methods

The treatment effect at follow-up was estimated using generalised linear models with a loge link and Poisson error. The loge of the baseline values were included in each model as explanatory variables to account for baseline differences. Standard errors were estimated using variance components to account for over dispersion in the models. Incidence rate ratios (IRRs) comparing the control versus intervention groups were estimated from these models along with their 95% confidence intervals and p-values.

Results

Recruitment

Recruitment into the trial was successful: the target sample size was achieved in just over 2 months (28 April 2014 to 7 July 2014), with the greatest rates of recruitment on days when a researcher was present to remind staff and patients about the study. Follow-up data were collected between 28 July 2014 and 2 July 2015. Participant flow is outlined in Figure 1. Due to a technical error in the online recruitment software, reasons for exclusion were only available for 20 participants out of 159. In total, 176 participants were randomised. Of these, seven withdrew from the study, one was a duplicate account, and nine were withdrawn due to being later found to be ineligible (they had been erroneously classed as eligible due to a technical error in the eligibility screening questionnaire).

Figure 1.

CONSORT diagram showing flow of participants.

Of 159 participants, 57 (36%) responded to the 3-month online questionnaire, which was the main outcome point (see Figure 1). An increase in the incentive from £10 (at 3 and 6 months), to £30 (at 12 months) appeared to have a positive effect on response rates: 79 out of 159 (50%) filled in the online questionnaire at 12 months. Data from clinic records on STIs and on sexual health service use were available for 149/159 (93.7%) participants.

Participants

Characteristics of participants are shown in Table 2. The mean age of participants is in line with data that suggest people in the 25–34 years age bracket are more likely to attend for STI testing,[33] and employment status and ethnicity were comparable with the general population.[34]

Table 2.

Demographic characteristics of participants.

Demographic characteristic	Intervention n = 84	Control n = 75
Mean (SD)
Age	29.3 (8.8)	29.5 (8.4)
n (%)
Employment status
School/college/training	10 (11.9)	19 (25.3)
Working	61 (72.6)	49 (65.3)
Unemployed	11 (13.1)	5 (6.7)
Long-term sick or disabled	1 (1.2)	0 (0)
Other	1 (1.2)	2 (2.7)
Ethnicity
White British/Irish/Other	60 (71.4)	50 (66.7)
Black British/Caribbean/  African/Other	11 (13.1)	12 (16.0)
Asian British/Indian/  Pakistani/Bengali	4 (4.8)	4 (5.3)
Chinese/other Asian	2 (2.4)	2 (2.7)
Mixed cultural background	4 (4.8)	4 (5.3)
Other	3 (3.6)	2 (2.7)
Prefer not to say	0 (0)	1 (1.3)

Website usage

Some 37% of participants in the intervention group visited the Men's Safer Sex website once (n = 31/84), 26% more than once (n = 22/84), and 37% did not see the website at all (n = 31/84). Twenty-two (26%) participants returned to the website after leaving the clinic, with 59% of these participants logging into the website twice in total (n = 13/22). The maximum number of return visits was eight. Participants visited a median number of 15 pages out of a total of 34 main topic or activity pages. Several factors hampered men’s access to the intervention website: technical problems with clinic Wi-Fi stability, time taken with online research procedures (registration, consent, baseline outcome measurement), and being called in to appointments before accessing the intervention website.

Descriptive statistics

Distributions of outcomes for condom use, STI diagnoses and potential mediators of behaviour change are reported in Table 3 and Table 4.

Table 3.

Condomless sex and STI diagnoses at baseline and follow-up.

	Control (n = 75)					Intervention (n = 84)
Outcome	Lower quartile	Median	Upper quartile	Max	N	Lower quartile	Median	Lower quartile	Max	N
Episodes of condomless vaginal or anal sex with a woman (in the past 3 months)
Baseline	2	6	20	50	72	1	4	20	155	72
3 months	3	12	30	100	34	2	10	40	100	23
Number of female sex partners (without condoms) in the past 3 months
Baseline	1	1	2	15	72	1	1	2	20	72
3 months	1	1	2	6	34	1	1	2	15	23
Number of male anal sex partners (without condoms) in the past 3 months
Baseline	0	0	0	0	72	0	0	0	3	72
3 months	0	0	0	1	34	0	0	1	3	23
STIs (clinic notes)
Baseline	0	0	1	3	69	0	0	1	2	80
12 months	0	0	0	4	69	0	0	0	2	80
Self-reported STI in the last 3 months (online)
Baseline	0	0	0	2	72	0	0	0	1	72
3 months	0	0	0	1	34	0	0	0	1	23

Table 4.

Potential mediators of behaviour change at baseline and follow-up.

Outcome	Control (n = 75)					Intervention (n = 84)
	Lower quartile	Median	Upper quartile	Max	N	Lower quartile	Median	Upper quartile	Max	N
Number of female partners in 3 months
(Baseline)	1	3	4	179	72	1	2	4	55	72
(3 months)	1	2	3	18	34	1	1	4	23	23
Number of male partners in 3 months
(Baseline)	0	0	0	5	72	0	0	0	15	72
(3 months)	0	0	0	0	34	0	0	0	4	23
Episodes of condomless sex due to being drunk/high in the last 3 months
(Baseline)	0	0	2	20	72	0	0	1	30	72
(3 months)	0	0	3	25	34	0	0	1	5	23
Self-efficacy score
(3 months)	36	40	45	50	33	40	44	46	50	21
Pleasure score
(Baseline)	21	25	29	38	72	19	24	31	41	72
(3 months)	20	24	27	32	34	21	25	30	36	23
Condom problems
(3 months)	1	2	4	8	34	1	2	3	5	23
Motivation
(Baseline)	2	3	4	4	71	2	3	4	4	69
(3 months)	2	3	4	4	32	1	3	3	4	21
Intention
(Baseline)	2	3	3	4	68	2	3	4	4	67
(3 months)	2	3	3	4	32	1	3	3	4	21
Evaluation of condom use
(3 months)	3	4	4	4	34	4	4	4	4	22
Knowledge score
(3 months)	8	9	10	11	34	9	10	11	11	23
Communication score
(3 months)	0	2	4	5	34	1	2	4	5	23
Identity score
(3 months)	25	27	29	35	34	25	28	30	35	23

It can be seen that there was an apparent increase in episodes of condomless sex reported by men in both the intervention and control groups over the 3 months of follow-up from baseline (Table 3). The median numbers of episodes of condomless sex reported at 3 months were small (12 and 10 for the control and intervention groups, respectively) and attrition was large, so these results are subject to bias. The median numbers of (condomless sex with) male and female partners, and STI over 3 months were very small (median 0 to 1). We present these results descriptively without significance testing since the small numbers preclude meaningful analyses.

Table 4 shows the distribution of outcomes which may be mediators of change (i.e. factors potentially influencing condom use and STI acquisition). We present these results descriptively without significance testing since the small numbers preclude meaningful analyses.

Group comparisons for condomless sex and STI

The main aim of the feasibility study was to assess response rates to follow-up outcome data, and the study was not powered to assess differences between groups. Measures of condomless sex with female partners were similar in the intervention and control groups at 3 months (IRR 1.01; 95% CI 0.52 to 1.96) (see Table 5). In total, 23 diagnoses of acute STI (one or more over 12 months) were recorded in the clinical notes of 15 trial participants (seven non-specific urethritis; seven Chlamydia trachomatis; three Neisseria gonorrhoea; two epididymitis; two Molluscum contagiosum; and two people were treated as chlamydia contacts). New acute STI diagnoses were recorded for 8.8% (7/80) of men in the intervention group, and 13.0% (9/69) in the control group over the course of 12 months. There was no statistically significant difference between the groups (IRR 0.75; 95% CI 0.29 to 1.90).

Table 5.

Group comparisons for condomless sex and STI diagnoses.

Outcome	Baseline		Follow-up		IRR	95% confidence interval	p-value	n
	Control Median (n) Inter-quartile range	Intervention Median (n) Inter-quartile range	Control Median (n) Inter-quartile range	Intervention Median (n) Inter-quartile range
Number of episodes of condomless sex with a woman (in 3 months, at 3-month follow-up)	6 (72) 2,20	4 (72) 1,20	12 (34) 3,30	10 (23) 2,40	1.01	(0.52,1.96)	0.975	55
Number of female partners (condomless sex) (in 3 months, at 3-month follow-up)	1 (72) 1,2	1 (72) 1,2	1 (34) 1,2	1 (23) 1,2	1.09	(0.69,1.71)	0.712	55
Number of male partners (condomless sex) (in 3 months, at 3-month follow-up)	0 (72) 0,0	0 (72) 0,0	0 (34) 0,0	0 (23) 0,1	1.72	(0.60,4.93)	0.318	55
Number of STI diagnoses from clinic notes (over 1 year, at 12-month follow-up)	0 (69) 0,1	0 (80) 0,1	0 (69) 0,0	0 (80) 0,0	0.75	(0.29,1.90)	0.543	149

Discussion

In this study we explored the feasibility of a full-scale online RCT (in sexual health clinic settings) of the Men's Safer Sex website for MSW. Recruitment into the trial was successful, and retrieval of clinic records for clinical STI diagnoses was 94%. A larger incentive (£30 voucher) offered at 12 months and resolution of technical problems appeared to have a positive influence on the response rate for the online questionnaire (an increase from 36% to 50%). However, response rates remained poor despite repeated email prompts, text messages and phone calls. A third of the intervention group did not see the Men's Safer Sex website, and there were no differences in main outcomes of interest (condomless sex at 3 months, STI acquisition over 12 months), which was as expected in this small feasibility trial. There was no reported harm from the intervention website or online trial. We discuss the implications of this feasibility trial for the conduct of online trials of digital interventions in sexual health clinic settings.

Technical and practical problems

We encountered technical issues which hampered recruitment, access to the intervention website and online data collection at all time points. It was difficult to set up and maintain access to Wi-Fi for patients in clinic waiting rooms, since the process for permission and set-up was complicated, internet connections were often poor and staff were not always confident about remedying internet access problems. Our protocols for data security demanded passwords with a high level of security, but these requirements are likely to have made it harder to remember the passwords.

There were substantial software development errors that impacted on recruitment of eligible participants, and upon data collection and access to the intervention (e.g. errors in links to the outcome questionnaire, login and password reset functions). Despite careful specification of requirements, and manual testing by the research staff, the extent of these problems were realised only after patients had been recruited.

The reach and scalability of digital interventions is potentially excellent, and interest in digital media technology for health has exploded over the past decade or so, but the NHS lags behind many other institutions in terms of patient access to digital services.[13,17] Reliable software frameworks for online RCTs are also needed, which are customisable, user-friendly, and which meet requirements for secure data protection and data storage.

Engagement with the MenSS website

Engagement with digital interventions for health promotion can be a major challenge.[35] We placed the intervention in clinic waiting rooms to take advantage of the waiting time that is common in drop-in sexual health clinics, and this depended upon clinic flow. A third of the intervention group did not see the MenSS website, which may have been due to a lack of available time: we shortened the length of the outcome questionnaire by capturing only key outcomes at baseline, but research procedures were still time-consuming (informed consent, online registration, outcome data collection), especially if participants were affected by the technical problems described above.

We sent emails at 1-month intervals to prompt men to access the intervention, and were pleased to find that a third of participants visited the website after their initial clinic visit, since we had anticipated that the main window of opportunity for an intervention would be at the time of the clinic visit. We offered financial incentives for follow-up data collection, but not for engagement with the intervention itself. Effective methods for engagement need to be established, for example integrating health promotion websites as a step in a patient pathway through the clinic.[17]

STI diagnoses in clinic records

The retrieval rate for data on STI diagnoses collected from clinic records was 94%, searching by names given, date attended, email address or telephone number. The location of data from clinic records could be improved by requesting additional information from participants (e.g. date of birth, address, clinic number); however, soliciting additional identifiable data may discourage some from participating in research. Half of the sample (72/159) returned to the same clinic within a year; however, STI diagnoses and service use are likely to have been underestimated, since participants are free to attend other sexual health clinics and/or general practice, and records are not shared between all clinics. Differential service use would undermine the validity of clinically recorded STI diagnosis as an outcome if patients in different experimental arms used other services at different rates. It might be possible to capture STI diagnoses made at other sites by using Hospital Episode Statistics and GP practice data. However, patients in sexual health clinics can choose pseudonyms, which would make it difficult to link clinical records. Regardless of which services were used, clinic records are likely to underestimate the cumulative incidence of STI, particularly if STI are asymptomatic (and do not trigger a visit to a clinic).

Despite these drawbacks, recording STI diagnoses from clinical notes has several advantages over self-reported online outcome measurement: clinic record data are not subject to recall bias or social desirability bias, and data collection from records is cheaper in terms of researcher time and costs of participant incentives. In addition, filling in a sexual health questionnaire may prompt participants to reflect on their behaviour (for both control and intervention participants),[25] potentially reducing the apparent effect of an intervention. The cumulative incidence of acute STIs over 1 year was 9.4%, which compares with an annual recorded rate of 0.85% for new STI diagnoses for men attending sexual health services in England.[36] We were therefore successful in recruiting men at substantial risk of STI.

Online outcome measurement

Online trials have a number of potential advantages including automated enrolment, randomisation, data entry and reminders.[17] Online questionnaires can capture data on a range of outcomes, including sexual wellbeing outcomes, which reflect participants’ priorities.[25,37] However, while online recruitment can be good, there can be high drop-out rates which substantially undermine internal validity.[17,38] The response rates for online questionnaires sent by email were poor (maximally 50%), despite an incentive of £30 for the final questionnaire. We used evidence-based methods to enhance retention, such as contacting participants multiple times via different routes (by personalised email, text message and with telephone calls).[24,38,39] However, we do not know whether emails ended up in ‘Junk Mail’, nor how many participants were put off by the initial technical problems with password access and malfunctioning links to the outcome questionnaire. The larger incentive (£30) offered at 12 months appeared to have a positive influence on the response rate (an increase from 36% to 50%), which is in line with studies which tested the effect of larger incentives.[24,40] However, it is hard to evaluate the effect of the larger incentive, since the technical problems were resolved by 12 months, which may have also contributed to the improved response rate.

Conclusion

Our findings regarding the conduct of an online RCT of a website for sexual health for men in a clinic setting may be applicable more widely. We have shown that using STI diagnoses from clinical records as a biological outcome is feasible, but we did not achieve adequate response rates for self-reported online outcomes, which limits the measurement of sexual wellbeing and exploration of the mechanism of action of the intervention. Response rates improved with a larger-value voucher (and once technical problems had been remedied).

There are opportunities for exploring the potential of digital media interventions, and considerable policy support in the UK, but there are problems to be overcome before realising their potential in NHS settings, which include ensuring the reliability of software, patient access to IT, data security protocols which are not over-burdensome for patients, minimally time-consuming research procedures (e.g. collecting outcome data at follow-up only) and facilitating patient engagement with digital interventions, for example by integrating digital interventions into routine clinical pathways.[13]