Facilitating mental health help-seeking by young adults with a dedicated online program: a feasibility study of Link.

OBJECTIVE: To explore the feasibility of a dedicated online youth mental health help-seeking intervention and to evaluate using a randomised controlled trial (RCT) study design in order to identify any modifications needed before commencement of the full-scale RCT.
DESIGN: A pilot RCT with 1:1 randomisation to either the intervention or comparison arm.
SETTING: An online study conducted Australia-wide. PARTICIPANTS: 18-25 year olds living in Australia were recruited via social media. INTERVENTION: Link is a dedicated online mental health help-seeking navigation tool that matches user's mental health issues, severity and service-type preferences (online, phone and face-to-face) with appropriate youth-friendly services. The comparison arm was usual help-seeking strategies with a link to Google.com. MAIN OUTCOME MEASURES: The primary outcome was the number of acceptability and feasibility criteria successfully met. Intervention and study design acceptability and feasibility were assessed by nine criteria. Secondary outcomes, via online surveys (at baseline, 1 week and 1 month) measured service use, help-seeking intentions, psychological distress, barriers to help-seeking, attitudes towards mental health help-seeking, mental health literacy, satisfaction and trust.
RESULTS: Fifty-one participants were randomised (intervention: n=24; comparison: n=27). Three out of four of the intervention and two out of five of the study design criteria were met. Unmet criteria could be addressed by modifications to the study design. Qualitative analysis demonstrated that Link was useful to participants and may have increased their positive experiences towards help-seeking. There were no observable differences between arms in any outcome measures and no harms were detected.
CONCLUSION: Generally, the Link intervention and study design were acceptable and feasible with modifications suggested for the four out of nine unmet criteria. The main trial will hence have shorter surveys and a simpler recruitment process, use positive affect as the primary outcome and will not link to Google.com for the comparison arm. TRIAL REGISTRATION NUMBER: Australian New Zealand Clinical Trials Registry, ACTRN12614000386639. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

RCT Entities:   Population Interventions Outcomes

This study provides important insights into the feasibility of a unique internet intervention for mental health help-seeking for young adults and the study design in preparation for a full-scale randomised controlled trial.

The study uses social media as an innovative technique to recruit 18–24 year olds across Australia, reflecting how young adults would learn about the intervention in a real-world scenario.

Following the Medical Research Council’s complex intervention guidelines, the intervention is underpinned by a sound theoretical framework ensuring that the design matches the intended goals.

General practitioners, psychological experts, other service providers, technical experts and young adults were involved in a process of codesign to develop the Link intervention.

This pilot study was not powered to assess the statistical significance of the outcomes and as such the results are descriptive, not generalisable, and should be interpreted with caution.

Introduction

Mental health disorders account for the highest burden of disease in adolescence and young adulthood.1 Up to one quarter of young people experience mental health problems with substantial negative effects on interpersonal relationships, functioning at school and work, general health and well-being.2 While there has been significant investment in mental health service promotion and delivery over the last decade,3 only 35% of young people experiencing mental health problems seek professional help.4 5

The major barriers preventing young people from seeking help include lack of recognition of mental health problems,6 lack of awareness about appropriate mental health services,7 8 not being ready to seek help,2 lack of clinical detection9 and the stigma associated with mental illness and seeking professional help.10

Facilitated access to treatment services is necessary to improve mental health outcomes,11 particularly interventions aimed at increasing young people’s willingness or readiness to seek help.12 The international Delphi panel on mental disorders suggested technological solutions may improve treatments, access to care and advance prevention and early intervention strategies.13 Young people find information and access mental health services on the internet and often prefer anonymous sources of help to traditional services.14–16 Yet, young people report that the help-seeking journey can be complex and repetitive.17 Non-dedicated search engines such as Google, while providing a vast array of information and sources, do not discern which services are appropriate for young people and may provide inaccurate and misleading advice and information.17 Furthermore, help-seeking outcomes associated with existing websites, search engines and dedicated online interventions are rarely evaluated.18 19

There is a paucity of evidence for online help seeking interventions and where there are studies, they are poorly designed.19 We sought to address this gap with a rigorous process, following the guidance of the Medical Research Council, involving an iterative approach across intervention development, feasibility testing and evaluation using a randomised trial, before further study on widespread implementation.20 Help-seeking is a complex behaviour to influence and designing an intervention aimed at improving help-seeking online must address multiple components of behaviour change including overcoming barriers to change, as well as providing algorithms to cluster user’s symptoms into related condition categories and thereby direct users to appropriate services likely to meet their needs. Our intervention, Link, is an online program aimed at facilitating mental health help-seeking for young adults aged 18–25 years of age. Our prior development phase examined several theories of help-seeking and behaviour change to determine a suitable theoretical framework with the Theory of Planned Behaviour selected.21 Second, an iterative codesign process with young adults (end users), health service providers and clinical experts tailored the program for use with young adults seeking help for mental health issues.17

In this paper, we report on the feasibility study that aimed to: (1) assess the acceptability and feasibility of the online intervention, Link, to guide young people to appropriate mental healthcare or information; (2) assess the acceptability and feasibility of the study design for the future full-scale randomised controlled trial (RCT) evaluation; and (3) identify any modifications needed to the intervention or the study design before proceeding to the main trial. This pilot was essential to ensure that the intervention elements were acceptable, appropriate and functioned optimally and that the study design was feasible, including the recruitment and randomisation strategy, suitability of the outcome measures and choice of the primary outcome measure for assessing effectiveness and cost-effectiveness in the future RCT. This feasibility study was not powered to assess the statistical significance of changes in the intervention versus comparison arm and, as such, results are descriptive, not generalisable, and should be interpreted with caution.

Methods

Study design

A two-arm RCT was undertaken. Individuals were baseline tested then randomised into balanced (1:1) parallel arms, the intervention arm (Link) or the comparison arm (‘usual search strategies’), and then measured for primary and secondary outcomes with surveys at 1 week and 1 month postrandomisation. Online recruitment and online surveys were used. The online survey service, QuON,22 provided a secure server for managing the trial phases, issuing surveys and collecting data with responses deidentified and stored securely at the University of Newcastle, Australia. This study is reported following the CONSORT guidelines.23 The CONSORT checklist for the trial is provided in the online supplementary file.

Participants

Recruitment

Two recruitment methods were trialled using various social media and online platforms such as Google Ads, Facebook advertising and Gumtree: (1) a static advertising campaign with eight advertisements coupled with keywords and (2) a dynamic advertising campaign, where advertisements were regularly modified to maintain a high profile and visibility on the various platforms. Advertisements were limited to those living in Australia (for all platforms) and aged between 18 and 25 years (only Facebook allowed specification of age). This online recruitment strategy was used to reach the population most likely to use the intervention in the real world and to emulate how the intervention would be advertised if found to be effective. Online recruitment was chosen over traditional mass media or mail-out campaigns as it has been found to be as representative as mass-media campaigns.24 25 Further advantages include being able to reach large numbers of participants quickly, a less confrontational format as there is no face-to-face contact and ease of participation with a click of a button.26 Recruitment using Facebook has been previously successful with this demographic.27

Inclusion criteria

In keeping with the intention of the Link intervention to provide a range of mental health service and information options according to a stepped care approach,28 the inclusion criteria were broad to encompass young adults with mild distress through to more severe mental health problems who have and have not previously sought help, and young people with or without clinical distress, seeking help for issues such as exam stress, relationship troubles, sleep or bullying. Young people had to be: (1) 18–25 years of age; (2) living in Australia; (3) able to provide three contact details such as a current valid email address and residential address and either a phone number or alternative email address; and (4) proficient enough in English to complete the surveys and intervention. Provision of a residential address ensured participants were living in Australia and allowed posting of gift vouchers for participation. The two other contact details were required to enable follow-up.

Procedure

Young people interested in the study clicked a link in the advertisements and were directed to the study website where a brief explanation of the study was provided along with check boxes confirming eligibility. Those eligible were directed to the information statement that fully explained the study objectives and procedures including the issuing of a $A25 gift voucher on completion of the study. If interested in participating, eligible young people were asked to provide their contact details and to tick a box indicating consent to participation. The online system then issued an email to consenting participants instructing them to verify that the email address was correct. They were then directed to complete the baseline (prerandomisation) survey using their email address as a username to ensure participants could only participate once. Participants chose their own password. During this pilot, it became clear that many participants discontinued their participation at the email verification stage, and hence this step was removed (June 13th, 1 month into the pilot) with consenting participants instead progressing straight to the baseline survey. In addition, the information about gift vouchers was moved to the brief explanation page on which participants landed from the recruitment advertisements, in an effort to improve recruitment. Following the baseline survey, participants were randomised into either the intervention or comparison arm. One week and 1 month postrandomisation measurement surveys were conducted in both arms.

Sample size

Consistent with feasibility studies, no power analyses were conducted as this study was not intended to assess effectiveness of the intervention relative to the comparison arm. We anticipated that a total sample size of 120 participants (60 in each arm) would be sufficient to test acceptability and feasibility, allowing for approximately 50% attrition, as is common in online recruitment strategies,29 to ultimately reach a sample of 60 (30 in each arm) at the 1-week timepoint.

Randomisation

An independent academic oversaw the randomisation process including generating the sequence for the random allocation using a random seed generator within QuON in random blocks of four, six or eight. Participants were randomly allocated to a trial arm after baseline and identified by a randomly allocated identification number. The researchers were blind to the randomisation sequence until data analysis had been carried out. Participants were emailed a link to either Google.com (comparison) or Link (intervention), and therefore it was not feasible to blind participants.

Interventions

Comparison arm: usual help-seeking strategies

Participants allocated to the comparison arm received the following instructions: ‘Please search for information and help for an issue you are currently facing using whatever strategies you would normally use whether it's online or offline. The below link will direct you to www.google.com to begin your search’.

The Link intervention arm

Participants allocated to the intervention arm were asked to seek help using Link, comprising three steps in a self-directed triage process (figure 1): (1) select issues (depression/anxiety, body image, drugs/alcohol, self-harm, suicide, bullying and relationship problems), (2) indicate severity on the Link severity scale (setting a level on an interactive pictorial sliding scale on a five-point scale to indicate level of severity from ‘I’m OK’, ‘It’s no big deal’, ‘It’s a lot to handle’, ‘It’s really tough’ and ‘It’s a huge deal’) and (3) select service preference (face-to-face, phone helpline, online chat or email therapy or online information).

Figure 1

The webpages of Link.

Three services appropriate to these specified needs and preferences were then presented with the following information: a description of the service, what to expect, costs and a link for direct access to the service. A ‘recommended’ service, in addition to the participants’ preferred service type, was also displayed based on clinical need. For example, a participant selecting suicidal thoughts as an issue with a preference for online help would also be recommended a 24-hour crisis telephone service (‘Lifeline’) in addition to their preferred service type. There were 31 youth-friendly services included in the Link directory. The program took a minimum of 30 s to complete (in order to reduce attrition during the program), but participants could spend longer exploring their issues or return at a later time to where they had ended a previous session if they wished. Intervention participants had access to Link for the duration of the trial (1 month in total). For the purposes of this study, participants accessed Link using their logon details to link their data with survey data. The development and prototype of the intervention is further described in another publication.21

Measures

Primary outcome

Based on previously used criteria,30 acceptability and feasibility of the intervention and study design are outlined in table 1; four criteria were used to assess the intervention and five criteria were used to assess the study design. Based on the number of criteria met, the study design or intervention were deemed ‘not feasible’ if no criteria were met, ‘feasible with modifications’ if some of the criteria were met, provided that modifications were possible, and ‘feasible with no changes’ if all the criteria were met.

Table 1

Acceptability and feasibility criteria, outcomes and contingency plans

Questions	Criteria	Met?	Outcomes and contingency plan
Acceptability and feasibility of the intervention
Will intervention participants complete Link at least once?	85% of participants log on at least once	No	18/24 (75%) accessed the intervention. However, 15/17 (88%) of those completing the 1-month assessment accessed the intervention. As well as emailing arm allocation to participants, the last page of the baseline survey will contain a link to the allocated intervention so participants can immediately and directly log onto the comparison condition or Link without checking their email account to access the link to their allocation. Another strategy to increase intervention access will be to send reminder emails containing the link to the allocation status and to expand the issues addressed by Link to include some of the issues indicated with a * in table 2.
Will there be any cross-contamination between arms?	No comparison participants use the intervention	Yes	0/27 comparison participants logged on to the intervention.
Did the intervention cause harm?	No serious negative reports or feedback	Yes	There were no negative reports.
Is the severity rating on the intervention useful?	The severity rating correlates with level of distress (k10) by at least 6	Yes	The correlation between severity and k10=0.69.
Acceptability and feasibility of study design and procedures
Can we recruit quickly enough using social media?	200 eligible participants reached within 3 months	No	320/2751 (7%) who saw the advertisement completed the inclusion criteria. 49/192 (26%) who did complete the criteria were ineligible, leaving a total of 143 young adults recruited in a 15-week period, 3 weeks longer than anticipated. As a result: 1. A new advertising campaign will be implemented including electronic direct mail, outsourced to a professional marketing manager, with weekly monitoring to adjust the language used in the ads and the keywords depending on the searches and interests of young people, other organisations using the keywords and the costs associated with the keywords; 2. the burden of participation will be reduced by redesigning the sign-up process and shortening the surveys; and 3. a new incentive scheme will be devised involving increasing the amount awarded to participants based on the number of surveys completed.
Were the measures in the survey useful?	All measures produce a meaningful and unique score	No	Some of the measures were qualitative in nature and difficult to interpret across a larger group of participants, including the mental health literacy measure and the perceptions of mental health items. The length of the survey, particularly due to these items, possibly contributed to attrition rates. After further discussion with the research team, young people and clinicians, changes in survey measures will be undertaken for the future RCT. The mental health literacy and perceptions of mental health items will not be included in the main trial. Given the theoretical considerations of how the Link intervention might work to relieve young people’s immediate distress when they realise there are options to address their mental health concerns, an additional measure of positive and negative affect, immediately postintervention will be used as a primary outcome for the main trial. The comparison condition will not include a link to Google.com in the main trial as this was suggesting a strategy rather than young people’s ‘usual strategies’ that may not include online. In addition, it was felt that landing participants on the Google.com search page presented an unfair comparison condition as the prototype of Link was narrower in the scope of issues capable of being addressed compared with Google.com. Google.com may also present misleading information as search results are not vetted.
Do most eligible participants enrol into study?	80% of eligible participants completed the baseline and randomisation process	No	136/320 (42.5%) did not complete the inclusion criteria or consent to participate. This may have been due to the number of contact details requested. A further 61/184 (33.2%) participants did not verify their email address and begin the baseline survey. 13/64 (20.3%) dropped out during the baseline survey before completing the randomisation process. Less contact details will be requested to improve the consent process. The verification of email will be removed allowing participants to begin the baseline survey more quickly. Feedback from participants indicated that the survey was too long. To further increase completion rates, surveys will be shorter.
Did the randomisation process work?	Equal numbers of participants in each group (within 10%) with similar demographics	Yes	27/51 (52.9%) were allocated to the comparison arm and 24/51 (47.1%) were allocated to the Link intervention arm.
Was the loss to follow-up reasonable?	At least half of the participants completing all surveys	Yes	34/51 (67%) completed all surveys. To further increase completion rates, surveys will be shorter.

RCT, randomised controlled trial.

Secondary outcomes

Help-seeking behaviours were assessed using the Mental Health Care Resource Use Questionnaire31; help-seeking intentions with the Stages of Change Questionnaire (range=1–4), with higher values indicating higher levels32; the General Help-Seeking Questionnaire (range 1–7), with higher values indicating increased likelihood of seeking help33; and two items created (‘I want to seek help for my problems’ and ‘I intend to seek help for my problems)’ using the Research-Based Education and Quality Improvement guidelines (range=1–7), with higher values indicating a higher level of intention.34 We assessed mental health with the Kessler Psychological Distress Scale (K10) (range=5–50), with higher values indicating higher psychological distress35; barriers to help-seeking with the Barriers to Adolescents Seeking Help Scale (range=11–66), with higher scores indicating more barriers7 36; quality of life with the Adolescent Quality of Life Scale37; mental health help-seeking perceptions with four created items (see online supplementary appendix A, supplementary figure A1) ranging from 4 to 28, with higher scores indicating positive perceptions; and mental health literacy using the Mental Health Literacy Vignette.38

Further assessments at the 1 week and 1 month surveys included: satisfaction and trust in the service and the likelihood of using the service again using the Client Satisfaction Questionnaire (range=8–40), with higher scores indicating higher satisfaction39; whether expectations were met using 10 items adapted from Retolaza and Grandes (see online supplementary appendix A, supplementary figure A2) ranging from 10 to 50, with higher scores indicating more expectations met40; and general feedback with four open-ended questions (What did you most like/dislike about the program? Do you have any suggestions for improvements? General comments).

The baseline surveys included all measures except for the questions about satisfaction and trust. The 1-week survey included only the questions about satisfaction and trust. The 1-month survey included all measures except measures of mental health literacy and perceptions.

Data management and analysis

Data from the online surveys and pathways through the Link intervention were collected into QuON. The Stata statistical package V.13 was used for data analysis.41 The primary outcome was the number of acceptability and feasibility items that were met. Descriptive statistics were used to describe the secondary outcomes. Effect sizes (Cohen’s d) were calculated using bootstrapped SD for the satisfaction and trust questions. For all other continuous outcomes, multiple regression analyses were performed in order to calculate postestimation effect sizes; however, these are not reported due to the small sample size.

Results

Study procedure

Figure 2 illustrates the participant flow throughout the study. The static advertising campaign ran from the 7th May to 12th June 2014 with a total of 23 participants randomised during these 5 weeks. Next, the dynamic campaign was used from 13th June to 19th August 2014 with a total of 28 participants randomised within this 11-week period. The majority of participants (45, 88%) were recruited via Facebook advertising.

Figure 2

The study flow diagram. * These participants did not complete the 1-week survey but completed the 1-month survey. Note: Withdrew before entering study group=started baseline then withdrew, or completed baseline and withdrew before randomisation.

Of the 24 participants allocated to the intervention group, 18 (75%) accessed Link. Only two participants (11.8%) who completed the 1-month follow-up assessment did not access Link.

Retention and assessment procedures

Once participants were randomised into the study, attrition rates reduced. From the 51 randomised, only six participants (11.8%; three from each arm) did not complete either of the surveys postintervention and were unable to be included in the outcome analysis. A further three (5.9%) did not complete the 1 week survey and six (11.8%) did not complete the 1 month survey, leaving a total of 34 (66.7%) participants with complete data.

Baseline data

The mean age of participants was 20.9 years (intervention: M=20.9, SD=2.1; comparison: M=21.0, SD=1.9). Other baseline characteristics are listed in table 2 by arm. There were no differences between groups using χ2 test and independent samples t-test statistics.

Table 2

Baseline characteristics by arm

	Comparison (n=27)	Link (n=24)	Total (n=51)
	n (%)	n (%)	n (%)
Female	22 (81.5)	17 (70.8)	39 (76.5)
Rural	9 (3.3)	4 (16.7)	13 (25.5)
State
Australian Capital Territory	4 (14.8)	1 (4.2)	5 (9.8)
New South Wales	4 (14.8)	4 (16.7)	8 (15.7)
Queensland	4 (14.8)	4 (16.7)	8 (15.7)
South Australia	0	3 (12.5)	3 (5.9)
Victoria	12 (44.4)	10 (41.7)	22 (43.1)
Western Australia	3 (11.1)	2 (8.3)	5 (9.8)
Language other than English	2 (7.4)	5 (20.8)	7 (13.7)
Completed higher education	15 (55.6)	14 (58.3)	29 (56.9)
High distress (K10>19)	21 (77.8)	18 (75.0)	39 (76.5)
Mental Health Literacy	25 (92.6)	20 (87.0)	51 (91.1)
Issues
Depression	14 (51.9)	19 (79.2)	33 (64.7)
Relationship problems	12 (44.4)	9 (37.5)	21 (41.2)
Body image	2 (7.4)	4 (16.7)	6 (11.8)
Alcohol/drug use	2 (7.4)	1 (4.2)	3 (5.9)
Bullying	0	2 (8.3)	2 (3.9)
University/school*	10 (37.0)	12 (50.0)	22 (43.1)
Financial issues*	5 (18.5)	10 (41.7)	15 (29.4)
Physical/chronic illness*	2 (7.4)	6 (25.0)	8 (15.7)
Concerns about the future*	3 (11.1)	4 (16.7)	7 (13.7)
Employment issues*	5 (18.5)	4 (16.7)	9 (17.6)
Trauma*	3 (11.1)	1 (4.2)	4 (7.8)
Other*	4 (14.8)	7 (29.2)	11 (21.6)

*Participants’ self-reported issues that were not listed in the intervention.

Note: More than one issue could be selected. Issues were coded from open-ended statements, hence participants could write as many issues as applied. The Other category included life and communication skills, concern for another person’s well-being, obtaining a driver’s licence, parenting and sexuality.

Of the seven issues included in Link, participants reported that they were seeking help for: depression or anxiety, relationship problems, alcohol or drug use, bullying and body image. No participants mentioned self-harm or suicide. In addition, many participants also mentioned issues not covered in Link as listed in table 2. Thirteen participants (21%) only described issues not covered in the program.

Acceptability and feasibility of the intervention and study design

The acceptability and feasibility outcomes are presented in table 1 with suggested modifications. Three of the four criteria were met for the intervention acceptability and feasibility criteria: no cross-contamination or adverse effects were reported, and the Link severity scale correlated with psychological distress (K10). The criterion of 85% of intervention participants logging onto Link was not met, with only 75% doing so. However, 88% of those completing the 1-month surveys logged onto Link, suggesting that failure to use Link may be a problem of engagement with the study measurement procedure. Therefore, the intervention was deemed feasible with minor modifications (listed in table 1) to the study design to increase the number of intervention participants using the intervention. It was also planned to expand the scope of pathways included in Link to address the other issues people described wanting assistance with (table 2).

For the study design, two out of five of the criteria were met: the randomisation process produced similar groups, and the expected completion rate was exceeded. The three criteria not met were the time it took to recruit participants, the usefulness of the measures used and enrolment into the study with minor modifications listed (table 1).

Secondary outcomes

Summary statistics of the measures used in the baseline and 1-month surveys are presented in table 3 in order to examine the acceptability and feasibility of using these measures in the main trial. No inferential statistics are reported due to the small sample size.

Table 3

Baseline and 1-month follow-up scores for help-seeking measures

	Comparison		Link
	Baseline (n=27)	One month (n=20)	Baseline (n=24)	One month (n=17)
Resources	n (%)	n (%)	n (%)	n (%)
Face to face	22 (81.50)	25 (92.60)	20 (83.30)	21 (87.50)
Online	11 (40.70)	14 (70.00)	8 (33.30)	8 (47.10)
None	4 (14.80)	2 (15.40)	3 (12.50)	2 (25.00)
	M (SD)	M (SD)	M (SD)	M (SD)
Distress (K10)	27.6 (9.36)	25.4 (9.78)	28.1 (9.07)	24.4 (7.06)
General Help-Seeking Questionnaire
Total	3.85 (0.81)	4.06 (0.85)	3.73 (0.92)	3.85 (0.62)
Professional	3.69 (1.29)	3.88 (1.42)	3.26 (1.53)	3.47 (1.21)
Personal	3.67 (1.27)	4.1 (1.19)	4.11 (1.12)	4.33 (0.86)
Online	4.57 (1.45)	4.35 (1.28)	3.71 (1.50)	3.41 (1.45)
None	2.3 (1.32)	2.7 (1.78)	2.75 (1.82)	2.06 (1.48)
Stages of Change Questionnaire
Precontemplation	1.72 (0.44)	1.78 (0.49)	1.91 (0.23)	1.36 (0.42)
Contemplation	2.73 (0.39)	2.65 (0.38)	2.64 (0.45)	2.65 (0.41)
Action	3.03 (0.52)	2.9 (0.66)	3.23 (0.75)	3.51 (0.40)
Maintenance	2.41 (0.52)	2.32 (0.57)	2.34 (0.80)	2.49 (0.50)
I want to seek help	3.96 (1.02)	3.56 (1.12)	3.71 (1.04)	3.67 (1.01)
I intend to seek help	4.15 (1.14)	3.7 (1.17)	3.71 (1.40)	3.41 (1.12)
Barriers to Adolescents Seeking Help	37.6 (7.39)	36.93 (7.32)	34.7 (10.10)	34.94 (6.91)
Mental health help seeking perceptions	17.5 (2.44)	17.9 (1.46)	17 (2.19)	17.94 (2.36)
Adolescent Quality of Life	0.5 (0.11)	0.47 (0.13)	0.5 (0.09)	0.54 (0.08)

There were no notable differences between groups (partial η 20.1) except for a small effect in the precontemplation scale of the Stages of Change Questionnaire (partial η20.4), with favourable results for the comparison group.

Satisfaction, trust and whether the participants’ expectations were met for the help-seeking strategies are presented in table 4. Intervention participants reported more satisfaction, expectations met and trust at post-test compared with the comparison arm (small effect sizes), with a small effect size favouring the comparison group for satisfaction at 1 month. There were no differences between arms for expectations met or trust at 1-month follow-up.

Table 4

Participants’ satisfaction, trust and whether their expectations were met

	Comparison (n=27)	Link (n=24)	Effect size
	M (SD)	M (SD)	d*
One-week outcomes
Satisfaction	19.52 (5.09)	20.75 (4.00)	0.3
Expectations met	32.14 (6.75)	34.20 (4.21)	0.4
Trust	3.38 (0.80)	3.55 (0.69)	0.2
One-month outcomes
Satisfaction	22.40 (4.15)	21.71 (4.36)	0.2
Expectations met	35.50 (7.90)	34.41 (7.86)	0.1
Trust	3.90 (0.72)	3.82 (0.73)	0.1

*Effect sizes are based on bootstrapped SD. A value between 0.2 and 0.5 is considered a small effect.

The usefulness of the Link severity scale

We compared the Link severity scale with K10 scores. A positive linear relationship (r=0.81) between the Link severity scale and the baseline scores of K10 was found (figure 3).

Figure 3

A scatterplot comparison of the Link severity scale with the baseline K10 scores with linear trend line.

Further examination of the qualitative responses revealed that participants in the Link intervention arm thought it was a useful program. In particular, the majority of participants in the Link arm found that it was quick, easy, self-directed, personalised and had lots of resources.

Some participants had specific problems (eg, chronic illness, financial, pregnancy and worry about the future) that were not addressed in Link, commenting that the ‘information was not appropriate’ (female, aged 19), ‘not understandable’ (female, aged 21), ‘impersonal’ (females, aged 19 and 21) or ‘too long’ (female, aged 23). Lack of trust in the accuracy of the information available on the internet was also a general concern for both the Link and comparison arms.

Discussion

This feasibility study piloted the acceptability and feasibility of the Link intervention and the study procedures to be used in the future RCT of Link, an online program to assist young adults seeking help for mental health problems. The results demonstrated that the study procedure, involving online surveys and the internet intervention, were feasible and acceptable with several minor modifications identified to enhance recruitment, intervention use and retention in the main trial. There were no indications that the Link or the Google arm caused harms in the participants.

Of the nine acceptability and feasibility criteria, four (logging onto and completing the intervention, recruitment rates, usefulness of the surveys and enrolment rates) were not met. These outcomes were examined to determine likely modifications to successfully meet the criteria in the main trial and are described below.

Will intervention participants complete Link at least once?

As the number of intervention participants who completed the intervention fell short of the expected number (75% instead of 85%), we will increase the visibility of the link to the intervention program in email reminders to participants to complete the program. Likewise, comparison participants will receive reminders to seek help using their usual strategies. As 88% of intervention participants who completed the 1-month survey also completed the intervention program, we anticipate that this minor modification will be sufficient to increase the number of intervention participants who access the Link intervention.

Can we recruit quickly enough using social media?

Two hundred 18–25 year olds living in Australia were needed to ensure 60 who would participate in the study. As the time frame for completing the study was limited by funding, it was necessary that this number was reached within 3 months. However, only 143 participants were recruited within a 15-week period. Therefore, two modifications will be implemented in the main trial to improve the recruitment rates: a new advertising campaign and a new reimbursement scheme.

New advertising campaign

Two methods of recruiting were trialled in this feasibility study, neither with much success. Further methods of online recruitment are needed to ensure an adequate sample size for the future main trial. Using email recruitment from young people signing up to targeted websites (such as ReachOut.com) may increase recruitment rates and will be explored for the main trial.

The dynamic nature of recruiting through Google and Facebook became apparent during the first weeks of our study. Due to the competitive nature of social media, flexible advertising with regular monitoring and adjustment is necessary to maintain visibility on social media streams. Increased flexibility of our advertising will increase exposure to young people and also reduce the costs associated with recruiting participants by choosing keywords that are not currently being used by other organisations. As advertising on social media is dynamic, and dependent on other organisations’ use of keywords, young people’s interests and searches as well as the costs associated with advertising, it is important we respond to the competitive nature of this advertising and have a dynamic and flexible advertising strategy with the ability to change the wording regularly and quickly. As this is a complex and time-consuming process, we will outsource this recruitment to a professional marketing manager with expertise in marketing and recruitment for research studies.

New reimbursement scheme

Alexander et al suggest that a small upfront payment with a higher incentive awarded for retention can increase recruitment and retention rates.42 A $A25 gift card was given to each participant who completed the 1-month survey. In order to reduce attrition rates early in the study procedure, rather than the $A25 gift card received at the end of the study, an incremental reimbursement schedule will be implemented with participants receiving increasing amounts per survey (eg, $10 for baseline, $15 after postintervention survey and $25 after a 1-month follow-up survey).

Were the measures in the survey useful?

We included a large number of measures in the feasibility trial to explore their usefulness with the understanding that only the most relevant would be retained in the main trial. We found that the K10, the General Help-Seeking Questionnaire, the Stages of Change Questionnaire, the Barriers to Adolescents Seeking Help Scale, the Adolescent Quality of Life Scale and the Client Satisfaction Questionnaire were the most meaningful. Qualitative analyses suggest that participants found the surveys too time-consuming, therefore shortening the surveys is likely to increase completion rates. Therefore, several measures that provided less meaningful results will be removed for the larger trial including the mental health literacy scale and the items we created based on the Research-Based Education and Quality Improvement guidelines.34 These measures are not well validated and are not primary outcomes so removing them is anticipated to improve the study design and retention.

One of the benefits reported by participants in the qualitative responses was the immediate increase in positive affect after seeking help using Link. This outcome was also discussed as a likely first step in engaging young people with a help-seeking journey during participatory workshops with service providers, researchers and young people. Therefore, the Positive and Negative Affect Schedule will be used to assess positive affect after participants have been exposed to their intervention, as the primary outcome.43

Do most eligible participants enrol into study?

There was high attrition early in the study procedure indicating that methods to increase retention rates during enrolment into the study are necessary. First, the sign up process was time-consuming and participants may have had concerns about their privacy. Hence, an email address and a phone number only will be required for the main trial. Second, participants were asked to validate the survey on their email account before obtaining access to the baseline survey in the beginning of the pilot study. One of the changes made during the pilot study was a direct link to the baseline survey without the email validation process. This may reduce drop-off at this point. Third, 20% of participants (n=13) dropped out after completing the baseline survey and before completing the randomisation process, indicating that processes to encourage participants to continue with the study are important here as well. The language used, length of the survey and the look of the website may have influenced the retention rates at this step. Further methods are needed to increase these rates such as including regular reminders via email and SMS.

The Link severity scale

Of interest, this study presents some validation for the severity scale used in Link, as it consistently correlated with scores on the K10. This suggests that the Link severity scale may be a valid measure of the impact of mental health problems on daily life and therefore will be retained within the Link intervention.

As discussed above, several measures will be removed from the larger trial and the outcome of positive affect will be used as a primary outcome. Some of the current measures used (Stages of Change Questionnaire, Barriers to Adolescents Seeking Help Scale and General Help Seeking Questionnaire) are not well validated and therefore, while useful as secondary outcomes, are not suitable as a primary outcome. The Mental Health Care Resource Use Questionnaire was also useful and necessary as a secondary outcome; however, as the aim of Link is to direct young people to services appropriate to their needs and preferences, it is difficult to interpret from the services used whether this has occurred. An increase in service use is not necessarily useful for participants with low mental healthcare needs for example.

One of the benefits of online help-seeking strategies is that information is immediately available and the sense of relief felt by participants once an avenue of help is suggested. Participants indicated that they found Link helpful and easy to use. Providing avenues of care to participants may increase positive emotions and broaden personal resources for coping and help-seeking. Some of the features of Link based on the Theory of Planned Behaviour also tap into the concept of positive psychology.44 Increasing mental health literacy and providing young people with a sense of empowerment and meaning improves coping skills,44 particularly problem-focused coping such as help-seeking. By providing avenues for help-seeking in Link, in line with the Theory of Planned Behaviour, positive emotions are likely to occur leading to a positive experience of seeking help. Therefore, we expect that positive affect will be an appropriate primary outcome.43 This measure is well-validated and likely to mediate the relationship between beliefs, and help-seeking intentions and behaviours.

Many coping strategies that increase positive affect are reinforced in Link such as: positive reappraisal, goal-directed problem-focused coping, increasing the repertoire of coping strategies, relaxation and behavioural therapies and increasing positive meaning of issues.44–46 In line with this theory, Link aims to increase connections with others, improve distraction skills and increase self-esteem. Furthermore, positive emotions also improve coping skills and increase the likelihood of future emotional well-being.47

Strengths and limitations

Conducting this feasibility study was a useful way to optimise the intervention and the study design with many issues uncovered before commencement of the main RCT. There were many strengths including the novel recruitment strategies employed allowing a broad range of young adults to be involved in the study. The online survey, data management and randomisation were generally successful with few issues found.

While no differences between arms were anticipated due to the exploratory nature of this study, providing comparison participants with the link to Google was a limitation as it may have suggested this avenue of help-seeking and led participants to a help-seeking method not ordinarily in their repertoire. Furthermore, as Google is a well-developed program with years of programming to perfect the search engine, it is highly advanced compared with the Link prototype, which currently only maps pathways for seven issues and includes 31 services. Therefore, the comparison arm in the larger trial will exclude the link to Google and instead direct comparison participants to use their usual strategies for seeking help.

A measure of positive and negative affect was not included in this study. This outcome is a key element to mental health help-seeking, as positive affect associated with the act of help-seeking is important to continued help-seeking behaviour and obtaining help if needed. Many of the outcome measures used are not well validated or widely used. This is in part due to the small number of publications in the field, but also because of the complexity of mental health ‘help-seeking’ as a focus compared with mental health or psychological distress.

Conclusions

In this feasibility study, we found that the proposed study design and the Link intervention were feasible and acceptable to participants with some modifications. These will mainly include improving the recruitment strategies, lessening the burden on participants during sign-up and by shortening the surveys, choosing a different primary outcome measure to determine positive affect and using a more realistic comparison condition to elicit ‘usual help-seeking strategies’. These are important features and processes to consider in developing and implementing a complex intervention. It is difficult to determine from this pilot trial whether Link will effectively improve help-seeking or positive affect; however, incorporating the improvements identified as a result of this study, the main randomised controlled trial will allow us to investigate the effects of Link on positive affect and help-seeking.