Development and pilot evaluation of an online psychoeducational program for suicide prevention among university students: A randomised controlled trial.

INTRODUCTION: Suicide is the second leading cause of death for the university aged population globally. A significant proportion of students with suicidal ideation or behaviours do not seek professional help. Few primary suicide prevention programs have specifically targeted help seeking for suicidal ideation or behaviours among university students.
METHODS: This study reported the development and pilot test of a brief, two-module online psychoeducational program (ProHelp) that aimed to encourage help seeking for suicidal ideation and behaviours among university students. The program consists of two five-minute modules that address the risk factors and warning signs of suicide, stigmatising attitudes, and perceived barriers to help seeking. 156 Chinese university students and 101 Australian university students were recruited to evaluate the effectiveness of this program at post-test and one-month follow-up. Participants were randomly assigned to the psychoeducational program or an attention control program.
RESULTS: Of the Chinese and Australian students who were randomised into the study, around 50% completed the two-day post-test survey, and 30% completed the one-month follow-up survey. Although no significant difference was found between the control and experimental group on professional help-seeking beliefs and intentions, both groups' help-seeking attitudes increased during the study (p = 0.003 for the post-test survey, and p = 0.008 for the follow-up survey). The experimental group in both countries demonstrated a significant improvement in suicide literacy at the post-test survey (p = 0.015) compared to control. Qualitative feedback indicated that the ProHelp program was user-friendly, clear, and helpful.
CONCLUSIONS: This study provides initial evidence that a brief online psychoeducational program could enhance university students' suicide literacy in both China and Australia. It also suggests that increasing suicide literacy might not be sufficient to improve students' help seeking, although effect sizes indicated that this low-intensity online approach shows promise in encouraging more positive beliefs towards help seeking and preparedness to help individuals with suicidal ideation among young people.

Introduction

Suicide is the second leading cause of death among the university-aged population globally (Turecki and Brent, 2016). It is estimated that nearly 7.5 per 100,000 university students die of suicide each year (Silverman et al., 1997), around 32% university students having lifetime suicidal ideation (Chan et al., 2008), 8% undergraduates and 5% graduate students having lifetime suicidal attempts (Drum et al., 2009).

To lower the risk of suicide among university students, different types of suicide prevention programs have been implemented, including psychoeducational programs (Abbey et al., 1989, Holdwick, 1999), gatekeeper training programs (Hashimoto et al., 2016, Pasco et al., 2012, Tompkins and Witt, 2009), and promotion of services (Joffe, 2008). Although most of them had positive influences on university students' attitudes and knowledge of suicide and suicide prevention, few of them measured help seeking as outcomes (Harrod et al., 2014). In spite of the fact that the ultimate goal of suicide preventions is to save students' lives, assisting students at risk of suicide to receive timely treatment is also an important goal, as less than half of the university students with suicidal behaviours reporting using any mental health services (Schweitzer et al., 1995).

Actively seeking professional help is crucial to suicide prevention. Evidence suggests students who sought help from mental health services were half as likely to attempt suicide as those who did not (Drum et al., 2009). To fill in the gap between the prevalence of suicidal behaviours and actual service use among university students, this study aimed to develop an online program to facilitate potential help seeking for suicidal ideation and behaviours. The effectiveness of this program was further evaluated by a randomised controlled trial (RCT) in Chinese and Australian university students.

Factors influencing help seeking for suicidal ideation or behaviours

Previous studies among young people have identified several factors associated with seeking professional help for suicidal ideation and behaviours. These findings were summarised below to inform the development of this online program.

Shamefulness and discomfort associated with mental illness and relevant treatment was described as one of young people's greatest concerns in seeking help for suicidal ideation and behaviours (Curtis, 2010, Czyz et al., 2013), which was echoed by the interviews from the informants of young suicide decedents (Moskos et al., 2007, Tornblom et al., 2015). Although quantitative studies among the students in Australia, China or US (Chan et al., 2014, Yakunina et al., 2010) have not found a significant effect of stigma on professional help-seeking intentions using the Stigma of Suicide Scale (SOSS, Batterham et al., 2013) or the Stigma of Suicide Scale (SSS, Yakunina et al., 2010), stigmatising attitudes towards suicidal individuals may still play an important role in young peoples' professional help-seeking attitudes and intentions, as indicated by the findings in a large community sample (Calear et al., 2014).

Family and friends also affect young people’ decision and deeds of seeking professional help, although the current findings are not consistent. Higher professional help-seeking intentions or likelihood of service use was found to be associated with greater willingness or actual disclosure of suicidal ideation to family members or friends (Wilson et al., 2011, Wong et al., 2014). However, studies also suggest that stronger social support only facilitate help seeking from non-professional sources, but not from professional sources (Yakunina et al., 2010). Warm and trusting relationships may impede treatment by decreasing the subjective perception of distress (Downs and Eisenberg, 2012) and perceived need for professional help (Czyz et al., 2013).

Self-reliance is another common self-recognised help-seeking barricade among young people (Curtis, 2010, Czyz et al., 2013, Freedenthal and Stiffman, 2007). In our previous study, a higher level of self-reliance was significantly associated with lower professional help-seeking attitudes among both Chinese (p = 0.010) and Australian university students (p < 0.001). Meanwhile, mental health literacy levels including knowledge of suicide prevention (Calear et al., 2014), and ability to identify and describe emotions (Ciarrochi et al., 2003, Ciarrochi and Deane, 2001) have also been found to influence young people's professional help seeking.

In addition to the above factors which are amenable to intervention, demographic factors such as being male (Ciarrochi and Deane, 2001) and mental health factors such as presence of severe suicidal ideation and mental health issues have also been found to be negatively associated with professional help-seeking intentions (Carlton and Deane, 2000, Deane et al., 2001, Wilson et al., 2005a).

Development of the “ProHelp” program

Informed by evidence of factors influencing professional help seeking for suicidal ideation and behaviours among young people (Han et al., 2017), an online psychoeducational program (ProHelp) composed of two modules was developed. Each module was designed to take about 5 min to complete.

The first module addresses suicide literacy, suicide and help-seeking stigma, and available help-seeking sources. The section on suicide literacy covers a broad range of areas, including recognising the signs, symptoms and risk factors of suicide, and how to manage and seek help for suicidality. Improving participants' mental health literacy levels is a commonly used strategy in early intervention programs for mental disorders and suicide (Beautrais et al., 2007, Kelly et al., 2007).

Specifically, the first module comprises six scenarios. The first four scenarios focus on improving participant suicide literacy, including (i) facts and misconceptions about suicidal ideation and attempts, (ii) facts and misconceptions about talking about suicide, (iii) risk factors associated with suicide, and (iv) warning signs of suicide. The content is based on the published literature (Hunter Institute of Mental Health, 2012, Rudd et al., 2006) and the Literacy of Suicide Scale developed by Calear et al. (2014). The next two scenarios are focused on the stigma surrounding suicide including stigma towards individuals with suicidal ideation or behaviour and stigma towards seeking help for suicidal ideation or behaviours. In addition, sources of help for suicidality in Australia and China are provided at the end of this module.

The second module addresses self-reliance, social support, and myths about mental health professionals. Although previous studies suggest that self-reliance may be a substantial barrier to disclosing suicidal ideation or seeking professional help even in the presence of severe symptoms of mental illness or suicidal behaviour that require professional help (Labouliere et al., 2015, Ortega and Alegría, 2002), it is noteworthy that self-reliance is also a source of resilience that could benefit health care and recovery (Bhagwanjee and Stewart, 1999, Ortega and Alegría, 2002). Therefore, the content of this program restricted the focus of self-reliance to the negative influence of self-reliance in specific contexts, where people find it difficult to manage by themselves (such as when they are having thoughts about suicide).

This module also addresses the important role that social support from family and friends can have in the help-seeking process. Several previous interventions have produced encouraging results on improving social support (Hogan et al., 2002) through teaching general psychosocial skills such as interpersonal communications skills (Brand et al., 1995, Richey et al., 1991) and improving coping strategies (Glanz et al., 2008). In this study, skill trainings on how to effectively communicate with other people and visualise one's social connections were offered to improve perceived social support. Although more comprehensive strategies may also be effective for building perceived social support (Cohen et al., 2000), the low-intensity approach chosen for this intervention was deemed to be more appropriate in the context of a universal population-based prevention program.

An attentional control program was developed to provide a credible comparison condition for the trial of the ProHelp program. The attention control program comprised two modules of similar length to the psychoeducational program. The control program content are about healthy life styles, with no content directly relevant to suicide.

To improve program adherence and user experience, parallax scrolling technology (module one) and animation (module one and module two) were used to illustrate the content of the psychoeducational program. Parallax scrolling is an emerging technique in computer graphics and web design. It generates a 3D illusion by making background images move slower than foreground images (Frederick et al., 2015), which can quickly capture an audience's attention and proves to be effective in guided storytelling (Thomas, 2014).

Study aim

The effectiveness of the “ProHelp” program was evaluated among Chinese and Australian university students, as suicide is a leading cause of death among both countries' university-aged population (Australian Bureau of Statistics, 2016, Wang et al., 2014). A cross-cultural evaluation of this online program could provide initial evidence to inform development of effective suicide prevention interventions in higher education under both Asian and Western cultures.

Methods

Eligibility criteria and random allocation procedure

The study was designed as a two-armed double-blind RCT. The Australian site was in English, and the Chinese site was in Simplified Chinese. These two sites ran separately with different website addresses. Participants were invited to register using a valid email address in order to receive the invitations to the program and surveys.

Eligibility was based on 1) being aged between 18 and 30, 2) being a current tertiary student, 3) having a valid email address, 4) having no suicide attempts in the past year, and 5) having no self-reported diagnosis of a psychotic disorder such as schizophrenia. The program is designed to raise public awareness of suicide prevention. To support participants' safety, a list of available sources of help was provided to all participants, including those students who were excluded on the basis of a recent suicide attempt or diagnosed psychotic disorder. Participants were randomised using a computerised adaptive randomisation procedure to the psychoeducational program or attentional control program using a block size of four (1:1 ratio). All participants were blinded to the allocation and stratified by sex and severity of suicidal ideation during this process. Severity of suicidal ideation was measured by the Suicidal Ideation Attributes Scale (Van Spijker et al., 2014). This scale ranges from 0 (no suicidal ideation over the past month) to 50, with higher scores indicating greater severity of suicidal ideation.

Sample size

The sample size was calculated based on the expected effect of the program on the primary outcome measures, i.e. help-seeking intentions based on power calculations performed in G*Power 3.1 (Faul et al., 2007). A target sample of 80 participants in each condition were based on a relevant effect-size of 0.5 (Cohen's d) suggested by previous findings (Gulliver et al., 2012, Taylor-Rodgers and Batterham, 2014) with alpha = 0.05 and power = 0.8, accounting for an expected drop-out rate of 20%.

Participants and procedure

University students in Australia and China were recruited to complete a pilot test of the ProHelp online program. Australian university students were recruited through online advertising on the social network site Facebook and an online recruiting platform, SONA, from the Research School of Psychology at the Australian National University. The Facebook advertisement received 369 clicks and 50 students were invited from the SONA platform. There were 135 respondents recruited from these two sources, of whom 101 were eligible. Respondents cannot be broken down by recruitment source as identifying information was removed from database. The students recruited from the SONA platform were eligible to request 1-hour course credit for their participation. Chinese university students were recruited through teachers' invitations in class from the School of Marxism at the Huazhong University of Science and Technology, and Counseling and Support Services at the Shanghai Jiao Tong University. In total, 231 respondents were screened, and 156 of them were eligible. Fig. 1 presents the flow of participants through the trial. The program contents and scales followed the translation, and back-translation procedure.

Fig. 1

Flow of participants through trial.

Ethics statement

The study received ethics approval from the Human Research Ethics Committee at the Australian National University (protocol number 2015/659), and approval to conduct the research at the School of Maxism in the Huazhong University of Science and Technology, and Counseling and Support Services in the Shanghai Jiao Tong University. The conduct of all research was consistent with the ethical principles of the Declaration of Helsinki.

Measures

Primary outcomes

The primary outcomes were professional help-seeking beliefs, attitudes and intentions.

Professional help-seeking beliefs and intentions items were based on the General Help-Seeking Questionnaire (GHSQ, Wilson et al., 2005b). Participants were asked to rate whether they would seek help if they had suicidal thoughts, from a number of professional sources including psychologists, psychiatrists, and other mental health workers. Help-seeking beliefs were rated for each source on a four-point scale from (1) “Highly unhelpful” to (4) “Highly helpful”, and help-seeking intentions were rated for each source from (1) “Highly unlikely” to (4) “Highly likely”. This scale has been widely used in measuring beliefs and intentions for help seeking among students (Ciarrochi and Deane, 2001, Wilson and Deane, 2010) and the general population (Calear et al., 2014). The items can be analysed individually by presenting the percentage likelihood to seek help from a certain source (Wilson and Deane, 2010) or collectively using the average score (Chan et al., 2014), depending on the research purpose. In this study, scores on professional help-seeking beliefs and intentions were assessed by averaging the scores (ranging from one to four).

Professional help-seeking attitudes were measured by an updated version of the 10-item Attitudes Towards Seeking Professional Psychological Help Scale (ATSPPHS-SF) (Calear et al., 2014) with a better Flesch-Kincaid reading level and good Cronbach's alpha value of 0.84, compared to the original scale developed by Fischer and Farina (1995). The rewritten items include: “Talking with a professional about my personal or emotional problems is not the best way to resolve them”, “If I was having personal or emotional problems, the first thing I would do is seek professional help”, “If I was having personal or emotional problems, I am sure that seeing a professional would be helpful”, and “I would admire a person who dealt with their problems without getting professional help”. Each item is rated on a four-point Likert scale from (0) “Disagree” to (3) “Agree”. Items 2, 4, 8, 9 and 10 were reversed scored, and scores of the help-seeking attitudes scale were assessed as the sum of responses to all items, ranging from 0 to 30, with higher scores indicating more positive attitudes towards seeking professional psychological help. The internal consistency of the ATSPPHS-SF in the present study was 0.65 in China and 0.77 in Australia.

Secondary outcomes

Secondary outcomes included suicide literacy, suicide stigma, social support, self-reliance, preparedness to assist a suicidal friend, and participant evaluation of the program.

Suicide literacy was measured by the 11-item version of Literacy of Suicide Scale (short-form) (Han et al., 2016). The scale covers four domains: (i) signs and symptoms (3 items), (ii) causes or the nature of suicide (4 items), (iii) risk factors (2 items), and (iv) treatment and prevention (2 items). Each of the items on the LOSS is responded to on a 3-point scale (“True”, “False”, or “I don't know”), with correct responses allocated a score of 1 and incorrect or “I don't know” responses assigned a score of 0. Total scale scores are calculated by summing correct scores with a range from 0 to 11.

Suicide stigma was measured by the 12-item version of the Stigma of Suicide Scale (short-form) scale (Han et al., 2016). Each item comprises a one- or two-word descriptor of a “typical” person who dies by suicide (e.g., brave, immoral, lonely), and is rated on a 5-point Likert scale ranging from (1) “strongly disagree” to (5) “strongly agree”. The scale showed a three-factor structure, with the primary factor assessing stigma towards people who die by suicide (five items), a second factor assessing the normalization or glorification of suicide (four items), and a final factor assessing the attribution of suicide to isolation or depression (three items). The subscales of the SOSS are obtained by calculating the mean response to all items associated with each subscale. In the RCT study, Cronbach's alphas for the three subscales were 0.77 (stigma), 0.78 (glorification), and 0.74 (isolation) among Chinese university students, and 0.90 (stigma), 0.78 (glorification), 0.89 (isolation) among Australian university students.

Social support was measured by the Supportive and Negative Interactions with Relatives and Friends scale (Schuster et al., 1990). The scale investigates both the influence of supportive and negative interactions. Items are scored on a 4-point scale from (0) “never” to (3) “often”. Supportive and negative interactions were summed up separately on family and friends. Cronbach's alphas for the three-item supportive interactions subscales were 0.81 (family) and 0.82 (friend) in China, and 0.85 (family) and 0.83 (friend) in Australia. Negative interactions assessed on two-item subscales had adequate internal consistency: 0.80 (family) and 0.73 (friend) in China, and 0.80 (family) and 0.65 (friend) in Australia.

Self-reliance was measured by using a subscale, need for control and self-reliance from the Barriers to Help Seeking Scale (BHSS, Mansfield et al., 2005) with higher scores indicating greater need for autonomy and self-reliance. The BHSS contains ten items using a 5-point Likert-type scale to rate each item from (0) “very unimportant” to (4) “very important”. It showed good internal consistency in the previous study with Cronbach's alpha of 0.91 (Mansfield et al., 2005). The internal consistency in this study was also good with 0.84 in China, and 0.89 in Australia.

Preparedness to assist a suicidal friend was measured by a series of questions previously used in an intervention to encourage help seeking for depression (Costin et al., 2009). The term “depression” was replaced by “suicide” to meet the needs of this study. Participants were asked to answer how much they agreed with the following statements, if a close friend came to them and said that they had been thinking about suicide for several weeks: 1) “I would take my friend's situation seriously”, 2) “I would be comfortable in listening to my friend”, 3) “I would feel confident in knowing how to act”, 4) “I would ask my friend about his/her feelings”, etc. The answers were rated based on a 5-point Likert scale from (0) “strongly disagree” to (4) “strongly agree”. A total score is produced by summing up all eight items.

Participants' experience of the ProHelp program was measured by the Internet Evaluation and Utility Questionnaire (IEUQ) (Ritterband et al., 2008, Thorndike et al., 2008), which contains 15 items with two open-ended questions. This questionnaire assesses the usability, likeability, and usefulness of an online intervention using a 5-point Likert scale from (1) “not at all” to (5) “very”. The two open-ended questions ask the participant to list the most helpful parts and least helpful parts of the program.

Other outcomes

In addition, demographic information including age, sex, and country were collected. This information was used to test whether the control and the experimental groups differed on these characteristics at baseline and if they moderated the between-group effect on outcomes. Mental health status including depression measure by the Patient Health Questionnaire-9 (PHQ-9, Spitzer et al., 1999), anxiety measured by the Generalised Anxiety Disorder-7 (GAD-7, Spitzer et al., 2006), and severity of suicidal ideation measured by the Suicidal Ideation Attributes Scale (Van Spijker et al., 2014) were collected. All the Cronbach alphas for these scales were above 0.80 in both countries. Appendix 1 presents the overview of the measures used in this study.

Analysis

Differences in the participants characteristics at baseline were compared by condition between Chinese and Australian university students using either Chi-square tests or independent sample t-tests for categorical and continuous variables respectively. The primary and secondary outcomes were analysed based on an Intention to Treat basis, including all participants randomised regardless of their level of adherence to the program or trial drop-out. The participants from China and Australia were combined to increase the sample size and reduce the impact of drop-out. The main and interaction effects of country on the outcomes were incorporated into the statistical models to take account of any differences between Chinese and Australian university students at baseline survey. Linear mixed model repeated measures analyses were used to investigate the changes between groups across time (pre to post-test, and pre to follow-up test) on professional help-seeking beliefs, attitudes, intentions, suicide literacy, suicide stigma, social support, self-reliance, and preparedness to provide help, with country included as an independent variable in the models. These models assume that withdrawal data are missing at random, an estimation assumption that is least biased for analysing trial data. As all three-way interactions between country, time, and condition were not significant, these interaction terms were removed from the models. Within-person variation was modelled by using an unstructured covariance matrix and degrees of freedom were estimated using Satterthwaite's correction (Verbeke and Molenberghs, 2009). Relative improvements of outcomes in the experimental group compared to the control group were calculated by Cohen's d using the pooled pretest standard deviation for weighting the differences of pre-post means (Morris and DeShon, 2002). Frequency analysis and t-tests were used to investigate satisfaction ratings at the one-month follow-up survey between Chinese and Australian university students and between experiment and control conditions. Despite including multiple outcomes, the significance level was set at p < 0.05, as the study was a pilot to provide preliminary evidence for the efficacy of this intervention. All statistical analyses were conducted using SPSS version 23 (IBM Corp, Chicago IL).

Results

Baseline characteristics

Participant characteristics at baseline were compared by condition between Chinese and Australian university students. No significant difference was found between the control and experimental groups except that the Chinese experimental group had significantly lower negative friend support than the control group (t (154) = 2.16, p = 0.033). However, significant differences were found between Chinese and Australian university students across quite a few demographic and psychological variables (see Table 1). Australian university students were slightly older (t (131) = 6.74, p < 0.001), female dominant (χ (1) = 24.55, p < 0.001), had significantly higher self-reported depression (t (173) = 5.68, p < 0.001), anxiety (t (157) = − 5.97, p < 0.001), and suicidal ideation in the preceding month (χ (1) = 34.83, p < 0.001) than Chinese university students. Chinese university students had lower levels of knowledge of suicide prevention (t (255) = − 6.15, p < 0.001) and higher stigma towards suicidal individuals (t (255) = 9.92, p < 0.001), while Australian students were more likely to normalise or glorify suicidal individuals (t (255) = − 4.84, p < 0.001), and attribute suicide to isolation (t (255) = 2.44, p = 0.015). Chinese students reported more positive (t (141) = 4.06, p < 0.001) and less negative interactions with family (t (194) = − 2.51, p = 0.013), but less positive interactions with friends (t (255) = − 2.09, p = 0.038) than Australian students. No significant differences were found on the levels of self-reliance, or negative interactions with friends between two countries. In addition, Chinese students were less prepared to provide help for suicidal individuals (t (255) = − 7.16, p < 0.001), and held less positive attitudes towards seeking professional help (t (181) = − 3.42, p = 0.001) than Australian students, but had higher intentions to seek help from mental health professionals (t (191) = 2.53, p = 0.012). No significant difference was found for professional help-seeking beliefs between the two countries. In addition, no significant changes were found in the levels of depression, anxiety, or suicidal ideation before and after the program in both control and experimental groups.

Table 1

Descriptive statistics for Chinese and Australian university students.

	Chinese (N = 156)	Australian (N = 101)	χ2/t
	Frequency or mean(% or SD)	Frequency or mean(% or SD)
Age	18.57 (1.02)	20.07 (2.08)	− 6.74⁎⁎⁎
Female%	78 (50.0)	81 (80.2)	24.55⁎⁎⁎
PHQ-9 depression	6.26 (4.74)	10.40 (6.23)	− 5.68⁎⁎⁎
GAD-7 anxiety	4.66 (4.19)	8.92 (6.32)	− 5.97⁎⁎⁎
Suicidal ideation past month%	13 (8.3)	39 (38.6)	34.83⁎⁎⁎
Literacy of suicide	6.29 (2.27)	7.97 (1.93)	− 6.15⁎⁎⁎
Stigma of suicide scale
Stigma	2.99 (0.78)	1.97 (0.86)	9.92⁎⁎⁎
Glorification	1.97 (0.80)	2.46 (0.74)	− 4.84⁎⁎⁎
Isolation	3.96 (0.66)	4.18 (0.73)	− 2.44⁎
Self-reliance	18.79 (7.11)	16.99 (8.61)	1.75
Social support
Family supportive	5.53 (0.90)	4.81 (1.61)	4.06⁎⁎⁎
Family negative	3.85 (2.06)	4.56 (2.35)	− 2.51⁎
Friend supportive	4.43 (1.45)	4.80 (1.32)	− 2.09⁎
Friend negative	3.35 (1.70)	3.24 (1.82)	0.49
Preparedness to provide help	20.85 (4.73)	25.39 (5.30)	− 7.16⁎⁎⁎
Professional help-seeking beliefs	3.07 (0.62)	3.14 (0.75)	− 0.85
Professional help-seeking attitudes	16.17 (3.97)	18.18 (4.95)	− 3.42⁎⁎
Professional help-seeking intentions	2.88 (0.75)	2.62 (0.87)	2.53⁎

Bold values indicate p < 0.05.

Note: SD, Standard Deviation.

p < 0.05.

p < 0.01.

p < 0.001.

Primary outcomes

Table 2 shows the mixed model estimates for professional help-seeking beliefs, attitudes, and intentions. No significant difference was found on the interaction of group × condition on all the help-seeking outcomes. However, a significant main effect of time on professional help-seeking attitudes was demonstrated at both post-test (p = 0.003) and follow-up time points (p = 0.008), indicating that both the control and experimental group experienced an increase in positive attitudes towards seeking professional help for suicidal ideation, but with quite small effect sizes (Cohen's d = 0.08 for the post-test survey, and Cohen's d = 0.14 for the one-month follow up survey). The effect sizes of the interaction of group × condition on professional help-seeking beliefs (Cohen's d = 0.38 for the post-test survey, and Cohen's d = 0.63 for the one-month follow up survey) indicated some promise of this program in encouraging more positive help-seeking beliefs among university students.

Table 2

Mixed model repeated measure estimates for professional help seeking for suicidal ideation.

	Unstandardised estimate	SE	df	t	p	Cohen's d
Professional help-seeking beliefs
Time (post vs. pre)	− 0.09	0.06	161	− 1.55	0.124
Time (follow-up vs. pre)	− 0.03	0.07	107	− 0.51	0.608
Condition (experiment. vs. control)	0.07	0.08	254	0.91	0.364
Country (Australia vs. China)	0.05	0.08	251	0.70	0.487
Time (post vs. pre) × Condition	0.10	0.08	161	1.21	0.228	0.38
Time (follow-up vs. pre) × Condition	0.13	0.10	106	1.38	0.169	0.63
Professional help-seeking attitudes
Time (post vs. pre)	1.01	0.33	166	3.03	0.003
Time (follow-up vs. pre)	1.09	0.41	126	2.70	0.008
Condition (experiment. vs. control)	0.75	0.55	254	1.38	0.17
Country (Australia vs. China)	2.08	0.52	249	3.96	< 0.001
Time (post vs. pre) × Condition	− 0.13	0.49	167	− 0.26	0.795	0.08
Time (follow-up vs. pre) × Condition	− 0.36	0.59	125	− 0.62	0.539	0.14
Professional help-seeking intentions
Time (post vs. pre)	− 0.02	0.06	161	− 0.40	0.688
Time (follow-up vs. pre)	− 0.05	0.08	114	− 0.61	0.542
Condition (experiment. vs. control)	0.01	0.09	254	0.06	0.953
Country (Australia vs. China)	− 0.21	0.09	249	− 2.29	0.023
Time (post vs. pre) × Condition	0.04	0.08	162	0.46	0.646	0.25
Time (follow-up vs. pre) × Condition	0.05	0.12	114	0.45	0.651	0.39

Bold values indicate p < 0.05.

Secondary outcomes

Table 3 presents the mixed model estimates for the secondary outcomes. A significant difference was found on the interaction of group × condition on the level of suicide literacy at the post-test time point (p = 0.015) with a moderate effect size (Cohen's d = 0.40). This statistic indicates that the change in suicide literacy was significantly greater in the experimental condition than in the control condition at the post-test. However, the effect was not maintained at the one-month follow-up (see Fig. 2).

Table 3

Mixed model repeated measure estimates for suicide literacy, suicide stigma, self-reliance, and preparedness to help.

	Unstandardised estimate	SE	df	t	p	Cohen's d
Literacy
Time (post vs. pre)	0.13	0.27	254	0.47	0.640
Time (follow-up vs. pre)	0.02	0.17	161	0.12	0.903
Condition (experiment vs. control)	0.45	0.21	100	2.14	0.035
Country (Australia vs. China)	1.54	0.26	238	6.01	< 0.001
Time (post vs. pre) × Condition	0.61	0.25	162	2.46	0.015	0.40
Time (follow-up vs. pre) × Condition	0.17	0.30	99	0.56	0.577	0.20
Stigma
Time (post vs. pre)	− 0.07	0.06	163	− 1.18	0.241
Time (follow-up vs. pre)	0.05	0.09	104	0.55	0.584
Condition (experiment vs. control)	0.03	0.10	254	0.30	0.767
Country (Australia vs. China)	− 1.01	0.10	254	− 10.19	< 0.001
Time (post vs. pre) × Condition	− 0.07	0.09	164	− 0.86	0.392	− 0.22
Time (follow-up vs. pre) × Condition	− 0.07	0.13	104	− 0.55	0.581	− 0.15
Glorification
Time (post vs. pre)	− 0.01	0.06	165	− 0.16	0.872
Time (follow-up vs. pre)	− 0.03	0.08	103	− 0.41	0.681
Condition (experiment vs. control)	− 0.11	0.10	254	− 1.16	0.245
Country (Australia vs. China)	0.47	0.09	242	5.03	0.001
Time (post vs. pre) × Condition	− 0.08	0.09	166	− 0.84	0.402	− 0.07
Time (follow-up vs. pre) × Condition	0.03	0.12	103	0.26	0.793	0.05
Isolation
Time (post vs. pre)	− 0.10	0.07	166	− 1.55	0.124
Time (follow-up vs. pre)	− 0.17	0.09	105	− 1.85	0.067
Condition (experiment vs. control)	− 0.06	0.09	253	− 0.72	0.47
Country (Australia vs. China)	0.29	0.08	222	3.75	0.001
Time (post vs. pre) × Condition	0.19	0.10	168	1.96	0.051	0.36
Time (follow-up vs. pre) × Condition	0.19	0.13	104	1.51	0.134	0.42
Preparedness to provide help
Time (post vs. pre)	− 0.11	0.41	163	− 0.25	0.800
Time (follow-up vs. pre)	− 0.08	0.55	120	− 0.14	0.885
Condition (experiment vs. control)	− 0.27	0.62	254	− 0.43	0.665
Country (Australia vs. China)	4.46	0.60	247	7.47	< 0.001
Time (post vs. pre) × Condition	0.61	0.61	163	1.00	0.320	0.24
Time (follow-up vs. pre) × Condition	0.25	0.79	119	0.31	0.758	0.19
Family supportive
Time (post vs. pre)	0.06	0.09	165	0.69	0.493
Time (follow-up vs. pre)	0.11	0.12	104	0.91	0.365
Condition (experiment. vs. control)	0.29	0.15	254	1.90	0.059
Country (Australia vs. China)	− 0.68	0.15	254	− 4.57	< 0.001
Time (post vs. pre) × Condition	− 0.14	0.14	165	− 1.04	0.301	− 0.04
Time (follow-up vs. pre) × Condition	− 0.21	0.17	102	− 1.23	0.221	0.06
Family negative
Time (post vs. pre)	0.20	0.15	165	1.35	0.178
Time (follow-up vs. pre)	0.01	0.19	116	0.07	0.942
Condition (experiment. vs. control)	− 0.04	0.27	254	− 0.14	0.890
Country (Australia vs. China)	0.56	0.26	243	2.18	0.030
Time (post vs. pre) × Condition	− 0.21	0.22	165	− 0.96	0.337	− 0.05
Time (follow-up vs. pre) × Condition	− 0.46	0.28	115	− 1.66	0.101	− 0.25
Friend supportive
Time (post vs. pre)	0.02	0.11	166	0.18	0.861
Time (follow-up vs. pre)	0.08	0.15	115	0.53	0.598
Condition (experiment. vs. control)	− 0.04	0.18	254	− 0.23	0.815
Country (Australia vs. China)	0.32	0.17	250	1.90	0.058
Time (post vs. pre) × Condition	− 0.02	0.16	167	− 0.09	0.927	0.08
Time (follow-up vs. pre) × Condition	0.02	0.21	113	0.09	0.929	0.27
Friend negative
Time (post vs. pre)	0.06	0.14	166	0.39	0.697
Time (follow-up vs. pre)	0.13	0.18	103	0.71	0.477
Condition (experiment. vs. control)	− 0.42	0.22	254	− 1.95	0.053
Country (Australia vs. China)	− 0.21	0.21	243	− 1.00	0.317
Time (post vs. pre) × Condition	− 0.01	0.21	167	− 0.03	0.977	0.00
Time (follow-up vs. pre) × Condition	0.00	0.26	103	0.00	0.998	− 0.06
Self-reliance
Time (post vs. pre)	0.15	0.55	162	0.27	0.786
Time (follow-up vs. pre)	− 0.78	0.80	104	− 0.97	0.336
Condition (experiment. vs. control)	− 0.89	0.97	254	− 0.92	0.360
Country (Australia vs. China)	− 1.47	0.94	246	− 1.57	0.118
Time (post vs. pre) × Condition	− 0.50	0.81	163	− 0.62	0.534	− 0.06
Time (follow-up vs. pre) × Condition	1.09	1.16	104	0.94	0.352	0.21

Bold values indicate p < 0.05.

Fig. 2

Mixed model repeated measure estimates for suicide literacy.

Although no significant difference was found between the experimental and the control groups on the other secondary outcomes, preparedness to help a person with suicidal ideation showed a positive change in the experimental group with a small effective size after the intervention (Cohen's d = 0.24 at the post-test, and Cohen's d = 0.19 at the one-month follow-up time point). In addition, a non-significant difference was found on the interaction of group × condition on the level of attribution of suicide to isolation at the post-test time point with a small effect size (p = 0.051, Cohen's d = 0.36). Although not significant at p < 0.05, this finding indicated marginal evidence that attribution of suicide to isolation may have increased in the experimental condition compared to the control condition.

Utility and acceptability

Chinese and Australian university students' experience of using this program was presented in Table 4. Overall, both Chinese and Australian university students reported favourable reactions to the ProHelp program. Australian university students tended to have more positive comments of the program than Chinese university students. Both countries' university students found the program was easy and convenient to use. They also had low concern about privacy while using the program. The design of the program was less appealing to Chinese university students than Australian university students based on the results from items three to five, but both Chinese and Australian university students were generally satisfied with the program, and thought it acceptable, useful, and easy to understand. They also tended to trust the content of the website, and reported high likelihood of coming back if they needed learn more about suicide prevention from a website. Both countries' university students also favoured the program being delivered by the internet. In addition, participating students' qualitative feedback regarding the most helpful and least helpful aspects of the program was collected. Although few participants (15 Chinese university students and 23 Australian university students) opted to provide qualitative feedback, around one-fourth of Australian (N = 4, 25.0%) and Chinese (N = 6, 26.1%) students mentioned that the presentation of the program was engaging: “The videos were a cute way of presenting very basic information”, and that information of suicide prevention was helpful (N = 1, 6.3% Chinese university students vs. N = 6, 26.1% Australian university students): “[The program contains] the variety of available information, everything important was covered.” No unhelpful aspects and suggestions were reported.

Table 4

Evaluation of the psychoeducational program.

Item	ChineseM (SD)(N = 27)	AustralianM (SD)(N = 29)	t	p
1) How easy was the web program to use?	3.44 (0.93)	4.10 (1.14)	− 2.35	0.022
2) How convenient was the web program to use?	3.30 (1.03)	4.28 (1.00)	− 3.62	0.001
3) How much did the web program keep your interest and attention?	2.63 (0.97)	3.76 (1.02)	− 4.24	< 0.001
4) How much did you like the web program?	2.85 (0.86)	3.79 (0.94)	− 3.89	< 0.001
5) How much did you like the way the web program looked?	2.70 (1.17)	3.62 (1.27)	− 2.81	0.007
6) How concerned were you about your privacy in using this web program?	2.52 (1.25)	1.79 (1.21)	2.21	0.032
7) How satisfied were you with the web program?	3.33 (0.92)	3.86 (0.92)	− 2.16	0.036
8) How acceptable did you find the web program?	3.56 (1.01)	4.17 (0.89)	− 2.43	0.019
9) How useful did you find the information in the web program?	3.30 (0.87)	4.07 (0.84)	− 3.38	0.001
10) How easy was the information to understand in this web program?	3.56 (0.93)	4.52 (0.74)	− 4.29	< 0.001
11) How trustworthy was the information in the web program?	3.33 (0.92)	4.48 (0.63)	− 5.48	< 0.001
12) If you want to learn more about suicide prevention or professional help resources in the future, how likely would you be to come back to this web program?	3.41 (1.08)	3.41 (1.32)	− 0.02	0.984
13) How good of a method was the internet for delivering this program?	3.26 (1.06)	4.34 (0.97)	− 4.00	< 0.001

Bold values indicate p < 0.05.

Note. M, Mean, SD, Standard Deviation.

Discussion

ProHelp is an online psychoeducational program developed based on the factors identified to be associated with professional help seeking by literature. This two-module brief online psychoeducational program had a short-term positive influence on suicide literacy in both Chinese and Australian university students. However, the increased suicide literacy was not sufficient to influence students' attitudes or intentions to seek help for suicidal ideation, although effect sizes suggested some promise of this intervention approach in increasing help-seeking beliefs.

The limited positive influence of this program on help-seeking outcomes may be related to its failure to modify levels of self-reliance, which was identified as an important barrier for professional help seeking for suicidal ideation and behaviours (Bruffaerts et al., 2011, Czyz et al., 2013). Up to now, few studies have investigated how self-reliance could be modified in health interventions aiming to facilitate help seeking. This study provided preliminary evidence that a simple psychoeducation program might not be sufficient to impact on levels of self-reliance among Chinese and Australian university students. More intensive and longer interventions that contain coping skill training might be a worthwhile approach, considering self-reliance is also a type of coping strategy among young people (Seyedfatemi et al., 2007). In addition, it is also important to develop scales that can more specifically measure situation-related self-reliance, to differentiate general autonomy from the types of extreme self-reliance that may be associated with reluctance to seek professional help in a crisis.

Nevertheless, this low-intensity online psychoeducational program showed some promise in improving students' professional help-seeking beliefs, and preparedness to help individuals with suicidal ideation, as indicated by the small to medium effect sizes. As the presentation of this program is similar to that of promotional videos used in public service announcements, the findings of this study suggest that short videos containing suicide prevention information may improve students' knowledge of suicide prevention at a population level, although the influence may be restricted to a short-time period. Repeating or extending the modules, or adding greater opportunity for self-reflection to the program (Sandars, 2009) using greater interactivity or by using persuasive design principles might help increase or prolong the positive influence.

In addition, this online psychoeducational program received positive feedback especially from the Australian university students. The majority of the students thought the program was easy to understand and engaging. They also favoured using internet as a delivery approach. Using the internet to deliver suicide prevention interventions could be a promising approach for suicide prevention among young people as it has the potential to be more cost-effective compared to traditional campaigns run by counseling staff or teachers (King et al., 2015).

It is also noteworthy that Australian university students reported higher levels of depression and anxiety, and higher prevalence of suicidal ideation than their Chinese counterparts. These findings indicate that Australian university students may have poorer mental health than their Chinese counterparts, although Chinese university students' reluctance to disclose their mental health conditions may also play a role in this process. Mental health and suicide-related stigma are well documented in China (Xu et al., 2017). Chinese students have also been found to have less knowledge about mental health and suicide prevention (Wong et al., 2017). They may thus be less aware of their symptoms, and may be afraid of public disclosure about their mental health status. Nevertheless, the severity of mental health symptoms did not moderate the effectiveness of the program.

It should be noted that the findings in the current study should be viewed as preliminary due to the limitations. Firstly, the sample size was relatively small and restricted to the university students from several specific universities. Students dropped out between baseline, post-test, and one-month follow-up surveys, which further reduced the sample size and power to find effects. All of these factors may limit the generalisability of the current findings. Secondly, the study measured the influence of the program until one month after the intervention, which is a relatively short period. Longer durations and more intensive program materials may help further validate the impact of the current findings. Thirdly, this study was conducted among the general student population with a small proportion of students reporting suicidal ideation. Future studies may target students at elevated risk of suicidal behaviours, such as those with current suicidal ideation or mental health issues. Nevertheless, a universal prevention approach may be more practical to implement and have wider impacts of peers supporting friends who experience suicidal thoughts. Fourthly, the sample was recruited by self-selection. This may lead to a limited sampling of students who are reluctant to seek help. While noting that such interventions may impact more broadly on provision of help to others, future studies may highlight the brief, and one-time nature of the intervention, and use a language familiar to young people to help better reach the non-help-seeking population (Ward-Ciesielski, 2013, Ward-Ciesielski et al., 2017). Another limitation of the program is that no quiz or feedback was provided to the participants. Providing immediate feedback about the correctness of the answer and corresponding explanations has been suggested to benefit students' learning outcomes (Van der Kleij et al., 2015). The lack of reflective components in the program may limit its scope for sustainable improvement. Future programs may include quizzes and feedback to help reinforce the influence. Finally, testing the ultimate aim of the intervention, to reduce suicidal behaviour through increasing help-seeking behaviours, requires long-term follow-up of large samples, as help seeking behaviour and suicidal behaviours are relatively rare outcomes in the general population.

Despite these limitations, the findings of the current study support the use of a low-intensive online psychoeducational program to increase suicide literacy among Chinese and Australian university students, although this may not be sufficient to significantly influence help-seeking outcomes in the long term. The effect sizes of students' professional help-seeking beliefs, and preparedness to help suicidal individuals, suggest this brief online psychoeducational program has promise in improving students' readiness to seek help for suicidal ideation or behaviours. Future suicide prevention interventions among university students may consider using internet as a delivery approach as it is generally favoured by students. In addition, further research with a large sample of university students is warranted to determine the robustness of the current findings, and explore how to improve the help-seeking outcomes by reducing students' levels of self-reliance.

Conflict of interest

None.

Variable	Measures	S	B	P	1
Age	Question	X
Student	Question	X
Diagnosis of schizophrenia	Question	X
Location and language	Questions	X
Absence suicide attempt in past month	Question	X
Demographics	Questions		X
Severity of suicidal ideation	SIDAS		X		X
Depression and anxiety	PHQ-9 & GAD-7		X		X
Help-seeking intentions	Modified GHSQ		X	X	X
Help-seeking beliefs	Modified GHSQ		X	X	X
Help-seeking attitudes	ATSPPHS-SF		X	X	X
Preparedness to assist a person with suicidality	Questions		X	X	X
Literacy of suicide	LOSS		X	X	X
Stigma of suicide	SOSS		X	X	X
Social support	SNIRF		X	X	X
Self-reliance	BHSS		X	X	X
Evaluation and utility of website (experiment group only)	IEUQ			X

Note. S, screening survey, B, baseline survey, P, posttest survey, 1, one-month follow-up survey, SIDAS, the Suicidal Ideation Attributes Scale, GHSQ, the General Help-Seeking Questionnaire, ATSPPHS-SF, Attitudes Towards Seeking Professional Psychological Help Scale, LOSS, Literacy of Suicidal Scale, SOSS, the Stigma of Suicide Scale, SNIRF, the Supportive and Negative Interactions with Relatives and Friends scale, BHSS, the Barriers to Help Seeking Scale, IEUQ, the Internet Evaluation and Utility Questionnaire.