Online support system for students in higher education: Proof-of-concept study.

BACKGROUND: Providing support to the increasing numbers of students facing mental health difficulties in higher education (HE) can be difficult due to stigma or lack of resources. Alternative and/or complementary sources of support are needed, such as online interventions that are recognised for their therapeutic value and cost-effectiveness.
OBJECTIVES: We aim to provide evidence supporting the conceptual and practical value of a newly developed online multimedia intervention system for HE students who face mild to moderate symptoms of anxiety and depression and study-skills difficulties.
METHODS: Students from five universities were invited to participate in a cross-sectional proof-of-concept study. Students were invited through the universities' internal communication channels. Following demonstration of each part of the system, students completed a survey with quantitative and qualitative questions.
RESULTS: Response was largely positive. Positive responses on the features of the questionnaire ranged between 65% and 86%; on the features of the workshops ranged between 57% and 91%; on 'My place' ranged between 65% and 79%; on the animated videos ranged between 79% and 92%; and on the overall system ranged between 78% and 89%. Participants indicated areas for improvement and ways in which such improvement could be accomplished; these then guided the development of the system.
CONCLUSIONS: The results confirm the need for such a system. It can complement student support services (SSS) by dealing with cases with mild to moderate difficulties, hence allowing SSS to prioritise and effectively address more severe cases. Potentially this method can provide a meaningful alternative to SSS; this is worth investigating further.

Background

Students in higher education (HE) typically gain a wealth of valuable experiences (for example managing finances, deadlines and adult relationships) leading to sudden personal and social development. This is often accompanied by concerns about study success, exam stresses, managing study costs and coping with increased independence.[1] Substantial numbers of HE students face mental health difficulties[2-6] and study-skills difficulties.[4,6-9] Such difficulties prevent students from achieving their true potential and enjoying the university experience, and link to unhealthy lifestyle and risk behaviours, such as binge drinking and substance misuse.[10] Alongside the economic burden that university studies entail, these difficulties may partly explain why more than one in five HE students drop out.[11] HE institutions (HEI) have an obligation to provide wellbeing and academic support to their students, but on-site counselling services struggle to cope with the increasing demands.[12]

A recent study[13] reported that 8% of students identified themselves as ‘having a mental health problem but not seeking diagnosis’ and another 10% as ‘having been diagnosed with a mental health problem, and I believe it still applies to me’. Moreover, official statistics from the Higher Education Statistics Agency (HESA)[3] suggest an upward trend in the numbers of students declaring a mental health issue; looking at 2012 to 2013, the numbers have more than doubled. Depression, anxiety, suicidality, eating disorders and alcohol abuse are only some of the most prevalent mental health difficulties reported among students.[14-16] For example, a study on 2843 students showed that prevalence of depression or anxiety was 15.6% among undergraduate students (13% for graduate students)[14] while a meta-analysis showed a weighted mean depression prevalence of 30.6% among students (ranging between 10% and 84.5%).[17] HESA, furthermore, reports that a very high percentage of students face psychological and study-skills difficulties, such as exam-related stress (90.5%) and stress related to time management and deadlines (83.3%). Half of all students experience further stress pertaining to having enough money to get by and self-image.[18] To some extent, social and cultural factors contribute to the challenges that HE students face. Prominent amongst them are family and interpersonal relationship issues,[19] lower staff-to-student ratios following an increase in student intake[5] and increases in HEI fees[7] The literature links such stressors to risks for common mental health problems including anxiety, depression, alcohol misuse, illicit drug use, as well as eating disorders, self-harm and low memory retention, and difficulty with working or socialising with others.[5,20]

Against this background, on-site student support services (SSS) in HEI struggle to cope with demand as universities face significant budget reductions,[21] despite the reported reluctance of students to seek support. In particular, the Royal College of Psychiatrists[5] indicated that just 4% of HE students seek help from a counsellor for emotional and psychological difficulties each year. In another recent study, only 30% of students stated that they would feel comfortable reporting concerns to an academic, and less than 7% would turn to an external agency or helpline if they needed immediate support.[18] Despite the evident reluctance, the demand for support has nonetheless increased, resulting in long waiting lists and lengthy waiting times.[5] In some cases, the average waiting time between referral and assessment in HE counselling is nine weeks.[22] Getting help via private counselling might be a solution, but usually at a cost not affordable to students.

Increased tuition fees add further pressure on HEIs to be accountable to students. SSS is a key area in which HEIs must demonstrate in practice both quality and value for money. This will help them to meet their pastoral role, optimise their resources and subsequently enhance students’ experience and satisfaction. Ultimately, well-functioning SSS benefit HEIs by reducing the number of student dropouts, increasing annual intake of students, improving position in rankings, and retaining funding. This is even more pertinent within the current context of significant budget cuts of up to 40% over a four-year period,[23] increased inter-institutional competition and pressures to widen access to under-represented groups.[24]

HEI urgently need to restructure and update their SSS. The Royal College of Psychiatrists[5] suggested that NHS health and counselling services and services within HEI could increase their ability to meet demand for support by following two recommendations. First, prioritise demand on the basis of the ‘severity of distress, disability, impact on academic progress, and likelihood of benefit in response to treatment offered’. However, this would mean that the needs of students experiencing mild to moderate difficulties would remain largely unmet. Second, ‘increase the availability of, and access to, self-help programmes such as proprietary or web-based interactive cognitive-behavioural therapy (CBT)’ (Royal College of Psychiatrists, 2011: 20).[5]

In reality, online systems have been widely and effectively used with a range of health conditions and populations. They have been tested and found to be valuable for people who have survived disasters and mass violence (terrorist or other)[25] including veterans,[26] for people with problems such as panic attacks, posttraumatic stress disorder, alcohol abuse, smoking cessation, depression and anxiety,[27] and for patients with chronic illnesses.[28] In education too, the synergy between technology and pedagogical philosophy (such as Universal Design; Universal Design is a paradigm for designing instructional methods, curricula and assessments that are welcoming and accessible to students with a wide range of characteristics, including those related to race, ethnicity, native language, gender, age and disability[29]) has made possible what was previously impossible for underrepresented groups such as, for example, people with disabilities.[30] Technology and purpose put together hold a great promise for diverse and hard-to-reach populations[31] such as ethnic minorities, the socially avoidant or individuals who fail to enact self-help behaviours.[32] A lot of these features are typical among student populations, which makes all the more relevant the development and use of online support systems for them. Preliminary research (conducted by iConcipio Ltd in 2011, and not published to date) surveying 13 UK HEIs revealed a unanimously positive response to an online system that could provide effective services at a lower cost per student. iConcipio Ltd was launched to develop such a system. This paper focuses on the design of this system, and the results from a proof-of-concept study of the system with HEI students.

iConcipio’s system: Description and stages of development

iConcipio has developed an innovative online multimedia system supporting mood, wellbeing, study skills and everyday functioning of HE students, built on current scientific knowledge and best practice from the psychological and educational fields. The design of the system was conceived by iConcipio’s CEO and Founder, Patapia Tzotzoli and was enabled through the advisory role of two boards: the Universities Advisory Board (UAB) and the Academics Advisory Board (AAB). Ann Conloy (Director of Students Services), Fiona Denney (Head of Graduate Studies), Garry Horrocks (Director of Student Experience) and Maya Schram (Head of the Compass Student Centre) all work at King’s College London, comprising the UAB. Elizabeth Barley (University of West London), Mark Haddad (City University London), Jane Hutton (King’s College Hospital), Nicole de Zoysa (King’s College Hospital) and Caroline Wood (University College London) comprise the AAB. The Universities Advisory Board (UAB) ensured that the system meets the requirements of students and HEI services. Meanwhile, iConcipio’s research and clinical team of senior academic and clinical psychologists from different HEIs and NHS hospitals ensured that the system follows best practices and research relevant guidelines. During the summer of 2011 iConcipio was awarded a Proof of Concept Grant (Smart Award) from the Technology Strategy Board. Mosaic Films, a BAFTA award-winning company, produced the demonstration of the system. The system requires no human involvement, other than for technical support purposes and for monitoring of the messaging service (which handles messages about users’ progress and looks for unsuitable discussion contents among participants in the system forum); such involvement typically totals about 2 hours per day.

The development of the system’s design advanced through three key stages: (1) market research; (2) proof-of-concept study; and (3) development of the prototype and feasibility study.

Stage 1: Market research

Initially, two market research projects were conducted seeking feedback from the two main stakeholders: counsellors and psychologists working in HEI SSS and HE students. This stage afforded an understanding of whether there is a place in the market for a system such as that under development.

The first project took place between June and August 2011. Semi-structured interviews were conducted on how existing services operate and their common practices, needs and challenges. The second project took place between February and April 2012 via an online survey. Through the distribution of flyers describing the study, a sum of 61 conveniently sampled students responded to the online questionnaire.[33] The questionnaire had a selection of open and closed questions on identified difficulties that students face at university, their views about online support systems and the features that would make an online support system useful and appealing.

Stage 2: Proof-of-concept study

In summer 2011, after being awarded the Proof of Concept Grant (Smart Award), iConcipio commissioned Mosaic Films to produce a demonstration of the system (hereafter referred to as ‘the demo’). A proof-of-concept study using the demo was conducted with selected UK university partners to collect direct feedback from students about the system. The aim of this testing was to collect feedback from the intended recipients (the students) about the main aspects of the proposed design, the system’s contents, the aesthetics and the process of delivery. In addition to student feedback, university providers were invited to provide feedback during this stage. Executives from each university had the opportunity to discuss results from the proof-of-concept study and contribute to the refinement and finalisation of the system.

Stage 3: Development of the prototype and feasibility study

A prototype of iConcipio’s system has been developed and a fine-tuning of its features is currently underway. Moreover, a feasibility study protocol has been now finalised.[34] As the feasibility study stage is a work in progress, it is discussed in detail following the presentation of the proof-of-concept study.

Systematic review evidence has indicated that self-help psychological therapies (using written materials, audio recordings, computer programmes or web-delivery systems) are effective for a wide range of mental health problems.[35,36] In particular, meta-analytic studies of online interventions (randomised trials or randomised control trials) targeting anxiety symptoms and disorders and stress, or both anxiety and depression symptoms at the same time among HE students, showed that about half (47%) reported at least one significant positive effect in the intervention group compared to the control while 29% failed to find a significant effect.[37] Another meta-analysis[38] of online interventions among HE students supports their effect in improving anxiety, stress and depression when treatment groups were compared to inactive controls. This effect, however, did not hold when the treatment groups were compared to active controls or when the intervention group was compared to active controls or alternative interventions. Further to these, computerised systems, especially computer-aided cognitive-behavioural therapy (CBT), have been found to be as effective as face-to-face interventions for some problems. They reduce overlong waiting lists, offer increased convenience and confidentiality and do not carry the potential stigma related to face-to-face treatment.[39,40] Moreover, web-based support systems can offer access to psychological and study-skills support for a complete cohort of students across each HEI, overcoming the time and resource limitations to which traditional SSS are subject.

iConcipio’s system is proposed as a stand-alone or as a complementary source of support to traditional face-to-face methods currently provided to HE students. It is an all-in-one web-based system, designed to address HE students’ psychological and study-skills difficulties. It targets mild to moderate difficulties. Students with more significant problems first identify themselves via a set of screening statements, and are then given advice regarding how to seek more appropriate means of support (be it the SSS of their HE institution, mainstream helplines or help centres). One of the system’s most pioneering elements is its ‘bottom up’ design, following either a needs- or symptoms-assessment approach (Questionnaire route) or a techniques-driven approach (Workshop route), with both routes offering personalised support on how to better ‘cope’ with present challenges. Such support does not depend on personality or condition categories and is not based on generic wellbeing advice. It is instead custom-made and avoids a fixed support plan underpinned by a ‘one size fits all’ philosophy. Figure 1 presents the layout of the system.

Figure 1.

Layout of the system.

The system offers an intervention built on implicit and explicit state-of-the-art psychological and educational techniques and practices. The techniques employed are derived from CBT, an effective psychological treatment approach for a wide range of emotional difficulties and associated behaviours.[41,42] For the sample problem, students can select techniques drawing from either the cognitive or behavioural approach, thereby allowing flexibility and enhancing acceptability. The techniques comprise the most up-to-date practical skills advice, which have been extensively tested and shown to improve students’ performance over the years.[43-45]

Students wishing to explore their difficulties can take one of two routes, after they create an account and log into their personalised view of the system. Access to the system requires access to a computer with a typical internet connection. Those who want more guided help can answer an interactive questionnaire, which leads to a taxonomy defining their difficulties (Questionnaire route). The questionnaire differentiates between predominant features of anxiety, of depression, or of mixed anxiety and depression. The questionnaire statements have been adapted from established tools, including the Hospital Anxiety and Depression Scale (HADS),[46] the seven-item Generalized Anxiety Disorder Scale (GAD-7),[47] the Patient Health Questionnaire (PHQ-9),[48] and the structure interview, Mini International Neuropsychiatric Interview (MINI).[49] The HADS, GAD-7 and MINI informed the choice of anxiety questions. The HADS, PHQ-9 and MINI informed the choice of depression questions. As an alternative to the Questionnaire route, students can enter the Workshops (Workshops route) and select two areas of difficulty for which they then access techniques that can help them overcome their difficulties (see Figure 2).

Figure 2.

Questionnaire and Workshop routes.

Each route then yields a personalised package of evidence-based psychological techniques (e.g. a progressive muscle relaxation technique) and/or study-skills techniques (e.g. effective time management). The techniques are presented in animated audio-visual format. Students can practice these techniques and use a rating system to monitor their progress over time. An overview of the different sections of the system is presented in Table 1. There is also an online peer-support network, which allows students to connect anonymously with others. The self-monitoring process is supported by an automated messaging service that monitors the plans and progress of individuals, and generates messages depending on individual users’ needs. For example, in the case of students who have entered the system, messages typically encourage them to return to the system to practise the techniques and complete the post-intervention measures. In other cases, however, for example when first-time users have been screened as having severe difficulties, or in cases where individuals do not seem to benefit from the system and do not show progress, the messages encourage them to seek additional help from formal support services. The messages are sent to the user’s inbox within the system, to their personal email accounts and, if they choose, to their personal mobile phone. The messaging service is monitored by expert facilitators, such as trained practitioners, researchers, and students (typically holders of PsyDs or close to completion of PsyD programmes). For a full review of the system please refer to Appendix 1, Figures 7–22.

Table 1.

Sections of the system.

Sections	Description
Home	Welcome message and description of the system.
Let’s go	Terms and conditions agreement.
	Screening questions.
	If the system can help the user, then: (a) The user can fill in a questionnaire and identify exactly where her/his difficulties are; Or (b) If areas of difficulty are known, the user can enter the workshops and select two areas where they access techniques and strategies to overcome these difficulties.  • If the user cannot find the topic that interests her/him, the system invites her/him to write to the developers about it;  • If the user can find the topic that interests them, the system invites him/her to assess how severely these issues influence one’s life (a tailored plan of action can then be found under the individual’s MyPlace).
MyPlace	The personalised version of the system, tailored to the user’s profile, input and requests, where one can find:  • A proposed set of techniques based on the questionnaires previously filled in or workshops selected;  • A plan of actions within the system (the user can opt out);  • A reminder of the scheduled activities can be seen in a separate window;  • Progress on techniques already visited (for example, indicating that one of five techniques have been visited);  • The techniques presented in the form of videos;  • The user’s personalised notes. Emails are sent to assess progress. If progress is not good other approaches are proposed to the user.
Forum	Synchronous and asynchronous secure communication environment for users.
FAQ	Addressing frequently asked questions.
Contact us	Contact details and forms.
Login/settings	Log in/log out options and settings for the personalised accounts of users.

What follows is a full description of the proof-of-concept study methodology, findings and a discussion of the resulting conclusions.

Methods

Participants

University SSS were directly approached by iConcipio and informed of the proof-of-concept study in spring 2012. This was achieved via a circular email sent by the Director of Student Experience at King’s College London to the Association of Managers of Student Services in Higher Education (AMOSSHE) (the UK Student Services Organisation) mailing list. During summer 2012, meetings were held with HEIs showing interest in finding out more about the system and the proof-of-concept study. Subsequently, five universities were selected to participate in the study: King’s College London, the University of Warwick, the University of Edinburgh, Bournemouth University and the University of Roehampton. Participating institutions varied in terms of geographical location, size, rural or urban environment, and campus type (campus or non-campus) to maximise variation in responses collected.

Universities used their internal communication channels to promote a link to the study webpage to undergraduate and postgraduate students. Half way through the six-week period of the study, a reminder email was sent. Some universities, at their discretion, provided a prize to encourage participation. A sample of 873 students resulted from this process. The number of respondents per question varied, either due to cumulative attrition from the survey or due to invalid responses. Ethical approval for this project was granted by the Kings College London (KCL) College Research Ethics Committee (CREC), Psychiatry, Nursing and Midwifery Research Ethics Subcommittee (PNM RESC).

Materials and procedure

The proof-of-concept study took place between September and October 2012. During this period, a demo of the system was administered to the participating students of the five HEIs. The demo of the system was developed as an online-hosted system, comprising slides and short videos, which explained to students how the system would work when fully developed. In particular, the slides and the videos demonstrated: (1) the screening process that will identify those with more significant difficulties and re-direct them to appropriate sources of support, whilst those with mild to moderate difficulties could proceed by accessing the techniques via one of two routes; (2) the Questionnaire route (where techniques for dealing with difficulties are sent post-screening via a questionnaire); (3) the Workshop route (where participants choose techniques to deal with their identified psychological and/or study challenges); (4) their personal space where the techniques are hosted (My Place) and how it functions; and (5) an example of a psychological (relaxation) technique in 3D animated video format.

The site followed a linear process, and five surveys were embedded within this to collect feedback from students on all parts of the demonstration. These surveys consisted of: (1) five questions about the Questionnaire route; (2) five questions about the Workshops route; (3) six questions about My Place; (4) five questions about the animated video; and (5) fifteen general questions about the system. The 36 questions included in the above five surveys were either open-ended (e.g. ‘What might be the barriers for using such a system?’) or required a categorical response (e.g. ‘Did you like the checking slide?’) with a forced-choice between ‘Yes’, ‘So-so’, and ‘No’. For the categorical questions, there was the option to provide further free-text information. The full set of closed questions of this survey can be seen in Appendix 2, Tables 3–12.

Table 3.

Feedback on the questionnaires (n = 873).

Questions	Responses (n (%)
	Yes	So-so	No
Did you like the way the questions were phrased?	569 (65)	240 (28)	64 (7)
Did you like the interactivity and usability of the questionnaire?	752 (86)	85 (10)	36 (4)
Did you like the checking slide?	662 (76)	147 (17)	64 (7)
Did you like the pre-rating slide?	600 (69)	198 (23)	75 (8)
Would you write to us if you did not agree with the difficulties identified?	598 (68)		275 (32)
Total	873

Table 12.

Other negative feedback, potential barriers to use, and potential alternative means of self-help.

Questions	n (%)
What did you dislike about the system?
Nothing disliked	123 (33.7)
Patronising/childish	45 (12.3)
Appearance/colours	44 (22.1)
Total	365
What might be the barriers for using such a system?
Lack of time	73 (25)
Stigma	41 (14)
Total	292
If you were to use other help available –for example self-help books, counselling services etc. – which would you use and why?
Talking to friends/family – ‘because they know me better’	175 (50.7)
Counsellor – ‘because they are professionals’	57 (16.5)
Books –‘because they have credible information’	49 (14.2)
Total	345

Data collection

Responses from participants were collected automatically in three files, comprising: (a) a list of email addresses of students who opted to participate in the focus groups; (b) the time stamps of when a page was submitted; and (c) the respondents’ data input. Data were stored online and exported in password-protected Microsoft Excel files. The email addresses of the students were stored in encrypted format in the database and were decrypted on generation of comma-separated values (csv) files. They were stored without any link to the students’ responses, even in the database, in order to preserve full anonymity.

Data analysis

The quantitative data collected from this survey were analysed using IBM SPSS Statistics for Windows v.19. Summaries of the quantitative data were generated and presented in the form of tables and charts; these were produced using Microsoft Excel 2010. Qualitative data, generated through the free-text questions, were managed in Microsoft Excel 2010 and were analysed through thematic analysis [TA][50] With regards to data analysis, first, a total of five iConcipio research assistants worked each on data from one out of five HEIs participating in this study, under the supervision of the principal investigator. At a later stage, a senior research officer from iConcipio also analysed data from all institutions. Agreement between the research assistants and the research officer was excellent (agreement was reached in 94% of the cases). Across the board, two main clusters of themes were identified: the ‘positive’ feedback and the ‘negative’ feedback. During the analysis process, themes were identified and analysed, leading to the development of sub-clusters, all fitting under two main clusters. Thematic analysis was selected as the analysis method on the premise that it can provide flexibility, both in terms of the depth one could take the analysis to, and in terms of the epistemological assumptions underpinning it. In this instance, the aim was to ‘organise and describe the data set in (rich) detail’ (Braun and Clarke, 2006: 6),[51] and TA provided an excellent tool to achieve this.

Results

Participant characteristics

The full sample recruited were 873 students, 38.3% of which were male. Over half of them were between 20 to 25 years, equally dispersed between humanities and sciences. A large majority identified as white (72.5%) and a fair majority as UK students (64%). For full details refer to Table 2. Participants agreed to participate for various reasons. In particular, 52% reported that they thought the system was ‘a great idea and wanted to find out more’. Twenty-one percent reported they ‘experience difficulties at the university’ and 18% reported they ‘had nothing better to do’. Five percent mentioned that they had ‘friends who had difficulties’; and 4% reported other reasons, such as ‘finding the topic interesting’, ‘wanting to improve university experience’, ‘were concerned that difficulties will arise’, ‘taking the suggestion by University’, ‘curiosity’ or ‘past experiences’. The variety of reasons given enhances the significance of the results presented later, given the different perspective these participants took when experiencing and assessing the system.

Table 2.

Sample characteristics.

Demographic variables	n (%)
Gender
Male	339 (38.8)
Female	534 (61.2)
Age group
19 or below	204 (23.4)
20–25	466 (53.4)
26–30	103 (11.8)
Over 31	100 (11.5)
Ethnicity
Any white background	633 (72.5)
Any mixed background	31 (3.6)
Black/Black British	71 (8.1)
Asian/Asian British	17 (1.9)
Chinese or any other ethnic background	89 (10.2)
Prefer not to disclose	32 (3.7)
Status
UK student	560 (64.1)
EU student	158 (18.1)
International student (non-EU)	155 (17.8)
Discipline
Arts and Humanities	426 (48.8)
Sciences	447 (51.2)
Total	873

Respondent feedback

Results are presented in the following subsections, each of which corresponds to one of the five surveys administered. Quantitative and qualitative information is presented in the text (with reference to the tables, which can be found in Appendix 2).

On the questionnaire

In total 873 respondents provided feedback about the questionnaire, which was designed to identify psychological and study-skills difficulties. The feedback on the questionnaire was very positive (Appendix 2, Table 3). Between 65% and 86% of students gave positive ratings for phrasing, interactivity and usability of the questionnaires, as well as whether they liked the checking and pre-rating slides. Furthermore, 68% of respondents stated that they would contact the developer of the system should they feel that the list of difficulties identified was incomplete or inaccurate. Moreover, students suggested two improvements to the system. First, they suggested that questions asked could be shorter and more numerous, to cater for varying needs. Second, the graphics in the pre-rating slide could be revised to appeal more to older age groups.

On the workshops

In total, 668 respondents provided feedback on the workshops survey (Appendix 2, Table 4). For workshop titles see Figure 3. Between 88% and 91% of respondents provided positive feedback (i.e. responded ‘Yes’) regarding the structure, phrasing and presentation of the workshops. The question with the lowest percentage of positive responses referred to the usefulness of the option for students to take up to two workshops across the three categories, with 57% of participants responding positively. There was also a high percentage of respondents who would communicate topics not addressed to the developers (71% of participants responded positively).

Table 4.

Feedback on the workshops (n = 668).

Questions	Responses (n (%)
	Yes	So-so	No
Did you like the way topics are structured into categories?	607 (91)	52 (8)	9 (1)
Did you like the way the workshops were phrased and presented?	587 (88)	69 (10)	12 (2)
How useful was the option to take up to two of these workshops across the three categories?	378 (57)	255 (38)	35 (5)
Would you write back to us to suggest a topic we have not addressed?	474 (71)		194 (29)
Total	668

Figure 3.

Workshop titles.

When asked to propose additional topics the system could cover, students suggested the following: (a) sex, relationships and interactions; (b) study concerns and coping with academic life (i.e. work-life balance, exam stress and procrastination); (c) self-esteem and other self-related concerns; (d) difficulties with tutors/supervisors (especially relevant for PhD students); (e) dealing with sadness (depression, bereavement and self-harm); and (f) coping with change (homesickness, moving off campus). Again, respondents provided two suggestions for the system. First, they proposed that students be given the opportunity to select more than two workshops at a time. Second, they proposed that the option to report desired workshops that are not currently available should be easily accessible so that maximum feedback can be provided.

On My Place

In total, 578 respondents provided feedback on the students’ personal space, known as My Place, (Appendix 2, Table 5). Figure 4 presents a snapshot of My Place. This area allows students to tailor their learning approach (decide on a time plan to watch and practice the videos, as well as to receive reminders and messages of encouragement). Furthermore, this is where they can access their techniques and keep track of the videos they have watched, make notes and monitor their progress over time. Feedback on My Place was generally positive, with positive responses across questions ranging between 65% and 79%. The least favourable feature was the reminders and encouragement to use My Place (65% positive responses), followed by note-taking on thoughts about the techniques (67% positive responses).

Table 5.

Feedback on My Place (n = 578).

Questions	Responses (n (%)
	Very	Somewhat	Not at all
How useful was My Place?	418 (72)	135 (24)	25 (4)
How useful are the reminders and words of encouragement about My Place?	376 (65)	165 (29)	37 (6)
How useful was it to keep track of the video techniques you have watched	444 (77)	105 (18)	29 (5)
How useful is it to make notes with your thoughts about the techniques?	385 (67)	152 (26)	41 (7)
How useful is it to monitor your progress over time?	456 (79)	105 (18)	17 (3)
Total	578

Figure 4.

My Place.

Respondents suggested that the words of encouragement and reminder messages should be restricted to a certain frequency, possibly selected by the user. Additionally, the scales used to monitor individual progress over time might benefit from including more scale points and a different design. Respondents felt that this section would benefit from offering the option to have online contact with a professional. Lastly, respondents suggested that certain functionalities, such as ‘bookmarking’ their favourite techniques and a ‘blackout’ button to escape to another site if the system is used in a public space, would increase utility.

On the animated video technique

Feedback about the animation was provided by 549 respondents (Appendix 2, Table 6). Figure 5 presents a snapshot of an animated video. Between 79% and 92% of responses were positive, with the highest percentage given in response to the questions about the use of videos to illustrate the techniques. Levels of satisfaction were slightly lower for the character used in the videos, the narrator’s voice and the overall production of the video. Respondents provided two additional recommendations for the animated videos. First, they suggested that certain wordings of the spoken video content, as well as the colour scheme, could be altered to cater for older age groups. Second, the animation could be improved by offering customisation of the character and of the narrator’s voice.

Table 6.

Feedback on the animation (n = 549).

Questions	Responses (n (%))
	Yes	So-so	No
Did you like the use of videos to illustrate the techniques?	502 (92)	29 (5)	18 (3)
Did you like the character in the video?	432 (79)	87 (16)	30 (5)
Did you like the narrator’s voice?	456 (83)	71 (13)	22 (4)
Did you like the video’s production overall?	483 (88)	54 (10)	12 (2)
Total	549

Figure 5.

Video animation.

On the overall system (stylistic aspects)

Five hundred and forty-six respondents evaluated the aesthetics of the system (Appendix 2, Table 7). The proportion of participants selecting the most positive feedback category ranged from between 78% to 89% across all questions. Most respondents indicated that they liked the colours, the design, the fonts and the feel of the site. However, a minority of respondents stated that they found aspects of the design immature, including the colours. Some participants also stated that the content is attempting to do ‘too much’.

Table 7.

Feedback on stylistic aspects of the system (n = 546).

Questions	Responses (n (%))
	Yes	So-so	No
Did you like the colours of the website?	424 (78)	96 (17)	26 (5)
Did you like the fonts of the website?	487 (89)	45 (8)	14 (3)
Did you like the design and the layout?	486 (89)	37 (7)	23 (4)
Did you like the overall feel of the website?	466 (88)	65 (12)	15 (3)
Total	546

Recommendations for future content

Students were invited to provide feedback on the topics presented in the demo of the system. They were asked which further psychological difficulties and study skills they would like to see covered (see Appendix 2, Tables 8 and 9). They also provided feedback on any additional feature(s) they would like to see included in the system (Appendix 2, Table 10). Participants could select multiple additional features; percentages therefore represent the proportion of responses per item, rather than across all items.

Table 8.

What other topics, in terms of psychological difficulties, would you like to see addressed in the system? (n = 540).

Topic	Count	Yes (%)
Generalised anxiety	372	68.1
Depression	343	62.8
Anxiety in social settings	297	54.4
Perfectionism	267	48.9
Homesickness	233	42.7
Other (self-esteem, eating disorders, ADD, ADHD, dysthymia, sleep, social withdrawal, apathy, relationships, personality disorders, mood swings, OCD, phobias, competitiveness)	49	9.0

Table 9.

What other topics, in terms of study skills, would you like to see addressed in the system? (n = 540).

Topic	Yes (%)	Topic	Yes (%)
Procrastination	390 (73.0)	Effective diary keeping	182 (34.1)
Attention/concentration	334 (62.5)	Active learning	169 (31.6)
Time management	315 (59.0)	Organising ideas	167 (31.3)
Motivated learning	281 (52.6)	Overcoming writer’s block	160 (30.0)
Revision for exams	270 (50.6)	Active listening	145 (27.2)
Memory	252 (47.2)	Reflective learning	138 (25.8)
How to make notes	234 (43.8)	Receiving feedback	127 (23.8)
Improved reading	228 (42.7)	Making the group work	126 (23.6)
Organising notes	210 (39.3)	Structured writing	119 (22.3)
Saving time	210 (39.3)	Coping with discrimination	118 (22.1)
Presentation skills	202 (37.8)	Using feedback	102 (19.1)
Critical writing	199 (37.3)	Giving feedback	97 (18.2)
Dealing with difficult moments in group work	196 (36.7)	Space management	90 (16.9)
Improved comprehension	194 (36.3)	How to structure writing	68 (12.7)
Making your point in group work	192 (36.0)	Other (generating ideas, thesis preparation, using databases, managing multiple reading lists, effective communication with lecturers)	20 (3.7)
Being an effective group member	186 (34.8)

Table 10.

What other features would you like to see included in the system? (n = 540).

Feature	Yes (%)
Being able to browse all the different information and techniques available, plus ‘drag and drop’ option	461 (87.8)
Written information about techniques that you can use to effectively manage your difficulties	421 (80.2)
Written information to help make sense of your difficulties	386 (73.5)
Information about your university’s student support services/direction to professional help	372 (70.9)
Functionality as a mobile app	297 (56.6)
A user-support service allowing users to complete forms/send emails regarding technical problems with the website	338 (64.4)
Functionality in mobile devices	262 (49.9)
Videos with testimonials from real people	256 (48.8)
A forum for discussion with other students who have similar difficulties	237 (45.1)
Recommendations of self-help books	186 (35.4)

Psychological difficulties: the most common psychological difficulties suggested (from the most frequently reported to the least) included generalised anxiety (68.1%), depression (62.8%), anxiety in social settings (54.4%), perfectionism (48.9%) and homesickness (42.7%), as well as other difficulties such as attention deficit disorder (ADD), eating disorders and sleep disorders (9%) (Appendix 2, Table 8).

Study skills: respondents suggested several additional categories of study skills they would like to be covered by the system. Most common were (presented in order from the most frequently reported to the least): procrastination (73%), attention/concentration (62.5%), time management (59%), motivated learning (52.6%), exam revision (50.6%), memory (47.2%), note making (43.8%), and improved reading (42.7%) (Appendix 2, Table 9).

Other features: respondents provided some other more general suggestions on the use of the system. They suggested they would like to be able to browse all the different information available (87.8%) and they would like to be able to have a ‘drag and drop option’. Seven students reported that they wished to receive written information about techniques one can use to effectively manage difficulties and to help make sense of one’s difficulty (80.2%). Other functionalities that students also mentioned they would like to see included: information about their university’s student support services/direction to professional help (70.9%); a user support service (64.4%); functionality as a mobile application (56.6%); video testimonials from real people (48.8%); a forum for discussion with other students facing difficulties (45.1%); and recommendations of self-help books (35.4%) (Appendix 2, Table 10).

Name of system

Participants were asked to give their opinion about the name of the system, ‘mypassporter’. Participants did not respond particularly favourably to this name, with only 40% reporting that they liked it. Feedback suggested that participants would prefer a name that would give an indication of what the system does. More specifically, references to ‘me’ or ‘my’ were extremely common in names suggested by the students, possibly implying that people like the feeling of ownership that is associated with such pronouns. In addition, references to ‘aid’, ‘help’ and other synonyms were common, as were references to ‘students’ or ‘universities’.

Usefulness of the system and potential frequency of use

When judging the usefulness of the system (Appendix 2, Table 11), just over half of the students reported that they found the system useful. In terms of potential frequency of use, less than half of the students reported that they would use it ‘very frequently’ while more than half of them stated that they would use it ‘somewhat frequently’. Moreover, approximately half of the students reported they found the system ‘very useful’, with only 5% reporting that they found the system to be ‘not at all useful’. In addition, with regards intended frequency of use, only 6% of the students reported they would use the system ‘not at all often’.

Table 11.

Feedback on the system’s usefulness (n = 489).

Questions	Responses (n (%))
	Very	Somewhat		Not at all
How useful might such a system be to you?	266 (54)		200 (41)	23 (5)
How often might you use such a system?	192 (39)		270 (55)	27 (3)
Total	489

Least likeable features of the system

Asked about least likeable features of the system, one-third of the students stated that they did not specifically dislike anything about the system. Only 12% found the system more suitable for younger students, and about the same amount did not like the appearance and colours used in the demo (Appendix 2, Table 12).

Barriers to using the system and alternative options

Two barriers to use were identified by respondents: lack of time (25%) and fear of stigma (12%) (Appendix 2, Table 12). When asked about alternative methods of help that they would consider, students frequently reported that they would approach friends or family members ‘as they know them better’ (50.7%), they would consider talking to a counsellor ‘because they are professionals’ (16.5%), and that they would use books as they have ‘credible information’ (14.2%) (Appendix 2, Table 12).

Overall impression

The final question provided the opportunity to assess the students’ overall impression of the system, which was very positive. The majority of students were impressed with the system and stressed the advantage of privacy that it offers. One stated, ‘Really quite impressed – didn't expect to be’. Another commented, ‘Overall, I am impressed. I think it’s a great idea, something I know a lot of students would find useful whether or not they would admit it to their friends. It’s helpful, user-friendly, the colours are vibrant and the layout is trendy/modern’. Others found it a great idea and either drew from their own past experiences to conclude that it was something they would have benefited from if in place when they needed it or drew from what they knew to be true for other students. For example, one stated, ‘It's definitely a good idea that I know would have been fantastic to have had when having trouble myself last academic year’. Another reported, ‘Good idea, should be implemented asap. Many students suffer and this is an accessible means of calming and helping students’. Respondents noted the thoughtfulness and skilfulness they believed had gone into its development, stating, ‘It’s been excellently built … and very thoughtfully done’.

Most students (77%) provided positive feedback of the system; some provided a mix of positive and negative comments (16%); and a minority provided only negative feedback (7%). There were a couple of dismissive comments, for example, ‘Poor, not at all useful. You are trying to re-invent the wheel’. Certain participants questioned the clarity of the aim of this venture, ‘It’s ok but I’m not really clear what its aim is that other bits of the university aren’t already supplying’. Some users, while acknowledging the importance of the system, raised concerns about the quality of its final features, for example ‘It could be very good, but it depends on the quality of the information’, and ‘really good idea but somehow the website is not really strong’. Others raised their concern about the role of the system, saying, ‘It’s nicely presented, I thought the video was sweet but I come back to the fact I don’t believe that this kind of system can help. I don’t think it has the ability to give people the amount or kind of help they need and it is very general’. Finally, some participants communicated their concern about the effectiveness of such a system as the main or a complementary method of support. One student responded, ‘I liked the issues addressed, but it seems difficult to deal with personal issues through animations: I think that real life examples and perhaps the possibility to organise personal meetings/support groups would be more effective’. Another suggested, ‘I think it is a good idea for the resources on study skills and perhaps as a resource for relaxation techniques etc. but I don’t think that it could be used as a regimented system to help cope with personal problems’.

Discussion

In this study we provide evidence of students’ positive response regarding the need and the preliminary design of iConcipio’s online multimedia intervention system – aiming to support mood, wellbeing, study skills and everyday functioning in students in HE. The positive feedback received aligns with growing evidence on the need for computerised support for HE students.[5]

Comparing with feedback from other similar studies it is evident that students across the board praise peer-to-peer support (the social network function in our system) and request information and support on topics relevant to student life (such as homesickness, study stress or learning difficulties).[52] There are differences too. There have been studies reporting students’ requests for centralised information and access to professionals, for example;[52] there were no similar requests made in our study. As indicated by students’ feedback, the system is easy to use, it can cater for the needs of a complete cohort of students facing mild and moderate difficulties, and it can be used privately and anonymously. The study provided useful insight into how the system might be developed and improved to suit the needs and tastes of the target audience.

The comments and suggestions made by students as part of their feedback provided an excellent source of information and ideas from which iConcipio drew to finalise the design of the system’s prototype. First, the questionnaire items were simplified and the structure of the questionnaire was altered, such that future versions can be built by simply adding more questions. Second, the restrictive format of two workshops was removed and replaced with a full library. This library comprises all the video techniques from which students can directly choose those relevant to them, without any restriction on the number they can select at one time (see Figure 6). Third, in My Place, an option was added to allow students to choose whether or not they want messages to remind and encourage them to use the system. Furthermore, the frequency of these messages was reduced. Additionally, the note-taking option was linked with the social network, so students can choose whether they want their thoughts shared in the chat room or kept private in their profile. Fourth, the animated videos were enriched with more characters to offer variety. In future versions of the system, there is potential to add a greater degree of personalisation, for example to include the option to change the narrator’s voice. Fifth, wording and phrasing alongside various graphic design aspects of the system were reconsidered to ensure content was appropriate for older student groups. Sixth, certain functionalities such as blackout and bookmarking buttons were added. Finally, the tentative name of the system, MyPassporter, was changed to ‘MePlusMe’ to incorporate suggestions from the students.

Figure 6.

System update: from ‘Workshops’ route to ‘Library’ route.

Upon completion of the full system, a feasibility study (the third stage of development of this online intervention as briefly mentioned above) will be conducted to test the feasibility, acceptability and potential effects of the system.[34] This stage will further provide information regarding the extent of change between pre- and post-intervention ratings (mood, wellbeing, academic self-efficacy and everyday functioning), and will inform the implementation of future randomised trials assessing effectiveness.

MePlusMe aims to support HE students towards improved wellbeing and enhanced academic competence, either as a stand-alone service or complementary source of support to traditional face-to-face methods currently provided to HE students. It has the potential to reach a complete cohort of students, even those students whose mild to moderate needs are currently not met by SSS, due to limited recourses, or the hard-to-reach individuals who, for various reasons, do not seek actively for face-to-face support. The system is not designed for people with more severe difficulties; in order to address severe difficulties a different approach to the development and delivery of the system would have been considered altogether. However, students with severe difficulties can also benefit from using the system as it is. MePlusMe filters can identify students with severe difficulties and signpost them to other appropriate services such as their HEI SSS. Older research has shown the potential of online systems, with online measures being reported as better predictors of suicidal intentions than face-to-face interviews, for example.[31] More recently, systems for more severe cases have been developed: Deprexis,[53] for example, has been tested and found to be effective for people with severe depression.

Students using MePlusMe take control over their improvement, which could reinforce their motivation to change, self-efficacy and confidence. Furthermore, the built-in community of students could act as an additional support resource, normalising the students’ experiences, and giving them a feeling of belonging and fitting into a group. In addition to the above benefits for the students, MePlusMe could facilitate HEI’s SSS to optimise their resources. By focusing on mild to moderate difficulties, MePlusMe will emphasise prevention. Simultaneously, it will allow SSS to focus on the more pressing cases of students who face severe problems. This cost-effective solution could allow HEIs to enhance provision of pastoral and academic care for all students. Data analytics on students’ wellbeing and academic competence could further be provided, anonymously, to HEIs to inform their policies. Overall, HEI’s use of MePlusMe could improve student experience and help HEIs sustain improved course completion rates and obtain higher ratings, intake and income.

Online interventions, similar to MePlusMe, do not go without challenges. These include the potential for misunderstandings (due to lack of face-to-face opportunity for clarifications) or exclusion (due to reading and/or writing deficiencies of potential users), issues of dehumanisation of the therapeutic bond and, more importantly, issues of security, confidentiality and anonymity.[31] Suggestibility to communication modality[54] and preference however can offer a different view of such challenges. Indeed there are people who prefer to communicate at their own pace in the privacy on their own space and others that need the ‘right there and then’ face-to-face interaction; similarly there are people who read but cannot hear and others that hear but cannot read. Such variations highlight the need for multiple means of mental health provision that systems like MePlusMe can offer. Issues of security, confidentiality and anonymity are more complex as they require attention at the level of the design, technical development and delivery of an online intervention. MePlusMe, in particular, invites students who are willing to participate in the study to log into the system. They get authenticated using university-provided credentials via single sign-on (SSO) services (e.g. Shibboleth, OAuth, LDAP lookup). This process ensures that the individuals using the system are indeed students but they remain unidentifiable.

A set of additional challenges can be identified by focusing on the stakeholders’ response to the use of online interventions such as MePlusMe. Patients’ perceptions about and attitudes towards online interventions are positive across the board.[32,52] Positive is the response of academic staff who in their roles as academic advisors, tutors or supervisors are often students’ first point of contact when they face difficulties.[55] Practitioners, however, do not always have such positive response to online interventions.[31,32] There are those who accept the value of the medium and those who do not. Of those who do not appreciate online interventions, some feel threatened: it is hard to accept that patients trust an intervention to be as effective as face-to-face interaction with an actual counsellor.[32] Others stress issues such as the missing of non-verbal signals, misreading or cultural clashes.[54] The progressively increasing use of such technology, however, suggests that disadvantages of the medium, similar to those discussed here, are being recognised and addressed effectively in favour of those who can benefit from the use of this technology.

Turning to the proof-of-concept study in particular, the research aimed to obtain a representative sample of students by recruiting a large number of participants from a range of UK HEIs to provide feedback. One limitation of our study relates to the attrition rate from one question to the next. We anticipate that incorporating respondents’ suggestions will lower the attrition rates at the next stage. For now, we would expect that those who responded to the questions are likely the students who would be motivated to use the final system. This suggests that their feedback is most relevant for ensuring that the system meets the needs of the end users. Future research will seek to determine the reasons for attrition and the impact on the effectiveness of the intervention.

Conclusions

MePlusMe is the first online, personalised intervention to address mild to moderate psychological and/or study-skills difficulties in HE students. The conceptual and practical value of this system was investigated here in a proof-of-concept study and feedback was overwhelmingly positive. As a result, refinements and alterations were incorporated in the development of the prototype, the acceptability of which will be tested in a feasibility study that is currently under way.[34] A group of UK HEIs will participate in the feasibility study and will be given online access to MePlusMe. The feasibility study will further test recruitment procedures and outcome measures, and the system will be available for incorporation within HEIs as part of a soft launch period that will also facilitate a future trial of effectiveness, the last step before a full roll-out of MePlusMe to HEIs both nationally and internationally.