An Internet-based program for depression using activity and physiological sensors: efficacy, expectations, satisfaction, and ease of use.

PURPOSE: Computerized cognitive behavioral therapy (CCBT) has been shown to be efficacious. Moreover, CCBT can be enhanced by using physiological and activity sensors, but there is no evidence about the acceptability of all these tools. The objective of this study is to examine the efficacy, expectations, satisfaction, and ease of use of an Internet-based CCBT program for preventing depression, with and without sensors (electroencephalography, electrocardiograhpy ECG, and actigraphy), in a high-risk population (unemployed men). PATIENTS AND METHODS: Sixty participants at risk of depression (unemployed men) were randomly assigned to three experimental conditions: 1) intervention program (N=22), 2) intervention program plus sensors (N=19), and 3) control group (N=19). Participants completed depression, anxiety, positive and negative affect, and perceived stress measures. Furthermore, they also completed the measures for expectation, satisfaction, and the ease of use of the program.
RESULTS: Results showed that the two intervention groups improved significantly more than the control group on the clinical variables, and the improvements were greater in the group that used sensors than in the group that did not use them. Furthermore, participants in both intervention groups scored high on expectations and satisfaction with the CCBT program (with and without sensors). The mean score for usability was 88 out of 100 (standard deviation =12.32). No significant differences were found between groups on any of these variables.
CONCLUSION: This is the first study to analyze the efficacy, expectations, satisfaction, and ease of use of an Internet-based program using physiological and activity sensors. These results suggest that an Internet program for depression with or without physiological and activity sensors is effective, satisfactory, and easy to use.

RCT Entities:   Population Interventions Outcomes

Introduction

Depression is already estimated to be the second leading cause of disability worldwide.1 There is evidence of a significant relationship between depression, stress, and emotion regulation ability.2 In addition, people when depressed often use maladaptive emotion regulation strategies and have low coping behavior.3 Therefore, to prevent depression, it is important to develop strategies to promote adaptive emotion regulation strategies and resilience in people exposed to high levels of stress. Furthermore, we need to develop better tools to identify people at risk and design effective prevention programs.4 In addition, it is important to reach everyone in need. Less than 50% of people with emotional disorders receive appropriate treatment, and many patients are reluctant or find difficult to get help either due to its cost, the time required for its application, and/or the lack of well-trained professionals.5

Computerized cognitive behavioral therapy (CCBT) and the use of the Internet to support treatment implementation have been shown to be powerful means for its effective deployment in providing general mental health care helping to deliver interventions that promote healthy lifestyle and behaviors.6–8

Systematic reviews and meta-analyses support the efficacy of Internet-based CCBT for depression, anxiety disorders, chronic tension/migraine, trauma, insomnia, obesity, complicated grief, headache, and eating disorders.9–13

Research on this topic is growing at an impressive rate, but little is known about the expectations, acceptability, or satisfaction with CCBT.14–16 These variables can inform us about the feasibility of the intervention, which can help to optimize CCBT effectiveness.17,18 Although it is important to provide data about acceptability and opinions,19–23 most studies do not measure or report acceptability directly, but instead offer indirect indicators, such as take-up rates and patient dropout rates.14

Moreover, CCBT interventions may be improved by incorporating tools to collect useful information about progress, improvement, or stagnation throughout treatment and provide feedback to participants and clinicians in order to improve the effects of the intervention. The use of sensors, biosensors, or other technological tools within personalized health-care systems is a growing trend in the application of treatments.24,25 These emerging tools are fed by continuous technological advances, becoming wearable systems that can provide powerful support in health care for treating and preventing several psychological problems.26 Although there are a lot of mobile technologies for enhancing psychological wellness, there is little evidence supporting their efficacy, and even less regarding their acceptability.27,28 In the past few years, several European projects have focused on these tools, demonstrating the importance of this issue.2,29–33 All these projects have used noninvasive sensors and self-monitoring tools to enhance diagnosis, prevention, or treatment.34 As for sensors, it becomes important to ask not only about expectations and satisfaction, but also about the ease of use for participants. To the best of our knowledge, however, there are no published studies on their utility and acceptability. Moreover, to date, no study has examined variables such as participants’ expectations about them and satisfaction with an Internet self-administered CCBT program that includes the use of sensors. Furthermore, few studies have assessed usability issues in CCBT programs.35 So far, there are no studies on CCBT with sensors that directly assess these variables (expectations, satisfaction, and ease of use) to determine the level of acceptability by users.

Taking all of this into consideration, we developed a self-administered, multimedia and interactive Internet-based program. Its objective is to allow the individual to learn and practice adaptive ways to cope with depression and daily problems. It can be used with or without physiological and activity sensors.

The first aim of this study is to revise the efficacy of this CCBT program in people at risk of depression (unemployed men). The second aim is to analyze the participants’ expectations, satisfactions, and perceptions of ease of use of this CCBT program, both with and without the use of physiological and activity sensors. We hypothesize that the two intervention groups (with and without sensors) will improve significantly more than the control group on the clinical variables. Furthermore, taking into account that sensors provide additional information and feedback (electrocardiograhpy [ECG] for heart rate and motion, actigraphy [ACT] for ambulatory activity, electroencephalography [EEG] to derive affective status – sensitivity, and sleep quality using ECG and ACT), we hypothesize that the improvements on the clinical variables will be higher in the group with sensors than in the group without them. In addition, we expect all participants will score high on expectations, satisfaction, and ease of use of the CCBT program. We hypothesize that in both intervention groups (with and without sensors) the scores will be high, but we cannot predict whether they will be higher in one group than in the other. Although the group with sensors receives more feedback, their use might be cumbersome.

Material and methods

Intervention program

“Smiling is Fun” is an Internet-delivered, multimedia (video, images, etc), interactive, self-administered CCBT program for the prevention and treatment of depression and adjustment disorders. It allows the individual to learn and practice adaptive ways to cope with daily problems and depressive symptoms. It is designed for optimal use on a PC, but it can also be used on a tablet. “Smiling is Fun” combines the most effective psychological procedures for stress management with strategies to promote coping capacity, emotion regulation, and resilience. It is based on classic CBT techniques, such as behavioral activation,36,37 and it also includes other psychological strategies to improve positive mood.38 The intervention protocol consists of eight interactive modules. Each module includes exercises for practicing each technique. The program has two addition modules. The first one, named “Home module” (Figure 1), explains what “Smiling is Fun” is, its goal, who can benefit from it, terms and conditions, and who we are. The second one, “Welcome module,” informs users about the content of each module and explains how to benefit from it.

Figure 1

Home module.

The content of the eight modules is designed to break the vicious cycle of stress, isolation, depressive or anxiety symptoms, poor coping, and poorer functioning (self-efficacy). Each of the modules has the specific objectives described as follows: 1) “Motivation for change,” in which the advantages and disadvantages of change are analyzed and the importance of motivation for change is emphasized; 2) “Understanding emotional problems,” in which information regarding the recognition and understanding of emotional problems is provided; 3) “Learning to move on,” motivates the user to get involved in meaningful activities and to be engaged in their life; 4) “Learning to be flexible,” teaches the user how to consider different alternatives in order to be more flexible in interpreting situations; 5) “Learning to enjoy,” explains the importance of positive emotions and ways to promote them; 6) “Learning to live,” teaches the user how to identify their psychological strengths and the importance of carrying out activities based on their goals and values in life; 7) “Living and learning” focuses on developing and putting in practice the psychological strengths identified; and 8) “From now on, what else…?,” strengthens what has been learned during the program and emphasizes the importance of continuing the practice.

“Smiling is Fun” has three complementary transversal tools that accompany the user throughout all intervention program (IP). These tools are in the main menu of the IP and the participants can use them every day, when they access the program. 1) “Activity report,” it is a tool where the person is asked for his/her coping ability, mood state, and stress, on a 0–10 scale. Then, the user is asked about the degree of satisfaction with each activity performed during the past 24 hours and to what extent these activities are related to his/her own goals and values in life. Furthermore, the user is asked about the percentage of the day that he/she has been active or involved in his/her life for the last 24 hours (Figure 2). With all this information, the Activity Report provides graphical feedback to the users, showing the relationship between their coping ability, stress, and mood and the activities performed, and the benefits of being active. 2) “The calendar” is a tool that provides information about homework and tasks already achieved (listing the number of fulfilled tasks with a symbol represented by a yellow sun), reminding the user of those still outstanding, and it allows the users to know what point they stand throughout the program. Moreover, it marks with a green tick those days the users have been active in the Activity Report (Figure 3). 3) “How am I?,” which offers a set of graphs and feedback to chart the user’s progress, including activity level, emotional distress (anxiety and sadness), positive emotionality (active, enthusiastic, energetic, etc), negative emotionality (angry, fearful, stressed, tense, moody, etc), and information from sensors.

Figure 2

Activity report.

Figure 3

Calendar.

Sensors

“Smiling is Fun” includes several physiological and activity sensors: EEG and ECG sensors to detect subjects’ cognitive and physiological state, and ACT to characterize their physical activity (Figure 4). Participants can receive feedback from the sensors in graphic form.

Figure 4

From left to right, the heart rate and heart rate variability sensor worn on the neck, the activity sensor worn on the ankle, and the brain wave sensor.

Note: The red arrow indicates the correct position of the ACT sensor. The circular washer must be placed in the direction of the arrow.

Abbreviations: ECG, electrocardiography; ACT, actigraphy; EEG, electroencephalography.

The ECG sensor records heart rate, heart rate variability, and physical motion. This sensor provides information that may be an indicator of stress based on heart waveform variability.39,40 The ECG waveform is well known, and under normal conditions it behaves in a well-defined and very predictable manner.

The ECG sensor is used continuously for 24 hours. The information is stored in the sensors’ FLASH memory, which has a limited capacity of 27 hours. Therefore, a routine is defined whereby the user has to download data once a day. It works as follows: remove the ECG sensor, put it close to the home PC (explained in the Home PC platform section), run the download program on the home PC, put the sensor on the battery charger, get a second fully charged sensor, run the home PC software to activate it to start recording, and put it again on the body.

The ACT sensor records several physical motion parameters: resting, walking, jogging, running, stairs, and cycling. More specifically, it monitors the frequency and intensity of regular exercise, sports activity, and sleep patterns. Evidence shows that regular physical exercise is a moderator of mood and prevents and improves depression.41 The ACT sensor has been designed to be worn on the lower leg. It samples the three axis accelerations of the leg, and implements Hidden Markov Model-based classifiers that recognize activity and estimate total energy expenditure. The procedure follows the same routine as the one described for ECG. When users download data and activate the ECG sensor, they repeat the routine with the ACT sensor, which has also been worn for 24 hours.

The EEG sensor records the electrical potential gradient oscillating around the brain.42 It detects anterior brain activities related to emotional stress, and it has been used to detect changes in the behavior, sleep anomalies, and mental acuity related to depression.43 Alpha asymmetry and C0 complexity are of particular interest, as well as Beta/Alpha and Alpha/Theta power band ratios, as they can reflect a person’s mood. This sensor records the user’s EEG signals over periods of approximately 90 seconds to obtain sufficient data for the algorithms. Additionally, the sensor provides a means to grade the quality of contact between the electrodes and the skin on the forehead, so that the contact quality is sufficient for the raw data to be useful. The EEG recording session lasts for 3 minutes. The user locates the sensor on the forehead and performs several actions as defined by the home PC application, which stores data automatically. The participants were free to perform the EEG task every day, but they had to do it at least once a week during the entire trial (6–10 weeks).

Home PC platform

A home PC platform has been designed to enable the users to collect, download, and store information using the sensors. The home PC has a “look and feel” wizard that guides the users through the different steps. Participants receive a training session in the laboratory to practice all the processes.

Research design

A randomized controlled trial compared three experimental conditions: 1) IP (N=22): participants had access to the CCBT IP without sensors. 2) Intervention program plus sensors (IP + S, N=19): participants had access to the CCBT IP, and they used sensors. Participants in both intervention groups could use the three transversal tools explained in the Intervention program section. However, only the participants in the IP + S group had access to the see the graphical feedback regarding the sensor information, provided by the “How am I” tool. 3) Control condition (C, N=19): the participants answered the pretreatment questionnaires, waited 10 weeks, and then answered the posttreatment and follow-up questionnaires.

Sample

Unemployed men were selected because they are considered a high-risk population. As the Analyses of Epidemiologic Catchment Area panel data reveal, the unemployed have over twice the risk of becoming clinically depressed as those who continue to be employed.44 The sample consisted of 60 unemployed males. The mean age was 32.35 (SD =9.55). The mean duration of unemployment was 15.30 months. Regarding marital status, the majority of the participants (66.7%) were single, 30% were married or had a partner, and 3.3% were divorced. Regarding the educational level, 8 participants (13.3%) had a high-school diploma or less, 17 participants (28.3%) had some college, and 35 participants (58.3%) had a college degree or graduate degree. On the Beck Depression Inventory second edition (BDI-II; possible range of scores: 0–63), participants had a mean preassessment score of 4.82 (SD =4.61).

Inclusion and exclusion criteria

Inclusion criteria were as follows: unemployed men suffering stress due to financial concerns and family obligations, aged between 18 and 65, who agreed to participate in the study and signed an informed consent, and who had access to a computer with Internet connection. Exclusion criteria included proneness to skin allergies that might be exacerbated by wearing stick-on sensors, personal or first-degree relative history of depression/psychosis, BDI II ≥19, epilepsy, heart conditions (could interfere with ECG measurement), daily use of recreational drugs (eg, cannabis), regular use of sleep medication, regular use of medication for heart, and/or cardiovascular conditions (eg, antihypertensive).

Measures

Mini International Neuropsychiatric Interview (MINI)

This is a structured diagnostic psychiatric interview that yields key diagnoses from the Diagnostic and Statistical Manual of Mental Disorders, fourth edition and the International Classification of Diseases. The MINI can be administered by clinical interviewers who need only brief training. The MINI has been translated into Spanish and validated.45

Beck Depression Inventory II (BDI II)

The BDI II is one of the most widely used questionnaires for evaluating depression severity in clinical trials. The instrument has shown good internal consistency (Cronbach’s α of 0.76–0.95) and test–retest reliability of about 0.8.46 The Spanish version of this instrument has also shown high internal consistency for both the general population (Cronbach’s α of 0.87) and the clinical population (Cronbach’s α of 0.89).47

Overall Anxiety Severity and Impairment Scale (OASIS)

The OASIS consists of five items that measure the frequency and severity of anxiety, as well as the level of avoidance, work school/home interference, and social interference associated with anxiety. A psychometric analysis of the OASIS scale found good internal consistency (Cronbach’s α=0.80), test–retest reliability (k=5.82), and convergent validity.48

Positive and Negative Affect Scale (PANAS)

The PANAS consists of 20 items that evaluate two independent dimensions: positive affect (PA) and negative affect (NA). The range for each scale (10 items on each) is from 10 to 50. The Spanish version has demonstrated high internal consistency (0.89–0.91 for PA and NA, respectively, in women, and 0.87 and 0.89 for PA and NA, respectively, in men) in college students.49

Perceived Stress Scale (PSS)

The PSS is a 14-item self-report questionnaire that assesses the perceived stress associated to recent life situations. The Spanish validation of this scale has an internal consistency of 0.86.50

Expectation of Intervention Scale and Opinion of Intervention Scales

These questionnaires were adapted from Borkovec and Nau.51 Each scale consists of five items, rated from 0 (“strongly disagree”) to 10 (“strongly agree”), which cover how logical the intervention seemed, to what extent it satisfied the participant, whether it could be used to treat other psychological problems, and its usefulness for the participant’s specific problem. The expectation scale is applied once the intervention rationale has been explained, at the end of the second module. Its aim is to measure the participant’s subjective expectation of the intervention. The opinion scale is administered when the participant has finished using the program, and its aim is to assess satisfaction with this intervention. Our group has used these questionnaires in several studies.19

System Usability Scale (SUS)

The SUS scale was developed by Brooke52 to assess the usability of a given product or service. It is a simple, 10-item Likert scale that provides a global view of the subjective assessment of usability. The SUS has proven to be a valuable and robust tool for assessing the quality of a wide range of user interfaces. It has several characteristics that make it a good choice for general practitioners: it is technology neutral, allowing the evaluation of a wide range of interface technologies; it is quick and easy to use for participants and administrators; it can provide a single score; and it is easy to understand.53

Figure 5 shows the mapping of the adjective ratings corresponding to acceptability ranges and quartiles.53

Figure 5

A comparison of mean SUS scores by quartile, adjective ratings and the acceptability of the overall SUS score.

Abbreviation: SUS, System Usability Scale.

Statistics and data analysis

Group differences in demographic data and pretreatment measures were analyzed with one-way analysis of variance (ANOVA) and chi-square tests to find out if there were significant differences among the groups. For the clinical variables, all posttreatment analyses involved a conservative intention-to-treat (ITT) design, where missing data were addressed by carrying forward the last available data (ie, last-observation-carried forward model). To test the treatment efficacy and the differential efficacy of the three experimental conditions, mixed ANOVAs with repeated measures were used to compare the time effect on the measures (pre- to posttreatment, and pre, post, 3 month follow-up) and the time interactions among the conditions. The ANOVAs were followed by Sidak’s post hoc tests whenever it was needed. Because we found differences in the three experimental groups at pretreatment in the BDI-II variable, it was analyzed using univariate analyses of covariance (ANCOVAs), assigning pretreatment scores as the covariate. Effect sizes (Cohen’s d) were calculated for within- and between-group changes, based on the pooled standard deviation. The expectation, satisfaction, and usability scores were also analyzed using mixed ANOVA with repeated measures. All statistical analyses were conducted using IBM SPSS Statistics 20 (IBM Corporation, Armonk, NY, USA).

Participant’s flow

Regarding the flow of the participants throughout the investigation process, Figure 6 shows that 622 people were interested in the study. The most important reason for exclusion was sex of the participant (354 women were excluded because only unemployed men were studied). We called all men (n=286) by telephone (prescreening). About 189 participants were excluded after this phone interview (53 did not answer the phone call, 52 did not meet the inclusion criteria, 4 were not interested in the study, and 60 did not live in Castellón or Valencia) and 79 men were invited to participate and performed the face-to-face interview. Finally, 69 participants were included and randomly assigned to the three experimental conditions: IP + S, n=20; IP, n=25; C, n=24, and 60 participants did the pretrial assessment: IP + S, n=19; IP, n=22; C, n=19.

Figure 6

Flow chart.

Abbreviations: IP, intervention program group; IP + S, intervention program plus sensors group; C, control group; BDI-II, Beck Depression Inventory II.

During the IP, there were three dropouts, and all of them in the IP group. In group C, there were two dropouts at postassessment.

At the 3-month follow-up, there were two dropouts in the C group.

Procedure

The study was approved by the ethics committee at Jaume I University. Participants were recruited through direct contact with local job centers, advertisements in the media and Internet, and announcements to the university community. The advertisements described the research as a study on stress management and depression prevention, and as an opportunity for personal growth specially designed for unemployed people. The advertisements also stated that prospective participants would receive 15 Euros to cover travel expenses any time they had to come to the laboratory. Finally, the advertisements outlined the exclusion criteria and the final incentive offered to participants: the possibility of keeping the netbook they used during the study if they were in the group that wore sensors.

All the volunteers had to answer an online questionnaire to see if they met the minimum requirements for joining the study (unemployed men). Women were excluded. Next, we prescreened possible participants by phone to filter other important criteria: personal or first-degree relative history of depression, epilepsy, heart conditions, daily use of recreational drugs, sleep medication, or medication for heart and/or cardiovascular conditions (eg, antihypertensive).

Participants who passed the prescreening by phone were given a date for an initial face-to-face interview (screening) in our laboratory, where a research team member explained the study. Participants interested in participating signed an informed consent form. Subsequently, the investigator asked for demographic information, evaluated the presence of depression with the MINI and the BDI-II, and gathered other information about the inclusion and exclusion criteria. The selected participants were then randomly assigned to the three experimental conditions.

Finally, 60 participants took part in the study and were assigned to each of the three experimental conditions: IP =22, IP + S =19, C =19. All the participants completed baseline measures, including questionnaires. Only after filling in the assessment protocol were participants told to which group they had been allocated. This marked the beginning of the use of the CCBT program, which lasted between 6 and 10 weeks (participants were asked to complete a fixed number of CCBT recording sessions, but set their own pace for the number of sessions per week).

At the end of the second module, the participants in the intervention groups completed the scale on their expectations about this program (see “Measures” section). Both when finishing the program and at the 3-month follow-up, the participants had to again answer the assessment protocol. Furthermore, the participants in both intervention groups were asked about their level of satisfaction with the program and about its ease of use.

Results

Pretreatment data

The results indicated that there were no significant differences between the three groups before treatment on any of these variables: marital status (χ2 [2] =3.332; P=0.504), educational level (χ2 [2] =8.963; P=0.062), and age (F(2,57)=0.701; P=0.500).

Regarding the clinical characteristics (Table 1), the data indicate that there were no significant differences among the three groups on any of the variables before treatment, with the exception of BDI-II (F(2,57)=3.404, P=0.040). Post hoc analysis (Tukey HSD) revealed that the IP group scored higher than the other two conditions (IP + S and C).

Table 1

Clinical characteristics of participants at preassessment

Mea	Gr	N; M (SD)	F(2,59)	P-value
BDI-II	IP + S	19; 3.53 (3.31)	3.404	0.040
	IP	22; 6.77 (5.61)
	C	19; 3.84 (3.82)
	Total	60; 4.82 (4.61)
OA	IP + S	19; 1.21 (1.93)	2.200	0.120
	IP	22; 2.59 (2.34)
	C	19; 1.63 (2.19)
	Total	60; 1.85 (2.21)
PA+	IP + S	19; 28.16 (7.54)	2.670	0.078
	IP	22; 29.50 (8.06)
	C	19; 33.84 (8.24)
	Total	60; 30.45 (8.18)
PA−	IP + S	19; 14.21 (4.05)	0.910	0.408
	IP	22; 15.64 (4.49)
	C	19; 14.21 (3.07)
	Total	60; 14.73 (3.94)
PSS	IP + S	19; 4.37 (2.14)	2.461	0.094
	IP	22; 5.27 (2.75)
	C	19; 3.53 (2.59)
	Total	60; 4.43 (2.58)

Abbreviations: Mea, measures; Gr, group; N, number of participants; M, mean; SD, standard deviation; IP, intervention program group; IP + S, intervention program plus sensors group; C, control group; BDI II, Beck Depression Inventory II; OA, Overall Anxiety Severity and Impairment Scale; PA+, Positive Affect Scale; PA−, Negative Affect Scale; PSS, Perceived Stress Scale.

Differential effectiveness of three conditions: change in outcomes pretreatment, posttreatment, and follow-up

Means and standard deviations and effect sizes for all outcome measures are listed in Table 2.

Table 2

Mean, SD, and effect sizes for the outcome measures at pretreatment, posttreatment, and 3-month follow-up (ITT analysis)

Mea	Gr	N; M (SD)			Within-group effect size, d (95% CI)a	Between-group effect size, d (95% CI)b
		Pretreat	Posttreat	F-up (3m)
BDI-II	IP + S	19; 3.53 (3.31)	19; 2.16 (2.36)	19; 1.53 (1.84)	0.77 [−0.06 to 1.60]	0.69 [−1.72 to 0.35]
	IP	22; 6.77 (5.61)	22; 5.05 (5.33)	22; 3.86 (4.41)	0.59 [−0.87 to 2.05]	0.49 [−1.54 to 0.56]
	C	19; 3.84 (3.82)	19; 3.32 (4.76)	19; 3.16 (4.43)	0.17 [−1.11 to 1.45]	0.16 [−1.16 to 1.48]
	Total	60; 4.82 (4.61)	60; 3.58 (4.50)	60; 2.90 (3.87)
OA	IP + S	19; 1.21 (1.93)	19; 0.68 (1.45)	19; 0.63 (1.17)	0.37 [−0.12 to 0.87]	0.59 [−1.11 to 0.07]
	IP	22; 2.59 (2.34)	22; 1.64 (2.11)	22; 1.64 (2.12)	0.44 [−0.21 to 1.08]	0.73 [−1.24 to 0.23]
	C	19; 1.63 (2.19)	19; 1.26 (2.66)	19; 1.79 (1.99)	0.08 [−0.73 to 0.57]	0.08 [−0.69 to 0.54]
	Total	60; 1.85 (2.21)	60; 1.22 (1.80)	60; 1.37 (1.86)
PA+	IP + S	19; 28.16 (7.54)	19; 34.47 (10.66)	19; 34.00 (9.41)	0.70 [−3.34 to 1.94]	0.14 [−2.53 to 2.81]
	IP	22; 29.50 (8.06)	22; 32.50 (9.38)	22; 32.77 (8.53)	0.40 [−2.80 to 1.99]	0.14 [−2.53 to 2.81]
	C	19; 33.84 (8.24)	19; 34.26 (8.14)	19; 35.11 (8.86)	0.15 [−2.80 to 2.50]	0.28 [−2.97 to 2.42]
	Total	60; 30.45 (8.18)	60; 33.68 (9.33)	60; 33.90 (8.82)
PA−	IP + S	19; 14.21 (4.05)	19; 12.26 (2.73)	19; 12.11 (2.66)	0.63 [−0.43 to 1.69]	0.55 [−1.52 to 0.42]
	IP	22; 15.64 (4.49)	22; 13.23 (3.61)	22; 13.86 (3.70)	0.44 [−0.75 to 1.63]	0.70 [−1.67 to 0.25]
	C	19; 14.21 (3.07)	19; 14.47 (3.42)	19; 14.26 (3.51)	0.01 [−1.04 to −1.00]	0.11 [−1.19 to 0.97]
	Total	60; 14.73 (3.94)	60; 13.32 (3.36)	60; 13.43 (3.42)
PSS	IP + S	19; 4.37 (2.14)	19; 2.53 (2.41)	19; 3.21(2.40)	0.52 [−0.18 to 1.23]	0.38 [−1.09 to 0.32]
	IP	22; 5.27 (2.75)	22; 4.55 (2.44)	22; 4.09 (2.33)	0.47 [−0.26 to 1.21]	0.09 [−0.81 to 0.63]
	C	19; 3.53 (2.59)	19; 3.47 (2.37)	19; 3.42 (2.24)	0.04 [−0.70 to 0.80]	0.30 [−0.38 to 0.98]
	Total	60; 4.43 (2.58)	60; 3.57 (2.51)	60; 3.60 (2.32)

Notes:

Pre-F-up (3m).

IP + S vs IP; IP + S vs C; IP vs C.

Abbreviations: Mea, measures; Gr, group; N, number of participants; M, mean; SD, standard deviation; Pretreat, pretreatment assessment; Posttreat, posttreatment assessment; F-up (3m), 3-month follow-up assessment; IP, intervention program group; IP + S, intervention program plus sensors group; C, control group; BDI II, Beck Depression Inventory II; OA, Overall Anxiety Severity and Impairment Scale; PA+, Positive Affect Scale; PA−, Negative Affect Scale; PSS, Perceived Stress Scale.

The analyses revealed a significant time effect on all measures: BDI-II (F(2,114)=10.63, P<0.001), OASIS (F(2,114)=3.24, P=0.043), PANAS + (F(2,114)=12.99, P<0.001), PANAS − (F(2,114)=7.36, P=0.001), and PSS (F(2,144)=7.32, P=0.001).

More interesting, the analysis revealed a significant interaction effect on all measures, except OASIS.

Regarding the BDI-II, an ANCOVA was performed to control for differences found at pretreatment. Covariate analysis indicated that the groups did not differ significantly at posttreatment (F(2,56)=0.426, P=0.655).

Regarding PANAS PA scores (Time × group (F(4,114)=2.649, P=0.037), the IP + S group improved significantly pre- to posttreatment (P<0.001) and pretreatment to follow-up (P=0.001), and improvements obtained at posttreatment were maintained at follow-up. The IP group did not improve significantly pre- to posttreatment (P=0.114), but it improved significantly pretreatment to follow-up (P=0.050). The C group did not improve pre- to posttreatment (P=0.990) or pretreatment to follow-up (P=0.763). Regarding PANAS NA scores (Time × group (F(4,114)=2.503, P=0.046), the IP + S group improved significantly pre- to posttreatment (P=0.029) and pretreatment to follow-up (P=0.024), and improvements obtained at posttreatment were maintained at follow-up. In the IP group, the participants also improved significantly pre- to posttreatment (P=0.002) and pretreatment to follow-up (P=0.046), and improvements obtained at posttreatment were maintained at follow-up. The C group did not improve pre- to posttreatment (P=0.978) or pretreatment to follow-up (P=1).

As for perceived stress (PPS scores) (Time × group (F(4,114)=2.536, P=0.044), the IP + S group improved significantly pre- to posttreatment (P=0.002), and improvements obtained at posttreatment were maintained at follow-up. The IP group did not improve significantly pre-to posttreatment (P=0.337), but pretreatment to follow-up, the improvement was marginally significant (P=0.052). The C group did not improve pre- to posttreatment (P=0.999) or pretreatment to follow-up (P=0.996).

Regarding OASIS, the ANOVAs did not reveal a significant interaction effect (F(4,114)=0.426, P=0.655). However, the mean scores and the effect sizes suggest a benefit for the intervention groups. Their mood and anxiety improved more than in the control group pre- to posttreatment and pretreatment to follow up. In the C group, we observed only a minimal change.

Expectations and satisfaction

Table 3 lists the results for both intervention groups. As can be observed, before the treatment the scores were very high. The ANOVA did not find significant differences between the two groups on any measures: treatment logic (F(1,36)=0.009, P=0.925), treatment satisfaction (F(1,36)=0.056, P=0.815), recommending the treatment to others (F(1,36)=0.006, P=0.941), usefulness of the treatment for other psychological problems (F(1,36)=0.091, P=0.765), and usefulness of the treatment for the patient (F(1,36)=0.018, P=0.895).

Table 3

Means and standard deviations for expectations (pretrial) and satisfaction (posttrial)

Statements	Expectations N; M (SD)	Satisfaction N; M (SD)
How logical do you think this treatment is?
Total sample	41; 8.00 (1.66)	41; 8.34 (1.47)
IP	22; 8.00 (1.41)	22; 8.37 (1.71)
IP + S	19; 8.00 (1.94)	19; 8.32 (1.25)
How satisfied are you with the treatment received?
Total sample	41; 7.63 (1.97)	41; 8.03 (1.55)
IP	22; 7.59 (1.37)	22; 7.89 (1.82)
IP + S	19; 7.68 (2.54)	19; 8.16 (1.26)
To what extent do you feel confident recommending this treatment to a friend who had the same problems?
Total sample	41; 8.22 (2.10)	41; 8.79 (1.52)
IP	22; 8.27 (1.39)	22; 8.79 (1.13)
IP + S	19; 8.16 (2.75)	19; 8.79 (1.87)
Do you think this treatment could be useful for treating other psychological treatments?
Total sample	41; 8.22 (1.62)	41; 8.39 (1.15)
IP	22; 8.09 (1.15)	22; 8.37 (1.06)
IP + S	19; 8.37 (2.06)	19; 8.42 (1.26)
To what extent do you think the treatment was helpful for you?
Total sample	41; 6.90 (2.36)	41; 7.16 (2.07)
IP	22; 6.95 (2.01)	22; 6.95 (2.12)
IP + S	19; 6.84 (2.77)	19; 7.37 (2.06)

Abbreviations: N, Number of participants; M, mean; SD, standard deviation; IP, intervention program group; IP + S, intervention program plus sensors group.

After using the program, scores of participants on satisfaction were very high. The ANOVA did not show any differences between groups on treatment logic (F(1,36)=0.012, P=0.914), treatment satisfaction (F(1,36)=0.268, P=0.608), recommending the treatment to others (F(1,36)=0.002, P=0.963), usefulness of the treatment for other psychological problems (F(1,36)=0.019, P=0.890), or usefulness of the treatment for the patient (F(1,36)=0.385, P=0.539).

See the video of a participant giving his opinion about the CCBT program (Video S1).

Ease of use

Table 4 provides the results of the SUS overall score and items. ANOVAs (Table 5) did not show any significant effect, indicating that all participants found the system very usable and acceptable, and that wearing sensors did not affect a good rating in terms of the system’s usability and acceptability (see the video of a participant giving his opinion about the sensors (Video S2).

Table 4

Means and standard deviations for the SUS

Statements	Total sample N; M (SD)	IP N; M (SD)	IP + S N; M (SD)
I would like to use this system frequently.	38; 3.89 (1.20)	19; 3.79 (1.36)	19; 4.00 (1.05)
I found the product unnecessarily complex.	38; 1.42 (0.98)	19; 1.26 (0.93)	19; 1.58 (1.02)
I thought the system was easy to use.	38; 4.50 (0.69)	19; 4.53 (0.70)	19; 4.47 (0.69)
I think that I would need the support of a technical person to be able to use this system.	38; 1.24 (0.63)	19; 1.11 (0.31)	19; 1.37 (0.83)
I found the various functions in the system were well integrated.	38; 4.39 (0.79)	19; 4.42 (0.69)	19; 4.37 (0.89)
I thought there was too much inconsistency in this system.	38; 1.53 (0.86)	19; 1.47 (0.77)	19; 1.58 (0.96)
I would imagine that most people learn to use this system very quickly.	38; 4.63 (0.59)	19; 4.68 (0.58)	19; 4.58 (0.61)
I found the system very cumbersome to use.	38; 1.26 (0.64)	19; 1.21 (0.53)	19; 1.32 (0.75)
I felt very confident using the system.	38; 4.50 (0.76)	19; 4.58 (0.77)	19; 4.42 (0.77)
I needed to learn a lot of things before I could get going with this system.	38; 1.32 (0.77)	19; 1.11 (0.46)	19; 1.53 (0.096)
Overall SUS scoring.	38; 87.89 (12.32)	19; 86.18 (14.98)	19; 86.60 (9.02)

Abbreviations: N, number of participants; M, mean; SD, standard deviation; IP, intervention program group; IP + S, intervention program plus sensors group; SUS, System Usability Scale.

Table 5

ANOVA results for overall scoring and single items in the SUS

Statements	Group effect
	F(1,37)	P-value
I think that I would like to use this system frequently.	0.285	0.597
I found the product unnecessarily complex.	0.994	0.325
I thought the system was easy to use.	0.054	0.817
I think that I would need the support of a technical person to be able to use this system.	1.667	0.205
I found the various functions in the system were well integrated.	0.041	0.840
I thought there was too much inconsistency in this system.	0.138	0.712
I would imagine that most people learn to use this system very quickly.	0.298	0.589
I found the system very cumbersome to use.	0.248	0.621
I felt very confident using the system.	0.401	0.531
I needed to learn a lot of things before I could get going with this system.	2.954	0.094
SUS total scoring.	0.727	0.400

Abbreviations: ANOVA, analysis of variance; SUS, System Usability Scale.

Discussion

The first objective of this work was to analyze the efficacy of the self-administered treatment via the Internet (with and without sensors) in improving the different clinical variables evaluated in a population under prolonged stress (unemployed men) at risk of developing depression.

The results show that the two intervention groups improved more than the control group on their clinical variables, and that the improvements were greater posttreatment in the group that used sensors than in the group that did not use them. Considering the effect sizes, the IP + S group obtained medium effect sizes in all the variables, with the exception of the OASIS (small effect size). The IP group obtained a medium effect size on the BDI-II, and small effect sizes on the other measures. The effect sizes in the C group were minimal.

Furthermore, the improvements on some of these variables were maintained at the 3-month follow-up.

It is important to mention that participants were a population at risk of depression (unemployed men), but they were healthy volunteers with no psychological problems. Despite their long-term unemployment situation, their score on the BDI-II (4.82, SD =4.61) and other measures at pretreatment were low. Even so, depression and other clinical variables improved more in both intervention groups, compared to the control group. Our results are in line with those presented in systematic reviews and meta-analyses showing that computerized and Internet-based treatments are effective interventions.12,54–57

The other objective of this study was to analyze the expectations, satisfaction, and perceived ease of use by the participants in both intervention groups. The participants in this study reported positive expectations of the program, coinciding with other studies.9,14 The expectations were high for both groups, regardless of the use of sensors, suggesting that participants found the treatment very logical, useful even for other psychological problems, and they felt confident about recommending it to a friend. Participants’ opinions and of satisfaction after using the program were also high, with no statistically significant differences between both intervention groups.

The results obtained regarding the ease of use would place the IP between the third and fourth quartile, among the best products and close to truly superior products (Figure 5). It should be noted that the overall scores on usability (SUS questionnaire) for both groups were very positive. High ratings on satisfaction and ease of use have been obtained, indicating that “Smiling is Fun” with and without sensors is considered very usable. All these results support our first hypothesis, as they were positive in both intervention groups. The results highlight that all the participants, regardless of grouping, accepted the program very readily, whether they used sensors or not.

Our results are in line with the literature that indicates that people treated with CCBT Internet-based programs report high levels of acceptability, positive expectations, and high satisfaction (of those completing the treatment).9,13 Very few studies include instruments specifically designed to determine the participants’ attitude toward CCBT.21,23 As previously mentioned, most studies offer limited information on indicators of acceptability (recruitment, take-up, and dropout rates).14 Similarly, although there is an increasing use of noninvasive sensors and self-monitoring tools,34 we have no data on participants’ satisfaction and usability, specifically in the literature that use behavioral and physiological sensors in combination with CCBT. In our study, the participants seem to accept these technological innovations readily, and they are willing to use different kinds of sensors.

In conclusion, the results of this study show that the use of Internet-based psychological treatments can result in good participant evaluation and acceptability.

Sensors can provide participants and clinicians with very useful feedback and information (especially for adapting and tailoring the treatment to the needs of each user), but they also have drawbacks. First, in many cases they are still not very robust prototypes and can result in failures or problems when used. Second, they are not yet well integrated into the CCBT systems (in our case, the participants had to use two technological platforms), and this requires extra work. Despite all these potential difficulties, however, the observed trend is that more and more people are using different sensors and specific applications for health promotion.34 These findings open the door to the use of CCBT plus sensors. We think this is the beginning of a new era in the psychological intervention field. The use of CCBT Internet-delivered programs and sensors is innovative, and further developments in the field are inevitable, such as the union between virtual reality techniques and the Internet; the use of avatars that guide, motivate, and help participants throughout the treatment process; the inclusion of serious games designed for specific people with specific difficulties; and the possibility of delivering important parts of the treatment using mobile devices. Information and communication technologies (ICTs) have arrived on the psychological treatment scene, and they are here to stay.

We need to perform additional research to make the ICTs use easier and define who can benefit from what type of application or what type of sensor.

We are aware that this study has limitations, such as the sample size and difficulties related to external validity because all the participants were men. Another limitation was the availability of sensors. They were prototypes, and we had a maximum of 25 of each type, which determined the small sample size. It is possible that the small sample size influences the results obtained regarding the nonsignificant differences between intervention groups. Furthermore, the fact that the sensors were prototypes may have influenced the results, and more solidly built versions could produce more definitive results. The opportunity to keep the netbooks could be a quite important reward for participants in the IP + S group. It would have been interesting to analyze whether this had any influence on their satisfaction and opinion. In any case, the total cost of each notebook (€400) is not something extraordinary, given the registration tasks participants were asked to perform.

It should also be kept in mind that participants used several sensors, but the specific utility of each sensor was not assessed. In future studies, participants will be able to choose (or the clinician will be able to suggest) a specific sensor or group of sensors with a precise goal. Finally, data are limited at 3-month follow-up.

Conclusion

Despite its limitations, this is the first study to use an Internet-based program plus sensors, and it demonstrates that the self-administered program was effective with and without sensors. In some cases, the improvements were greater for the sensor group than for the group that did not use sensors (although no significant differences were found between the two intervention groups). Furthermore, the self-administered program is very easy to use and well accepted (with and without sensors), and it opens the door to the use of devices such as physiological sensors in the field of Internet-delivered psychological interventions.