Piloting a scalable, post-trauma psychosocial intervention in Tuvalu: the Skills for Life Adjustment and Resilience (SOLAR) program.

Background: The Skills for Life Adjustment and Resilience (SOLAR) programme is a brief, scalable, psychosocial skill-building programme designed to reduce distress and adjustment difficulties following disaster.
Objectives: We tested the feasibility, acceptability, efficacy and safety of a culturally adapted version of SOLAR in two remote, cyclone-affected communities in the Pacific Island nation of Tuvalu. Method: This pilot adopted a quasi-experimental, control design involving 99 participants. SOLAR was administered to the treatment group (n = 49) by local, non-specialist facilitators (i.e. 'Coaches') in a massed, group format across 5 consecutive days. The control group (n = 50) had access to Usual Care (UC). We compared group differences (post-intervention vs. post-control) with psychological distress being the primary outcome. We also examined whether changes were maintained at 6-month follow-up.
Results: Large, statistically significant group differences in psychological distress were observed after controlling for baseline scores in favour of the SOLAR group. Mean group outcomes were consistently lower at 6-month follow-up than at baseline. SOLAR was found to be acceptable and safe, and programme feedback from participants and Coaches was overwhelmingly positive. Conclusions: Findings contribute to emerging evidence that SOLAR is a flexible, culturally adaptable and scalable intervention that can support individual recovery and adjustment in the aftermath of disaster. RCTs to strengthen evidence of SOLAR's efficacy are warranted.

Introduction

Disasters can be defined as large-scale, traumatic events of natural or human origin that threaten harm or death to a large group of people; disrupt social processes, resources, networks and services; and impact mental and physical health outcomes (Goldmann & Galea, 2014).

Disaster exposure is associated with a range of adverse mental health consequences. Many disaster survivors experience persistent psychological distress and functional impairment in the medium- to long-term aftermath of exposure that falls below threshold criteria for psychiatric diagnosis (Norris et al., 2002). These survivors are at an increased risk for developing more severe disorder (O’Donnell et al., 2016), and therefore are an important target for early intervention (Forbes, O’Donnell, & Bryant, 2017).

There are few evidence-based interventions developed to treat subclinical distress and impairment in populations that have been impacted by adversity. The two interventions in this space that have attracted the greatest interest are Problem Management Plus (PM+) and Skills for Psychological Recovery (SPR). Both interventions involve task sharing – an approach that utilizes less specialized workers to deliver interventions in order to improve human resource efficiencies and extend treatment coverage within a constrained budget (Padmanathan & De Silva, 2013). This is an important feature of interventions intended for post-disaster settings, in which large numbers of people are simultaneously affected, and for interventions targeting populations in low- and middle-income countries (LMICs), where mental health service coverage is limited (Patel et al., 2018).

PM+ (Dawson et al., 2015) is a low-intensity intervention that was developed by the World Health Organization to address common, low-severity mental health problems in LMICs. It is delivered over five sessions, individually or in groups, and includes psychoeducation, slow breathing to manage stress, problem-solving strategies, behavioural activation, and strategies to strengthen social supports. PM+ has been subject to five randomized controlled trials to date. Across settings, it has proven significantly (p < .05) more effective at reducing distress, reducing participant-identified difficulties, and improving psychosocial functioning, relative to usual or enhanced usual care (Bryant et al., 2017; de Graaff et al., 2020; Rahman et al., 2016). PM+ does not include any specific trauma-focused treatment component, which is a potential limitation concerning its use with disaster survivors, given trauma-focused psychological strategies that facilitate processing of traumatic experiences are known to be efficacious with these populations (Forbes, O’Donnell, & Bryant, 2016).

Skills for Psychological Recovery (SPR) (Berkowitz et al., 2010) was developed in the USA by the National Center for PTSD and National Child Traumatic Stress Network. It is designed to support individuals following disaster (Berkowitz et al., 2010). SPR is a modular, skill-building programme that supports individuals to (i) build problem-solving skills, (ii) engage in positive activities, (iii) cope with physical and emotional reactions to stress (iv) adopt helpful thinking, and (v) rebuild social connections (Berkowitz et al., 2010). SPR is ideally delivered over a minimum of three to five sessions (Berkowitz et al., 2010). Providers of SPR are expected to determine which modules to prioritize for delivery and to tailor these to the individual (Berkowitz et al., 2010). This is a complex task requiring clinical judgement, which has been cited as a limitation to its widespread uptake (Forbes et al., 2016). SPR lacks robust evidence regarding the programme’s efficacy.

Skills fOr Life Adjustment and Resilience (SOLAR) is an internationally developed, low-intensity intervention designed to target distress and impairment following disaster and trauma (O’Donnell et al., 2020). The SOLAR programme was designed to be (i) brief, to maximize access and reduce delivery costs; (ii) adaptable, to support its use across varied settings and populations; and (iii) delivered by trained lay persons, to increase reach of the intervention following disaster and trauma. Importantly, SOLAR includes a trauma processing component, which is missing from other brief interventions. An initial pilot study of SOLAR delivered by local community workers after bushfires in Australia found it to be feasible, acceptable and safe to deliver (O’Donnell et al., 2020). It also provided preliminary evidence that SOLAR was efficacious in reducing post-traumatic stress, anxiety and depressive symptoms, although the small sample size prevented firm conclusions from being drawn.

Tuvalu is a Polynesian, Small Island Developing State (The World Bank, 2016) comprising nine low-lying islands and atolls. It is among the nations most vulnerable to climate change on account of its ecological vulnerability, most particularly to sea-level rise and cyclone, and its limited capacity for adaptation (Barnett & Campbell, 2010; Nunn & Carson, 2015; United Nations Development Programme, 2013). In March 2015 Tuvalu was impacted by Tropical Cyclone Pam. The outer island of Nui was hardest hit, with 40% of the population being displaced as a result. In the capital Funafuti, Cyclone Pam came to signify the impending threat of climate change for many locals, precipitating considerable psychological distress (Gibson, 2018; Gibson, Barnett, Haslam, & Kaplan, 2020b). The present study aimed to establish the feasibility, acceptability, efficacy and safety of a culturally adapted version of SOLAR among residents of Tuvalu exposed to Tropical Cyclone Pam using between-group comparisons on psychosocial outcomes post-SOLAR versus post-control.

Methods

Ethics

Ethics approval for this study was obtained from the University of Melbourne Human Ethics Committee (ID: 1852041) and the trial was registered retrospectively with the Australian New Zealand Clinical Trials Registry (see http://www.ANZCTR.org.au/ACTRN12619000577112.aspx). Local project approval was obtained from the Tuvalu Departments of Community Affairs, Home Affairs, and Gender Affairs. Island approval was obtained from the Island Chiefs (Pule Fenua) and Resident EKT Pastors of Nui and Funafuti atolls.

Adaptation of the SOLAR programme

SOLAR is made up of six modules that target skills for healthy living (psychoeducation; strategies to improve sleep, improve nutrition and increase exercise), managing strong emotions (cognitive control and acceptance strategies), getting back into life following disaster (values-based behavioural activation), coming to terms with disaster (information processing and experiential learning), managing worry and rumination (psychoeducation, arousal management and self-soothing), and maintaining healthy relationships (strategies to expand social engagement and mitigate the impacts of stress on relationships) (O’Donnell et al., 2020). SOLAR was developed by an international group of disaster, trauma, and mental health experts (O’Donnell et al., 2020). It is delivered over five sessions by trained, non-specialist or lay ‘Coaches’. The programme includes a Coach Manual and a Participant Workbook containing information, examples and practice exercises. These materials were adapted for this study to maximize cultural fit, while maintaining fidelity to the programme’s core elements. The adaptation process was informed by the British Medical Research Council Framework for the development and evaluation of complex interventions (Craig et al., 2008), and the cultural adaptation framework developed by Bernal and Sáez-Santiago (2006) to increase ecological validity. Consideration was also given to research expounding the outcomes of adaptations undertaken to deliver psychological interventions across cultures previously (e.g. Chowdhary et al., 2013).

Steps taken to gain the requisite cultural knowledge needed to inform adaptations included (i) consultation with Tuvaluan stakeholders and cultural informants; (ii) a review of programme materials and trial protocols by the first-named author, a psychologist with experience working in Tuvalu and cross-culturally; (iii) a focus group attended by eight Tuvaluan citizens; and (iv) feedback from Coaches during SOLAR Coach training, which resulted in iterative changes to programme materials in the field. Resulting adaptations made to the programme are shown in Supplementary Table 1, and are categorized by the eight elements of adaptation outlined by Bernal and Sáez-Santiago (2006).

Study design

Feasibility was conceptualized as the ability of lay Coaches, without prior mental health training or experience, to deliver SOLAR within their own communities after a short (6-day) training period with high levels of fidelity. Acceptability concerned participants’ responses to the programme, reflected in participants’ attendance rates and their qualitative feedback regarding the usefulness of the programme and uptake of skills taught. Efficacy was examined using a quasi-experimental, Usual Care (UC) control design to compare groups at post-intervention versus post-control. The primary outcome was psychological distress, with PSTD symptoms and impairment as secondary outcomes. Symptom change over time was also examined as a secondary measure of efficacy. Safety referred to the documentation of Serious Adverse Events reported by a participant, or observed to be affecting a participant, during the pilot period.

Participants from two islands were involved in the study. Participants on one island (Nui) were the active condition. They completed an assessment (pretreatment), received the SOLAR intervention, then completed another assessment (post-treatment). Participants on the second island (Funafuti) were the control condition. They underwent an assessment (pretreatment), could access usual care, then completed another assessment (post-treatment). Usual Care comprised informal familial, community and church-based supports. They then received the SOLAR intervention. All participants were followed up 6 months after receiving the intervention.

During the intervention period, participants were administered SOLAR in a group format (up to 10 participants per group) across 5 consecutive days. Two Coaches facilitated each group and 10 groups were run in total. The intervention period occurred in October–November 2018, approximately 3 years and 7 months after Tropical Cyclone Pam.

Baseline differences between the two groups were carefully examined and statistically controlled for during between-group analyses. Changes in outcomes pre-to-post-intervention and from post-intervention to 6-month follow-up were examined for both groups. The study design is illustrated in Figure 1.

Figure 1.

Study design

Measures

Feasibility

Coach training outcomes were measured using a purposively designed written assessment that comprised (i) a multiple choice quiz of programme content; (ii) clinical vignettes requiring Coaches to apply their knowledge to hypothetical scenarios; (iii) Coach confidence ratings. This assessment was conducted pre- and post-training. Research assistants also completed a checklist of facilitator skills while observing Coach role-plays during training. Fidelity monitoring was completed by local research assistants who observed 50% of all sessions and completed checklists outlining the key components of each session. Sessions were selected for fidelity monitoring to ensure (a) all Coach pairings were observed at least twice and (b) sessions across each day of the programme (1–5) were observed at least twice.

Acceptability

Acceptability was determined by the average number of sessions attended by participants, and participants’ programme feedback, surmised using descriptive statistics. The latter was collected during post-intervention and follow-up assessments. These included open-ended questions to gauge participants’ experiences of the SOLAR intervention and frequency of skill uptake subsequent to the programme, as well as participant ratings (theoretical range: 1–10) of the usefulness of SOLAR.

Efficacy

Provisional evidence of efficacy was determined by a) between-group comparisons on psychosocial outcomes post-SOLAR versus post-control, as measured by the HSCL-25 Tuvalu, PCL-5, TIC and PSYCHLOPS, outlined below, and b) symptom change over time according to these same measures.

The Hopkins symptom checklist-25 (HSCL-25) Tuvalu (Gibson, 2018), a culturally adapted version of the HSCL-25 (Mollica, McDonald, Massagli, & Silove, 2004; Parloff, Kelman, & Frank, 1954), was used to measure severity of psychological distress (anxiety and depression) during the previous week. The HSCL-25 Tuvalu has shown adequate internal consistency (Cronbach’s alpha (α) = .86) (Gibson, Barnett, Haslam, & Kaplan, 2020a). The scale comprises 25 items with 4-point Likert response options, where 1 = ‘not at all’ and 4 = ‘extremely’. Scores were summed and divided by 25 to give a total distress score, with scores >1.75 commonly interpreted as indicative of symptomatic levels of distress (Mollica et al., 2004). Bentler’s (2005) dimension-free greatest lower-bound reliability was .993 for the HSCL-25 Tuvalu pretest scores.

The PTSD Checklist for DSM-5 (PCL-5) (Weathers, Litz, Herman, Huska, & Keane, 1993) was used to measure the severity of post-traumatic stress symptoms during the previous week, following an adaptation process entailing translation, blind back-translation, consensus reaching and piloting, conducted by the research team. The PCL-5 has been widely used across cultures and routinely demonstrated high levels of internal consistency and convergent, discriminant and structural validity (Blevins, Weathers, Davis, Witte, & Domino, 2015). The scale comprises 20 items with 5-point Likert response options, where 0 = ‘not at all’ and 4 = ‘extremely’. Scores were summed to yield a total PTSD symptom score, with scores >33 being a widely used cut-off point for determining clinical significance (Weathers et al., 2013). Bentler’s (2005) dimension-free greatest lower-bound reliability was .998 for PCL-5 pretest scores.

The Tuvalu impairment checklist (TIC) (Gibson, 2018) was used to measure the impact of psychological distress on everyday functioning across five life domains typical of Tuvaluan adults. This checklist was developed following a survey of common domains of functioning common in contemporary Tuvalu, and has been used once previously (Gibson, 2018). The domains include (i) caring for other people in the household, (ii) contributing to household tasks, (iii) attending or assisting with community, church, or social events, (iv) engaging in leisure activities, and (v) study/work (where applicable). It comprises five items with 4-point Likert response options, where the level of difficulty performing tasks in each domain is scored between 1 = ‘not at all’, and 4 = ‘extremely’. Scores were summed and divided by the number of applicable items to yield a total impairment score.

The psychological outcomes profiles (PSYCHLOPS) (Ashworth et al., 2004) was used to measure participant-identified difficulties. On this measure, participants identify up to two key problems they presently face, then rate on a 5-point Likert scale how ‘affected’ they have been by each problem, how ‘difficult’ it has been to complete activities impacted by each problem, and how they have felt overall during the past week. Scores on these four items are summed to yield a total ‘self-identified difficulties’ score. The PSYCHLOPS was used following an adaptation process entailing translation, blind back-translation, consensus reaching and piloting, undertaken by the research team. It has been shown to be sensitive to clinical change after therapy and to have satisfactory levels of internal reliability and convergent, concurrent and construct validity (Ashworth et al., 2005).

Safety

An adverse event was defined as any undesirable experience occurring to a participant during the study, irrespective of its perceived relationship to the research trial, which was reported by a participant or observed by a coach or member of the research team. Protocol stipulated that these events be recorded on a specific database and escalated to a local advisory board.

Assessment procedures

Tuvaluan Research Assistants carried out screening and assessments during face-to-face interviews conducted in Tuvaluan. Eligibility screening involved administration of an interview schedule comprising the HSCL-25 (Tuvalu), PCL-5, and TIC. Individuals ≥18 years of age were eligible for inclusion if they reported mild-to-moderate symptoms of anxiety, depression, and/or posttraumatic stress symptoms, together with resulting functional impairment in one or more areas of daily life. Specifically, individuals who scored (i) between 1.25 and 1.75 (inclusive) on the HSCL-25 (Tuvalu), or (ii) between 15 and 33 (inclusive) on the PCL-5, and (iii) >1 on the TIC, indicative of at least mild impairment in one domain, were eligible for inclusion. Individuals who scored above cut-off scores on the HSCL-25 or PCL-5 routinely used to determine clinical significance were eligible for inclusion pending the outcome of a risk assessment. Structured risks assessments were conducted by trained research assistants to determine participants’ risk of harm to self or others. High-risk participants were deemed ineligible, except where additional support and monitoring to ensure safety were in place. This decision was made in recognition of the lack of alternative mental health interventions available to residents in this setting.

Screening scores served as baseline (Time 1, T1) scores for eligible participants. At T1, participants also completed a demographic questionnaire, a questionnaire concerning their experience during Cyclone Pam, and a Tuvaluan language version of the Psychological Outcomes Profiles (PSYCHLOPS).

At post-intervention and 6-month follow-up assessments, the same outcome measures were administered, together with a programme experience survey. Participants were assessed at post-intervention if they had attended ≥1 SOLAR session.

Coach recruitment, training and supervision

Twelve Coaches were recruited via nomination by project partner organizations. Coaches were required to be ≥18 years of age, fluent in Tuvaluan and English, and have experience working or volunteering in a disaster response, community support or leadership role in Tuvalu. Although no prior mental health training or experience was required, six Coaches had recently commenced a counselling degree.

Coach training was delivered in English across 6 days with an interpreter present throughout. The training programme was 4 days longer than that provided in the initial, Australian SOLAR pilot. This was deemed necessary given the novelty of the programme in this setting, the need to discuss unfamiliar concepts and their cultural equivalence, and the need for intermittent interpreting. All Coaches were required to demonstrate competence prior to delivering the programme.

In the field, group supervision was provided to Coaches daily for the 5 days spent delivering the programme. Coaches also attended a focus group at the end of the programme to share their experiences and feedback.

Participant recruitment

Ninety-nine community members across Nui (n = 49) and Funafuti (n = 50) were invited to participate in SOLAR following self-referral in response to trial advertisements and subsequent eligibility screening. Participants whose homes were severely damaged during Cyclone Pam were also directly approached by Research Assistants and invited to participate. Figure 2 provides details of participant recruitment and retention.

Figure 2.

Flow diagram illustrating participant recruitment and retention

Data analyses

Efficacy was determined by comparing group scores (post-SOLAR versus post-control) on each outcome using an intent-to-treat process. Mplus Version 8.4 (Muthén & Muthén, 1998–2019) was used to estimate a series of one-way Analysis of Covariance (ANCOVA) models for each outcome variable using methods described by Green and Thompson (2012). A series of modifications to the ANCOVA approach were conducted to accommodate the study’s use of a quasi-experimental design, in which non-random assignment led to violations to ANCOVA assumptions. An example of the Mplus code used for these ANCOVAs is given in the Supplementary materials.

Each ANCOVA compared the total scores on the outcome variable between SOLAR and the control group at post-intervention/control, while controlling for baseline scores on the relevant outcome. For each ANCOVA, the grand-mean centred covariate (i.e. subtracting the grand mean of the baseline outcome variable from all scores on that measure) was included as a predictor in the model, together with a constant (i.e. intercept). Mplus Version 8.4 was used for the ANCOVA analyses. Missing data was handled using full information maximum likelihood via MLR estimation. We used an intent-to-treat analysis to ensure that all participants screened into the pilot were included in the analysis and analysed according to their originally assigned groups. This resulted in all 99 cases used within each of the ANCOVAs.

In each ANCOVA, the slope coefficients for the covariate were constrained to be equal across groups, consistent with the assumption of homogeneity of slopes, while error variances were constrained to be equal across groups, consistent with the homogeneity of variance assumption. The MLR x2 estimates for each of the three models were compared against a model in which these constraints were freed using the chi-square difference test, which was recalled using Satorra and Bentler (2010) scaled chi-square to adjust for multivariate normality. Where evidence indicated these assumptions were unreasonable (i.e. the scaled chi-square difference test was significant), they were accommodated in Mplus by freeing the covariate slopes and error variances of the groups to differ (Fan & Hancock, 2012; Green & Thompson, 2012). When freeing these equality constraints, the model remains just-identified, which can be used to directly assess the ANCOVA hypothesis by constraining the intercepts to be equal across groups.

When freeing the regression slopes for the covariate to differ, the groups are no longer equated on the covariate, which poses interpretational challenges for ANCOVA. This problem was addressed by using Potthoff’s (1964) modification to the Johnson–Neyman procedure (Aiken & West, 1991; Johnson, 2016). The procedure allows for the identification of cases that are equated on the covariate, while allowing for covariate slopes to differ between the two groups. These cases are selected through identification of the point of intersection (crossover point) of the regression lines for the two groups, and what are termed simultaneous regions of significance (SROS). The SROS means that with 95% confidence (for α = 05), ‘we can state that the two groups […] are different simultaneously for all points contained in it’ (Potthoff, 1964, p. 244). For all group analyses using this procedure, reported in results below, at least 80% of the total sample for each outcome were equated on the covariate, meaning the findings reported reflect the majority of participants.

Having used the SROS to identify cases equated on the covariate, the magnitude of the difference between the two groups for these ‘covariate equated cases’ is reported as a mean difference with 95% confidence interval, and was indexed using Glass’ delta or Cohen’s d. The baseline standard deviation for all participants recruited (n = 99) was used as the estimate of the population standard deviation in the denominator of Cohen’s d, and the baseline standard deviation for all 49 scores from the control group was used as the estimate of population standard deviation in the denominator of Glass’ delta.

To understand the trajectory and magnitude of symptom change over time, paired-sample t-tests with a Holm’s sequential Bonferroni correction was used to compare within-group changes in outcome scores (HSCL-25 (Tuvalu), PCL-5, TIC, PSYCHLOPS). Repeated measures effect size estimates (dRM) with 95% Confidence Intervals (CIs) were produced to quantify the magnitude of participant within-group change in each outcome, from (i) pre to post-intervention, and (ii) post-intervention to 6-month follow-up. Mplus Version 8.4 (Muthén, & Muthén, 1998–2019) was used for the analysis.

Missing data was handled using full information maximum likelihood via MLR estimation under Rubin’s (1987) missing at random assumption. An intent-to-treat analysis was again used to ensure that all patients screened into the pilot were included in the analysis and analysed according to the originally assigned groups. This resulted in all 99 cases being used within each of the paired-sample t-tests. The Holm-Bonferroni correction was calculated in excel (Gaetano, 2018) to maintain the family-wise Type I error rate for each of the paired sample t-tests.

Results

Baseline sample characteristics

Aggregate demographic and cyclone-related data are shown in Table 1, with data segmented by site. Participants at the two sites varied demographically, reflecting characteristic demographic differences between outer island and main island residents. The two samples also differed in their experience of fear during Cyclone Pam and extent of personal damage arising from it. The statistical significance of baseline differences between the two groups was determined using independent sample t-tests, or Pearson’s chi-square test for independent samples where the relevant variable was binary.

Table 1.

Sample demographic data

Demographic characteristics	Funafuti	Nui
Age**	M = 34.12; Range: 18–71	M = 50.02; Range: 20–74
Female*	N = 38 (76%)	N = 28 (57%)
Currently married**	N = 28 (56%)	N = 44 (90%)
Number of children*	M = 2.60; Range: 0–6	M = 3.71; Range: 0–11
Number of people per household**	M = 11.02; Range: 3–31	M = 6.31; Range: 1–19
EKT Church members*	N = 36 (43%)	N = 49 (100%)
Years of formal schooling**	M = 10.82; Range: 7–13	M = 7.92; Range: 0–13
Vocational or tertiary training**	N = 21 (42%)	N = 2 (4%)
Receiving remittances*	N = 16 (32%)	N = 26 (53%)
Level of subsistence activity**	M = 2.04	M = 2.59
Level of difficulty meeting basic household needs**	M = 2.22; Range: 1–4; 1 = not difficult, 4 = extremely difficult	M = 2.78; Range: 1–4; 1 = not difficult, 4 = extremely difficult
Cyclone experience
Level of fear experienced during cyclone**	M = 2.86; Range: 1–4; 1 = not at all, 4 = extremely	M = 3.82; Range: 1–4; 1 = not at all, 4 = extremely
Damage to personal property or land**	N = 26 (38%)	N = 49 (100%)
Level of increase in day-to-day difficulties following cyclone**	M = 2.46; Range: 1–4; 1 = no more difficult than usual, 4 = extremely difficult	M = 3.16; Range: 1–4; 1 = no more difficult than usual, 4 = extremely difficult

*p < .05; **p < .01.

Acceptability

Attendance rates

Eighty per cent of participants attended one or more SOLAR sessions. Among attendees (n = 79), there was a high degree of session attendance, with the majority of participants attending an average of 4 sessions (SD = 1.25).

Post-intervention participant feedback

At post-intervention, attendees’ ratings of the usefulness of the SOLAR programme ranged from 7 to 10, with a mean of 9.03 (SD = 1.04). All attendees reported that the programme was personally useful and/or important, and all reported that they would recommend the programme to others.

Six-month follow-up feedback

Sixty-three attendees completed a follow-up assessment 6-months after their post-intervention assessment. Attendees’ ratings of the usefulness of the skills taught in the programme during the preceding 6 months ranged from 5 to 10, with a mean of 8.87 (SD = 0.96).

Open-ended survey data was collected from 52 attendees at follow-up, of whom 51 reported continuing to regularly practice at least one of the skills taught in the programme. Of these 52 attendees, 37 reported having shared the skills with other community or family members, 50 reported a desire to attend a follow-up/booster session if it were available, and 47 reported a desire to repeat the programme if it were offered again in the future.

Feasibility

Effectiveness of coach training programme

Pre-to-post training analyses indicated large, statistically significant improvements in Coaches’ knowledge of SOLAR programme content, t(10) = 4.36, p = .001, effect size (dRM) = 1.76, 95% CI: [0.41, 3.11]; their ability to apply that knowledge in response to example vignettes, t(10) = 19.10, p < .001, effect size (dRM) = 6.83, 95% CI: [3.15, 10.51]; and their confidence delivering the programme, t(9) = 2.98, p = .015, effect size (dRM) = 1.26, 95% CI: [0.07, 2.45]. Figures S1, S2, and S3 in Supplementary Materials show changes pre-to-post training.

Fidelity ratings

Research assistants attended 50% of sessions as observers to rate Coaches’ fidelity to the SOLAR protocol, according to session-specific checklists. In 21 out of 25 sessions (84%), all programme components were delivered. In four cases, part of one component was missed, or else delivered in insufficient depth or without complementary activities to cement learning. In three cases, this was remedied in a subsequent session.

Efficacy

Between-group analyses

Support for the efficacy of SOLAR was found for all four outcomes, with large, statistically significant between-group differences observed after controlling for baseline scores in favour of the SOLAR group in each analysis.

As described above, a series of one-way ANCOVAs with varying model constraints were run for each analysis to assess population constraints. These results are reported in Supplementary Table 2. The results reported below concern the model of best fit.

Distress

ANCOVA of HSCL-25 (Tuvalu) scores, using Potthoff’s (1964) modification to the Johnson–Neyman procedure (Aiken & West, 1991; Johnson, 2016), revealed a large, statistically significant difference between post-intervention/control means for the two groups, after controlling for baseline scores. This mean difference was 0.520 [95% CI: 0.646, 0.395], with the intervention group adjusted mean statistically significantly lower than that of the control group (see Table 3). Glass’ delta for this difference was 1.106 [95% CI: 0.839, 1.373].

Table 3.

Group comparisons for scores falling with the SROS regions of the Johnson–Neyman procedure

Model	AdjM Intervention group	AdjM Control group	Mean difference with 95% CI	Effect size with 95% CI
HSCL-25	1.291	1.812	0.520 [0.646, 0.395]	1.106 [0.839, 1.373]a
PCL-5	12.357	34.075	21.718 [18.484, 24.953]	1.575 [1.341, 1.810]a
Impairment	1.315	2.269	0.955 [0.810, 1.099]	1.316 [1.117, 1.516]a
PSYCHLOPS	4.687	12.84	8.153 [7.088, 9.218]	2.403 [1.987, 2.819]b

Note. HSCL-25 = Hopkins Symptom Checklist; PCL-5 = Post-traumatic stress disorder checklist for DSM-5; Impairment = Tuvalu Impairment Checklist; SROS = simultaneous regions of significance; AdjM = adjusted means on the post-test measure for participants who are at the mean on the pretest measure; a Glass’ delta; bCohen’s d.

These findings refer to the majority of participants whose HSCL-25 Tuvalu pretest scores fell above the upper SROS (i.e. 90.90% of the sample), of whom 46 were in the intervention group. No significant differences in post-intervention/control scores were identified for the nine participants whose scores fell outside the SROS.

Table 2 shows the results of the Johnson–Neyman procedure for each analysis. Table 3 shows group comparisons for those scores falling within the SROS regions of the Johnson–Neyman procedure for each analysis.

Table 2.

Results of Johnson–Neyman procedure

Model	Crossover		SROS				n Int group	n Control group
	Cent	Uncent	CentLower	Cent Upper	Uncent Lower	Uncent Upper
HSCL-25	−0.902	1.052	−2.356	−0.447	0.616	1.43	46	44
PCL-5	−28.259	8.629	−55.874	−17.02	4.85	18.084	44	40
Impairment	−0.458	0.112	−0.837	−0.204	−0.95	0.85	44	50

Note. HSCL-25 = Hopkins Symptom Checklist; PCL-5 = Post-traumatic stress disorder checklist for DSM-5; Impairment = Tuvalu Impairment Checklist; SROS = simultaneous regions of significance; Cent = centred; Uncent = Uncentred; n = sample size of cases falling within the lower and upper SROS regions.

Post-traumatic stress symptoms

ACOVA of PCL-5 scores, again using Potthoff’s (1964) modification to the Johnson–Neyman procedure, likewise showed a large, statistically significant difference between groups (Glass’ delta = 1.575 [1.341, 1.810]), with significantly greater declines in PTSD symptoms observed in the intervention group (see Table 3).

These findings refer to the majority of participants (n = 84) whose baseline scores fell above the upper SROS value (see Table 2). No difference was found between groups for the remaining participants (n = 14).

Impairment

ACOVA of TIC scores, using Potthoff’s (1964) modification to the Johnson–Neyman procedure, showed there was a large, statistically significant difference between post-intervention/control means for the two groups after controlling for baseline scores, with greater reductions in impairment seen in the intervention group (Glass’ delta = 1.316 [1.117, 1.516]) (see Table 3). In this analysis, only five scores fell below the upper SROS value.

Self-identified problems

ACOVA of PSYCHLOPS scores did not require Potthoff’s (1964) modification to the Johnson–Neyman procedure, as the two ANCOVA assumptions for the homogeneity of regression slopes and error variances were satisfied (i.e. Models 4a and 4b returned an exact fit to the data, and did not significantly differ – see Supplementary Table 2). For this analysis, Model 4 c was interpreted, in which intercepts were constrained to equality.

The chi-square test for Model 4 c was statistically significant, x2 (N = 99, 3) = 110.636, p < .001 and returned a statistically significant chi-square different test result, Δ x2 (N = 99, 1) = 151.147, p < .001 when compared against Model 4b. This indicates that the intercepts were statistically, significantly different between the two groups, with significantly greater declines in PSYCHLOPS scores seen in the intervention group (Cohen’s d = 2.403 [95% CI: 1.987, 2.819]).

To aid interpretation of changes in PSYCHLOPS scores, the self-identified difficulties reported by participants were categorized into themes (see Supplementary Figures S4 and S5 for the number of participants that reported each theme). The most commonly reported difficulties among Nui residents were, in order: (i) ongoing financial impacts of Cyclone Pam, (ii) lack of financial resources, and (iii) ongoing psychological impacts of Cyclone Pam. The most commonly reported difficulties among Funafuti residents were as follows: (i) lack of financial resources, (ii) familial relationships, and (iii) burden of care for extended family and kin. Improvements on the PSYCHLOPS indicate that SOLAR positively impacted (i) how ‘affected’ by these problems participants reported having been in the previous week, (ii) how ‘hard’ it was to do things because of these problems during the previous week, and (iii) how participants ‘felt in themselves’ during the previous week.

Symptom change over time

Descriptive statistics for both the Funafuti and Nui samples indicated improvements across all outcome variables pre-to-post intervention, which were retained to varying degrees at 6-month follow-up. Results are shown in Table 4.

Table 4.

Mean aggregate scores at each assessment point and paired sample t-tests comparing changes over time by sample

	Pre-control	Pre-intervention		Post-intervention	6-month follow-up	Pre-to-post change							Post-to-follow-up change							Pre-to-follow-up change
	M(SE)	M(SE)		M(SE)	M(SE)			95% CI								95% CI						95% CI
						dRM			Lower	Upper	p		dRM		Lower		Upper	p		dRM			Lower	Upper	p
Funafuti
HSCL-25 Tuvalu	1.89 (0.07)	1.75 (0.07)		1.29 (0.03)	1.51 (0.07)	−2.66			−3.60	−1.72	<0.001		1.88		0.98		2.79	0.010		−1.12			−1.93	−0.31	0.020
PCL-5	33.88 (1.95)	31.12 (1.88)		14.03 (1.25)	21.97 (2.95)	−0.13			−0.17	−0.10	<0.001		0.04		0.02		0.06	0.020		−0.04			−0.07	−0.01	0.050
TIC	2.32 (0.10)	2.27 (0.10)		1.37 (0.05)	1.75 (0.10)	−2.51			−3.20	−1.82	<0.001		1.56		0.84		2.29	<0.001		−1.26			−1.89	−0.63	0.001
PSYCHLOPS	15.04 (0.52)	12.84 (0.49)		5.26 (0.45)	5.32 (0.73)	−0.77			−1.02	−0.53	<0.001		−0.77		−1.02		−0.53	<0.001		−0.52			−0.80	−0.26	0.001
Nui
HSCL-25 Tuvalu		2.28 (0.12)		1.29 (0.03)	1.17 (0.05)	−2.14			−2.51	−1.77	<0.001		−1.68		−2.59		−0.77	0.010		−2.39			−2.84	−1.94	<0.001
PCL-5		43.74 (2.72)		12.91 (1.14)	13.60 (0.68)	−0.12			−0.15	−0.09	<0.001		0.02		−0.03		0.07	0.590		−0.12			−0.15	−0.09	<0.001
TIC		2.40 (0.10)		1.314 (0.03)	1.58 (0.04)	−3.10			−4.03	−2.17	<0.001		6.42		4.12		8.72	<0.001		−2.32			−3.10	−1.54	<0.001
PSYCHLOPS		13.14 (0.45)		4.68 (0.44)	5.81 (0.44)	−0.92			−1.25	−0.59	<0.001		0.15		0.02		0.28	0.110		−0.90			−1.32	−0.49	<0.001

For both samples, dRM estimates for pre-to-post intervention change indicated very large improvements in distress and impairment, and a small improvement in PTSD-symptoms. dRM estimates for self-identified difficulties indicated large (Nui) and moderate (Funafuti) improvements respectively. The 95% CIs for each dRM point estimate from pre-to-post intervention excluded zero, and was therefore consistent with statistically significant effects at the p < .05 criterion level.

While both samples improved from pre-to-post intervention, there was an increase in symptoms at follow-up for one or both samples on each outcome, although follow-up scores remained significantly lower than pre-intervention scores (see Table 4).

Safety

Serious adverse events

No serious adverse events were identified during the trial.

Discussion

Summary of findings

This study found support for the acceptability, feasibility, efficacy and safety of a culturally adapted version of SOLAR provided by lay Coaches to cyclone-affected residents in Tuvalu. High programme attendance rates and consistently positive participant feedback attested to the programme’s acceptability, while coach training outcomes and fidelity records showed it was feasible for locals recruited from within the targeted, disaster-affected communities to facilitate SOLAR after a short period of training. SOLAR proved safe to deliver, including for individuals with very high levels of distress and/or PTSD symptoms at baseline, and was associated with large, statistically significant improvements in all outcomes pre- to post-intervention. Some benefit was retained at 6-month follow-up across all outcomes, although symptom scores at 6-month follow-up were consistently higher than at post-intervention, with one exception (distress scores for the Nui sample showed a further, significant decline between post-intervention and follow-up). Results provide provisional evidence of efficacy, with improvements being consistently and significantly greater for the intervention group relative to the UC control group.

Synthesis with existing evidence

It is well established that many disaster survivors experience persistent psychological distress and functional impairment in the aftermath of disaster that falls below threshold criteria for psychiatric diagnosis (Norris et al., 2002). Nonetheless, there are presently few post-disaster interventions tailored to the needs of these survivors.

The feasibility, acceptability and efficacy of SOLAR in Tuvalu, as demonstrated in this pilot, compares favourably with the feasibility, acceptibility, accessibility and efficacy of PM+ in settings where it has been empirically investigated. With regard to acceptability, PM+ is generally met with positive feedback and sessions are well attended, although concerns about possible stigma being associated with programme participation has been raised in certain settings (de Graaff et al., 2020; Sangraula et al., 2020). In this trial, SOLAR was met with consistently positive feedback and session attendance rates were comparable (Rahman et al., 2016; Sangraula et al., 2020). Encouragingly, 71% of the participants interviewed at follow-up reported that they had shared the skills they learned in SOLAR with friends or family, indicating that participants were unconcerned about their programme participation being public knowledge. It may be that the focus on resilience building within the SOLAR programme helped alleviate possible stigma associated with doing the programme.

A comparison of facilitator fidelity to programme protocols in the present trial, compared to trials of PM+, is also encouraging, with the per cent of sessions adhering to protocol being comparable, if not superior, in the present study (de Graaff et al., 2020; Rahman et al., 2016; Sangraula et al., 2020). This may be attributable to the close supervision arrangements in place in this trial. It is noteworthy that whereas PM+ is often delivered using a train the trainer model, whereby a ‘master trainer’ trains local mental health professionals, who subsequently train lay facilitators (e.g. Rahman et al., (2016), in the present study, the international SOLAR trainer provided training directly to Tuvaluan coaches. Train-the-trainer models, if effective, could expand the reach of SOLAR, and their impact on feasibility outcomes should be explored and evaluated in future trials.

With regard to efficacy, a comparison of the effect sizes of post-intervention between-group differences identified in the present study and those reported in trials of PM+ using that utilized the same measures (i.e. the HSCL-25, PCL-5, PSYCHLOPS), suggests that SOLAR performs comparatively well when it comes to immediate, post-intervention reductions in distress, PTSD symptoms, and participants’ self-identified difficulties (de Graaff et al., 2020). This is suggestive, not conclusive, since various differences between study protocols prohibit direct comparisons. It does, however, encourage continued investigation into the efficacy of SOLAR across contexts and using more rigorous study designs. In particular, the relative efficacy of SOLAR for reducing PTSD-symptoms, and the mechanism of action behind this, is an important avenue for future research, given the inclusion of a module targeted processing of traumatic memory is a feature that distinguishes SOLAR from PM+.

In summary, SOLAR appears to be a promising intervention, suited to task sharing, and may help address the current gap in psychosocial interventions targeting subclinical symptoms in post-disaster settings.

Limitations

Although this study design included a control condition, it is limited by its lack of a randomization procedure. To address this limitation, the analyses we used to examine group differences statistically controlled for baseline differences on each outcome measure. However, randomized trials are needed to yield more robust evidence of efficacy. Future trials involving an active control, rather than UC, would also provide insight into the particular benefits of SOLAR relative to alternative interventions. This is the first time the outcomes of a mental health intervention in Tuvalu have been published, so we cannot yet comment on the outcomes we observed in relation to other possible modes of intervention in this setting.

The primary outcome measure in this study has been validated in the Tuvaluan population; however, we recognize that the translated versions of the PCL-5 and PSYCHLOPS have not been validated. While substantial effort was made to adapt and translate each outcome measure to improve its cultural validity, we cannot be certain how these processes altered the construct, content or criterion validity of the measures.

Finally, this study did not systematically examine the sustainability of SOLAR, which is something warranting in-depth investigation in future trials. During Coach focus groups, all Coaches believed they would use concepts and activities from the SOLAR programme in their day-to-day work, and one Coach intended to rollout the SOLAR programme with womens’ groups in her community. Further, 73% of the participants who completed 6-month follow up assessments reported having shared some of the information they had learned in the programme with non-participants. These are promising signs, but future trials would benefit from a better resourced implementation plan extending beyond the programme’s initial roll-out that could provide insights into what benefits the programme has for the broader community in the longer term.

Implications and conclusions

The findings of the present study contribute to emerging evidence that the Skills fOr Life Adjustment and Resilience (SOLAR) programme constitutes an acceptable, feasible, safe and effective intervention for disaster survivors experiencing distress and impairment years after a disaster event.

We attribute the programme’s acceptability both to the programme’s content, as well as the careful cultural adaptation process undergone. Its acceptability in this pilot, where it was delivered in a massed group format in a Small Island Developing State, together with previous demonstration of its acceptability when delivered via weekly, individual sessions in Australia (O’Donnell et al., 2020), speaks to SOLAR’s versatility, and provides support for its potential uptake across culturally and economically diverse settings.

The feasibility of SOLAR in this setting provides confidence in its potential roll out in other LMICs settings where mental health literacy is limited. A more nuanced examination of fidelity, and investigation of the extent to which Coach-related variables, individual training outcomes, and supervision arrangements influenced programme outcomes, would be useful lines of enquiry to further understand feasibility and constraints in subsequent trials.

It is promising that SOLAR was safe to deliver to individuals reporting very high levels of distress or post-trauma symptoms at baseline. Notably, of the 53 participants who scored in the potentially clinically significant range on either the HSCL-25 Tuvalu or PCL-5 at baseline, no participant scored in that range at post-intervention. While individuals with severe symptoms are not the primary target of SOLAR, which is intended for subclinical populations, it is reassuring that highly distressed individuals need not be turned away in situations where SOLAR is the only intervention available.

The preliminary evidence of efficacy observed in this pilot is encouraging and warrants its continued research. To improve retention of benefit, future trials may benefit from a booster session or practice reminders following completion of the intervention. The present findings suggest this would be well received, with 98% of participants who completed 6-month follow-up assessments stating they would attend such a session if it were available.

We conclude that SOLAR constitutes a promising step towards redressing the significant gap in mental healthcare typical in post-disaster settings; a gap that is likely to grow given the increasing threat of ecological disaster globally.

Click here for additional data file.