Satisfaction and Mental Health Outcomes Associated with a Large Regional Helpline.

We surveyed users of a behavioral health helpline serving New York City and surroundings, to assess their helpline experiences, changes in psychological distress after contacting the helpline, and factors associated with differences in these measures. We surveyed users twice: roughly 2 weeks following their helpline contact, from 4/2019 to 9/2019 (N = 1097 respondents) and again 6 months following contact, from 10/2019 to 3/2020 (N = 732 respondents). Eighty-nine percent of respondents reported that contacting the helpline helped them deal a little or a lot more effectively with their problems. Rates of psychological distress decreased from 41.3% 2 weeks following helpline contact to 29.0% 6 months after (P < 0.05). Improvements in psychological distress were found across a range of demographic characteristics and were greatest for repeat users. Users reported broadly positive experiences with the helpline and improved psychological distress 6 months later. Behavioral health helplines can offer beneficial services to diverse populations, complementing the formal behavioral healthcare system.

Introduction

According to the 2018 National Survey on Drug Use and Health, approximately 19% of U.S. adults (47.6 million) reported experiencing mental illness in the past year (Substance Abuse & Mental Health Services Administration, 2019). Fewer than half of these (20.6 million, or 43%) indicated receiving mental health services during that timeframe (Substance Abuse & Mental Health Services Administration, 2019). Likewise, roughly 8% of U.S. adolescents and adults reported a substance use disorder or needing substance use treatment (21.2 million), but only 3.7 million (17%) received substance use treatment in the past year (Substance Abuse & Mental Health Services Administration, 2019). Additionally, many individuals experience sub-threshold but clinically significant mental health distress that could benefit from intervention (Prochaska et al., 2012). Individuals in need may not engage with formal behavioral health services for many reasons, including stigma, expense, inability to find a provider, or linguistic or cultural barriers (Clement et al., 2015; Mechanic, 2002; Mojtabai, 2005; Mojtabai et al., 2011).

Behavioral health helplines (also called hotlines, lifelines, warmlines, or crisis lines) assist individuals who may not use other behavioral health services or who need immediate support (Middleton et al., 2016). Thousands of helplines exist worldwide, many serving specific geographic areas (e.g., countries, states, or metropolitan areas). Helplines provide a variety of services, including short-term counseling, suicide prevention or other crisis intervention, peer support, information and referral, and follow-up services (Gould et al., 2012; Ramchand et al., 2017; Shor & Birnbaum, 2012), and often serve specific populations, such as veterans or adolescents (Dalgin et al., 2011; Knox et al., 2012; Price et al., 2016). Helplines may also be a starting point for receiving referrals to behavioral health providers and accessing services (Coveney et al., 2012; Gould et al., 2007; Kalafat et al., 2007; Middleton et al., 2016). In October 2020, the US federal government passed The National Suicide Hotline Designation Act, which created a national suicide prevention and mental health crisis hotline system that can be accessed nationally using a three-digit number (988), instead of the 10-digit number used previously (U.S. Government Publishing Office, 2020).

Studies of helplines have found high satisfaction (Biggs et al., 2015; Britton et al., 2013; Coveney et al., 2012) and suggest helpline use is associated with reduced suicidality (Gould et al., 2007, 2018), distress (Ramchand et al., 2017), and hopelessness (Gould et al., 2007; Kalafat et al., 2007). However, few studies have assessed differences in helpline experiences across user demographic characteristics (Gould et al., 2018), and most studies focused on users’ short-term experiences and psychological states, such as during the contact (Biggs et al., 2015) or within the subsequent 2–3 weeks (Gould et al., 2007, 2012; Kalafat et al., 2007; Millstein, 2009). Limited research exists on longer-term user mental health status outcomes after helpline contact (Dalgin et al., 2018; Gould et al., 2018), and small sample sizes have limited assessment of outcomes among diverse subpopulations.

This study builds on a longitudinal survey of users of NYC Well, a behavioral health helpline serving New York City (NYC) and surrounding areas, conducted as part of a broader evaluation of NYC Well (Sparks et al., 2020). Launched in October 2016 under a citywide behavioral health initiative overseen by the NYC Mayor’s Office, NYC Well is a free and confidential service for New Yorkers seeking emotional support, suicide prevention or other crisis intervention, peer support, information and treatment referral, connection to mobile crisis teams, and follow-up services for behavioral health concerns. NYC Well helpline operators, which include both counselors and Peer Support Specialists, serve a large and diverse population of over 200,000 users each year (NYC Mayor’s Office of Operations, 2020). Counselors and Peer Support Specialists go through a 2- to 3-week training period that includes detailed discussions of protocols and approaches to providing NYC Well’s service, role modeling overseen by a trained senior counselor or Peer Support Specialist, and a set of observed interactions with NYC Well users. Users are not always assigned the same operator when they call back. Each call is treated as a “new” call in the phone triage. We surveyed helpline users of NYC Well roughly 2 weeks after contacting the helpline and again 6 months after to assess user experiences and changes in psychological distress, as well as factors associated with user experiences and mental health outcomes.

Methods

Data

Survey Sample

From April 1 to August 4, 2019, helpline providers invited all eligible individuals who contacted NYC Well by phone, text, or online chat to participate in a follow-up survey at the end of their helpline contact. To be eligible to participate, individuals had to be 13 years of age or older and speak English or Spanish. Users were not eligible if they were experiencing an acute behavioral health crisis (e.g., suicidal intent, impaired functioning or psychological distress, as assessed by their helpline provider), or if they were a clinician or service provider contacting the helpline on behalf of a patient or client. While users were eligible to participate no matter their assessed level of psychological distress in the prior month, users contacting the program in acute crisis were excluded due to concerns about the clinical safety and appropriateness of asking these individuals to complete the survey, and their ability to provide informed consent during their service interaction. The survey sample included individuals contacting the program on their own behalf and those contacting on behalf of someone else, although as noted below those contacting on behalf of someone else were excluded from the present analyses. We identified 43,132 unique helpline users making at least one contact during the recruitment period. Of 43,132 unique users contacting the helpline at least once during the study period, 31,460 (73%) met eligibility criteria, and 2283 users (7%) agreed to be contacted for the follow-up survey. Helpline providers retained discretion to not invite individuals to participate if not clinically appropriate and did not track when users refused or were unable to participate in the survey (i.e., contact ended abruptly). Among users eligible for the survey, those recruited had more helpline contacts; higher documented mental health risk levels, substance use, and documented suicidal ideation and intent; and more complete administrative data than those who were not recruited (see Supplementary Appendix for additional detail). While users who were recruited for the survey represented only 7% of all users contacting the helpline during the recruitment period, they accounted for 47% of the contacts made during the recruitment period. Therefore, while the users who agreed to participate in this study may not be representative of the population served by the helpline overall, they represent an important constituency.

Compliance with Ethical Standards

Individuals who consented to participate in this study also consented to having their NYC Well administrative records linked to their survey responses for analyses. This study was approved by the REDACTED Institutional Review Board. The supplementary materials provide additional detail on eligibility and recruitment.

Survey Data Collection

We surveyed NYC Well users about how they learned about the helpline, their experiences with it, recent use of other behavioral health services, self-reported mental health status, and demographic characteristics. We contacted consenting NYC Well users for the initial follow-up survey starting 2 weeks after their contact to the helpline, from April to September 2019 (N = 1097; response rate = 48.1%), and again 6 months after their helpline contact, from October 2019 to March 2020 (N = 732; response rate = 68.3%). We completed data collection before COVID-19 began to escalate in NYC, in March 2020. We fielded both surveys via email with links to a web survey; users who did not respond to the initial email received telephone follow-up. Respondents were provided a $30 gift card incentive for participating in the 2-week follow-up survey; a $5 incentive for confirming and validating their contact information at 3 months’ follow-up; and a $25 gift card incentive for completing the six-month follow-up survey.

Administrative Data

The helpline compiles information for each contact, including a unique user identification number; the date, time, and mode of contact; and user characteristics collected by providers (including demographic characteristics and history of suicidality and drug use). We linked user-level administrative data with the survey sample and survey responses.

Analytic Sample

This study focused on experiences and self-reported mental health status among helpline users contacting the program on their own behalf. We also limited the analytic sample to respondents providing valid responses to all survey questions used in the multivariate regression analyses, which resulted in the exclusion of 39 respondents. The final analytic sample size for this study was 857 respondents completing the 2-week follow-up survey and 595 completing the six-month follow-up survey.

Measures

Demographic Characteristics

We derived demographic information from survey questions included in the survey fielded 2 weeks following helpline contact, including age, gender, race, ethnicity, education, and whether the user had insurance.

Helpline Interaction

We obtained NYC Well administrative data on number and mode of helpline contacts. The survey asked whether or not users received a referral to or contact information for another provider during their helpline contact (Gould et al., 2012).

Experiences and Satisfaction with Helpline

We used three measures of experience and satisfaction with the helpline, adapted from a satisfaction survey fielded by NYC Well, collected in the survey fielded 2 weeks following helpline contact: Contacting NYC Well Helped Deal More Effectively with Problems (helped a lot, helped a little, didn’t help or hurt, made things a little worse, and made things a lot worse); Satisfaction with Experience with NYC Well (very much satisfied; somewhat satisfied; somewhat dissatisfied; or very dissatisfied); and Would Recommend NYC Well to a Friend in Need of Similar Help (would definitely recommend, would probably recommend; would probably not recommend, and would definitely not recommend).

We created binary analytic measures reflecting the most positive experiences: contact helped deal a lot more effectively with problems (vs a little, no difference, or worse); very satisfied with experience (vs somewhat satisfied or dissatisfied); and would definitely recommend to a friend (vs probably or would not recommend). We used top-box coding for these measures given the distribution of responses (a high proportion of respondents selected the best experience for each measure) and because even the second best response (e.g., probably recommend) indicates that the experience was not entirely positive (Elliott et al., 2009).

Mental Health Status

In both the 2-week and six-month follow-up surveys, we asked respondents to complete the Kessler 6 Psychological Distress Scale (K6), a six-item series of validated questions assessing psychological distress (Kessler et al., 2003; McGinty et al., 2020). The K6 requires respondents to report how often they felt the following during the past 30 days: nervous; hopeless; restless/fidgety; depressed; that everything was an effort; and worthless. Following the K6 guidelines, we scored each response on a scale from 0 (none of the time) to 4 (all of the time), and summed responses to generate a composite score ranging from 0 to 24 (Kessler et al., 2003). Individuals with scores between 0 and 4 were assessed as having no psychological distress, between 5 and 12 were assessed as having moderate psychological distress, and of 13 or higher were assessed as having serious psychological distress (Kessler et al., 2003).

Analysis

Survey Weights and Generalizability of the Survey Data

We used administrative data from the helpline to assess whether sampled users differed from those who were eligible to be recruited but were not; we also assessed differences between survey respondents and sampled non-respondents. In general, survey respondents were similar to non-respondents; however, the survey sample disproportionately represented repeat users and differed across a variety of dimensions, including having a greater share of users with documented substance use and suicidal ideation and intent, relative to users who were not selected for the survey sample. Because of the large magnitude of these differences, incorporating sampling weights introduced an unacceptable degree of imprecision into our analyses. We therefore used non-response weights to account for differences between the population of helpline users recruited to the sample and those who responded to the survey, without incorporating additional sampling adjustment to account for selection into the recruited sample. We calculated two non-response weights, both weighting respondents to reflect the entire survey sample: one for all respondents to the 2-week follow-up survey, and one for respondents to both surveys. Weighted analyses of both surveys can be considered representative of the users selected into the survey sample, who made roughly half of the contacts to the helpline during the study period, but not generalizable to the broader population of eligible or all helpline users. Supplementary Appendix B of the supplementary materials includes additional detail.

Summary Statistics

We used responses from the 2-week follow-up survey to calculate summary statistics on user characteristics, their experiences with the helpline, and changes in mental health status from the 2-week follow-up survey to the six-month follow-up survey. Bivariate differences in demographic characteristics and helpline experiences by level of psychological distress were calculated using chi-squared tests.

Multivariate Analyses

We used cross-sectional logistic regression to assess factors associated with helpline experiences reported in the 2-week follow-up survey. Dependent variables included binary measures reflecting positive experiences with the helpline. Explanatory measures included age, gender, race, Hispanic ethnicity, education, having health insurance, self-reported level of psychological distress, helpline contact mode, and number of helpline contacts during the study recruitment period.

To assess factors associated with changes in serious psychological distress between two surveys, we used generalized estimating equations (GEE) regression with a probit link to account for repeated observations within users over time (Ballinger, 2004). The dependent variable was serious psychological distress. Explanatory measures included demographic characteristics, mode and number of contacts, and receipt of a referral.

Results

The helpline served a diverse group of users across several demographic characteristics (Table 1). For example, among users recruited for the study, 37% of users identified as White, 30% as Black or African American, 8% as Asian, and 26% as other/multiple races; and 26% identified as Hispanic ethnicity. Among surveyed users, 75% contacted the helpline more than once, most frequently via telephone (75%), and 64% reported receiving a referral during their contact.

Table 1

Characteristics of helpline users

Characteristics	No. (%)
Age
13–17	28 (3.7)
18–34	480 (54.5)
35–54	214 (25.8)
55–64	96 (11.3)
65+	39 (4.7)
Gender
Male	274 (34.8)
Female	549 (61.4)
Other, transgender, or non-conforming	34 (3.7)
Race
White	316 (36.5)
Black or African American	259 (30.1)
Asian	69 (7.9)
Other or multiple	213 (25.5)
Hispanic ethnicity	219 (25.9)
Education
Less than high school graduate	85 (10.6)
High school graduate or some college	418 (49.7)
4-Year college degree or more	354 (39.8)
Had health insurance	733 (85.2)
Mode of helpline contact
Call only	622 (74.2)
Chat only	42 (4.4)
Text/SMS only	62 (6.7)
Multiple modes	131 (14.7)
Number of contacts over roughly 4 month period
1	221 (25.8)
2–5	342 (40.5)
6–19	182 (21.8)
20–99	85 (7.9)
100+	27 (4.0)
Receipt of referral to or information for a provider
Did not receive referral to or information for a provider	303 (36.0)
Received referral to or information for a provider	554 (64.0)

N = 857. Estimates in this table reflect user self-reported characteristics roughly 2 weeks after contact with the helpline and measures from the helpline’s administrative data. Surveys were collected from users contacting the helpline between April and August 2019. Estimates are weighted to adjust for survey non-response, such that weighted results can be considered representative of those who were recruited to complete the survey

Users recruited for the study reported broadly positive experiences interacting with the helpline when initially surveyed 2 weeks following their contact (Table 2). Sixty percent of users reported that their interaction with the helpline helped them deal a lot more effectively with problems, and another 29% reported that the helpline helped deal a little more effectively with problems. Likewise, nearly 70% reported being very satisfied with their helpline experience and 74% reported they would definitely recommend the helpline to a friend in need of similar help.

Table 2

User helpline experiences reported 2 weeks after contacting the helpline

User helpline experiences	No. (%)
Whether contact helped deal more effectively with problems
Helped a lot	500 (59.5)
Helped a little	255 (29.2)
Didn’t help or hurt	93 (10.5)
Little or a lot worse	8 (0.8)
Satisfaction with experience
Very satisfied	593 (69.9)
Somewhat satisfied	224 (25.7)
Somewhat or very dissatisfied	39 (4.4)
Would recommend to a friend in need of similar help
Definitely yes	634 (74.7)
Probably yes	181 (21.0)
Probably or definitely not	41 (4.3)

N = 857. Estimates in this table reflect user self-reported helpline experiences as recalled roughly 2 weeks after contact with the helpline. Surveys were collected from users contacting the helpline between April and August 2019. Estimates are weighted to adjust for survey non-response, such that weighted results can be considered representative of those who were recruited to complete the survey

After adjusting for other user characteristics, users with serious psychological distress were less likely to report that contacting the helpline helped deal a lot more effectively with problems than users with no psychological distress (− 11% points; 95% confidence interval (CI) − 21 to − 2), and users with moderate psychological distress were less likely to report that they would recommend the helpline than users with no psychological distress (− 8% points; 95% CI − 16 to − 0) (Table 3). Additionally, Asian respondents reported consistently worse experiences relative to White respondents, and users interacting only by chat reported consistently worse experiences relative to those who called the hotline. We found no statistically significant differences in experiences between users who did and did not recall receiving a referral to another provider.

Table 3

User characteristics associated with helpline experiences reported 2 weeks after contacting the helpline

User characteristics	Regression-adjusted percentage point difference relative to reference group (95% confidence interval)
	Contact helped deal a lot more effectively with problems (vs a little, no difference, or worse)	Very satisfied with experience (vs somewhat satisfied or dissatisfied)	Would definitely recommend to a friend (vs probably or would not recommend)
Age (ref: 18–34)
13–17	4.1 (− 18.5 to 26.6)	− 9.2 (− 31.9 to 13.5)	− 23.6 (− 48.5 to 1.3)
35–54	2.9 (− 5.3 to 11.2)	2.2 (− 5.2 to 9.6)	1.4 (− 5.6 to 8.5)
55–64	4.1 (− 7.2 to 15.3)	− 6.4 (− 17.7 to 4.9)	2.3 (− 7.5 to 12.1)
65+	− 1.6 (− 19.5 to 16.2)	− 13.4 (− 31.4 to 4.6)	− 5.3 (− 21.8 to 11.3)
Gender (ref: female)
Male	− 5.2 (− 12.7 to 2.2)	− 5.1 (− 12.2 to 2.1)	− 2.8 (− 9.3 to 3.7)
Other, transgender, or non-conforming	− 1.9 (− 19.8 to 15.9)	− 2.7 (− 17.8 to 12.3)	− 18.6 (− 36.2 to − 1.1)*
Race (ref: White)
Black or African American	− 0.1 (− 8.8 to 8.5)	5.0 (− 3.0 to 13.0)	6.5 (− 0.8 to 13.8)
Asian	− 21.7 (− 35.5 to − 7.8)*	− 10.5 (− 24.4 to 3.3)	− 14.1 (− 27.4 to − 0.7)*
Other or multiple	− 0.2 (− 9.5 to 9.2)	2.5 (− 6.7 to 11.7)	− 1.1 (− 10.1 to 7.8)
Hispanic (ref: not Hispanic)	0.2 (− 8.3 to 8.7)	− 1.3 (− 9.9 to 7.2)	9.9 (2.6 to 17.2)*
Education (ref: high school/some college)
Less than high school graduate	4.5 (− 8.5 to 17.4)	7.6 (− 3.5 to 18.7)	16.7 (8.5 to 25.0)*
4-Year college degree or more	− 5.6 (− 12.9 to 1.8)	− 0.8 (− 7.8 to 6.3)	− 0.9 (− 7.4 to 5.7)
Had health insurance (ref: none)	− 3.6 (− 13.1 to 5.8)	0.6 (− 8.4 to 9.6)	0.2 (− 8.5 to 8.8)
Psychological distress (ref: no)
Moderate psychological distress	− 8.6 (− 17.9 to 0.8)	− 7.5 (− 16.2 to 1.3)	− 8.4 (− 16.4 to − 0.3)*
Serious psychological distress	− 11.2 (− 20.6 to − 1.7)*	− 6.4 (− 15.1 to 2.3)	− 6.8 (− 14.8 to 1.3)
Mode of helpline contact (ref: call only)
Chat only	− 31.5 (− 47.0 to − 16.0)*	− 32.2 (− 49.3 to − 15.2)*	− 13.3 (− 29.1 to 2.5)
Text/SMS only	− 2.4 (− 15.7 to 10.8)	− 7.4 (− 20.2 to 5.4)	− 4.9 (− 16.9 to 7.0)
Multiple modes	− 2.3 (− 12.0 to 7.4)	− 4.4 (− 13.6 to 4.9)	0.5 (− 7.7 to 8.8)
Number of contacts over roughly 4 months (ref: 1)
2–5	− 0.8 (− 9.6 to 7.9)	− 2.6 (− 11.1 to 5.9)	2.9 (− 5.2 to 10.9)
6–19	7.6 (− 2.4 to 17.6)	2.5 (− 6.8 to 11.9)	10.1 (1.3 to 18.8)*
20+	6.6 (− 5.5 to 18.6)	5.5 (− 5.4 to 16.4)	7.4 (− 3.2 to 18.1)
Receipt of referral to or information for a provider	− 1.7 (− 9.0 to 5.6)	2.2 (− 4.9 to 9.3)	2.5 (− 4.2 to 9.1)
N	857	857	857
F-value	1.8	1.6	2.5
P-value from F-test	0.011	0.046	< 0.001

Estimates in this table reflect user self-reported characteristics and helpline experiences 2 weeks after contact with the helpline. Surveys were collected from users contacting the helpline between April and August 2019. Estimates are weighted to adjust for survey non-response, such that weighted results can be considered representative of those who were recruited to complete the survey

*P < 0.05

Users’ self-reported psychological distress improved between the 2-week follow-up survey and the six-month follow-up survey (Table 4). Across all users responding to both surveys, 41% reported serious psychological distress in the 2-week follow-up survey, relative to 29% in the six-month follow-up survey, a decrease of 12% points (95% CI − 17 to − 8). This improvement in psychological distress held across a broad set of demographic subgroups, including age (with the exception of the oldest and youngest users, where findings were not statistically significant), gender, race and ethnicity.

Table 4

Changes in proportion of respondents with serious psychological distress from 2 weeks after helpline contact and 6 months later

Population	N	Two-week follow-up survey, %	Six-month follow-up survey, %	Difference in serious psychological distress, percentage points	95% Confidence interval for difference, percentage points
Overall	595	41.3	29.0	− 12.3	− 16.8 to − 7.8*
Age
13–17	16	56.1	60.2	4.1	− 25.8 to 34.0
18–34	336	48.7	32.9	− 15.9	− 22.1 to − 9.6*
35–54	154	37.3	27.6	− 9.8	− 18.7 to − 0.8*
55–64	66	24.9	11.7	− 13.2	− 24.3 to − 2.1*
65 +	23	12.4	21.5	9.1	− 12.2 to 30.5
Gender
Male	187	39.6	26.4	− 13.2	− 21.0 to − 5.4*
Female	382	41.9	29.9	− 12.0	− 18.1 to − 6.0*
Other, transgender, or non-conforming	26	47.1	39.9	− 7.2	− 20.4 to 6.0
Race
White	224	46.7	35.0	− 11.7	− 19.6 to − 3.9*
Black or African American	183	34.2	27.3	− 6.8	− 15.2 to 1.6
Asian	44	38.8	25.9	− 13.0	− 33.4 to 7.5
Other or multiple	144	42.8	23.8	− 19.0	− 28.2 to − 9.8*
Ethnicity
Not Hispanic	443	43.8	29.8	− 14.0	− 19.1 to − 8.8*
Hispanic	152	34.7	26.8	− 7.8	− 18.4 to 2.7
Number of contacts over roughly 4 months
1	142	33.9	36.4	2.6	− 7.4 to 12.5
2–5	250	44.6	23.5	− 21.1	− 28.3 to − 13.8*
6–19	119	42.8	30.6	− 12.2	− 22.0 to − 2.4*
20 +	84	43.8	29.1	− 14.7	− 26.7 to − 2.7*
Mode
Call only	426	41.7	30.0	− 11.7	− 17.1 to − 6.2*
Chat only	26	44.2	14.3	− 29.9	− 51.0 to − 8.8*
Text/SMS only	44	28.0	20.0	− 8.0	− 23.5 to 7.5
Multiple modes	99	44.5	34.1	− 10.5	− 20.9 to 0.0
Receipt of referral to or information for a provider
Did not receive referral	237	40.0	31.4	− 8.6	− 16.7 to − 0.5*
Received referral	495	42.0	27.6	− 14.4	− 20.2 to − 8.6*

Estimates in this table reflect user self-reported characteristics and psychological distress 2 weeks and 6 months after contact with the helpline. Surveys were collected from users contacting the helpline between April and August 2019. Estimates are weighted to adjust for survey non-response, such that weighted results can be considered representative of those who were recruited to complete the survey. F value = 98.3 (P < 0.001)

*P < 0.05

Among the subset of respondents contacting the helpline multiple times, the prevalence of serious psychological distress was reduced between 12 and 21% points from 2 weeks to 6 months after their helpline contact. In contrast, respondents contacting the NYC Well helpline only once had a lower prevalence of serious psychological distress 2 weeks after their contact, but also reported little change over time (chi-squared test for equality across groups = 13.99, P = 0.003). There were no statistically significant differences in degree of improvement by contact mode or by referral receipt, although improvements in serious psychological distress for respondents contacting the helpline by online chat appeared substantially larger in magnitude (− 30% points; 95% CI: − 51 to − 9) than for users contacting the helpline by other modes. Similarly, though the magnitude of improvement for users receiving referrals (− 14% points) appeared larger than those for who did not receive a referral (− 9% points), differences by referral receipt were not statistically significant.

Discussion

This study assessed self-reported helpline experiences and changes in psychological distress 2 weeks after contacting the helpline and again at 6 months post-contact. Users reported broadly positive experiences with the helpline. The proportion of users reporting serious psychological distress 6 months after contacting the helpline was significantly reduced, relative to follow-up 2 weeks after. These findings may be informative for policymakers and operators of other helplines. NYC Well provides a broad set of services to a large and diverse population, which facilitates meaningful assessments of differences in experiences and psychological distress across user subpopulations.

To our knowledge, this is the first quantitative study with a large sample size to report findings on changes in mental health status 6 months after contacting a helpline. Our finding of improved psychological distress between 2 weeks and 6 months after helpline contact is consistent with other studies reporting improvements in mental health status (including suicidality) within a few weeks of contacting helplines (Gould et al., 2007, 2018; Kalafat et al., 2007). While Dalgin et al. (2018) found no statistically significant differences over time in the Recovery Assessment Scale, a measure designed to assess community integration, among helpline callers in a follow-up survey conducted longitudinally over 3 years, the study had a sample size of only 48 unique respondents. Users contacting the helpline multiple times and by internet chat or phone reported the greatest improvements in psychological distress; in contrast, users contacting the helpline only a single time or only by texting reported relatively lower improvements in psychological distress. To the extent feasible, helplines might explore the feasibility of facilitating repeated exchanges or deeper relationships with users. Users who received a referral to another provider also reported greater improvement in psychological distress than those who did not. Although the findings about call mode and receipt of referrals were not powered to detect statistically significant differences, these findings nonetheless offer insight to NYC Well and other helplines about which types of interactions are most beneficial. However, these improvements cannot be causally attributed to helpline use and many factors may have contributed to these improvements over that timeframe.

This study found that NYC Well users were satisfied with their helpline interactions. Most users reported that the helpline helped deal more effectively with problems, that they were satisfied with their overall experience, and that they would recommend the service to a friend. This is consistent with several other studies of helpline satisfaction (Biggs et al., 2015; Coveney et al., 2012; King, 1977; Millstein, 2009). While estimates of helpline satisfaction may be influenced by acquiescence bias (Millstein, 2009), helpline users may often contact helplines at a low point in their lives, and users across multiple studies and helplines have consistently reported positive experiences.

Both user experiences and improvement in self-reported psychological distress varied by demographic characteristics. Asian respondents reported consistently worse experiences relative to White respondents. These findings may reflect cultural differences in helpline use (Sue et al., 2012) and in how people respond to care experience surveys (Chung et al., 2016; Mayer et al., 2016). Additionally, relative to users reporting no psychological distress roughly 2 weeks following helpline contact, users reporting moderate or serious psychological distress were less likely to report positive experiences with the helpline. Users with moderate or serious psychological distress may have had more complex needs than those with no distress, which helplines may not be able to fully address during a single interaction.

Roughly three-quarters of users exclusively contacted the helpline via phone call, and 15% used multiple modes; relatively few users used only text or chat to contact the helpline. However, those users interacting only by chat reported consistently worse experiences relative to users who called the helpline. While offering chat may expand access to services, users contacting the helpline by chat may experience delayed response times relative to users contacting the helpline by phone, which could be a factor in explaining the relatively worse experiences reported by those users (Brody et al., 2020; Predmore et al., 2017). As helplines increasingly add online chat options, determining best practices for providing high quality interactions and needed care may be an important implementation consideration. Notably, while users contacting the helpline via chat and text would face similar delays in response times, text users reported similar experiences to call-only and multi-mode users; it is possible that users contacting the helpline by text (on a mobile phone) had an easier time multitasking between responses than users contacting the helpline via online chat.

These findings suggest opportunities for future research. First, few studies have examined the effects of using helplines beyond 6 weeks; an improved understanding of whether and how helplines promote longer term benefits for users would inform how helplines fit into the behavioral health system and help policymakers and helpline operators allocate resources more efficiently. For example, to what extent do helplines substitute vs. complement other behavioral health services, including outpatient therapy, urgent/emergent care, and care in inpatient settings? Research should consider how helplines—which by their very nature offer virtual, open access to behavioral health supports, typically on a 24/7 basis—are responding to and providing services during natural disasters or public health emergencies, when sectors of the formal behavioral healthcare system may be less accessible. Second, another important opportunity for future research is to better understand differences in helpline experiences and outcomes by race and ethnicity and the reasons for those differences. For example, are there approaches or strategies helplines can implement to improve their cultural or linguistic competencies? Third, identifying a valid comparison group poses unique challenges for studies of helplines, at least in part because the most important factors associated with the decision to use a helpline may not be available to researchers. Nonetheless, future research should consider whether it might be feasible to identify a comparison group for studies of helplines using administrative data, such as Medicaid claims/encounters or electronic health record data, especially in the context of a natural experiment (e.g., a helpline becoming active in one state or region at a given time, but not another state or region). The implementation of the National Suicide Hotline Designation Act of 2020 may provide a unique opportunity for such a study (Substance Abuse and Mental Health Services Administration, 2021).

This study has limitations. First, its findings cannot be interpreted causally. This study reports findings from a survey of helpline users following their contact to the helpline; it was not possible to identify and survey a similar comparison group of non-users or to survey users prior to their helpline contact, and other factors outside of NYC Well that we were not able to control for could have contributed to decreases in psychological distress (e.g., receiving behavioral health care from other providers, time to recover from an acute behavioral health episode). We were not able to obtain information about other activities individuals took after contacting the hotline that may have contributed to their change in distress. However, this study provides important context about the value of the helpline among users of the helpline, including across diverse populations. Second, survey respondents were not representative of all helpline users. As discussed in the methods section, exclusion criteria were applied when initially asking helpline users for consent to participate in the evaluation survey. Additionally, helpline users who consented to participate in the survey differed from eligible users who did not consent. Relative to those who did not consent to participate, recruited participants had more helpline contacts and higher documented risk of substance use and suicidal ideation and intent (Supplementary Appendix B). However, while the users who were recruited for the survey represented only about 7% of all helpline users during the study recruitment period, they accounted for 47% of helpline contacts during that time and response rates to follow-up surveys among users who agreed to participate in the study were relatively high, 48% for the first follow-up survey and 68% for the second follow-up survey. Third, some subgroups had small sample sizes, including the youngest and oldest age groups, Asian respondents, and respondents identifying as other gender, transgender, or gender non-conforming. This limited our ability to identify statistically significant differences of small and moderate magnitudes involving these groups. Fourth, we were not able to identify and collect survey responses from NYC Well users prior to or immediately after their helpline contact, and it is possible that the behavioral health status of users may have changed between the time they contacted the helpline and when they responded to the 2-week follow-up survey. As the survey used multiple modes (email and phone) and was collected by a third-party evaluator rather than helpline staff, workflows did not allow for immediate follow-up, and we were not able to identify users for a potential baseline survey until they made contact with the helpline. However, follow-up surveys even months after an encounter are common, especially for surveys about experiences with health care services (Elliott et al., 2009; Trombley et al., 2019). Additionally, the primary behavioral health outcome used by the study was the Kessler 6 measure of psychological distress, which asks respondents to report how often they felt adverse symptoms during the past 30 days, and which included the period when users contacted the helpline.

Our study of the NYC Well program suggests that behavioral health helplines play an important part of the behavioral health care system, reaching diverse communities and providing counseling and other supports to individuals in need, many of whom report symptoms of moderate to serious psychological distress. Those who contact helplines such as NYC Well may find the services they receive help them manage their problems and reduce distress. The virtual and 24/7 access provided by helplines may be especially critical during the natural disasters and public health emergencies like the COVID-19 pandemic, when the formal behavioral healthcare system may be less accessible and anxiety levels and social isolation may be heightened (Costa et al., 2020). Helplines may also offer an important alternative to using emergency services, along with other resources such as mobile crisis response (Fendrich et al., 2019) or referrals to other mental health services.

Below is the link to the electronic supplementary material.

Supplementary file 1 (DOCX 50 kb)