Evaluation of participant reluctance, confidence, and self-reported behaviors since being trained in a pharmacy Mental Health First Aid initiative.

In the U.S., an estimated one in five individuals experience a mental illness annually which contribute to significant human and economic cost. Pharmacists serving in a public health capacity are positioned to provide first aid level intervention to people experiencing a mental health crisis. Research on pharmacy professionals (pharmacists, technicians, students) undergoing training in Mental Health First Aid (MHFA) can provide evidence of the potential benefits of such training. The objectives of this study were to 1) describe the reluctance and confidence to intervene in mental health crises of pharmacy professionals previously trained in MHFA, 2) describe their self-reported use of MHFA behaviors since becoming trained, and 3) describe participant open-ended feedback on their MHFA training.
MATERIALS AND METHODS: An electronic survey was disseminated in May and June, 2019 using a four-email sequence to pharmacy professionals who had completed MHFA training from one of five pharmacist MHFA trainers throughout 2018. Domains included demographics, six Likert-type reluctance items, seven Likert-type confidence items for performing MHFA skills, and frequency of using a set of nine MHFA skills since being trained. Prompts collected open-ended feedback related to MHFA experiences and training. Descriptive statistics were used for scaled and multiple-choice items and a basic content analysis was performed on the open-ended items to group them into similar topics.
RESULTS: Ninety-eight out of 227 participants responded to the survey yielding a response rate of 44%. Participants reported high levels of disagreement to a set of reluctance items for intervening and overall high levels of confidence in performing a range of MHFA skills. Participant self-reported use of a set of MHFA skills ranged from 19% to 82% since being trained in MHFA. Almost half (44%) of participants had asked someone if they were considering suicide. A majority (61%) also had referred someone to resources because of a mental health crisis. Open-ended responses included positive experiences alongside important challenges to using MHFA in practice and recommendations including additional training focused on the pharmacy setting.
CONCLUSIONS: Pharmacy professionals in this evaluation reported little reluctance and high confidence related to using MHFA training and reported use of MHFA skills since being trained.

Introduction

It is estimated that one in five U.S. adults experience a mental illness annually but fewer than half receive treatment in a given year. [1] Untreated and undertreated mental illness also has a significant economic cost with an estimated 193.2 billion in lost productivity every year. [1] A large portion of the U.S. has a deficiency of mental health service providers and significant gaps in access to mental health services. [2]

Because of the high prevalence of under treated mental illness, there has been a public health effort to train non-mental health professionals to provide first aid level care to people experiencing mental illnesses and mental health crises. [3-5] Mental Health First Aid is one such training and is supported by the National Council on Behavioral health. The goal of MHFA is to train community members to decrease stigma toward mental illness, serve as a “first line of support” or gatekeepers for distressed people, and help people in crisis seek further assistance. [6] Mental Health First Aid U.S.A.is based on a five-step action plan that uses the acronym ALGEE®: assess for risk of suicide or self-harm, listen non-judgmentally, give reassurance and information, encourage appropriate professional help, and encourage self-help and other support strategies. [6]

Pharmacists are accessible and trusted health care professionals [7] that potentially can intervene and refer these individuals to help. [4, 8–11] Persons experiencing mental health crises, including those with warning signs of suicide, can present to pharmacies, clinic appointments, or other pharmacy practice settings during routine interactions such as when people procure medications, seek advice over the telephone, or engage with other pharmacy services. [8,10-14] Studies in pharmacy, however, cite a variety of barriers to intervening [8, 10, 15, 16], including a lack of confidence by the pharmacist [14, 17] and a need for additional training. [13,17] Pharmacists, especially those in the community-based setting, could play a key role in identifying people in mental health crises and refer them to appropriate professional help. [10, 11, 14, 18, 19]

Research suggests Mental Health First Aid (MHFA) can increase trainee knowledge, decrease reluctance, decrease stigma, and improve confidence related to intervening with persons experiencing mental health crises, including suicidal ideation. [20,21] Studies also suggest that MHFA training increases behaviors like identifying people in mental health crises and making referrals in the months and years following the initial MHFA training. [21-23]

Formal evaluations of MHFA and other gatekeeper training programs for pharmacy trainees beyond the initial training are lacking. [24, 25] Educational interventions related to mental health crisis intervention during pharmacy school and offered as continuing education for practicing pharmacists and pharmacy technicians may have an impact on improving confidence and knowledge and decreasing stigma, [3, 12, 24, 26–31] but more research is needed on how often trainees use gatekeeper skills in practice and their feelings of competency months and years after their initial training. Research on the longitudinal experience of pharmacy professionals that have been trained as mental health gatekeepers through programs like MHFA may provide evidence of program effectiveness and impact. The objectives of this research are to describe: 1) the reluctance and confidence of pharmacy professionals previously trained in MHFA, 2) self-reported use of MHFA behaviors since becoming trained, and 3) participant open-ended feedback on their MHFA training.

Materials and methods

This evaluation was based on an initiative by the National Community Pharmacy Association, Iowa Pharmacy Association, and the Community Pharmacy Foundation where funding was provided for 5 pharmacists to become accredited by the National Council on Behavioral Health to conduct MHFA U.S.A. training sessions. The initial week-long training of these 5 pharmacists occurred in 2017 or 2018. These trainers then facilitated MHFA training events for over 200 pharmacists, pharmacy faculty, pharmacy students, and pharmacy technicians in 2018. The MHFA training sessions were offered through a state pharmacy association to local individuals via an online signup, to student pharmacist groups, and to workshop attendees. Pharmacists attending any of the MHFA training sessions could receive eight hours of continuing education credit for their participation. This study was approved by the University of Iowa Institutional Review Board on May 8, 2019 IRB #201905722.

MHFA is an eight-hour training that introduces non-mental health professionals and the lay public to a five-step action plan for assessing individuals and making referrals for people experiencing mental health crises or illnesses. [6] The MHFA training sessions were delivered in-person either over a single 8-hour session or two 4-hour sessions to groups in Iowa, California, Florida, Indiana, and Oklahoma.

This study was guided by evaluation standards and used a single group post-only design. [27] Data were collected using an anonymous electronic survey administered through Qualtrics (Provo, UT). A sequence of four email contacts was used which include a pre-notification email, initial email, and two reminder emails. People could email the research team if they wanted to opt out of future emails. Each of the four emails was sent approximately six to eight days apart in order to stagger the day of the week that people were receiving emails. Four of the five pharmacist trainers provided email lists for the participants they had trained using the email addresses participants submit as part of their certification. One pharmacist was not used to supply participant lists as they provided MHFA in conjunction with one of the other trainers. Emails were sent between May and June, 2019 which was approximately 6–18 months following the trainings.

The survey included domains informed by a gatekeeper evaluation model by Burnette et al. [3] The domains included demographics, reluctance, confidence, and use of MHFA behaviors. Demographics included age, self-identified gender, and professional role. Reluctance was assessed using six items associated with reluctance to engage with mental health crises and were based on previous gatekeeper research. [26,32] Reluctance items included suicide and mental health misconceptions “If someone contemplating suicide does not seek assistance, there is nothing I can do to help” and environmental factors like “I am too busy to provide mental health first aid at work.” Confidence in performing MHFA skills was assessed using seven items and based on established MHFA self-evaluation questions. [6, 33] Confidence items focused on the ALGEE behaviors and included “Ask someone if they are thinking about suicide” and “Encourage someone experiencing a mental health crisis to seek professional help.” Use of MHFA behaviors was assessed using 9-items based on the MHFA training and other studies. [6, 22, 33] For these self-reported behaviors, participants selected if they had done them 0,1,2,3, 4 or more times, or not applicable. The self-reported behaviors included both thoughts like “Thought someone’s behavior might indicate they are having suicidal thoughts” and behaviors like “Asked someone if they are considering suicide” and “Referred someone to appropriate resources because you were concerned they were considering suicide.” Both thoughts and behaviors were assessed in the event that behaviors were rare. A median number of times was calculated by treating 4 or more as 4. Median and interquartile ranges were calculated for scaled items that demonstrated a non-normal distribution

Six open-ended text boxes were available for participants to provide free text responses. These included: Without using any identifying information, do you have any experience with MHFA you wish to share?; What major challenges or problems have you faced while using MHFA in your role as a pharmacist? What improvements would you recommend for future MHFA training recruitment? What suggestions would you propose related to how MHFA training is delivered? What suggestions would you propose for further supporting pharmacists in the area of mental healthcare? What do you still need help with? and Is there anything else on this topic not included in the previous questions that you feel important to tell us?

The survey was not piloted on the participants because we did not want to decrease the people available to take the survey. Multiple rounds of feedback were obtained from outside the research team using experts in evaluation and suicidology. Survey questions were phrased generally as opposed to referring specifically to the respondent’s professional role because as a gateleeper training, the goal is to give people skills to use regardless of the setting.

Descriptive statistics including frequencies, medians, and percents were calculated for all scaled items and frequencies and percents were calculated for multiple choice multiple-choice responses. Reliability coefficients were calculated for the confidence and reluctance scales. Open-ended responses were analyzed using a basic descriptive content analysis. [34] This included applying codes to describe text segments from the open-ended responses in an interactive process to group similar ideas from the participants. One author conducted an initial sorting and another author provided feedback. Discrepancies were resolved through discussion.

Results

Ninety-eight out of 227 invited participants responded to the survey which participants completed between 6 and 18 months since their training. Four emails were undeliverable, yielding a usable response rate of 44%. The median survey-completion time was 7 minutes. Sixty-two percent of respondents were female. Almost half of respondents were practicing pharmacists (46%) and 23% were student pharmacists (Table 1). Half of participants were trained between January and June, 2018.

Table 1

Demographics of survey respondents who participated in MHFA training (n = 98).

Variable	Frequency	Percentage
Gender
Female	61	62%
Male	22	22%
Non-binary gender	1	1%
Age category
Age below 25	9	9%
Age 25–34	32	33%
Age 35–44	20	20%
Age 45–54	8	8%
Age 55–64	13	13%
Age above 65	2	2%
Pharmacy role
Practicing pharmacist	45	46%
Student pharmacist	23	23%
Pharmacy school faculty	6	6%
Pharmacy technician	2	2%
Training period
Trained January-March, 2018	16	16%
Trained April-June, 2018	34	35%
Trained July-September, 2018	27	28%
Trained October-December, 2018	20	20%

Demographics not completed for n = 14.

Responses to the reluctance items favored disagreement suggesting respondents were willing to intervene with people experiencing a mental health crisis (Table 2). Participants most frequently reported strong disagreement to the statement that “I am too busy to provide mental health first aid at work (N = 57).”

Table 2

Responses to reluctance items for engaging in a mental health crisis (N = 98).

	Percent response
Item	1 = SD	2 = D	3 = N	4 = A	5 = SA	NA	NR	Median (IQR)
There is very little that I can do to help if someone thinking about suicide doesn’t acknowledge the situation	34	40	8	7	3	0	8	2 (1)
If someone contemplating suicide does not seek assistance, there is nothing I can do to help	48	34	5	3	2	0	8	1 (1)
If someone in a mental health crisis refuses to seek help, it should not be forced upon then	18	40	15	11	6	0	9	2 (1)
I cannot understand why anyone would contemplate suicide	56	18	12	1	3	1	8	1 (1)
I am too busy to provide mental health first aid at work	58	19	7	4	3	0	8	1 (1)
I do not know most patients well enough to know when they are in a mental health crisis	27	29	15	9	6	6	8	2 (2)

SD = Strongly disagree, D = Somewhat disagree, N = Neither agree or disagree, A = Somewhat agree, SA = Strongly agree, NA = Not Applicable, NR = No Response, IQR = Interquartile range

Cronbach’s Alpha = 0.79

Responses to positively worded confidence items favored positive answers with 5 of 7 item frequencies demonstrating strong agreement with the specific MHFA skill (Table 3). The greatest number of respondents strongly agreed they could listen non-judgmentally (n = 70) and encourage someone experiencing a mental health crisis to seek help (n = 64).

Table 3

Responses to confidence items for performing Mental Health First Aid skills (N = 98).

	Percent response
Item: I am confident I can. . . .	1 = SD	2 = D	3 = N	4 = A	5 = SA	NR	Median (IQR)
Recognize the signs that someone may need MHFA	1	1	0	46	39	11	4 (1)
Ask someone if they are thinking about suicide	0	0	2	46	39	11	4 (1)
Listen non-judgmentally to someone experiencing a mental health crisis	0	0	2	15	70	11	5 (0)
Offer basic "first aid" level information to someone experiencing a mental health crisis	0	0	1	32	54	11	5 (1)
Offer reassurance to someone experiencing a mental health crisis	0	0	0	35	52	11	5 (1)
Encourage someone experiencing a mental health crisis to seek professional help	0	0	0	23	64	11	5 (1)
Encourage self-help strategies for someone experiencing a mental health crisis	0	2	3	33	49	11	5 (1)

SD = Strongly disagree, D = Somewhat disagree, N = Neither agree or disagree, A = Somewhat agree, SA = Strongly agree, NA = Not Applicable, NR = No Response, IQR = Interquartile range

Cronbach’s Alpha = 0.82

Participant self-reported use of MHFA skills ranged from 19% to 82% using the skill since being trained in MHFA (Table 4). Most had asked someone about their distressed mood (82%), with 28% doing so 4 or more times. Almost half (44%) of participants had asked someone if they were considering suicide. A majority (61%) also had referred someone to resources because they were concerned they might be experiencing a mental health crisis.

Table 4

Respondent self-reported use of Mental Health First Aid skills since training (N = 98).

	Percent response
	No	Yes	NA	NR	Median times if Yes (IQR)
Thought someone’s behavior might indicate they are having a mental health crisis	13	72	1	13	2 (1)
Thought someone’s behavior might indicate they are having suicidal thoughts	28	57	2	13	2 (1)
Asked someone about their distressed mood	4	82	0	14	3 (2)
Asked someone if they are considering suicide	40	44	2	14	2 (1)
Listened non-judgmentally to someone experiencing a mental health crisis	4	80	2	14	3 (2)
Referred someone to appropriate resources because you were concerned they might be experiencing a mental health crisis	22	61	2	14	2 (2)
Referred someone to appropriate resources because you were concerned they were considering suicide	37	45	3	15	1 (1)
Engaged with a mental health crisis resource on behalf of someone	51	32	3	14	2 (2)
Engaged with emergency medical or police services because of someone experiencing a mental health crisis	63	19	4	13 (	2 (2)

NA = Not Applicable, NR = No Response median = if yes, how often? 1,2,3,4 or more times

The responses to the open-ended questions indicated positive MHFA experiences, challenges to using MHFA in practice, and insights for training improvements. For MHFA experiences, 10 shared their success stories in using MHFA skills in practice and 3 shared positive thoughts on the training.

“I talked to a patient on the phone who, I believe, was having a panic attack. She had questions about her new medication and asked several times if she was going to die. I recognized the feeling of dying as one of the signs of a panic attack and reassured her she was ok and talked her through it. She seemed to get better as I talked to her and seemed perfectly fine the next day.”

“A very positive experience. Some of the dialog examples have come in handy in a multitude of situations, including with both employees and patients.”

Twenty-two participants shared challenges or problems they have faced while using MHFA. Eleven participants described time constraints as a major challenge for using MHFA in practice. Participants stated that they do not have enough time given their workload to interact with people to determine the nature of their crisis, provide listening and support, and create and execute an intervention plan.

“[A busy community pharmacy setting] is not at all an ideal environment to even quickly assess if patients may need further counseling or reassurance of their symptoms. If the line is long, you will have the manager and even other customers in line irate and pushy to get that patient in front of you out of the door as soon as possible.”‘

While time constraints were reported as a challenge, some participants were still motivated to help people. For example, one participant stated that time “is a factor” but “not an excuse for not stepping up when needed.” Similarly, another participant indicated that helping people in the pharmacy requires “a lot of time” but it is “worth it.” Four participants indicated not having enough skills and resources as a challenge. One participant stated, “Realizing I still have a lot to learn, that people react to crisis situations in so many different ways.” Another participant said it is “a challenge to remember what was taught” and another found it challenging to differentiate a mental health crisis from “a bad day.”

Other challenges related to better understanding local resources and “understanding the system” and a lack of high-quality mental health providers in the area, including challenges with insurance coverage. Respondents also voiced concerns about privacy for these discussions and not having enough background information about their patients hampering their abilities to effectively intervene. Three respondents also saw stigma as a barrier, including other staff which may have stigmatizing beliefs about people with mental illness or self-stigmatizing beliefs held by people that may keep them from approaching the pharmacist.

The participants also shared insights on ways to improve the training and recruitment efforts. Ten participants recommended training more people, across settings and positions.

“all healthcare providers should be required to learn”

There also was interest in having the MHFA training occur at work and while being paid. For the content of training, there was interest in making it more specific to the pharmacy setting, including how MHFA could be used in brief pharmacy encounters. Two participants indicated that the content of the training was “repetitive” and suggested there could be higher level courses. Two participants suggested providing additional resources, such as a “brief refresher”, a phone app, and/or a card for “basic steps” that could be used as a reminder. Lastly, there was an interest in quality continuing education to maintain learning.

Discussion

Overall, people completing the survey reported disagreement to items assessing their reluctance to engage in MHFA and high levels of confidence in performing a range of MHFA behaviors. These pharmacy participants also reported using a range of MHFA skills since their training, often on more than one occasion. This is among the first longitudinal evaluation of a MHFA or other gatekeeper training in pharmacy to include self-reported behaviors since being trained.

The largest group of respondents was practicing pharmacists followed by student pharmacists. The students that responded were nearing the end of their schooling and may be speaking from practice experience on clinical rotations or during part-time work. The sample had a large participation of younger respondents, but this can be attributed to some of the MHFA sessions being targeted to students through a pharmacy school. Exposing student pharmacists to mental health crisis intervention training may have benefits both in and out of their work settings. [11, 24, 28, 30]

Regarding participant reluctance, most provided responses that are considered conducive to intervening with people experiencing mental health crises. One of the strongest held beliefs was disagreement that “I am too busy to provide mental health first aid at work.” While encouraging, this contradicts some of the open-ended responses and findings from other research suggesting it is hard to engage in mental health crisis intervention in the community pharmacy setting. [14-16] These answers may be subject to social desirability bias and suggest an aspirational state. Alternatively, consistent with some of the open-ended responses, participants may be committed to make time should the situation warrant. Although, people may be less likely to engage if a pharmacist seems too busy, lacks an established relationship, or feels stigmatized at the pharmacy. [16, 35]

The reluctance item with the lowest score was “I do not know most patients well enough to know when they are in a mental health crisis.” While responses, on average, were more favorable than neutral, this appears to be a potential barrier to consider. Pharmacies should continue to work to better understand the people that use their pharmacy and establish relationships as doing so can decrease stigma and increase help-seeking. [10, 15, 35]. While having a prior relationship has been reported by pharmacists as a facilitator for mental health crisis intervention [15] it should not be a requirement as MHFA is intended for any person, regardless of background, to help someone without needing a prior relationship. [6]

Respondents were highly confident in their abilities to use MHFA skills. They were most confident in their ability to listen non-judgmentally to someone experiencing a mental health crisis. Non-judgment is associated with decreased stigma, which MHFA evaluation studies have shown [20, 21], although stigma was not measured in the present study. This very high perception of listening non-judgmentally may be part an aspirational belief as remaining out of judgment can be difficult [36] and respondents could be over-estimating their abilities as this was one of the more subjective skills. There is evidence pharmacists tend toward medication-centric communication rather than patient-centric communication, including with mental illness. [37, 38]

Confidence also was high for encouraging someone experiencing a mental health crisis to seek professional help. We cannot be certain if participants interpreted this item as giving out the national suicide hotline number, encouraging someone to see a therapist, or something else. These overall high levels of confidence warrant testing as some research shows inconsistent performance of crisis intervention skills by pharmacists and students such as directly asking about suicide. [33, 39]

Encouragingly, respondents used a range of MHFA skills in the months following their initial training. Most participants had asked someone about their distressed mood and almost half reported asking someone if they were having suicidal thoughts and a similar percent referred someone to appropriate resources because of suicidal ideation. A caveat to these self-reported behaviors is that we do not know if these happened at work with a pharmacy patron, at work with a co-worker, or in someone’s personal life outside of work. Either way, these interventions represent potentially life-saving interventions. These MHFA trained pharmacy professionals appeared to ask about suicide in greater numbers than a general pharmacist population where only 14% had asked about suicide and more commonly the information was volunteered. [14]

The open-ended responses offer several areas for improvement and future work related to training. First, there was interest in continuing education after initial training. This could take the form of a refresher course, or intervening in mental health crises in pharmacy-specific environments such as the community or retail setting as the standard MFHA training does not address the environmental constraints of a community pharmacy. Such trainings are needed as the initial MHFA training certification is only for 3 years and the recertification course focuses on reviewing basic concepts rather than adding new concepts.

There also was interest in additional resources like a pocket guide or other brief ways to stay engaged with the skills presented in the training. For example, Washington State has mandated suicide prevention training for pharmacists [40] and has developed materials. There are other suicide focused trainings that pharmacists can take, such as question persuade refer (QPR) [41]. Some open-ended comments pointed out a perceived lack of local mental health professionals. Shortages of mental health professionals have been reported, especially in rural areas [2]. Pharmacy professionals also may need more training in navigating local mental health resources.

These data also raise several organizational concerns. Pharmacies should consider how to accommodate a pharmacist taking time out to interact with someone experiencing a mental health crisis. This could require considerations of staffing levels and private consultation areas [15, 16]. There also was a recommendation to have the MHFA training during work hours or on the clock. Employers should consider this as a benefit to employment and strengthening their workforce, including technicians and management.

This evaluation study has several limitations. With a single group design, it cannot be determined what proportion of participant beliefs and self-reported use of MHFA behaviors are attributable to their participation in one of the pharmacist-led MHFA training sessions. Experimental designs are needed. Also, it was not possible to evaluate differences among trainers. All surveys were disseminated at one time point and no adjustments were made based on when in 2018 the trainee completed their MHFA training. The survey data are based on self-report and the MHFA behaviors described in this report were not based on observation and may have variability in interpretation and may be subject to social desirability bias. There also could be response bias as most participants voluntarily pursued MHFA training and volunteered to take the evaluation survey. While the response rate was high for an electronic survey, non-responders may have had different experiences since being certified such as not having used MHFA in practice. While there were participants from multiple U.S. states, larger samples would be needed for greater confidence in generalizability.

This study also suggests several areas for future research. First, there is a need for larger randomized studies with pre and post evaluations, and longer term follow up. There also is a need to collect observational data to supplement self-report to determine if pharmacy professionals are effective in their mental health crisis interventions. Training topics that could supplement MHFA training could relate to restricting access to medications as a lethal means [9, 34] and counseling on the suicide risk associated with antidepressants. [42, 43] Calling for, and developing trainings and resources for making modifications to work environments are additional opportunities to improve how pharmacy professionals engage in mental health crisis intervention which would require further development and evaluation. Lastly, future work should consider measuring stigma and environmental factors as these have been identified as theoretically important to modeling mental health crisis interventions. [3]

Conclusions

This study found pharmacy professionals trained by pharmacist MHFA trainers had low reluctance and high levels of confidence in using the range of skills taught in the 8-hour MHFA program. Respondents also reported engaging in a range of MHFA behaviors since being trained such as asking about suicide, referring someone to resources, and engaging with a mental health crisis resource on someone’s behalf. While pharmacy professionals were positive about the training, there was interest in pharmacy-specific continuing education such as providing MHFA within realistic pharmacy environments.

(DOCX)

Click here for additional data file.

13 Jan 2020

PONE-D-19-33742

Evaluation of Mental Health First Aid training initiative for pharmacy

PLOS ONE

Dear Dr Witry,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please provide a response to the included reviews.

We would appreciate receiving your revised manuscript by Feb 27 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Andrew Soundy

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses.

For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information.

3. Thank you for stating the following in the Competing Interests section:

'AP was one of the pharmacist MHFA facilitators.'

a. Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these.

Please note that we cannot proceed with consideration of your article until this information has been declared.

b. Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Partly

Reviewer #4: Partly

Reviewer #5: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

Reviewer #5: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

Reviewer #5: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Introduction, Lines 59-62: Not sure the relevance of discussing MHFA on attitudes since this was not an objective of your study. Consider deletion

Line 71: What data-informed changes do you anticipate or might result? What has the literature shown? Has previous studies shown a low uptake of MHFA or low use of skills of those who are trained? Would be nice to have a little more justification as to why this study was undertaken

Lines 95-100: Appreciate that this survey was developed from a literature search and the MHFA training assessment questions, but were any efforts undertaken to validate or pilot your survey? Are the MHFA questions validated? This is a somewhat major limitation of the study if not addressed in my opinion.

Lines 101-104: It would be nice to understand at a deeper level what your content analysis looked like and how it was coded. Where any validated frameworks used to develop these open-ended questions? If not, how were they created/validated to ensure they measured what they were designed to measure?

Table 1 -- might consider use of subheadings for each demographic o more clearly differentiate these for your readers

Lines 113-135: Might consider differentiating the reluctance items in your methods so that your reader is anticipating this. Delineating this differences and making it clear to your reader what these mean/what they were assessing

Tables 2 and 3 -- might consider presenting data as % vs numbers -- would be easier to interpret. Table 4 presents both n and % -- suggest being consistent between all tables

Lines 228--236: I think this is a little out of place and doesn't focus on the most important findings of your study and putting them into context -- an important point to highlight might be other trainings for pharmacist intervention in mental health, such as WA state where training for suicide prevention is required. Pharmacists expressed interest in continuing education/revisiting training, which is required to maintain the certification -- the MHFA is only good for 3 years. I would like to see a better analysis of what pharmacy education literature has shown as some of their findings were different than yours which is important with a significant n being student pharmacists. Additionally, Australia and Canada are much further in engaging pharmacists in this healthcare role. What can we learn from them?

Limitation-- also that the survey was not validated. Pharmacists who didn't respond might not have responded because they aren't using these skills and concerns for judgment/failing

Reviewer #2: Research into the impact of MHFA training on behavioral changes in trained pharmacists is needed. This paper provides a beginning to that evidence base. Since it was a post-training survey, the information may have been more impactful if a pre-training survey had been taken by the trainees, a consideration for future research.

Line 84: "assessing patients" This sentence is part of a paragraph discussing general application of MHFA, not specifically health care professionals. Consider whether this should be "assessing individuals" as the term "patient" implies a health care professional, not a member of the general public.

Lines 88-89: "The survey was anonymous." This sentence seems repetitive since the previous sentence states that if was an anonymous electronic survey.

Lines 91-92: Email lists. Were these email lists that were kept by the pharmacists trainers with a view to doing a study? Were these the email lists that were obtained at the start of the training that are required by MHFA for submission and certification? Was MHFA contacted about the use of these email lists for research? Were the participants who provided their email addresses at their MHFA training made aware that their email may be used in a future research study and were they given the opportunity to opt out of having their email on a research list?

Lines 161: "understating the system", should this be "understanding the system"?

Lines 200-204: This paragraph discusses how the respondents felt that they didn't know their patients well enough to be providing mental health information. Please consider discussing here how MHFA training is intended for the lay public to be able to help anyone, including a stranger or someone that they don't know well. This is a basic tenet of the training, that it is first aid that can be provided to anyone. This should be included to remind the reader that MHFA is not meant to be used only with people that the trained person knows well.

Reviewer #3: Thank you for the invitation to review the paper by Witry et al. I think this paper is adding to current literature in attempting to understand how MHFA training is used in pharmacy practice. Previous research on interventions to improve confidence and willingness to engage with people who experience mental illness tend to report pre/immediate post, self-report evaluations of the intervention and lack longitudinal follow up. There is limited research on the topic of suicide prevention in pharmacy practice or on the contribution of pharmacists beyond the dispensing of medication. While the study has many limitations as acknowledged by the authors, it is important to publish the results of this study to build evidence on the effectiveness of MHFA as a potential educational intervention for pharmacists. Below are some comments and suggestions regarding the paper that I hope the authors will find useful.

Introduction

• Title: This paper focusses on the use of MHFA as gatekeeper training for suicide or crisis management. I feel this should be reflected in the title.

• Line 19: “describe the reluctance and confidence” to do what?

• Line 45: Is the purpose of MHFA in the US to bridge gaps in access to mental health services? Perhaps the purpose of MHFA as described in the MHFA manual could be described here so that we know we are not training pharmacists as amateur counsellors.

• Line 65: The new Australian and Canadian study may be helpful to contextualise here https://ps.psychiatryonline.org/doi/10.1176/appi.ps.201900244

Methods

• How were participants recruited for the training?

• Line 83. MHFA studies usually include a slightly more detailed description of the intervention such as an overview of the content and the action plan. This would be useful to the reader as the US version is clearly different to the original Australian and European versions and most of the pharmacy research to date uses the Australian version. Did you use pharmacy specific scenarios?

• Line: Already stated the survey was anonymous.

• Line 96: The questions here seem to address attitudes to suicide prevention as much as knowledge.

• Were the questions specifically related to the use of MHFA in the pharmacy setting? Not use of the skills in a personal context or with colleague/ student? Did the question say ‘since completing MHFA have you….?

• Line 103: Was the content analysis carried out by one author alone? Quotes selected to demonstrate themes.

Results

• In general it would be useful if there were more descriptive titles in the tables so they could stand alone.

• Line 113. I find this sentence a little confusing. Perhaps clarify?

• Line 140: Any example of a success story worth sharing?

Discussion

• Line 180: I would find it more readable to simplify/clarify the language of the paper summary here.

• Line 221: The most useful addition to current literature in my opinion is the aspect on how often the pharmacists used their skills. It is a shame there was no baseline comparison.

• Line 253: Can continuing education address environmental concerns? Need to assess the work environment and identify ways of engaging with all patients privately.

Reviewer #4: Abstract:

• If room could a background be given before objectives?

• ‘4-contact design’ is unclear?

• Seems unnecessary to state ‘four emails were undeliverable in the abstract’?

• Line 33 – ‘the skill’. MHFA is about more than one skill?

Background:

• Suggest mental health ‘service’ providers

• Paragraph 3 refs 18 and 19 – sentence refers to MHFA but only one ref is about MHFA? Also there are systematic reviews and meta-analyses of effects of MHFA

• Paragraph 4 – unclear what ‘programs’ this sentence is referring to? A comprehensive discussion of current evidence base would be better.

• A concern with this paper is the discussion of ‘behaviors’ and last paragraph of introduction talks about increases in behaviors post MHFA training. It is important to note these are self-reported behaviors and rely on participants to report whether they used skills in real life or present fictitious scenarios and ask what a participant would do in these situations. It is important to note this as a limitation.

• ‘Such work also may prompt data informed changes’ – unclear what is meant by this?

Methods

• Were MHFA instructors accredited with MHFA USA?

• Reference to ref 27 on line 87 is unclear?

• Was there ethical approval to conduct this study and to email participants of the MHFA training about the study?

• I have concerns about the validity of the survey instrument. ‘Reluctance to engage’ is not a knowledge construct and appears to be more about attitudes towards providing MHFA? Can more information be provided about the psychometric properties of this scale and how it was constructed/developed?

• It is unclear what the open-ended questions were?

Results:

• Table 1 – what is P1, P2 etc?

• Not appropriate to say ‘skewed to disagreement’ or ‘skewed positive’, something needs to be statistically significant or not

• If data is skewed in one direction it is not appropriate to calculate means, it should be median, or just report percent of participants with each response.

• Suggest in tables 2 and 3 to put the labels of 1-5 in the columns to make it easier to follow

• Table 4 – says if people answer yea then they answer either ‘1, 2, 3, 4+ times’. How is a mean calculated if just 4 or more? Or was this a free response to nominate any number?

• Again – unclear on what open-ended questions were about?

Discussion:

• I feel the discussion needs some reshaping as I don’t think the claims can be appropriately backed up by the data presented in this study. There is no pre-MHFA data so it is hard to say there are high levels of confidence in MHFA skills post training when there is no baseline data. Also concerns again about he validity of the knowledge/reluctance items.

• It reports to be the first longitudinal evaluation of MHFA in pharmacy, but there is no baseline and it is unclear as to what the timeframe is that the surveys have been completed after training completion.

• Behaviors – again this should be used with care and referred to as ‘self-reported behaviors’

• Lines 217-220 – Lack of mental health services and confidence of respondents seem to be very different points? This paragraph needs some reworking

• The discussion lacks reference to a lot of work done in the space (e.g. Ashoorian et al 2018 Early Int Psych, Rodgers et al. 2019 Adv Ment Health, Boukouvalas et al. 2018 Soc Psych Psychiatr Epidemiol)

Minor:

A number of language suggestions:

• Report numbers in words when less than 10

• ‘Trainings’ – suggest training programs

• ‘Persons’ – people

• ‘mental health condition’ – suggest mental illness not condition as condition is not a diagnosed illness

• Use people living with a mental illness not patients. Mental illness can be very stigmatizing and people find it labelling to be called a ‘patient’.

Reviewer #5: Evaluation of Mental Health First Aid training initiative for pharmacy

PLOS ONE

The present manuscript is a report of a survey evaluating the outcomes of pharmacy professionals trained in MHFA. Outcomes explored include reluctance and confidence as a result of MHFA, MHFA behaviors since training, and feedback on their MHFA training.

1. Abstract: Please clarify what pharmacy professionals mean.

2. Abstract: Please indicate timing of the survey as it relates to the training.

3. Abstract: Awkward wording “conducive” responses. Please consider rewording in this section and elsewhere.

4. Introduction: Is “gatekeeper training” language typical since this language seems somewhat new and may be confusing to readers? Please clarify this language early in the paper so readers are not confused with other references to gatekeepers in healthcare.

5. Introduction: It may be helpful if authors clarify that the first aid level of care in lines 46-47 are not intended to meet the gap of the disparity in access since first aiders are not trained to provide professional-level services but rather to catch more folks missed by the system due to the disparity and that need to get into the system then managed by trained professionals. It is important to clarify this point early since most readers would be confused as to how MHFA really is addressing the gap in access to services. In some ways, it could widen the gap since there could be more folks identified by MHFA that just won’t get the needed services because there is a lack of providers, etc. Please clarify and address.

6. Introduction: Authors need to give a brief synopsis about what MHFA is, what is covered by MHFA, and how it is similar or different from other trainings. Why evaluating MHFA vs. other trainings? It should be noted that NCBH is the organization supporting MHFA training/certification. Please add.

7. Introduction: There is a small literature involving pharmacists/pharmacy students and MHFA and was surprised it was not briefly presented- at least highlighting what we do know about MHFA from these studies. Please consider adding this section.

8. Materials and Methods: It might be useful to clarify that the 5 pharmacists trained to conduct MHFA trainings were presumably trained through a week-long instructor training on how to lead MHFA trainings provided by the National Council for Behavioral Health (NCBH). Please add clarification.

9. Materials and Methods: The authors are not completely accurate that the US version is an 8-hour training. It actually can be delivered in multiple versions of two 4-hour sessions, or four 2-hr sessions. It is not clear there is a “US version”. The authors probably meant that the version their 5 pharmacists completed was 8-hour but doesn’t mean the only version available. Please clarify.

10. Materials and Methods: Please clarify when survey was administered with regards to when MHFA attendees attended their respective trainings. It would be good to note since it could be that survey was administered much further out in time from some trainings and not others. It would be useful to note if the later trainees had different perspectives from the earlier trainees. Please also indicate how many trainings were provided by each instructor? Is there any way to track responses to the trainer and see if a particular trainer had an impact vs. others? Please clarify.

11. Materials and Methods: The authors really need to provide more information about the knowledge, confidence, and use of skills items with greater details. Readers need to have a better sense of what these items measure- even examples of 2-3 items would be helpful. I would include some details in methods and refer to table for actual items. Reliability coefficients for scales should be in methods section. Please also state what types of descriptive statistics was used for which data. Please address.

12. Materials and Methods: Is knowledge truly knowledge or attitudes? Authors used an attitudinal scale of agreement and items seem more attitudes than knowledge. Please justify items as knowledge vs. attitudes and explain how agreement scale is reflection of knowledge vs. attitudes. Knowledge is usually assessed more as nominal items of correct or not.

13. Materials and Methods: Was there only one coder? Please clarify multiple coders and how discrepancies were resolved.

14. Materials and Methods: Please indicate at end of methods IRB approval.

15. Results: Table 1. Authors might want to collapse ages into fewer categories to save journal space. Also, not sure the need for P1-P4 student designations and likely lost on an international readership. It would be interesting to have collected prior experience with mental illness, etc. This might be an important to note as either a limitation or future direction. Some literature shows prior experience with mental illness either clinically or in personal lives affects attitudes and skills.

16. Results: Tables 2 and 3. Please provide n (frequency) AND % for Strongly Disagree to Strongly Agree. Most tables like this abbreviate SD for Strongly Disagree, D= Disagree, etc. See all numbers as column headers and in rows is somewhat confusing and distracting. Authors might include medians as well. Please indicate total sample size in table headers.

17. Results: Table 4. Total sample size responding to this item should be included in table header.

18. Discussion: Please consider above comments as they relate to comments in this section. Additional limitations include: generalizability of findings to other settings, lack of validated measures, lack of a comparison group and baseline measures to discern differences across time, potential temporal issues between time of trainings and survey (memory recall biases), and inability to discern trainer effects on outcomes.

19. Discussion: Authors please consider existing literature on MFHA and other trainings and relate current finding/results to such prior works. Connecting the work and results with any conceptual frameworks related to the topic would be a plus and add value to this work.

20. General comments: Authors might consider getting permission NCBH about publishing based on their program. It can’t hurt if not already done.

21. Discussion/Conclusions: Overall, these sections are generally well written with many excellent points brought up. It would been useful if the authors has a future directions section where they can propose are next steps/directions based on work and findings. They talked about future research possibilities at different points of the discussions but not as it own section with defined priorities and plans. Such a focused section based on findings and limitations will help readers see this as new program of research helping to establish the best scientific evidence for the effective use of MHFA in pharmacy practice.

This manuscript does offer some new information compared to prior publications. However, it is significant methodological limitations and could be strengthened in several areas that lack clarity and specificity. This reviewer wishes the authors the best as they pursue this work and seek publication.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

Reviewer #5: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

28 Feb 2020

PONE-D-19-33742

Evaluation of Mental Health First Aid training initiative for pharmacy

PLOS ONE

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming.

--I believe we have adopted the style. I did not see a web resource for file naming, but I copied what was in the video.

Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses.

--We have added additional detail on the questionnaire, including a copy.

For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information.

--Attaching

3. Thank you for stating the following in the Competing Interests section:

--'AP was one of the pharmacist MHFA facilitators.'

"This does not alter our adherence to PLOS ONE policies on sharing data and materials.”

4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

--It is probably more appropriate for this data to be available upon request. We have edited the cover letter to request this change.

Reviewer #1:

Introduction, Lines 59-62: Not sure the relevance of discussing MHFA on attitudes since this was not an objective of your study. Consider deletion

--By attitudes we meant confidence and reluctance, we have revised this to be clearer.

Line 71: What data-informed changes do you anticipate or might result? What has the literature shown? Has previous studies shown a low uptake of MHFA or low use of skills of those who are trained? Would be nice to have a little more justification as to why this study was undertaken.

--Our thinking was that if a certain item had low confidence or a certain behavior seemed rare, that it could be targeted for additional training. But we have deleted this to streamline the manuscript.

Lines 95-100: Appreciate that this survey was developed from a literature search and the MHFA training assessment questions, but were any efforts undertaken to validate or pilot your survey? Are the MHFA questions validated? This is a somewhat major limitation of the study if not addressed in my opinion.

--We did get feedback from persons outside the research team from evaluation and suicidology. It is difficult to validate these questionnaires, and other authors have pointed this out when publishing their work. Suicide is fortunately relatively rare. So we tend to use instruments that have demonstrated utility in previous studies, certainly more work is needed that tests the link between attitudes and behaviors. This has been added to the discussion.

Lines 101-104: It would be nice to understand at a deeper level what your content analysis looked like and how it was coded. Where any validated frameworks used to develop these open-ended questions? If not, how were they created/validated to ensure they measured what they were designed to measure?

--This was a very basic, non-interpretive content analysis where we grouped like responses together. We have added a bit more detail including that 1 coder provided an initial sorting and another reviewed the coded data and selected representative quotes,

Table 1 -- might consider use of subheadings for each demographic o more clearly differentiate these for your readers

--Updated. Thank you.

Lines 113-135: Might consider differentiating the reluctance items in your methods so that your reader is anticipating this. Delineating this differences and making it clear to your reader what these mean/what they were assessing

--We have changed this to be more clear throughout, using reluctance instead of knowledge in most occurrences.

Tables 2 and 3 -- might consider presenting data as % vs numbers -- would be easier to interpret. Table 4 presents both n and % -- suggest being consistent between all tables

--Changed to percents only. Also added median as recommended by another reviewer

Lines 228--236: I think this is a little out of place and doesn't focus on the most important findings of your study and putting them into context -- an important point to highlight might be other trainings for pharmacist intervention in mental health, such as WA state where training for suicide prevention is required. Pharmacists expressed interest in continuing education/revisiting training, which is required to maintain the certification -- the MHFA is only good for 3 years. I would like to see a better analysis of what pharmacy education literature has shown as some of their findings were different than yours which is important with a significant n being student pharmacists. Additionally, Australia and Canada are much further in engaging pharmacists in this healthcare role. What can we learn from them?

--We have added the need for recertification at 3 years. And added a reference to Washington State and QPR. We have added more comparison to other studies elsewhere in the introduction and discussion.

Limitation-- also that the survey was not validated. Pharmacists who didn't respond might not have responded because they aren't using these skills and concerns for judgment/failing

--Thank you, we added the detail you suggested about non-response bias to the limitations

Reviewer #2: Research into the impact of MHFA training on behavioral changes in trained pharmacists is needed. This paper provides a beginning to that evidence base. Since it was a post-training survey, the information may have been more impactful if a pre-training survey had been taken by the trainees, a consideration for future research.

--Agreed, we have this in future research and limitations.

Line 84: "assessing patients" This sentence is part of a paragraph discussing general application of MHFA, not specifically health care professionals. Consider whether this should be "assessing individuals" as the term "patient" implies a health care professional, not a member of the general public.

--Good catch, thank you, we have made this change.

Lines 88-89: "The survey was anonymous." This sentence seems repetitive since the previous sentence states that if was an anonymous electronic survey.

--Deleted

Lines 91-92: Email lists. Were these email lists that were kept by the pharmacists trainers with a view to doing a study? Were these the email lists that were obtained at the start of the training that are required by MHFA for submission and certification? Was MHFA contacted about the use of these email lists for research? Were the participants who provided their email addresses at their MHFA training made aware that their email may be used in a future research study and were they given the opportunity to opt out of having their email on a research list?

--We have added details about opting out and where the email addresses came from. We did discuss or research project with MHFA USA and were encouraged to proceed. IRB has been added and details about protecting human subjects has been added.

Lines 161: "understating the system", should this be "understanding the system"?

--Thank you for catching this. Changed

Lines 200-204: This paragraph discusses how the respondents felt that they didn't know their patients well enough to be providing mental health information. Please consider discussing here how MHFA training is intended for the lay public to be able to help anyone, including a stranger or someone that they don't know well. This is a basic tenet of the training, that it is first aid that can be provided to anyone. This should be included to remind the reader that MHFA is not meant to be used only with people that the trained person knows well.

--This is a good point and we have added “However, MHFA is intended for any person, regardless of background, to help someone without needing a prior relationship.( 6)”

Reviewer #3: Thank you for the invitation to review the paper by Witry et al. I think this paper is adding to current literature in attempting to understand how MHFA training is used in pharmacy practice. Previous research on interventions to improve confidence and willingness to engage with people who experience mental illness tend to report pre/immediate post, self-report evaluations of the intervention and lack longitudinal follow up. There is limited research on the topic of suicide prevention in pharmacy practice or on the contribution of pharmacists beyond the dispensing of medication. While the study has many limitations as acknowledged by the authors, it is important to publish the results of this study to build evidence on the effectiveness of MHFA as a potential educational intervention for pharmacists. Below are some comments and suggestions regarding the paper that I hope the authors will find useful.

--Thank you for your review

Introduction

Title: This paper focusses on the use of MHFA as gatekeeper training for suicide or crisis management. I feel this should be reflected in the title.

--Title has been updated to better reflect content.

Line 19: “describe the reluctance and confidence” to do what?

--We have added “to intervene in mental health crises” to clarify.

Line 45: Is the purpose of MHFA in the US to bridge gaps in access to mental health services? Perhaps the purpose of MHFA as described in the MHFA manual could be described here so that we know we are not training pharmacists as amateur counsellors.

--Great recommendation, we have added this line to the end of the first paragraph. Mental Health First Aid is one such training with the goal of training community members to serve as a “first line of support” for distressed persons and helping the person in crisis seek further assistance.

Line 65: The new Australian and Canadian study may be helpful to contextualise here https://ps.psychiatryonline.org/doi/10.1176/appi.ps.201900244

--Thank you, we have added to the introduction and discussion.

Methods

How were participants recruited for the training?

---Recruitment varied, we added some detail to the methods. The trainings were offered through a state pharmacy association to local individuals via an online signup, to student pharmacist groups, and to workshop attendees,

Line 83. MHFA studies usually include a slightly more detailed description of the intervention such as an overview of the content and the action plan. This would be useful to the reader as the US version is clearly different to the original Australian and European versions and most of the pharmacy research to date uses the Australian version. Did you use pharmacy specific scenarios?

---We have added additional detail, thank you for the recommendation. Mental Health First Aid is one such training with the goal of training community members to serve as a “first line of support” for distressed persons and helping the person in crisis seek further assistance.( 6) Mental Health First Aid U.S.A. is based on a 5-step action plan that uses the acronym ALGEE®: assess for risk of suicide or self-harm, listen non-judgmentally, give reassurance and information, encourage appropriate professional help, and encourage self-help and other support strategies.(6)

Line: Already stated the survey was anonymous.

--Deleted.

Line 96: The questions here seem to address attitudes to suicide prevention as much as knowledge.

--We have edited this and are consistently referring to “reluctance” instead of knowledge. This is recommended by the RAND article.

Were the questions specifically related to the use of MHFA in the pharmacy setting? Not use of the skills in a personal context or with colleague/ student?

--We chose to be general and have added this statement to the methods. Survey questions were phrased generally as opposed to referring specifically to the respondent’s professional role because as a gateleeper training, the goal is to give persons skills to use regardless of the setting. We have added this clarification to the discussion.

Did the question say ‘since completing MHFA have you….?

--We have attached the survey. The survey contained this prompt prior to asking about behaviors. Please report your best estimate of the number of times that you have had experience with the situation since completing MHFA training.

Line 103: Was the content analysis carried out by one author alone? Quotes selected to demonstrate themes.

--Two authors worked on the content analysis. We are hesitant to call these themes given it’s a content analysis.

Results

In general it would be useful if there were more descriptive titles in the tables so they could stand alone.

--Thank you, we have made the tables more descriptive.

Line 113. I find this sentence a little confusing. Perhaps clarify?

--We have reworded all uses of conducive per other reviewer. This should help clarify.

Line 140: Any example of a success story worth sharing?

---Added

Discussion

Line 180: I would find it more readable to simplify/clarify the language of the paper summary here.

--We have clarified, removing conducive throughout.

Line 221: The most useful addition to current literature in my opinion is the aspect on how often the pharmacists used their skills. It is a shame there was no baseline comparison.

--Agreed.

Line 253: Can continuing education address environmental concerns? Need to assess the work environment and identify ways of engaging with all patients privately.

--We have added more detail to the body.

Reviewer #4:

Abstract: If room could a background be given before objectives?

--Added

‘4-contact design’ is unclear?

--We have edited this throughout to 4 email sequence.

Seems unnecessary to state ‘four emails were undeliverable in the abstract’?

--deleted

Line 33 – ‘the skill’. MHFA is about more than one skill?

--This has been clarified.

Background:

Suggest mental health ‘service’ providers

--changed

Paragraph 3 refs 18 and 19 – sentence refers to MHFA but only one ref is about MHFA? Also there are systematic reviews and meta-analyses of effects of MHFA

--Added, thank you for the suggestion,

Paragraph 4 – unclear what ‘programs’ this sentence is referring to? A comprehensive discussion of current evidence base would be better.

--This paragraph has been rewritten to address this concern.

A concern with this paper is the discussion of ‘behaviors’ and last paragraph of introduction talks about increases in behaviors post MHFA training. It is important to note these are self-reported behaviors and rely on participants to report whether they used skills in real life or present fictitious scenarios and ask what a participant would do in these situations. It is important to note this as a limitation.

--Agreed. We have emphasized this more as a limitation.

‘Such work also may prompt data informed changes’ – unclear what is meant by this?

---We have clarified this. Also, variation in confidence, reluctance, and self-reported behaviors could prompt supplemental education.(

Methods

Were MHFA instructors accredited with MHFA USA?

--Yes, we have added this clarification.

Reference to ref 27 on line 87 is unclear?

--The evaluation is available from the group that conducted it.

Was there ethical approval to conduct this study and to email participants of the MHFA training about the study?

--Yes we have added the IRB information and detail about how persons could opt out.

I have concerns about the validity of the survey instrument. ‘Reluctance to engage’ is not a knowledge construct and appears to be more about attitudes towards providing MHFA? Can more information be provided about the psychometric properties of this scale and how it was constructed/developed?

--We based this on the RAND gatekeeper review study and have clarified by removing reference to knowledge. We provide 2 references for the reluctance items. The Cronbach alphas are provided underneath the two tables. These items we adapted from other articles.

It is unclear what the open-ended questions were?

--Thanks for pointing this out. We have added them to the methods.

Results:

Table 1 – what is P1, P2 etc?

--Year in the pharmacy program, we have deleted and just say pharmacy students

Not appropriate to say ‘skewed to disagreement’ or ‘skewed positive’, something needs to be statistically significant or not

---We have removed the term skew given its statistical meaning.

If data is skewed in one direction it is not appropriate to calculate means, it should be median, or just report percent of participants with each response.

--We have removed mean and Std dev in favor of median and IQR.

Suggest in tables 2 and 3 to put the labels of 1-5 in the columns to make it easier to follow

--Added, thank you for the recommendation.

Table 4 – says if people answer yea then they answer either ‘1, 2, 3, 4+ times’. How is a mean calculated if just 4 or more? Or was this a free response to nominate any number?

--We have clarified in the methods and are presenting a median and IQR>

Again – unclear on what open-ended questions were about?

--Added to the methods.

Discussion:

I feel the discussion needs some reshaping as I don’t think the claims can be appropriately backed up by the data presented in this study. There is no pre-MHFA data so it is hard to say there are high levels of confidence in MHFA skills post training when there is no baseline data. Also concerns again about he validity of the knowledge/reluctance items.

--We have made changes to the discussion and point out in the limitations the lack of pre data.

It reports to be the first longitudinal evaluation of MHFA in pharmacy, but there is no baseline and it is unclear as to what the timeframe is that the surveys have been completed after training completion.

---We have calculated the timeframe of between 6 and 18 months since training and added that to the results.

Behaviors – again this should be used with care and referred to as ‘self-reported behaviors’

--- We have added this throughout.

Lines 217-220 – Lack of mental health services and confidence of respondents seem to be very different points? This paragraph needs some reworking

---We have moved this statement out and put with other open ended items in a subsequent paragraph, and added additional context.

The discussion lacks reference to a lot of work done in the space (e.g. Ashoorian et al 2018 Early Int Psych, Rodgers et al. 2019 Adv Ment Health, Boukouvalas et al. 2018 Soc Psych Psychiatr Epidemiol)

---Thank you for these references,. We have worked to incorporate,

Minor:

A number of language suggestions:

• Report numbers in words when less than 10 - changed

• ‘Trainings’ – suggest training programs - changed

• ‘Persons’ – people - changed

• ‘mental health condition’ – suggest mental illness not condition as condition is not a diagnosed illness - changed

• Use people living with a mental illness not patients. Mental illness can be very stigmatizing and people find it labelling to be called a ‘patient’. Changed most occurrences. Pharmacists may call their patrons patients, especially if they are in a service role.

---All adopted, thank you.

Reviewer #5: Evaluation of Mental Health First Aid training initiative for pharmacy

PLOS ONE

The present manuscript is a report of a survey evaluating the outcomes of pharmacy professionals trained in MHFA. Outcomes explored include reluctance and confidence as a result of MHFA, MHFA behaviors since training, and feedback on their MHFA training.

Abstract: Please clarify what pharmacy professionals mean.

--pharmacists, technicians, students, - clarified.

Abstract: Please indicate timing of the survey as it relates to the training.

---We do say that it was administered in May and June 2019 and the training sessions occurred in 2018, Changed during to throughout,

Abstract: Awkward wording “conducive” responses. Please consider rewording in this section and elsewhere.

---Conducive has been eliminated throughout.

Introduction: Is “gatekeeper training” language typical since this language seems somewhat new and may be confusing to readers? Please clarify this language early in the paper so readers are not confused with other references to gatekeepers in healthcare.

---We have introduced this term earlier in a revised introduction.

Introduction: It may be helpful if authors clarify that the first aid level of care in lines 46-47 are not intended to meet the gap of the disparity in access since first aiders are not trained to provide professional-level services but rather to catch more folks missed by the system due to the disparity and that need to get into the system then managed by trained professionals. It is important to clarify this point early since most readers would be confused as to how MHFA really is addressing the gap in access to services. In some ways, it could widen the gap since there could be more folks identified by MHFA that just won’t get the needed services because there is a lack of providers, etc. Please clarify and address.

---We have added a new paragraph about MHFA, but feel it is beyond the scope of this paper to raise the concern that making referrals will be a burden on the mental health care infrastructure.

Introduction: Authors need to give a brief synopsis about what MHFA is, what is covered by MHFA, and how it is similar or different from other trainings. Why evaluating MHFA vs. other trainings? It should be noted that NCBH is the organization supporting MHFA training/certification. Please add.

---Added

Introduction: There is a small literature involving pharmacists/pharmacy students and MHFA and was surprised it was not briefly presented- at least highlighting what we do know about MHFA from these studies. Please consider adding this section.

----We have added more pharmacy studies to the introduction.

Materials and Methods: It might be useful to clarify that the 5 pharmacists trained to conduct MHFA trainings were presumably trained through a week-long instructor training on how to lead MHFA trainings provided by the National Council for Behavioral Health (NCBH). Please add clarification.

---added

Materials and Methods: The authors are not completely accurate that the US version is an 8-hour training. It actually can be delivered in multiple versions of two 4-hour sessions, or four 2-hr sessions. It is not clear there is a “US version”. The authors probably meant that the version their 5 pharmacists completed was 8-hour but doesn’t mean the only version available. Please clarify.

---We have clarified that sessions were either 1 8 hour session or 2 4 hour sessions..

Materials and Methods: Please clarify when survey was administered with regards to when MHFA attendees attended their respective trainings. It would be good to note since it could be that survey was administered much further out in time from some trainings and not others. It would be useful to note if the later trainees had different perspectives from the earlier trainees. Please also indicate how many trainings were provided by each instructor? Is there any way to track responses to the trainer and see if a particular trainer had an impact vs. others? Please clarify.

----We have added that the survey was administered between 6 and 18 months following the trainings. We have decided not to do sub-analyses based on time since training because there would be too many confounding variables. Because the survey was anonymous, we cannot tie participant responses to an individual trainer. We have added to the limitations “Also, it was not possible to evaluate differences among trainers nor were adjustments made based on the gap between training and survey completion.”

Materials and Methods: The authors really need to provide more information about the knowledge, confidence, and use of skills items with greater details. Readers need to have a better sense of what these items measure- even examples of 2-3 items would be helpful. I would include some details in methods and refer to table for actual items. Reliability coefficients for scales should be in methods section. Please also state what types of descriptive statistics was used for which data. Please address.

---We have added example items and their corresponding concepts. We have included “Descriptive statistics were calculated for all scaled and multiple-choice responses.”

Materials and Methods: Is knowledge truly knowledge or attitudes? Authors used an attitudinal scale of agreement and items seem more attitudes than knowledge. Please justify items as knowledge vs. attitudes and explain how agreement scale is reflection of knowledge vs. attitudes. Knowledge is usually assessed more as nominal items of correct or not.

---We changed knowledge to reluctance which should address this..

Materials and Methods: Was there only one coder? Please clarify multiple coders and how discrepancies were resolved.

---Added description of coding and discussions to methods

Materials and Methods: Please indicate at end of methods IRB approval.

---Added at the beginning of methods.

Results: Table 1. Authors might want to collapse ages into fewer categories to save journal space. Also, not sure the need for P1-P4 student designations and likely lost on an international readership. It would be interesting to have collected prior experience with mental illness, etc. This might be an important to note as either a limitation or future direction. Some literature shows prior experience with mental illness either clinically or in personal lives affects attitudes and skills.

---year in pharmacy school deleted

Results: Tables 2 and 3. Please provide n (frequency) AND % for Strongly Disagree to Strongly Agree. Most tables like this abbreviate SD for Strongly Disagree, D= Disagree, etc. See all numbers as column headers and in rows is somewhat confusing and distracting. Authors might include medians as well. Please indicate total sample size in table headers.

We have changed to median and IQR. We have clarified the tables. Added sample size to table headers. Other reviewers have recommended just using % since the n is so close to 100.

Results: Table 4. Total sample size responding to this item should be included in table header.

---Done

Discussion: Please consider above comments as they relate to comments in this section. Additional limitations include: generalizability of findings to other settings, lack of validated measures, lack of a comparison group and baseline measures to discern differences across time, potential temporal issues between time of trainings and survey (memory recall biases), and inability to discern trainer effects on outcomes.

---We have added to the limitations

Discussion: Authors please consider existing literature on MFHA and other trainings and relate current finding/results to such prior works. Connecting the work and results with any conceptual frameworks related to the topic would be a plus and add value to this work.

---Added additional focus on Burnette RAND model

General comments: Authors might consider getting permission NCBH about publishing based on their program. It can’t hurt if not already done.

----We already have been in contact.

Discussion/Conclusions: Overall, these sections are generally well written with many excellent points brought up. It would been useful if the authors has a future directions section where they can propose are next steps/directions based on work and findings. They talked about future research possibilities at different points of the discussions but not as it own section with defined priorities and plans. Such a focused section based on findings and limitations will help readers see this as new program of research helping to establish the best scientific evidence for the effective use of MHFA in pharmacy practice.

----We have added future research section and removed from other places in the discussion.

This manuscript does offer some new information compared to prior publications. However, it is significant methodological limitations and could be strengthened in several areas that lack clarity and specificity. This reviewer wishes the authors the best as they pursue this work and seek publication.

---Thank you, we have aggregated the future research ideas into its own section following limitations.

Submitted filename: Response to reviewers.docx

Click here for additional data file.

14 Apr 2020

PONE-D-19-33742R1

Evaluation of participant reluctance, confidence, and self-reported behaviors since being trained in a pharmacy Mental Health First Aid initiative

PLOS ONE

Dear Dr Witry,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please consider the comments from reviewer 2 and acknowledge any limitations not already included and resubmit.

We would appreciate receiving your revised manuscript by 28 April 2020. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Andrew Soundy

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

Reviewer #5: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #5: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #5: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #5: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #5: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for your work to address the comments made by your reviewers. Your time, consideration of these requests and efforts are greatly appreciated. However, based on my concerns and concerns that were raised by at least two other reviewers regarding the validity of your survey tool/findings, I cannot recommend publication. While MHFA and other mental health roles are growing in community pharmacy practice and it is important to understand the impact of these trainings, I feel the analysis is too focused on self-reported post training and did not include a baseline assessment that hinders its findings. My greatest concern was that a validated survey tool was not used -- there are several validated attitude scales regarding suicide (ATTS, Understanding of Suicidal Patients Scale) that could have been used. Further, a qualitative framework could have been used (such as the Theory of Planned Behavior) if you felt that the validated surveys were not a good fit.

Additional comments:

1) LIne 71 -- suggest using language suggested by APhA community-based

2) Suggest including reference to the survey in your methods if intended to be included in your published article

3) Domains unclear (Line 120). Only mention demographics and then . --> suggest listing all domains and then moving into description of each

4) Time since training being 6-18 months. I feel the survey not being consistently conducted xx amount of time since being completed is a significant limitation as this could impact retention of concepts, how much they are using, etc

5) Relevance of data in lines 234-239 since the national behavorial council created the training and is concerned with fidelity and does not allow adjustments to training?

Reviewer #2: All comments have been appropriately addressed, appreciate author responses to reviewer recommendations.

Reviewer #5: I think the authors addressed my primary concerns. I will say there may be still some typos like line 130 "participants". I am still not entirely convinced how knowledge is reluctance but will accept their attempt here. I might have suggested that they just refer to reluctance and not view it as knowledge at all. Further, I didn't see anywhere why the focus on MHFA over other trainings like MHFA.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Carol A. Ott, PharmD, BCPP

Reviewer #5: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

14 Apr 2020

Please see attached response to reviewers document

Submitted filename: PLoS RR 4_14_2020.docx

Click here for additional data file.

20 Apr 2020

Evaluation of participant reluctance, confidence, and self-reported behaviors since being trained in a pharmacy Mental Health First Aid initiative

PONE-D-19-33742R2

Dear Dr. Witry,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Andrew Soundy

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

23 Apr 2020

PONE-D-19-33742R2

Evaluation of participant reluctance, confidence, and self-reported behaviors since being trained in a pharmacy Mental Health First Aid initiative

Dear Dr. Witry:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Andrew Soundy

Academic Editor

PLOS ONE