Exploring Mental Health Providers' Interest in Using Web and Mobile-Based Tools in their Practices.

A growing number of Internet sites and mobile applications are being developed intended for use in clinical practice. However, during the development process (e.g., creating features and determining use cases), the needs and interests of providers are often overlooked. We explored providers' interests using a mixed-methods approach incorporating both qualitative and quantitative research methods. A first study used an interview approach to identify the challenges providers faced, tools they used, and any use of computers and apps specifically. Fifteen providers from both the United States and Canada completed the interview and recordings were transcribed and analyzed using a constructivist grounded theory approach. Four primary themes were identified including challenges, potential tools, access and usability. A second study used a brief survey completed by 132 providers at a large healthcare system to explore current use of and potential interest in Internet and mobile technologies. Although many providers (80.9%) reported recommending some form of technology to patients, this was mostly Internet websites that were predominantly informational/psychoeducational in nature. Overall, these studies combine to suggest a strong interest in websites and apps for use in clinical settings while highlighting potential areas (ease of use, patient security and privacy) that should be considered in the design and deployment of these tools.

Introduction

Healthcare is being revolutionized by the rapid development and expanding use of digital health tools. These tools include a diverse repertoire of resources such as information storage and access (e.g., patient portals and electronic medical records), communication (e.g., e-mail, text messaging, and video conferencing), and Internet websites, mobile apps, wearables and sensors aimed to promote behavior change. These tools occupy various places within healthcare systems. Some are patient facing (e.g., self-help websites or self-management apps), others are provider facing (e.g., electronic medical records or clinician support tools), while still others help bridge patient-provider communication (e.g., technology-mediated communication or supported interventions). Given this, various stakeholders are involved in the design, development, funding, deployment, and use of these tools (Schueller et al., 2014). The needs of each of these stakeholders can and should be considered during the process of developing these tools and the services that surround them to ensure successful uptake, use and impact (Wu and Wang, 2005).

For the current investigation, we focus on a specific subset of stakeholders, providers of mental health services. Providers are key stakeholders because they are both end users of these tools and because they are gatekeepers to clinical knowledge whom patients rely on for opinions about clinical resources (East and Havard, 2015). A recent report found over 165,000 health apps were available in public app marketplaces with 29% of disease specific apps targeting mental health (IMS Institute, 2015). The adoption of these apps, however, is quite low, with only 36 apps accounting for nearly half of all downloads. The adoption of apps by patients greatly improves when “prescribed” by providers. Mental health apps, in fact, enjoy the highest “fill rate” (i.e., the rate at which patients download apps that their provider recommends) at 72%, when compared to 55% for medication apps or 48% for fitness apps (IMS Institute, 2015). Thus, understanding providers' interest, including their needs and concerns, is imperative to getting these tools in the hands of patients.

Understanding the perspectives and needs of end users is a common practice in user-centered (Norman and Draper, 1986) and participatory design approaches (Schuler and Namioka, 1993), which have become extremely influential in the process of creating software products (Muller, 2003). In light of these approaches, design work usually begins with a user needs analysis that involves characterizing the end users, understanding their goals and activities, identifying common situations, and appreciating their requirements and preferences (Booth, 1989). Design work is becoming increasingly common in the mental health space as clinical researchers are adopting techniques such as user-centered design (Bruns et al., 2015, Kelders et al., 2013) and usability testing (Vilardaga et al., 2015) into the development process. Specific recommendations for engaging the relevant stakeholders for mental health practice, such as the patient-clinician-designer framework, provide clear recommendations about how to deal with areas specific to this domain such as recognizing the different evaluations goals of each group (Marcu et al., 2011). More work in this vein, especially from a formative approach, could help provide specific recommendations about what providers want when it comes to technologies to enhance their clinical practice.

Much of the research focused on understanding the capacities, needs, and wants of end users has focused on patients. Extant studies have shown that mental health outpatients have the relevant technological infrastructure to access Internet and mobile interventions. Cell phone ownership in this population meets or exceeds the national average (Campbell et al., 2014, Torous et al., 2014a, Torous et al., 2014b). Many of these phones are smartphones (Torous et al., 2014a, Torous et al., 2014b) although mental health outpatients might be slightly more likely to share phones than the general population (Campbell et al., 2014). Patients also report a willingness to use smartphone apps and texting to promote their treatment and mental health (Campbell et al., 2014, Torous et al., 2014a, Torous et al., 2014b) and text messaging interventions are successful and well-liked (Aguilera and Muñoz, 2011).

Provider input on design may also be critical to ensure adoption; however, limited research is available. A recent study aimed to design a program to help implement an online technological platform into existing treatment resources for perinatal depression and anxiety for a given healthcare system (Baumel and Schueller, 2016). In this study, providers were presented with the online platform and interviewed as to how this platform could complement existing treatment and to identify potential problems with using this platform. Providers emphasized the need to train users of the platform – both those providing support and those receiving support – to ensure quality and safety of care. Furthermore, providers outlined several other necessary safeguards to protect patient safety including levels of acuity for which such care would be inappropriate and providing information that the platform was not appropriate for emergency support or crisis situations. Thus, this study suggests that quality and safety are two important considerations in the design of such systems; considerations that may have been missed if the focus had only been on patients and not providers. Although these providers gave a wealth of information through an in-depth interview, the generalizability of these findings are limited in that it was a small group of providers (five) from a single healthcare system.

In light of this, we were interested in exploring providers' attitudes and interests in using technology in clinical treatment using a mixed-methods approach – both qualitative and quantitative data – in diverse populations. This investigation consists of two studies. The first study was a sample of ‘front-line’ clinical providers based in the United States and Canada that provided qualitative data as to how applications could address the clinical challenges they encounter. Qualitative methods are useful because they provide detailed information about why a provider might be interested, or even opposed to using technology and highlight key themes to be addressed. The second study was a survey of mental health providers in a large healthcare system. This study produced quantitative information regarding the prevalence of certain attitudes and interests and helps reinforce themes identified through the qualitative study.

Study 1 — interview study

Participants and procedures

Participants were (N = 15) mental health providers with a primary position in an outpatient clinical setting that was unaffiliated with a major health network or hospital system. Such providers were selected because they themselves determine what tools to use in clinical care and are responsible for integrating these tools into their practice. Providers in major health networks are more likely to be involved in national rollouts of specific treatment protocols, receive directives to use specific clinical practices, and have access to a dedicated infrastructure that directs the use of technology (e.g., EMR).

A reputational case sampling approach was used to identify providers (Miles and Huberman, 1994). This approach uses the recommendations of key stakeholders to identify participants. For the current study, directors of nationally accredited training programs were identified as key stakeholders. These individuals were selected based on their ability to identify high quality therapists because of their access to 1) prior graduates of their program and 2) therapists in their area. Inclusion criteria included having greater than 20 h of patient contact scheduled per week, having been in practice at least 2 years since obtaining their license, and being in a community or private practice. Training directors were e-mailed and asked to share with their alumni. Prospective participants completed an internet-based screener to determine inclusion/exclusion criteria and gather descriptive information. A total of 27 providers completed the survey and 15 agreed to be contacted for an interview. The remaining participants could not be reached or declined to participate for a response rate of 56%.

Descriptive information for the obtained sample of N = 15 individuals is provided in Table 1. The sample was predominantly female (86.7%), with an average age of 40.6 (SD = 8.59). Of note is that the sample had an average of 25.73 h of patient contact per week and spent an average of 10.73 h completing auxiliary support work including notes, billing, paperwork, and contacting other providers to coordinate care per week. Twelve respondents identified themselves as providing cognitive-behavioral therapy, 3 as behavior therapy. No providers reported using apps currently in their practice.

Table 1

Descriptive statistics for sample in Study 1.

	N	%
Female	13	86.7
Degree
MSW	1	6.7
Clinical Psychology PhD	10	66.7
RN	1	6.7
Other	3	20.0
	M	SD
Age	40.6	8.59
Numbers of years licensed	9.67	5.81
Hours of patients scheduled per week	27.20	13.77
Hours of patient contact per week	25.73	6.10
Auxiliary hours per week	10.73	5.13
% of clinical time adhered to EBP	81.07	18.47

Interviews were conducted via telephone given that participants were located across the United States (n = 14) and Canada (n = 1) and were audio recorded. Qualitative interviews consisted of 6 questions and were designed to build discussion about challenges faced in clinical practice, tools used in clinical practices, and use of technology in clinical practice broadly. All questions were open-ended with follow-up questions asked as needed to probe specific answers. Mean interview time was 28.02 (SD = 5.29) minutes. Interviews were transcribed for coding.

Data analysis

Coding used a constructivist grounded theory approach (Mills et al., 2006). Two coders, a clinical psychologist and a trained bachelor's level research assistant, reviewed all interviews three times individually. The first review involved reading and listening to all interviews for thematic content. Through several discussions, the coders then identified primary themes that were used to create an initial codebook to guide specific coding. The interviews were reviewed a second time using line-by-line coding. Codes were created according to the preliminary themes identified in the codebook. The coders meet weekly during this process to discuss the development of new codes, revisions of themes, and issues related to assigning codes. After the initial coding was completed for all interviews, the coders reviewed the codebook and made revisions to the listed codes and themes. Using the revised codebook, the coders reviewed the interviews a third time and adjusted their codes accordingly. Coding was then compared across coders and disagreements were resolved until complete agreement was reached. The coders then reviewed all of the interviews a final time together to ensure that all codes and themes were appropriately applied.

Results

Four primary themes were identified that included: challenges providers faced in treatment, proposed tools that would facilitate the delivery of treatment, issues related to access of mobile applications (security, privacy, costs), and usability issues of applications in treatment (Table 2).

Table 2

Themes identified in Study 1 and exemplar quotes.

Theme	%
Challenges of treatment: Patients have low adherence to treatment activities.“I know a lot of people just aren't going to do the homework.”“They might be attending sessions, but they might fully be doing the work behind the sessions.”	93
Challenges of treatment: Poor attendance.“Not showing up.”“People can't get to session due to lack of transportation.”	93
Challenges of treatment: Complex presentations.“When one [patient] looks like they'd be a good fit for manualized therapy with one primary disorder. But it happens that as you get into it a lot of other stuff is going on and you have to make it cognitive therapy which doesn't necessarily fit the protocol.”	86
Proposed tools: Remote monitoring“I think if we could do something like that for anxiety, like track your panic attacks, when they happen and why they happen and what kind of symptoms you have.”	53
Proposed tools: Method to parse literature“If there were a place that I felt like concrete advances in empirically-based treatments. You know, some kind of listserv or website or journal or whatever where that kind of information were presented in fairly concrete terms such that it was easy to glean.”	40
Proposed tools: Supervision“I would really find helpful regular supervision.”“There are so many great trainings that are so expensive or you know a lot of them we can get paid for because of CE credit stuff but if there were some way to access you know some of this stuff is online.”	33
Security and privacy concerns“I guess my only concern would be confidentiality. Making sure that it were a secure enough thing that if I were going to do clinical work, it would be safe.”“Well it would depend on what information is contained in the mobile app. Because if it was like a confidentiality thing, then I wouldn't want to be walking around with people's identifying information on a mobile application.”	66
Usability“It would have to be user friendly. So easy to use.”“I would want to know that it has some added value. For that research has shown that it has potential to have some added value beyond what I am already doing.”	66

Challenges that providers face in treatment

Consensus on key challenges emerged as a majority of the sample identified similar themes regarding the provision of services. Nearly the entire sample (93%) identified patient lack of adherence as a primary barrier to successful therapy. Several providers noted low levels of homework compliance. Others commented that some patients are unmotivated or unwilling to engage in specific activities that are part of treatment. Difficulty with session attendance was also highlighted by nearly all interviewers (93%). Interestingly, poor attendance was primarily attributed to the patient's limited resources such as not having the means for transportation. These factors often co-occurred within the same cases.

Another commonly identified challenge (86%) was the comorbidity and complexity of the presenting problem. The providers suggested that most patients did not conform to a single diagnosis and thus it was challenging to select an appropriate evidence-based treatment protocol. Another suggested many patients have symptom profiles that change during the course of treatment such that an initial treatment strategy no longer fit and substantial changes to the treatment plan were needed. This issue made the delivery of evidence-based treatments difficult. Finally, approximately half (53%) of the providers identified that many of their patients lacked support outside of the therapy office, which they hypothesized contributed to many of their ongoing difficulties.

Proposed tools to facilitate treatment

Providers identified several different tools that they felt would improve their ability to provide care. Approximately half of the providers (53%) identified a need to monitor the behavior or symptoms of their patients remotely. The providers proposed that remote monitoring tools would improve their assessment of a patient's presenting symptoms by knowing what specific symptoms were present during a challenging experience. For example, it could allow for the gathering of important contextual information as to what may incite a panic attack. Such knowledge could allow them to tailor treatment to the specific patient. Remote monitoring could also improve accountability of the patient by determining if and when specific treatment related tasks were completed.

A second tool identified by a substantial portion of the sample (40%) was a method to efficiently parse the scientific literature. Several providers reported having limited access to journals and scientific databases. Additionally, they stated they had limited time to review new studies that often present complex or contradictory findings. However, all cited a strong desire to remain current with the empirical literature. A tool that allowed the providers to efficiently navigate the current scientific literature for a given patient or remain current with new techniques was identified as having the potential to make a substantial impact on practice.

A third of the sample (33%) stated that they would like technology-mediated supervision. They reported that it is difficult to obtain supervision or consultation from colleagues in the field due to costs and time limitations. A method that would allow the providers to interact with one another was identified as a potentially powerful tool.

Access of mobile applications

The major concerns regarding access related to security, privacy and cost. A majority (66%) of the sample cited security and privacy as a challenge that would limit their use in care. Providers wanted assurances that patient information would be protected and secure when communicating via a mobile application. There was a clear consensus that applications that could not make such guarantees would not be used. Cost was another issue that several providers raised as a barrier to access (60%). Several stated that their patient population had limited access to mobile phones or would not have the means to purchase an app. Although providers reported a wide range in how much patients should pay for an app (from free to $100), the modal recommended price was $1. This modal cost is consistent with other app costs.

Usability

The final theme related to the usability apps in treatment. The providers expressed concern that applications would interfere with treatment if they were hard to use (66%). None of the providers felt they were capable of providing advanced technical assistance on using an app nor did they want to spend session time going through an involved tutorial. Relatedly, a majority of providers (66%) stated that any new technology should improve efficiency in their practice and replace current tasks as opposed to increasing their workload. Providers already spend a considerable amount of time completing auxiliary tasks and were unwilling to take on additional work. That is, providers wanted technology to improve efficacy and replace cumbersome tasks rather than create new ones. A majority of providers felt that mobile applications should add value to their clinical practice (53%) by giving them something they could not otherwise obtain with minimal to no increase in workload. Finally, a majority emphasized that clear empirical support for any tool was an important consideration in their willingness to adopt the strategy (53%).

Conclusions

The qualitative study provided useful information into solutions that are likely to increase the adoption of technological tools by mental health providers. Clear consensus was reached on several topics. Providers were most interested in solutions that would extend their reach beyond the therapy room. Specifically, tools that would allow them monitor behavior or promote adherence to homework were most likely to be adopted, which addressed common clinical challenges. Furthermore, security, privacy, and cost were all major considerations. These providers identified several potential tools that could add value to their clinical practice. Some were patient facing such as remote monitoring. Interestingly, others were provider-focused, such as additional tools to improving access to the empirical literature and colleagues. This finding speaks to the need for improved dissemination efforts that are brief and easily accessed by those who are in clinical settings.

The providers also gave several insights into what would increase their adoption of future technologies. Providers have numerous demands as shown by the weekly number of patient contact hours and auxiliary hours. Technology solutions must be easy to use and highly intuitive such that they do not place additional burden on providers. Usability studies and consultation with colleagues in fields of engineering and computer science are integral to new software development to ensure that a usability threshold is met. Applications that have poor usability are unlikely to be adopted universally or endorsed by providers. It is worth noting that given the recruitment method for these providers (i.e., recommendations from training directors), they might be more strongly focused on delivering evidence-based practices than what is typically found in routine clinical practice. Thus, we wanted to conduct additional research in a setting that might have broad generalizability for clinical providers.

Study 2 —survey study

For Study 2, we were interested in investigating whether similar patterns would emerge using a large sample. We opted to use quantitative rather than qualitative methods for this purpose, as it would allow for more data, albeit less detailed, from a wider number of participants. We recruited providers from a single healthcare setting because although they might have similarity in terms of their background and practice, these biases would likely not influence their interest in providing technology (at the time there were no widespread efforts to use technology at this healthcare setting).

A survey was created and e-mailed to the listserv of providers within a large healthcare system (approximate n = 200 providers). Within a 4-week period, 132 providers responded to the survey (~ 66% response rate). Respondents were entered into a drawing for a gift card. Descriptive statistics are displayed in Table 3. The sample was predominantly female (79.5%), with an average age of 41.3 (SD = 11.72). Additionally, providers tended to be cognitive-behavioral in orientation (n = 69, 52.2%), with 22 endorsing eclectic (16.7%), 19 other (14.4%), 10 family systems (7.6%), 4 humanistic/experiential (3.0%), and 3 psychodynamic (2.7%).

Table 3

Descriptive statistics for sample in Study 2.

	n	%
Female	105	79.5
Degree
Masters-level	98	74.2
PhD or PsyD	13	9.8
Other	19	14.4
	M	SD
Age	41.3	11.72
Years in practice	10.0	8.80
Hours of patient contact per week	28.45	14.80

Survey measure

The survey was a brief measure consisting of 35 questions taking participants approximately 10–15 min to complete. In addition to demographic questions, the survey contained questions about providers' access and use of technology in general as well as in their clinical practice. Most of these questions required either a yes/no, Likert-type, or multiple option response, however three free response questions asked participants to report their “biggest excitements about using technology in [their] practice”, “biggest concerns about using technology in [their] practice”, and any additional feedback or comments.

Responses are presented below as percentages. For yes/no questions, we report the percentage of respondents indicating yes. For multiple option response questions, we note that the question was posed allowing for multiple answers and report percentage of respondents endorsing each answer choice. For Likert-type responses, participants responded on a 4-point scale consisting of “not at all interested”, “somewhat interested”, “very interested”, and “extremely interested”. For these questions we present the percentage of respondents who endorsed either “very interested” or “extremely interested” as these indicate a high degree of agreement with the statement. Lastly, free response answers were coded and collapsed to identify key themes or common responses in these items. For some questions, participants were asked to identify examples to reinforce their initial response and these examples were treated similarly to the free response questions.

Responses to this survey are displayed in detail in Table 4. Providers appear quite comfortable in using technology to communicate with patients. We surveyed both providers' current technological communication methods and their interest in using these methods. The largest discrepancy between current use and interest were for Internet sites (3.7% vs. 24.5%) and mobile apps (0.9% vs. 25.4%) suggesting a need to develop such resources. A majority of providers (80.9%) reported that they recommend some form of web or mobile resources to their patients. Most of the Internet sites recommended (47.2%), however, were of professional organizations (e.g., NAMI, NIMH, OCF), and not for treatment facilitation. Other categories included medical sites (9.4%; e.g., WebMD, MayoClinic, drugs.com, etc.), social Services (7.5%, e.g., housing, social security, etc.), support groups (7.5%, e.g., AA, NA, 12 Step), general psychoeducation (6.6%), general wellness (6.6%, e.g., fitness/diet, cooking), general web resources (6.6%), others practices (5.6%), and 2.8% who referred specifically to DBT-selfhelp.com. In line with the theoretical orientation of the providers, the largest category of mobile apps were CBT-based (20.8%), 17.7% were relaxation apps and the remaining a mixture including meditation, mood tracking, DBT, exposure, guided imagery, music, social stories, blood alcohol calculators, diet/fitness, time management, and sleep. Some of the particular apps named included iCBT, Mood Kit, Sparkpeople, iCouch, and LoseIt.

Table 4

Providers use and interest in web and mobile resources and tools (N = 132).

	Currently used	Interested in using
Modes of communication
e-Mail	69.2%	69.3%
Text messaging	27.1%	37.7%
Internet sites	3.7%	24.5%
Mobile apps	0.9%	25.4%
Recommended in practice
Internet sites	77.2%
Mobile apps	19.0%
Both	80.9%
Information interested in receiving
Scheduling requests	64.1%
Patient questions or concerns	58.9%
Treatment progress	53.8%
Homework completion	52.9%
Medication adherence	35.9%
Other	8.5%
Features of interest
Internet site providing lessons	61.9%
Internet site providing tools	60.2%
Mobile app providing lessons	59.7%
Mobile app providing tools	56.8%
Internet site tracking symptoms with patient feedback	50.4%
Internet site tracking symptoms with provider feedback	50.0%
Mobile app tracking symptoms with provider feedback	47.0%
Mobile app tracking symptoms with patient feedback	42.8%
Text message tips	25.2%
Text message symptom tracking	24.3%
Reasons for using Internet sites or mobile apps
Convenience	75.0%
Speed	46.2%
Cost	17.8%
Other	20.0%
Reasons against using Internet sites or mobile apps
Prefer to limit contact to face-to-face visits	46.7%
No need for these tools	21.1%
Information from these tools is unreliable	18.3%
No interest in these tools	7.3%
Other	29.3%

We then asked providers about the types of tasks they would want to accomplish using technological tools. The most common response was scheduling requests. Eight and a half percent of respondents endorsed “other” although all but one of these respondents indicated that they would not want to receive any information through technological tools. Thus, although overall it appears a large number of providers would be interested in these tools; a considerably minority report that they would not use these tools in their practice.

We next presented a series of ten features that could be available via Internet sites, mobile apps, or text messaging. In general, Internet sites appeared slightly more popular than mobile apps with a majority of providers being very or extremely interested in an Internet site that helped patients understand and manage their symptoms through the provision of lessons. A similar pattern was found for symptom tracking and feedback. Internet sites had slightly higher rates of endorsement than mobile apps. Text message tips and check-ins had the lowest levels of support.

The next set of questions addressed why providers would or would not use technological tools, the tasks they would most be interested having technological tools to support, and the features they would be interested in having included in these technological tools. Free responses questions presented several reasons why providers would or would not use these tools. The most popular reason providers would want to use tools was convenience. Twenty percent of respondents indicated some other concern. Although these other responses represented a variety of interests, a few providers (n = 4) noted an interest in using a form of communication and resources that are popular with patients. Thus, at least a few providers note they believe their patients want these tools. The most common reason providers would avoid using such tools was a desire to limit their connections with patients to face-to-face visits. The “other” response was endorsed by 29.3% of respondents with most of these responses related and security. In fact, privacy and security was the third most endorsed concern overall (20.1%).

For the free response questions, 101 providers completed the question regarding their biggest excitement about using technology in their practice. The most common responses related to allowing the use of treatment skills and strategies in the real-world, outside of session (27.8%) and convenience (27.8%). One-hundred and four providers completed the question regarding their biggest concern. Again, security/privacy/confidentiality was the largest concern (51.0%), other noteworthy concerns were reducing face-to-face communication (15.4%), overuse by patients and boundary crossing (6.7%) and the time involved in using technology (6.7%).

Although a majority of providers reported recommending Internet websites and mobile apps to patients, these websites and apps do not seem to offer the features and possibilities that providers are interested in. In fact, providers appear to want convenient resources to help support patients to apply skills and lessons learned in the therapy room into the real world. Security and privacy is a major concern of providers and managing data security and privacy should be a major concern of technologies hoping to be integrated into clinical care. Surprisingly, given their success at delivering efficacious interventions, text messaging was rated noticeably lower than comparable features on either Internet sites or mobile apps. It could be that providers believe that text messaging would require more work, but additional research would be needed to confirm whether that were the case.

Discussion

Taken together, the results of these studies suggest that although providers report a high level of interest in using websites and mobile apps to support mental health treatment, very few providers are doing so. Many providers do report recommending some form of website, however, these are largely informational and not designed to actively support treatment. This gap is most apparent with regards to mobile apps. In our first study, no providers reported using an app in their practice. In our second study, although one in five providers reported recommending apps, nearly no one reported using an app that facilitate communication, a feature that providers reported wanting. Collectively, these studies identify critical provider concerns that serve as barriers to implementation, as well as providers' perceived needs that could influence the design of future tools and services that make use of them. As key stakeholders in the process of moving technological tools into practice, addressing these concerns and needs are important to ensure that tools eventually reach the hands of patients.

Security and privacy emerged as major concerns. Security and privacy are complex because they require understanding ethical and legal obligations as a healthcare professional, an evolving regulatory environment with regard to technological communications, and evolving competency and acceptability from both providers and patients. As such, it is important to keep track of relevant legislation such as the Health Insurance Portability and Accountability Act (HIPAA, Health Information Technology for Economic and Clinical Health Act (HITECH), and other laws impacting web or mobile technologies (Hughes & Goldstein, 2015). It appears, however, that provider liability is only one piece of security and privacy concerns. Another factor is that security and privacy are necessary to make patients feel safe to disclose information. If providers are unable to trust information coming from technologies, they are unlikely to find these technologies useful for their clinical practice.

Another concern is how websites and apps will fit into providers' workflows. Technological resources will be adopted if they make providers' lives easier and enhance their ability to provide high-quality clinical services. As such, apps must be integrated into providers' workflow. For example, Google Calendar enjoys widespread adoption when it can ease the process of coordinating scheduling through features such as invites, shared calendars, and reminders. Therefore, it serves as not just another thing to do (enter an event in one's Google Calendar) but simplifies other processes in one's life. Similarly, websites and apps need to simplify providers' workflows. In these studies, providers offered several ideas for ways technologies could add value to their practice: increasing patient adherence and helping providers implement evidence-based practices. We found preferences for structured tools over text messages as websites and apps that provide structure and content with minimal provider involvement can be helpful in easing provider burden. It is worth noting, however, that this does not remove all burden of involvement from providers. Technological tools are used more often and result in better clinical outcomes when providers remain involved and aware of how patients are using these tools (e.g., Hilvert-Bruce et al., 2012, Mohr et al., 2011). Thus, technological tools should be developed in ways to facilitate this involvement while ensuring that providers can make efficient use of their time.

Providers appear most interest in tools that allow patients to receive clinical skills and strategies outside of scheduled sessions. Providers realize that patients and their mental health conditions are complex. People are likely to have multiple mental health needs (Kessler et al., 2005) and its unlikely all of these needs can be addressed during each session. Furthermore, additional information or skills might be useful for a given patient that a provider does not have time to cover. Internet sites and mobile apps can help fill in some of these gaps (Price et al., 2014). However, tools and resources need to be easy to use and find so that providers do not have to search for them and patients will have consistent, easy access to them.

Another unmet need identified in this study was additional tools aimed at improving access to the empirical literature and colleagues. Such tools exist, such as PracticeWise, which provides a searchable database of randomized controlled trials of treatments for mental disorders among children and adolescents, as well as tools for using practice elements from these trials to create individualized, modular-based treatments for specific clients (Chorpita et al., 2013, Chorpita et al., 2014). Unfortunately, PracticeWise was not listed by any of the providers in this study, and therefore may not be well recognized as a resource for individual providers. Further, no comparable collection exists in the area of adult mental health treatment and its development could be of considerable benefit to mental health care providers.

In the second study, we found a slight preference for Internet sites over mobile apps. In our experience working with these clinics, this is often driven by a belief on the part of providers that patients do not own mobile phones or would not want to use phones for mental health purposes. This is in stark comparison to broad trends in smartphone ownership (Smith, 2015) as well as in psychiatric outpatient populations (Campbell et al., 2014, Torous et al., 2014a, Torous et al., 2014b). More work, however, needs to address if provider beliefs impact the types of tools they report interest in or would be willing to recommend to patients. Another possibility is that this is due to the age of our sample. In both studies, the average age of the respondent was approximately 40 years, and therefore it could be that younger clinicians are more comfortable and familiar with mobile apps.

We should acknowledge a few limitations with this study. First, Study 1 was based on a small sample of 15 participants and Study 2, albeit larger in sample size, drew all participants from a single healthcare organization. It is unclear, therefore, how these findings would generalize to providers in other care settings. These providers might not be representative of the larger population of mental health providers. For example, these providers overwhelmingly supported evidence-based practices and cognitive-behavioral therapy. In defense of the external validity of this study to different contexts it is worth noting, that participants in the second study drew from multiple care contexts: acute care, outpatient group practice, a community mental health center, and school-based mental health providers. This might lessen this limitation and help indicate broad support for bringing more instances of technology into clinical practice. Most of the respondents came from the United States, although technological tools for mental health have enjoyed much wider deployment in other countries (e.g., Kenter et al., 2015, Titov et al., 2015). The diverse samples and methodologies used across Studies 1 and 2 combined with the similarity in findings across these studies lends support to the generalizability of conclusions drawn from our results. An additional approach could have been to give providers more context about the types of tools we were asking about by providing examples possibly even screenshots or existing websites or mobile apps. We decided, however, to engage in formative work to uncover areas of needs that could hopefully influence subsequent development of Internet sites and mobile apps. We should note that although our findings address which tools providers are currently using; it does not address why alternative technologies are not being used. This could represent a lack of knowledge of available resources or a reluctance to use them. However, it did appear from the features and capabilities that providers reported an interest in that if web and mobile resources could meet their needs, providers would be interested in using them.

Conclusions

Providers are unwilling to endorse technology-based resources unless the value added to clinical practice is clear. It is unlikely they will see value in these resources, unless we first strive to understand what they value and then build these resources to incorporate these values. This paper contributes to providers' viewpoint of perceived needs and barriers related to technological tools. Providers, however, are only one stakeholder with an interest in using these tools in clinical practice. Future work should also explore other stakeholders (e.g., patients, payers, developers) and engage in processes that bring together these stakeholders during processes of creation, deployment, and evaluation. Nevertheless, this paper provides clear guidance as to the types of tools providers might want and concerns that need to be addressed within these tools.