Challenges of Building, Deploying, and Using AI-Enabled Telepsychiatry Platforms for Clinical Practice Among Urban Indians: A Qualitative Study.

BACKGROUND: Published literature shows the overall challenges associated with artificial intelligence (AI)-enabled medicine and telepsychiatry more from the western perspective, with no specific mention from the perspective of individual stakeholders or Indians. This study was conceptualized to understand the perceived challenges of building, deploying, and using AI-enabled telepsychiatry for clinical practice from the perspectives of psychiatrist, patients, and the technology experts (who build such services) in urban India.
METHODS: Between February 2020 and April 2020, a semistructured topic guide was drafted for qualitative exploratory study among psychiatrists (n = 14), their patients (n = 14), technology experts (n = 13), and Chief Executive Officers (CEOs) (n = 5) of health technology incubation centers. Interviews were conducted over the phone, recorded, and analyzed using the grounded theory approach.
RESULTS: Almost all respondents cited ethical, legal, accountability, and regulatory implications as challenges. The major issues stated by patients were privacy/confidentiality, ethical violations, security/ hacking, and data ownership. Psychiatrists cited lack of clinical validation, lack of established studies or trials, iatrogenic risk, and healthcare infrastructure issues as the main challenges. Technology experts stated data-related issues as the major challenge. The CEOs quoted the lack of interdisciplinary experts as one of the main challenges in building deployable AI-enabled telepsychiatry in India.
CONCLUSIONS: There are challenges to deploy an AI-enabled telepsychiatry platform in India. There is a need to constitute an interdisciplinary team to systematically address these challenges. Deployment of AI-enabled telepsychiatry is not possible without clinical validation and addressing current challenges.

Key Messages:

Deploying AI-enabled telepsychiatry has several ethical, legal, accountability, and regulatory problems along with issues related to privacy/confidentiality, security/ hacking, ownership of data, iatrogenic risk, and lack of interdisciplinary experts to build these solutions. These challenges must be systematically addressed in order to develop an explainable AI-enabled telepsychiatry platform in India.

Telepsychiatry helps in reducing the treatment gap in accessing quality mental health care.[1] Studies show that telepsychiatry consultations offer similar results as the conventional face-to-face therapy and have been touted as an alternative when traditional care is inaccessible.[2] In mental health, artificial intelligence (AI) can be used to enhance the decision-making abilities of mental health practitioners and help tailor personalized treatments for patients.[3] AI-based web and smartphone services are known to improve user experience and optimize personal mental health care services.[4] Harnessing big data in psychiatry is another utility of AI-based services.[5-10] In psychiatry, the use of deep learning has also reduced the need for large volumes of data otherwise required for disease prediction.[11]

Despite the potential opportunities in using AI-enabled telepsychiatry services, several challenges have been reported. Mental health care providers have raised concerns about therapeutic rapport with the patients and the inadequacy of handling psychiatric emergencies.[12] Studies report legal, ethical, and administrative barriers apart from lack of cost-effectiveness data that affect the large-scale implementation of AI-enabled telepsychiatry services for primary care.[13] Cross-cultural barriers affect the incorporation of existing telepsychiatry programs into the current health systems.[14] Apart from the specific issues related to AI-enabled telepsychiatry, there are general issues with AI in medicine, such as validation, privacy, security, biases, urban/rural gaps, language barriers, financial constraints, technical issues, training and professional issues, professionalism issues, and problems related to patient–provider relationships.[15-20] All these challenges have limited the use of AI and telepsychiatry for primary care.

Published literature shows the overall challenges associated with AI-enabled medicine and telepsychiatry more from the western perspective, with no specific mention from individual stakeholders’ perspectives. But, studies trying to understand perceived challenges must be exploratory in nature and employ qualitative methods. There are no studies that are qualitative and specifically focused on AI-enabled telepsychiatry from the perspective of individuals in India. Also, no studies addressed the challenges from a holistic perspective, including every stakeholder in context. This study was conceptualized to understand the perceived challenges of building, deploying, and using AI-enabled telepsychiatry for clinical practice, from the perspectives of psychiatrists, patients, and the technology experts who build such services.

Materials and Methods

The study followed the Consolidated Criteria for Reporting Qualitative Studies (COREQ) guidelines. In December 2019, we framed a set of research questions (Box 1) to study the perceived challenges in building, deploying, and using AI-enabled telepsychiatry for clinical practice in India. We sought the perspectives of psychiatrists, patients, technology experts who build such services, and CEOs of health technology incubation centers in India. An initial informal review of published literature showed that most of the studies were quantitative and systems perspective in nature. None of the studies focussed on the provider–user perspective or were from the point of view of the technology experts who actually build and use such AI-based telepsychiatry platforms. Also, most of the experiences in building the platform have been published in the Institute of Electrical and Electronic Engineers conferences, purely from the technology viewpoint, with fewer implications for psychiatrists. There is no real-time knowledge on the perception and attitudes of the users and providers too. Therefore, a qualitative exploratory study will help build basic knowledge of the user–provider–developer perspective on the challenges faced while building, deploying, and using AI-enabled telepsychiatry for clinical practice.

What are the problems/issues/challenges in using technology for the clinical practice of psychiatry?

What are the issues raised by a patient while using technology to access care for mental health? What are the challenges perceived by the patients while using telepsychiatry platforms? What are the suggestions and expectations of the patients for developing a feasible AI-enabled telepsychiatry model?

Are the psychiatrists comfortable with using technology for providing clinical care and support? What are the issues present in the present models that utilize AI and telepsychiatry? What is the reason that AI-enabled telepsychiatry is still not developed or deployed in large scale?

What are the issues faced by technology experts while trying to build, deploy, and validate AI-based telepsychiatry models for mental health? Why is that not many start-ups or companies focus on addressing mental health issues using AI-enabled telepsychiatry as much as they focus on other noncommunicable diseases?

Between February 2020 and April 2020, a semistructured topic guide was drafted for this qualitative exploratory study. The semistructured interview guide primarily focussed on the following three questions:

What do you think is the role of AI-enabled telepsychiatry in clinical practice today and in the future?

What do you think are the challenges in having AI-enabled telepsychiatry platforms for routine clinical care? What are the challenges you face while using such services?

How do you think the challenges can be overcome?

The guide was pilot tested in Chennai with a psychiatrist (who fulfilled the eligibility criteria but was not included as a part of the main research). These questions were reframed for each participant and iteratively revised to ask more specific questions as the study progressed. Institutional Ethics Committee approval was obtained.

The interviews were conducted by the corresponding author, who is a male medical doctor with more than 10 years of experience in social sciences and behavioral research using qualitative methods. He is currently serving as a scientist and faculty and in various leadership roles in government institutions and public and private organizations. The interviewer first contacted psychiatrists in Bangalore and Chennai from the first author’s network of psychiatry professionals, using the telephone. The study objectives and rationale were explained to them.

Psychiatrists who had used web-based/phone-based telemedicine services in the last six months (August 2019–January 2020) for consulting patients were selected. Only practitioners who were fluent in English and practicing in urban areas of Chennai, Mumbai, Bangalore, or Delhi were contacted. The reason for limiting to an urban population was to have maximum homogeneity. Including rural service care providers and patients lead to a lot of variation, due to India’s wide variety of rural cultural landscape. Such variations in sampling will affect the generalizability of the findings. Also, the penetration of such services in the rural areas is less and not uniform, affecting the sampling procedures.

Informed consent was obtained. First, a psychiatrist was contacted, a telephonic interview was conducted (on a different, convenient day/date/time), and the audio was recorded. Then, the psychiatrist was asked to refer any one of his patients to take an interview. This was done to ensure that we capture the perceptions of both a user and a provider of the same platform. The psychiatrists connected the research team with similar psychiatrists who were using telepsychiatry platforms (snowball sampling).

Technology experts in leadership positions who have at least one year of experience in building AI-based telepsychiatry platforms were contacted over the telephone. They had directly involved in developing, deploying, and validating AI-based psychiatry/telepsychiatry programs. They were selected from the open database of health technology start-ups of NASSCOM. English-speaking, consenting candidates were selected for telephonic interviews after obtaining informed consent. Audios were recorded. Technology experts were from Chennai, Bangalore, Hyderabad, Mumbai, and Delhi. We also contacted five CEOs of health technology incubation centers in India who have been actively involved in evaluating proposals from start-ups for funding and support. They were chosen to be interviewed because they understand the entrepreneurial landscape and have firsthand knowledge of different health technology applications and how they are being implemented in India.

The interviews were stopped after attaining data saturation. None of them refused or dropped out of the study.

The analysis was done by both the authors together. The recorded audios were analyzed using the grounded theory approach. The audio-recorded in-depth interviews were transcribed as extended notes and were initially explored using focused coding and constant comparative method.[21,22] A few emergent codes and categories were identified from the text and added to the existing codes gained from observational studies. Axial coding was done to develop connections between categories derived from all data. Themes were identified by looking for similarities, differences, and relationships between categories.[23]

Results

Table 1 shows the demographic characteristics of the psychiatrists and patients interviewed. Table 2 shows the demographic characteristics of the technology experts and the CEOs. The interviews were manually analyzed. Initially, codes were derived, which were converted to categories. Then, the categories were classified into themes. Table 3 shows the thematic framework and the frequency of the occurrence of the themes and categories across all respondents. The major recurrent theme was ethical, legal, accountability, and regulatory issues (182), followed by challenges related to data (128). Other themes were health system infrastructure (98), technology (78), knowledge gaps/deficit (69), AI-related (63), clinical practice (46), human resources and skills (31), and attitudes and perception toward AI (21). The following section demonstrates themes and categories along with suitable illustrative quotes (for significant themes).

Table 1.

Demographic Characteristics of the Psychiatrists and Patients

		Psychiatrists	Patients
Age		35.5 years (SD = 3.72)	29.72 years (SD = 4.91)
Sex		Males = 7Female = 7	Males = 9Female = 5
Residence	Chennai	6	6
	Bangalore	5	5
	Delhi	3	3
Years of experience as a psychiatrist		8.1 (SD = 2.17)
Years of experience using telepsychiatry platform/any AI-based application for providing service		2.34 (SD = 1.62)
Years as a patient			5.23 (SD = 2.18)
Years of experience using telepsychiatry platform/any AI-based application for taking service			1.98 (SD = 0.54)
Average duration of the interview		23 min (SD = 3 min)	20 min (SD = 2 min)

Table 2.

Demographic Characteristics of the Technology Experts and CEOs of Incubation Centers

		Technology Experts	CEOs
Age		42.16 years (SD = 5.27)	51.09 years (SD = 2.98)
Gender		Males = 11Females = 2	Males = 4Female = 1
Residence	Chennai	2	1
	Bangalore	7	1
	Delhi	2	1
	Mumbai	2	1
	Bhubaneshwar		1
Years of experience		9.15 years (SD = 1.07)	4.02 years (SD = 2.66)
Years of experience building telepsychiatry platform/any AI-based application for providing service		2.45 years (SD = 0.67)	Not applicable
Average duration of the interview		30 min (SD = 4 min)	45 min (SD = 7 min)

Table 3.

Summary of Categories and Themes and Their Frequency Respondents-wise

Themes (Frequency of the Theme)	Categories/Codes (Frequency of the Code)	Respondents (the Numbers Refer to the Frequency of Responses)
		Psychiatrists (n = 14)	Patients (n = 14)	Technology Experts (n = 13)	CEOs of Incubation Centers (n = 5)
Knowledge gaps/deficit (69)	Lack of clarity on problem statement (18)			13	5
	Lack of understanding of ground reality (15)	10			5
	Lack of studies/research (14)	14
	Lack of proof of principle/proof of concept (22)	4		13	5
Attitudes and perception (21)	Existential threat (2)				2
	Myth of being replaced (8)	5			3
	Lack of awareness among doctors (11)			8	3
Challenges related to data (128)	High predictors and low sample size (paucity of data) (15)			13	2
	Lack of common data schema and standards (13)			13
	Lack of data on the minority (19)	6		13
	Lack of established biomarkers (13)			13
	Emergence of new types of digital data (22)	9		13
	Differences in the data used for assessment (20)	9		13
	High volume of unstructured qualitative data (13)			13
	Missing values and inconsistencies (13)			13
Ethical, legal, accountability, and regulatory (182)	Lack of specific AI-enabled telepsychiatry guidelines (16)	5		6	5
	Privacy and confidentiality (46)	14	14	13	5
	Security and hacking (46)	14	14	13	5
	Ethical issues (28)	14	14
	Data ownership (28)	14	14
	Legal framework (18)			13	5
AI-related (63)	Black-box approach (18)			13	5
	Lack of clinical validation (16)	11			5
	Algorithmic bias (11)			6	5
	Requirement of a large amount of data to train AI (18)			13	5
Health system infrastructure (98)	Financial issues (27)	12	10		5
	Urban–rural differences (19)	14			5
	Cultural and language (19)	14			5
	Regional variation (21)	11		10
	Inequalities in healthcare access (12)	12
Human resources and skills (31)	Need for reskilling and training (8)	3			5
	Lack of interdisciplinary experts (23)	5		13	5
Technology (78)	Hardware and software (27)	4	5	13	5
	Maintenance (21)	3		13	5
	Connectivity (30)	8	4	13	5
Clinical practice (46)	Fear of flawed algorithm leading to iatrogenic risk (19)	14			5
	Problems in patient–provider relationship (17)	10	7
	Low computer literacy (26)	12	9		5

Ethical, Legal, Accountability, and Regulatory Issues

Privacy/Confidentiality

“I am not sure if whatever I speak with my doctor is not overheard by others or recorded…. Even now when I use video consultation, I hesitate to tell everything as I would otherwise tell my doctor in person….” (Patient, Bangalore, using telepsychiatry services for last eight months)

Security/Hacking

“There is always a possibility of hackers taking over telemedicine platforms and causing data theft. Apart from that, when there are security lapses, the possibility of stealing vital bank information from the mobile (that is used for accessing the mental health service) is also possible….” (Technology expert, Mumbai, two years’ experience in building a telemedicine platform)

Lack of Specific AI-Enabled Telepsychiatry Guidelines

“The telemedicine guidelines issued by the Indian Government is generic and does not specifically address issues related to telepsychiatry…also…the guidelines need changes when AI is integrated…these are yet to be regulated in the country….” (CEO, Health Technology Incubation Center, Mumbai)

Ethical Issues

“The extent of involvement of AI in telepsychiatry has a lot of ethical issues…like…how can we rely on AI for automated evaluation and diagnosis…who will be notified when there is a clinical diagnosis…if the patient is notified of something that causes suicide risk, who will regulate this…such questions raise serious ethical concerns” (Psychiatrist, Chennai, using telepsychiatry platform for the last one year)

Data Ownership

“A potential issue with telepsychiatry and AI is who will own this data? Does it belong to the patient alone? if it belongs to the patient, what is the level of access that can be granted to the developers or service providers?… these questions remain unanswered largely…these issues are common with other domains as well, like cardiology, ophthalmology, etc.…but clinical data in psychiatry is very sensitive and has several social implications as well…so the issue of data ownership must be sorted out before we embark on innovation” (CEO, Health Technology Incubation Center, Bangalore)

Legal Framework

“Several legal issues are yet to be clarified…for instance, if there is a misdiagnosis or missed diagnosis…who will the patient sue…. Doctor? Developer? Platform owners? Also, does indemnity insurance for doctors cover this particular aspect?… All these discrepancies in legal frameworks must be addressed before large-scale deployment of the telepsychiatry platforms integrated with AI….” (Psychiatrist, Delhi, using telepsychiatry platform for the last eight months)

Challenges Related to Data

Low-N, High-P (Paucity of Data) (High Predictors and Low Sample Size)

“We need voluminous data to build AI models…one problem with data in psychiatry is the low sample size and a high number of predictors…also, it is very hard to collect data when it comes to mental health…there are no open sources…this makes it very hard to build reliable algorithms….” (Technology expert, Chennai, 14 months’ experience in building a telemedicine platform)

Lack of Common Data Schema and Standards

“Any data used in technology has to follow certain schema and standards…though there are various standards for clinical data…specific standards for data related to mental health are yet to be formulated….”(CEO, Health Technology Incubation Center, Bhubaneshwar)

Lack of Data on Minority

“The data related to mental health is scarce in India…in our country where diverse minority groups constitute the majority of the population, we need specific data on such population to account for various sociodemographic differences…or else, we will end up trying to build a model with one size fits all approach….” (Psychiatrist, Chennai, using telepsychiatry platform for the last six months)

Lack of Established Biomarkers

“Diagnosis using AI primarily uses classification based on established biomarkers…biomarkers in psychiatry based on these data (voice, text, and images) are yet to be validated…unless we develop such biomarkers, we cannot build a robust model for diagnosis using automated evaluation….” (Technology expert, Chennai, 17 months’ experience in building a telemedicine platform)

Emergence of New Types of Digital Data

“Data in psychiatry have predominantly been self-reported questionnaires and clinical assessment using various tools…but with the digital penetration…, data is no longer just numbers…it is images, text, voice, social media [posts], etc…with this new types of data, there is a need for revised diagnostic criteria using digital data….” (Psychiatrist, Mumbai, using a telepsychiatry platform for the last seven months)

Differences in the Data Used for Assessment

“The data used for clinical decision making in psychiatry practice is very different from the data used for building AI models…in practice, it is predominantly clinical assessment that determines the diagnosis…whereas, in AI models, it is temporal qualitative data like voice, text, and images…we call it digital phenotyping…which requires a lot of research to be done…this affects the development of suitable models….” (CEO, Health Technology Incubation Center, Chennai)

High Volume of Unstructured Qualitative Data

“AI mainly uses quantitative structured data…whereas the data collected from patients is mainly qualitative and unstructured…patient-to-patient variation is also maximum…noise-to-signal ratio is misleading as it is difficult to ascertain which is noise and which is signal….” (Technology expert, Mumbai, 19 months’ experience in building a telemedicine platform)

Missing Values and Inconsistencies

“Another important issue with data is the number of missing values and inconsistencies when it comes to data related to psychiatry…it is hard to fill in those gaps…. We need either a proper data collection mechanism, or we might have to devise a novel approach to reduce such inconsistencies….” (Technology expert, Bangalore, two years’ experience in building a telemedicine platform)

Health System Infrastructure

Most of the psychiatrists reported financial issues (regarding payment and reimbursement) as an important issue. Other issues in the healthcare system, such as regional variations, urban–rural differences, language and cultural barriers, and inequalities in healthcare access, were reported as challenges.

Technology

Most of the respondents, including patients, stated that getting a good mobile phone for telepsychiatry services is a major challenge, as most of the psychiatric patients do not have such devices. Technology experts reported building software and constantly debugging it as a challenge. CEOs of the technology incubation centers stated the maintenance of the platforms as a major technology-related challenge.

The major technology-related issue reported by patients as well as psychiatrists was connectivity.

“All these devices, technology, AI, etc., require high-speed internet…. Majority of the hospitals do not have internet…also, synchronous video consultation requires a good device with high-speed internet…patients who have basic livelihood issues cannot afford a device or internet….” (Psychiatrist, Chennai, using a telepsychiatry platform for the last five months)

Knowledge Gaps/Deficit

Most of the psychiatrists stated that technology experts do not understand the ground reality in psychiatry.

“Most of the solutions they (technology experts) develop tend to have ethical, legal, or regulatory barriers. There is also a lack of well-established trials and studies to understand the applicability of AI-related technology. They build solutions with no real-time applications. Some of them are not feasible to be implemented at all at this point of time in India….”

(Psychiatrist, Delhi, using telepsychiatry platform for the last 13 months)

On the other hand, the technology experts stated that there is no clarity on the problem statement for which a solution needs to be developed.

“…any psychiatric disorder is a broad problem statement. Finding specific solutions is challenging unless clear end-points are given to us…we don’t understand what psychiatrists want to achieve through AI or any health technology…they have to clearly tell us what their goals are…like diagnosis, treatment adherence [monitoring], etc….”

(Technology expert, Mumbai, two years’ experience in building a telemedicine platform)

The CEOs highlighted lack of proof of principle/proof of concept.

“Any model requires a concrete proof of concept…or at least a proof of principle…simple hypothetical frameworks cannot be tested or validated…grants are required for large-scale testing for which good proof of concepts are a must….”

(CEO, Health Technology Incubation Center, Chennai)

AI-Related

Some of the AI-related issues reported were the black-box approach used in AI algorithms. This lack of transparency affects the credibility of the technology. Also, there is a lack of clinical validation of the algorithms used. Other issues are the requirement of a large amount of data to train AI and the algorithmic bias that can ensue when AI is trained with only one type of data.

Clinical Practice

The challenges in clinical practice are related to the computer literacy of the service provider and the patient. Also, patients and psychiatrists who prefer an in-person consultation feel the lacuna while using online services. Some psychiatrists stated that having a flawed algorithm leading to misdiagnosis or missed diagnosis may lead to an iatrogenic risk.

Human Resources and Skills

All the CEOs emphasized the need for reskilling and training mental health care workers to handle AI-enabled telepsychiatry platforms. All the CEOs and technology experts reported lack of interdisciplinary experts as a major challenge in the development and validation of AI in mental health.

“People who understand psychiatry do not understand technology, while none of the technology experts talk the language of the psychiatrists…this is a serious gap…this is why it is suggested that interdisciplinary experts are required who understand the language of both worlds and help bridge this knowledge gap….” (CEO, Health Technology Incubation Center, Bangalore)

Attitudes and Perception Toward AI

Other less relatively reported challenges are related to attitudes and perceptions toward AI. Some of the psychiatrists thought that AI will be an existential threat to humanity or may replace them in the near future. Another challenge is the general lack of awareness among doctors on what AI is and what it is not.

Discussion

This is the first study from India that analyzed the challenges of AI-enabled telepsychiatry from an interdisciplinary perspective using a qualitative approach, which is essential for evidence-based policymaking in healthcare.[24] In this study, the problems related to ethical, legal, accountability, and regulatory implications were cited as challenges by almost all respondents. The major issues stated by patients were privacy and confidentiality, ethical violations, security and hacking, and data ownership. Other issues, from the patients’ perspective, were related to procuring the hardware, installing software, lack of consistent internet connectivity, and low computer literacy. On the other hand, psychiatrists cited lack of clinical validation, lack of established studies or trials, and healthcare infrastructure issues as the main challenges. Improperly trained AI algorithm leading to misdiagnosis or missed diagnosis and the subsequent iatrogenic risk was another major concern for the psychiatrists.

Technology experts stated data-related issues as the major challenge. These include the lack of a standard schema for data related to psychiatry, inability to collect uniform and consistent data, low sample size but a high number of predictors (low-N, high-P), absence of data accounting standards in mental health, the emergence of new types of digital data (such as voice, social media, and images), differences in the assessment practices used clinically and using an AI, lack of established biomarkers in digital data, and a huge volume of unstructured qualitative data. Apart from the challenges related to data, all the technology experts quoted the lack of clarity on problem statements as a main reason for being unable to build an explainable AI model for psychiatry.

CEOs of health technology incubation centers quoted the lack of interdisciplinary experts as one of the main challenges in building deployable AI-enabled telepsychiatry in India. Also, there are no specific telemedicine guidelines for AI-enabled telepsychiatry.[25] Lack of proof of concept is another challenge cited by them. Besides, the existing workforce in mental healthcare must be reskilled along with training new staff to handle technology-enabled practice in clinical psychiatry.

Most of these challenges have been reported by several small-scale studies and reviews elsewhere;[26-35] but this is the first time that these challenges have been reported from the perspectives of various primary stakeholders, namely patients, psychiatrists, and technology experts. The added perspective of CEOs of health technology incubation centers makes the findings more robust. The firsthand knowledge gained in this study can be used for effectively building sustainable models in AI-enabled telepsychiatry. The study has the limitation of covering only urban Indians; but this is justified by the fact that most of the Indians who use telepsychiatry or AI-enabled services come from urban areas.

Maybe, in the future, if there are deployable AI-enabled telepsychiatry for rural areas, then we will be able to interview them and get a different insight. At this point, in India, it is not feasible to include the rural population to study the challenges in AI-enabled telepsychiatry.

There are challenges to deploy an AI-enabled telepsychiatry platform in India. There is a need to constitute an interdisciplinary team to systematically address these challenges.[36] Deployment of AI-enabled telepsychiatry is not possible without clinical validation and addressing current challenges.