Training nonspecialists in clinical evaluation for telepsychiatry using videoconferencing: A feasibility and effectiveness study.

AIMS: Videoconferencing-based telepsychiatry has been used successfully for the assessment and management of psychiatric disorders. However, training mental health professionals through videoconferencing has seldom been attempted. Online decision support systems for diagnosing psychiatric disorders had been developed earlier at the Postgraduate Institute of Medical Education and Research (PGIMER), Chandigarh, as a part of a project for delivering telepsychiatric services to remote areas. The feasibility of videoconferencing for training nonspecialist staff in the use of the online tool to diagnose psychiatric disorders was examined. The effectiveness of training was evaluated using ratings of diagnostic agreement between trainees and trainers and estimations of training costs.
METHODS: The Skype platform was used for videoconferences (VCs). Broadband internet connections had bandwidths of 4 mbps and speeds of 512 kbps. A total of 62 training sessions were conducted by the PGIMER team for remote-site teams using role-play techniques and actual patient interviews.
RESULTS: Videoconferencing-based training was considered to be convenient, satisfactory, and useful by all the participants. Diagnostic agreement between trainees and trainers was 89%-100%. Such training also appeared to be cost-effective. The main problems encountered were poor connectivity and poor audiovisual quality of the VCs.
CONCLUSIONS: Videoconferencing can be feasible and effective for training nonspecialists to diagnose psychiatric disorders. Copyright:

INTRODUCTION

Videoconferencing telepsychiatry has been established as an effective means for delivering mental health services. Telepsychiatric care is comparable to conventional in-person care across a range of outcomes including and diagnosis and assessment of psychiatric disorders, patient satisfaction, effectiveness of mental health-care, cost-effectiveness, and safety and security of patients.[12] However, compared to in-person services, the added advantages of telepsychiatric services include greater access for people in remote and underserved areas, reduced transportation and scheduling difficulties for patients, improved patient engagement, and lessened stigma of receiving care.

Telepsychiatric services are ideally suited for delivering of mental health care in low- and middle-income countries like India. The scarcity of trained personnel and mental health resources, and an unequal distribution of the available resources between urban and rural areas, has resulted in poor access to mental health services, particularly for rural population.[3] Telepsychiatry can be used to bridge this mental health gap in treatment. Telepsychiatric services can be used for providing consultation, supervision, and capacity building, thereby augmenting in-person care in India.[4] Efforts have been made to reduce the mental health gap in India under the aegis of the National Mental Health Programme by strengthening existing resources, developing new ones, and enhancing workforce.[35] Such efforts have included training mental health professionals in person using different formats. However, in-person training of health workers from remote areas by bringing them over to the tertiary care centers is expensive in terms of time and money and also impractical for reasons of travel and stay away from place of work. Many of these limitations are overcome by training through videoconferencing, which offers training at site without the need for relocation. Trainees can continue with their usual clinical routines and also avoid the costs of traveling. Videoconferencing-based training is better for maintaining continuity of training with the regular supervision and feedback. The application of training skills while seeing patients can also be improved with real-time videoconferencing sessions.

However, despite the sizeable body of research on the effectiveness of telepsychiatry, research on videoconferencing-based training is limited.[678] Nevertheless, such training appears to be no different from conventional in-person training. Moreover, professionals who receive training have more positive perceptions of telepsychiatric care and are more likely to use telepsychiatric services.

Therefore, this study examined the feasibility and effectiveness of training nonspecialist staff in diagnosing mental disorders. The study was a part of a project at the Postgraduate Institute of Medical Education and Research (PGIMER), Chandigarh, for delivering telepsychiatric services to remote areas. As a part of this project, an online tool for the diagnosis of common psychiatric disorders was developed. The feasibility of videoconferencing for the use of this online tool to diagnose psychiatric disorders was examined. The effectiveness of training was evaluated using ratings of diagnostic agreement between trainees and trainers and estimations of training costs.

MATERIALS AND METHODS

Approval and consent

The study was approved by the institute research and ethics committees. Written informed consent was obtained for all patients and family members before inclusion. Other ethical safeguards were maintained during the conduct of the study.

The online diagnostic interview

The computerized decision support system used in this study was a fully automated, net-based tool for diagnosing 18 common psychiatric disorders among adults and children in bilingual formats. The diagnostic part of the application had a screening submodule and more detailed diagnostic submodules that generated a final International Classification of Diseases-10 diagnosis. The diagnoses generated by the tool were found to be highly reliable, feasible, with sufficient sensitivity and specificity.[4]

Location

The nodal site was at the PGIMER, Chandigarh. The three remote sites were in district hospitals in Himachal Pradesh, Uttarakhand, and Jammu and Kashmir.

Trainers and trainees

The team at nodal site included psychiatrists and clinical psychologists. Teams at remote sites included psychologists, social workers, and technicians.

The team at nodal site had an experience of clinically evaluating psychiatric patients for a minimum of 6 months to 24 months. The nodal team was trained face to face by consultant psychiatrists in PGIMER. The teams at peripheral sites had limited to no experience of working with psychiatric patients. The team at nodal site acted as the trainers, and the teams at peripheral sites were the trainees. The patients undergoing treatment for psychiatric illnesses at nodal and the peripheral sites, and their caregivers, participated as subjects for diagnostic assessment and management.

Equipment and tools

The Skype platform was used for videoconferences (VCs). Broadband internet connections had bandwidths of 4 mbps and speeds of 512 kbps.

The training was conducted in the use of automated structured diagnostic tool including screening submodule and diagnostic submodules of the automated net-based telepsychiatry application.

The training process

The training was conducted in five phases. The first phase of training involved demonstration, discussions, and role plays. The use of the application and the diagnostic rules were taught during this phase. Phase two involved observation where patient interviews were conducted by trainers at the nodal site and observed by trainees at the remote sites. Patients were recruited from the inpatient ward of the Department of Psychiatry at the PGIMER, Chandigarh, after obtaining informed consent. Phase three involved supervised interviewing where patient interviews were conducted by the trainees at remote sites and supervised by the trainers at the nodal site. Patients were recruited by the trainees from their respective outpatient departments after obtaining consent for videoconferencing assessments. At the end of these sessions, feedback was offered to the trainees on their interviews and use of the application. The fourth phase involved passive rating by trainees where trainers carried out diagnostic interviews through real-time videoconferencing. The fifth and the final phase involved continued supervision and feedback provided to the trainees as well as passive rating by trainers for quality assurance over a period of 4 to 6 weeks. Phone calls and E-mail were used in addition to VCs during this phase. Booster sessions were taken after 3–9 months for consolidation of gains after training. The entire process of training was completed in 62 videoconferencing sessions spread across 2½ months. The total duration of training was about 43 h.

Outcomes assessed

The feasibility of training was assessed in terms of satisfaction with the modes of training, the use of videoconferencing for training, the hands-on training approach, continuous supervision and feedback provided, and the relevance of such training for their daily clinical work.

Effectiveness of training was rated during the simultaneous but independent assessments of patients (n = 30) by trainees and trainers. Interviews were conducted by the trainees who rated on the application (active raters), and simultaneously, the trainers who observed the interview in real-time rated the responses on the application (passive rater). The diagnostic agreement between trainees and trainers was evaluated both at the level of screening and more detailed diagnostic evaluations and compared for percentage agreement.

Costs of VC-based training under different heads were estimated and compared with assumed costs based on an earlier experience of an in-person meeting that was conducted earlier on the same group of trainees at the nodal site.

RESULTS

Feasibility of videoconference-based training

Videoconferencing as a mode of training was quite acceptable and satisfactory to the majority of the participants [Table 1]. The major hindrance to a satisfying training session was the poor strength of the internet signal, more so at the J and K site, which compromised the audiovisual quality of the VCs. However, all participants were quite satisfied with the process of training. Almost all participants perceived hands-on training and booster training sessions to be quite useful. Feedback was obtained after the training as well [Table 2]. Trainees at all three sites felt that the training was useful and educative. They perceived improvement in their psychiatric interviewing skills and their confidence in diagnosing psychiatric disorders. The other advantages of training via VC that were reported were avoiding the need for traveling, being able to fit training sessions in their routine work, and receiving information in “small doses” over a longer duration.

Table 1

Feasibility of training by videoconferencing

Variables to assess feasibility	Trainee (n=6)	Trainer (n=4)
Acceptability of mode of training
Not acceptable	-	-
Somewhat acceptable	-	1
Quite acceptable	6	3
Satisfaction with the mode of training
Not satisfied	-	-
Somewhat satisfied	2	1
Quite satisfied	4	3
Satisfaction with the process of training
Not satisfied	-	-
Somewhat satisfied	-	-
Quite satisfied	6	4
Perceived usefulness of hands-on training
Not useful	-	-
Somewhat useful	-	-
Quite useful	6	4
Usefulness of booster training sessions
Not useful	-	-
Somewhat useful	1	-
Quite useful	5	4
Overall usefulness of training
Not useful	-	-
Somewhat useful	-	-
Quite useful	6	4

Table 2

Posttraining feedback received from the trainees at three remote sites

	Remote sites
	Himachal Pradesh	Jammu and Kashmir	Uttarakhand
Usefulness	Very useful	Very useful	Very useful
Educative value	Quite educative	Very educative	Very educative
Improvement in interviewing skills	Improved	Significantly improved	Significantly improved

Effectiveness of videoconference-based training

Diagnostic agreements between trainees and trainers.

The percentage agreement between ratings of trainees and trainers ranged from 80% to 100% for most questions of both the screening and diagnostic submodules. Submodule and Level 2 screening questions of diagnostic submodules [Table 3]. The percentage agreements for screening for mania (level 1) and for depression (level 2) were comparatively lower at 73% and 63%, respectively.

Table 3

Effectiveness of training by videoconferencing

Screening (level-I)	Percentage agreement	Diagnostic sub-module-Screening (level-II)	Percentage agreement
Mood disorder (mania)	73	Mania	90
Mood disorder (depression)	87	Depression	62
GAD	80	GAD	87
Panic disorder	87	Panic	94
Phobic disorder	80	Phobic disorder	80
OCD	93	OCD	88
Reaction to severe stress	87	Reaction to severe stress	98
Somatoform disorder	87	Somatoform disorders	90
Dissociative disorder (a)	93	Dissociative disorder	87
Dissociative disorder (b)	93	-	-
Alcohol dependence	93	Alcohol dependence	93
Substance dependence	87	Substance dependence	80
Psychosis (a)	87	Psychosis	86
Psychosis (b)	80	-	-
Psychosis (c)	87	-	-
Organic disorders	93	Organic disorders	100
Sexual dysfunction	100	Sexual dysfunction	100
Mental retardation	100	Mental retardation	100
		Neurasthenia	100

GAD – Generalized anxiety disorder; OCD – Obsessive–compulsive disorder

Costs of videoconference-based training

The total costs of training through videoconferencing were estimated to be 100,734 rupees (cost of equipment 53,334 rupees and operating costs 47,400 rupees). The assumed costs of in-person training extrapolated from an earlier in-person meeting involving the same participants were estimated to be 268,725 rupees. Thus, the cost differential (167,991 rupees) was about 150% less, in favor of VC-based training.

DISCUSSION

Training of health-care providers in identification and management of psychiatric disorders remains a key strategy in enhancing the human resources needed delivery of mental health workers. This is especially true for India, given the shortage of trained mental heal professionals. Apart from increasing opportunities for training in different aspects of mental health care, the quality of training needs to be enhanced, and the process of training needs to be made feasible, convenient, and cost-effective. However, there is a lack of consensus on the most effective methods for training mental health professionals.[9] Commonly utilized conventional methods, such as reading material, use of manuals, or brief training workshops, are often insufficient at increasing professional knowledge and skill.[10] It has also been found that there is a need for ongoing support (e.g., consultation and supervision) and contextual factors (e.g., organizational resources) to increase the effectiveness of training.[11] Nevertheless, a meta-analysis found that internet-based learning was equivalent to traditional learning methods and was better than no learning interventions.[12] Moreover, a recent meta-analysis found that web-based training may result in significant increases in knowledge, skill, self-efficacy, and use.[13] Although the evidence is rather limited, training in telehealth and telepsychiatry also appears to be equivalent to in-person training.[67] Training in telepsychiatry helps in the acquisition of knowledge and skills, fosters more positive attitudes, and leads to increased use of telepsychiatry care.[68] On the other hand, the lack of training of training opportunities in telepsychiatry constitutes a significant barrier to the wider implementation of telepsychiatric-based services.

The results of the current study demonstrated that videoconferencing-based training for diagnosing psychiatric disorders was feasible and acceptable for nonspecialist mental health professionals. The process of training was relatively brief and could be conducted over the net. Some of the benefits of this kind of training reported by the trainee's flexibility in training schedules, observing and learning from trainers, constant supervision and feedback, phased learning which avoided information overload, and opportunities to apply skills acquired in routine clinical work. Such training also led to saving of resources such as time and travel costs and appropriate use of existing resources. Finally, it did not disrupt their work or lead to neglect of patient care. The effectiveness of training was indicated by the high rates of diagnostic agreement between the trainees and their trainers. Finally, a preliminary cost analysis suggested that videoconferencing-based training might be less costly than in-person training. However, adequate network connectivity was required for such training to be useful.

It has been suggested that it is vital to minimize the “transactional distance,” which is the communication gap between the instructor and his or her learners when using videoconferencing for educational purposes.[9] Didactic teaching is less effective for a VC audience than for a live audience. Hence, more interactive strategies with elements of active engagement and learning should be employed.

Our training program incorporated several interactive elements along with hands-on training. We consider that these strategies resulted in effectiveness of the program as reflected in the preliminary findings. Furthermore, all the trainees felt more confident in evaluating a patient for psychiatric ailments. Moreover, the clinical decision support system (CDSS) by virtue of its design acts as a self-educating tool, and improvement in diagnostic skills is expected with repeated use. In addition, the use of videoconferencing facilitates ongoing supervision and monitoring, which is further expected to improve skills, provide timely support to providers at remote sites, and enhance their confidence. Such lack of adequate supervision and monitoring is a major drawback of traditional methods of training. Most of the participants found the mode of training acceptable as well as satisfactory, though the major obstacle to satisfaction was the poor quality of audiovisual communication and interruption due to poor internet signal strength in a few sessions. However, considering the difficult geographical accessibility of, the trainees from J and K site reckoned that connecting through Information and Communication Technology modes is more feasible and practical as compared to physical relocation.

Therefore, the hands-on approach used in the current training program, along with its interactive elements, may have contributed to its greater acceptance and effectiveness. Greater attention to constant interaction and feedback, ongoing support, and a phased approach were also valued by the trainees.

CONCLUSION

The results of cost analysis that focused only on monetary aspects of the training program are definitely positive and add to its economic feasibility. The largest costs associated with traditional in-person training were the costs associated with travel, which alone crossed the costs of infrastructure required for teletraining. Moreover, benefits of online training and use of CDSS, along with the provision of supervision and monitoring in real-time, such as improved quality of health-care services and improved health-related outcomes, are expected to outweigh in the long run the initial costs of infrastructure development as seen with most telepsychiatry and telemedicine programs. Thus, training in clinical interviewing using the telepsychiatry application through VC is not only effective, feasible, practical, and potentially useful but also economical method of training.

Telepsychiatric services are becoming vital adjuncts to in-person care, an often the sole means of care in times of crisis such as the current pandemic. A safe, effective, and acceptable way to train mental health professionals in telepsychiatric care can enhance the usefulness of telepsychiatry. Overcoming the limitation of connectivity at a national level will result in an e-world of learning and clinical practice. This will improve the care delivery services. More in-depth research into methods of videoconferencing-based training is required, particularly in India.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.