Role of WhatsApp-based discussions in improving residents' knowledge of post-operative pain management: a pilot study.

BACKGROUND: To provide a platform for the dissemination of basic knowledge of pain management, a WhatsApp group was created by residents and consultants. Common clinical scenarios, resident queries, and important instructions to be followed by residents with respect to running the Acute Pain Service were discussed in the group. This study evaluates the benefits of this interaction.
METHODS: This study was approved by the hospital ethics board and was registered with the Clinical Trial Registry of India. Second- and third-year anesthesia residents were included in a WhatsApp group, along with consultants (board certified anesthesiologists with a special interest in pain). Pain knowledge assessment was performed pre- and post-discussion using a standard 22-point questionnaire. A feedback form, which included self-rated confidence scores (1-10, 10-most confident) and opinions about the 3-month WhatsApp discussion, was collected. Improvements in the documentation in clinical sheets post-discussion were also analyzed.
RESULTS: A total of 38 residents were included in the WhatsApp group. An improvement in the percentage of correct answers from 69.1% (pre-discussion) to 73.6% (post-discussion) was observed (P = 0.031). Improvements in the self-rated residents' confidence levels were also noted (P < 0.05). A total of 37 residents felt that the WhatsApp-based discussion was useful. Documentation of the details of epidural blockade in clinical sheets improved from 30% to 100%.
CONCLUSIONS: The WhatsApp discussion improved residents' knowledge and confidence levels, and also resulted in improved documentation of essential details in the clinical notes. This form of education is promising and should be explored in future studies.

Introduction

Pain relief after surgery is a major challenge and is essential for improving outcomes [1]. Acute Pain Service (APS) Team, a multidisciplinary team often led by anesthesiologists, aims to provide optimal pain relief to all patients after surgery [1]. Although APS is well established in developed countries [2], this concept is still emerging in India and few centers offer these services [3]. Administration, manpower issues [3], and continual education of health care providers [2] are important areas to be addressed for pain services.

Pain education for health professionals has repeatedly been identified as an important step to improve pain management practices [45]. Previous studies have shown that pain education can lead to changes in knowledge, attitude, and practice regarding pain management [678910]. At our hospital, it remains challenging to train and retrain new recruits about the basic concept of acute pain management. Current training is in the form of didactic lectures and interactions with consultants during regular pain posting and with senior colleagues during the evening rounds. Restrictions exist in the available time slots for teaching pain management, which should be combined with training in other areas of anesthesia and critical care. Hence, novel approaches to explore the interaction between consultants and residents are required.

With improvements in technology, smart phone applications for instant messaging, such as WhatsApp, have been used to link groups of people together in an effective and interactive way [11]. WhatsApp is a method of sending and receiving messages to and from individuals or groups with additional features of sending images, videos, and links. This application has allowed the creation of small groups and communications within its boundaries [11]. To improve the interaction between residents and consultants, a WhatsApp group was created by the APS. The group was used as a medium to discuss common clinical scenarios, resident queries, and convey important instructions to be followed by residents with respect to running the APS. This study was conducted to evaluate the benefits of this WhatsApp-based discussion on residents' knowledge of post-operative pain management. The primary aim was to explore the benefits of the 3-month informal WhatsApp training by evaluating pre- and post-discussion responses to standard pain questionnaires. We also evaluated improvements in residents' self-rated confidence scores and documentation in the APS clinical sheets.

Materials and Methods

This educational study was exempted from review by the hospital ethics board and registered with Clinical Trials Registry India (CTRI/201511/006377). All of the second- and third-year anesthesia residents, who were in the department from March 2015 to June 2015, were included. First-year anesthesia residents who joined the department in May 2015 were excluded. Participation of residents was voluntary. Filling and providing feedback were considered implied consent for inclusion in the study. This educational research was based on pre- and postintervention models [1213]. Prior to the WhatsApp discussion, feedback was collected from all of the residents with respect to their current confidence level, the ability to identify problems and report them to seniors, and their ability to deal with bedside issues and guide junior colleagues in the evening rounds (Appendix 1). They were asked to rate their confidence level from 1 to 10 (10, most confident). Residents were asked to fill out the questionnaire based on literature to represent their current understandings of pain management [141516]. This was followed by an impromptu WhatsApp-based discussion that lasted 3 months. At the end of the discussion, the questionnaire was readministered. The benefits of the discussion were assessed by comparing the pre- and post-knowledge scores, as well as by evaluating changes in the confidence levels. Improvements in APS patient sheet documentation were also examined.

Assessment instrument

A standard 22-item questionnaire used in a prior study was administered (Appendix 1) [16]. The questionnaire focuses on domains that are considered minimal but crucial competences regarding pain and its management [16]. This questionnaire included questions that dealt with pain assessment (Items 1,4,6,11,12,13,15,16,21), opioid-related issues (Items 2,5,8,10,17,18,19,20), general principles of pain management (Items 3,4,7,16,22), pain management issues in children (Items 8, 9), and the non-pharmacological aspect of pain management (Item 14) [16]. The questionnaire included 22 statements for which the participants had to indicate whether they agreed or disagreed; an option of ‘Do not know’ was included. In addition to the knowledge-based questions, we added three questions based on the resident's self-evaluation of confidence pertaining to the practice of pain management. The resident had to rate their confidence levels on a score of 1–10 (10, most confident) based on three components: to identify problems and report to seniors, to deal with bed side issues and make decisions, and to guide junior colleagues in the evening rounds. At the end of 3 months, we explored how helpful the WhatsApp interactive discussion was. This was captured by one of three options: Yes / No / Not sure. The feedback forms were handed over or emailed to the residents pre- and post-discussion, and they were given a couple of days to answer and return the forms. The replies were coded for each resident and their identity was kept anonymous. Every resident had the right to not respond.

WhatsApp-based discussion

The WhatsApp-based discussion was impromptu and based on scenarios provided by consultants (board certified anesthesiologists with a special interest in post-operative pain) and difficulties posted by residents. Each question was followed by resident replies. Assignments and further readings were proposed by the consultant. Assignments included focused questions where residents had to perform a thorough literature search and share their search results. Each scenario continued for 4–5 days to allow all of the residents to respond, followed by a brief summary by the consultant. Thus, the learning was directed by the residents' queries and their opinions, and was not based on any pre-defined syllabus. The total number of topics discussed included scenarios posted by consultants and resident-generated queries. The response rate to each topic was noted.

Impact on documentation

During the WhatsApp discussion, a need for adequate documentation in the APS sheets was repeatedly emphasized. To understand the benefit of the discussion on the resident's actual practice, two fields of the APS patient sheet were selected; documentation of details of segmental block following local anesthetic bolus through epidural analgesia catheters using ice (ice test), and details of deep vein thrombosis (DVT) prophylaxis that the patient received. Prior to the WhatsApp discussion, documented details of the ice test reached 30%, while none of the charts had documented details of DVT prophylaxis. These two factors were randomly selected based on importance and due to their poor compliance. After the WhatsApp discussion, all of the APS clinical sheets were checked by an independent observer (not part of the APS team) for 1 month (September 2015) and compliance was noted.

Statistical analysis

An individual question was checked for overall correctness, and the percent of correct answers was noted. A score of 1 was given for the right answer and 0 was given for the wrong answer or an unknown answer. If a question was not attempted or more than one option was scored, it was marked 0. Thus, the maximum score achievable for an individual participant was 22, which equaled a 100% correct answer score. The scores were represented as mean ± SD. The difference between the pre- and post-discussion score was compared using the paired Student's t-test. Knowledge in each domain was also analyzed and the percentage of correct answers was grouped as 100, 99–70, and less than 70 for each domain. Improvement in knowledge for each domain post-discussion was examined using the McNemar's test. Domains on issues of children and non-pharmaco-logical aspects of pain had two and one items, respectively, and the data were grouped into 100% and less than 70%. The self-rated confidence scores pre- and post-discussion were matched and compared using the paired Student's t-test. Improvement in documentation and feedback on the WhatsApp discussion was expressed as a percentage. All of the analyses were performed using IBM® SPSS ver. 22.0 (IBM Corp., Armonk, NY, USA) and MS Excel®. Results with a P value < 0.05 were considered statistically significant.

Results

The WhatsApp discussion was conducted from 19th March 2015 to 17th June 2015 for a duration of 3 months. A total of 38 residents were included in the WhatsApp group. A total of 17 topics were covered, which included 14 consultant-generated scenarios and 3 resident-generated queries (Appendix 2). The scenarios included topics for further reading for residents and advisories were posted by consultants in the group. The response rate varied from 39.5% to 84.2%, with an average of 60.7%. The pre-questionnaires were completed by all 38 residents, and 37 residents returned the post-discussion feedback forms. With respect to the pain questionnaires, a pre-test score of 15.2 ± 3 (69.1%) and post-test score of 16.2 ± 3 (73.6%) were measured. The difference between the paired pre- and post-test scores was statistically significant (P = 0.031). Based on the analysis of individual domains from the questionnaire (Fig. 1), an increase in the number of students scoring 100% post-discussion was seen in all of the domains, with the exception of the domain on non-pharmacological aspects of pain. This increase was significant for the domain dealing with general principles of pain management. A significant difference was also noted in the self-rated confidence scores for all three aspects of reporting to seniors, making decisions, and guiding juniors (Fig. 2). When asked about the usefulness of the program, 37 of the 38 residents felt that the 3-month WhatsApp discussion was useful. After 3 months of WhatsApp discussion, documentation of the details of DVT prophylaxis increased from 0% to 3%, while documentation of the ice test increased from 30% to 100%.

Fig. 1

Graph showing the effects of the WhatsApp-based discussion on each domain of pain knowledge. Pre- and post-discussion scores were compared using the McNemar's test. A: Domain on assessment of pain, P = 0.079 (response to Items 1,4,6,11,12,13,15,16,21). B: Domain on opioid related issues, P = 0.515 (response to Items 2,5,8,10,17,18,19,20). C: Domain on general principles of pain, P = 0.038 (response to Items 3,4,7,16,22). D: Domain on issues in children, P = 0.549 (response to Items 8 and 9). E: Domain on non-pharmacologic aspect, P = 0.070 (response to Items 14).

Fig. 2

Comparison of self-rated confidence scores pre- and post-WhatsApp discussion; the mean value was compared using the Student's paired t-test. Confidence to report to seniors: Self rated confidence score to identify problems and report to seniors: 10-most confident, P = 0.001. Confidence to make decisions: Self rated confidence score to deal with bed side issues and make decisions: 10-most confident, P = 0.001. Confidence to guide juniors: Self rated confidence score to guide juniors in the evening rounds: 10-most confident, P = 0.003.

Discussion

After the WhatsApp-based discussion, an increase in the mean scores from a pre-test score of 15.2 ± 3 (69.1%) to a post-test score of 16.2 ± 3 (73.6%) was noted, which was a statistically significant difference (P = 0.031). Residents' self-rated scores, with respect to confidence in reporting to seniors, making decisions, and guiding junior colleagues, showed significant improvements. All of the residents felt that the WhatsApp-based discussion was useful. We observed a 3% improvement in the documentation of details of DVT prophylaxis in clinical sheets, while the recording of details for the ice test for the extent of epidural blockade increased from 30% to 100%.

Social media has been used in previous studies [171819]. For example, third-year medical students at the University of Michigan were introduced to a Twitter-based educational tool to maximize their educational experience [18]. The feedback suggested that the students perceived the twitter experience to be a unique means of communicating with the faculty, residents, and clerkship.

In our hospital, residents work at variable times and at numerous locations and are often unable to attend all scheduled educational sessions. Therefore, we felt the need to provide an a-synchronistic approach to learning. Electronic media tools may be particularly useful because they act as a repository and can be accessed at times convenient for the learner [17]. We used WhatsApp as an instant messaging system imparting pain knowledge. WhatsApp is a smartphone application that operates on all types of devices and operating systems. The technical advantage of WhatsApp includes its simplicity, low cost, and privacy [11]. The obvious advantage for us was that consultants could post their scenarios at any convenient time and the residents could respond within a stipulated period, mainly after working hours. With residents posting in different locations with different working schedules, this was a major advantage. All of the residents being linked in one common group helped to address queries in a common forum, in addition to stimulating reading in various topics within a defined time frame. The assignment generated during the discussion ensured a thorough review of the literature by residents, and the results were shared with the entire group, benefiting everyone.

Interpretation of the adequacy of baseline pain knowledge is a matter of debate, with few considering a cutoff of 80% as a good (acceptable) level of knowledge, while 70% is often considered adequate in online courses [16]. Our study found that the pre-discussion test score was 69.1%. Considering 70% as the cutoff point, the baseline pain knowledge of our residents was less than adequate. The post-discussion score of 73.6% is suggestive of the positive impact of the WhatsApp discussion on pain knowledge.

In our study, we used the self-rated score to judge the resident's confidence. The residents rated their confidence scores on a scale of 1–10 (10, most confident). The limitation of using self-rated scores is that self-assessment is easily affected by variables and can lead to over- or under-estimation of performance [20]. However, self-assessment tools have been used and validated in numerous medical fields, including surgery [21]. In the absence of validated tools for assessing the ability to treat pain and make decisions, we used self-rated confidence scales based on three aspects: reporting to seniors, making decisions, and training juniors. We observed an improvement in all three areas after the WhatsApp-based discussion. Although we observed statistical significance, this improvement may only represent a small change in the actual scores, which could be explained by the fact that the discussion lasted for 3 months and included theoretical knowledge of various aspects. Practical bedside experience, which is an important aspect of resident training, was not a component of the WhatsApp discussion.

Pain education can improve pain management [12]. Educational and quality improvement initiatives in a student's knowledge on pain management ultimately improve practices and the patients' quality of life [16]. Previous audits in our institute showed poor compliance to protocols, particularly with respect to the documentation of relevant details in the clinical sheets [22]. We determined if our WhatsApp-based discussion led to improved documentation. Assuming that the positive effects would phase out, it was essential to examine the long-term effects; thus, we screened the clinical sheets for documentation 3 months post-discussion. An improvement in documentation in DVT prophylaxis to 3% was initially observed, supporting the need for regular and repeated emphasis. However, an increase to 100% in the documentation of details of the ice test after the 3-month discussion suggests that changes in relevant areas could persist.

An important limitation of this study is the tool used to evaluate pain knowledge. The questionnaires used in previous studies are focused on information associated with the treatment of pain (mainly chronic or cancer), and there is no specific questionnaire looking into various aspects of acute pain management, including regional analgesia. The WhatsApp group was started instinctively; in addition, the content of the group discussion was impromptu. The topics and assignments were guided based on responses from the students. The questionnaire used in the study did not focus on acute pain management, and not all topics included in the questionnaire were discussed in the group. This could explain the small (although statistically significant) absolute difference in the pre- and post-discussion scores. However, this positive change supports the need for further studies. In addition, it is important to have a more appropriate questionnaire focusing on various aspects of acute pain management as a study tool. Our center is now evaluating the role of WhatsApp as part of a comprehensive teaching program, and we hope the results of this trial (CTRI/2016/05/006902) will increase our understanding of using social media as an educational tool.

There were various responses to scenarios, with an average response rate of 60.7%. Although WhatsApp allows the user to check if the message has been received and read by all members of the group, an actual understanding of the scientific matter posted in the WhatsApp group cannot be guaranteed in such social media based discussions. However, this limitation also exists in traditional classroom teaching. In addition, in this study, the questionnaires were either handed or emailed to the students directly. Although the residents were instructed not to consult external sources, the questionnaires were not answered in a monitored environment. Thus, the possibility of consulting medical texts cannot be ruled out. Because referral to medical texts would result in much higher scores than the current scores of 69.1% and 73.6% pre- and post-discussion, we believe that the students answered the questions honestly.

In conclusion, the informal WhatsApp-based discussion led to improvements in residents' knowledge and self-rated confidence levels, as well as improvements in the documentation of essential details in the clinical notes. This mode of education is promising and should be explored in future studies.