Role of written examination in the assessment of attitude ethics and communication in medical students: Perceptions of medical faculties.

BACKGROUND: The study aims to record the perceptions of medical faculties regarding the effectiveness of theory-based examination to assess the newly introduced competencies of attitude ethics and communication (AETCOM) in the competency-based medical curriculum for Indian medical graduates.
MATERIALS AND METHODS: This is an analytical cross-sectional study performed on the month of February 2020 where a prevalidated questionnaire consisting of components of AETCOM was e-mailed to the teaching faculties of IQ City Medical College via Google Forms. Consenting faculties responded. Their results were analyzed by inbuilt Google statistics and were cross-verified with SPSS 20.0.
RESULTS: Sixty percent faculties strongly agree regarding the beneficial role of mandatory inclusion of AETCOM competencies in competency-based medical education. About 61.66% of faculties strongly agree that both formative assessment and summative assessment of AETCOM are essential. Although 48.33% of faculties believed that theoretical questions can be used to assess AETCOM, 51.66% of faculties do not agree that theory examination serves as an effective tool to assess AETCOM. They believe that AETCOM cannot be written on paper and attitude can change in reality when facing a real-world clinical scenario in contrast to what is written in answer script during creative writing.
CONCLUSIONS: Assessment of AETCOM is essential and it should be preferably done via a practical approach in a real-world simulated scenario and not by written theoretical examination. Copyright:

Introduction

India has got a very low doctor–population ratio of 1:1800 compared to the standard proposed by the World Health Organization[1] Even with 542 medical colleges and an annual intake of over 80,000 candidates, the desired doctor–population ratio is yet to be achieved, and the Medical Council of India (MCI) has set the year 2031 to achieve that goal. With adequate input of resources in the near future, this may be possible; however, another issue that needs to be addressed simultaneously is the academic quality of the medical curriculum. Unless both of these issues are tackled, the intended outcome of producing competent and dedicated doctors will not be possible.[2] The medical curriculum and training programs by MCI were designed to address specific learning objectives addressing primarily three domains: cognitive, psychomotor, and affective, also known as the head, hand, and heart, respectively. However, until recently, the traditional way of medical education in India dealt mainly with the head compared to the hand and nearly neglected the heart. Hence, it failed to produce a clinician who could provide holistic care (i.e., preventive, curative, and palliative care) with empathy and compassion.[34] There has been a gradual rise of mistrust of the general population upon the medical fraternity due to incidences of medical negligence, misconduct, and unethical practices. This has led to many incidences of violence and legal complications, especially toward the junior medical professionals. All of these demanded a dire need for revision of the existing medical curriculum.[56] Hence, MCI drastically revised the traditional curriculum and introduced the competency-based medical curriculum and made it mandatory to implement it for undergraduate medical batch from the year 2019 as per the latest Graduate Medical Education Regulation (GMER 2019).[7] In competency-based medical education (CBME), the outcome is expressed in terms of competencies. To address the affective domain, the CBME curriculum has integrated the attitude, ethics, and communication (AETCOM) module,[8] which addresses the “heart” and stresses on the development of proper attitude and communication skills and enables medical graduates to practice ethically in a real-world scenario. The revised medical curriculum and AETCOM module nicely distribute various levels of AETCOM competencies across all disciplines. When it comes to the assessment of these attributes, various newer evaluation methods, summative and formative, have been proposed which starts with the test of knowledge domain and ends with an assessment of affective domain, i.e., behavioral practice in such a manner that skill and performance assessment of a trainee would provide a more realistic picture in working clinical settings.[910] While methods such as objective structured clinical examination (OSCE) can be a tool to assess the competence of attitude and communication (ATCOM) in a summative assessment examination setting, a better holistic approach with 360° analysis by continuous formative evaluation technique based on the feedback assessment system from multiple stakeholders like parents, teachers, mentors, and other peers can effectively evaluate a student's competence in behavioral performance with the help of any preformulated scale or survey.[11] The preferred methods can be direct observation and feedback using workplace-based assessment tools which may provide valuable information about the ability of the medical student to function as a working professional in real-world clinical setting.[41213] However recently, as per the GMER 2019 and MCI Assessment module,[14] a theory-based question is mandatory in a summative examination of every discipline. Even with many newer assessment methods taught to us in the MCI basic teacher's training programs, many institutes still stick to the traditional theory and viva voce-based practical examination. This has placed a great emphasis on the role of theory examination to assess AETCOM. To address this issue, this study aims to look at the perceptions of faculties regarding their take on the matter of how theory examination can serve as an effective tool to assess AETCOM.

Materials and Methods

This analytical cross-sectional questionnaire-based [Table 1] study was carried out in IQ City Medical College. The institutional ethics committee approval and the consent of the dean were obtained vide ethical clearance certificate no. IQMC/IEC/LTR/19/06/13 (07) dated December 10, 2019. The questionnaire comprising various aspects on AETCOM module was prevalidated by institutional ethics and research committee and was sent by e-mail as Google Form. Consenting faculty members filled up the form online. The questionnaire comprised two sections. The first section used a Likert scale[15] based questionnaire focused on their perceptions regarding various aspects of AETCOM that included the competencies, teaching–learning methods, and assessments. It was followed by open-ended questions which asked them to name a method of their choice for assessing AETCOM module along with its justification. Responses were received from 96 faculty members. Among them, the responses of faculty members who did not attend AETCOM workshops by MCI were not considered. The responses were analyzed by an inbuilt Google Statistics available with Google forms. The quantitative data were further statistically cross-verified with SPSS 20.0 (IBM, Armonk, New York, United States). On applying Kolmogorov–Smirnov test, the value of P was found to be significant; therefore, the distribution was skewed. Thus, central tendency and dispersion of data were expressed in median and interquartile range (IQR), respectively. The qualitative data from the open-ended question were thematically analyzed.

Table 1

Test questionnaire (circulated as Google Form)

Details of faculty
Name:
Designation:
Department:
Teachers training workshop attended (more than one option selectable)
CISP
RBCW
AETCOM module workshop
BCW
None
Statements/questions	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
1. The mandatory implementation of AETCOM as competencies will be beneficial to the Indian Medical Graduates
2. Both formative and summative assessments of AETCOM are essential
3. Written/theory examination serves as a suitable tool to assess AETCOM
3a. If you choose “neutral” “agree” or “strongly agree”, please provide a method of choice for your assessment of AETCOM by theory examination
3b. If you chose “disagree” or “strongly disagree”, justify your choice briefly within 20 words

AETCOM=Attitude ethics and communication, CISP=Curriculum implementation support program, RBCW=Revised basic course workshop, BCW=Basic course workshop

Results

Ninety-six faculty members responded to the e-mail by filling up the Google Form questionnaire. Apart from their name and designation, the questionnaire included a section regarding information of their teachers' training workshops attended. On analysis, it was found that 22 attended Curriculum Implementation Support Program (CISP) workshop along with a Revised Basic Course Workshop (RBCW) along with AETCOM module, 38 attended RBCW/basic course workshop (BCW) along with AETCOM module but not CISP, and 13 faculty members attended RBCW/BCW but not AETCOM module. Twenty-three faculty members were yet to attend any MCI teachers' training workshop. This distribution is represented in Figure 1.

Figure 1

Distribution of participant faculties according to their teachers training program exposure

This study only documents the perceptions of faculties trained in AETCOM module; hence, the responses of faculties who did RBCW/BCW without AETCOM workshop and the faculties who were untrained in any workshops have been excluded from the study. Responses of the eligible 60 faculties to the various questionnaires were analyzed.

A composite stacked bar in Figure 2 shows a comparative analysis of the responses of the first three Likert[15] scale based questions.

Figure 2

Question 1: The mandatory implementation of attitude, ethics, and communication as competencies will be beneficial to the Indian Medical Graduates. Question 2: Both formative and summative assessments of attitude, ethics, and communication are essential. Q3: Written/theory examination serves as a suitable tool to assess attitude, ethics, and communication

When asked about when asked whether the mandatory implementation of AETCOM as competencies will be beneficial to the Indian Medical Graduates, there was a positive response from majority (81.66%) of the participants, i.e., 60% of faculties “strongly agreed” and 21.66% “agreed.” About 61.66% of them “strongly agreed” and 20% “agreed” when asked whether formative and summative assessments of AETCOM are essential, as stated in GMER 2019. However, when it came to their opinion regarding using theory question or written examination to assess AETCOM, there was a multitude of varied responses. Although 10% and 16.6% faculties “agreed” and “strongly agreed,” respectively, to the statement, majority of responses were skewed to the side of “disagree” and “strongly disagree” which were 28.33% and 23.33%, respectively. Hence, a cumulative 51.66% or more than half of the faculty participants did not believe that the theory examination is a suitable tool for assessment of AETCOM. A comparison of median values and the IQR with their significance is shown in Table 2.

Table 2

Comparison of medians of the responses of the statements

Statements	Median	Interquartile range	Significance
1. The mandatory implementation of AETCOM as competencies will be beneficial to the Indian Medical Graduates	5	1	Majority “Strongly Agree”
2. Both formative and summative assessments of AETCOM are essential	5	1	Majority “Strongly Agree”
3. Written/theory examination serves as a suitable tool to assess AETCOM	2	2	Majority “Disagree”

Explanation of score: 1=Strongly disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly agree, AETCOM=Attitude ethics and communication

The next part of the questionnaire was a subsection of statement 3 where the participants who chose “neutral” “agree” or “strongly agree” were asked to write a method of choice for assessing AETCOM. It is to be noted only a handful of participants seemed to react neutral or positively to this statement. Four types of theoretical assessment tool were proposed by them. They were “SAQs” (short answer type questions), “short notes,” “creative writing experiences,” and “problem based MCQS.”

The majority of participants who did not agree that the theory question was not a suitable method to assess AETCOM were asked to justify their answer within 50 words. Since it was an open-ended question, the responses were thematically analyzed and are shown in Table 3.

Table 3

Thematic analysis of qualitative responses of Section 3b of the test questionnaire

Reasons for disagreeing to the theoretical assessment of AETCOM

Cannot reflect the attitude

Communication skills cannot be written on paper

Nonverbal communication skills cannot be tested

Lack of question bank for testing the new competencies

Tool for testing knowledge domain cannot test the affective domain

Attitude changes from paper to reality

Students differ in what they write and how they behave

Possibility of repetition of items when every discipline has to frame a theory question

No specific model answers available for evaluating theory question

Marking of creative writing may be subjective

A doctor cannot behave on paper

AETCOM=Attitude ethics and communication

Apart from those mentioned in Table 3, there were other responses citing example of various other tools for assessment such as objective structured practical examination (OSPE)/OSCE and methods such as video simulation questionnaire and role-play feedback, etc., However, those responses were not considered as the logic was to provide justification against the role of written examination to evaluate AETCOM as there is a separate practical examination where AETCOM will be tested, and the above-mentioned tools can be used for the purpose which showed a lack of understanding of the statement by the participants.

Discussion

The educationists have well expressed that expertise is favored over competence as the ultimate goal in CBME.[16] The competency-based training programs must define these standards of an acceptable level of expertise. Not only for the outcome competencies but also these should be well defined for the milestones that the medical trainee will be achieving at the end of curriculum completion. A criterion-based approach is best adopted to establish these cutoff standards.[917] This should be the standard of an absolute level of competence and independent of the competence of the other students as adopting the norm-based approach risks a possibility of setting a standard below the acceptable level of expertise. This applies to both the cognitive, psychomotor, and affective skills, and AETCOM being the affective domain is extremely vital to fully mold a medical graduate into a competent doctor to serve the community.

The recent movement toward CBME in India has seen a rapid evolution from the stage of discussion and planning since the release of the GMER 1997[18] by MCI, which was followed by various meetings where reforms were suggested and the “Vision 2015” was published by the MCI in 2011.[19] Here, competency-based outcomes for “Indian Medical Graduate” were outlined. To serve as a “Basic Doctor” or a physician of the first contact, the medical graduate would need to play five roles, namely clinician, leader and team member, communicator, lifelong learner, and professional (who is ethical, responsive, and accountable to patients, community, and profession). The competency was defined as “desired and observable ability in the real world scenario” and the specific competencies to perform the above roles to be developed were also specified. However, these deliberations did not discuss any assessment methods but only mentioned that the assessment will be criterion referenced. This was followed by the ATCOM module which was quickly revised to AETCOM module[8] where framework for teaching AETCOM was outlined. In 2019, the Assessment module[14] was published by MCI and then the revised GMER document where MCI has suggested a mandatory assessment of AETCOM in the form of continuous formative assessment and end-term summative assessment. This information has been well circulated by MCI via the CISP workshops conducted by the regional centers and subsequently by nodal centers. This is well evident in our results as majority of the faculties strongly agree to the essentiality of the formative and summative assessments of AETCOM. As per GMER-2019, the theory question in all summative examinations across all disciplines should feature a mandatory question from AETCOM in addition to practical examination. This has led to a difference of opinions as to the effectiveness of a tool that primarily tests the knowledge domain to assess an attribute that mostly belongs to affective domain.

As seen from our results, majority of the participants “strongly agree” with the implementation of AETCOM as a part of the undergraduate curriculum. Dedicated teaching–learning methods to impart AETCOM have to be done across all phases starting with Phase 1, i.e., First Professional MBBS following the AETCOM module. The various reasons put forward by the participants of the study provide valid and pertinent reasons why theoretical assessment of AETCOM cannot be justified.

A theory question often fails to provide the circumstances that would affect his reasoning and behavior which may be quite contrasting when a young doctor faces the challenges in real-life scenario. Hence, in addition to the record-keeping and reflections, as it is being done as a part of continuous formative assessment, to test AETCOM it is better to stress on the practical aspect of the examination where assessment method will be via OSPE/OSCE stations with checklists pertaining to AETCOM. Stations featuring Role-play, simulated patients, and direct observations do a better justice in this regard.

It is to be noted that this is the first batch on which the competencies have been implemented. Formative assessment in the form of logbook record-keeping and reflection after each session of teaching–learning is being done, but the current batch is yet to face their summative assessment, and hence, the speculations and perceptions of faculties are yet to be tested in the real world.

As the AETCOM module has been introduced this year and any summative assessment is yet to be taken for it, this study is the first of its kind to analyze and review the teachers' feedback on the use of theoretical examination. However, the study sample incudes faculties of one medical college only and the perceptions may differ among faculties of other medical colleges, so further studies using the same model are needed in future to substantiate the findings of this study. In addition, a follow-up set of data can be obtained with the same set of the study population 1 year later when the examinations will be over. Then, the faculties who are merely speculating now will have definitive evidence in the form of assessment results and feedback and can provide a better response from their proper understanding of the new areas of the competency-based medical curriculum. The follow-up project has been planned by the authors in future; however, faculty attrition rate has to be considered as with time new faculties join, old faculties resign or retire.

However, there is no denying the fact that the GMER has provided an excellent model which can be tested and validated and the results in upcoming years can retrospectively prove or disprove the effectivity of theory examination to assess AETCOM. That being said, the GMER and AETCOM module is more of a guideline and every college and university are free to adopt and form their own working plan of assessment. Whatever the plan of action is, a record of every proceeding, is to be maintained and sent to the MCI regional center. Regional center will then compile all the activities of the sister medical college and hospitals and communicate to the nodal center and subsequently to MCI.

Conclusion

The AETCOM brings a new life to the medical curriculum. It should be taught and assessed continuously as well as at the end of each term. The assessment of the same should preferably be done in a practical based holistic 3600 approach. Although theoretical tools like SAQs, problem-based MCQs, and creative writing experiences may be used, they may not be the best way to evaluate AETCOM. However, this is still in the inception stage and implementation and real-world evaluation in upcoming years will definitely provide insight into this matter and will help up to formulate tools to effectively assess the AETCOM skills in Indian medical graduates.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.