Development and implementation of a competency-based module for teaching research methodology to medical undergraduates.

CONTEXT: Research experience helps an undergraduate student to understand published works, to learn teamwork, and even to consider research as a career. Few medical institutions have attempted to engage undergraduates in research experience. Competency-based medical education has emerged as a core strategy to educate and assess medical students worldwide. AIMS: This study aims to develop and implement a competency-based research methodology training module for undergraduate students and find out students' perception about this. SETTINGS AND
DESIGN: A cross-sectional study of mixed design was undertaken in the Department of Community Medicine, UCMS, Delhi, India.
METHODOLOGY: A competency-based research training module was developed and implemented with 4th semester undergraduate students posted in the department. Students' feedback about the module was obtained. STATISTICAL ANALYSIS USED: For quantitative variables, means, ranges, medians, and percentages were calculated. To find out students' perception about the posting a qualitative analysis was done.
RESULTS: The module was implemented with 25 students posted in the department in May 2017. However, feedback was obtained from 23 students. About 83% of the students reported as highly satisfied with the posting, 61% of the students mentioned that after completion of this posting, they felt motivated to do further research. A qualitative analysis of the feedback showed that students found the project helped them to enhance their knowledge and develop skills.
CONCLUSIONS: Competency-based research methodology training can serve as a tool for teaching research methodology to undergraduate students.

Introduction

Research experience helps an undergraduate student to understand published works, to learn team work, and might also help to consider research as a career. Research skills can be taught through a series of lectures and tutorials. Some institutions follow problem-based curricula, e-learning, research electives, compulsory research projects, and programs for volunteers.[123] There is also facilitation of research training by different organization like Indian Council of Medical Research-Short Term Studentship projects.[4] Competency-based medical education (CBME) has emerged as a core strategy to educate and assess medical students worldwide. CBME is learner centered, flexible and gives importance on formative assessment and feedback. Research training based on CBME focus on outcomes and prepares students for actual professional practice. Teaching–learning activities are more skill-based, involving hands-on experience.[5] Assessment in competency-based research training (CBRT) is workplace based and milestones achieved at the end of the posting need to be predecided. Incorporating research-based competencies are challenging and require careful planning and attention.[6] The authors planned to introduce CBRT to undergraduate medical students with the following objectives:

To develop and implement a competency-based module for teaching research methodology to undergraduate students

To find out the level of satisfaction, self-perceived gain in knowledge and skill, and motivation to pursue medical research in future among the undergraduate students after completion of the module.

Methodology

Materials and Methods

A cross-sectional study of mixed study design (both qualitative and quantitative components) was conducted in the Department of Community Medicine, in a Medical College of Delhi, from February 2017 to December 2017. This project was a part of regular routine teaching–learning program of the department. Undergraduate students are posted in the department of community medicine in 4th semester for “project posting” for 21 days for 3 h each day. In this posting, usually, students become familiar with research method and they carry out a small research project. These projects are usually descriptive cross-sectional studies where data are collected from community, outdoor and indoor patients, or student community. The students are expected to develop a research proposal, collect and analyze data, and present it for assessment. The group is guided by 3–4 supervisors. One of the supervisors is a faculty of the community medicine department. Others are usually senior and junior residents of the department.

Development of the module

A competency-based module was developed with the help of faculties and residents. Standard procedure was followed[67] for development and implementation of the module. The steps were as follows:

Identification of competencies: The following steps were followed

Review of literature

Identification of key competence by opinions of the experts (faculties and senior residents)

Identifying critical elements of research and behavior

Describing regular competencies and their components: Having identified the main competence domain, competencies within each domain was developed and discussed.

Content identification and program organization: For each competency corresponding content was identified. Sequencing of the whole program was done keeping time and resources in mind. Assessment with feedback sessions was given priority. Supervisors, i.e., residents were sensitized and trained for the whole exercise

Planning for assessment: Observable and measurable forms of competencies were described. A longitudinal assessment program was defined with selected assessment tools

Implementation of the program: The undergraduate was students were briefed in the beginning of the posting about this program and accompanied assessment method

Evaluation of the program: At the end of the posting, students completed a feedback pro forma by giving inputs about their perception about this posting. The information was collected about an undergraduate student's level of satisfaction, self-perceived level of motivation and interest to conduct medical research and students’ self-perceived gain in research methodology-related knowledge and skills. Mean and median value of perceived competence was calculated. Supporting examples from students responses-theme both positively and negatively oriented were analyzed.

Operational definition of competence in the current context

An undergraduate student at the end of this teaching–learning program will be able to explore relevant literature, formulate a research question and objectives, design and undertake appropriate methods to address the question, collect and analyze data and then present the findings. This posting also should enable an undergraduate student to build on soft skills such as communication, leadership, interpersonal relationship, and critical thinking.

The difference between traditional and competency-based research training is shown in Table 1.

Table 1

Difference between the traditional and competency-based training to teach research methodology to medical undergraduates

Traditional research training	CBRT
Usually emphasis is on the process	Emphasis was on the process but more on the outcome. Expected competencies were more clearly defined. Teaching-Learning sessions were in line with expected competencies
Students are assessed as a group	Students were assessed on an individual basis
Expected outcome/competencies are not communicated	Outcomes/competencies were clearly defined and communicated to the students
Subjected to the individual supervisor s planning with the departmental goal in sight	It was structured and modular teaching with departmental goal in sight
More emphasis on end-posting assessment with group presentation of project	Emphasis on assessment throughout the posting with end-posting written test and individual presentation of the project
	Assessment schemes were more carefully designed
	Assessment is planned with having a match with desired competency
No importance is usually given on soft skills	Due importance was given on soft skills like teamwork, communication, behavior in the field

CBRT=Competency-based research training

All the 25 students posted in the department of community medicine, in May, were invited to participate in the present study and give feedback regarding this module using a semi-structured questionnaire. This questionnaire had two sections. In the first section, students were asked to rank their level of satisfaction, motivation to do future research and if they found the activity as interesting in a scale of 0–5 which were further categorized as low, medium, and high with equal score at each level. The perceived gain in knowledge and skill in different areas of research was assessed on a Likert scale of 1–5. The second section of the questionnaire had questions about their perception about the posting: “List things that you liked about this teaching–learning program,” and “list things that you did not like about this teaching–learning program.”

For quantitative variables, mean and range, median, and percentage were calculated. To find out students’ perception about the posting, interview data were examined aiming to obtain the emerging themes. The initial analysis revealed a number of basic themes that were arranged to form organizing themes. Student's responses both positively and negatively oriented were reported as verbatim.

Approval of Institute's Ethics Review Committee was obtained before initiation of the study. Informed written consent was obtained from all students for their feedback.

Results

Identification of core competence and program organization

We were able to identify with four competencies: (1) research related knowledge, (2) research related skills, (3) behavior and communication competence, and (4) ethics and human subject protection. We also tried to describe each of these competencies with a list of subcompetencies [Table 2].

Table 2

Identification of core competence domain and description

Competence domain	Subcompetency/description
Research-related knowledge (knowledge of research methodology, method, and techniques)	The student understands the purpose, concept, and criteria for choosing a topic for research
	The student has an understanding of the different type of research method, study design
	The student has the knowledge of doing a literature review and can develop a research question
	The student knows different elements of a research protocol
	The student has an understanding of basic statistics including classification of variables, measures of central tendency and dispersion
Research related skills (application of knowledge in small scale research projects)	The student can formulate a simple research question, frame objectives aligned to the research question and can choose an appropriate study design to address the research question
	The student can contribute to develop a questionnaire for data collection
	The student can calculate the sample size for a prevalence study
	The student can perform probability sampling and can recruit study sample
	The student can contribute to writing an informed consent form
	The student can contribute to developing the overall operational plan of the study The student can write a protocol
	The student can extract and carry out basic analysis of key data sets (using MS excel) by producing frequencies, tables, graphs, calculating rate, ratio, proportion, and cross-tabulations; interprets the key findings from this
	Identify and articulate whether or not any conclusions drawn from analyses of data are valid and based on the material provided
Behavioral/communication competence	Shows the ability to
	Communicate with community members
	Interact appropriately with peers
	Behave and conduct herself in a professional manner in the community
Knowledge and skills of ethics and human subject protection	Demonstrate an understanding of ethics and regulation of research while dealing with human subjects
	Ensures confidentiality, privacy, and autonomy of research participants
	Understand the need for ethical approval to be obtained before research activities are initiated

Once the core competencies were identified and described, for each of the subcompetencies the related competency domain (knowledge, skill, attitude, and communication), teaching–learning method, desired level achieved according to the assessment framework of Miller's pyramid[8] (knows, knows how, shows, and performs) were decided. Assessment methods were also decided [Table 3].

Table 3

Different competency, related domain, teaching-learning, and assessment method

Competency	Domain Knoweldge (K), Skill (S), Attitude (A), Communication (C)	Teaching-learning method	Level in Millers Pyramid Knows (K), Knows How (KH), Shows (S), Shows How (SH) Performs (P)	Assessment method	Assessment Type (F/S)
The student understands the purpose, concept, and criteria for choosing a topic for research	K	Case-based discussion	K	End posting assessment: Short question	S
The student has an understanding of the different type of research method, study design	K	Tutorial	KH	Short exercises	F
The student can formulate a clear, answerable relevant simple research question, frame objectives aligned to it and can choose an appropriate study design to address the research question	K	Discussion	KH	Short question	F
The student has the knowledge of doing a literature review	K	Discussion and observes supervisor	KH	On-line game	F
Student performs literature search using search engines and keywords	S		SH	OSPE station	S
The student knows different elements of a research protocol	K	Tutorial	K	Crossword puzzle	F
The Student can discuss writing a protocol with different elements of research	K	Student group led discussion using Fish Bowl method	KH	Peer Review by fellow students	F
The student knows and can contribute to develop a questionnaire for data collection	K	Lecture followed by discussion	K	-	-
The student can calculate the sample size for a prevalence study	S	Hands-on training	S	Problem-based exercise	S
The student can perform probability sampling and can recruit study sample					F
They have an understanding of basic statistics including classification of variables, measures of central tendency and dispersion	K	Lecture followed by discussion	K	End posting assessment: Short question	S
The student can extract and analyze data using MS-Excel Presentation of research finding by creating tables, graphs, calculating rate, ratio, proportion	S	Hands-on training	S	OSPE station	F and S
The student can contribute to writing an informed consent form	K	Discussion	KH	-	F
The student can contribute to developing the overall operational plan of the study	S	Formulation of research protocol	S	Observation	F
Demonstrate an understanding of ethics and regulation of research while dealing with human subjects	K	Discussion	KH	-
The student shows the ability to communicate effectively with community members	A,C	Discussion	S	Observation using a checklist	-

OSPE=Objective structured practical examination, K=Knowledge, K=Knows, S=Skill, S=Shows, S=Summative, A=Attitude, C=Communication, KH=Knows How, P=Performs, F=Formative

Implementation of the module

The module was implemented with 25 undergraduate students from May 1 to May 21 2017. Overall working days available for implementation were 17 days. One faculty and two residents were supervisors. Students worked in small groups, facilitated by supervisors, choose topics within the groups, developed a series a research questions, set appropriate study designs. The supervisors facilitated the discussion on how the whole group wished to proceed. Each student collected, entered data into a common MS excel format. Data were checked for accuracy and cleaned by the supervisors. The compiled data were mailed to all students, and they did a basic analysis of the data and compiled a report individually. Students were provided with the formative assessment and feedback throughout the project duration.

Evaluation of the program

Students’ perception about the module

Most of the students (83%) reported their level of satisfaction as high (83%). About 61% reported this posting highly motivated them to do research in future. More than half of the students (52%) found this teaching–learning activity as moderately interesting and 43% found it very interesting [Table 4].

Table 4

How effective was the posting: Students’ perceptions

Variable	Level
	Low (%)	Moderate (%)	High (%)
Level of satisfaction	0	4 (17)	19 (83)
“This posting motivated you to do research in future.”	1 (4)	8 (35)	14 (61)
Found the posting interesting	1 (4)	12 (52)	10 (43)

The median value for students’ self-perceived gain in different areas of research was as follows: literature search-4, study design-4, data collection-3, and data analysis-3 [Table 5].

Table 5

Self-perceived gain in knowledge and skill on different components of research on a Likert scale of 1-5

Perceived gain in knowledge and skill	Mean (range)	Median
Literature search	3.7 (2, 5)	4
Study design	3.4 (1, 5)	4
Data collection	3.4 (2, 5)	3
Data analysis	3.8 (2, 5)	3
Report writing	3.7 (2, 5)	3

Qualitative analysis of students’ perception about the posting elicited both positive and negative responses. For positively oriented responses toward this posting six themes were identified: addition to knowledge, research skill development, development of interest in research, felt motivated to do research in future, development of soft skills and presence of a conducive environment. Students reported that they came to know about different components of research, learnt data analysis, and enjoyed assessments as fun activities [Table 6].

Table 6

Supporting Verbatim examples from students responses-theme positively oriented toward this posting following the question: List things which you liked about this teaching-learning program

Theme	Verbatim example
Knowledge	“My idea of research was it has to be laboratory based” “Learnt about areas of research,” “came to know what is meant by research,” “became familiar with different components of research,” “broadened my horizon”
Skill	“Learnt data analysis,” “got an opportunity to refresh my MS Excel skills.” “learnt data collection,” “learnt something about how to do a research”
Interest	“Developed some interest in field of research,” “it was interesting as I was doing research for the first time,” “An interesting and interactive programme that taught us all research components as well as motivated us to learn more and perform more in this field”
Motivated to do future research	“In future shall like to do a research project,” “all the things that we have learnt will definitely help us in future,” “I can now try for my own research paper as now I am aware how research project is done,” “the posting helped me and motivated me to do research work for my further medical career”
Soft-skill	“Interesting tasks performed apart from academics and learnt new skills,” “improved my conversation skill and confidence,” “we learnt to work as a team,” “learnt to talk to community members,” “student from weak background can do good,” “hesitation to represent project work got reduced,” “this posting helped me to open up,”
Conducive environment	“Discussion among students and teachers everybody shared their view,” “nontraditional seating arrangement made it less threatening less monotonous,” “The way things were taught was amazing” “interactive classes equal opportunity given to all students” “liked the idea that I knew from the beginning how I shall be assessed” “I found the assessments were fun activities”

Qualitative analysis of the responses negatively oriented to this posting showed that some students found it difficult that the research topic chosen by the group had no relation with medicine. The use of computer was also a problem reported by the students. Students did not like the idea of getting assessed on a regular basis. They also reported less time available for doing a research project [Table 7].

Table 7

Supporting Verbatim examples from students responses-theme negatively oriented toward this posting following the question: List things which you did not like about this teaching-learning program

Theme	Verbatim example
No relation to medical field	“No interaction with patients,” “learning to do literature search might not help in medical field,” “did not give me any information about medicine”
Computer	“If computer access would have been provided, e.g., in library quality of work would have been better,” “Lots of work depending on computer and internet,” “Did not like doing computer work”
Group dynamic	“Groups should have been divided along the motivation to do research” “I wanted to do this seriously whereas many of my batch mates were not interested”
Expectation not met	“We were not taught about any data analysis software” “Our research work should get published”
High workload	“High work pressure right before examination,” “lot of study and work to do,” “I did not like the homeworks”
Assessment	“I don’t like the fact that our work is getting assessed”
Less time	“Less days for data collection,” “I needed more days for data analysis” “we needed more days for report writing”

Discussion

The importance of integration of research in undergraduate medical students’ curriculum has been highlighted in many reports[91011] Medical Council of India (MCI) in the document of Regulations on Graduate Medical Education (2016) states that students should be having the skill to carry out a small research project.

Competency is a observable and it integrates multiple dimensions such as knowledge, skill, and attitude.[12] CBME is focused on outcomes, and the teaching–learning strategy is aligned to the intended outcomes.[13] CBME does not specify any particular teaching–learning strategy and keeps these options open as per the learners’ characteristics and intended outcomes.[1415]

For research training of medical undergraduates, we tried to follow the principles of CBME.

Overall, it was found that students were satisfied and motivated to do further research after completion of this posting. Now, at this age of CBME, this study may help educators to define and standardize the competencies which are required for research skills for medical undergraduates. However, this study has the following limitations. First, this work is limited to only a small sample of students. Second, the data were collected from the students at the end of their semester, and so competing demand of assignment and examination may have hindered their participation and responses. Third, the findings were based on Students’ self-rating of competence and thus subjected to the participant over or underestimation. Fourth, as the study was time bound, we could not wait for each student to work in his/her own pace which is usually recommended for competency based training.

Finally, the present study being noncomparative in nature, we cannot conclude whether this leads to a better learning outcome. We expect students participated in these particular teaching–learning activities to do better in the field of medical research. There is definitely an opportunity to follow these students longitudinally to measure further development of their research skills. This longitudinal evaluation will assist with providing outcomes regarding the development of research skills and work practice (communication, teamwork, and ethical practice). Following the recommendation of MCI, there is a move toward the development of competency-based approach in medical education in India. It is important to conduct comparative studies in different settings between CBRT and traditional method in achieving the desired research competencies.

Conclusions

The results of this study suggested the use of CBRT as a teaching tool can help medical educators in forming effective undergraduate medical curriculum. Students enjoyed and were satisfied with the associated teaching–learning activities and formative assessment process. However, more studies are required to conclude whether CBRT meets the expectation associated with its implementation and inclusion in undergraduate students’ curriculum.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.