Design of formative assessment model for professional behavior using stages of change theory.

Background: Professionalism is a core competency of physicians. This study was conducted to design a model for formative assessment of professional commitment in medical students according to stages of change theory.
Methods: In this qualitative study, data were collected through literature review & focus group interviews in the Tehran University of Medical Sciences in 2013 and analyzed using content analysis approach.
Results: Review of the literature and results of focus group interviews led to design a formative assessment model of professional commitment in three phases, including pre-contemplation, contemplation, and readiness for behavior change that each one has interventional and assessment components. In the second phase of the study, experts' opinion collected in two main categories: the educational environment (factors related to students, students' assessment and educational program); and administrative problems (factors related to subcultures, policymakers or managers and budget). Moreover, there was a section of recommendations for each category related to curriculum, professors, students, assessments, making culture, the staff and reinforcing administrative factors.
Conclusion: This type of framework analysis made it possible to develop a conceptual model that could be effective on forming the professional commitment and behavioral change in medical students.

Introduction

Professionalism is one of the main competencies that all medical students should develop during their education (1). Over past two decades, professionalism has attracted a lot of attention in medicine which its results can be traced in medical education curriculum content, assessment tools of training programs and criteria for accreditation in medical education institutions which are reviewed and updated frequently (2). Nowadays, patients, doctors and medical professionals receive numerous warnings for lack of professionalism that may lead to damaging patients’ welfare and health as well as physician morale (3). To deal with such lack of professionalism, the medical education system must focus on change and modification of behaviors among medical students. The main aim of this study was to design a formative assessment model for medical students’ professionalism. The suggested model is based on learning theories and focus on Prochaska and Diclemente’s model of behavior change stages.

It is assumed in behavior changing stages that when people try to make a successful individual change they pass predictable stages and periods. These stages include: pre-contemplation, contemplation, readiness, action, and maintenance. Stages of change theory suggest that people in the change process need to pass different stages that require different interventions (4-8). This theory supports the idea that the needs of each individual must be considered before designing interventions to change their behavior; then, the intervention should be designed based on their level of preparedness (7-10). Shirazi et al (2008), Moeini and colleagues (2010) and Abasgholizade (2012) applied the behavior change pattern to improve clinical practice and suggested positive and effective results (6,10,11). At Stanford University, an educational workshop suggested a program for unprofessional behavior change using the behavior change pattern, though, the program had an individual approach and did not systematically use that pattern for formative assessment among medical students. We aimed to design a practical pattern using opinions of some of medical education experts and training officials.

Methods

This study was performed with qualitative approach within several phases. The initial phase was to determine dimensions and the components of the evaluation of professional commitment. All documents including papers, books, reports, any programs and theses in Persian and English language on the evaluation, rehabilitation of professional behavior field and behavioral change pattern were studied. The used keywords for retrieving relevant literature were “professionalism” “professional commitment”, “stages of change theory”, “evaluation”, “assessment” and “medical students”, separately and in combination, in scientific databases such as PubMed, Medline, Google Scholar, Science Direct and Ovid. Persian translations of the used keyboards were used to search the Persian databases including IranMedex, Magiran and SID. References in the literature were also used to find relevant documents. The results were categorized into principles of professional behavior evaluation, and stages of change theory and its application in the professional evaluation.

In the second phase, during multiple correspondences with experts from Stanford University and according to their advice for using theory of behavior change stages to handle unprofessional behavior, we designed and developed a primary pattern.

In the third phase, the prepared primary pattern was discussed in a focus group discussion followed by a deep interview with ten experts in medical education. The experts had experience of educational and executive management in medical educational system and had some research in professionalism commitment or medical ethics fields. The comments of the experts were collected. Interview questions were focused on efficacy of the pattern, challenges of efficiency as well as barriers of effectiveness in modifying professional behavior among Iranian medical students.

Identified experts were invited verbally or by invitation letter and their consent for participating in group discussion and interview session was obtained. One day before interview session, the primary pattern and interview questions were sent to them via email. All sessions were recorded and transcribed accordingly. The chief topics were coded and classified. Sampling continued until data saturation occurred. Final formative assessment pattern for professional commitment was designed and modified according to points expressed in the sessions and interviews.

Results

Recommended pattern for formative assessment of professional behavior

Providing feedback to students’ unprofessional behavior is necessary for their rehabilitation. Since the clinical interaction to observe unprofessional behavior occurs during internship, the intern’s behavior should be followed. For providing better feedback master of the ward held some meetings and talked with the interns and residents about unprofessional behavior. When unprofessional behaviors persisted, the students’ educational ward might have been changed or their course extended between 1-6 months, depending on the type and severity of abuse. One of the scientific group members was selected as program chief who continuously supervised all students’ activities and discussed on their behavior. Students were encouraged to talk about their problems in a confidential and supportive environment. The program chief should evaluate the students’ readiness to change behavior, and then consider essential intervention.

Pre-contemplation phase

In this phase, students refused their mistakes and were not ready to change their behaviors. In this phase the most important intervention, more than clarifying expectations, is showing how far the student behavior is from desired behavior. The chief of program transferred the right of choosing evaluator (faculties, students or peers) to the students themselves and prepared all of the coordination for their behavioral assessment by checklist. The evaluator asked other students to complete an evaluation form in order to assess the behaviors of other peers using comparable diagram. Afterwards, the program chief discussed different aspects of their behaviors compared with their peers into some questions as follows:

√ Why did they assess your behavior as being weaker than others?

√ What portion of your role is an undesirable behavior?

The program chief might request some of these activities according to level of student’s behaviors:

√ Moral susceptibility training: work in charities, writing papers on altruism, dutifulness, respect for people’s confidence to their physician and role of physician in term of defending from patients’ rights, etc.

√ Mental health services (mental health assessment, consultation, cognitive behaviors, etc.)

It was expected that students would have challenge feelings, accept their role in negative assessment of their behaviors by others and think deeply about it (Table 1).

Table 1

A brief formative evaluation pattern for professionalism

Stage of change	Definition	Manifestation	Intervention	Tool	Expected result
Pre-contemplation	Lack of knowledge of problems, denial or apathy	Denial, knowledge deficiency, projection, want to place blame on someone or something	Understand student point of viewState problem/determine commentsExplain the expectationstake them into challenging situation and showing distance between their behaviors with suitable ones	Comparative Performance TableMSFBehavioral checklist	Challenge between aims and behaviorsIncrease awareness and move to next phase
Contemplation	Orientation with uncertainty	Went into discussion, uncertainty on him/herself, resistance to change program	Empathy with students troubles Describing causes Determine the barrier	MSFReflective writing	Increase of challengeReducing the barrier
Readiness	Determined to change	Awareness, accept problems and be sure about change	CollaborationAdherence to commitmentsHelp develop a practical plan	Peer assessmentReflective writing	Change of behavior

Contemplation phase

In this phase, students had uncertainty about changing their behaviors. They sometimes show professional behavior and sometimes undesirable behaviors. During discussion sessions, they accepted the need for behavior changes but they failed to show suitable behaviors in all situations. The most important work in this phase was empathy and helping students to find barriers to behavioral changes and its facilitators. In the interview with students, the program chief tried to force students to continue thinking about this situation and to determine the barriers and opportunities to their behavior changes as a result.

√ What causes lack of changing?

√ What were factors responsible for lack of changing in your behavior? What were the barriers to behavior change?

√ Do you think that you need to learn some tools to recognize behavior changes?

√ What things, people or program are helpful for you to change your behavior?

√ Do you think about learning something new which may change your behavior?

Then, the program chief with receiving help from students prepared a program for resolving barriers of behavior change. In order to help students pass this phase, the program chief might request some of the following activities:

√ Personal development counseling (time management, emotional intelligence, etc.)

√ Mental health services (mental health assessment, consultation, cognitive therapy, etc.)

√ Participating in training courses for communicative skills, problem solving and team working

The program chief used multi source function (MSF) for assessment of students’ behaviors and requested the students to write about their contemplation on professional behavior changes in a clinical environment. It was expected in this phase that students could show better control on their barriers.

Readiness

In this phase, students intensively wanted to change their behaviors and were confident that they can change their behaviors. In this phase, the program chief introduced all regulations relating to behavior changes. Further focus on behavior modification required students to write about contemplation and to continue evaluation of own performance. In this phase, students might have needed to be peer assessed frequently to assure about their own professional performance. If students performed suitably for six months, they would pass the rehabilitation phase successfully.

It seemed that in each educational year, one to three students needed to rehabilitation program. If the students were in the contemplation and readiness phases, there was no need to other phases. The program chief reports the details of behavior changes and rehabilitation phases to the next chief of the ward and one of the faculties to control students’ program. If there was a need to personal development and communication skills programs, the program chief would arrange it. To follow up, the program chief could request assessment of students' behavior from the ward physicians, peers and residents.

Results of focus discussion groups and interviews

In focus discussion groups section, the participated experts believed that with taking into consideration of the challenges, problems and modification of existing substructures suggested pattern will be effective. Thus, they proposed some challenges and problems in two main categories: challenges of educational environment and executive problems (Table 2).

Table 2

Challenges of model and recommendations

Theme	Category	Subcategory	Quotations	Recommendations
Educational environment issues	Teacher related factors	Time deficiency for professional commitment education and evaluation	“our teachers do not have enough time, a few specific teachers may have time and energy for students, but few”	1.Increase rates of faculty for work in this area2. Formal education training for teachers
		Lack of knowledge in behavioral science	“Maybe some teachers who are directly responsible for the student did not have some features that are needed for correct behavior because they did not pass any courses about it ”	3. Empowering teachers in Behavioral Science4. Enhancing and promoting a culture of feedback
		Lack of feedback culture	“The biggest challenge is that in our culture we have no such discussion and teacher-student relation atmosphere is not that kind”
	Student related factors	Considerable influence of peers	“ there is a large gap between our old and new generation, such discussion may also have a weak result, now they take considerable effect from peer review and peer education more than media and take their roles”	1. Accept the consequences of behavior by students2. Punishing students if the offending behavior continues3. Strengthening students in professional commitment4. Engage students in activities in the field of professional commitment5. Emotional support for students6. Provide role model
	Evaluation related factors	Lack of good tools	“Some of these (non-professional behavior) are not subject in self-expression, and this means they cannot be properly assessed”	1. The use of this model in the evaluation2. Strengthen the evaluation system by increasing the number of training evaluations and assessment
		Making no difference for future	“ the disciplinary committee also does not come to deal with these behaviors, however they deal with other issues such as dressing seriously”	3. Strengthening social tools with workshops4. Uniform evaluation
		Being superficial and insufficient	“Our students are very clever and intelligent, so they can deceive themselves and others”	5.No discrimination among students
	Educational program related factors	Rotation in the wards for short periods of time	“The next point is that we do not have a model for this. What does it means? It means that if a student came to my ward, and the model is running, he/ she would be followed-up but they go to another ward and spend no more time in my ward”	1. Increase the frequency of professional commitment education2. Bring Behavioral Science in curriculum3. Increasing and strengthening theoretical interventions4. Implementation of model in wards that are familiar with this area
	Budget related factors	Lack of funding	“ Teachers are not in need of money, can it is not possible to pay them much to provide mentorship since you too are limited ”
Executive issues	Ministerial instructions	Existence of assistant Exam	“All students want to transfer their shift to another and go into quiet room and read their books because the residential exam determines their future”
		Lack of instruction against unprofessional behavior	“Of course I do not want to talk anymore and bring doubt in your work but you do not know the rules, a student had psychosis but there is no any legal right to discontinue her education, also, if a student complains about you, you will be punished”	1. Strengthening infrastructures and supportive policies for professional commitment and ground reform 2. Assessment of teacher and students’ attitude to comply with program performances 3. Modify the rules by eliminating certain by-laws and regulate some new by-laws 4. All stakeholders should be sensitive the issue of professional commitment to education 5. Creating committed professional atmosphere

The educational environment in Iran suffers from current conditions of the universities and ministry instructions, teacher problems (such as lack of time or lack of adequate training skills), lack of assessment of professional commitment, also students' insufficient knowledge of the medical profession (Table 2).

Other related issues with this pattern included lack of feedback culture, accepting more advice from their peers, and lack of strong tools for assessment and identifying the professional behaviors. The experts suggested some activities such as change of rules, detecting and supporting suitable assessment tools for commitment to professionalism, faculty empowerment, presenting pattern of behavior, and finally establishing professional commitment atmosphere (Table 2).

In current study, several factors were identified as factors influencing evaluation of professional commitment. These factors which are effective on formation, reinvention and continuation of professional commitment are reported as: continuous feedback, constructive evaluation, and rehabilitation.

The stage of change behavior theory supports all 3 aforementioned factors. Based on this theory and the opinions received from Stanford University colleagues, pattern of formative assessment of professional commitment was designed and developed in accordance with educational atmosphere in Iran medical universities.

Learning environment problems may lead to perceive inferior status for professional commitment in educational, executive and evaluative staff. Hence, professional commitment is abandoned with different excuses such as shortage of time and budget, and also personalization. According to a study (12), formal training of medical professionalism and its assessment are a must to give medical students basic values and prepare them for social activities. According to this study, considering the current situation, it is better to focus on formative evaluations (12).

Another study (13) recommended that we should pay a rapt attention to strengthening intimate relationships with interns, creating a politely atmosphere between faculty and residents to give a model of professionalism to interns; respecting interns during their education, respecting nurses, and highlighting professional behavior of outstanding faculties (13).

Another study (14) reviewed monitoring program for medical students between 2000 and 2008 and showed that for raising successful students effective trainers for professional commitment are needed (14). Also, the study showed that feedback culture or giving and receiving feedback is an essential factor in changing or establishing a behavior (14). Thus, teachers and students should be involved in professional commitment programs practically.

Motivations to improve performance and ensure learning prompt the need to provide feedback. Without feedback students’ errors remain uncorrected, good performance will not be boosted, empirical capabilities are not acquired and no confidence will be inspired.

Lack of sufficient knowledge about the professional commitment should be considered in assessments of medical students. In most countries, the used methods to assess students’ capabilities and enrolling criteria to examine their professionalism commitment and readiness vary from that of Iran of which is mostly based on students’ knowledge; therefore, they could not be compared flawlessly. Evaluating system for choosing medical student in Iran is considered to be malfunctioning in some parts which may cause serious problems (15). Unfortunately, few valid tools exist to predict the students' capacities regarding personality fit for medical profession and professionalism (15). Nonetheless, it seems that if medical students are supported and respected by the education system at all stages of their training and evaluation, they would promote themselves and progress in their career. However, the role of medical universities in improving student admission process, formal education, and removal of unprofessional behavior should not be neglected.

Summative evaluation could be the most challenging part of our professional commitment model. Considering problems, some of which, including personalization of evaluation, lack of trained teachers for evaluation of professional commitment, defects in reporting abuse, lack of sensitivity to badly-behaved students and officials, and ineffectiveness of results of evaluation in the students’ educational future we should focus on formative method. Therefore, formative evaluation factors should be regarded in designing and developing assessment tools for measuring professional commitment.

Designing a uniform system of evaluation with standard criteria has been reported as the best strategy to increase interns' adherence to professionalism in clinical wards of hospitals affiliated to Tehran University of Medical Sciences (13).

In 1995, at the University of California, School of Medicine, a study was conducted on medical students and in a professional evaluation form unprofessional behavior of first and second-year students recorded and sent to the head of the school. According to this evaluation form, if two or more reports of unprofessional behavior were submitted for a student at the first two years and another one at the third and fourth years, the student's behavior was considered as inappropriate and he/she might be expelled from school even though he/she had passed all of required courses (13).

Evaluation by peers in areas such as affective domain, communication skills and professional commitment is unfruitful unless assessments would be made continuous and daily (16).

All of factors which may lead to failure of medical education and reducing the dignity of the medical profession should be considered thoroughly and authorities should realize that it is necessary to do any required action to reduce students’ professional behavior abuse. In this regard, formative evaluation model based on the stages of change theory is recommended as an effective and helpful model in assessment and promotion of professional commitment in medical students. Based on this model, it is necessary to focus on repetition and rehabilitation to change uncommitted behaviors (1). Also, behavioral interventions should be performed in accordance with the principles of behavioral sciences.

Conclusion

Evaluation of professionalism in medical schools is significantly affected by different ways such as personalization. This issue has become one of the main concerns of medical experts and professors of medical ethics in medical schools. Therefore, to increase students' adherence to professional behavior, a comprehensive and uniform system for training and evaluation of professional commitment as an effective way is emphasized by experts. To address this need, presenting a formative evaluation model and a pilot implementation are recommended. Also, training workshops seem inevitable.