A modified tool for "reflective practice" in medical education: Adaptation of the REFLECT rubric in Persian.

CONTEXT: Reflection is a learnable process that enhances long-life learning, clinical decision-making, and can foster empathy and professionalism. One of the methods for teaching reflection is "reflective writing" that is conducted in "reflective practice." Some tools have been developed to assess "reflective capacity," and The Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT) is one of them. AIMS: This study aimed to adapt the REFLECT rubric in Persian. SETTINGS AND
DESIGN: This quantitative study was conducted in a medical school with the participation of medical interns through census sampling, and in three stages, including translation, pilot study, and main study. SUBJECTS AND METHODS: Persian translation was obtained by the "forward/backward translation" method. We made some changes in the tool and used it in the pilot and main study to confirm validity and reliability. STATISTICAL ANALYSIS USED: Cronbach's alpha coefficient, Pearson correlation, and Cohen's kappa were applied for statistical analysis. Data analysis was performed using SPSS23.
RESULTS: We inserted a numerical value of 1-4 at the reflection levels, and also removed Axis 2 and an optional writing component. In the pilot study, face and content validity was confirmed involving 10 interns and five medical education specialists. Then, 67 interns participated in the main study, and we measured the reliability of the tool by internal consistency through Cronbach's alpha (0.83) and test-retest through correlation coefficient (0.89). The size of the agreement was measured to determine the inter-rater reliability by Cohen's kappa (0.84).
CONCLUSIONS: The modified REFLECT version is a valid and reliable tool that can help us to assess reflective capacity. The use of this tool is recommended for reflective practice in medicine. Copyright:

Introduction

Today emotion in medicine is neglected, and although professional development is still important, it is not considered as an important goal in medical education. Research in pre-1960s shows that the lack of emotions has reduced doctor's effectiveness. At the end of the 1990s, reflection emerged in the literature, and “Medical Humanities” was presented as an approach to connect humanities and medical sciences,[12] and teaching reflection in medicine increased.[1] Reflection means a process in a person's thoughts that related to experience and can be decomposed and interpreted to make awareness. Hence, the created understanding in this process can be used in the same condition in the future.[3] Accordingly, reflection on past experiences will lead to deeper learning and better performance. Moreover, reflection is a base for professional development; hence, practitioners can aware of their inner values and attitudes.[4] Besides, reflective and critical thinking can enhance “long-life learning” and improve “clinical decision-making,”[56] also helps physicians to choose difficult or ethical decisions when encountering complicated cases in clinical practice.[7]

On the other hand, reflection can foster empathy and professionalism, and “medical humanities” is an approach that presented reflective writing as a valuable tool to improve “reflective capacity.”[8] This capacity is an important ability that lets the physicians be mindful, interested, aware, and prepared to identify and correct faults,[9] therefore fostering reflection improves professionalism, which is a core competency in medicine.[310]

Hence, reflection is a learnable process, so teaching methods can be used to foster it, and one of the teaching methods of reflection is reflective writing.[10111213] An important tip in this method is offering guidance and feedback to learners to make particular skills.[1] Teaching reflective writing is conducted in “Reflective practice,” which is an approach in medical education and considers medical students’ and practitioner concerns. These practices are “the higher-order intellectual and affective activities” in practitioners that involve “to critically analyze and evaluate their experiences to lead to new understandings and appreciation of the way they think and operate in the clinical setting.” In the traditional approach of clinical education, physicians are evaluated in skills and outcomes, and trainees gain competencies in procedures only through “repetitive guided practice.”[1]

Assessing reflection can increase learning, and medical teachers can analyze “problem-solving” and clinical “decision-making” process of learners too. Reflective tools are used for assessing reflection levels in narratives. These tools were prepared by theories of reflection and reflective thinking[514] that can have a positive educational impact if we presented to learners as guidance before practice.[5] Some studies have used these reflective tools for teaching reflection.[101516]

Various tools have been developed to assess reflection that addressed reflection levels and guided feedback.[17181920] Mezirow proposed a classification for reflective evaluation that is divided into three parts, including nonreflective, reflective, and critical reflection. This model was used for reflective journals and blogs. Besides, Boud et al. presented a conceptual framework that has been used in seven stages to evaluate the reflective process in journals and blogs. The combination of these two models can be used to create a deeper understanding of reflective writing.[52122] Some studies applied a combination of the Mezirow's model and the Boud et al. framework to identify the reflection process. In general, reflection divides into reflection and critical reflection and includes seven steps. Step 1 is without reflection, steps 2–4 are the reflection, and steps 5–7 are the critical reflection. Critical reflection is at a higher level of reflection. In reflection, new insights and understanding are obtained; but in critical reflection, understanding is much deeper and dilemmas that have been considered by a person before, are identified.[52122232425] Tsingos-Lucas et al. used a reflective tool for assessing reflection in pharmacy students and assessed the reflective narratives of students. This tool was produced in the 7 stages and 3 levels of reflections that its details were extracted from a previous tool used in dentistry education.[16212226] A modified tool was provided by Kember et al., to evaluate reflection levels in students, which has four levels “habitual action, understanding, reflection, and critical reflection.”[27] Moreover, the “Reflection Evaluation for Learners’ Enhanced Competencies Tool” (REFLECT) rubric has developed and recommended by Wald et al. aimed to improve reflection, self-directed learning, and self-assessment as a qualitative and formative assessment to provide written feedback to each learner.[10]

Among the tools reviewed, we chose the REFLECT rubric that was introduced principled and more comprehensive than others. As explained, reflection needs to be trained and should be assessed to improve professional development. There is no tool in Persian to teach and assess reflection, so we performed this study to present the modified REFLECT as a valid and reliable tool in Persian Version.

Subjects and Methods

This study performed in a quantitative methodology at a medical school and in three phases for adapting the REFLECT rubric that included translation, pilot study, and main study. The sampling method was census, and the study participants were medical students of internal medicine internships.

The Reflection Evaluation for Learners’ Enhanced Competencies Tool rubric

An innovative rubric called REFLECT was developed by Wald et al. at Brown University after a comprehensive literature review and several phases, that assess reflective levels in medical students’ narratives. This rubric helps to improve reflection by presenting written feedback to students and is a psychometric tool that was confirmed its “interrater reliability, face validity, feasibility, and acceptability.” This rubric was designed to analyze writing through qualitative and formative assessment that improves providing feedback to learners. The REFLECT includes four reflection levels includes (Level 1: Usual action without reflection; Level 2: Thoughtful action or introspection; Level 3: Reflection; and Level 3: Critical Reflection). In addition, the REFLECT focuses the criteria for each level and explains five writing components in a narration (”Range of writing, Presence, Description of the conflict or disorienting dilemma, Attention to emotions, Analysis and construction of meaning”). Furthermore, this tool evaluates narratives for “transformative reflection, and learning and confirmatory learning.” The process of utilizing this tool for narrative analysis involves four stages that include: (1) reading the whole reflective narrative; (2) fragmenting and analyzing the components of narration; (3) investigating the whole narrative taking into account the second stage or “Gestalt;” and (4) justifying and confirming the reflection levels and learning results that have assigned to the narration). The original version of the REFLECT in the English language was produced for formative assessment and has no scoring for reflection levels.[10]

We selected the REFLECT rubric after reviewing the literature as a proper tool for assessing reflection in medical narratives because the levels of reflection and the writing components have described well, and we could enhance providing feedback to learners too. We received permission from one of the rubric's providers for using this tool in our study.

The modified Reflection Evaluation for Learners’ Enhanced Competencies Tool

Forward/backward translation

The tool was translated into Persian by three medical education specialists. We selected the final translation after agreement by the panel of researchers, then an English translator translated it back to English. We compared two English translations (original and back-translated versions), and translations were very closely matched, then Persian translation was approved.

Pilot study

Reflection levels and writing components of the tool were approved by five medical education specialists through reviewing the related literature in the field of reflection and narrative writing.

Participants

Ten medical interns participated in this phase (nonparticipants in the main study).

Validity

The translated tool was used in a pilot study, and some changes were made in the tool based on better understanding from the students’ view and easy application for analyzing Persian medical narratives. A confirmed medical narrative was selected by a narrative analyst and gave to the medical interns along with the tool. We explained the tool and questioned each student separately about levels and components in the narrative based on the tool, and noted the ambiguous points. Then, we held a meeting with the participation of all the research members and corrected the ambiguities.

Main study

Participants included 67 medical interns.

Reliability

We asked the interns to write reflective narratives about clinical encounters with their patients two times, at an interval of 2 months in test and re-test. Then, two raters assessed students’ narratives using the REFLECT rubric. We measured internal consistency through Cronbach's alpha coefficients to determine reliability. We measured the correlation of scores between the writing components’ and time stability in test-retest too.

The inter-rater reliability

We measured the size of the agreement between two raters by Cohen's kappa coefficient. Two raters assessed 40 Student's narratives independently, and then we measured the coefficient of agreement between them.

Data analysis

All analyses were performed using SPSS-23 software (IBM, Armonk, NY, USA), and significance in statistical tests was considered as P < 0.05.

Ethics

Ethical approval for this study was obtained from the University Research Center with the ethics code. IR.MUI.REC.1396.3.472.

Results

We modified the REFLECT after verifying the final translation. Changes were made to the tool based on the pilot study.

The changes included

The removal of Axis 2 about “critical reflection,” and the removal of the “optional minor criterion”

Determining a 4-degree Likert scale in four Levels in reflection, “from habitual action to critical reflection.” Therefore, we added a numeric value in four reflection levels to assess changes in learners’ reflective capacity.

Validity

Face and content validity of the tool was confirmed by five medical education specialists who were well-versed in professionalism. In addition, the tool was revised for ease of use and understanding of each component, with the comments of 10 students (6 men [60%] and 4 women [40%]) participating in the pilot study, and the necessary modifications were made.

Participants of the main study were 67 medical students (response rate 97.06%) included 32 women (47.8%) and 35 men (52.2%) from 23 to 28 years old with a mean age of 24.05 ± 1.62 years.

Reliability

Interrater reliability

Kappa agreement coefficient was measured (0.84, and P < 0.000) that expressed a very good agreement between raters.

Internal consistency

The raters assessed 67 written narratives of medical students based on the tool, and Cronbach's alpha coefficient was 0.83.

Intra-scale correlations

Spearman–Brown correlation coefficient was measured. Each writing components with others showed a significant correlation (P < 0.05) [Table 1].

Table 1

Correlations between writing components

Writing components	Correlation coefficient
	1	2	3	4	5***
1. “Range of writing”	1.000	0.580**	0.681**	0.537**	0.227
2. “Presence”	0.580**	1.000	0.738**	0.666**	0.379**
3. “Description of the conflict or disorienting dilemma”	0.681**	0.738**	1.000	0.615**	0.277*
4. “Attention to emotions”	0.537**	0.666**	0.615**	1.000	0.273*
5. “Analysis and construction of meaning”	0.227	0.379**	0.277*	0.273*	1.000
Total score	0.767**	0.875**	0.854**	0.864**	0.367**

*P<0.05, **P<0.000 and (n=67), ***1-5: Writing components

Test-retest reliability

We conducted two test sessions with a 2-month interval for medical students, and to evaluate the reliability of the tool, no educational intervention was provided for medical students. The tool stability was measured by the Spearman–Brown correlation coefficient (0.89 and P < 0.000). Therefore, the scores of the test and retest were highly correlated (P < 0.05) [Table 2].

Table 2

Correlation between writing components and total score in test-retest

Test Writing components	Retest
	Correlation coefficient					Total score of re-test
	1	2	3	4	5***
1. “Range of writing”	0.863**	0.491**	0.604**	0.436**	0.328**	0.749**
2. “Presence”	0.531**	0.675**	0.565**	0.525**	0.245*	0.726**
3. “Description of the conflict or disorienting dilemma”	0.531**	0.432**	0.580**	0.513**	0.418**	0.664**
4. “Attention to emotions”	0.478**	0.643**	0.509**	0.785**	0.314*	0.779**
5. “Analysis and construction of meaning”	0.392**	0.304*	0.208*	0.253*	0.386**	0.386**
Total score of test	0.711**	0.681**	0.653**	0.675**	0.405**	0.869**

*P<0.05, **P<0.000 and (n=67), ***1-5: Writing components

The modified REFLECT in English was checked by one of the developers of the original version.[10] A copy of the final modified REFLECT is attached [Appendix 1].

Appendix 1

English version of the modified REFLECT rubric (“REFLECT: Reflection Evaluation For Learners’ Enhanced Competencies Tool”)

	Level
	Ranking	1	2	3	4
Row	Writing components	Usual action without reflection	Thoughtful action or introspection	Reflection	Critical reflection
1	“Range of writing”	Descriptive, superficial writing approach (event report, vague impressions) without reflection or introspection	Descriptive, accurate writing and non-reflective impressions approach	Moving toward reflection, beyond reporting or descriptive writing (e.g. trying to understand, question or analyze the event)	Exploring and criticizing the assumptions, values, beliefs and/or biases and outcomes of the action (present and future)
2	“Presence” (as a sense of being present in the narrative)	No relative presence of the writer’s sense	Relative presence of the writer’s sense	Full and immense presence of the writer’s sense	Full presence of the writer’s sense
3	“Description of the conflict or disorienting dilemma” (causing desperation)	No description of the discomfort dilemma, conflict, challenge or concern	No or weak description of the dilemma, conflict, challenge or concern	Description of the discomfort dilemma, conflict, challenge or concern	Full description of the discomfort dilemma, conflict, challenge or concern encompassing various views, evaluation of alternative descriptions and challenging assumptions
4	“Attention to emotions”	Little or no recognition of or attention to emotions	Recognition of emotions, yet without exploration or attention	Recognition, exploration and attention to emotions	Recognition, exploration and attention to emotions and gaining an emotional insight
5	“Analysis and construction of meaning”	No analysis or construction of meaning	Little or unclear analysis or construction of meaning	Relative analysis and construction of meaning	Comprehensive analysis and construction of meaning

Discussion

So far, few tools developed to assess reflection. Considering that there was no tool for this purpose in Persian, we conducted this study to adapt the REFLECT rubric in this language. The modified tool includes four reflection levels and five writing components, and guides learners and educators for practice and giving feedback. The results of this study showed that the modified REFLECT is a valid and reliable tool for teaching and assessing reflection that can be used in formative and summative assessments at every level of the learning. Considering no educational intervention was provided to students, as shown in Table 1, the correlation of writing components 1 with 5 was not significant. Because attaining high levels of reflection in Component 5 require training and practice, this should be achieved through educational intervention. Thus, in general, component 5 is less correlated with other components. However, as shown in Table 2, the components of writing in the test and retest were correlated significantly with each other.

The REFLECT rubric in the original version was applied for formative assessment during reflection training that helped to give written feedback to learners’ narratives.[10] In our study, the numeric value was used at reflection levels to do a summative assessment, and determine the impact of teaching reflection in future educational interventions, and assess changes in learners’ capacity. Furthermore, because few of Persian written narratives had “critical reflection,” we deleted the Axis 2 that contains “transformational learning,” and “confirmatory learning” which need a deep training and practicing reflective writing.

Some studies emphasized that teaching reflection through writing can make a significant improvement in “self-awareness, professionalism, and humanism,” and help students to learn empathy and promote compassion in clinical skills.[328] There are few tools for teaching and evaluating reflective narratives, and some studies applied the REFLECT.

McNeill et al. examined how practitioners applying reflection on their practice and writing about it in e-portfolio. The researchers used a grading system to determine the level of reflection and analyzed using a qualitative method. Findings showed that practitioners presented reflection in the documents.[18] Hoffman et al.'s study addressed the relation of reflection and professionalism errors in medical students. This study, that was done in a retrospective case–control in medical students at Indiana University showed that there is a significant correlation between them.[29] They used a validated tool to assess reflection in medical students[17] that included seven reflection levels.[29] Tracey et al. used the REFLECT rubric and investigated the application of reflective assignments in instructional designers. In general, the use of this tool and providing feedback was reported usefully for improving reflection and to address the weaknesses of the learners on the reflection levels.[30] Patterson et al. used the REFLECT rubric to analyze reflective narratives in medical students. Assessing with this tool showed that reflective skill improved in 50% of students after teaching and practicing reflection.[31] Huang et al. held the narrative medicine program and used the REFLECT rubric to assess students’ narratives. This tool was used as an educational tool during the program, not for summative assessment.[32] The mentioned studies did not use the REFLECT for summative assessment of learners. However, our study was intended to use this tool for summative and formative assessments.

Miller-Kuhlmann et al. compared two rubrics which had been used for the reflection assessment in previous studies. These tools included the REFLECT and the “Reflection-on-Action” which were based on the reflection theory. Both the rubrics were used to help teachers in educating and evaluating narrative writings. In this study for comparing the tools quantitatively, the REFLECT rubric was ranked from 1 to 4 in reflection levels, such as in our study. Furthermore, these tools were compared to the aspects of assessing reflective writing, strengths, and weaknesses. Both tools have advantages and challenges in education and are used to evaluate reflective narratives, so educators must choose one with considering their unique benefits and educational challenges. For example, the REFLECT rubric provides more details for presenting feedback on reflection levels, the length of time, it takes for training is longer, but using the Reflection-on-Action rubric is simpler in practice.[33]

Considering the results of previous studies on the positive educational effects of using the REFLECT rubric in analyzing reflection levels and improving reflection in learners, we recommend applying the modified version of this valid and reliable tool to assess in the summative form besides the formative evaluation. In addition, using this tool, due to its concurrent capabilities of training and assessing, can be a novelty solution to the formal educational system to assess reflective capacity in learners which is a prerequisite for professional development.

In addition, strength point of applying the modified version is determining learners’ changes in educational interventions. Using this tool is useful for universities that intend to present “reflective practice” for the first time to medical students. In Persian, we have no tool to assess reflection, so this tool can be used for teaching reflection during medical ethics or professionalism as a teaching method in the formal medical curriculum. It is recommended to use the original version of REFLECT to foster “critical reflection” and “transformative learning.” This tool can be used in reflective practice for medical students and health-care professionals.

We were restricted to find related literature about this tool because it is not widely used yet. Furthermore, the tools related to teaching and assessing reflection were limited. However, we tried to address the implications of this tool by reviewing the limited studies that had been done.

Conclusions

Improving reflection can help practitioners in clinical and ethical decision-making and problem-solving to promote clinical performance in physicians. Moreover, teaching reflection is a basis to promote professionalism. The modified REFLECT tool is a valid and reliable tool that can help us to assess reflective capacity in medical students and can promote reflection through formative and summative assessments and presenting feedback about reflection levels in narrative components. Therefore, introducing and using this tool in reflective practice can be effective in professional development.

Financial support and sponsorship

This study was funded by the National Agency for Strategic Research in Medical Education. Tehran. Iran. Grant No. 970001, and was part of a project with the ethics code. IR.MUI.REC.1396.3.472 at Isfahan University of Medical Sciences in Iran.

Conflicts of interest

There are no conflicts of interest.