Development an instrument assessing residents' attitude towards professionalism lapses in training.

PURPOSE: Medical professionalism is a fundamental competency for all physicians and continuous development of professionalism during residency training is crucial. The purpose of this study was to develop an instrument assessing residents' attitudes toward unprofessional behaviors.
METHODS: A questionnaire survey was conducted in cooperation with the Korea Resident Association from May to July 2013. A total of 317 residents from seven university-affiliated hospitals in South Korea participated in the survey.
RESULTS: In the exploratory factor analysis, seven factors were extracted from the data; factor loadings of the 44 items ranged between 0.40 and 0.89. Through iterative discussion, three items below 0.45 were deleted and one additional item was removed due to its irrelevance. Twelve items included in Factor 1 were divided into two different categories. A final version of the questionnaire containing 40 items in eight categories was assessed using confirmatory factor analysis. It was deemed to have a good fit; the root mean square error of approximation and comparative fit index were 0.07 and 0.9, respectively. The reliability (Cronbach's α) of the inventory was 0.97.
CONCLUSION: The items of this instrument encompass a broad range of residents' behaviors in clinical practice, research, and publication. In addition, it includes some types of misconduct that can be considered unique features of the authors' cultural backgrounds. We recommend this instrument as an assessment tool to diagnose residents' perceptions and attitudes towards professionalism lapses and to provide insight regarding potential improvement in professionalism education.

Introduction

During the last two decades, attention on medical professionalism has increased as a fundamental competence of physicians [1]. In Western countries, various studies have been conducted to understand medical professionalism and embed it in the education field. As a result of previous studies, a concrete concept of medical professionalism has been drawn up, and essential values, principles, and codes of conduct for doctors have been established [2,3,4]. In addition, other previous studies have established a framework and standards for assessing medical professionalism [2,5], which have been employed in evaluating the effects of professionalism education [3,4]. Most medical educators now agree with the importance of evaluating medical professionalism since it induces physicians or students to compensate for their professional defects [5,6].

To evaluate professionalism, the process of identifying unprofessional behavior is necessary as a basic step [7]. Behavioral assessments are proxy measures, resting on the assumption that observed behaviors are reflective of underlying dimensions [3]. Therefore, earlier studies on medical professionalism have focused on the assessment of unprofessional behavior [8]. In particular, many previous studies have paid attention to the professional lapses of residents due to the concern that unprofessional behaviors during training may lead to unprofessional behavior later on when in unsupervised practice [9,10]. Evidence for this assumption has come from several studies, which have demonstrated that physicians who have been disciplined for unprofessional behavior during their practice were more likely to have records of irresponsible or unprofessional behavior during medical school years or residency training [9]. Although some current studies have begun to adopt direct evaluation of professional behavior, the assessment of unprofessional behavior is more widely used in the medical education field as the definition of professionalism remains incomplete [8]. However, studies for assessing neither unprofessional behavior, nor direct evaluation of professional behavior have not been widely conducted in Asian cultures.

The residency is a critical period in the formation of professional identity and development of professionalism, and is accompanied by the struggle to operationalize the learning of professional behaviors. Knowing more about residents’ perceptions of unprofessional behaviors and the prevalence of lapses in professionalism would enhance definitions and facilitate support for the development of professionalism skills and behaviors at the graduate level [9,11]. A valid and reliable instrument to assess unprofessional behaviors can help us to identify program needs and training gaps at both individual and environmental levels.

As a set of values, medical professionalism is inevitably influenced by a certain culture in which it is practiced. Some studies have drawn attention to cultural differences in the conceptualization of medical professionalism in response to varied societal needs [12,13]. An individual’s attitude towards unprofessional behavior reflects the context of their cultural and social background. Therefore, we need professionalism assessment tools which can embrace both specific cultural backgrounds of medical practices and core common factors of medical professionalism regardless of difference in culture.

In this study, the authors aimed to develop an inventory assessing residents’ attitudes towards unprofessional behaviors which reflects medical professionalism in our own unique medical contexts and cultural backgrounds.

Subjects and methods

An inventory assessing residents’ attitudes toward unprofessional conduct was developed in three steps: questionnaire construction, questionnaire survey, and psychometric analysis of the inventory (Fig. 1).

Fig. 1.

Development Process of the Instrument to Assess Residents’ Attitudes towards Professionalism Lapses

1. Development of operational definition of medical professionalism

As the first step in constructing survey items, an operational definition of medical professionalism was developed through literature review. Professionalism recommendations by the Accreditation Council for Graduate Medical Education (common program requirements), American Board of Internal Medicine (the physician charter), as well as definitions of professionalism by the Association of American Medical Colleges and other efforts [5,14,15,16] to establish a professionalism framework were thoroughly reviewed by the authors.

After extracting the core components of medical professionalism from the previous literature, the authors selected those concepts that were repeatedly mentioned and created an operational definition of medical professionalism with the follow domains: (1) clinical competence [5,14,15]; (2) commitment to ongoing professional development [5,14]; (3) adherence to ethical standards [14,15,16]; (4) promotion of the public good [14,15]; (5) adherence to core human values such as integrity, honesty, altruism, humility, respect for diversity, and transparency with respect to potential conflicts of interest [5,14,15]; (6) taking responsibility for their own health and well-being and that of their colleagues [14]; (7) physician-led regulation [5,15]; (8) understanding that physicians are accountable to those served, to society, to their profession, and to themselves [5,14,15]; (9) establishing cooperative relations with colleagues and other healthcare professions [5].

2. The development of a draft questionnaire

Based upon our operational definition of medical professionalism, in order to develop questionnaire items, the authors focused on common issues closely related with residency training in everyday practice. A draft questionnaire construction was begun under the frame of six domains: (1) clinical competence; (2) commitment to ongoing professional development; (3) adherence to ethical standards; (4) adherence to core human values, particularly integrity and honesty; (5) transparency with respect to potential conflicts of interest; (6) establishing cooperative relations with colleagues and other healthcare professionals.

We searched previous studies on unprofessional behavior by physicians using the keywords “misconduct,” “unethical behavior,” and “unprofessionalism.” Through content analysis of the articles [7,8,9,10], the common descriptors of unprofessional and unethical behaviors during residency training were extracted. To reflect our unique medical culture and actual training environment in the questionnaire items, the types of resident misconduct identified by the authors of previous studies [17] were incorporated into the items. Finally, we selected 39 unprofessional behaviors considered to reflect the authentic and realistic experience in the residency-training environment of Korea.

The questionnaire items consisted of asking residents about their attitudes toward misconduct under the leading question: What do you think of residents’ conduct with regard to each of the following types of behavior? Please select one answer that best reflects your opinion or views. The answer options consisted of a 4-point Likert scale from 1 (absolutely cannot be done), 2 (cannot be done), 3 (can be done depending on circumstances), to 4 (usually can be done).

3. Modification of questionnaire items by content validity assessment

Content validity was assessed by seven panelists: two residents (third- and fourth-year), three clinical faculty members (internal medicine, surgery, and pathology), and two experts in educational measurement. The residents and clinical faculty members focused on evaluating the appropriateness and authenticity of the described clinical situations and misconduct, and the clarity of expression in the questionnaire. The experts in educational measurement mainly assessed the appropriateness of the Likert scale for the response options. Following the expert panel’s suggestions, minor amendments were made: (1) the addition of five items which were commonly occurred in real clinical settings of South Korea (providing incorrect information to patients to conceal one’s ignorance; prescribing patients with unnecessary medication for doctor’s benefit; not reporting patient’s medical condition to supervisor to avoid blame; ignoring opinion of other healthcare professionals; criticizing colleague doctors or other healthcare professionals in front of patients), (2) sequencing items according to the relevance of contents, and (3) arranging items in order of the degree of severity of misconduct. As a result, a 44-item questionnaire was developed.

4. Survey

A survey using the 44-item questionnaire was conducted in cooperation with the Korea Resident Association from May to July 2013. The Korea Resident Association consists of resident representatives from training hospitals nationwide. Among the resident representatives, seven members volunteered to help with this survey, all of whom were from university-affiliated hospitals located in various regions of South Korea: Seoul (four hospitals), Goyang (one hospital), Suwon (one hospital), and Daejeon (one hospital). They helped us to recruit participants, and distributed and collected the survey forms. A small gratuity (US $90) was given to the resident representatives. All participation occurred on a voluntary basis after obtaining informed consent. Residents who agreed to participate in the study were asked to respond to the questionnaire. The data were collected anonymously, and no personal identifiers were used. A total of 317 residents completed the questionnaire.

5. The assessment of construct validity and reliability

We examined the construct validity of the questionnaire by factor analysis. Since prior knowledge of residents’ attitudes toward unprofessional conduct is lacking, we extracted factors by exploratory factor analysis and then verified the structure of variables by confirmatory factor analysis. A principal component and varimax rotation was used for the exploratory factor analysis.

In the confirmatory factor analysis of this study, two goodness-of-fit indices were used to evaluate the structural equation model. The first index was the root mean square error of approximation (RMSEA), which is an absolute fit measure that takes into account both structural error and parsimony, and which is not affected by sample size. The second was a comparative fit index (CFI), which is based on an assumption that is more realistic and that is also not affected by sample size.

The coefficient of internal consistency reliability was assessed by Cronbach’s α. For statistical analyses, IBM SPSS Statistics ver. 20.0 (IBM Corp., Armonk, USA) and Amos ver. 16.0 (IBM Corp.) were used.

Results

1. Exploratory factor analysis

After content validation, the remaining 44 items were suitable for factor analysis in terms of normal distribution, equal variance, type of scale (interval scale), and mutual independence. Bartlett’s test resulted in χ2=11,325.77 (p<0.001) and the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.94, indicating the data were fit for factor analysis. The results of the exploratory factor analysis are shown in Table 1. Seven factors were extracted from the data and factor loadings of the items ranged between 0.40 and 0.89.

Table 1.

Attributes Grouped Together in the Exploratory Factor Analysis for 44 Items

	Extracted factors
	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7
Item 19	0.738
Item 18	0.708
Item 17	0.692
Item 22	0.675
Item 20	0.673
Item 23	0.639
Item 15	0.635
Item 24	0.608
Item 16	0.592
Item 26	0.557
Item 21	0.552
Item 25	0.530
Item 30[a)]	0.430
Item 40		0.826
Item 37		0.825
Item 38		0.823
Item 39		0.773
Item 41		0.699
Item 34		0.691
Item 35		0.679
Item 33		0.671
Item 36		0.629
Item 11			0.808
Item 12			0.765
Item 10			0.712
Item 13			0.511
Item 8				0.889
Item 7				0.789
Item 9				0.740
Item 14[a)]				0.436
Item 2					0.679
Item 3					0.617
Item 5					0.600
Item 1					0.564
Item 6[a)]					0.477
Item 4[a)]					0.419
Item 29						0.667
Item 27						0.533
Item 28						0.531
Item 32						0.499
Item 31						0.467
Item 42							0.769
Item 44							0.697
Item 43							0.694
Eigenvalue	7.53	7.51	3.34	3.32	3.01	2.68	2.63
Percentage of explained variance	17.10	17.08	7.58	7.55	6.84	6.08	5.97

Items were deleted at the final analysis.

We conducted iterative discussion on three items that had factor loadings of less than 0.45: (1) “forcing junior residents or students to attend classes as their substitutes, for master’s or doctoral degree programs”; (2) “absence from work without prior notice to clinical team”; and (3) “insisting on doctors’ opinions regardless of patients’/caregivers’ preferences.” Among the three items, we deleted “substitute attendance” from the questionnaire because we regarded it as a situation that was specific to the authors’ institution, which requires mandatory attendance for some graduate courses. The second and third items were also removed because the authors could not reach an agreement regarding whether those behaviors were widely tolerated or only allowed in specific situations. One additional item (Item 6 in Table 1), “drinking that leads to residents having trouble working on the following day,” was removed from the inventory. Although its factor loading value was above 0.45, this item was not relevant to other items that were categorized within the same factor, that is, “misconduct lacking in respect for medical colleagues.” As a result, a 40-item questionnaire under seven categories was constructed.

2. Confirmatory factor analysis

Following the iterative and intensive discussion of the questionnaire, we decided that it was preferable to divide the 12 items in Factor 1 into two categories according to their content relevance: professionalism lapse related to medical practice (Category 1) and behaviors in contravention to patient respect and confidentiality (Category 7). As a result, we re-categorized the 40 items into eight factors (Table 2) and named them as follows: (1) dishonesty and unsafe practice in patient care, (2) conflicts of interest, (3) misconduct in research and publication, (4) irresponsible conduct while on duty, (5) physician impairment, (6) lack of respect for colleagues, (7) disrespectful to patients and breaching confidentiality, and (8) misconduct in authorship.

Table 2.

An Instrument Assessing Residents’ Attitudes towards Professionalism Lapses (40 Items)

Factor	Item[a)]
1. Dishonesty and unsafe practice in patient care	1. Charting unexamined physical findings
	2. Neglect in improving clinical knowledge and skills
	3. Practicing medicine beyond one’s capability on a voluntary basis
	4. Practicing medicine beyond one’s capability as ordered
	5. Providing incorrect information to patients to conceal one’s ignorance
	6. Prescribing patients with unnecessary medication for doctor’s benefit
	7. Not reporting patient’s medical condition to supervisor to avoid blame
	8. Concealing one’s medical error
	9. Overlooking colleagues’ medical errors
2. Conflicts of interest	10. Taking money or gifts from patients
	11. Conferring benefits to patients after taking money or gifts
	12. Having private relationship with patients
3. Misconduct in research and publication	13. Citing articles without referencing
	14. Combining passages from the Internet into an article
	15. Purchasing an article
	16. Referencing unread articles
	17. Fabricating whole data
	18. Fabricating partial data
	19. Altering data
	20. Falsifying statistical analysis process
	21. Submitting the same paper to different journals after modifying some details
4. Irresponsible conduct while on duty	22. Being late for work
	23. Sneaking out of hospital while on duty
	24. Sneaking out of hospital while on call
	25. Leaving work without completing hand over
5. Physician impairment	26. Use of illegal drugs
	27. Prescribing illegal drugs
	28. Inappropriate physical contact during physical examination
6. Disrespectful behavior to colleagues	29. Ignoring opinion of other healthcare professionals
	30. Treating other healthcare professionals in an abusive manner
	31. Criticizing colleague doctors or other healthcare professionals in front of patients
	32. Verbally or physically abusing junior doctors or students
7. Being disrespectful to patients and breaching confidentiality	33. Talking about patients in public places
	34. Talking about patients in personal spaces or on the internet
	35. Leaking patients’ medical information
	36. Humiliating patients with rude words
	37. Talking about patients with colleagues for fun, or slandering patients
8. Misconduct in authorship	38. Including a colleague who did not contribute to the paper as an author
	39. Including a colleague who did not contribute to the paper as an author, due to pressure from others
	40. Requesting one’s own inclusion as an author despite not contributing to the paper

4-Point Likert scale from 1 (absolutely cannot be done), 2 (cannot be done), 3 (can be done depending on circumstances,), to 4 (usually can be done).

A structural equation model was used for the confirmatory factor analysis of 40 items under eight categories (Fig. 2). As a result of the analysis, the χ2 was significant (p<0.01), and the RMSEA and CFI were 0.07 and 0.9, respectively, representing good fit for the instrument.

Fig. 2.

Measurement Model for an Instrument to Assess Residents’ Attitudes towards Professionalism Lapses

Item numbers are exactly same with those of the Table 2. The detailed information of each item is shown in Table 2.

3. Assessment of reliability

The Cronbach’s α for the questionnaire was 0.97, and it ranged from 0.75 to 0.97 for each category. The reliability coefficients of all the items and each category are shown in Table 3.

Table 3.

Reliability Coefficients of Total Items and Each Category

Category	Reliability
Total item: Residents’ attitudes toward unprofessional conduct	0.97
Factor 1: Dishonesty and unsafe practice in patient care	0.92
Factor 2: Conflicts of interest	0.76
Factor 3: Misconduct in research and publication	0.95
Factor 4: Irresponsible conduct while on duty	0.87
Factor 5: Physician impairment	0.86
Factor 6: Lack of respect to colleagues	0.75
Factor 7: Disrespectful to patients and breaching confidentiality	0.86
Factor 8: Misconduct in authorship	0.93

Discussion

We developed an inventory assessing residents’ attitudes toward unprofessional conduct based on the assumption that residents’ perceptions of unethical behaviors or misconduct may reflect their level of standards of professionalism. The 40-item inventory was developed and validated using exploratory and confirmatory factor analysis. Because we could not find any validated tools assessing residents’ professionalism in our culture, we identified latent variables through exploratory factor analysis and categorized them into eight factors. We then constructed the structural model and verified it using confirmatory factor analysis. As a result, the fit indices of the model were acceptable, as was the reliability of the entire questionnaire and its factors.

Residency is a crucial period for the development of professional identity as doctors, as residents fully engage in performing various tasks in the real world. The level of a resident’s professionalism can influence their clinical performance, patient safety, and quality of patient care. In a previous study, a resident group with a low level of professionalism was reported to have a high risk of disciplinary action [8]. Therefore, previous studies have suggested the importance of residents’ professionalism and methods for its enhancement [9,10,11,18].

This study is distinct from previous studies in a number of ways. First, our instrument includes more comprehensive domains of behavior that may occur in the actual training environment. Most previous papers investigated unprofessional behaviors in specific circumstances [10,18,19] or domains, such as alcohol and other substance abuse [20]. However, our inventory encompassed a wide range of types of resident misconduct, including patient care, respect for patients and colleagues, responsibility for work, physician impairment, and research and publication.

Second, we tried to incorporate some specific features originating from different sociocultural backgrounds into the inventory: for example, Item 11 (“Conferring benefits on patients after accepting money or gifts”) has rarely been reported by previous studies from Western cultures. One of the distinctive features of this study was the gifted authorship who did not contribute to the paper as an author. Publication ethics, misrepresentation, plagiarism, and falsification were frequently identified resident misconducts in previous Western studies [19,21], whereas gifted authorship has rarely been reported. More interestingly, residents demonstrated a relatively tolerant attitude toward gifted authorship. A cross-cultural study on medical professionalism between Taiwan and Canada reported the prominence of Confucian relationalism and recommended further cross-cultural studies on medical professionalism to move the field beyond a Western individualist perspective [12,13]. In this study, the authors regarded Korea as one of the cultures rooted in Confucianism, in which relationalism is more pronounced. This is a distinctive feature of Korean culture. As such, we would like to raise these authorship issues for educational purposes.

We expected that the mere action of completing this questionnaire would provide residents with an opportunity for self-reflection on the unprofessional behaviors described in the inventory. Furthermore, we expect that this tool may help to identify the level of residents’ personal standards for professionalism, and the areas to which medical educators should pay attention in order to improve residents’ attitudes toward professionalism. Our inventory can be used to enhance residents’ abilities in assessing and reflecting upon their professionalism.

Our study has several limitations. First, we developed the questionnaire items based on real-life practice situations faced by residents; however, it was impossible to cover all aspects of practice. Second, sample size may be not adequate for generalization; we will conduct the survey about more cases. Finally, although we did our best to define and identify unprofessional or undesirable behaviors, establishing a clear definition of unprofessional behaviors was challenging. Furthermore, we must recognize that the trend of educational research on medical professionalism is moving from the evaluation of unprofessional behavior to that of professional behavior [8]. Although this study has several limitations, the authors conclude that our inventory was a timely attempt to explore residents’ attitudes and level of awareness of medical professionalism, in considering Korean medical society where studies on medical professionalism have only just commenced. Furthermore, we hope this questionnaire instrument can increase academic interest in developing more valid and reliable tools for assessing medical professionalism.

In conclusion, the authors suggest that the inventory of this study may contribute to identifying residents’ perceptions and levels of consciousness on the seriousness of each type of unprofessional behavior and to define the areas of professionalism that require educational reinforcement at both individual and institutional levels.