Accounting for professionalism: an innovative point system to assess resident professionalism.

BACKGROUND: Professionalism is a core competency for residency required by the Accreditation Council of Graduate Medical Education. We sought a means to objectively assess professionalism among internal medicine and transitional year residents. INNOVATION: We established a point system to document unprofessional behaviors demonstrated by internal medicine and transitional year residents along with opportunities to redeem such negative points by deliberate positive professional acts. The intent of the policy is to assist residents in becoming aware of what constitutes unprofessional behavior and to provide opportunities for remediation by accruing positive points. A committee of core faculty and department leadership including the program director and clinic nurse manager determines professionalism points assigned. Negative points might be awarded for tardiness to mandatory or volunteered for events without a valid excuse, late evaluations or other paperwork required by the department, non-attendance at meetings prepaid by the department, and inappropriate use of personal days or leave. Examples of actions through which positive points can be gained to erase negative points include delivery of a mentored pre-conference talk, noon conference, medical student case/shelf review session, or a written reflection.
RESULTS: Between 2009 and 2012, 83 residents have trained in our program. Seventeen categorical internal medicine and two transitional year residents have been assigned points. A total of 55 negative points have been assigned and 19 points have been remediated. There appears to be a trend of fewer negative points and more positive points being assigned over each of the past three academic years.
CONCLUSION: Commitment to personal professional behavior is a lifelong process that residents must commit to during their training. A professionalism policy, which employs a point system, has been instituted in our programs and may be a novel tool to promote awareness and underscore the merits of the professionalism competency.

Stern et al. stated that ‘professionalism is demonstrated through a foundation of clinical competence, communication skills and ethical and legal understanding, upon which is built the wise application of the principles of … excellence, humanism, accountability, and altruism’ (1). The American Board of Internal Medicine Foundation, the ACP-ASIM Foundation, and the European Federation of Internal Medicine have put forth a medical professionalism charter for physicians (2). Professionalism is an Accreditation Council of Graduate Medical Education (ACGME) core competency and a key component of the American Board of Internal Medicine (ABIM) Milestone Project (3, 4). As a construct, professionalism is broad and difficult to operationalize. Delineation of methods to reliably assess and quantify professionalism among trainees and in clinical settings remains a challenge (5–7). Assessment methods include direct observation, portfolio self-assessment, global and multisource evaluation, and critical incident reports (8). Longitudinal studies have demonstrated the association of poor performance and behavior during residency with a greater risk for state licensing board actions against practicing physicians, and suggest that remediation strategies during residency are needed (9). Adopting a framework for addressing unprofessional or disruptive behavior may help promote a better work environment, assist with risk management, and improve patient safety (10). We describe a novel point method to attempt to promote awareness and quantify professional behaviors, both positive and negative, among internal medicine and transitional year residents seeking to identify a system to both remediate negative points and reward positive professional behaviors.

Methods

Our local institutional review board approved this study. Internal medicine and transitional year residents at our ACGME accredited training program at The University of Tennessee College of Medicine, Chattanooga, were included in the implementation of a new professionalism policy in 2009. The number of residents each year was 36 (30 internal medicine and 6 transitional). During each academic year, two-thirds of the residents were American medical school graduates (allopathic and osteopathic) and one-third was international medical graduates. The initial policy was drafted by three PGY-3 internal medicine chief residents at that time, several core faculty including the program director and associate program director, the program administrator, and the resident continuity clinic nurse manager. Our professionalism policy states that all residents must master the principles of professionalism as outlined by the ACGME and the ACGME/ABIM Internal Medicine Milestones Project (4). The policy outlines actions to be taken if a breach in professionalism is identified. An Accountability Committee was initially formed in order to: 1) identify unprofessional behavior(s) which might attract negative points, 2) give timely feedback to the residents, and 3) generate an appropriate remediation plan. Members include core teaching faculty, chief residents, teaching subspecialists, and our clinic nurse manager. Examples of infractions deserving negative points that would result in immediate referral to the Accountability Committee include, but are not limited to the following:

Tasks and behaviors

Absence from, or persistent tardiness to, mandatory education events (e.g., Grand Rounds, morning report, noon teaching conference) without a valid excuse

Recurrent tardiness in submitting evaluations and/or department-required paperwork

Failure to respond to departmental requests in a timely fashion

Lack of timely attention to clinical duties

Unprofessional interactions with other residents, students, faculty, staff, or patients, such as cultural insensitivity toward patient(s) and/or coworkers, or issues of honesty or compromised integrity

Failure to ‘clear’ continuity clinic mailboxes and/or electronic health record messages in a timely manner (within 24–36 hours)

Tardiness in submitting team and critical care calendars on a monthly basis

Inappropriate use or abuse of personal days, leave policy, and/or paid registration at state or national meetings (e.g., traveling to a conference paid for by the department, but not attending any sessions)

Failure to complete medical record deficiencies in a timely manner (within 24–36 hours)

Inappropriate personal appearance.

The ultimate decision to assign points for negative behavior is at the discretion of the program director with input from core teaching faculty members. The Accountability Committee intervenes if a single resident accumulates equal to or greater than five points over the academic year or if he/she repeats past unprofessional behavior. It provides input into how these negative points are to be remediated. Each resident's semiannual evaluation template includes a notation of negative and/or positive accountability points.

When the Accountability Committee intervenes, the resident is required to attend a scheduled meeting and be prepared to explain to the committee why he or she has not met the stated requirement(s) or why he or she behaved in an unprofessional manner. Prior to this, the program director will have thoroughly investigated the behavior of the specific resident. The resident is notified prior to the meeting about the issue(s) to be discussed. Extenuating circumstances are considered along with the severity of the issue. The resident may request that one of the chief residents attend the meeting as a resident advocate. The program director, the associate program director, or a designee chairs the meeting which would be attended by core faculty, chief medical residents, the residency continuity clinic nurse manager, and the program administrator. After a discussion with the resident, the concern(s) are reviewed by the Accountability Committee. The committee may recommend that negative points be assigned, and may map out a specific plan and timeline to remediate or ‘erase’ those points. If the resident fails to resolve the identified concerns when he/she meets with the Accountability Committee during a follow-up meeting or if his/her unprofessional behavior continues, our Graduate Medical Education (GME) Institutional Performance Deficiency and Remediation process may begin. The program director will bring the resident back to the Accountability Committee to evaluate his/her overall performance and remediation so that the professionalism evaluation loop is concluded properly.

In 2010, an addition to the above policy included assigning positive professionalism points to residents for acts of professionalism and altruism which go above and beyond minimum requirements or expectations. Positive points would be accrued just as negative points are. In order to ‘erase’ negative points, residents could accumulate positive points.

Positive points might be assigned for:

Receiving unsolicited letters from patients or their friends and family praising a resident's competence, compassion, or empathy

Hosting residency applicants on a continuous basis during recruitment season

Organizing departmental celebrations or fundraisers

Organizing and volunteering for community-based health fairs

Giving talks about various health issues to community groups

Selflessly helping out with extra duty during unexpected events such as inclement weather or illness.

Results

In the academic year ending in 2009, a total of 10 residents either received negative professionalism points or were called before the Accountability Committee. Two of these residents were transitional and the other eight were categorical internal medicine. A total of 38 negative points were assigned that year. Four of the residents chose to remediate their negative points by completing faculty-mentored lectures. Twelve points were remediated during the year. Two of the residents received GME professionalism alerts and were eventually dismissed from the program.

In the academic year ending in 2010, a total of four residents received negative professionalism points. All four residents were categorical internal medicine. A total of four negative points were assigned. Two remediation positive points were completed that year. One GME professionalism alert was implemented.

In the academic year ending in 2011, a total of two categorical residents received six negative professionalism points (Tables 1 and 2). One of these residents was placed on GME professionalism alert. For this academic year, implementation of positive professionalism points for all residents, not just those with negative points, was started. Twelve positive professionalism points were accrued by seven residents.

Table 1

Negative professionalism points assigned

	2009	2010	2011	2012
Missed mandatory event (Grand Rounds, noon conference)	28	3
Poor communication issues with colleagues/consultants	2		1
Failure to complete medical records on time	3
Failure to show up at sponsored conference	5
Missed mandatory education retreat		1
Poor interactions with patients			2	4
Poor interactions with chief resident/consistent lack of insight into their problems			3
Failure to turn in required forms on time				3
Total	38	4	6	7

Table 2

Positive professionalism points assigned

	2009	2010	2011	2012
Clinic coverage during inclement weather				1
Minority health fair participant				11
Minority fair organizer				3
Breast cancer screening in community				1
Reflection essay		2	2
Mentored noon conference on professionalism			1
Organized resident retreat			2
Organized resident party			2
Led EHR transition among residents			2
Dinner with intern applicants on a consistent basis			1
Emergency department compliment			2
Total	n/a	2	12	16

For the academic year ending in 2012, a total of five categorical residents received seven negative points. Thirteen categorical residents were assigned 16 positive professionalism points (Fig. 1).

Fig. 1

Professionalism points.

*No. of residents with negative points/# of residents with positive points.

Discussion

Our study describes a method to quantify, as well as increase awareness of, the ACGME core competency of professionalism in residents in Internal Medicine and transitional year training programs at the same institution. A system of points, a professionalism ‘scorecard’, is used to assess professionalism activities judged to be unprofessional by our program director and core teaching faculty. Negative points were assigned for various infractions with an accompanying remediation pathway outlined by an Accountability Committee. Positive professionalism points were awarded for resident activities and behaviors which were exemplary. Interestingly, a previous internal medicine Residency Review Committee site visit had noted that a large proportion of time was spent on residents with problems. The addition of a positive point reward system has been one way to acknowledge those demonstrating significant positive professional behavior. Fellow residents can see these ‘rewards’ and hopefully internalize and subsequently demonstrate similar professional behavior.

Medical professionalism is complicated. There is no one definition or example one can identify to generate a common understanding. Measurement is even more complex. Swick states that: “attributes of medical professionalism reflect societal expectations as they relate to physicians’ responsibilities, not only to individual patients but to wider communities as well” (11). There have been many attempts to catalog behaviors which are professional and those which are not. McLachlan et al. have introduced the concept of the Clinical Conscientiousness Index (12, 13). The Index is based on observing and enumerating easily identifiable and observable tasks which are associated with professionalism. The collected data are used to help assess professionalism. Our point system is used to assess behaviors which are professionally appropriate or otherwise, and may be used in the future to dovetail with a tool such as the Clinical Conscientiousness Index. The use of multisource resident feedback, observed clinical or non-clinical encounters, patient surveys, and peer-generated concerns are integral tools used to help assess medical professionalism. Using them is one way to help code professionalism activities (negative or positive), to make the process as objective as possible in order to help our Accountability Committee better ‘quantify’ behaviors and to equitably apply points in a consistent and fair way.

Limitations of our study include its small sample size and its focus on one medical specialty at one institution. Collecting feedback from residents would have been important particularly from those assigned either negative or positive points and from those who have accumulated no points. It is critical for residents to have ‘buy in’ to our innovative point system which attempts to give concrete definitions and guidelines for maintenance of medical professionalism over the course of one's career.

The original intent of the Accountability Committee was to serve as a deterrent for unprofessional behavior. The assignment of fewer negative points over the past three academic years infers a heightened awareness among residents of what constitutes medical professionalism. This reduction in negative points is a measure of success and may, in fact, be a positive result of this study (Fig. 1).

Since 2009, there has been a steady rise in the number of residents receiving positive professionalism points. This may suggest that part of professionalism is a set of behaviors which residents can see, internalize, and subsequently demonstrate in their own professional lives (Fig. 1). We need to look carefully at whether those residents accruing positive points are different from those accumulating negative points. Would those demonstrating positive professionalism activities do so whether or not we had a point system in place?

Tracking point assignments over each academic year needs to be detailed, standardized, and consistent. Specific point numbers need to be assigned to specific behaviors. More work needs to be done to better codify what specific positive point activity might be used to offset negative points. Positive points, for example, given to residents for unsolicited letters of praise from patients’ families not involved in actual patient care may be biased and perhaps should be given less positive ‘point value’. Is preparing a set of student lectures, for instance, the appropriate way to erase negative points for professionalism core competency behaviors or tasks? This also raises the issue of whether any highly positive professional behaviors can ever compensate for specific highly unprofessional behaviors. Residents who accumulate negative points on more than one occasion and in more than one academic year have to be carefully and quickly identified requiring very explicit remediation plans to be put into action. Concrete time limits for remediating points also need to be given. Promotion to the next postgraduate year may be in jeopardy. Guidelines about whether negative points should be carried over from year to year need to be established. Whether graduating residents can leave behind negative points needs to be addressed. Tracking repeat resident offenders is essential to help understand whether the point system modified only certain negative behaviors (and not others) and in any way reinforced positive ones. Whether repeatedly earned negative points were for the same negative behaviors or different ones over time would be important to know in order to build a more appropriate, robust remediation plan.

Another limitation of our point system is the notion of residents learning to ‘play the system’ during residency and behaving admirably to accumulate or ‘bank’ positive points which they would have in place if negative behaviors emerged. This may be an example of the Hawthorne effect, a form of behavior adjustment whereby residents modify their behavior in response to knowing they are being scrutinized. In the future, monitoring these residents after they leave our program in order to assess their professional behavior would be important to see if appropriate behavior was, indeed, left behind or was translated into post-postgraduate life.

Our point system is an attempt to find a method to help promote awareness and assessment of medical professionalism among medical residents. It is a start to help clearly identify professional behaviors which are judged to be important. In addition, point assignments will be modified, fine-tuned, and consistent over time and experience with this model. Remedial paths will be altered so that the definition of medical professionalism will be clearer, more succinct, perceived to be less arbitrary, and will be an important parameter to include in a resident's learning/education portfolio. Over time, medical professionalism benchmarks and broad traits will be cataloged in a comprehensive, valid, and reliable way (14). Developing thresholds of specific behaviors which would trigger negative and/or positive points would be important.

Conclusions

Measurement of medical professionalism is difficult at best. Assessing professional behavior and its underlying motivation, honesty, integrity, accountability, and respect for others are complex and, at times, very subjective. We hope that our novel professionalism point system will continue to generate data, which will be important in how it might be adapted and ultimately used in other medical disciplines. Examining residents’ attitudes toward professionalism points, negative or positive, is important to ensure that there is no perceived arbitrariness to the assignment of points. Identifying ways of faculty ‘teaching’ of medical professionalism and its history are important. Codifying point assignments in order to be consistent and equitable is critical. Other assessment tools to evaluate how negative or positive professionalism points are assigned to residents might predict future professionalism behavior issues during postresidency activities such as fellowships, private practice, or medical staff credentialing (9). Investigation into whether professionalism points should be progressively weighted more heavily over the course of a resident's postgraduate residency is important. Equally important is identifying and tracking residents who repeatedly are assigned negative points. Developing appropriate time lines for remediating negative points is critical. Such criteria will help create additional benchmarks for residents to meet with regard to being promoted from one postgraduate year to the next. There is a need for unifying attributes and definitions of medical professionalism to help hold us accountable in our own professional lives as well as those of our students and residents (15).

Professionalism in medicine is a concept and a set of behaviors instilled in physicians in medical school, progressing through residency training, and ultimately translating itself into successful and safe medical practice (16). Specific professionalism competencies may vary depending on specialty-specific definitions, but the concept of professionalism remains the same (11, 14).

Considering the multidimensionality of professionalism, we created a novel point system which helps assess professionalism as a core competency based on behaviors, motivations, and actions. Remediation pathways may then become easier to define, institute, monitor, and standardize. Performance assessments and interventions would necessarily be clearer and more sharply focused. Our study is a work in progress. More needs to be done to incorporate this point system into the ACGME/ABIM Internal Medicine Milestones Project's professionalism component (4). Continuing to develop professionalism scorecards will hold us accountable to this core competency and its place in our lifelong learning portfolios.