The CARE Inventory: A Self-Reflective, Behavior-Based Instrument to Guide Professional Development and Mentorship of Academic Faculty.

Introduction: Traditional needs assessments often rely on self-reported skill levels. To gather more objective and growth-focused data, we developed a behavior-based inventory to measure perceived faculty competence and desired areas for growth in four common domains of academic medicine: clinical, administrative/leadership, research, and education (CARE).
Methods: Competencies in teaching, research, and professional development and leadership noted in the literature were used as the foundation of our instrument. Clinical service topics were added, and questions were vetted with the executive committee of our faculty development center. A behavior-based inventory was developed to enhance self-reporting of competency. The instrument was piloted with faculty at an external institution and revisions made prior to data collection.
Results: In May 2016, the CARE Inventory was sent to all full-time faculty members in the Ohio State University College of Medicine (n = ∼1,800). We received 350 responses (20% response rate). Individual reports were generated and sent to the faculty member and his/her identified mentor for individual professional development. Summary data were used in aggregate for professional development program planning. Discussion: Anchoring measurement to current and desired future behavior allows for more self-reflective and growth-oriented assessment for individuals, and results can inform tactical faculty development by both individual and program. We believe this is a scalable and generalizable instrument other academic medical and health sciences programs could use both as a needs assessment tool for program planning and for individualized development plans with faculty and their mentors.

Educational Objectives

By implementing the CARE Inventory, administrative leaders will be able to:

Stimulate self-reflection of academic faculty and gather data for individual professional development.

Aggregate individual data from the inventory to inform programming for faculty development.

Apply the inventory and the individual report as tools to assist career mentorship in academic health professions.

Introduction

Competencies of academic faculty members in different roles have been described by several authors, and the shift to competency-based assessments for learners and faculty is well underway.[1-5] Yet faculty members, especially junior faculty, often struggle with application of these competencies in their own professional development in academic medicine, in part due to limitations of the self-assessment model.

Ordinarily, self-assessment tools are used as a reflective activity to gauge competency or skill development along a continuum from novice to mastery. However, there is a human tendency to overrate one's abilities and overlook one's weaknesses.[6,7] Therefore, relying on traditional self-assessment tools to guide professional development can be misleading. Additionally, these traditional tools are merely point-in-time measures and do not provide direction for increasing focus or effort to achieve career goals. A survey of the literature, including MedEdPORTAL, did not reveal a tool for faculty competency assessments designed to overcome self-assessment biases.

To address these limitations, we developed a tool based on current academic faculty competencies identified from the literature and created a novel behavior-based inventory to overcome the limitations of traditional self-assessment. We focused on domains of competencies and career development in academic medicine, including clinical service skills, administrative and leadership skills, research and scholarship skills, and educator skills (CARE). The CARE Inventory asks for current self-assessment (e.g., “Have done this as part of a team”) and future self-assessment (e.g., “Would like to be fully independent”). Considering the change from current to future self suggests areas for continued professional development for the individual, and examining the degree of desired change in aggregate serves to identify common needs that inform faculty development programming.

This competency-based, behaviorally anchored instrument also serves as an essential tool for mentors and departmental leaders to help guide faculty members in focused career development. Ideally, a potential mentee should engage in self-reflection about his or her work and learning styles, as well as his or her knowledge and skill gaps, and write down short- and long-term goals before meeting with a mentor.[8] Actions of a good mentor have also been described and include fostering mentee self-reflection and helping the mentee to identify areas for performance improvement.[9] The CARE Inventory has been designed to assist mentees with this self-reflective process in a systematic fashion. The instrument can be used as a tool by mentees and mentors to identify the specific competencies that faculty members would most like to develop, thereby helping to guide or focus individual professional development activities.

Methods

To develop a broad view of faculty skills and competencies needed to be successful in academic medicine, we reviewed the literature. As the foundation of our instrument, we used competencies in teaching, research, and professional development and leadership noted by Stritter, Bland, and Youngblood[1] and by Milner, Gusic, and Thorndyke.[2] Additional sections on clinical service topics were added to the instrument to reflect the domains of faculty practice. Below are the domains of the CARE survey, as well as some sample items from it.

Demographic information:

Rank.

Track.

Gender.

Clinical service:

Create efficiencies of clinical practice.

Utilize electronic medical records to analyze practice patterns.

Admin/leadership:

Time management.

Performance improvement of supervisees.

Research:

Design research study.

Respond to reviewer critiques.

Educator skills:

Design curricula.

Provide clinical teaching at bedside.

Finally, demographic questions were included to identify who was using the instrument and to analyze data by subgroups if needed. The questions were vetted and refined with the executive committee of our faculty development center, whose members had a diverse range of experience and expertise. The executive committee comprised faculty members from assistant through full professor, representing several different clinical and research departments. Both physician and PhD scientists were represented, and the committee included the chairs of each subcommittee (Women in Medicine and Science, Education, Diversity and Inclusion, and Mentorship and Research).

After initial vetting and refinement of competencies, we focused on the development of a novel behavior-based inventory informed by the Accreditation Council for Graduate Medical Education's Next Accreditation System to enhance self-reporting for each competency.[4] This inventory contrasted with the more typical needs assessment instrument, which measures a perception of one's competency from novice to expert. Instead, the inventory asked faculty members to identify their experience currently and what they would like their experience to be within 2 years based on more observable standards (Table 1). For instance, on a scale of 1–5, for current state a 3 would be “Have done this independently with room for growth,” and a 4 in desired experience would be “Would like to be fully independent.” The questions and anchors were again vetted with the executive committee, refined, and then built in an online survey platform (Qualtrics, Provo, UT). The instrument was piloted with a small group of basic science and clinical faculty from another institution. Final revisions were made and the survey reviewed once more by the faculty development committee. The final revised CARE Inventory is included as Appendix A.

Table 1.

Behavior-Based Anchors for Inventory

Inventory Rating	Current	Desired Within 2 Years (Future)
1	Have not done this.	Will not do this.
2	Have done this with guidance or as part of a team.	Would like to do this with guidance or as part of a team.
3	Have done this independently with room for growth.	Would like to begin to do this independently.
4	Confident in my ability, fully independent.	Would like to be fully independent.
5	Others seek my advice.	Would like to be sought after for advice.

A link to the online CARE Inventory was sent to all full-time faculty members via email from the faculty development team. Data collected from the instrument were then disseminated in two ways. Since individuals did not have online access to their data after submission, individual summary reports were generated from the online survey platform and emailed back to the faculty members and their mentors (if identified) using a standardized template included as Appendix B. The summary reports allowed faculty members to have access to their answers to use for professional growth or mentoring activities.

Additionally, aggregate reports were generated from the online survey platform for each department, and a summary report was sent via email to the department chair using the template included as Appendix C. The summary reports were also rolled up for the entire Ohio State University College of Medicine (COM) and used by the faculty development committee to select topics for programming based on what the respondents in aggregate expressed the most interest in developing.

Results

In May 2016, the instrument was sent to all full-time faculty members at the COM (n = ∼1,800). Three hundred fifty responses were received, approximately a 20% response rate. The initial respondents were primarily assistant professors on the clinical track (Table 2). Faculty members who completed the survey received their data back in an individual report via email; this report was also sent to the faculty mentor if identified. Individual responses were also screened by the faculty development team to identify individuals self-reporting a current level 5 in a domain (“Others seek my advice”) as possible mentors.

Table 2.

Demographic Data of Respondents From 2016 Survey

Category	Responses
	No.	Percentage
Gender
Female	174	51.3
Male	165	48.7
Track
Tenure	58	17.5
Research	8	2.4
Clinical	239	72.2
Associated	19	5.7
Other/unknown	7	2.1
Rank
Assistant	220	66.3
Associate	68	20.5
Full	42	12.7
Years on faculty
0–3	110	33.2
4–6	75	22.7
7–10	52	15.7
>10	94	28.4

The data were also aggregated for analysis. Data were aggregated at the department level as well as across the entire COM. Since the aggregate data were a snapshot of faculty's current self-reported level of competency and desired growth, the areas with the greatest delta of desired change were identified as high priorities for program development. Table 3 shows the five competencies with the largest desired growth change from our 2016 survey. When we looked at the data in aggregate, a few interesting trends were revealed. For example, the aggregate data identified a group of faculty members who ranked themselves in the top third of current teaching experience but desired continued growth in these same areas. This group would therefore require more sophisticated professional development opportunities that enabled moving from proficiency to leadership, rather than from novice to competent. By examining the aggregate data, trends in areas for desired growth and type of growth desired suggested programming areas and types on which to focus limited faculty development resources.

Table 3.

Top Five Categories With Desired Growth From 2016 Survey of Faculty

Area for Desired Growth	Collect Data	Analyze Data	Project Leadership	Clinical Teaching	Small-Group Teaching
Current skill level
1	19%	21%	8%	1%	8%
2	23%	21%	13%	1%	5%
3	32%	27%	23%	21%	24%
4	17%	15%	31%	40%	41%
5	8%	8%	25%	27%	19%
Future skill level
1	7%	10%	0%	<1%	2%
2	12%	9%	5%	<1%	6%
3	15%	28%	8%	4%	7%
4	39%	33%	29%	21%	27%
5	20%	23%	49%	58%	50%

Percentages for each category column do not always add up to 100% either because there was also a “not applicable” choice or due to being rounded.

We also collected ordinal data about mentorship and desired leadership roles, as well as open-ended comments about career development. Approximately 87% of respondents said they either already had a formal leadership role or desired one in the future, and 63% reported having at least one mentor.

Discussion

Faculty members often do not receive formative feedback with respect to their performance across competencies, and they sometimes struggle to identify targeted areas for professional development. Formative feedback is especially lacking for faculty in education roles.[10,11] Mentorship can support the creation of individual development plans but requires mentors to encourage and guide mentee self-reflection. A missing link in the development of individual faculty members is a tool to trigger structured reflection in competency-based domains relevant to growth as an academician. The CARE Inventory offers several benefits. First, grouping competencies into common domains of clinical, administrative, research, and education helps faculty to focus on professional identity formation. For junior faculty deciding on a career path, responses to questions about desired growth relative to academic competencies serve to declare primary interests and help to define the career path. For example, a clinical faculty member who responds with desired growth in both education and leadership competencies begins to develop a professional identity as an educational leader. A mentor could guide this individual in the selection of training and development opportunities to gain necessary skills, as well as by providing the mentee with opportunities and stretch assignments required of educational leaders.

Anchoring measurements to current and desired future behavior allows for tactical faculty development for the individual faculty member and helps identify faculty at the institutional level who are already competent or expert. These expert faculty can then be called on to share expertise as part of a speakers' bureau and serve as mentors. Data from the CARE Inventory provide a bird's-eye view, which allows for better customization to target development opportunities for individuals. Furthermore, in aggregate, data from the CARE Inventory guide the selection of faculty development training and programming topics and orient facilitators to the skill level of participants.

The CARE Inventory is valuable at both the individual and institutional levels (Figure). At the individual level, it is a tool that guides mentees through a structured self-reflection process concerning professional competencies. At the institutional level, the inventory provides faculty development teams data to support the need for development programming. Using the data, programs could be directly marketed to target faculty as well as used to engage in conversations with chairs and other leaders. The CARE Inventory is generalizable to other academic medical institutions and health professions because the CARE domains are common to many allied health programs. The CARE Inventory has garnered interest from and been shared with other health professional schools, including dentistry and veterinary schools at our institution.

Figure.

Uses of data collected with the CARE Inventory. Abbreviation: IDPs, individualized development plans.

Experience from our first large-scale data collection led us to change the online data-collection method moving forward. This was largely based on local support and user familiarity with the two systems. The questions and behavioral anchors can be uploaded into any data-collection system supported by local institutions. Additionally, as with any survey-based initiative, response rates can be low. We found that departments whose local leaders sent faculty the instrument link directly had higher response rates than departments at which the link came from the faculty development team. This might be a strategy to increase response rates at other institutions.

Since our original large-scale data collection in 2016, we have used the CARE Inventory on an individual basis for entering faculty members as part of our onboarding process. Data from these rolling inventories are not included in this publication. We have also given mentors access to share the inventory with new mentees. We believe this is an important tool to help junior faculty members begin to understand the specific range of competencies to develop for success in academic medicine. We plan to continue to use the CARE Inventory in two ways moving forward: yearly for all new entering faculty and every 2–3 years as an aggregate needs assessment for all faculty.

The CARE Inventory is a self-reported instrument. While the behavior-based inventory was designed to overcome some of the limitations of self-assessment, other factors including social desirability bias may still impact the data. While we do not think the topics are particularly sensitive, having a forced choice based on behavior may help ameliorate this bias.[12,13]

When the instrument was sent to all faculty, we received feedback that senior faculty viewed it as more appropriate for junior faculty development. It was suggested that we might need higher level behaviors for senior faculty to fine-tune their competencies, rather than develop them. However, we did find the inventory to be a useful tool to identify expertise within the faculty.

Another limitation is the possible temptation to use this instrument for program evaluation. The purpose of the tool is for individual self-reflection and aggregate data collection as a snapshot needs assessment. It is not designed to measure year-to-year overall growth or programming effectiveness. We plan to administer the inventory every 2 years, and while some individual focused growth would be expected, faculty may shift their focus, not show growth in certain areas, from year to year. We also plan to use data over time to appreciate individual growth focusing on deepening and strengthening skill sets, not so much for routine program evaluation.

Finally, the CARE Inventory was created with a web-based survey tool. Depending on the sophistication of the system and the experience of users, the inventory may require significant manual manipulation in order to download data and create individual and aggregate reports.

Overall, we have found that this instrument is intuitive, based on published evidence about competencies important for success in academic medicine, and easily scalable and adaptable. It provides feedback to individuals and informs training and development needs from a programmatic perspective. Additionally, it fills a gap in mentorship by promoting competency-focused self-reflection and can serve as a valuable tool for mentors and mentees to use when contemplating professional identity formation and focused skill development as part of an individualized development plan.

A. CARE Inventory.docx

B. Individual Report Template.docx

C. Department Chair Sample Report.docx

All appendices are peer reviewed as integral parts of the Original Publication.