Development and Implementation of a Novel Learner-driven Precepting Model for Pulmonary Fellowship.

Background: Ambulatory education is currently underemphasized in pulmonary and critical care medicine (PCCM) fellowship training. Existing clinic precepting models, originally developed for students and residents, do not meet the unique needs of fellow-level trainees. Objective: We aimed to develop and implement a novel fellow-led precepting model to improve ambulatory education for PCCM trainees.
Methods: We performed a mixed-methods needs assessment, including surveys, focus groups, and direct observations, to explore perceptions of ambulatory training, define current precepting practices, and identify target areas for improvement. On the basis of these findings, we developed, implemented, and evaluated a novel model for PCCM outpatient precepting.
Results: A targeted needs assessment identified that current precepting practices did not meet fellows' needs for graduated autonomy, development of assessment and management skills, and self-directed learning. We developed and implemented a novel, learner-driven precepting model (Set the Stage, Tell the Story, Educational Goals, Preliminary Plan, Uncertainties, Plan Recap [STEP-UP]), designed to prioritize clinical reasoning skills and self-identification of learning goals. Implementation of the STEP-UP model improved perceptions of overall outpatient training and precepting. However, we faced several barriers to uptake of the new model, including increased cognitive burden of integrating a new process.
Conclusion: A robust assessment of ambulatory education at a single PCCM fellowship program identified a need to align precepting processes with the unique goals of advanced trainees. We developed a learner-driven precepting model focused on development of clinical reasoning skills and self-directed educational objectives. Additional study is warranted to refine, adapt, and test the model in different setting.

Pulmonary and critical care medicine (PCCM) fellowship programs are charged with preparing trainees for independent practice across the patient care continuum. Although several Accreditation Council for Graduate Medical Education milestones highlight competencies specific to outpatient practice (1, 2), clinical training remains inpatient focused (3). Unsurprisingly, fellows report feeling underprepared for independent outpatient pulmonary practice (4).

Initial efforts to improve ambulatory education in PCCM fellowship have focused on augmenting medical knowledge using standardized outpatient curricula (4). However, clinical experiences, with learning shepherded by faculty preceptors, remain the foundation of ambulatory education. Although precepting interactions can provide rich educational opportunities, time pressures, patient complexity, and preceptor variability can impede learning.

To address these limitations, several precepting models have been adopted in undergraduate and graduate medical education settings, including SNAPPS (Summarize Case, Narrow Differential, Analyze Differential, Probe the Preceptor, Plan Management, Select Issue for Self-directed Learning) (5) and One-Minute Preceptor (6). Structured precepting models have demonstrated multiple benefits, including improvements in clinical reasoning verbalization (7) and care efficiency (8) but have not been widely adopted in the fellowship setting. Existing models, developed for students and residents and focused primarily on obtaining history and forming differential diagnoses, may not be as well suited to the advanced developmental goals of fellowship-level trainees. Applying the reporter-interpreter-manager-educator competency framework (9), fellows have likely achieved competency as reporters and interpreters already and instead are focused on developing skills as managers and educators.

We aimed to improve the outpatient precepting process for PCCM fellows at the University of Pennsylvania by conducting a multifaceted local needs assessment to inform development of a novel precepting model. We then implemented, pilot-tested, and evaluated the model for acceptability, usability, and impact on learner-centered outcomes. A portion of this work was previously published in abstract format (10).

Methods

Setting

The University of Pennsylvania’s PCCM fellowship is a 3- to 4-year training program at a tertiary academic medical center, with six to eight fellows per class. Ambulatory experiences consist of longitudinal continuity clinic and rotating subspecialty clinics.

Fellows spend one half-day every other week in continuity clinic, managing their own patient panels with supervision from a volunteer pool of faculty preceptors. Patients are scheduled directly with fellows, who seek out preceptors to staff patients on a first-come, first-served basis outside of the examination room. Two to four preceptors are available during each session, scheduled at a ratio of one preceptor to two trainees.

Needs Assessment

To identify potential targets for improving ambulatory education, we performed a mixed-methods local needs assessment, including surveys, focus groups, and direct observation of precepting encounters. We started with surveys and observations to define the current state of precepting and then proceeded to focus groups to explore participant experiences and preferences in more detail.

A baseline survey was developed and pilot-tested with input from fellows and faculty with training in program assessment and survey design (11). The survey was anonymous and voluntary, with no incentives offered. Survey domains included: 1) perceptions of ambulatory education and precepting, 2) current precepting practices (including presentation methods and perceived time breakdowns), 3) ideal preceptor practices, and 4) barriers to effective precepting. The voluntary, confidential survey was distributed via an online platform to all fellows in June 2018. Likert scale responses were analyzed using summary statistics, and thematic analysis was performed on free-text comments.

Direct observations of precepting sessions were performed using convenience sampling from October and November 2018. Observations were collected by J.K.H. and C.B.C., both fellows at the time, using a standardized observation form to capture duration of precepting encounters and breakdown of time in different presentation domains (history, physical, data, assessment, and plan). Given the difficulty of separating teaching from patient care, teaching was not separately classified but instead was attributed to the corresponding presentation domain. We calculated mean precepting time, time to interruption by preceptors, and proportion of time spent discussing assessment and plan.

Finally, we convened three focus groups (two with PCCM faculty and one with fellows), facilitated by faculty experts in residency-level ambulatory education (D.A. and D.R.N.), to explore barriers to effective precepting and gauge openness to changing precepting structure and practices. Notes, including paraphrased quotations, were recorded during the focus group and later analyzed by three research team members to identify themes.

Intervention Development

On the basis of our needs assessment and review of published precepting models, we determined that a novel precepting model, borrowing aspects from existing models, was required to meet the unique needs of PCCM fellowship learners. We developed the STEP-UP model (Set the Stage, Tell the Story, Educational Goals, Preliminary Plan, Uncertainties, Plan Recap) (Table 1) to emphasize development of assessment and management skills and identification of self-directed learning objectives. Iterative changes were made to incorporate preceptor and program leadership feedback before implementation.

Table 1.

STEP-UP model for ambulatory precepting of pulmonary and critical care medicine fellows

	Model Step	Explanation	Example
S	Set the stage: align expectations with faculty	Fellow will identify which precepting model to use in the current encounter (STEP-UP vs. a traditional model).	“This case is very straightforward; I would like to proceed with the STEP-UP,” vs. “This is a complicated new patient and a diagnostic dilemma, and therefore it would be helpful to do a traditional full presentation.”
T	Tell the story: starting with the diagnosis (in <60 s)	Fellow will briefly summarize key history, physical, and data on the patient in <1 min, starting with their assessment.	“This is a patient with stable COPD. Briefly, he is a 60-yr-old heavy smoker, who developed dyspnea several years ago. He was diagnosed with COPD on the basis of obstruction on PFTs and flattened hemidiaphragms on CXR. He can walk several blocks before getting winded and has one exacerbation per year, managed as an outpatient. He is only on SABA therapy currently.”
E	Educational goals	Fellow will identify one or two potential learning goals for the encounter, which may include medical knowledge or direct observation of skills.	“I want to learn more about the evidence base of current smoking cessation resources but also would love to get feedback on my smoking cessation communication with this patient.”
P	Preliminary plan	Fellow will propose a preliminary plan, before input from the faculty member.	“My plan for the patient is to focus on smoking cessation. I will plan to start him on nicotine patches and gum. Otherwise, I would continue his current inhalers regimen. He is already up to date with lung cancer screening and vaccinations, and we already reviewed inhaler technique.”
U	Uncertainties and learning objectives	Preceptor will clarify any uncertainties with the presentation and address the learning objectives set by the fellow.	“Let’s talk about some options to help patients succeed with smoking cessation. What options are you familiar with?”
P	Plan recap	Fellow and preceptor will confirm plan before conclusion of the encounter.	“Ok, so in addition to his current inhalers, we will recommend varenicline to assist with smoking cessation.”

Definition of abbreviations: COPD = chronic obstructive pulmonary disease; CXR = chest X-ray; PFT = pulmonary function test; SABA = short-acting β-agonist; STEP-UP = Set the Stage, Tell the Story, Educational Goals, Preliminary Plan, Uncertainties, Plan Recap.

Intervention Dissemination

We used a multifaceted strategy to introduce the STEP-UP model to our learners and preceptors. First, we held an hour-long training session for fellows, including discussion of results from surveys, observations, and focus groups. The STEP-UP model was demonstrated and practiced using training videos and role play. Training was repeated for incoming fellows in July 2019.

Clinic preceptors were provided with written material explaining STEP-UP and links to training videos. In addition, signage describing the model was posted throughout precepting areas, and in-person guidance, including real-time feedback on model use, was provided for the first two clinics after implementation. Although model uptake was encouraged, usage was voluntary and left to the discretion of fellows and preceptors.

Follow-up surveys were distributed to fellows at 3 and 6 months after implementation. The 3-month survey focused on perceptions of ambulatory education and precepting, and the 6-month survey focused on uptake of the new model and perceived impact on the structure of precepting encounters. Both surveys collected input on barriers to implementation of STEP-UP. Summary statistics were used to analyze Likert scale responses, and thematic analysis was performed on free-text comments. Percentages and counts are reported, given the small sample sizes, varying response rates, and fluctuation in the number of eligible fellows. We did not assess for changes compared with baseline, given the small sample sizes.

All survey instruments are provided in Tables E1–E3 (data supplement). The study was reviewed by the University of Pennsylvania Institutional Review Board and deemed exempt.

Results

Seventeen out of 18 fellows responded to the baseline survey (94%). Only half of respondents (53%, n = 9) agreed they were satisfied with their current precepting experience. Although 94% of fellows (n = 17) agreed with the statement “I feel confident in my outpatient skill set,” no respondents indicated strong agreement. Eighty eight percent (n = 15) reported that no expectations were set for precepting sessions.

Fellows reported that they most often (65%, n = 11) used a traditional presentation format (chief complaint, history of present illness, review of systems, physical examination, data, assessment, and plan) during precepting encounters. The majority of fellows (65%, n = 11) indicated that less than half of a typical precepting encounter was spent discussing assessment and plan.

Direct observations of 14 precepting encounters were performed over a 2-month period, including four first-year, six second-year, and four third-year fellows. Mean precepting time was 10.1 minutes (standard deviation, 5.6 min; range, 2–24 min). Fellows were interrupted by preceptors after a mean of 2.3 minutes, and less than half (47%) of total precepting time was spent discussing assessment and plan.

Focus groups with fellows and faculty identified characteristics of ideal precepting encounters, as well as targetable areas for improvement, presented in Table 2. Improvement opportunities fell into three overarching themes: autonomy, precepting structure, and educational objectives. Overall, fellows emphasized a mismatch between their learning needs and the current precepting structure. Likewise, faculty recognized a need to improve the precepting process and expressed eagerness to adopt new approaches.

Table 2.

Targeted themes to improve precepting from fellow and faculty focus groups

	Fellow	Faculty
Autonomy	Want more autonomy, feel that faculty sometimes micromanage plans, are overly detail oriented	Support graduated autonomy for fellows on the basis of attainment of core competencies
	Have to present known patients as new patients at every visit, perceived as a lack of trust	Still need to balance fellow autonomy with patient safety
Presentation elements	Currently, feel preceptors focus too much on reporting history	Currently, feel fellows default to long, detailed presentations
	Want to prioritize clinical reasoning skills by spending more time on assessment and management	Supportive of streamlined history and examination reporting
	Want more uniform approach to precepting by faculty	Want a more uniform approach to precepting
Educational goals	Currently, feel they are not setting educational goals for precepting encounters	Unsure what topics fellows want to learn about
		Review of primary data viewed as teaching opportunity
	Would like to choose topics for teaching	Sometimes encounter resistance to teaching

Intervention Development and Implementation

On the basis of our needs assessment, we developed and implemented a novel, learner-driven precepting model, STEP-UP, to prioritize clinical reasoning skills and self-identification of learning objectives (Table 1). We established guidelines for appropriate use of STEP-UP, empowering fellows or preceptors to opt for a traditional presentation model if merited by an individual fellow’s learning or supervision needs or patient complexity (Figure 1). Regardless of the model used, learners were asked to identify educational objectives for each precepting encounter.

Figure 1.

Process for selection of precepting model. STEP-UP = Set the Stage, Tell the Story, Educational Goals, Preliminary Plan, Uncertainties, Plan Recap.

Program Evaluation

Seventy percent (n = 14/20) of fellows responded to a 3-month postintervention survey assessing perceptions of outpatient training and precepting experiences. Baseline and postimplementation results are reported in Table 3.

Table 3.

Fellows’ perceptions of outpatient training and precepting at baseline and 3 months after implementation of STEP-UP model

Survey Question	Agreed/Strongly Agreed*
	Baseline	Follow-up
Confident in outpatient skill set	17 (94)	6 (43)†
Satisfied with current precepting experience	12 (71)	12 (86)
Precepting improved patient care	11 (65)	13 (93)
Discussion with preceptor improves knowledge	10 (59)	13 (93)
Experienced progression of autonomy throughout training	12 (71)	14 (100)

Definition of abbreviation: STEP-UP = Set the Stage, Tell the Story, Educational Goals, Preliminary Plan, Uncertainties, Plan Recap.

Data are presented as n (%).

The eligible number of fellows increased from 18 to 20 and response rate decreased from 94% to 70% at 3-month follow-up.

The baseline study respondents included a higher proportion of second- and third-year fellows (78% vs. 50%).

Sixty-five percent (n = 13/20) of fellows responded to a 6-month postintervention survey. Fellows reported variable uptake of STEP-UP, with 46% (n = 6) using the model in <25% of precepting encounters. The majority of fellows (69%, n = 9) reported spending less than half of the precepting time discussing assessment and plan, compared with 65% in the initial survey. However, most (n = 10) fellows indicated they were reliably setting educational goals.

Barriers to use of STEP-UP were similar at 3 and 6 months, with fellows citing lack of familiarity, difficulty remembering, and increased cognitive load with the new model, with strongly ingrained habits favoring traditional presentation structure. One representative comment stated, “I’m probably doing something closer to STEP-UP than [the] traditional model, but [I’m] not necessarily intentionally trying to use STEP-UP.”

Discussion

Using a mixed-methods approach, we created a novel learner-driven precepting model to improve the ambulatory educational experience at a single, large PCCM fellowship. The STEP-UP model aims to prioritize clinical reasoning skills by shifting focus to assessment and management, while incorporating a streamlined, hypothesis-driven presentation of history, examination, and data. In addition, STEP-UP encourages self-directed learning by requiring fellows to identify educational objectives.

Although the study was not designed to detect changes in survey responses, fellows reported higher overall satisfaction with precepting and knowledge gained and noted more opportunities for graduated autonomy at 3-month follow-up. Interestingly, the proportion of fellows reporting confidence in their outpatient skill set decreased from 77% to 43%, most likely due to a decrease in proportion of upper-year respondents (77% vs. 50%). Overall, the lower response rate and shift in respondent demographics may have biased responses in favor of STEP-UP. In addition, changes in perceptions and reported behaviors may reflect maturation effect, given that there was no control group.

One strength of our study was the mixed-methods approach to intervention development, informed by a multimodal local needs assessment, educational theory, prior literature, and expert opinion. Specifically, by triangulating multiple techniques, we were able to mitigate potential measurement biases, such as the Hawthorne effect for observations and recall bias for surveys. Furthermore, involving major stakeholders from the outset of the initiative promoted engagement in the iterative process of improvement.

We faced challenges promoting uptake of STEP-UP, primarily due to ingrained presentation habits and higher cognitive load requirements of the new model. Given the uptake challenges, we did not perform repeat objective measurements of precepting sessions, focusing instead on fellows’ perceptions and barriers to implementation. On the basis of this feedback, we plan to develop a simplified version of STEP-UP, provide additional in-person guidance on model use, and evaluate model usage and fidelity using direct observation. Despite our goal of prioritizing clinical reasoning discussion, fellows did not perceive an increase in proportion of time spent discussing assessment and plan. This may reflect poor uptake of STEP-UP or that the time gained by cutting down detailed reporting was redistributed to different areas, such as identification of learning objectives.

Additional limitations include generalizability, as development of our model was based on a local needs assessment at a single program. Specifically, programs with rotating, availability-based preceptor selection, compared with settings where attendings are paired longitudinally with learners and able to review charts before precepting, may be most in need of a model like STEP-UP. However, we believe that the two primary goals of STEP-UP, to emphasize clinical reasoning skills and encourage self-directed learning, are likely to be shared by most fellowship-level learners. Importantly, STEP-UP is designed for learners who are already competent in information gathering and data interpretation, as hypothesis-driven presentations focused on management could mask learner deficits in these areas. Preceptor comfort or patient-level outcomes (i.e., diagnostic or management errors) were not assessed in this study but could be incorporated in future studies.

Given the lack of established precepting methodologies in subspecialty fellowship, we believe STEP-UP provides a novel approach to improving ambulatory education for PCCM fellows. Further adaptation and study of STEP-UP, as well as a broader needs assessment of outpatient educational strategies, is warranted to drive continued innovation in ambulatory education for PCCM trainees.