Meeting Milestones: Results of a Quality-Improvement Curriculum to Achieve Cost-Conscious Care.

There is a lack of residency education in cost-conscious care. We implemented a costing and quality improvement (QI) curriculum to Obstetrics and Gynaecology trainees using "Time-Driven Activity-Based Costing (TDABC)," and assessed its educational impact. The curriculum included didactic and practical portions. Pre-and post-knowledge surveys were obtained from 24 residents on self-perceived knowledge of key QI principles. Self-perceived knowledge, before and after the curriculum, was scored on a Likert scale from 0 to 5 points (0 is the least knowledge and 5 is the most knowledge). The mean scores reported an increase in knowledge of clinical guideline development (pre = 1.19 vs. post = 3.07, p = 0.0052); confidence in participating in QI work (pre = 1.75 vs. post = 3.42 points, p < 0.0001); and knowledge in communicating QI principles (pre = 1.89, post = 3.17, p < 0.0003). Our educational programme uses the TDABC method and the residents' clinical experience effectively to teach residents cost-conscious care.

Background

Studies show that cost-conscious care does not result in poor patient outcomes [1]. In academic medicine, though, formulating a comprehensive differential diagnosis and work-up has historically been preferred to a cost-conscious medical plan [2]. Consequently, there is a paucity of relevant teaching resources.

The Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS) identified lack of training in the domain of cost-conscious care. In 2015, ACGME moved to the Next Accreditation System (NAS) and “Milestones” framework. Milestones describe the learning trajectory throughout training with the goal of demonstrating a high level of competency [3], within milestones such as “Cost-Effective Care and Patient Advocacy,” “Patient Safety and Systems Approach to Medical Errors,” and “Quality Improvement Process.”

There have been attempts to create curricula on cost-conscious care [4,5]. Some argue that these attempts may be significantly limited [6], as the results showed only a temporary effect [7]. This may be because traditional didactic-based curricula are often decoupled from the realities of clinical workflow. Other data show practice patterns acquired in residency will persist for the rest of professional life [8]. New methods of teaching adult learners that integrate an understanding of clinical workflow may be more successful in teaching cost-conscious care.

We introduced the Time-Driven Activity-Based Costing (TDABC) concept as a vehicle for learning value, cost-conscious care, and quality improvement. TDABC was developed at Harvard Business School [9]; it breaks up clinical care into individual steps using process mapping, and assigns cost to each step on a per-minute basis. Process maps provide a visual depiction of clinical workflow: specific symbols are used to represent when a decision is required, when an activity occurs, and when an outcome results. All steps and costs are visible, making it easier to identify areas that do not add value. Case studies using this approach have been linked to improved systems performance [10,11]. We hypothesised that TDABC, as part of a QI curriculum, would provide a way to teach quality improvement and cost-conscious care.

Methods

Twenty-four female obstetrics-gynaecology residents, or physicians-in-training, participated. The residents’ post-graduate year ranged from 1 to 4. The first portion of the curriculum was the educational component led by four OB/GYN generalists, including two with particular expertise in the field of quality improvement. There were three didactic sessions that focused on the principles of process mapping, the basic skills of cost analysis, and approaches to designing a systematic process improvement intervention.

The second portion focused on process mapping and cost accounting. The residents were separated into five teams and assigned to create process maps for vaginal birth after caesarean, spontaneous vaginal delivery and caesarean delivery. Residents created two process maps for each of these scenarios (Figure 1). The first was created from memory with the goal of learning how to create process maps. A second was made from direct observation of the process in order to improve process maps in an iterative fashion through direct observation and feedback. Maps were then scored by blinded reviewers.

Figure 1.

Process map for women presenting for rule-out labour.

In addition, trainees played a cost accounting game we created called, “The Process Is Right”. This game presented the learner with a childbirth scenario in steps and forced decisions at key points. The cost of each decision was totalled. Twenty-one residents played the game.

Residents completed pre- and post-curriculum surveys to assess understanding of value-based care (Appendix). They were asked about their knowledge on the following QI principles: root cause analysis, process mapping, adverse event reporting at BIDMC, Plan-Do-Study-Act (PDSA) cycle, run charts, clinical guideline development, confidence in participating in QI project, ability to communicate to colleague about QI principles, role of QI as part of the physician’s professional activities, difference between pricing and cost, difference between quality and value, and just culture. The knowledge scale was scored on a Likert scale from 0 to 5, with 0 as least knowledge and 5 as most knowledge.

All statistical analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC, USA). All tests were two-sided, with P < 0.05 considered to be statistically significant. Data were presented as median and interquartile range (IQR). Wilcoxon signed-rank test were used to compare paired medians.

Results

For “The Process is Right” game, the 21 residents spent median amount of $617.14, with range of $474.24–$1,426.04. The resident who won the game was able to obtain a $474.24 cost for a safe vaginal delivery.

A cross-sectional survey was completed before and after the QI curriculum. Twenty-one residents completed the pre-surveys and 21 residents completed the post-survey. Eighteen residents filled out both the pre- and post-survey (85.7%). When asked about knowledge of process maps, the average score was 1.12. After the curriculum and having had actual experience creating process maps as a team, the post-curriculum average score was 3.49, a significant improvement (< 0.0001) (Table 1). Residents reported a positive increase in knowledge on clinical guideline development, with a pre-curriculum average score of 1.19, with the post-curriculum average score of 3.07 points (p = 0.0052).

Table 1.

Changes in self-perceived knowledge scales on core quality improvement principles, before and after the QI curriculum.

Factor	Pre (n = 18)	Post (n = 18)	Change (n = 18)	p-Value
Process mapping	1.12 (0.74, 2.12)*	3.49 (2.97, 4.09)	1.99 (0.90, 2.89)*	< .0001
Clinical guideline development	1.19 (0.27, 2.00)***	3.07 (2.50, 3.70)	1.63 (0.41, 3.27) ***	0.0052
Ability to identify QI opportunities in case presentations and day-to-day patient care	2.01 (1.23, 2.48) **	3.72 (3.37, 4.20)	1.38 (0.73, 2.37) **	< .0001
Confidence in participating in a QI project	1.75 (0.96, 3.05)	3.42 (3.06, 4.24)	1.44 (0.58, 3.07)	< .0001
Ability to communicate with colleagues about QI principles	1.89 (1.27, 2.30) *	3.17 (2.93, 3.48)	1.09 (0.70, 2.23) *	0.0003
Role of QI as part of a physician’s professional activities	2.00 (1.59, 2.72) **	3.42 (3.06, 4.03)	1.11 (0.79, 1.64) **	0.0002
The difference between quality and value	2.06 (1.00, 3.03) ***	3.27 (2.63, 4.04)	1.05 (0.17, 2.10) ***	0.0017

Data are presented as median and interquartile range.

*Missing two responses.

**Missing one response.

***Missing three responses.

When asked about their ability to identify QI opportunities in case presentations and day-to-day patient care, there was a significant improvement in knowledge with the average pre-curriculum score of 2.01 to a post-curriculum average score of 3.72 (p < 0.0001). The residents’ confidence in participating in a QI project increased from 1.75 to 3.42 (p < 0.0001). Residents also felt more knowledgeable in communicating with colleagues about QI principles (pre-curriculum average score = 1.89, post-curriculum average score = 3.17, p < 0.0003).

Discussion

Residents’ extensive exposure to the clinical environment confers an understanding of hospital workflow. A resident can appreciate the minutia of the admission, inpatient and discharge process, describe impediments to care, and frequently have ideas for improvement. However, the ideas for improvement frequently arise without consideration to cost or value.

After exposure to this curriculum, residents stated that they became more aware of how they can change the distribution of resources in order to spend their time valuably. This curriculum also provides a forum for residents to devise ideas for quality improvement, again relying on their unique position and ample time on the clinical units.

Residents were able to apply their knowledge and personal experience with the cost-accounting game of “The Process is Right”. While the objective of the game was to achieve safe vaginal delivery at the lowest cost, the underlying goal is for residents to appreciate how every step of clinical decision-making does alter the total cost, which may or may not affect the outcome. The wide range of spending for the game perhaps reflects the clinical practice variation that a QI curriculum like ours can aim to resolve.

Our curriculum integrates QI into training programmes. With knowledge of quality improvement, residents will graduate with better understanding of the value of healthcare. This is relevant to residents’ careers after residency, as with new healthcare legislation in the news [12], patient satisfaction and outcomes will likely be more scrutinised.

The strength of our curriculum is that it allows residents to achieve their ACGME milestones. By including residents of all levels, our comprehensive curriculum allowed residents to achieve their milestones earlier in their training. This method of teaching cost-conscious care is novel as it combines process mapping and the cost of the individual steps of care into the science of improvement and does so in a way that trainees can readily understand it. This curriculum also gave trainees the opportunity to increase value and improve care processes.

The limitation of this project is our small sample size, and the cost-accounting portion is difficult to replicate unless the game is available. The costs are unique to our institution. However, the curriculum is adaptable for other institutions. Another limitation was that we did not have a control group for comparison as all of our residents participated in this curriculum. By introducing stewardship of cost-conscious care, we wanted all of our residents to carry this skill set into their professional careers and make an impact with their clinical decision-making.

Conclusion

Our curriculum is an effective educational programme to increase residents’ self-perceived knowledge on core quality improvement principles. The curriculum relies on residents’ unique perspective and familiarity with workflow, integrated with the TDABC method, to design interventions to decrease cost.