Impact of an educational training program on pharmacists' documentation practice at a teaching hospital.

OBJECTIVES: This study aims to determine the impact of an educational training program on the quantity and quality of the pharmacists' documentation practice at the Winchester District Memorial Hospital.
METHODS: This study is a part of an evaluation for continuous quality improvement, performed applying a pre- and post-test model. The primary endpoint was the number of Pharmacists' Patient-Care Records in patient charts with the listed action codes. Charts of 80 patients were surveyed at three points (two months before and one and three months after the sessions) for 240 charts. Additionally, a pre- and post-educational questionnaire was administered using case scenarios and the Pharmacists' Patient-Care Record code system.
RESULTS: The number of charts containing Pharmacists' Patient-Care Records with Action Codes listed is 70%, 73%, and 64% in the first, second, and third chart review rounds, respectively. According to the pre- and post-educational session questionnaire, the average score per pharmacist is 47% and 73%, respectively.
CONCLUSION: The initial estimate of the documentation practice is 50% or less, which is an underestimation. Furthermore, the quantity of pharmacists' documentation is not significantly affected by the educational sessions; however, the quality is improved.

Introduction

The Ontario College of Pharmacists (OCP) emphasise the importance of documentation in the provision of pharmaceutical care. Being a standard of practice, documentation should be factual, complete, timely, and organised. At our centre, The Winchester District Memorial Hospital (WDMH), a standardised Pharmacists Patient-Care Record form was developed and implemented in 2018. However, no formal training has been provided to staff pharmacists since then. Thus, this lack of formal training has resulted in 50% or less pharmacist documentation practice. In 2008, a review paper adduced common perceptions considered as challenges to documentation by pharmacists., These include gaps in understanding the rationale of documentation, professional standards, legal considerations, and liabilities, along with a lack of experience.,

Furthermore, the provision of quality services is listed as a key element in the Canadian Society of Hospital Pharmacists’ (CSHP) Professional Standards for Hospital Pharmacy Practice. Under this standard, components of practice include providing seamless pharmaceutical and patient care services and ensuring documentation of patient care activities. Our goal is to implement a quality improvement intervention to optimise pharmacist documentation practice at WDMH.

According to a retrospective observational study at a Canadian University Teaching Hospital which assessed the level of documentation in patients' medical records, out of a total of 779 patient charts, 72.3% were considered to have minimal documentation (at least 1 intervention described in writing), 55.5% had sufficient documentation (at least 1 note written during the patient's hospitalization), and 10.4% had extensive documentation (appropriate number of notes associated with the duration of hospitalization). The investigators have suggested further analysis of educational presentations and documents to raise pharmacists' and students' awareness regarding practice standards for documentation.

According to the Pharmacy Mission Statement for The Winchester District Memorial Hospital (WDMH), as part of a multi-disciplinary team, pharmacists are responsible for monitoring and optimizing the outcomes of individual patients’ drug therapy (referenced from an internal document, Appendix A). Additionally, as a part of the provision of pharmaceutical care, pharmacists are expected to document their activities and interventions. As noted in WDMH Clinical Documentation Process (referenced from an internal document, Appendix A), the documentation of pharmaceutical care in the patient care record serves the following purposes:

Communication with the health care team

Effective interdisciplinary care

Providing a written record of quality of care

Documenting a record of services provided by pharmacists

Liability protection.

Currently at WDMH, pharmacist documentation remains a permanent part of the patient care record in three sections of the patient chart. Pharmacists document all verbal and written suggestions, activities, and monitoring and interventions in the Pharmacists Patient-Care Record (Appendix B). Pharmacists’ suggestions are documented irrespective of whether they are implemented or not. This record is placed in the Allied Health/Miscellaneous section of the patient chart and at the end of the progress notes.

Pharmacists also communicate with the health care team to support continuity of care by recording recommendations and interventions in the interdisciplinary progress notes. In the physician order sheets, pharmacists may document recommendations (as ‘Pharmacy Suggests’), pharmacy auto-substitutions, clarification of orders, verbal orders, or orders for pharmacist-initiated therapeutic drug monitoring. In addition, the current documentation system at WDMH has not been evaluated recently. Pharmacist documentation is believed to differ among various clinical services. A training session had been provided with the implementation of the system in 2018. However, since then, no formal follow-up has been conducted on documentation practices. Recently hired pharmacists are provided a copy of the Pharmacy Department's Clinical Documentation Process (Appendix A) and introduced to documentation practices during an educational session with their Chief Clinical Pharmacist. Changes in staffing and current lack of formal refresher training for documentation practices may have impacted the extent and quality of pharmacist documentation practice at WDMH. Moreover, there is a lack of published research on documenting pharmacist interventions. To date, studies have focused mostly on the implementation of pharmacist documentation systems in a hospital setting and the comparison of different documentation systems. A recent survey of pharmacy directors of American hospitals showed that 61% were dissatisfied with their current documentation system for pharmacist interventions. Therefore, evaluating the current pharmacist documentation practice at WDMH and the impact of an educational session were deemed to constitute valuable research opportunities. Potential benefits of this study include improving the quality and extent of documentation by clarifying the practice for pharmacists and reinforcing the need for documentation, as well as identifying areas requiring improvement.

Materials and Methods

Study design

This study evaluates continuous quality improvement through a pre- and post-test model. The primary analysis endpoints include the number of Pharmacist's Patient-Care Records in patient charts with action codes listed. Eighty patient charts are surveyed two months before the educational sessions, and then 80 charts one month and three months after these sessions (Appendix C).

A questionnaire is completed by WDMH pharmacists before and after the educational sessions, using 15 case scenarios to determine their preferred action codes (Appendix D). Incidentally, action codes are correctly used approximately 50% of the time. These codes are selected based on assumptions due to lack of previous formal training on documentation skills. The number of correct codes used in the pre- and post-training questionnaires serves as a surrogate endpoint to analyse a possible tendency towards an improvement in the quality of documentation by the pharmacists and a standardisation of the use of documentation codes. The questionnaires are of equivalent difficulty and in a multiple-choice format. The pre-training questionnaire include scenarios from training sessions provided to pharmacists at WDMH in 2009. Additionally, the post-training questionnaire contains scenarios of a similar structure developed by the principal investigator with feedback from the two Chief Clinical Pharmacists and the Pharmacy Director. The questionnaires are sent to all pharmacists working at WDMH, with an expected response rate of at least 70%. While the pre-survey encourage the pharmacists to identify barriers to their documentation practice, the post training questionnaire include a subjective evaluation of the quality of the educational session.

Study setting

Following approval of the study by WDMH Research Ethics Board (REB), charts have been reviewed at WDMH Campus. To obtain 80 charts for each of the three chart reviews, pharmacists are randomly selected at the WDMH with a total of 16 pharmacists. The principal study investigator then randomly selects five charts for data collection from the clinical services or wards on which each of the selected pharmacists are currently working (see Figure 1, Planned Chart Selection). For consistency, the documentation of pharmacists selected in the initial chart review are surveyed in all three chart reviews.

Figure 1

Planned chart selection.

Eighty patient charts are surveyed at three points in time (two months before the educational sessions and one and three months after the sessions) for a total of 240 charts. The timing of the chart reviews is not disclosed prior to their occurrence to minimise temporary changes in documentation practice. The questionnaire is administered using a web-based survey engine (www.surveymonkey.com) and the pharmacists notified via e-mail. The pre-training questionnaire is distributed one month before the educational sessions, while the post training questionnaire is distributed one month after the sessions. A reminder is sent to the pharmacists one week before the deadlines to encourage completion of the questionnaires.

Inclusion/exclusion criteria

Charts are surveyed for patients who have been in the hospital for at least 5 days but less than 14 days in order to standardise the number of days recorded by pharmacists in patient charts.

Description of intervention

The mandatory educational session for pharmacists is conducted on four separate days (1 h each over lunchtime). The purpose of documentation is emphasised: the Pharmacy Department's Clinical Documentation Process and how to use the action codes. Furthermore, the sessions are interactive, and the scenarios presented in the pre-training questionnaire are reviewed to determine the appropriate useable codes.

Results

Data analysis

The data collected and the results from the questionnaire are analysed using descriptive statistics (Table 1, Primary and Secondary Endpoints). The quantity of pharmacist documentation is estimated by determining the number of blue sheets with action codes listed. The quality of pharmacist documentation is estimated using the results of the pre- and post-educational session questionnaires, as well as by comparing action codes on the Pharmacist's Patient-Care Record to the usage description for each code.

Table 1

Primary and secondary endpoints.

Primary Endpoint

Number of PPCR with AC listed

Secondary Endpoints

Number of PPCR in patient charts

Number of days since admission

Number of weekend days

Number of pharmacist entries on PPCR

Percent recorded entry dates/total

Percent recorded AC/suggested AC

Percent recorded pharmacist initials/total entries

Number of pharmacists/PPCR

Percent of signatures (bottom of the sheet)

Percent of dates (bottom of the sheet)

Percent legibility of signatures/dates (bottom of the sheet)

Number of entries/PPCR

Percent follow-up plan noted/entry (P,O,T)

Number of days since last entry

Number of days from admission to last entry

Number of pharmacist notes in physician orders

Percent of pharmacist notes in physician orders noted on PPCR

Number of pharmacist notes in progress notes

Percent of pharmacist notes in progress notes noted on PPCR

PPCR: Pharmacist Patient Care Record, AC: Action Codes, P: progress notes, O: physician orders, T: team awareness.

Quantity of documentation

In total, the documentation of 23 pharmacists is surveyed, as there were scheduling changes (i.e. summer holidays) and lack of charts meeting inclusion criteria. The primary outcome is the number of Pharmacist's Patient-Care Records with action codes listed. The chart review #1 shows 70% of charts containing Pharmacist's Patient-Care Records with action codes listed, with no difference between the participants (Table 2, Chart Reviews: Pharmacist's Patient-Care Records and Action Codes). This result is an underestimate of our initial hypothesis of documentation practice of 50% or less. Additionally, there is no difference in the amount of documentation one month after the educational sessions, as shown by chart review #2 (73% of charts contained Pharmacist's Patient-Care Records with action codes listed). Three months after the educational sessions, the quantity of documentation is not significantly different from that seen in the first or second chart reviews (64% of charts contain Pharmacist's Patient-Care Records with action codes listed). Moreover, the number of action codes per patient chart does not vary between the three chart reviews (Table 3, Chart Reviews: Action Codes per Patient Chart). The average number of weekend days is slightly higher in chart review #2 (3.8 weekend days) than the other chart reviews (2.5 and 3.0 weekend days for chart reviews #1 and #2), but this does not appear to change the extent of documentation.

Table 2

Chart reviews: pharmacist's patient-care records and action codes.

Chart Reviewa	PPCR with AC listed/total charts (Batch #1 and Batch #2)	PPCR with AC listed/total charts (Batch #1)	PPCR with AC listed/total charts (Batch #2)
#1	56/80 = 0.7	28/40 = 0.7	28/40 = 0.7
#2	58/80 = 0.73	32/40 = 0.8	26/40 = 0.65
#3	51/80 = 0.64	27/40 = 0.68	24/40 = 0.6

PPCR: Pharmacist's Patient-Care Records, AC: Action codes.

Timing of Chart Reviews: #1: 2 months pre-educational training sessions; #2: 1 month post-educational training sessions; #3: 3 months post-educational training sessions.

Table 3

Chart reviews: action codes per patient chart.

	Chart Review #1	Chart Review #2	Chart Review #3	All Chart Reviews
Total AC/Total Charts	260 codes/80 charts	237 codes/80 charts	210 codes/80 charts	707 codes/240 charts
Average AC/chart	3.25	2.96	2.63	2.95

AC: Action Codes.

The most used codes include MON (Efficacy/Toxicity Monitoring) (173/707 = 24%), NEW (Chart Review/New Patient Work-up) (162/707 = 23%), and ADD (Drug Added) (88/707 = 12%). The least used codes include ADR (Adverse Drug Reaction Identified) (0/707 = 0%), CLAR (Order Clarified) (4/707 = 0.6%), and INT (Drug Interaction Managed) (3/707 = 0.4%). Pharmacist's Patient-Care Records are found in 89%, 96%, and 89% of patient charts in chart reviews #1, #2, and #3, respectively. Table 4, Data Collection, provides information of other secondary endpoints.

Table 4

Data Collection.

	Chart Review #1	Chart Review #2	Chart Review #3	Average of all Chart Reviews
Primary Endpoints # of PPCR with AC listed	56	58	51	55
Secondary Endpoints (Averages)
% of PPCR in patient charts	71	77	71	73
# of days since admission	8.35	9.4	8.8	8.8
# of weekend days	2.5	3.8	3.0	3.1
# of pharmacist entries	2.5	2.5	2.4	2.5
% recorded entry dates/total	97%	97.2%	95.6%	95.1%
% recorded AC/suggested AC	77.5%	81.2%	76.6%	78.4%
% recorded pharmacist initials/total entries	82.2%	78.8%	86.5%	82.7%
# of pharmacists/PPCR	1.3	1.3	1.4	1.3
% of signatures (bottom of the sheet)	90.2%	93.2%	85.3%	89.5%
% of dates (bottom of the sheet)	82.1%	81%	72.5%	78.6%
% legibility of signatures & dates (bottom of the sheet)	96.5%	93.9%	86%	92.1%
% follow-up plan noted/entry (P,O,T)	40.7%	32.9%	32.8%	35.4%
# of days since last entry	2.3	4.6	4.4	3.7
# of days from admission to last entry	6	5.1	5.4	5.5
# of pharmacist notes in physician orders	75	79	77	77
% of pharmacist notes in physician orders noted on PPCR	26.7%	16.5%	10.4%	17.7%
# of pharmacist notes in progress notes	11	8	6	8.3
% of pharmacist notes in progress notes noted on PPCR	45.5%	25%	50%	40%

PPCR: Pharmacist's Patient Care Record, AC: Action Codes, P: progress notes, O: physician orders, T: team awareness.

Quality of documentation

An area for improvement is seen in noting the follow-up plan for action codes. In all chart reviews, only 35.4% of follow-up plans (i.e. notes in progress notes, physician orders, or informing the team) are noted, with little difference between chart reviews. Of all notes written by pharmacists in the physician orders and interdisciplinary progress notes, only an average of 17.8% and 39.8% are noted in the Pharmacist's Patient-Care Record in all three chart reviews, respectively (Table 4, Data Collection).

Discussion

Prior to the training sessions, the quantity of documentation is significantly higher (70%) than the initial estimate of 50% or less due to the assumption of lack of formal training in documentation. This result indicates that pharmacists at WDMH documented patients' charts to a higher degree than expected and highlights their understanding of seamless care for patients and communication among the members of the healthcare team. The quantity of documentation does not significantly differ following the educational training sessions as noted by the results of the chart reviews: 70%, 73%, and 64% of charts containing Pharmacist's Patient-Care Records with action codes listed in chart reviews #1, #2, and #3, respectively. No significant difference is noted in the quantity of pharmacist documentation across participants. One explanation for this observation may be due to challenges in modifying habits within a short time frame. For incentivizing documentation skills in the future, further avenues should be explored to increase the quantity of documentation to the intended level of 80% or more through regular reviews of pharmacist documentation, as well as anonymous publication of the highest, lowest, and average amount of documentation. Increasing pharmacist staffing levels may also help improve quantity of documentation by allotting more time to documentation. A target of 100% pharmacist documentation may be unrealistic at WDMH considering short lengths of stay and current staffing issues.

According to the results of the questionnaires, the quality of documentation improved following the educational sessions. The average scores of 47% and 73% (before and after the educational sessions, respectively) demonstrate a better understanding of the definition of the action codes. In this respect, the educational sessions were beneficial to the pharmacists. According to OCP, systematic records with established codes can address issues of incomplete and inconsistent documentation. Action codes encourage systematic documentation which supports the results of our study from the perspective of continuous quality improvement. However, the extent of pharmacist documentation may have been affected by shorter lengths of stay and the number of weekend days (when pharmacist documentation is minimal due to low staffing).

To maximise the documentation time for in-patient charts, selected patients include those admitted for at least 5 days and less than 14 days. The three chart reviews showed similar number of days since admission, with an average of 8.8 days since admission in all these reviews. Additionally, there were similar number of pharmacist entries in the three chart reviews. Since this is a retrospective review of pharmacist documentation in patient charts, it is difficult to analyse the quality of pharmacist documentation (unless present when an intervention is made, it is impossible to determine whether the correct documentation was performed). Therefore, questionnaires are developed as a surrogate marker. Response rates for the two questionnaires were 54% and 42%, respectively. The lower response rate in the second survey may partially be due to its organisation during the summer months, when many pharmacists are on vacation.

The initial questionnaire contains scenarios from training sessions for pharmacists at WDMH in 2009. The principal investigator developed the second survey following the same style as the first (scenarios presented, then possible choices of codes provided as multiple-choice answers). After enhancing the clarity of the questions, the members of the pharmacy management team conduct a pilot test on the questionnaires. Overall, the complexity of the scenarios remained the same between the two questionnaires.

The sessions received an overall positive response; in the second questionnaire, 75% of pharmacist responders found the documentation session to be helpful and another 15% found it to be somewhat helpful.

Since this is a continuous quality improvement study, we could not conduct a traditional randomised, placebo-controlled trial, which may have introduced some methodological bias. In addition, as it was not possible at the time to conduct the study in multiple centres to obtain a greater sample size, we used surrogate markers to evaluate our primary and secondary outcomes based on our intervention of educational training. This limited the number of pharmacists to be included in our study, along with some challenges due to scheduling issues. Furthermore, to maximise the internal validity and applicability of the results, we highlight the opportunities for further research using a systematic approach in a multi-site, randomised, placebo-controlled trial, with long-term follow up.

Our centre can further explore the application of an electronic clinical decision support system (CDSS) which serves as a tool for automated, near real-time surveillance, alerting, analysis, and reporting. Some clinical applications of CDSS include timely identification of potential adverse drug events, IV to PO antibiotic conversion opportunities, drug-bug mismatches, and discontinuation or de-escalation opportunities facilitating pharmacist documentation. After the CDSS infrastructure implementation at an acute care community hospital in East Texas, documentation of clinical interventions by pharmacists demonstrated value and resulted in the approval and hiring of two additional full-time clinical pharmacists. Thus, CDSS may prove beneficial for our centre at WDMH.

Conclusion

The initial estimate of documentation practice at 50% or less is an underestimation. Although the quantity of pharmacist documentation is not significantly affected by the educational sessions for one month and three months’ chart reviews, the quality of documentation improved.

There is a substantial increase in the average score of correct action scores selected per pharmacist on the post questionnaires (surrogate marker of quality of documentation). The average score per pharmacist on the pre-educational session questionnaire is 47% (7/15 scenarios with correct action codes selected). The average score per pharmacist on the post-educational session questionnaire is 73% (11/15 scenarios with correct action codes selected).

Although the number of action codes satisfying the specified definitions is similar between all three chart reviews (82%, 87%, and 87%, for chart reviews #1, #2, and #3, respectively), this is a less sensitive measure of quality of documentation, as it is difficult to retrospectively assess the appropriateness of the utilisation of the codes. However, as a quality indicator, documentation practice has improved from the education sessions and shows benefits in clinical practice. Compliance with recording dates and initials for entries, as well as signatures and dates on the bottom of the Pharmacists Patient-Care Records, is 80–90%, respectively, with little difference between the chart reviews. According to all chart reviews, pharmacist signatures are legible 92.1% of the time.

Attendance of the educational training sessions

Out of a possible 52 attendees, 38 pharmacists attended one of the sessions over the four days (73%). This included five pharmacy residents. Limitations are attributed to conducting the study in a single site with variability in pharmacist scheduling because some pharmacists were on vacation.

Questionnaire response rate

For the first questionnaire, there are 42 responders out of a possible total of 78 pharmacists (response rate: 54%). However, some of the pharmacists contacted are casual employees who may not have felt comfortable answering the questionnaire, or did not read their WDMH e-mail within the time frame allotted for the questionnaire. For the second questionnaire, fewer pharmacists are contacted (the casual pharmacists were not contacted because they did not attend the training sessions). Additionally, there are 25 responders out of a possible total of 59 pharmacists (response rate: 42%).

Recommendations

As this was a continuous quality improvement study, the study findings are limited to WDMH. Nonetheless, hospitals using a similar pharmacist documentation code system may note an improvement in the quality of pharmacist documentation if educational sessions and questionnaires similar to the ones described in this study are utilised for training and refresher sessions.

Barriers to pharmacist documentation were identified in the initial survey. The issues listed in Table 5 demonstrate that familiarity with the action codes, albeit not the most identified barrier, was considered an important barrier in 50% of respondents. Some responders in the second questionnaire suggested that pharmacists may benefit from periodic refresher sessions.

Table 5

Barriers to documentation – responses from Questionnaire #1.

Please rate the following according to what you consider to be important barriers to documentation in your practice
	Very Important	Important	Somewhat Important	Not Important	Response Total
Time	57% (24)	29% (12)	12% (5)	2% (1)	42
Chart availability	31% (13)	36% (15)	33% (14)	0% (0)	42
Familiarity with action codes for the blue sheet	24% (10)	26% (11)	40% (17)	10% (4)	42
Total Respondents					42

Areas for improvement were identified from the study. Seven ‘Documentation Dilemmas’ identified were forwarded to the Clinical Affairs committee of the Pharmacy Department for resolution. Some of these dilemmas arose from some confusion among the pharmacists regarding documenting in the chart versus documenting their workload for workload measurement. Additionally, ward clerks should be reminded to place the Pharmacist's Patient-Care Records in all patient charts. The action code definitions and WDMH Clinical Documentation Process should be placed on WDMH Infonet (the hospital's intranet system) for easy access by all pharmacists.

Pharmacists should also be reminded to note the follow-up plan for action codes on the Pharmacist's Patient-Care Record for demonstrating previous actions to facilitate further interventions (i.e. any progress notes, recorded notes in physician orders, or the need to notify the team). Furthermore, pharmacists need to sign legibly and mark the date at the bottom of the Pharmacist's Patient-Care Record while writing their first entry.

Periodic refresher sessions may be beneficial. In addition, recently hired pharmacists should complete a questionnaire with scenarios similar to those used in this study.

Source of funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors have no conflict of interest to declare.

Ethical approval

This study was approved by ERB committee, received on 17 Dec 2020, number# 8945-23.

Authors' contribution

AE conceptualised and prepared the original manuscript, conducted data curation, analysed the paper, conducted literature search, collected data, as well as completed writing, reviewing and editing the manuscript, and brainstormed ideas. YT conducted research, provided research materials, collected and organised data, and conducted the literature review. AEL analysed, interpreted data, and conducted the literature review. ZY assisted in writing the initial and final draft of the article, and provided logistic support. JJ conducted research, provided research materials, collected and organised data, conducted literature review. All authors have critically reviewed and approved the final draft and are responsible for the content and similarity index of the manuscript.