Timing of surgical antibiotic prophylaxis administration: complexities of analysis.

BACKGROUND: The timing of prophylactic antibiotic administration is a patient safety outcome that is recurrently tracked and reported. The interpretation of these data has important implications for patient safety practices. However, diverse data collection methods and approaches to analysis impede knowledge building in this field. This paper makes explicit several challenges to quantifying the timing of prophylactic antibiotics that we encountered during a recent study and offers a suggested protocol for resolving these challenges. CHALLENGES: Two clear challenges manifested during the data extraction process: the actual classification of antibiotic timing, and the additional complication of multiple antibiotic regimens with different timing classifications in a single case. A formalized protocol was developed for dealing with incomplete, ambiguous and unclear documentation. A hierarchical coding system was implemented for managing cases with multiple antibiotic regimens.
INTERPRETATION: Researchers who are tracking prophylactic antibiotic timing as an outcome measure should be aware that documentation of antibiotic timing in the patient chart is frequently incomplete and unclear, and these inconsistencies should be accounted for in analyses. We have developed a systematic method for dealing with specific problematic patterns encountered in the data. We propose that the general adoption of a systematic approach to analysis of this type of data will allow for cross-study comparisons and ensure that interpretation of results is on the basis of timing practices rather than documentation practices.

Introduction

The timing of prophylactic antibiotic administration is a patient safety outcome that is recurrently tracked and reported. Compliance to antibiotic prophylaxis guidelines [1-9], the effect of interventions on antibiotic prophylaxis timing [10-20], and the relationship between prophylaxis timing and the incidence of surgical site infection [21-24] are quality measures that are described in the literature. The results of the published work are unanimous in suggesting an association between properly timed antibiotic prophylaxis and a decreased rate of surgical site infections [21-30]. It is apparent that these timing data are worthwhile data to collect and the interpretation of these data has important implications for patient safety practices. However, diverse data collection methods and approaches to analysis make comparisons across studies difficult and impede knowledge building in this field. This paper describes several challenges to quantifying the timing of prophylactic antibiotics that we encountered during a recent study (Lingard L, Regehr G, Cartmill C, Orser B, Espin S, Bohnen J, Reznick R, Baker R, Rotstein L, Doran D: Evaluation of a preoperative team briefing: Improved communication routine results in improved clinical practice, submitted) and offers a suggested protocol for resolving these challenges in order to standardize future analyses and facilitate the comparison of future studies in this field.

Background

We implemented a pre-operative team briefing that used a checklist as a guide and included members from all three professions in the operating room: nursing, anesthesia and surgery. As a marker of the briefing's effect on patient safety, we evaluated its impact on the timing of antibiotic prophylaxis administration, using a pre-intervention, post-intervention study design. The original objective of this research was to analyze the antibiotic timing data as a binary variable (on-time, within one hour prior to incision or not-on-time) and perform a simple chi-squared analysis between the pre and post phases of the study. However, while the vast majority of studies that offer data on antibiotic prophylaxis timing present simple data collection methods that lead to straight-forward analyses and interpretation, we found that the collection of data on antibiotic prophylaxis timing was a complex process requiring intricate data deciphering, coding and decision-making about the specific data to include and exclude from analysis. Two clear challenges manifested during the data extraction process: the actual classification of antibiotic timing, and the additional complication of multiple antibiotic regimens with different timing classifications in a single case. Each of these will be elaborated in turn.

Challenges

Data extraction and classification

The first challenge was in attempting to fit all of the data extracted from the patient charts into one of two categories (on-time or not-on-time). For this study, antibiotics that were documented in the chart documents as administered within 1 hour prior to the time of incision recorded on the operative record were coded as on-time, a time period used extensively in the current literature to distinguish timely and untimely antibiotic administration[1,3,4,10-16,18,19,22,24,31-33]. Antibiotics that were documented as administered outside of this 1 hour window were coded as discordant (clearly not on time).

However, we encountered many instances of incomplete, ambiguous, and unclear documentation in the chart documents in a manner that appears to have been experienced by other researchers who have tried to document antibiotic timing[3,4,8,9,13,15,19,20]. However, despite these identified ambiguities and problems of documentation in the literature, methods of handling complex chart data are quite inconsistent across the studies and reported only sporadically. Several studies report the exclusion of cases without documentation of the antibiotic administered or the time administered[3,4,15]. Several other studies make note of the variability in antibiotic administration documentation practices[9,13], but do not provide details of how this variability was accounted for in analyses. A few studies note the problem of missing data[8,19], but do not provide specific details of how these missing data were interpreted or dealt with in analyses.

Because of the high level of ambiguity and undocumented cases in our research sample (20% of cases) we were hesitant to exclude these cases from the sample. Thus, we developed a formalized protocol for dealing with these cases (Table 1). Situations where preoperative antibiotics were ordered but not documented as given, or where antibiotics were neither ordered nor documented as given but were a requirement of the procedure, were given a code of not-documented. Antibiotics that were documented as given, but without clear chart documentation of the associated timing, were coded as ambiguous-timing.

Table 1

Description of antibiotic codes applied to data documented in patient charts

Codes applied to data	Description of coded data	N cases with code*
On-time	Antibiotic documented; time documented; time within1 hour prior to incision	449
Discordant	Antibiotic request documented; delivery time documented; time not within 1 hour prior to incision	93
Not-documented	Antibiotic request not documented and delivery not documented	95
Ambiguous-timing	Antibiotic request documented; delivery documented but not with associated time	42

*Total cases collapsed across pre-test and post-test study phases.

For the purposes of analysis (to determine the effect of the intervention on antibiotic timing) the data describing various problems with timing were combined in a variety of ways to represent three levels of timeliness problems: clearly not on time; clearly problematic; and potentially problematic (Table 2). Separate χ2 analyses were performed for these three different levels of timeliness problems for the purposes of understanding the impact of the various categorizations on the conclusions.

Table 2

Codes and categories included in each χ2 analysis:

	categories used in analyses	Codes included in categories of untimeliness	N cases in category*
	On time		449
Analysis 1	clearly-not-on-time	discordant	93
Analysis 2	clearly-problematic	discordant + not documented	188
Analysis 3	potentially-problematic	discordant + not documented + ambiguous timing	230

*Total cases collapsed across pre-test and post-test study phases.

Multiple antibiotic regimens

The second significant challenge we encountered was in interpreting prophylaxis regimens that involved multiple antibiotics. Every possible combination of not-documented, ambiguous-timing, not-on-time and on-time codes was encountered for procedures with multiple antibiotic regimens.

Again, this predicament has been described sporadically in the literature and dealt with in a variety of ways. Some studies have avoided the predicament by studying procedures and populations that routinely require a single antibiotic for prophylaxis[7,9,12,14,23]. Several studies evaluate each drug separately and then assess the complete antibiotic course by combining the separate evaluations, with divergences from the guidelines for one drug leading to a final assessment of the total prophylactic course as discordant with the guidelines[5,17,24]. Hawn et al. analyze only the first antibiotic administered in the dataset[4].

In our study, we decided that each case should have a single code regarding antibiotic timeliness even if multiple antibiotics were to be administered. To accomplish this, each antibiotic relevant to the case was evaluated independently and given an antibiotic code according to the rules described in table 1. Then, to assess overall antibiotic timeliness of the case, these codes were combined in a hierarchical fashion and one of these codes chosen to represent the entire case as follows. Given that patient safety is unequivocally at risk when antibiotics are given either too early or too late, relative to the time of incision [21-29], and based on the approach described in the literature[5,17,24], we decided that a case would be coded as on-time only if every prescribed or administered antibiotic in the procedure was given on time. For cases with at least one discordant antibiotic, the case was coded as discordant, regardless of the codes for the other antibiotics. A code of not-documented took precedence over a code of ambiguous-timing.

Conclusion & implications for research

Researchers who are tracking prophylactic antibiotic timing as an outcome measure should be aware that documentation of antibiotic timing in the patient chart is frequently incomplete and unclear, and these inconsistencies should be accounted for in analyses. While some studies appear to have avoided encountering these complexities by using data collection methods such as 'real-time' data collection [5,31] or collecting antibiotic timing data in the OR[11,14,15], such techniques are not always possible, leading to reliance on the documentation from the patient chart. Previous studies provide hints for how to manage these complexities, but we have proposed a systematic method for dealing with specific patterns encountered in the data. There are several advantages to the method we have developed. The use of this hierarchical coding model and performing three separate χ2 analyses allowed us to keep data in the sample rather than excluding a large proportion of cases because of poor documentation practices. Although the clearly-not-on-time analysis is most conservative and provides the most certainty that timeliness is due to real administration practices rather than documentation practices, it provides the smallest sample size, as it requires the elimination of cases with ambiguous timing in the documentation and cases with no documentation regarding antibiotics. By contrast, the potentially-problematic coding provides a larger sample, but we are less confident that the data represent administration rather than documentation practices. This method allows the researcher to balance the issues of sample size and data-accuracy and to explore this balance from three different levels of certainty. Future researchers will need to decide whether this proposed plan is suitable to the data sources and consider tailoring the system to account for variability in antibiotic characteristics (it may be appropriate to classify timeliness using a cut-off point other than 1 hr for certain antibiotics with different biological properties). While this analysis classifies antibiotic administration practices strictly based on timing, overall appropriateness of the antibiotic choice was not considered. Future research might investigate how to classify clearly inappropriate cases of antibiotic administration, such as the administration of a medication that is contra-indicated due to patient allergy, in the context of this type of classification system. We propose that the general adoption of a systematic approach to analysis of this type of data will allow for cross-study comparisons and ensure that interpretation of results is on the basis of timing practices rather than documentation practices.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

CC contributed to data acquisition, analysis and interpretation of data, statistical analysis, drafting of the manuscript, and provided administrative, technical and material support. LL contributed to study design, data acquisition, analysis and interpretation of data, drafting of the manuscript, obtaining funding and study supervision. GR contributed to study design, analysis and interpretation of data, statistical analysis and drafting of the manuscript. SE contributed to study design, data acquisition, analysis and interpretation of data and critical revision of the manuscript. JB contributed to study design, analysis and interpretation of data and critical revision of the manuscript. RB contributed to study design, analysis and interpretation of data and critical revision of the manuscript. LR contributed to study design, analysis and interpretation of data and critical revision of the manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/9/43/prepub