The BACH classification of long bone osteomyelitis.

OBJECTIVES: The aim of this study was to assess the clinical application of, and optimize the variables used in, the BACH classification of long-bone osteomyelitis.
METHODS: A total of 30 clinicians from a variety of specialities classified 20 anonymized cases of long-bone osteomyelitis using BACH. Cases were derived from patients who presented to specialist centres in the United Kingdom between October 2016 and April 2017. Accuracy and Fleiss' kappa (Fκ) were calculated for each variable. Bone involvement (B-variable) was assessed further by nine clinicians who classified ten additional cases of long bone osteomyelitis using a 3D clinical imaging package. Thresholds for defining multidrug-resistant (MDR) isolates were optimized using results from a further analysis of 253 long bone osteomyelitis cases.
RESULTS: The B-variable had a classification accuracy of 77.0%, which improved to 95.7% when using a 3D clinical imaging package (p < 0.01). The A-variable demonstrated difficulty in the accuracy of classification for increasingly resistant isolates (A1 (non-resistant), 94.4%; A2 (MDR), 46.7%; A3 (extensively or pan-drug-resistant), 10.0%). Further analysis demonstrated that isolates with four or more resistant test results or less than 80% sensitive susceptibility test results had a 98.1% (95% confidence interval (CI) 96.6 to 99.6) and 98.8% (95% CI 98.1 to 100.0) correlation with MDR status, respectively. The coverage of the soft tissues (C-variable) and the host status (H-variable) both had a substantial agreement between users and a classification accuracy of 92.5% and 91.2%, respectively.
CONCLUSIONS: The BACH classification system can be applied accurately by users with a variety of clinical backgrounds. Accuracy of B-classification was improved using 3D imaging. The use of the A-variable has been optimized based on susceptibility testing results.Cite this article: A. J. Hotchen, M. Dudareva, J. Y. Ferguson, P. Sendi, M. A. McNally. The BACH classification of long bone osteomyelitis. Bone Joint Res 2019;8:459-468. DOI: 10.1302/2046-3758.810.BJR-2019-0050.R1.

BACH is a new classification system for long bone osteomyelitis. This study serves to validate this classification in terms of interuser assessment and optimize specific variables that are included in the classification.

BACH can be applied with good accuracy and reproducibility to varied cases of long bone osteomyelitis.

To classify the bone involvement of osteomyelitis, a clinical imaging utility should be used that allows 3D visualization of the pattern of infection.

The absolute number of resistant susceptibility tests performed for a bacterial isolate has a high correlation with multidrug resistance as defined by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID).

This study assesses the reproducibility of the BACH classification system with users who have a variety of clinical backgrounds and experience in the management of osteomyelitis.

This study proposes a classification system that can be used in clinical practice and in subsequent prospective studies.

Introduction

Osteomyelitis is a complex disease that carries a significant burden for patients.[1] Specialist knowledge and treatment are essential for the successful management of patients with bone and joint infections.[2] A method of stratifying patients is necessary to ensure that referral to specialist teams occurs at an appropriate time during the management process. One tool to address this need is a clinically applicable method of classifying osteomyelitis.

With the aim of establishing a new classification system, we previously performed a systematic review that revealed 13 classification systems for osteomyelitis.[3] After an in-depth analysis and using a multidisciplinary approach, we hypothesized that four key variables for classification would guide prognosis and requirement for referral to a specialist treatment centre. These were the following, from which the acronym BACH arises: 1) the bone involvement (B-variable); 2) the antimicrobial options (A-variable); 3) the coverage of the soft tissue (C-variable); and 4) the host status (H-variable).[3] The bone involvement was divided into three options based on the presence of cavitary involvement (which includes a cortical, medullary, and non-segmental corticomedullary infection) (B1), segmental involvement (B2), or the concomitant involvement of a joint irrespective of the segmental or cavitary infection (B3). The antimicrobial options used the European Society of Microbiology and Infectious Diseases (ESCMID) criteria for stratifying isolates into multidrug-resistant (MDR), extensively drug-resistant (XDR), and pan-drug-resistant (PDR) isolates,[4] and in case of a foreign body, whether an antimicrobial compound with activity against adhering (biofilm) bacteria was available. The coverage of the soft tissues was classified based on whether patients required plastic surgical expertise for skin closure after surgical excision of the infection. The host status stratified patients based on the presence or absence of comorbidities such as diabetes mellitus, vascular or immune compromise, or the presence of recurrent osteomyelitis following previous reconstructive surgery. The host status also accounted for patients who did not require an operation, were not fit enough to receive an anaesthetic, or declined surgery.

Based on the classification of these individual variables, the complexity of osteomyelitis can be determined. Each variable in BACH is stratified into either ‘uncomplicated’ or ‘complex’. In two variables, the antimicrobial options and the host status, osteomyelitis can also be stratified as having ‘few or no options available’. The overall complexity of the osteomyelitis is determined by classification of the most severely classified variable.

Several steps are mandatory to validate a classification system. First, we analyzed the applicability to a retrospective cohort. The promising results of this investigation have been reported elsewhere.[5] The objectives of the present study are: 1) to evaluate BACH using an interuser assessment; 2) to refine these variables according to this assessment; and 3) to adapt BACH in view of the outcomes of these assessments. Based on these, version 3 of the BACH classification system will be presented.

Materials and Methods

The process of modification and optimization of the BACH classification system is illustrated in Figure 1. In a primary analysis, we measured the applicability of the classification system. In a secondary analysis, we assessed the performance of criteria that demonstrated suboptimal performance in the primary analysis.

Fig. 1

Flowchart presenting the development of the BACH classification of osteomyelitis. Here, the development of BACH version 2 to BACH version 3 is presented. The results of the retrospective assessment have been published previously.[5]

Primary interuser analysis

In all, 20 cases of osteomyelitis were defined. Histories of included cases derived from patients who attended one of two specialist bone infection centres within the United Kingdom (Bone Infection Unit, Oxford University Hospitals, Oxford, United Kingdom and Addenbrooke’s Hospital, Cambridge, United Kingdom). All cases had the diagnosis of osteomyelitis confirmed using a validated composite protocol, as described elsewhere.[6,7] Cases received surgical treatment from October 2016 to April 2017. Anonymized details of the infection were extracted from the patient’s medical notes and entered onto a secure online form. Information presented to the assessors included: the osteomyelitis history (aetiology, site of infection, radiology imaging); microbiology (isolates with susceptibility testing); photographs of the soft tissues overlying the site of infection; and patient comorbidity. Radiology images were presented as single slices of MRI, CT, positive emission tomography (PET)-CT, or radiographs (anteroposterior and lateral) chosen to best illustrate the condition. Users were allowed as much time as required to complete the assessment.

A total of 30 users were recruited to complete the classification of the 20 cases. This number of users allowed inclusion of a wide range osteomyelitis experience, clinical specialty, and centre. Owing to this variation in the user background, this assessment contains comparably more users than reported in previous studies.[8] Users were required to be practising as a trauma and orthopaedic surgeon, plastic surgeon, or physician (infectious diseases, microbiologist, or anaesthetist) and to have completed either membership or fellowship examination in their speciality. Users were invited from a range of hospitals throughout the United Kingdom, Europe, and North America. Users were given an ‘interuser key’ that explained how to apply the classification system without assistance.

Results were presented using a Fleiss’ kappa (Fκ) and interpreted according to Landis and Koch.[9] The percentage classification accuracy was calculated by comparing the user answer with the reference answer, defined by the authors. Linear regression was performed to ascertain whether there was significant association between user attributes and the overall accuracy of classification.

Secondary interuser analysis

Bone involvement: The 3D bone involvement analysis was performed to establish whether there was a limitation with using single images to classify the bone involvement (giving a single slice of a 2D image impression only) and whether this could be improved by using a 3D image viewer (PACS, Insignia Medical Systems, Basingstoke, United Kingdom) as in real clinical practice (thus allowing a 3D impression of the complete bone involvement). The secondary interuser assessment was performed with a selection of nine users who had completed the primary interuser assessment. All nine users were based in one centre owing to access to the 3D clinical imaging system being required. Ten cases that were not included in the primary interuser assessment were selected at random from a pool of cases based on their bone involvement score. All included cases had undergone surgical treatment at the Bone Infection Unit, Oxford University Hospitals, Oxford, United Kingdom between 2013 and 2017. Results were presented using the Fκ and the percentage accuracy. To ensure that the sample population was representative of the total population, the Kolmogrov–Smirnov test was performed. A one-way analysis of variance (ANOVA) was performed to assess the difference between the groups.

Antimicrobial options: microbiology analysis: All cases of long bone osteomyelitis that received surgery in the Bone Infection Unit, Oxford University Hospitals, Oxford, United Kingdom between 2013 and 2017 were included for analysis (n = 253). For validation purposes, a second cohort from the same centre who received debridement surgery between 2001 and 2004 (n = 176) were used. The total number of sensitive and resistant susceptibility tests were recorded for each isolate. For the later 2013 to 2017 cohort, all ESCMID classifiable isolates were classified into MDR or non-MDR based on the ESCMID criteria.[4] Receiver operating characteristic (ROC) curves were constructed for the number of resistant susceptibility tests and the percentage of susceptibility test results that were sensitive. For these measures, a Youden index was calculated[10] and the optimum threshold for defining MDR was therefore defined. This threshold was then validated in an earlier cohort from 2001 to 2004 using the same methods as above.

The coverage of soft tissue and host status: The coverage of the soft tissues and host status variables functioned well in the primary analysis and therefore secondary analysis was not performed.

Results

Bone involvement

The interuser assessment of the bone involvement variable demonstrated accuracy of 77.0% (95% CI 71.5 to 82.5) among users. It had a ‘fair agreement’ among users with an Fκ of 0.479 (95% CI 0.475 to 0.483). The returned answer correlated with the reference answer in 72.3% of B1 cases, 95.0% of B2 cases, and 82.0% of B3 cases. Closer review of the cases that scored in the lowest quartile for accuracy demonstrated that the 2D nature of the online form used for the data collection was a key limitation in the accuracy of classification. This was therefore explored further in a secondary analysis. Other explanations for lower accuracy scores were due to unclear radiology reports and distorted anatomy.

The accuracy of classification in the 3D bone assessment using webPACS was 95.7% (95% CI 90.8 to 100.0), which was significantly higher when compared with the users’ scores in the 2D assessment (p = 0.028, paired t-test) and with the total population who completed the 2D assessment (p = 0.009, ANOVA with Dunnett’s post hoc test) (Fig. 2). This result was substantiated by an improvement in the Fκ in the 3D group to 0.899 (95% CI 0.869 to 0.929), demonstrating an almost perfect agreement between users. The sample population who took the 3D assessment were deemed to be representative of the total population who took the 2D assessment (p = 0.981, Kolmogrov–Smirnov test).

Fig. 2

Chart showing the results of the classification of the bone involvement variable using a 3D web-based picture archiving and communication system (webPACS) system, as performed in clinical practice, versus using a screenshot of the images alone (2D group). The sample population demonstrated a significant improvement when completing the 3D analysis compared with both the total interuser population who completed the 2D analysis (95.7% (95% confidence interval (CI) 90.8 to 100.0) vs 77.0% (95% CI 71.2 to 82.8); p = 0.009, analysis of variance with Dunnett’s post hoc test and the sample population when completing the 2D analysis (79.3% (95% CI 62.3 to 96.3); p = 0.028, Student’s t-test).

Antimicrobial options

The accuracy of classification of the antimicrobial options was 84.4% (95% CI 81.3 to 87.0) with a Fκ of 0.815 (95% CI 0.811 to 0.819). This indicated that there was an almost perfect agreement among users. Despite this, microorganisms with multiple resistant susceptibility tests did not return an accurate answer with the reference answer. The returned answer correlated with the reference answer in only 46.7% of the A2 group and 10.0% of the A3 group (Fig. 3). Therefore, an easier method than the proposed classification was deemed important.

Fig. 3

Results for the antimicrobial options variable in the interuser assessment. a) Bar chart showing the individual answers in each of the cases for the antimicrobial options variable, organized by the reference score. The dotted line is the mean score for the antimicrobial options category (92.2%). b) Heat map demonstrating the returned antimicrobial options score versus the reference antimicrobial options score.

The baseline characteristics for the cohorts are shown in Table I. For the 2013 to 2017 cohort, 72/222 (32.4%) cases had a culture negative osteomyelitis. There were 247 isolates, with 181 (73.3%) being ESCMID-classifiable and 66 (26.7%) being non-ESCMID-classifiable (Fig. 4a). The ESCMID-classifiable isolates comprised of Staphylococcus aureus (94/181, 51.9%), Enterobacteriaceae (57/181, 31.5%), Pseudomonas aeruginosa (15/181, 8.3%), and Enterococcus spp. (15/181, 8.3%) (Fig. 4b). A total of 40 (22.1%) were MDR, XDR, or PDR (for simplicity, termed MDR+) according to ESCMID classification. A similar number of susceptibility tests were performed for non-MDR+ and MDR+ (mean 16.7 vs 16.1).

Table I.

Summary of the baseline characteristics of the two cohorts included in the further analysis of the microbiology

Characteristic	2013 to 2017 cohort	2001 to 2004 cohort
Patients, n
Total	253	176
Excluded	31	10
Included	222	166
Age
Mean age, yrs (sd; range)	51.3 (16.0; 17 to 88)	45.0 (16.5; 12 to 90)
Sex, n (%)
Male	166 (74.8)	121 (72.9)
Female	56 (25.2)	45 (21.1)
Growth of microorganisms, n (%)
Culture-negative osteomyelitis	72 (32.4)	47 (28.3)
Culture-positive osteomyelitis	150 (67.6)	119 (71.7)

Fig. 4

Summary of investigations into the development of the antimicrobial options classification. a) Flowchart of patients in the 2013 to 2017 cohort and the isolates grown on microbiology culture. b) Chart showing the proportion of European Society of Clinical Microbiology and Infectious Diseases (ESCMID) isolates compared between the early (2001 to 2004) and late (2013 to 2017) cohort. c) Receiver operating characteristic (ROC) curve of number of resistant susceptibility tests for ESCMID classifiable isolates in the 2013 to 2017 cohort (area under the curve 98.1% (95% confidence interval (CI) 96.6 to 99.6)). d) ROC curve demonstrating using the percentage of sensitive susceptibility tests for the ESCMID classifiable isolates in the 2013 to 2017 cohort (area under the curve 98.8% (95% CI 98.1 to 100.0)). e) Scatterplot of each isolate from the 2013 to 2017 cohort with percentage resistant susceptibility tests against the total number of susceptibility tests performed. The dotted line represents four resistant tests. Crosses represent multidrug-resistant (MDR) isolates and coloured dots represent non-MDR isolates, according to ESCMID. This substantiates the results of the ROC analysis where 98.3% of MDR isolates are situated on or above the dotted line (four resistant tests).

The number of resistant susceptibility tests were counted for each ESCMID classifiable isolate. ROC curves were constructed using the number of resistant susceptibility tests and then the percentage of sensitive susceptibility test results, versus criteria positive according to MDR+ (Figs 4c and 4d). The area under the curve (AUC) was calculated for each of these ROC curves. Using the number of resistant susceptibility tests, the AUC was 98.1% (95% CI 96.6 to 99.6); using the percentage of sensitive susceptibility tests, the AUC was 98.8% (95% CI 98.1 to 100.0). The Youden index demonstrated that optimum cut-off was the presence of four or more resistant susceptibility tests (Youden index 0.85) and less than 78.5% sensitive susceptibility tests (Youden index 0.96). Using these thresholds, there was a false negative rate of 1.1% (2/181) and a false positive rate of 7.2% (13/181) in this cohort (Table II). This suggests that the presence of resistance to four or more antibiotic susceptibility tests or a total of less than 80% susceptibility tests sensitive is a good measure of multidrug resistance, as defined by the ESCMID criteria (Fig 4e).

Table II.

Summary of the classification of isolates in the 2013 to 2017 cohort when using the thresholds determined in the further assessment of microbiology. Isolates are classified as either European Society of Microbiology and Infectious Diseases (ESCMID) multidrug-resistant (MDR+; reference test) or ESCMID non-MDR+ and this is compared with using either four or more resistant susceptibility tests or less than 80% susceptibility tests sensitive as a measure of an isolate having either high resistance or low resistance

Resistance	Reference test
	ESCMID non-MDR+	ESCMID MDR+
Low resistance	128 (true negative)	2 (false negative)
High resistance	13 (false positive)	38 (true positive)

To validate this method of classifying multidrug-resistant isolates, a similar analysis was performed using an earlier cohort of patients who received surgery in the same centre between 2001 and 2004 (n = 176). This cohort was comparable in terms of baseline characteristics to the 2013 to 2017 cohort (Table I). Here, the number of resistant susceptibility tests gave an AUC of 91.0% (95% CI 85.2 to 96.8) with an optimum threshold of four or more. The percentage of sensitive susceptibility tests gave an AUC of 91.2% (95% CI 85.4 to 96.9) with an optimum threshold of less than 75.8%.

Coverage of the soft tissues

In the interuser analysis, the coverage of the soft-tissue variable gave an accuracy of 92.5% (95% CI 88.5 to 95.6) with a Fκ of 0.729 (95% CI 0.723 to 0.734). Linear regression demonstrated that clinicians who practised within the United Kingdom were more likely to perform better on the coverage of the soft-tissue variable (94.6% vs 85.7%; p = 0.03). However, all plastic surgeons (n = 3) included practised in the United Kingdom and scored a mean of 98.3% on the coverage of the soft tissue variable, which could explain this discrepancy. The three cases that were classified the least accurately were those that included a small (2 cm to 3 cm) lesion on the anterior tibia. In all of these cases, a soft-tissue procedure was required by a plastic surgeon.

Host status

The host status gave an accuracy of 91.2% (95% CI 89.0 to 93.3) and a Fκ of 0.719 (95% CI 0.714 to 0.725) indicating a substantial agreement among users. Comorbidities that were consistently scored (> 95.0% of the time) as H2 included immune disease (e.g. autoimmune neutropenia), chronic kidney disease, and vascular pathology (e.g. stroke). Despite this, comorbidity such as diabetes mellitus or smoking were scored as H2 between 50% and 75% of the time.

Discussion

The present study has demonstrated that the BACH classification system is able to be accurately applied to adults with long bone osteomyelitis by users who have a variety of backgrounds and little previous experience of the system. It has identified the need for optimization of the antimicrobial options variable that allows stratification of resistant isolates by non-specialists.

The definition of the bone involvement variable has remained consistent since the first version of BACH. The investigations performed as part of this study demonstrate the importance of using software that can give a 3D representation of the bone involvement, which improves the agreement and accuracy of the classification. Despite this improvement, the combination of clinical information and imaging must be used to make the classification. By stratifying patients into those who have a nonunion (segmental involvement) or adjacent joint involvement early in the disease process, specialist reconstructive options may be offered at an early stage in management. This is supported by previous work that has demonstrated the difficulty in the management of infected nonunion, even in specialist centres.[11,12]

The initial antimicrobial options variable was based upon the ESCMID criteria for the definitions of MDR, XDR, and PDR.[4] The classification of the multidrug-resistant isolates were classified with a disappointing accuracy of 46.7% for the A2 group (MDR isolates) and 10.0% in the A3 group (XDR and PDR isolates). Despite these criteria being detailed, they are not comprehensive and account for only a few species (i.e. Enterobacteriaceae, Enterococcus spp., Pseudomonas aeruginosa, Acinetobacter spp., and Staphylococcus aureus). In an attempt to ensure that all isolates were classifiable, the version of the classification system used in the interuser assessment included a line that stated that the A2 group should include “bacteria that are not part of the ESCMID criteria with resistance to appropriate antimicrobial agent”. Early identification of multidrug-resistant isolates is essential, as delay in appropriate treatment is thought to be the main cause for mortality, rather than the virulence of the bacteria per se.[13,14] Therefore, an alternative and simple method for the identification of these isolates by a non-specialist is important, especially in cases where access to a specialist infectious diseases physician is not available. Using the number of resistant or sensitive susceptibility tests means that the classification is easily applicable by a non-specialist. The limitation to this method is the variation between laboratories for the susceptibility testing that is performed for each isolate. However, guidelines for susceptibility testing, such as the European Committee on Antimicrobial Susceptibility (EUCAST) guidelines[15] and British Society of Antimicrobial Chemotherapy recommendations, are increasingly being applied.[16] This ensures that relevant and appropriate susceptibilities are being tested as part of a standardized operating procedure.[17] In any case, there is evidence to suggest the importance of inclusion of a specialist infectious disease physician or clinical microbiologist in the care of bone and joint infection.[18]

The coverage of the soft tissues is based on the requirement on whether a wound can be closed directly or whether specialist soft-tissue coverage is required. The accuracy of this variable was 92.5% in the interuser assessment. This variable does not offer advice on when specialist expertise is required. However, this system may be beneficial because it can account for the individual expertise of the surgeon or treating department. The benefit of including a plastic surgeon for limb reconstruction has been reported in the setting of lower limb trauma[19] and also applies to bone infection.[18,20]

The host status can be complex to classify with multiple scoring systems available for general use[21,22] and for use specifically in bone and joint infection.[2325] The distinction between defining how well controlled a comorbidity is and the severity of disease is important in outlining the prognosis to patients. Host status in osteomyelitis can fall into a number of categories: patients without health compromise who wish to have definitive treatment; patients with reversible or irreversible health compromise who wish to have treatment; and patients who decline treatment or who are unfit for definitive treatment. These groups all have differing prognosis. Depending on the progression of their comorbidity, age, and consent for surgery, they may then progress to either H1, H2, or H3 at a future timepoint.

The principle aim of BACH is to guide the need for complex elements of management in long bone osteomyelitis. Version 3 of BACH is shown in Figure 5, with the user key presented as Supplementary Material. Each variable is able to determine the complexity of management and requirement for multidisciplinary input. The overall complexity is governed by the outcome of the most severely classified variable. ‘Uncomplicated’ osteomyelitis may be managed at a non-specialist centre, but either ‘complex’ or ‘limited options available’ osteomyelitis should be referred early to a centre with appropriate expertise. In addition, BACH may also be able to offer prognosis to patients in terms of patient-reported outcomes, surgical complications, and medical complications. However, this has not been assessed by this study. In view of this, a prospective evaluation is planned to assess the classification system for both prognosis and management of bone and joint infection, with potential further refinement in the future.

Fig. 5

The BACH classification system for long bone osteomyelitis. The four key variables are headers with the corresponding criteria for making the classification in each. The green band denotes ‘uncomplicated’ osteomyelitis that can be managed at a non-specialist centre, the amber band denotes ‘complex’ osteomyelitis that should be managed at a centre with specialist expertise, and the red band is ‘limited options available’. The overall complexity of the osteomyelitis is determined by the band of the most severely classified variable.

There were limitations to this study. First, we wanted to ensure that the user key was read in its entirety prior to classification of the cases. In view of this, the password to gain access to the online form was placed at the foot of the user key document. Although this ensured that the document was opened, it did not ensure that it would be studied thoroughly. This is a pragmatic approach to using the classification system, which may reflect uptake in real clinical practice. To ensure that users understood the classification system, users could have been asked to complete a number of practice cases with feedback on performance prior to the full assessment. Second, cases included in the primary interuser assessment were selected to represent the frequency of clinical cases from the retrospective cohort.[5] This resulted in an unequal representation of each variable (for example, there were 13 cases of B1, two cases of B2, and five cases of B3) but allows the resultant accuracy and Fκ scores to indicate the application of BACH in real clinical practice. Third, the alternative method for classification of MDR isolates by non-specialist clinicians was not reassessed using a second interuser assessment. We hypothesize that counting both the total number of performed susceptibility tests and number of sensitive/resistant test results would be easily applicable by clinicians. In addition, the method depends on relevant and appropriate susceptibility testing of a meaningful number of antibiotics. However, this will be re-evaluated for accuracy and agreement between users in the future. Finally, the improvement observed in the secondary interuser assessment of the bone involvement variable could be attributable the classification of only one variable and the inclusion of fewer cases. In the primary assessment, a total of 80 classifications were required by each user (20 cases each with four variables), which compares with ten classifications (ten cases with only one variable) in the secondary assessment. The higher demand in the number of classifications for the primary assessment could have been overbearing for clinicians not experienced with the classification system, resulting in lower scores.

In summary, BACH is a new classification for bone and joint infection. This study has optimized the variables that are included and has demonstrated the applicability of the classification system by an interuser assessment. The results of this study lead the way for the completion of a prospective assessment of BACH in cases of long bone osteomyelitis to assess the management and prognosis of this complex condition.