Development of quality indicators for the management of Staphylococcus aureus bacteraemia.

BACKGROUND: Staphylococcus aureus bacteraemia (SAB) is a serious and often fatal infectious disease. The quality of management of SAB is modifiable and can thus affect the outcome. Quality indicators (QIs) can be used to measure the quality of care of the various aspects of SAB management in hospitals, enabling professionals to identify targets for improvement and stimulating them to take action.
OBJECTIVES: To develop QIs for the management of hospitalized patients with SAB.
METHODS: A RAND-modified Delphi procedure was used to develop a set of QIs for the management of SAB in hospitalized patients. First, available QIs for the management of SAB were extracted from the literature published since 1 January 2000 (MEDLINE and Embase databases). Thereafter, an international multidisciplinary expert panel appraised these QIs during two questionnaire rounds with an intervening face-to-face meeting.
RESULTS: The literature search resulted in a list of 39 potential QIs. After appraisal by 30 medical specialists, 25 QIs describing recommended care at patient level were selected. These QIs defined appropriate follow-up blood cultures (n=2), echocardiography (n=6), source control (n=4), antibiotic therapy (n=7), antibiotic dose adjustment (n=2), intravenous-to-oral switch (n=2), infectious disease consultation (n=1) and medical discharge report (n=1).
CONCLUSIONS: A set of 25 QIs for the management of SAB for hospitalized patients was developed by using a RAND-modified Delphi procedure among international experts. These QIs can measure the quality of various aspects of SAB management. This information can be fed back to the relevant stakeholders in order to identify improvement targets and optimize care.

Introduction

Staphylococcus aureus bacteraemia (SAB) is a serious and often fatal infection and one of the leading causes of community-acquired and healthcare-associated bacteraemia. In the USA the incidence of community-acquired SAB is 45.7 per 100000 persons per year. SAB often has a complicated course with metastatic infections such as endocarditis, vertebral osteomyelitis or infection of prosthetic material and has a 30 day mortality of around 20% and a 90 day mortality of 30%.

Several factors determine the outcome in patients with SAB. Besides patient characteristics (such as comorbidity), the setting in which the infection is acquired, the site of infection,, strain virulence factors and biofilm formation by S. aureus, and methicillin resistance are associated with mortality. Many of these factors are intrinsically unmodifiable. A modifiable factor that determines the outcome in patients with SAB is the quality of SAB management. One prospective observational study showed significant differences in crude mortality following SAB infection between hospitals that could only partly be explained by differences in patient characteristics and pathogen factors, suggesting that differences in the quality of the care provided at hospital sites play a role. A recent study indeed showed that hospital site was a stronger predictor of the use of echocardiography in SAB cohorts than endocarditis risk factors.

Quality indicators (QIs) are ‘measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality, and hence change in the quality, of care provided’. As such, QIs can be used to measure the quality of various aspects of SAB management in hospitals. This will enable professionals involved in the management of SAB patients to determine which aspects of their management offer room for improvement. In addition, the provision of comparative QI scores to professionals and/or hospitals (i.e. audit and feedback) may stimulate them to take action and improve the quality of SAB care. The aim of this study was to develop QIs for the management of hospitalized patients with SAB.

Materials and methods

Study design

A RAND-modified Delphi procedure was used to develop a set of QIs to define and measure the quality of care provided to patients with SAB. Possible QIs and defining conditions (DCs) were extracted from the literature (Step 1). DCs are conditions that help operationalize QIs; for example, by specifying a recommended time span within which the recommended care should be provided (Table 1). Subsequently, the QIs and associated DCs were presented to an international and multidisciplinary expert panel. The individual panel members had the task of appraising the set of QIs and DCs during two questionnaire rounds (Steps 2 and 4). Between questionnaire rounds, a face-to-face meeting was organized for discussion among experts (Step 3).

Table 1.

Example of a QI and an accompanying DC

QI	DC
Follow-up blood cultures after initiation of antimicrobial therapy should be done regardless of clinical evolution.	The optimal time to obtain the first follow-up blood culture after initiation of antimicrobial therapy is 48 h.

All the experts consented to participate in the study and were aware that their responses would be used for research purposes. Ethics approval from a medical ethical committee was not required.

Step 1. Literature review

For a systematic review of available literature on potential QIs, the MEDLINE and Embase databases were searched for relevant studies in May 2017. The complete search strings are included in the Supplementary Methods (available as Supplementary data at JAC Online). Studies written in English and published after January 2000 that provided potential QIs for the management of SAB in adult patients were included. QIs encompass ‘outcome indicators’, which specify the ultimate goals of the care given; ‘process indicators’, which refer to the actual care delivered to the patients; and ‘structure indicators’, which refer to the organizational structures of a healthcare system. Studies only involving paediatric patients and case reports were excluded.

Two researchers (J. L. J. and T. W. vdV.) independently screened titles and abstracts to identify potentially eligible studies. In case of discrepancy, a third reviewer (J. tO.) was consulted for final judgement. Articles without an abstract were automatically included for full-text review. All selected full-text articles were screened by one reviewer (J. L. J., medical doctor) for eligibility using the predefined inclusion and exclusion criteria. One reviewer (J. L. J.) extracted potential QIs from the included articles.

Thereafter, four reviewers (J. tO., J. L. J., J. S. and M. E. J. L. H.) grouped recommendations into domains addressing similar aspects of care. Next, in consensus, they merged recommendations and removed duplicates, after which the selected recommendations were translated into potential QIs. This list of QIs was compared with a previously validated set of QIs that measure appropriate antibiotic use in hospitalized adults. QIs from this set of QIs were added to the list if they addressed a topic not yet covered by the QIs extracted from the literature and were relevant to the management of patients with SAB; these QIs were rephrased to make them SAB specific.

Study characteristics and risk of bias

To describe the characteristics of original studies included, we collected the following variables: study period, country, setting, number of patients included, study design, details of the intervention/control condition, and outcome measures. Furthermore, the risk of bias of each included study was assessed using design-specific checklists (e.g. the Cochrane Risk of Bias tool and Newcastle–Ottawa Scale). From the reviews included we extracted the following variables based on the PRISMA checklist: databases searched, time frame of literature search, number of studies included, design of studies included, intervention, and outcome measures. From consensus studies we summarized the following characteristics: type of consensus procedure, objective, year performed, included experts’ country of origin and profession, and sources of recommendations.

Step 2. First questionnaire round

The potentially relevant QIs with accompanying DCs were converted into a questionnaire in a Microsoft© Word file (Table S1) and sent by e-mail to the expert panel on 18 October 2017.

Fifty-one (inter)national medical professionals (MDs) with extensive experience in the management of patients with SAB were invited to join the expert panel. Some experts were approached because of their involvement in internationally acknowledged studies on SAB and others were expert members of guideline committees. Thirty-three (65%) of the 51 experts consented to participate. The panel members were asked to appraise the potential QIs on the basis of their knowledge and experience of the management of patients with SAB. A Likert scale was used, ranging from 1 (very irrelevant) to 9 (very relevant) and including the option ‘cannot assess’, to appraise the relevance of the potential QI for assessing the quality of SAB management. The panel members could provide suggestions for rephrasing QIs/DCs, as well as proposals for additional QIs/DCs.

Results from the first questionnaire were analysed using IBM SPSS Statistics version 24. QIs were accepted if the median score was ≥8 and ≥70% of the scores were in the top tertile (7, 8 or 9). If the median score was <7, the QI was excluded. QIs with a median score of ≥8 but <70% of the scores in the top tertile and QIs with a median score of 7 and ≥70% of the scores in the top tertile (7, 8 or 9) were discussed during the consensus meeting. QIs were—irrespective of the score—also labelled for discussion if more than four panel members made similar comments regarding the phrasing. DCs were accepted if ≥80% of the experts agreed on the suggested condition to help operationalize the QI. DCs that scored lower were discussed during the expert panel meeting.

We performed a subgroup analysis, comparing median relevance scores between non-Dutch and Dutch experts.

Step 3. Expert panel meeting

Prior to the expert panel meeting the panel members received a personalized feedback report providing information on the first-round scores, including their individual scores. All expert panel members who had participated in the first round were invited to attend the meeting, either in person or by telephone. The meeting was chaired by an experienced member of the research team (J. S.) and was held on 4 December 2017. During the face-to-face meeting, the QIs and DCs were ranked for discussion and those newly suggested were discussed and subsequently accepted or excluded.

Step 4. Second-round questionnaire

In Step 4, along with the second questionnaire a detailed report of the consensus meeting was sent to the panel members (19 January 2018). The expert panel members who had participated in the first round were asked to approve the adjusted QIs and DCs. Rephrased QIs and DCs were accepted if at least 70% of the full expert panel agreed (yes/no) on the adjustments made. Newly proposed QIs were appraised on a Likert scale as described above.

Results

The literature searches resulted in 1580 articles, of which 1259 remained after removing duplicates. Finally, 13 articles were included (Figure 1)., From these 13 papers and from a previously validated set of QIs that measure appropriate antibiotic use in hospitalized adults, 39 potential QIs for the management of SAB in hospitalized patients were extracted. Of these, 37 were process indicators and 2 were outcome indicators. No structure indicators were found. These potential 39 QIs and accompanying DCs were converted into a written questionnaire.

Figure 1.

Flow chart of the literature search to identify potential QIs for the recommended care of SAB in hospitalized patients. The asterisk indicates that one of the QIs was already accepted in the first questionnaire round but was rephrased in the face-to-face meeting when a QI addressing a similar aspect of care was excluded.

Description of the studies included and risk of bias of the cohort studies are presented in Tables S2 and S3.

Thirty of the 33 experts returned the questionnaire (8 clinical microbiologists, 17 infectious disease specialists, 2 clinical microbiologist/infectious disease specialists, 1 infection control and prevention specialist, 1 cardiologist and 1 nuclear medicine physician; response rate 91%).

The experts originated from the Netherlands (19), Germany (2), England (1), Denmark (1), Spain (1), Canada (1), the USA (4) and Japan (1). Eight of the Dutch participants were members of the national guideline committee for SAB.

In this step, 18 of the 39 initial QIs were accepted and 7 were excluded (Table S1).

For nine preliminary accepted and three preliminary excluded QIs, relevant comments on the phrasing were provided (i.e. more than four panel members had made similar comments about the QI or the accompanying DC of a QI); these QIs were therefore labelled ‘for discussion’. One QI was labelled ‘for discussion’ on the basis of the scoring [QI 20 (Table S1), median 8, 69% in highest tertile]. One QI was not appraised by the experts because of a layout problem in the questionnaire and was therefore labelled ‘for discussion’ [QI 19 (Table S1)]. No new QIs were proposed. Where applicable, agreement was sought on DCs to help operationalize QIs.

In a subgroup analysis, scores on the first-round questionnaires were compared between the 11 non-Dutch and 19 Dutch experts. The median relevance scores were identical or within 1 point for 33 of the 39 QIs. For the following QIs differences of >1 point existed between the two groups: transthoracic echocardiography should be performed in patients with SAB (QI 7; non-Dutch median score 8 versus Dutch median score 5); transthoracic echocardiography should always be performed within 14 days after detection of SAB (QI 13; 9 versus 6); 18F-FDG PET/CT scan should be performed in patients with risk factors for complicated SAB (QI 14; 3 versus 5); cardiac devices should be removed when the cardiac devices are suspected to be infected in patients with SAB (QI 18; 9 versus 6); joint prosthesis should undergo debridement when the joint prosthesis is suspected to be infected in patients with SAB (QI 20; 9 versus 7); and in-hospital mortality of SAB could well serve as an easy-to-assess quality-of-care indicator for infectious diseases inpatient care (QI 38; 8 versus 5). Of these six QIs, both the Dutch and non-Dutch experts excluded QI 14. The other QIs, except the outcome QI 38 (rejected), were discussed and excluded during the panel meeting.

Step 3. Face-to-face meeting

Seven Dutch panel members (one clinical microbiologist and six infectious disease specialists) physically attended the consensus meeting (23%), and four international panel members (one clinical microbiologist, one infectious disease specialist, one infectious disease specialist/clinical microbiologist and one internist) attended the meeting by telephone (13%).

During the meeting, the 14 QIs labelled ‘for discussion’ were presented. Discussion of the relevant comments regarding the three initially excluded QIs did not alter this decision. Based on the discussion of relevant comments regarding the 9 preliminary accepted QIs (while taking the 18 accepted QIs into account), 3 QIs were combined into 2 and were rephrased, 2 were removed because of overlap with a third one (which remained unchanged and accepted), 1 was rephrased, and 2 remained unchanged and accepted. The QI labelled ‘for discussion’ on the basis of the scoring [QI 20 (Table S1)] was excluded but resulted in a rephrasing of a previously accepted QI (QI 21). The QI with the layout problem [QI 19 (Table S1)] was rephrased and was to be appraised in the second questionnaire round. One new QI was proposed following this discussion. Thus, following the meeting, in total 20 QIs were accepted and 6 QIs were to be presented to the expert panel in the second questionnaire round [4 rephrased QIs for approval and 2 QIs for appraisal (QI 19 and the newly proposed QI, QI 11b)]. Additionally, in line with the QIs discussed, DCs were discussed, rephrased or added. Most DCs operationalized exact time periods.

Step 4. Second questionnaire round

In Step 4, the expert panel members who had participated in the first round were asked to approve (yes/no) the four rephrased QIs and to appraise two QIs (QI 19 and QI 11b) and accompanying DCs. All 30 panel members returned the second questionnaire. The rephrased QIs and accompanying DCs were approved and definitely accepted. The newly proposed indicator QI 11b was rejected. QI 19 was accepted (median 9, 100% in highest tertile).

This led to a final set of 25 QIs and accompanying DCs for the management of SAB (Table 2). The ultimate QI set reflects the following domains: follow-up blood cultures (n=2), echocardiography (n=6), source control (n=4), antibiotic therapy (n=7), antibiotic dose adjustment (n=2), intravenous-to-oral switch (n=2), infectious disease consultation (n=1) and medical discharge report (n=1).

Table 2.

Final list of 25 QIs and 10 associated DCs for the management of patients with SAB

QI	DC	Reference
Blood cultures
QI 1. Follow-up blood cultures after initiation of antimicrobial therapy should be done regardless of clinical evolution.	The optimal time to obtain the first follow-up blood cultures after initiation of antimicrobial therapy is after 48 h.	17
QI 2. Collection of repeat blood cultures should be performed until first negative blood culture.	The optimal interval to obtain repeat blood cultures is after 48 h.	15 , 16 , 18 , 19 , 21 , 22 , 24 , 25
Echocardiography
QI 3. Transthoracic echocardiography should be performed in patients with predisposing cardiac conditions for endocarditis.a	The optimal time to perform transthoracic echocardiography in patients with SAB and predisposing cardiac conditions for endocarditis is preferably at 3–5 days, but not later than 14 days.	17 , 19 , 24 , 25
QI 4. Transthoracic echocardiography should be performed in patients with risk factors for complicated SAB.a,b	The optimal time to perform transthoracic echocardiography in patients with risk factors for complicated SAB is preferably at 3–5 days, but not later than 14 days.	17 , 19 , 24 , 25
QI 5. Transthoracic echocardiography should be performed in patients with diagnosed complicated SAB.a,c	The optimal time to perform transthoracic echocardiography in patients with diagnosed complicated SAB is preferably as soon as possible, but not later than 72 h after first positive blood culture	17 , 19 , 24 , 25
QI 6. Transoesophageal echocardiography should be performed in patients with SAB and predisposing cardiac conditions for endocarditis.	The optimal time to perform transoesophageal echocardiography in patients with SAB and predisposing cardiac conditions for endocarditis is preferably at 3–5 days, but not later than 14 days.	17 , 19 , 24 , 25
QI 7. Transoesophageal echocardiography should be performed in patients with risk factors for complicated SAB.b	The optimal time to perform transoesophageal echocardiography in patients with risk factors for complicated SAB is preferably at 3–5 days, but not later than 14 days.	17 , 19 , 24 , 25
QI 8. Transoesophageal echocardiography should be performed in patients with diagnosed complicated SAB.c	The optimal time to perform transoesophageal echocardiography in patients with diagnosed complicated SAB is preferably as soon as possible, but not later than 72 h after first positive blood culture.	17 , 19 , 24 , 25
Non-antibiotic therapeutic interventions
QI 9. After detection of SAB a vascular catheter should always be removed.	The optimal time of removal of vascular catheter after detection of SAB is right away and at least within 24 h.	15 , 17–19 , 23–25
QI 10. Cardiovascular implantable electronic devices should be removed when these devices are confirmed to be infected in patients with SAB.		15 , 21 , 23
QI 11. A joint prosthesis should undergo debridement and/or should be surgically removed when the joint prosthesis is confirmed to be infected in patients with SAB.		21 , 23
QI 12. An abscess should be drained in patients with SAB.	The optimal time of drainage of an abscess in patients with SAB is within 24 h.	17 , 19 , 23
Antibiotic treatment
QI 13. Initial antibiotic therapy should be administered intravenously in patients with SAB.		14 , 15 , 17–19 , 23 , 24
QI 14. Initial therapy should be intravenous (flu)cloxacillin (or nafcillin or oxacillin) or cefazolin in the case of methicillin-susceptible strains in patients with SAB.		15–17 , 19 , 22–25
QI 15. Antibiotic therapy should be initiated within 24 h after first positive blood culture.		22 , 23
QI 16. Appropriate treatment should be adapted within the first 24 h after a methicillin susceptibility result is available, if so required.		15 , 17 , 21
QI 17. The dosage of antibiotic treatment should be according to (national) guidelines in patients with SAB.		11
QI 18. Appropriate duration of intravenous antibiotic treatment should be at least 14 days for uncomplicated SAB.		4 , 15 , 17–19 , 21 , 23–25
QI 19. Appropriate duration of intravenous antibiotic treatment should be at least 28 days for SAB complicated by metastatic abscesses or deep foci of infection.d		15 , 17–19 , 23–25
QI 20. Therapeutic drug monitoring should be performed when SAB is treated with vancomycin.		16 , 17 , 19 , 22
QI 21. Antibiotic treatment therapy for patients with SAB should be adjusted according renal function.		11
QI 22. Intravenous-to-oral switch should not be performed in uncomplicated SAB after 48–72 h.		14
QI 23. Intravenous-to-oral switch should not be performed in complicated SAB after 48–72 h.		14
Other management aspects
QI 24. Infectious disease specialist consultation should be performed in patients with SAB.		4 , 15 , 20–22 , 24 , 25
QI 25. SAB should be documented in the medical discharge summary.		18

The performance of a transoesophageal echocardiography as first-line diagnostic modality obviates the need for transthoracic echocardiography.

Patients with one of the following: community acquisition, signs of infection >48 h before initiation of appropriate treatment, fever >72 h after initiation of appropriate treatment, and/or positive blood cultures >48 h after initiation of appropriate treatment.

Uncomplicated bacteraemia: exclusion of endocarditis and other metastatic sites of infection, the absence of implanted prostheses, clearance of bacteraemia within 4 days for patients with repeat blood cultures, and defervescence within 72 h after the initiation of effective therapy. Complicated bacteraemia: cases not meeting the criteria for uncomplicated bacteraemia.

Patients with endocarditis are not included.

Discussion

In this study, we developed a set of QIs and their accompanying DCs for the management of patients hospitalized with SAB using a RAND-modified Delphi procedure. The QIs and DCs describe the domains of microbiological diagnostics, adjunctive imaging and therapy. These QIs make it possible to measure the various steps in the care of patients hospitalized with SAB and to set priorities for targeted improvement interventions. In addition, when used among various professionals or hospitals, comparative QI scores can be fed back to encourage recipients’ action to address discrepancies between desired and actual quality of care.

Antimicrobial stewardship teams are perfectly equipped to measure and improve the quality of antimicrobial use, and are composed of the core disciplines, i.e. infectious disease specialist, clinical microbiologist and hospital pharmacist, involved in the management of patients with SAB. Antimicrobial stewardship teams increasingly ensure the performance of bedside consultations as defined by one of the accepted QIs. It has been demonstrated that it is possible and effective to include optimizing SAB management as a stewardship objective into antimicrobial stewardship programmes,, and bedside consultation by an infectious disease physician has been shown to be associated with decreased mortality in patients with SAB.,,,,, Obviously, close collaboration should exist with other specialists involved in the management of SAB, such as radiologists, cardiologists, cardiothoracic surgeons and endocarditis teams.

The two outcome QIs on attributable mortality were excluded by the panel members, arguing that the often-present and significant comorbidity prevents the exact cause of death from being determined. Nevertheless, as they specify the ultimate goals of the care provided, outcome QIs provide valuable information. Crude outcome QIs are less suitable since these are strongly influenced by patient factors. Primary endpoints for use in clinical trials as proposed in a consensus study by Harris et al. could be the starting point for development of outcome QIs.

This study has several strengths. First, a RAND-modified Delphi procedure was performed, in which scientific evidence is combined with expert opinion.,, Second, an international multidisciplinary expert panel was selected in which all the main specialties in the treatment of SAB were represented. The size of the panel and the diversity of its members have contributed to the reliability and validity of the results of this Delphi procedure, although infectious disease physicians were overrepresented. Third, the involvement of experts from MRSA-endemic regions and the inclusion of detailed QIs on antimicrobial treatment mean that this is a generic set of QIs that is not restricted to the management of patients with MSSA bacteraemia, but is also applicable to those with MRSA bacteraemia.

This study also has limitations. First, only 31% of the panel members attended the consensus meeting. However, a detailed report from the consensus meeting was sent to the all panel members and important changes in QIs made during the meeting were presented for approval or appraisal in the second-round questionnaire. This approach, together with high response rates to both questionnaires and the high approval (100%) in the second round, reassures us that our results are valid. Second, we had a large group of experts with an overrepresentation of Dutch experts. Subgroup analysis, however, only showed significant differences in the scores of the first questionnaire round for six QIs. Five of these came back for discussion to reach consensus in the face-to-face meeting. Imbalance in group composition was not an issue since all experts had to agree on acceptance or rejection of the QIs. Third, the quality of most of the studies from which we derived the QIs was not high. This leaves controversies on some management aspects, i.e. optimal antibiotic dosing, optimal antibiotic treatment of MRSA, and route of administration. This also implies that recent developments that have not yet been evaluated in studies with a robust design or with which most experts do not have a lot of clinical experience are likely to be excluded. In this study, in our opinion, this could be the case with the use of 18F-FDG PET/CT scan. Future studies should shed light on these issues and provide guidance for improvement of the management of SAB., However, if there is only a limited scientific base for recommendations on SAB management, a systematic procedure, e.g. the RAND-modified Delphi procedure, that combines available evidence and expert opinion, is necessary to develop QIs.

In conclusion, this RAND-modified Delphi study resulted in a set of 25 QIs for the management of hospitalized patients with SAB. This is the first important step to be able to evaluate the different processes of SAB management and to provide feedback to the relevant stakeholders in order to optimize care. Before applying these QIs, the clinimetric properties (measurability, applicability, reliability, improvement potential and case-mix stability) need to be assessed. Ultimately, use of these QIs in an infrastructure of collaborating medical specialties may contribute to decreasing the high mortality of this common infection.

Click here for additional data file.