Economic evaluation of antimicrobial stewardship in primary care: a systematic review and quality assessment.

BACKGROUND: Primary care accounts for 80%-90% of antimicrobial prescriptions, making this setting an important focus for antimicrobial stewardship (AMS) interventions.
OBJECTIVES: To collate the findings and critically appraise the qualities of economic evaluation studies of AMS or related interventions aimed at reducing inappropriate antimicrobial prescribing in primary care.
METHODS: A systematic review of economic evaluations of interventions aimed at reducing inappropriate antimicrobial prescribing in primary care was performed. Published literature were retrieved through a search of Medline, Embase, EconLit and Web of Science databases for the period 2010 to 2020. The quality of the studies was assessed using the Consensus on Health Economic Criteria checklist and Good Practice Guidelines for Decision-Analytic Modelling in Health Technology Assessment.
RESULTS: Of the 2722 records identified, 12 studies were included in the review (8 trial-based and 4 modelled evaluations). The most common AMS interventions were communication skills training for health professionals and C-reactive protein point-of-care testing (CRP-POCT). Types of economic evaluations included in the review were cost-effectiveness (7 studies), cost-utility (1), cost-benefit (2), cost-effectiveness and cost-utility (1) and cost analysis (1). While six of the studies found AMS interventions to be cost-effective, the other six reported them as not cost-effective or inconclusive. The quality of the studies ranged from good to low.
CONCLUSIONS: There were significant variations in cost-effectiveness of AMS interventions across studies and depending on the inclusion of cost components such as the cost of antimicrobial resistance. However, communication skills training and CRP-POCT were frequently cost-effective or cost-beneficial for reducing inappropriate antimicrobial prescribing.

Introduction

Inappropriate use of antimicrobials is a global healthcare issue driving antimicrobial resistance (AMR), which leads to treatment failure and healthcare cost escalation.[1] Antimicrobial stewardship (AMS) refers to systematic programmes aimed at promoting rational use of antimicrobials, contributing to reduced risk of resistance development, improved patient outcomes and reduction in treatment costs.[2,3] AMR has reached a state of ‘silent pandemic’, mainly driven by inappropriate prescription and use of antimicrobials.[4,5] If current prescribing practices continue, it is expected that, by 2050, 10 million lives will be lost globally each year due to AMR and economic output will be reduced by US$100 trillion.[6] According to the Global Research on Antimicrobial Resistance estimate, there were 4.95 million deaths associated with bacterial resistance in 2019.[7] Therefore, there is an urgent need to identify and implement strategies to reduce inappropriate antimicrobial prescribing.

Primary care has been recognized as a crucial setting where AMS interventions can have a major impact due to the volume of prescriptions and associated AMR burden contributed by this setting.[8,9] As a result, both the WHO and several countries including Australia have developed strategies to tackle AMR in primary care.[10,11] However, currently, there is limited evidence on the effectiveness and cost-effectiveness of AMS interventions in reducing inappropriate antimicrobial prescribing in primary care.

Previous studies reported that well-implemented AMS programmes are effective in reducing inappropriate prescribing and use of antimicrobials in a hospital setting.[12-14] For example, an AMS intervention implemented across 47 South African hospitals was associated with a reduction in the mean antibiotic DDDs per 100 patient-days from 101.38 to 83.04. The study concluded that even healthcare facilities with limited infectious diseases expertise can achieve substantial returns through AMS.[14] An AMS programme that aimed to educate physicians in a university hospital in Taiwan, for instance, documented a 13% reduction in inpatient antibiotic consumption.[15] A systematic review and meta-analysis of AMS programmes in Asia also reported reductions in antibiotic usage, healthcare costs and mortality rates associated with the implementation of the programmes.[12]

There were also a few previous reviews that aimed at assessing the clinical and economic value of AMS programmes in hospital settings.[16-19] A review by Nathwani et al.[17] reported that a reduction in length of hospital stay (LOS) and antibiotic expenditure was reported by 85% and 92% of the studies, respectively. While the average cost saving due to the implementation of an AMS programme in the USA was $732 per patient, this cost saving was mainly driven by a reduction in length of hospital stay. Similarly, a systematic review and meta-analysis by Karanika et al.[18] reported that implementation of AMS programmes was associated with reductions of 8.9% in length of hospital stay, 19.1% in antimicrobial consumption and 33.9% in antimicrobial cost. Smith and Coast[20] argue that these cost saving estimates are conservative since they do not account for the cost of future AMR.

Apart from a lack of focus on studies in primary care, there were other limitations in previous reviews.[16-18] First, there has not been a critical appraisal of the models based on the type of economic evaluations that have been performed and robust quality assessment of the studies. Second, assessment of efforts to account for the cost of AMR in the economic evaluations is absent, despite this being an important cost element but usually missed by researchers.[21] Most importantly, there has been no review of economic evaluation studies of AMS programmes at the primary care level despite this setting accounting for 80%–90% of antimicrobial prescriptions.[22]

The objectives of this study were to collate the findings and perform a robust quality assessment of economic evaluation studies and critical appraisal of the models used for AMS interventions in primary care.

Methods

The study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).[23] A protocol for this systematic review was prepared and registered at the International Prospective Register of Systematic Reviews (PROSPERO: CRD42021247776). Figure 1 shows the PRISMA flow chart of the article selection process followed.

Figure 1.

PRISMA flow chart of article selection.

Data sources and search strategy

English language literature on the topic were retrieved through a search of Medline, Embase, EconLit and Web of Science databases for the period January 2010 to October 2021. Database search strategies consisting of concepts including AMS, decision analytic modelling and cost-effectiveness were developed and adapted to each of the databases (see Table S1, available as Supplementary data at JAC Online). A Google scholar search, reference list checking and forward citation checks of included papers were also done to retrieve additional relevant studies that were not captured by the database search.

Search results management and article selection

All database search results were uploaded to Endnote®, duplicate records were removed and the remaining records were exported to Covidence.[24] After further duplicate checking and removal, titles and abstracts of the remaining papers were independently screened by two of the authors (B.L.W. and G.M.), with discrepancies resolved through discussion. The full texts of the papers that passed the title and abstract screening were independently assessed by the two authors (B.L.W. and G.M.) against the eligibility criteria. Discrepancies were resolved through discussion and the opinion of a third author (T.C.). Full or partial economic evaluations of one or more AMS programmes or other interventions aimed at reducing inappropriate antimicrobial prescribing in primary care were included. Systematic reviews and meta-analyses, editorials and commentaries were excluded. There were no restrictions with respect to type of interventions, type of infection, study population or comparator. Conference abstracts for which full-text articles were not available through online searching or contacting authors were also excluded.

Data extraction

Two authors (B.L.W. and N.G.-F.) independently extracted data from the articles included in the review using a pre-agreed data extraction template. Discrepancies in data extraction between the two authors were resolved through discussion and the opinion of a third author (G.M.). Extracted data included: (i) study characteristics such as setting, aim, intervention and population details; and (ii) summary of the economic evaluation methods and reported results including comparator, analysis type, perspective, time horizon, included costs and main findings.

Quality assessment and critical appraisal

The Consensus on Health Economic Criteria (CHEC) list by Evers et al.[25] was used to assess the quality of both trial-based and modelled economic evaluations. The CHEC list consists of 19 items each addressing various economic evaluation subjects including economic study design, time horizon, cost and outcome valuation. Each of the 19 items is marked as yes, no, partially and not applicable.[25] The percentage of fulfilled items by each study and the percentage of studies complying with each of the checklist items were calculated. Overall study quality was regarded as ‘excellent’, ‘good’, ‘moderate’ and ‘low’, which were defined as 100%, >75% to <100%, >50% to ≤75% and ≤50% of the checklist items fulfilled, respectively.[26,27]

Critical appraisal of the decision analytic models used in the model-based economic evaluations was performed using the Philips et al.[28] Good Practice Guidelines for Decision-Analytic Modelling in Health Technology Assessment, hereafter called the Philips et al.[28] checklist. The checklist has 58 items designed to assess three main aspects of decision analytic models: structure (23 items), data (30 items) and consistency (5 items) (see Table 4 and Table S2).

Table 4.

Quality assessment of model-based economic evaluations using the Philips et al.[28] checklist

Dimensions of quality		Question items[a]	Gong et al.,[35] 2019	Hunter,[31] 2015	Holmes et al.,[30] 2018	Lubell et al.,[36] 2018
Model structure
S1	statement of decision problem/objective	S11	yes	no	yes	yes
		S12	unsure	no	yes	yes
		S13	no	yes	no	no
S2	statement of scope and perspective	S21	yes	yes	yes	yes
		S22	no	yes	unsure	N/A
		S23	no	no	yes	partially
		S24	unsure	unsure	yes	no
S3	rationale for structure	S31	yes	unsure	no	N/A
		S32	yes	yes	yes	N/A
		S33	no	no	no	N/A
		S34	no	yes	no	N/A
		S35	no	yes	unsure	N/A
S4	structural assumptions	S41	yes	unsure	unsure	N/A
		S42	yes	unsure	unsure	N/A
S5	strategies/comparators	S51	yes	yes	yes	yes
		S52	no	unsure	no	yes
		S53	N/A	unsure	yes	no
S6	model type	S61	yes	unsure	unsure	N/A
S7	time horizon	S71	unsure	yes	no	unsure
		S72	no	yes	no	unsure
		S73	yes	yes	no	N/A
S8	disease states/pathways	S81	yes	unsure	unsure	N/A
S9	cycle length	S91	no	partially	yes	N/A
Data
D1	data identification	D11	unsure	no	yes	unsure
		D12	unsure	no	N/A	no
		D13	unsure	partially	yes	N/A
		D14	unsure	no	yes	no
		D15	no	no	unsure	unsure
		D16	N/A	N/A	N/A	N/A
D2	pre-model data analysis	D21	N/A	no	No	N/A
D2a	pre-model: baseline data	D2a1	unsure	yes	unsure	Yes
		D2a2	yes	yes	yes	N/A
		D2a3	no	no	no	N/A
		D2a4	no	no	no	N/A
D2b	pre-model: treatment effects	D2b1	unsure	N/A	N/A	N/A
		D2b2	no	no	unsure	yes
		D2b3	no	partially	no	unsure
D2c	pre-model: quality of life weights (utility)	D2c1	no	yes	yes	N/A
		D2c2	yes	yes	yes	N/A
		D2c3	yes	no	unsure	N/A
D3	data incorporation	D31	no	partially	yes	yes
		D32	unsure	unsure	unsure	unsure
		D33	unsure	yes	yes	unsure
		D34	N/A	no	no	N/A
		D35	N/A	unsure	unsure	N/A
D4	assessment of uncertainty	D41	no	partially	unsure	unsure
		D42	no	no	unsure	no
D4a	uncertainty: methodological	D4a1	unsure	no	unsure	no
D4b	uncertainty: structural	D4b1	no	no	no	N/A
D4c	uncertainty: heterogeneity	D4c1	unsure	yes	unsure	N/A
D4d	uncertainty: parameter	D4d1	N/A	no	unsure	yes
		D4d2	no	yes	yes	N/A
		D4d3	no	no	no	unsure
Consistency
C1	internal consistency	C11	no	no	no	unsure
C2	external consistency	C21	yes	yes	yes	yes
		C22	unsure	unsure	unsure	unsure
		C23	unsure	no	no	N/A
		C24	yes	no	no	N/A

N/A, not applicable.

Descriptions of the question items are presented in Table S2.

Results

A total of 2722 studies were identified through the database and other searches, 777 of which were removed as duplicates. After duplicates were removed, the titles and abstracts of 1945 papers were screened for eligibility. The full texts of 140 papers were assessed against the eligibility criteria, with 12 papers included in the review (Figure 1).

Study characteristics

Eight of the 12 included studies were trial-based evaluations, while 4 were model-based evaluations. The studies were conducted in the UK,[29-31] the Netherlands,[32,33] Canada,[22] China,[34] the USA,[35] Vietnam[36] and in multiple countries.[37-39] Table 1 presents a summary of the AMS studies included in the review.

Table 1.

Summary of the AMS studies included in the review

Author, year	Country	Intervention type or components	Intervention period	Study design	Sample size and population
Cals et al.,[32] 2011	the Netherlands	GP use of CRP-POCT GP communication skills training	2 years	cluster randomized trial	431 adults (≥18 years) with LRTI
Dekker et al.,[33] 2019	the Netherlands	online training GPs on: prudent antibiotic use child-specific information communication skills for parents: information booklet on RTI and advice on antibiotic use	winter seasons of 2013–14 and 2014–15	trial based	153 children in the intervention group and 107 children in the control group
Zhang et al.,[34] 2018	China	for prescribers: clinical guidelines on URTI management and training on using guidelines and peer review meetings for patients and caregivers: videos with messages on appropriate use of antimicrobials	6 months	cluster RCT	25 hospitals, 12 interventions and 13 controls (4800 prescriptions of children aged 2–14 years)
Gong et al.,[35] 2019	USA	education on appropriate ARTI treatment computerized clinical decision support to suggest non-antibiotic treatment choices requiring free-text justification into patient’s health record when prescribing antibiotics sending periodic e-mails to prescribers about their rate of inappropriate antibiotic prescribing relative to peers	18 months	modelling	45-year-old adults with signs and symptoms of ARTI presenting to a healthcare provider
Holmes et al.,[30] 2018	UK	pragmatic use of testing, which is reflective of routine clinical practice testing according to clinical guidelines	3 months	modelling	71 adults presenting with ARTI symptoms
Hunter,[31] 2015	UK	GP plus CRP-POCT practice nurse plus CRP-POCT GP plus CRP-POCT and communication training	N/A	modelling	cohorts of 100 hypothetical 50-year-old patients with RTI symptoms
Mamun et al.,[22] 2019	Canada	guidelines and continuing health education for prescribers direct outreach through schools, day cares and community care facilities public campaigns ranging from transit ads to social media	overall intervention period is 2005 to 2014 but varies for the different components	multimodal interventional study	general population
Oppong et al.,[37] 2013	Sweden and Norway	CRP-POCT	28 days of patient follow-up	observational study	370 patients (≥18 years) with a presentation suggesting LRTI
Oppong et al.,[38] 2018	Belgium, the Netherlands, Poland, Spain and UK	training GPs in the use of CRP testing and/or communication skills	4 weeks	multinational cluster RCT based	patients who presented with RTIs
Lubell et al.,[36] 2018	Vietnam	CRP-POCT	not stated	RCT and modelling	acute respiratory infection
Butler et al.,[39] 2018	England, the Netherlands, Spain and Wales	CRP-POCT	July 2013 and August 2014	RCT	614 female adults (≥18 years) with uncomplicated UTI
Ward,[29] 2018	UK	CRP-POCT	6 months	trial based	141 patients with viral or self-limiting LRTI

ARTI, acute RTI; LRTI, lower RTI; N/A, not applicable; RCT, randomized controlled trial; URTI, upper RTI; UTI, urinary tract infection.

Interventions and comparators

The most common AMS interventions were communication skills training for health professionals and C-reactive protein point-of-care testing (CRP-POCT). Some studies also included interventions such as educating health professionals and/or patients[35] as well as clinical guideline development and promotion.[34] Behavioural interventions consisting of suggested alternatives, accountable justification and peer comparison were also implemented in the study by Gong et al.[35] The implementation period of the AMS interventions in the studies ranged from 28 days to 2 years. While usual/standard/routine care was used as the comparator in 11 of the studies, the comparator in Mamun et al.[22] was the pre-intervention part of the time series data.

Aims of the interventions

Reducing inappropriate prescribing for adults and/or children with respiratory tract infection (RTI) was the main aim of the great majority of the interventions in the reviewed studies. The interventions in 10 of the 12 studies focused on inappropriate prescribing for upper/lower/acute RTI. While Butler et al.[39] targeted urinary tract infection, the set of interventions by Mamun et al.[22] had no focus on any specific condition.

Study participants and sample size

The study participants in six of the studies[30-32,35,37,39] were adults, while two studies included children[33,34] and other studies included both children and adults or the general population.[22,29,36,38] The sample size of the studies ranged from 71 patients[30] to 614 patients.[39]

Types of analyses

The types of analyses conducted in the studies included in the review were cost-effectiveness analysis,[30-34,37,39] cost-utility analysis,[35] cost-benefit analysis,[22,36] cost-effectiveness and cost-utility analyses[38] and costing analysis[29] (Table 2).

Table 2.

Summary of the economic evaluation methods employed and reported results

Author, year	Control/comparator	Analysis/model type	Perspective	Time horizon	Included cost components	Any consideration for AMR cost	Discounting	Findings
Cals et al.,[32] 2011	usual care	CEA	healthcare payer	28 days	days off work, medication and other medical costs, GP’s communication skills training costs	possible long-term effects on AMR were regarded as intangible costs and, therefore, not included in the analysis	neither costs nor effects were discounted	GP communication skills training and CRP-POCT are cost-effective both individually and in combination compared with usual care at no WTP or WTP of as low as $121.70 per 1% reduction in antibiotic prescribing for LRTI; GP communication skills training is the most cost-effective of the three interventions
Dekker et al.,[33] 2019	usual care	CEA	societal	2 weeks	costs of non-prescription medications, additional childcare and parents’ loss of work productivity and transportation costs for up to 2 weeks following the index consultation, GP’s time spent in following online training and annualized intervention development costs	the authors discussed that cost of AMR was not considered due to uncertainty in the available data and this may have underestimated the real cost savings of the interventions	N/A	the mean antibiotic prescription rate was 12% lower in the intervention group; and costs were €10.27 per child higher in the intervention group resulting in an ICER of €0.85 per percentage reduction in antibiotic prescribing, which is equivalent to €0.32 per prevented antibiotic course
Zhang et al.,[34] 2018	usual care	CEA	healthcare provider	6 months (time horizon of the trial)	direct costs: costs of consultation (time cost of doctor), prescription monitoring process and peer-review meetings (time cost of participants) andmedication costs	the time horizon of the model, which didn't allow capture of long-term effects such as increased AMR, is mentioned as a limitation	as the time horizon of the trial was <12 months, no discounting was applied	APR in the intervention group reduced by 29.23% at an additional cost of $1.02 per patient compared with the usual care group, producing an ICER of $0.03 per percentage point reduction in antibiotic prescribing
Gong et al.,[35] 2019	both no intervention and provider education on guidelines for appropriate treatment of ARTIs	CUA using Markov model	US societal	30 years	costs of intervention implementation, outpatient visits, hospitalization and treatment of complications	the analysis included model parameters such as rates of baseline resistance, conversion of susceptible to resistant strains and costs of resistant infections	3% discounting rate applied to all costs and QALYs	all the three interventions (suggested alternatives, accountable justification and peer comparison groups) had lower costs but higher QALYs compared with provider education
Holmes et al.,[30] 2018	standard care (no CRP-POCT)	decision analytic model-based CEA	UK NHS	28 days	GP and independent nurse prescriber consultations, CRP-POCT, antibiotic prescription and treatment of adverse drug reactions; prescription medication costs and dispensing fees; 2016–17 UK£	a scenario analysis was conducted where the impact of inappropriate prescribing on antibiotic resistance was assessed based on costs extracted from the literature	discounting was not done due to the short time horizon of the model	in patients with ARTI and based on routine practice, the ICERs of CRP-POCT were £19 705/QALY and £16.07 per antibiotic prescription avoided; following clinical guidelines, CRP-POCT in patients with LRTIs costs £4390/QALY and £9.31 per antibiotic prescription avoided; at a WTP of £20 000/QALY, the probabilities of CRP-POCT being cost-effective were 0.49 (ARTI) and 0.84 (LRTI); CRP-POCT as implemented in routine practice is appreciably less cost-effective than when adhering to clinical guidelines
Hunter,[31] 2015	current standard GP practice (no CRP test)	decisiontree and Markov model	health service (NHS England)	3 years	incremental costs of CRP test, the costs associated with managing an RTI and GP training costs; 2012/2013 UK£	no	3.5% discount rate was applied to future costs and effects	GP plus CRP-POCT and practice nurse plus CRP-POCT have a higher NMB than current practice; although providing communication training in addition to the GP CRP-POCT results in reduced risk of infection and antibiotic prescribing, the benefits were outweighed by the additional cost of training
Mamun et al.,[22] 2019	the pre-intervention segment of the time series data	CBA using interrupted time series analysis	patients and theirthird party insurers	19 years	cost of antibiotics	no	adjusting for unit drug price took care of both inflation and changes in real prices overtime	the intervention was associated with a reduction in average monthly prescription rate of 14.5% and 31% (CAD2404.90) in the monthly total cost of antibiotics; the programme has been effective in cost-benefit terms and, therefore, should be considered for universal adoption in Canadian healthcare systems; in 2014, CAD1 spent on the DBND programme was associated with conservative savings of CAD76.20
Oppong et al.,[37] 2013	usual care	hierarchical regression	health service	28 days	healthcare resource use includes primary care clinic visits, nurse visits, hospital admissions and drug prescriptions	no	not stated	CRP-POCT costs €112.70 per patient prescription avoided or €9391/QALY; CRP-POCT is likely to provide a cost-effective diagnostic intervention both in terms of reducing antibiotic prescribing and QALYs
Oppong et al.,[38] 2018	usual care	CUA and CEA	health service	28 days	costs of consultations with health professionals, use of medications, medical investigations and hospital admissions	yes, the cost of resistance obtained from another study was added to every antibiotic prescription	not stated	training in communication skills is the most cost-effective option; however, excluding the cost of AMR resulted in usual care being the most cost-effective option
Lubell et al.,[36] 2018	routine care	CBA	societal	14 days	cost of CRP readers and reagents and cost of AMR	yes, the cost of AMR per antibiotic prescribed was included	discounting was not applied as all costs were assumed to be incurred at the time patients presented at the facility	use of CRP-POCT in the context of primary care in low- and middle-income countries is likely to incur a modest incremental cost but this can be offset by the economic costs of AMR averted, provided adherence to their results is high
Butler et al.,[39] 2018	standard care	CEA	not stated	14 days	no information was provided on included cost components	no	not stated	there was no statistically significantdifference between study arms in antibiotic use; in addition, there was no evidence of any differences in recovery, patient enablement, UTI recurrence, re-consultation, antibiotic resistance and hospitalizations at follow-up; CRP-POCT was not cost-effective
Ward,[29] 2018	usual care	costing		6 months	costs of cartridge for CRP-POCT and additional health professional consultation time due to the introduction of the test	no	not stated	CRP-POCT has the potential to facilitate AMS in primary care; however, care needs to be taken to ensure it is used in a cost-effective and evidence-based manner

APR, antimicrobial prescription rate; ARTI, acute RTI; CBA, cost-benefit analysis; CEA, cost-effectiveness analysis; CUA, cost-utility analysis; LRTI, lower RTI; N/A, not applicable; NMB, net monetary benefit; UTI, urinary tract infection.

Perspective

Health service/payer including the UK NHS was the most popular perspective taken by five of the studies.[31,32,34,37,38] Healthcare provider and societal perspectives were employed by two[22,34] and three[33,35,36] studies, respectively.

AMR cost consideration

Three studies[30,36,38] accounted for the cost of AMR either as part of the main analysis or in a scenario analysis. Three other studies reported that the cost of AMR was not accounted for due to being considered intangible,[32] the short time horizon of the analysis[34] or the high uncertainty of its estimation.[33] In Lubell et al.[36] and Oppong et al.[38] the future cost of AMR was found to be an important factor affecting the cost-effectiveness of the intervention. Four studies[22,29,31,39] did not report the cost of AMR.

Time horizon and discounting

Most of the studies employed short time horizons ranging from 2 weeks to 6 months, considering the short clinical prognosis of the infections. All, except two of the modelled evaluations,[31,35] did not apply discounting to future costs and outcomes. The study by Gong et al.[35] employed a 3% discount rate for costs and effects and Hunter[31] used 3.5%. While some studies justified the absence of discounting in terms of the short time horizon of the analyses, others did not mention this metric.

Economic outcome measures

The most popular economic outcome measure was cost per unit reduction in antibiotic prescription adopted by six (50%) of the studies. Four of the studies[30,31,37,38] also calculated cost per quality-adjusted life year (QALY) in addition to cost per unit reduction in an antibiotic prescription. The net monetary benefit of the interventions was also calculated in the studies by Hunter[31] and Lubell et al.[36] Mean total cost per unit increase in concordant antibiotic prescribing was employed by Butler et al.[39]

Cost-effectiveness

Five of the 12 studies[22,32,35-37] concluded that CRP-POCT and communication skills training for health professionals were cost-effective/beneficial interventions in reducing inappropriate antimicrobial prescribing. Cals et al.[32] found that communication skills training for GPs and CRP-POCT were cost-effective both individually and in combination compared with usual care at willingness to pay (WTP) of as low as $0 per a 1% reduction in antibiotic prescribing for lower RTI. GP communication skills training was the most cost-effective of the three interventions. Oppong et al.[37] also reported that CRP-POCT was a cost-effective diagnostic intervention both in terms of reducing antibiotic prescribing and QALYs gained costing €112.70 per patient prescription avoided or €9391 per QALY gain. Similarly, GP training in communication skills was the most cost-effective intervention to reduce antibiotic prescribing for RTIs in the Oppong et al.[38] study. The study in Vietnam concluded that, provided adherence to test results is high, POCT can be a valuable intervention even in low- and middle-income countries since its incremental costs can be offset by the economic burden of AMR that it can avert.[36] The cost-benefit analysis of the Do Bugs Need Drugs (DBND) multimodal community-based intervention in Canada showed that $1 spent on the programme was associated with conservative savings of CAD76.20.[22] All of the three components of the behavioural interventions (suggested alternatives, accountable justification and peer comparison groups) of Gong et al.[35] also had lower costs and higher QALYs compared with provider education.

Two other studies[33,34] calculated incremental costs and outcomes or incremental cost-effectiveness ratios (ICERs) without commenting on the cost-effectiveness of the interventions. Dekker et al.[33] estimated an ICER of €0.85 per percentage reduction in antibiotic prescribing, which was equivalent to €0.32 per prevented antibiotic course for interventions involving CRP-POCT and GP communication skills training in the Netherlands primary care. An ICER of US$0.03 per percentage point reduction in antibiotic prescribing was calculated for a multifaceted intervention focusing on prescribers and children and their parents in China.[34]

Four studies reported that CRP-POCT[30,31,36,39] and communication skills training[31] were not cost-effective in reducing inappropriate antimicrobial prescribing. The main reason for lack of cost-effectiveness was non-adherence to test results or not delaying prescribing until test results were known. The modelling studies for the UK primary care also reported that POCT was less cost-effective compared with adhering to clinical guidelines[30] and the benefits of communication skills training were outweighed by the additional cost of training.[31]

Quality assessment

Based on the CHEC list, none of the studies reached an ‘excellent’ quality mark. Five of them were graded as ‘good’ quality;[30-32,34,35] another five were graded as ‘moderate’ quality[22,33,36,37,39] and one was graded as ‘low’ quality.[29] The items that were fulfilled by the least number of the studies were related to the discussion of variables’ distributional issues, discounting, outcome evaluation and relevant cost item identification. Scoring of the CHEC list is provided in Table 3.

Table 3.

Quality assessment of the studies according to the CHEC list[25]

Quality assessment item	Cals et al.,[32] 2011	Dekker et al.,[33] 2019	Zhang et al.,[34] 2018	Mamun et al.,[22] 2019	Oppong et al.,[37] 2013	Oppong et al.,[38] 2018	Butler et al.,[39] 2018	Ward,[29] 2018	Gong et al.,[35] 2019	Hunter,[31] 2015	Holmes et al.,[30] 2018	Lubell et al.,[36] 2018	Studies fulfilling criterion (%)
Clearly described population	yes	yes	yes	yes	yes	yes	yes	no	yes	yes	yes	yes	91.67
Clearly described competing alternatives	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	100.00
Well-defined research question	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	100.00
Appropriate economic design	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	100.00
Appropriate time horizon	yes	yes	yes	yes	yes	no	yes	no	yes	yes	yes	no	75.00
Appropriate perspective	yes	yes	yes	yes	yes	yes	no	no	no	yes	no	yes	66.67
Important and relevant costs identified	no	yes	yes	yes	no	no	no	no	yes	no	yes	yes	50.00
Costs measured appropriately	yes	yes	yes	yes	yes	yes	no	no	yes	no	yes	yes	75.00
Costs valued appropriately	yes	yes	yes	yes	yes	yes	no	no	yes	no	yes	no	66.67
Important and relevant outcomes identified	yes	yes	yes	yes	yes	yes	no	yes	yes	yes	yes	yes	91.67
Outcomes measured appropriately	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	100.00
Outcomes valued appropriately	no	no	no	no	no	yes	no	no	yes	yes	yes	no	33.33
Incremental analysis performed	yes	yes	yes	no	no	yes	yes	no	yes	yes	yes	yes	75.00
Costs and outcomes discounted appropriately	no	no	no	no	no	no	no	no	yes	yes	yes	no	25.00
Appropriate sensitivity analysis	yes	yes	yes	no	no	yes	no	no	yes	yes	yes	yes	66.67
Conclusions follow the data	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	yes	100.00
Study discusses the generalizability	yes	no	yes	no	yes	yes	no	yes	no	yes	yes	yes	66.67
Article indicates no potential conflict of interest	yes	no	no	yes	yes	no	yes	yes	yes	yes	yes	yes	75.00
Ethical and distributional issues discussed appropriately	partially	partially	partially	partially	partially	partially	yes	partially	no	partially	no	no	8.33
Items fulfilled (%) and overall study quality	78.95 (good)	73.68 (moderate)	78.95 (good)	68.42 (moderate)	68.42 (moderate)	73.68 (low)	52.63 (moderate)	42.11 (low)	84.21 (good)	78.95 (good)	89.47 (good)	73.68 (moderate)

Critical appraisal of the models

Overall, the four model-based studies had a ‘yes’ scoring for 13, 18, 19 and 10 of the 58 items on the Philips et al.[28] checklist. None of the studies had a ‘yes’ scoring for more than a third (20/58) of the items on the checklist. The items assessing the data aspect of the models were the least fulfilled ones where none of the studies got a ‘yes’ for 16 of the 30 items focusing on this dimension. None of the studies provided information on any consideration of competing theories on the model structure. Transparent and appropriate data identification methods were presented only in one[30] of the four modelling studies. None of the studies provided information related to justification of data sources, data quality assessment, pre-modelling data analysis methods, half cycle correction, treatment effects synthesis techniques, assumptions regarding the continuing effect of treatment after treatment completion, appropriateness of assumptions and choices, description and justification of distributions chosen for each parameter, and uncertainty assessment. Concerning consistency, no study provided information on whether the mathematical logic of the model had been tested before use, explanation of counter-intuitive results and model calibration against independent data. Details of critical appraisal of the modelling studies using the Good Practice Guidelines for Decision-Analytic Modelling in Health Technology Assessment are presented in Table 4.

Discussion

This review aimed to determine and critically appraise the findings of economic evaluations of AMS interventions in primary care and assess the qualities of the studies and accompanying decision analytic models. Overall, there has been very limited research on the subject – only 12 eligible papers were found despite our search having no restriction on study country, disease condition or type of intervention. The quality of reporting and/or conduct of the studies was low; none of the studies reached an ‘excellent’ quality mark when assessed against the CHEC list. Similarly, critical appraisal of the decision analytic models used in the modelled evaluations using the Philips et al.[28] checklist demonstrated low-quality scores for the structure, data and consistency dimensions of the models.

Common AMS interventions and their cost-effectiveness

CRP-POCT and communication skills training for health professionals were the most common AMS interventions in primary care. While six studies found these interventions to be cost-effective in reducing inappropriate prescribing, the other six studies reported them as not cost-effective or did not reach a conclusion regarding cost-effectiveness. However, the variability in the costs and outcomes collected, the time horizon of the studies and the low quality of reporting means it is difficult to assess the cost-effectiveness of these interventions. The lack of adequate research and inconclusive cost-effectiveness findings warrant further research.

The reported cost-effectiveness of AMS interventions in reducing inappropriate antimicrobial prescribing in primary care was mixed, warranting further research on the subject. While six of the studies found communication skills training for health professionals and CRP testing cost-effective, four other studies concluded that these interventions were not cost-effective. As discussed in a previous study,[40] for an intervention to be considered cost-effective, findings from different modelling approaches and analyses should corroborate each other. Therefore, there is still a need to investigate the cost-effectiveness of these interventions. On the other hand, there are studies that calculated incremental costs and outcomes without coming to a conclusion on cost-effectiveness.[33,34] This could be partly due to difficulty in interpreting the findings because of the lack of a standard, generally acceptable threshold of WTP for some outcomes that are more meaningful for AMS interventions such as cost per unit reduction in inappropriate prescribing and cost per course of antimicrobial treatment avoided.

Costs included in the economic evaluations

Frequently included cost components were costs of CRP testing, medications and other health service costs such as GP consultation mostly from the healthcare service/payer perspective. There is a strong argument for accounting for the future cost of AMR in the economic evaluation of AMS interventions.[20] Although there is a huge uncertainty in estimating the future cost of AMR, Shrestha et al.[41] conducted a modelling study, which can be used to estimate the AMR cost burden that is expected from each prescription for an antimicrobial agent. Despite this trend of paying attention to AMR costs, this review found that only two studies accounted for the future cost of AMR.[30,36] One of these two studies also assessed and found that the cost-effectiveness of the interventions was dependent on this cost component.[36] Therefore, we believe it is important to both further advance the methods that enable this cost component with enhanced certainty and account for it in future economic evaluations. At a minimum, if a study cannot account for the future cost of AMR, it needs to acknowledge that the benefits of an AMS intervention evaluated under such a situation represent a conservative estimate and the likely benefit is higher. AMS interventions are usually paid for by health systems or specific health facilities and, therefore, the use of the health service/payer perspective by the majority of the studies is appropriate. However, the impact of AMR is not limited to the individual receiving antibiotic therapy and, therefore, inclusion of a societal perspective, at least as a sensitivity analysis, is pertinent.

Both quality assessment of the studies and critical appraisal of the decision analytic models indicated inadequate quality levels. For instance, quality assessment of the included studies against the CHEC list showed that none of the studies reached an ‘excellent’ quality mark and only five studies were graded as ‘good’ quality (i.e. fulfilled only >75% to <100% of the 19 items). Only one study (8.33%) fully fulfilled the item on appropriate discussion of ethical and distributional issues. Similarly the four decision analytic models fulfilled only 13 (22.4%),[35] 18 (31%),[31] 19 (32.8%)[30] and 10 (17.2%)[36] of the 58 items on the Philips et al.[28] checklist. We appreciate the findings of the quality assessment are dependent on adequate reporting. However, detailed and transparent presentation of the methods used in economic evaluations in line with relevant guidelines is vital for confidence in policy implementation.

Strengths and limitations

This review contributes to the understanding of how AMS economic evaluations should be designed and implemented in primary care. The quality of the included studies and the decision analytic models used within these studies were assessed using established quality checklists.[25,28] One important limitation is that the studies included in this review are heterogeneous because of differences in the study setting, ASP interventions, and effectiveness and cost-effectiveness measures. Therefore, it is difficult to determine which AMS intervention is the most cost-effective.

Conclusions

The review found that CRP-POCT and communication skills training were the most popular AMS interventions in primary care for which economic evaluations have been conducted. While the quality of the studies was low the findings on the cost-effectiveness of these interventions was mixed. The findings of this review warrant a need for further research of improved quality to provide evidence on the value for money of AMS interventions in primary care.

Click here for additional data file.