Proxy indicators to estimate the appropriateness of medications prescribed by paediatricians in infectious diseases: a cross-sectional observational study based on reimbursement data.

BACKGROUND: We previously developed proxy indicators (PIs) that can be used to estimate the appropriateness of medications used for infectious diseases (in particular antibiotics) in primary care, based on routine reimbursement data that do not include clinical indications.
OBJECTIVES: To: (i) select the PIs that are relevant for children and estimate current appropriateness of medications used for infectious diseases by French paediatricians and its variability while using these PIs; (ii) assess the clinimetric properties of these PIs using a large regional reimbursement database; and (iii) compare performance scores for each PI between paediatricians and GPs in the paediatric population.
METHODS: For all individuals living in north-eastern France, a cross-sectional observational study was performed analysing National Health Insurance data (available at prescriber and patient levels) regarding antibiotics prescribed by their paediatricians in 2017. We measured performance scores of the PIs, and we tested their clinimetric properties, i.e. measurability, applicability and room for improvement.
RESULTS: We included 116 paediatricians who prescribed a total of 44 146 antibiotic treatments in 2017. For all four selected PIs (seasonal variation of total antibiotic use, amoxicillin/second-line antibiotics ratio, co-prescription of anti-inflammatory drugs and antibiotics), we found large variations between paediatricians. Regarding clinimetric properties, all PIs were measurable and applicable, and showed high improvement potential. Performance scores did not differ between these 116 paediatricians and 3087 GPs.
CONCLUSIONS: This set of four proxy indicators might be used to estimate appropriateness of prescribing in children in an automated way within antibiotic stewardship programmes.

Introduction

Antimicrobial resistance, in particular antibiotic resistance, is a global threat. To address this public health issue, two main strategies must be implemented: infection prevention and control as well as antimicrobial stewardship programmes. Metrics or indicators reflecting the appropriateness of antibiotic prescriptions are needed, initially to assess whether current antibiotic use is appropriate or whether improvement is necessary, and then, if this assessment shows targets for improvement, to optimize antibiotic use relying upon a multifaceted strategy that includes audit and feedback, for which, again, such metrics are crucial.

A recent literature review identified quality indicators that could be used to assess the appropriateness of antibiotics prescribed in the outpatient setting. However, all these quality indicators need data on the clinical indication to be calculated. Since data on clinical indication are rarely available in electronic medical records or in routine reimbursement databases, quality indicators usually rely on manual collection of data, which is time-consuming. Finding a way to collect data reflecting the appropriateness of antibiotic use based on routine automated data collection is urgently needed. Recently published studies have started addressing this need.

The vast majority of antibiotics are prescribed in the outpatient setting, with children accounting for a significant proportion of antibiotic use. Some studies have already suggested easy-to-measure indicators to estimate the appropriateness of antibiotic use in the paediatric population.,, A few of these studies developed ‘proxy’ indicators (PIs), i.e. metrics that can be calculated from data on the quantity of antibiotic use, that do not need clinical indication data and that still indirectly reflect appropriateness of antibiotic use, provided that they are associated with a clear target., As an example, Piovani et al. used the proportion of amoxicillin out of total antibiotic use as a PI, with a target of 50% or more. Quality indicators accurately reflect the appropriateness of each antibiotic prescription. PIs can only strongly suggest that antibiotic use at an aggregated level (not the prescription) is appropriate or not, depending on whether or not the set target is met.

Building on a previous study targeting antibiotics prescribed by GPs, our objectives were to: (i) select the PIs that are relevant for children and estimate current appropriateness of medications used for infectious diseases (antibiotics and anti-inflammatory drugs) by French paediatricians and its variability while using PIs; (ii) assess the clinimetric properties of these PIs using a large regional reimbursement database; and (iii) compare performance scores for each PI between paediatricians and GPs in the paediatric population.

Methods

Study setting and population

This study was conducted in two regions of north-eastern France: the Lorraine region (2 346 000 inhabitants) and the Champagne-Ardenne region (1 339 270 inhabitants); the methodology is similar to the one used in our study targeting GPs. We included paediatricians practising in these two regions in 2017, who took care of at least 100 different patients during the year and wrote at least 10 prescriptions for antibiotics. In France, both GPs and paediatricians can take care of children, depending on the parents’ choice. It is exceptional for primary care paediatricians to have a specialized practice.

Data source and study design

In France, individuals pay health service fees, which are refunded by the National Health Insurance (NHI). Every inhabitant, child or adult, whatever his/her income and professional status, nationality or age, is covered by the NHI programme.

Data regarding the quantity of antibiotics and anti-inflammatory drugs prescribed and dispensed by community pharmacies are available in the NHI databases, as these medications are subject to reimbursement by the French NHI. Each time a prescribed drug is dispensed to a given patient, information on the medication dispensed (including the type and number of dispensed packages), the prescriber (professional identification number) and the patient identification number are recorded and electronically sent to the Regional Health Insurance Fund. Presence of a chronic disease (‘affection de longue durée’) is identified by the NHI since there is exemption from health insurance co-payments when chronic diseases are particularly costly. However, the NHI databases do not provide any information about clinical indications/diagnoses related to the specific prescription or the total duration (in days) of treatment.

We performed a cross-sectional observational study analysing data regarding systemic antibiotics [J01 code according to the Anatomical Therapeutic Chemical (ATC) 2017 classification] and anti-inflammatory drugs (M01A code for systemic non-steroidal anti-inflammatory drugs and H02AB for systemic corticosteroids) prescribed by paediatricians in 2017 for all individuals living in Lorraine and Champagne-Ardenne. Data included each occasion on which these drugs were prescribed by paediatricians and dispensed by community pharmacies during the year. They were collected from the outpatient reimbursement database of the north-eastern France (Grand Est – CNAM) Regional Health Insurance Fund as part of its routine work. This fund covers salaried workers and their family but also other socio-professional groups, such as the unemployed, and accounted for 94.5% of the population in 2017.

PIs to estimate the appropriateness of prescriptions

Starting from a recently published literature review and structured consensus procedure of outpatient quality indicators, we have already identified and validated PIs estimating the appropriateness of antibiotics prescribed by GPs to adults and children. Based on this previous work, we identified, with input from paediatrician colleagues, four PIs that could be operationalized (i.e. translated into numerators/denominators, with targets) while using routine reimbursement data, to estimate the appropriateness of medication prescriptions for infectious diseases in children in France at paediatrician level. Comparing these fractions with the target levels provides an indication of whether antibiotic use is appropriate or not. Since NHI databases do not provide any information on children’s weight, we could not use the ‘estimated duration of treatment’ PI we designed previously.

The four PIs are described in Table 1 with their numerators, denominators and targets. Most (upper and lower) respiratory tract infections occurring during the cold-weather season are viral infections. Prescription of antibiotics should therefore be relatively stable during the year and this stability is assessed by PI 1. PI 2 focuses on preferred prescribing of first-line antibiotics, such as amoxicillin, rather than second-line antibiotics for all patients. The two last PIs (PI 3 and PI 4) are not exclusively focused on antibiotics but on the co-prescription of anti-inflammatory drugs (non-steroidal and steroidal) and antibiotics. Co-prescription is usually not recommended, according to national guidelines. For all PIs, a low value indicates high quality of care, except for PI 2, where a high value indicates high quality of care. The scientific evidence base supporting the design of each PI is detailed in Table S1 (available as Supplementary data at JAC-AMR Online).

Table 1.

List of PIs to estimate the appropriateness of medications prescribed for infectious diseases by paediatricians

PI	Numerator description	Denominator description	Unit	Target value	Target patients
PI 1: Seasonal variation of total antibiotic use (%)	[number of prescriptions of antibiotics (J01) during the cold-weather season (January–March and October–December)/number of prescriptions of antibiotics (J01) during the hot-weather season (April–September) − 1] × 100		percentage of prescriptions per year	<20%	all patients
PI 2: Amoxicillin/second-line antibiotics (ratio)	number of prescriptions of amoxicillin (J01CA04)	number of prescriptions of amoxicillin/clavulanic acid (J01CR02) + quinolones (J01M) + cephalosporins (J01D) + MLSK (J01F)	number of prescriptions per year	>1	all patients
PI 3: Co-prescription antibiotic + systemic NSAIDs (%)	number of antibiotic(s) (J01) + systemic NSAID(s) (M01A) co-prescribed on the same day	total number of antibiotic prescriptions	percentage of prescriptions per year	<5%	all patients
PI 4: Co-prescription antibiotic + systemic corticosteroids (%)	number of antibiotic(s) (J01) + systemic corticosteroid(s) (H02AB) co-prescribed on the same day	total number of antibiotic prescriptions	percentage of prescriptions per year	<5%	all patients

MLSK, macrolides, lincosamides, streptogramins and ketolides; NSAIDs, non-steroidal anti-inflammatory drugs.

Each PI was calculated at paediatrician level to estimate the appropriateness of prescriptions by French paediatricians and its variability. The unit of measure at patient level was the antibiotic treatment, i.e. the antibiotic dispensed on a given day in 2017 by a community pharmacy following a prescription by a paediatrician. If two different antibiotics were prescribed by a paediatrician and dispensed on the same day, they were counted separately. We defined numerators and denominators for all PIs and used thresholds/cut-off values to indicate high quality of care. We ended up with a paediatrician score for each PI, which is an aggregated score (aggregating prescriptions made by a paediatrician to all his/her patients).

Assessment of clinimetric properties

Measurement or clinimetric properties of the PIs affect the interpretation of data and should therefore be tested. To demonstrate the value of our PIs as measurement instruments to estimate the appropriateness of prescriptions in routine practice, we evaluated the following clinimetric properties.,

(i) Measurability

Measurability is defined as the availability of data required to calculate the indicator. An indicator was considered measurable if data necessary to calculate it were available for more than 75% of the cases, i.e. data were missing in <25% of cases.

(ii) Applicability

A PI was considered applicable if it could be calculated from data extracted for more than 75% of paediatricians. In practice, a PI score could not be calculated for a given paediatrician if fewer than 10 prescriptions were identified for the denominator.

(iii) Potential room for improvement

Potential room for improvement measures the sensitivity of a PI in detecting variability in appropriateness of prescriptions between physicians and over time. It is expressed as 100% minus the performance score, with performance expressing the percentage of paediatricians who reached the PI target. High performance scores make indicators less sensitive and therefore less useful in routine practice, so the potential room for improvement for a PI was considered to be low if it was ≤15%.

Ethics approval

As our study was retrospective and did not modify the medical care of patients, and complete anonymity was preserved at both patient and physician levels, no ethics committee approval was required, in accordance with French law.

Statistical analysis

PI results were calculated at the paediatrician level; they are presented as medians and IQRs since data were not normally distributed. The performance scores, i.e. the percentage of paediatricians who reached the target, were also calculated. Measurability, applicability and improvement potential are presented as percentages. The performance score (percentage of physicians who reached the target) calculated for 3087 GPs in our previous study for each of the four PIs considered here was compared with the performance score of paediatricians by using χ2 tests in patients <16 years old. All analyses were performed with SAS Enterprise Guide version 7.1 (SAS Institute Inc., Cary, NC, USA).

Results

Population characteristics

Out of the 126 paediatricians practising in the Lorraine and the Champagne-Ardenne regions in 2017, we included 116 who met the inclusion criteria. Their mean age was 55.6 ± 10.2 years and 46% were men. They took care, on average, of 1251 ± 576 different patients in 2017 (a patient can, in France, visit several paediatricians or GPs), with a total of 3112 ± 1724 consultations. The 116 included paediatricians prescribed in 2017 a total of 44 146 antibiotic treatments that were dispensed to 22 638 different patients. Regarding their patients’ characteristics, 10% were aged ≥16 years and 2% had a chronic disease.

Appropriateness of antibiotic use by French paediatricians

Results concerning the four PIs (Table 2) showed wide variations of prescribing (large IQRs) between paediatricians, particularly for seasonal variation of total antibiotic use. Performances were particularly low for three out of four PIs: seasonal variation of total antibiotic use (PI 1), co-prescription of non-steroidal anti-inflammatory drugs (PI 3) and systemic corticosteroids (PI 4).

Table 2.

Results for the four proxy indicators, calculated at paediatrician level

PI	Target value	Median (IQR)	Percentage of paediatricians who reached the target (performance)
PI 1: Seasonal variation of total antibiotic use (%)	<20%	88.7 (59.2–126.7)	6.9
PI 2: Amoxicillin/second-line antibiotics (ratio)	>1	1.9 (1.0–3.2)	74.1
PI 3: Co-prescription antibiotic + NSAIDs (%)	<5%	9.1 (4.3–16.6)	25.9
PI 4: Co-prescription antibiotic + corticosteroids (%)	<5%	9.4 (4.1–19.2)	29.3

NSAIDs, non-steroidal anti-inflammatory drugs.

Clinimetric properties of the proxy indicators (Table 3)

As data required to calculate the PIs were collected from the outpatient reimbursement database of the Regional Health Insurance Fund and as all medications studied are reimbursed by the French NHI, there were no missing data and all the PIs were measurable in 100% of the cases.

PIs could occasionally not be calculated for some paediatricians when the denominator was <10 prescriptions. However, applicability was always higher than 75%.

As previously mentioned, PI performances were low, and as a consequence potential room for improvement was considerable, varying from 25.9% (PI 2) to 93.1% (PI 1).

Comparison of performance scores between paediatricians and GPs (Table 4)

For patients <16 years old, paediatricians’ and GPs’ performances did not differ for any of the four PIs. However, PI 2 performance was 75.0% for paediatricians compared with 66.9% for GPs, with P = 0.066, suggesting that paediatricians tend to prescribe first-line antibiotics more often than GPs in the paediatric population.

Discussion

Based on reimbursement data only, we could identify four PIs estimating the appropriateness of prescriptions, in particular antibiotics, by French paediatricians. All showed good measurability, applicability and potential room for improvement. Our set of PIs encompasses both unnecessary prescriptions (e.g. overuse during the winter season) and inappropriate prescribing (e.g. co-prescriptions of antibiotics and anti-inflammatory drugs). Performance ranged from 7% to 74% and was particularly low for three out of four indicators, suggesting a large amount of room for improvement. Performance variability was also high between paediatricians (large IQRs). Performance levels identified in our study for paediatricians did not differ significantly from the ones found for GPs.

As already discussed in detail in our previous work targeting GPs, the PIs we developed are easily measurable at national level using administrative databases and provide less opportunity to ‘game’ the system by adapting the coding of diagnoses. They could be used in different ways to reach public health objectives, both at the prescriber level or further aggregated at a geographical level. With the aim that physicians themselves will use the data to initiate improvement interventions, the PIs could be used to perform automated audit and feedback in almost real time. They might be displayed as personalized antibiotic profiles, with an action plan, at the prescriber level. The PIs could also be used by regional antibiotic stewardship networks/teams and regional or national health authorities as a screening/diagnostic tool to guide and adapt stewardship interventions. The third option for use of these PIs could be that they are integrated into the existing NHI pay-for-performance system. There are currently two indicators in place in France for paediatricians focusing on antibiotic prescribing (both on the proportion of children treated with third-generation cephalosporins over the year, out of children receiving antibiotics). This French programme has led to an almost 4% absolute reduction in antibiotic prescribing for these particular indicators. Our PIs could finally be integrated into a public reporting system, like the PHE Fingertips initiative., The next step should be the validation of these PIs, i.e. a comparison with proper quality indicators, based on clinical diagnoses. This could be done either by conducting an audit using manual data collection on a sample of antibiotic prescriptions or by using quality indicators monitored in an automated way in settings where clinical indications are routinely available in databases.

Very few studies have described similar PIs for children in the outpatient setting. Piovani et al. developed two PIs at prescriber level based on 2011 reimbursement data in one region (Lombardy) in Italy and included 1164 paediatricians. The percentage of paediatricians who reached the target of having prescribed amoxicillin to at least 50% of patients was 12.8%. The percentage of paediatricians who reached the target of having prescribed cephalosporins or macrolides (second-line antibiotics) to less than 10% of patients was 54.0%. The authors acknowledged that, for their first PI, phenoxymethylpenicillin (which is not marketed in Italy) might also be considered in addition to amoxicillin in certain countries, depending on availability and national guidelines. There are no published data on the clinimetric properties of comparable proxy indicators. Some countries, however, use such PIs: for example, in Belgium an amoxicillin/co-amoxiclav ratio is used, with an 80/20 target. Other studies have suggested certain quantity metrics to estimate the appropriateness of antibiotic use in children., Based on the Access, Watch, Reserve (AWaRe) grouping of the WHO Essential Medicines List, Hsia et al. defined three metrics: access percentage, amoxicillin index and access-to-watch index. In another study, de Bie et al. used two metrics: amoxicillin index and amoxicillin/broad-spectrum antibiotics ratio. However, these metrics were not associated with a target, which makes it difficult to differentiate inappropriate from appropriate practices.

Our work is original, but it presents several potential limitations. First, we agree that the targets we set are debatable since they are partly based on expert opinion. A structured consensus procedure involving a large group of stakeholders might be useful to validate these targets. Second, the NHI reimbursement database gives information on the dispensing of antibiotics, not real use by the patient.

In conclusion, we have defined a set of four proxy indicators to estimate the appropriateness of prescriptions, in particular antibiotics, at prescriber level for infectious diseases in children, that are easily calculable based on reimbursement data only. We believe that other countries, especially in settings in which antibiotic indication cannot be ascertained, could adopt the same approach and develop their own set of PIs for outpatient paediatric prescriptions. The latter could be used within antibiotic stewardship programmes to measure and improve antibiotic use. According to their national context/guidelines, they may select different antibiotics or set different targets.

Funding

This study was carried out as part of our routine work.

Transparency declarations

None to declare.

Authors’ contributions

C.P. initiated the study. C.P. wrote the protocol, which was reviewed by all authors. O.P. analysed the data. N.T. and C.P. interpreted the data. N.T. and C.P. wrote the manuscript, which was reviewed by O.P., J.S. and M.E.J.L.H.

Supplementary data

Table S1 is available as Supplementary data at JAC-AMR Online.

Click here for additional data file.

Table 3.

Clinimetric properties of the proxy indicators

PI	Measurability, missing data (%)	Applicability n (%)	Improvement potential (%) (100 − performance)
PI 1: Seasonal variation of total antibiotic use (%)	0	103 (88.8)	93.1
PI 2: Amoxicillin/second-line antibiotics (ratio)	0	104 (89.7)	25.9
PI 3: Co-prescription antibiotic + NSAIDs (%)	0	112 (96.6)	74.1
PI 4: Co-prescription antibiotic + corticosteroids (%)	0	109 (94.0)	70.7

NSAIDs, non-steroidal anti-inflammatory drugs.

Table 4.

Comparison of performance scores between paediatricians and GPs, in patients <16 years old

	Paediatricians’ performance (%)	GPs’ performance6 (%)	P a
PI 1: Seasonal variation of total antibiotic use (%)	6.9	11.7	0.112
PI 2: Amoxicillin/second-line antibiotics (ratio)	75.0	66.9	0.066
PI 3: Co-prescription antibiotic + NSAIDs (%)	31.0	26.5	0.273
PI 4: Co-prescription antibiotic + corticosteroids (%)	31.0	25.3	0.161

NSAIDs, non-steroidal anti-inflammatory drugs.

χ2 test.