Validating a popular outpatient antibiotic database to reliably identify high prescribing physicians for patients 65 years of age and older.

OBJECTIVE: Many jurisdictions lack comprehensive population-based antibiotic use data and rely on third party companies, most commonly IQVIA. Our objective was to validate the accuracy of the IQVIA Xponent antibiotic database in identifying high prescribing physicians compared to the reference standard of a highly accurate population-wide database of outpatient antimicrobial dispensing for patients ≥65 years.
METHODS: We conducted this study between 1 March 2016 and 28 February 2017 in Ontario, Canada. We evaluated the agreement and correlation between the databases using kappa statistics and Bland-Altman plots. We also assessed performance characteristics for Xponent to accurately identify high prescribing physicians with sensitivity, specificity, positive predictive value (PPV), and negative predictive value.
RESULTS: We included 9,272 physicians. The Xponent database has a specificity of 92.4% (95%CI 92.0%-92.8%) and PPV of 77.2% (95%CI 76.0%-78.4%) for correctly identifying the top 25th percentile of physicians by antibiotic volume. In the sensitivity analysis, 94% of the top 25th percentile physicians in Xponent were within the top 40th percentile in the reference database. The mean number of antibiotic prescriptions per physician were similar with a relative difference of -0.4% and 2.7% for female and male patients, respectively. The error was greater in rural areas with a relative difference of -8.4% and -5.6% per physician for female and male patients, respectively. The weighted kappa for quartile agreement was 0.68 (95%CI 0.67-0.69).
CONCLUSION: We validated the IQVIA Xponent antibiotic database to identify high prescribing physicians for patients ≥65 years, and identified some important limitations. Collecting accurate population-based antibiotic use data will remain vital to global antimicrobial stewardship efforts.

Introduction

Rising antimicrobial resistance is a global public health threat jeopardizing multiple advances in modern medicine.[1] Antibiotic overuse in humans is associated with resistance at both the individual [2] and ecological [3] levels, and is the most important modifiable driver of antimicrobial resistance. An essential component of global, national, and regional antimicrobial stewardship strategies is improved surveillance of antibiotic use.[4-6]

Various forms of antibiotic use data are available including dispensing data compiled by pharmacies (e.g. National Prescription Audit® CompuScript® or GPM®)[7] [8] or funders (e.g. Ontario Drug Benefit) [9] which can be population based. Other databases include dispensed data at the prescriber level (e.g. Xponent®) [10] or by prescription from electronic medical records.[11] These data sources have important differences and may not correlate over time. [11,12] Many jurisdictions do not have access to valid population-based community antibiotic use data and rely on third party proprietary data. The most common source of antibiotic use data globally is IQVIA, formerly IMS Health. IQVIA maintains a variety of prescription drug databases. In the United States (U.S.) and Canada, the IQVIA Xponent database is derived from 65–70% of the populations’ dispensed prescriptions [7,13] IQVIA then applies a proprietary geospatial extrapolation algorithm. The data are then sold to third parties representing 100% complete prescription data. The projection methodology is internally validated by the company.[14] Despite some uncertainty in the data validity multiple organizations in Europe, Asia, and North America, such as the U.S. Centers for Disease Control and Prevention and Health Canada, rely on these data for both research and surveillance of antibiotic use patterns.[7,13-23] In Canada we have used IQVIA data to describe overall antibiotic use,[7,24] to identify predictors of prolonged antibiotic durations,[10] and we are currently studying the impact of providing audit and feedback letters from these data to primary care physicians (https://www.clinicaltrials.gov/ct2/show/NCT03776383).

The Ontario Drug Benefit (ODB) database has population-based drug dispensing data for medications funded by the ODB program, for all persons in Ontario, Canada over 65 years of age and has been previously validated compared to chart abstraction to be over 99% accurate.[25] Our objective in this study was to validate the accuracy of the IQVIA antibiotic Xponent database to identify high prescribing physicians compared to ODB as the gold standard for patients 65 years of age and older.

Methods

Setting

We conducted this study at ICES (formerly the Institute for Clinical Evaluative Sciences) in Ontario, Canada. ICES is a not-for-profit research institute and a prescribed entity under Ontario’s Personal Health Information Protection Act with permission to securely collect and store personal health information. Ontario has universal health insurance which includes virtually all residents (excluding recent migrants within the previous 3 months, those residing on an indigenous reserve, and military personnel). All persons 65 years of age or older, as well as select low income individuals or those on disability, have publically funded drug insurance through the ODB which includes most commonly used antibiotics.[26] The study population consisted of physicians prescribing antibiotics to patients 65 years of age or older, between 1 March 2016 and 28 February 2017. We have previously observed large disparities in antibiotic prescribing to male and female patients 65 years of age and older,[7] therefore we sought to validate physician prescribing to male and female patients separately.

Data sources

Xponent is an IQVIA database with dispensed antibiotic prescription counts aggregated at the physician prescriber-level. Non-physician prescribers (i.e. dentists, nurse practitioners, optometrists, etc.) are not included in this database. Antibiotic prescription counts were provided for total and 13 antibiotic class specific groupings. Antibiotics were defined as World Health Organization Anatomical Therapeutic Chemical Classification J01 antibacterials for systemic use limited to those taken orally and dispensed from an outpatient pharmacy (S1 Table). Counts of antibiotic prescriptions were limited to outpatient new prescriptions, excluding refills, topical, and intravenous medications. The data also includes antibiotic prescribing rates (number of antibiotic prescriptions/number of total prescriptions by that physician) and proportion of antibiotic prescriptions that were prolonged in duration which we defined as >8 days.[10] The data was broken down by patient age and sex strata; all patients, males <18 years, females <18 years, males 18–64 years, females 18–64 years, males >64 years, and females >64 years. We limited this study to males and females >64 years for which we had complete population data in ODB to act as the reference standard.

IQVIA creates the Xponent database by obtaining prescription data directly from 2,187 (49.8%) of Ontario’s 4391 pharmacies. As of 2017 this amounted to 61.3% of all Ontario prescriptions. IQVIA then incorporates sales and insurance data, as well as the geographical location of pharmacies not captured, into a patented geospatial projection algorithm and extrapolates to estimate all physician prescribed antibiotics. The methodology is proprietary, but according to the company is routinely internally validated.[15,27] Xponent was linked to ICES using unique College of Physicians and Surgeons of Ontario numbers.

We used ODB, held at ICES, from persons ≥65 years of age as the reference standard. The ODB database contains all dispensed medication claims prescribed by physicians from a formulary of 4400 prescription drugs funded through the ODB program. The ODB database has been previously validated against pharmacy chart abstraction and determined to be >99% accurate.[25] To exclude refilled prescriptions from both Xponent and ODB, we defined refills in ODB as the same drug being dispensed for the same patient, prescribed by the same physician for the identical dose and duration within 14 days of the expected end date of the previous prescription. The only antibiotic expected to be potentially poorly captured in ODB was doxycycline as was not funded by the ODB at the time of this study. We used the Ontario Health Insurance Plan database to ascertain the number of outpatient visits.

We excluded physicians who prescribed <9 antibiotics to patients ≥65 years of age during the study year in either Xponent or ODB databases. We used this cut off, which removes 5% of all antibiotic prescriptions, in order to exclude infrequent antibiotic prescribers. We excluded physicians with <200 total outpatient visits in the year based on billing claims in the Ontario Health Insurance Plan database. We also excluded the few physicians who had illogical antibiotic prescribing rates of ≤0% or ≥100%, or if there were more antibiotic prescriptions than patient visits.

Statistical analysis

We performed multiple analyses to evaluate the agreement and correlation between Xponent and ODB antibiotic data in patients ≥65 years of age. Our primary interest with the data was to be able to reliably identify high antibiotic prescribing physicians. We organized both Xponent and ODB in quartiles based on the volume of antibiotic prescriptions per physician. We first dichotomized the quartiles as high (top quartile) and low (bottom three quartiles) prescribers to calculate the performance characteristics of accurately identifying a high prescriber in Xponent. We first did this for all physicians and then restricted to primary care physicians only since this specialty prescribes majority of the outpatient antibiotics.[24] We reported sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), with 95% confidence intervals (CIs). As a sensitivity analysis, we adjusted the definition of high prescribers in ODB from the top 25% to 30% and to 40% in order to evaluate the degree of Xponent error. For instance, if 20% of high prescribers were misclassified in Xponent, but 90% of these were in the top 40% of ODB, this provides more reassurance related to the magnitude of error in the data. If low prescribing physicians in the bottom 60% were misclassified as high prescribers, this would represent a more clinically significant misclassification.

We descriptively compared the mean number of antibiotics prescribed per physician. Then we used the quartiles and evaluated agreement using weighted kappa statistics with 95% CIs for total antibiotics as well as subclasses of antibiotics. We did this for male and female patients separately as well as by urban and rural locations of practice. Rural areas were defined by the physician location of practice residing in a town with a population of <10,000. Kappa statistics were considered poor if <0, slight if 0–0.20, fair if 0.21–0.40, moderate if 0.41–0.60, substantial if 0.61–0.80, and almost perfect if 0.81–1.00.[28] To quantify the magnitude of the error we calculated the relative difference in the number of antibiotics per physician as (Xp-ODB)/ODB x100; where Xp is the number of antibiotics per physician in Xponent and ODB is the number of antibiotics per physician in the ODB database.

We evaluated the total antibiotic prescribing rate (number of antibiotic prescriptions per 100 total prescriptions) using Bland-Altman plots.[29] Bland-Altman plots allow one to visualize the error between two data sources by plotting the difference of the values on the Y-axis (Xponent-ODB) against the mean of the values on the X-axis ((Xponent+ODB)/2), along with plots of the two standard deviation (SD) limits above and below the mean difference. We performed this separately for male and female patients. IQVIA data does not contain numbers of patient visits, therefore we assessed the correlation of the antibiotic prescribing rate (number of antibiotic prescriptions per 100 total prescriptions) to number of antibiotic prescriptions per 100 patient visits using Spearman correlation coefficients.

To validate the antibiotic duration data, we defined antibiotic prescription durations that were >8 days as prolonged.[10] Similar to the above analysis we organized physicians into quartiles of proportion of prolonged durations and calculated performance characteristics, weighted kappa statistics, and Bland-Altman plots as described above for all physicians and primary care physicians only. Results were reported in accordance with guidelines for validation studies with administrative data.[30]

Ethics

This project has Ethics Research Board approval from Public Health Ontario 2017–032.02.

Results

The Xponent database had 25,678 unique physicians and 24,878 (97%) were successfully linked to ODB at ICES using College of Physicians and Surgeons identifiers. After applying the study exclusion criteria 14,671 were excluded for prescribing <9 antibiotics, 919 for <200 patient visits, and 16 for illogical prescribing rates, leaving 9,272 physicians included in this validation analysis (Fig 1).

Fig 1

Flow chart of study exclusions.

Physician characteristics

These physicians were mostly male (68%) and 83% were primary care physicians. The majority of these physicians saw ≥20 patients daily, 90% worked in an urban area, and 34% were not Canadian trained (Table 1).

Table 1

Demographic characteristics of the 9,272 physicians included in this study.

Physician characteristic	Number (%)
Age, mean (SD)*	52.3 (11.4)
Years since medical graduation^
Early career (<11 years)	3,309 (35.7%)
Middle career 11–24 years)	2,396 (25.8%)
Late career (≥25 years)	3,567 (38.5%)
Gender^
Female	3,000 (32.4%)
Male	6,272 (67.6%)
Specialty^
Primary care**	7,734 (83.3%)
Internal medicine and subspecialties	669 (7.3%)
Surgical specialty	562 (6.1%)
Emergency medicine	200 (2.2%)
Dermatology	68 (0.7%)
Other	39 (<0.1%)
Area of practice
Urban	8,376 (90.3%)
Rural	896 (9.7%)
Location of medical training
Canada	6,093 (65.7%)
Outside Canada	2,346 (25.3%)
Missing	833 (9.0%)
Median daily patient visits
0–9	34 (0.4%)
10–19	2,035 (22.0%)
≥20	7,203 (77.7%)
Average patient Chronic Disease Score per physician***
Low (≤4)	994 (10.7%)
Medium (5)	3,504 (37.8%)
High (≥6)	4,770 (51.5%)

*8.8% of physicians were missing a value for age

**Primary care defined as specialties of family medicine, general practice, or community medicine

***The Chronic Disease Score is a comorbidity index based on pharmaceutical data,[34] Missing data from 4 physicians. ^These variables derived from Xponent, the other variables are from ICES.

Performance characteristics

The Xponent database has a specificity of 92.4% (95%CI 92.0%-92.8%) and PPV of 77.2% (95%CI 76.0%-78.4%) for correctly identifying the top 25th percentile of physicians by antibiotic prescribing volume. The performance measures of correctly identifying the top 25th percentile of prolonged duration prescribers was slightly better with a specificity of 94.0% (95%CI 93.6%-94.4%) and PPV of 82.0% (95%CI 80.8%-83.0%). There was little difference when the cohort was limited to only primary care physicians (Table 2). The sensitivity analysis demonstrated that the magnitude of the misclassification was relatively small. In the primary analysis 77% of physicians were correctly classified in Xponent as top 25th percentile prescribers, however 85% were at least within the top 30th percentile, and 94% in the top 40th percentile of ODB prescribers (Fig 2).

Table 2

Performance characteristics of Xponent in defining the top quartile (25% of physicians) as compared to the reference standard of the Ontario Drug Benefit (ODB) for antibiotic volume and the proportion of prolonged duration prescriptions.

Comparator in ODB	TP	FN	FP	TN	Sensitivity (95% CI)	Specificity (95% CI)	Positive Predictive Value (95% CI)	Negative Predictive value (95% CI)
Comparing the highest 25% in IQVIA to the highest 25% in ODB
All physicians antibiotic volume	3580	1056	1056	12852	77.2% (76.0%-78.4%)	92.4% (92.0%-92.8%)	77.2% (76.0%-78.4%)	92.4% (92.0%-92.8%)
Family physicians antibiotic volume	2924	942	942	10660	75.6% (74.3%-77.0%)	91.9% (91.4%-92.4%)	75.6% (74.3%-77.0%)	91.9% (91.4%-92.4%)
All physician proportion prolonged duration	3799	837	837	13071	82.0% (80.8%-83.0%)	94.0% (93.6%-94.4%)	82.0% (80.8%-83.0%)	94.0% (93.6%-94.4%)
Family physician proportion prolonged duration	3173	693	693	10909	82.1% (80.8%-83.3%)	94.0% (93.6%-94.5%)	82.1% (80.8%-83.3%)	94.0% (93.6%-94.5%)
Sensitivity analysis
Comparing the highest 25% in IQVIA to the highest 30% in ODB
All physicians antibiotic volume	3949	1613	687	12295	71% (69.8%-72.2%)	94.7% (94.3%-95.1%)	85.2% (84.1%-86.2%)	88.4% (87.9%-88.9%)
Family physicians antibiotic volume	3240	1400	626	10202	69.8% (68.5%-71.2%)	94.2% (93.8%-94.7%)	83.8% (82.6%-85.0%)	87.9% (87.3%-88.5%)
All physician proportion prolonged duration	4126	1436	510	12472	74.2% (73.0%-75.3%)	96.1% (95.7%-96.4%)	89.0% (88.1%-89.9%)	89.7% (89.2%-90.2%)
Family physician proportion prolonged duration	3431	1209	435	10393	73.9% (72.7%-75.2%)	96.0% (95.6%-96.3%)	88.8% (87.7%-89.7%)	89.6% (89.0%-90.1%)
Comparing the highest 25% in IQVIA to the highest 40% in ODB
All physicians antibiotic volume	4371	3045	265	10863	58.9% (57.8%-60.1%)	97.6% (97.3%-97.9%)	94.3% (93.6%-94.9%)	78.1% (77.4%-78.8%)
Family physicians antibiotic volume	3601	2585	265	9017	58.2% (57.0%-59.5%)	97.2% (96.8%-97.5%)	93.2% (92.3%-93.9%)	77.7% (77.0%-78.5%)
All physician proportion prolonged duration	4421	2995	215	10913	59.6% (58.5%-60.7%)	98.1% (97.8%-98.3%)	95.4% (94.7%-96.0%)	78.5% (77.8%-79.2%)
Family physician proportion prolonged duration	3666	2520	200	9082	59.3% (58.0%-60.5%)	97.9% (97.5%-98.1%)	94.8% (94.1%-95.5%)	78.3% (77.5%-79.0%)

TP = True Positives; FN = False Negatives; FP = False Positives; TN = True Negatives; Prolonged duration defined as >8 days

Fig 2

Positive Predictive Values, with 95% confidence intervals, comparing the highest quartile from Xponent to ODB to identify high volume and high prolonged duration prescribers.

Agreement

We descriptively compared the mean number of antibiotic prescriptions per physician and found them to be similar with a relative difference of -0.4% and 2.7% for female and male patients, respectively. The error was greater in rural areas with a relative difference of -8.4% and -5.6% per physician for female and male patients, respectively. There were some notable differences between antibiotic subclasses, particularly for trimethoprim and/or sulfamethoxazole (Table 3). Overall, the weighted kappa for quartile agreement between Xponent and ODB was substantial at 0.68 (95%CI 0.67–0.69) for both female and male patients.

Table 3

Agreement between physician antibiotic prescribing quartiles in Xponent compared to the Ontario Drug Benefit (ODB) database.

	Mean number of antibiotic prescriptions per physician			Agreement between quartiles
Stratum	Xponent	ODB	Relative difference*	Weighted kappa (95% CI)
Female Patients
Total antibiotics	85.2	85.5	-0.4%	0.68 (0.67–0.69)
Rural regions	79.6	86.8	-8.4%	0.69 (0.65–0.72)
Urban regions	85.8	85.4	0.5%	0.68 (0.67–0.69)
Antibiotic subclasses
Penicillins without Beta-Lactamase Inhibitor	11.8	11.8	-0.5%	0.71 (0.70–0.72)
Penicillins with Beta-Lactamase Inhibitor	4.8	4.9	-2.0%	0.64 (0.63–0.65)
Cephalosporins (First Generation)	8.9	9.3	-4.0%	0.68 (0.67–0.69)
Cephalosporins (Second/Third Generation)	4.7	4.8	-1.6%	0.69 (0.68–0.70)
Fluoroquinolones (Second Generation)	11.4	11.1	3.4%	0.69 (0.68–0.70)
Fluoroquinolones (Third Generation)	5.9	5.9	0.3%	0.65 (0.64–0.66)
Macrolides	12.8	12.7	1.1%	0.73 (0.72–0.74)
Trimethoprim and/or Sulphonamides	4.9	5.6	-13.1%	0.64 (0.62–0.65)
Nitrofurantoin	12.8	13.8	-7.4%	0.71 (0.70–0.72)
Tetracyclines**	1.5	0.6	167.5%	0.42 (0.40–0.43)
Lincosamides	1.4	1.4	0.3%	0.70 (0.69–0.71)
Metronidazole	2.1	2.1	1.5%	0.67(0.66–0.68)
Other	1.5	1.7	-12.1%	0.81 (0.80–0.82)
Male Patients
Total antibiotics	57.9	56.3	2.7%	0.68 (0.67–0.69)
Rural regions	51.8	54.9	-5.6%	0.69(0.66–0.73)
Urban regions	58.5	56.5	3.6%	0.68 (0.66–0.69)
Antibiotic subclasses
Penicillins without Beta-Lactamase Inhibitor	8.6	8.6	0.7%	0.69 (0.68–0.70)
Penicillins with Beta-Lactamase Inhibitor	4.1	4.1	-0.6%	0.71 (0.70–0.72)
Cephalosporins (First Generation)	7.7	7.6	1.5%	0.68 (0.67–0.69)
Cephalosporins (Second/Third Generation)	3.7	3.7	-0.8%	0.66 (0.64–0.67)
Fluoroquinolones (Second Generation)	9.4	8.6	9.2%	0.65 (0.64–0.66)
Fluoroquinolones (Third Generation)	5.1	5.0	1.9%	0.62 (0.61–0.63)
Macrolides	9.3	9.3	0.5%	0.73 (0.73–0.74)
Trimethoprim and/or Sulphonamides	2.9	3.3	-12.5%	0.68 (0.67–0.69)
Nitrofurantoin	2.7	2.8	-3.3%	0.69 (0.68–0.70)
Tetracyclines**	1.4	0.6	116.4%	0.46 (0.45–0.48)
Lincosamides	1.0	1.0	5.4%	0.74 (0.73–0.75)
Metronidazole	1.5	1.5	-0.5%	0.68 (0.67–0.70)
Other	0.3	0.3	-20.0%	0.90 (0.89–0.90)

*(Xponent-ODB)/ODB x 100

**Doxycycline was not covered under the ODB program during this time period likely resulting in falsely low counts of tetracyclines within ODB; ODB = Ontario Drug Benefit

We constructed Bland-Altman plots to visualize and quantify the agreement in antibiotic prescribing rate (number of antibiotic prescriptions per 100 total prescriptions) between Xponent and ODB (Fig 3). These databases had high agreement for this variable with a mean difference (Xponent rate minus ODB rate) of -0.5 antibiotics per 100 total prescriptions for female patients. Meaning, that on average the antibiotic rate was 0.5 antibiotics per 100 total prescriptions higher in ODB than Xponent. However there were substantial outliers (mean-2SD = -16.4 to mean+2SD 15.4). Results were similar for male patients (S1 Fig).

Fig 3

Bland-Altman Plot from 9,272 physicians comparing the antibiotic rates (antibiotic prescriptions per 100 total medications prescribed) between Xponent and Ontario Drug Benefit (ODB) for female patients only.

Dash line = Mean difference (Xponent-ODB) = -0.5; Dotted lines = mean-2SD = -16.4 to mean+2SD = 15.4.

The proportion of prolonged duration antibiotic prescriptions had overall substantial agreement as well between the data sources with more error observed in rural areas (Table 4). On the Bland-Altman plot the mean difference (Xponent-ODB) was -0.3 prolonged antibiotic prescriptions per 100 antibiotic prescriptions. Meaning, that on average the proportion of prolonged antibiotic duration prescriptions was 0.3% higher in ODB than Xponent. Similarly, substantial outliers were observed (mean-2SD -17.8 to mean+2SD 17.1) (Fig 4 and S2 Fig).

Table 4

Agreement between physician percent prolonged antibiotic prescription duration (defined as >8 days) quartiles in Xponent compared to the Ontario Drug Benefit (ODB) database.

	Mean percent prolonged duration per physician			Agreement between quartiles
Stratum	Xponent	ODB	Relative difference*	Weighted kappa (95% CI)
Female Patients
Total	31.3	31.5	-0.8%	0.74 (0.73–0.75)
Rural region	32.0	31.3	2.0%	0.69 (0.66–0.72)
Urban region	31.2	31.6	-1.1%	0.74 (0.73–0.75)
Male Patients
Total	38.5	39.2	-1.8%	0.73 (0.73–0.74)
Rural region	41.8	41.0	1.8%	0.69 (0.66–0.72)
Urban region	38.2	39.0	-2.1%	0.74 (0.73–0.75)

*(Xponent-ODB)/ODB x 100; ODB = Ontario Drug Benefit

Fig 4

Bland-Altman Plot from 9,272 physicians comparing the proportion of prolonged antibiotic duration (defined as >8 days) between Xponent and Ontario Drug Benefit (ODB) for female patients only.

Dash line = Mean difference (Xponent-ODB) = -0.3; Dotted lines = mean-2SD = -17.8 to mean+2SD = 17.1.

Physician ranking according to antibiotic prescriptions per 100 total prescriptions in Xponent was strongly correlated to the number of antibiotic prescriptions per 100 patient visits in ODB data for primary care physicians treating female (Spearman coefficient = 0.93 p<0.001, Fig 5) and male (Spearman coefficient = 0.93, p<0.001, S3 Fig) patients.

Fig 5

Correlation of the number of antibiotic prescriptions per 100 total prescriptions in Xponent compared to antibiotic prescriptions per 100 patient visits in the Ontario Drug Benefit (ODB) database for female patients by primary care physicians.

Spearman correlation = 0.93; p<0.001.

Discussion

Accurately monitoring population antibiotic use is a critical component of antimicrobial stewardship. We have validated that the IQVIA Xponent antibiotic database reliably identified high physician prescribers for patients 65 years of age and older. We identified that 23% of high antibiotic prescribing physicians were misclassified as being in the highest quartile of antibiotic prescribing volume, however only 6% of these physicians fell outside of the top 40% of prescribers. Furthermore, Xponent accurately captured the average number of antibiotic prescriptions per physician and the proportion of prolonged duration prescriptions, defined as >8 days. However, we identified an important limitation in the data with larger errors noted for physicians practicing in rural locations. This is most likely a reflection of a greater reliance on IQVIA’s projection algorithm as they collect less prescription data directly from pharmacies in rural locations. This error was slightly more pronounced in female patients. Further exploration regarding differences in data validity between male and female patients may be warranted.

Tan et al demonstrated the validity of hospital antibiotic purchasing data from IQVIA as a reliable metric of antibiotic utilization compared to internal pharmacy records.[31] Abstracts validating IQVIA antibiotic databases in other Canadian jurisdictions, performed with patients of all ages, have been previously presented.[32,33] Lin et al demonstrated substantial variability between different databases in the U.S. for estimating antibiotic use, including a database from IMS (renamed as IQVIA), highlighting the importance of reliable data sources for antibiotic use.[12] Both Canada and the U.S. rely on IQVIA for monitoring antibiotic use trends as well as assessing variability between regions and physicians.[7,13,15,24]

This study provides reassurance that IQVIA measures of total antibiotic prescribing in Xponent by physicians is valid. Furthermore, these results can assist public health departments, researchers, and policy makers towards appropriate uses of the data. We have demonstrated reasonable reliability of Xponent to identify high antibiotic prescribing physicians as well as reliably identifying physicians who prescribe prolonged antibiotic durations. Xponent is also quite reliable in estimating the number of antibiotic prescriptions per physician for both male and female patients. However, we caution the reliance on this database in rural areas or geographical locations where the data relies more heavily on the IQVIA projection algorithm. We demonstrated that antibiotic prescriptions per 100 total prescriptions is imperfect, but is highly correlated with antibiotic prescriptions per 100 patient visits for primary care physicians in patients 65 years of age or older. This is a potentially useful metric from pharmacy data to indirectly account for differences in patient volume, when patient visits are unavailable.

This study has some limitations. While we demonstrated that the misclassification of prescribers in the top 25th percentile was small, this error may be important to clinicians receiving feedback on their prescribing. Certain subclasses of antibiotics were less well correlated between IQVIA and ODB databases, particularly trimethoprim and/or sulfamethoxazole and nitrofurantoin. One possible explanation for this could be the differences between the databases in defining repeats as these drugs are frequently used for prophylaxis for urinary tract infections in seniors. The IQVIA database utilized an explicit field from pharmacy data to denote refill prescriptions. No such field exists in ODB, and our study definition of repeats may have resulted in some differences in counting new antibiotic prescriptions. In addition, tetracyclines appeared to be overestimated in Xponent, however it is more likely that doxycycline was underestimated in ODB given that it was one of the only antibiotic treatments not funded by ODB during the study period. ODB does not fund First Nations populations with Non-insured Health Benefits, however this represents a small proportion of the included patient population. In Ontario, we only have complete population pharmacy data for patients 65 years of age or older. As a result validation studies may need to be conducted in other jurisdictions to determine whether our findings are generalizable to younger patient age groups. Non-physician prescribers are not included in either ODB or Xponent, and other databases are needed to study these important antibiotic prescribing populations.

In conclusion, we have validated the IQVIA Xponent antibiotic database to identify high prescribing physicians for patients 65 years of age and older, and identified some important limitations. Collecting accurate population-based antibiotic use data will remain vital to global efforts to combat rising rates of antimicrobial resistance. Governments and public health organizations should prioritize the need for accurate, population-based antimicrobial use datasets. An understanding of the uses and limitations of available databases are crucial for sound research and public policy decisions.

Oral outpatient antibiotic drugs included in the various antibiotic classes.

(DOCX)

Click here for additional data file.

Bland-Altman plot from 9,272 physicians comparing the antibiotic rates (number of antibiotics prescribed per 100 total medications prescribed) between Xponent and Ontario Drug Benefit (ODB) for male patients only.

Dash line = Mean difference (Xponent-ODB) = -0.5; Dotted lines = mean-2SD = -16.0 to mean+2SD = 15.0.

(DOCX)

Click here for additional data file.

Bland-Altman plot from 9,272 physicians comparing the proportion of prolonged antibiotic duration (defined as >8 days) between Xponent and ODB for male patients only.

Dash line = Mean difference (Xponent-ODB) = -0.1; Dotted lines = mean-2SD = -21.0 to mean+2SD = 20.7.

(DOCX)

Click here for additional data file.

Correlation of antibiotic prescriptions per 100 total prescriptions in Xponent compared to antibiotic prescriptions per 100 patient visits in the Ontario Drug Benefit (ODB) database for male patients by primary care physicians.

(DOCX)

Click here for additional data file.

24 Jul 2019

PONE-D-19-17515

Validating a Popular Outpatient Antibiotic Database

PLOS ONE

Dear Dr. Schwartz,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR:

Overall this manuscript addresses an important topic and has used a robust  methodology and chose to use Bland Altman plots which was an excellent choice. However there are certain issues to address to improve the manuscript.

Please respond to all the peer reviewer comments . In addition please respond to the following Academic Editor 's comments

1.Reviewer 2 noted 2 previous citations presented in poster format that did comparisons from 2 Canadian provinces which had comprehensive data ( all ages)  on outpatient prescriptions and one compared the data from the Xponent database so the authors need to rephrase their comments " to our knowledge community antibiotic use data has neve been externally validated.." which appears in a couple of settings in the manuscript.

2. You mention about a variety of prescription drug databases in the Introduction. Please add a couple of sentences to describe the commonly used ones and their differences eg Compuscript, the Canadian Disease and Therapeutic Index ,  National Prescription Audit  data and National Sales and Prescriptions data for the benefit of the readership.

3. Provide an estimate of NIHB prescriptions that may have been missed .

4. Please explicitly  explain if the prescriptions from non physician prescribers eg dentists, midwives, podiatrists etc were excluded reliably from the Xponent database which has obvious important implications

5. Please describe the geographic pharmacy difference if possible and what implications it may have

6. The Discussion is missing a limitations paragraph and there are several important ones - some mentioned earlier in the manuscript  but should be highlighted plus other mentioned about the limitations of the Xponenet database, non captured scripts eg NIHB, other prescribers, missing data and its influence and impact of a sensitivity analysis of > 14 days use as opposed to 8 .

7. There are several typos that need to be fixed eg refs 7-35 , presume it is 7, 35, missing words in some of the sentences

8. The references are very sloppy and full of errors ( too numerous to count )  in case, non use of urls and date of access, inappropriate case, spacing, non use of italics for Latin terms , incomplete references, missing references, inappropriate journal abbreviations and appear not to have been proof read by the authors very carefully . Please clean up ALL errors in the references.

==============================

We would appreciate receiving your revised manuscript by August 25th 2019. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

John Conly, MD

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

3. Our internal editors have looked over your manuscript and determined that it is within the scope of our Antimicrobial Resistance call for papers. This collection of papers is headed by a team of Guest Editors for PLOS ONE: Kathryn Holt (Monash University and London School of Hygiene and Tropical Medicine), Alison H. Holmes (Imperial College London), Alessandro Cassini (WHO Infection Prevention and Control Global Unit), Jaap A. Wagenaar (Utrecht University). The Collection will encompass a diverse range of research articles; additional information can be found on our announcement page: https://collections.plos.org/s/antimicrobial-resistance. If you would like your manuscript to be considered for this collection, please let us know in your cover letter and we will ensure that your paper is treated as if you were responding to this call. If you would prefer to remove your manuscript from collection consideration, please specify this in the cover letter.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript:

This study was funded by Public Health Ontario

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

The authors received no specific funding for this work

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Overall, this is a well written paper on a topic of considerable importance. Recognizing the need for reliable outpatient antibiotic utilization data, this study attempts to validate a pre-existing database that can be used to optimize outpatient antimicrobial stewardship. The study limitations were well recognized and stated clearly [ie. insufficient correlation between specific antibiotics and inability to validate for a) patients <65 years old, b) refill prescriptions]. The study also identified a weakness in the database with respect to the rural population which was well explained. The article's greatest strength is its rigorous statistical analyses.

A few additional comments for author response are:

1. With a NPV of 73%, 23% of prescribers were classified as belonging to the highest quartile of antibiotic prescribing when really they belonged to the top 40% according to OBD (the gold standard database). From an antimicrobial stewardship perspective, this discrepancy is acceptable because it still represents a physician group that should be targeted for further stewardship education and optimization of antibiotic use. However, from a clinician's perspective, being labeled as within the top 25% is very different from top 40%, the former signalling higher end of normal and the latter potentially being close to the mean. This NPV may not be acceptable to a clinician receiving their antibiotic prescribing summary.

2. Given a large subset of physicians are likely not represented (eg. rural physician etc), it would be interesting to see how many registered physicians within Ontario are excluded from the Xponent database entirely and therefore would not be subject to antibiotic utilization scrutiny. Is it possible to match the physicians to the College of Physicians and Surgeons of Ontario to obtain this number?

3. The authors performed multiple separate analyses for female vs. male genders and demonstrated that there were was a slightly greater relative difference in prescribing for male patients. Nevertheless, the discussions section lacked explanations on the potential reasons and significance of this finding. This should be further elaborated on.

4. The antibiotic prescribing rate "antibiotic prescriptions/100 total prescriptions" is an interesting metric. It is conceivable that a clinician whom is prescribing antibiotics excessively would likely prescribe other drugs excessively as well thereby obscuring the increased antibiotic use. It was reassuring to see that "antibiotic prescriptions/100 total prescriptions" correlated well with "antibiotic prescriptions / 100 patient visits" given the denominator for antibiotic utilization is typically patient visits to take into account differences in patient volume. It is not clear to me where the patient visit denominator is coming from (is it from OBD or an alternate source) and could these not just be incorporated into the numerator generated from Xponent rather than relying on the "antibiotic prescriptions/100 total prescriptions".

Reviewer #2: This topic is of great interest to likely a small group of individuals. IQVIA formerly Brogan, formerly IMS has recorded data on antimicrobial and other drug utilization from retail pharmacies for many years. The validity of the methods to extrapolate from the sample of pharmacies to the whole population has never been public and therefore the science community has been left to wonder how well their proprietary methods work and can they be trusted for research. In addition to the cited work of Tan (reference 42), two other groups in Canada have completed similar work in conference papers and could be cited (Dalton, B; Sabuda D, Bresee L et al. External Validation of Estimates of Antibacterial Dispensing in the IMS Brogan Xponent® Database in a Canadian Province. IDWeek 2013 https://idsa.confex.com/idsa/2013/webprogram/Paper41244.html) & Chong M, Dutil L, Bhatia T, Marra F, Patrick DM. Assessing antimicrobial consumption

using two different methodologies in British Columbia. Can J Infect Dis Med Microbiol

2007; 18(1): 35 Abstract A3.

In general, the authors have conducted a careful analysis using appropriate methods, and this manuscript is likely deserving of publication. I have a few suggestions to help make small improvements in clarity.

The goal of this study was to reliably predict the top 25 percentile of prescribers of antimicrobials to patients 65 and older, by identifying them in a public database of all pharmacies and comparing to the IQVIA database. I think the authors could do a better job describing this objective in the title as one does not reflexively think that quantification and validation would be performed at a physician level. I think some discussion of how these data would relate to the exponent database derived data on population level antimicrobial utilization would be appropriate (or if not at all, please state) and the conclusion "that the xponent database is validated for patients 65 and older " is actually inaccurate considering the principle finding of validating the identification of high prescibers of antibiotics in a population of 65 and older. "

The definition of physician antibiotic prescribing rate is unclear. Is this based on number of prescriptions, days of therapy or DDD etc.

Physicians were excluded for "prescribing less than 9 antibiotics". Does this mean <9 antibiotic prescriptions over the time period? Can you explain this number?

Could you define "antibiotics" in the study methods better? eg "systemic antibacterials" rather than just referring reader to the supplement.

The methods and results for agreement are confusing. In the methods it is stated the agreement of quartile groupings was evaluated, so one expects results in terms of categorical analysis. However in the text of results and table 3 mean number of of antibiotics prescriptions per physicians is reported.

The agreement on antibiotic prescribing per 100 prescriptions was assessed by Bland Altman plots. I am unclear of the relevance of antibiotic prescription per 100 prescriptions. This should be discussed in background and discussion.

If there are a significant number of non physician prescribers in Ontario, are they captured in ODB and exponent? Even if their prescribing rates are lower than that of physicians, it would be useful to note if agreement is similar with non physicians.

In figures 5 and S3 one can observe correlation but there appears to be bias with discounting of the xponent values. This is not commented upon. Can the slope of the regression line be provided? It would also provide easier interpretation if the x and y axes were given the same scales and number values.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Bruce Dalton

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

8 Aug 2019

PONE-D-19-17515

Validating a Popular Outpatient Antibiotic Database

PLOS ONE

Dear Dr. Conly,

Thank you for the thoughtful review and helpful comments. We have incorporated all comments where possible and feel this has greatly improved the manuscript. We would like to be considered for the call for antimicrobial resistance papers.

The data for this study is not publically available. The data set from this study is held securely in coded form at ICES. While data sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/DAS. The full data set creation plan and underlying analytic code are available from the authors upon request, understanding that the programs may rely upon coding templates or macros that are unique to ICES. The IQVIA Xponent dataset is owned and proprietary by IQVIA. The contract between PHO and IQVIA does not permit us to share the data publically. The authors had no special access privileges and other researchers may purchase the data from IQVIA directly (www.iqvia.com).

Sincerely,

Kevin Schwartz

On behalf of the co-authors

==============================

ACADEMIC EDITOR:

Overall this manuscript addresses an important topic and has used a robust methodology and chose to use Bland Altman plots which was an excellent choice. However there are certain issues to address to improve the manuscript.

Please respond to all the peer reviewer comments . In addition please respond to the following Academic Editor 's comments

1.Reviewer 2 noted 2 previous citations presented in poster format that did comparisons from 2 Canadian provinces which had comprehensive data ( all ages) on outpatient prescriptions and one compared the data from the Xponent database so the authors need to rephrase their comments " to our knowledge community antibiotic use data has neve been externally validated.." which appears in a couple of settings in the manuscript.

Response: Reference to this being the first validation study has been removed

2. You mention about a variety of prescription drug databases in the Introduction. Please add a couple of sentences to describe the commonly used ones and their differences eg Compuscript, the Canadian Disease and Therapeutic Index , National Prescription Audit data and National Sales and Prescriptions data for the benefit of the readership.

Response: The following sentences have been added to line 53: “Various forms of antibiotic use data are available including dispensing data compiled by pharmacies (e.g. National Prescription Audit® Compuscript® or GPM®)[7] or funders (e.g. Ontario Drug Benefit)[8] which can be population based. Other databases include dispensed data at the prescriber level (e.g. Xponent®) [9] or by prescription from electronic medical records.[10] These data sources have important differences and may not correlate over time. [11,12]”

3. Provide an estimate of NIHB prescriptions that may have been missed .

Response: We do not have precise numbers for this. There are approximately 10,000 registered Indian Status seniors living in Ontario according to StatsCan (https://www12.statcan.gc.ca/nhs-enm/2011/as-sa/99-011-x/99-011-x2011001-eng.cfm); roughly 2 million Ontario seniors are eligible for ODB. We feel this is unlikely to introduce significant bias at the physician level. We have added this line to the limitations line 257: “ODB does not fund First Nations populations with Non-insured Health Benefits, however this represents a small proportion of the included patient population.”

4. Please explicitly explain if the prescriptions from non physician prescribers eg dentists, midwives, podiatrists etc were excluded reliably from the Xponent database which has obvious important implications

Response: Non-physicians are not captured. We added this sentence line 93: “Non-physician prescribers (i.e. dentists, nurse practitioners, optometrists, etc.) are not included in this database.”

5. Please describe the geographic pharmacy difference if possible and what implications it may have

Response: We do not have granular data on the included and not included pharmacies from IQVIA. They will not share specific information but do admit that rural and smaller independent pharmacies are less likely to be included. This is reflected in the discussion line 221: “However, we identified an important limitation in the data with larger errors noted for physicians practicing in rural locations. This is most likely a reflection of a greater reliance on IQVIA’s projection algorithm as they collect less prescription data directly from pharmacies in rural locations.”

6. The Discussion is missing a limitations paragraph and there are several important ones - some mentioned earlier in the manuscript but should be highlighted plus other mentioned about the limitations of the Xponenet database, non captured scripts eg NIHB, other prescribers, missing data and its influence and impact of a sensitivity analysis of > 14 days use as opposed to 8 .

Response: The penultimate discussion paragraph has been edited to reflect the study's limitations (Line 247)

7. There are several typos that need to be fixed eg refs 7-35 , presume it is 7, 35, missing words in some of the sentences

Response: We have corrected all identified typos

8. The references are very sloppy and full of errors ( too numerous to count ) in case, non use of urls and date of access, inappropriate case, spacing, non use of italics for Latin terms , incomplete references, missing references, inappropriate journal abbreviations and appear not to have been proof read by the authors very carefully . Please clean up ALL errors in the references.

Response: Referencing formatting have been corrected to PLOS one standards and all errors have been corrected.

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response: Manuscript modified to conform to these requirements

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Response: Done

3. Our internal editors have looked over your manuscript and determined that it is within the scope of our Antimicrobial Resistance call for papers. This collection of papers is headed by a team of Guest Editors for PLOS ONE: Kathryn Holt (Monash University and London School of Hygiene and Tropical Medicine), Alison H. Holmes (Imperial College London), Alessandro Cassini (WHO Infection Prevention and Control Global Unit), Jaap A. Wagenaar (Utrecht University). The Collection will encompass a diverse range of research articles; additional information can be found on our announcement page: https://collections.plos.org/s/antimicrobial-resistance. If you would like your manuscript to be considered for this collection, please let us know in your cover letter and we will ensure that your paper is treated as if you were responding to this call. If you would prefer to remove your manuscript from collection consideration, please specify this in the cover letter.

Response: We would like to be considered for the collection, and have added to the cover letter above.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

Response: We apologize for any confusion but ICES data and IQVIA data cannot be shared publically. We have provided the explanations in the cover letter above.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript:

This study was funded by Public Health Ontario

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

The authors received no specific funding for this work

Response: Please change to: This study was funded by Public Health Ontario. We have removed this line from the manuscript

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Response: Please see cover letter above for the explanation

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Overall, this is a well written paper on a topic of considerable importance. Recognizing the need for reliable outpatient antibiotic utilization data, this study attempts to validate a pre-existing database that can be used to optimize outpatient antimicrobial stewardship. The study limitations were well recognized and stated clearly [ie. insufficient correlation between specific antibiotics and inability to validate for a) patients <65 years old, b) refill prescriptions]. The study also identified a weakness in the database with respect to the rural population which was well explained. The article's greatest strength is its rigorous statistical analyses.

A few additional comments for author response are:

1. With a NPV of 73%, 23% of prescribers were classified as belonging to the highest quartile of antibiotic prescribing when really they belonged to the top 40% according to OBD (the gold standard database). From an antimicrobial stewardship perspective, this discrepancy is acceptable because it still represents a physician group that should be targeted for further stewardship education and optimization of antibiotic use. However, from a clinician's perspective, being labeled as within the top 25% is very different from top 40%, the former signalling higher end of normal and the latter potentially being close to the mean. This NPV may not be acceptable to a clinician receiving their antibiotic prescribing summary.

Response: Thank for this comment. We have added this line to the limitations paragraph line 247): “While we demonstrated that the misclassification of prescribers in the top 25th percentile was small, this error may be important to clinicians receiving feedback on their prescribing.”

2. Given a large subset of physicians are likely not represented (eg. rural physician etc), it would be interesting to see how many registered physicians within Ontario are excluded from the Xponent database entirely and therefore would not be subject to antibiotic utilization scrutiny. Is it possible to match the physicians to the College of Physicians and Surgeons of Ontario to obtain this number?

Response: We cannot be certain if some physicians are not captured as we did not obtain all physicians in the province, only those that prescribed 1+ antibiotics in the time period. We did use CPSO numbers as the link between databases. Our linkage success rate was 97%. We clarified in line 168 that we used CPSO numbers: “The Xponent database had 25,678 unique physicians and 24,878 (97%) were successfully linked to ODB at ICES using College of Physicians and Surgeons identifiers.”

3. The authors performed multiple separate analyses for female vs. male genders and demonstrated that there were was a slightly greater relative difference in prescribing for male patients. Nevertheless, the discussions section lacked explanations on the potential reasons and significance of this finding. This should be further elaborated on.

Response: Thank you for this comment. For the most part we found the analysis for male and female patients to be similar. However there did appear to be a difference in validity between male and female patients in rural areas. Reasons for this are unclear and we have added this line to the discussion (line 224): “This error was slightly more pronounced in female patients. Further exploration for the differences between male and female patients may be warranted.”

4. The antibiotic prescribing rate "antibiotic prescriptions/100 total prescriptions" is an interesting metric. It is conceivable that a clinician whom is prescribing antibiotics excessively would likely prescribe other drugs excessively as well thereby obscuring the increased antibiotic use. It was reassuring to see that "antibiotic prescriptions/100 total prescriptions" correlated well with "antibiotic prescriptions / 100 patient visits" given the denominator for antibiotic utilization is typically patient visits to take into account differences in patient volume. It is not clear to me where the patient visit denominator is coming from (is it from OBD or an alternate source) and could these not just be incorporated into the numerator generated from Xponent rather than relying on the "antibiotic prescriptions/100 total prescriptions".

Response: Outpatient visits were obtained from the OHIP database. We have clarified this in the methods line 119. To do this comparison we linked IQVIA data to ICES. We feel it adds value to leave the prescriptions per 100 total prescription variable as some jurisdictions may not have the capacity to link to administrative data, and even for those that do it adds a layer of complexity. We completely agree with your analysis of the potential limitations of this variable and the implications for it from our study.

Reviewer #2: This topic is of great interest to likely a small group of individuals. IQVIA formerly Brogan, formerly IMS has recorded data on antimicrobial and other drug utilization from retail pharmacies for many years. The validity of the methods to extrapolate from the sample of pharmacies to the whole population has never been public and therefore the science community has been left to wonder how well their proprietary methods work and can they be trusted for research. In addition to the cited work of Tan (reference 42), two other groups in Canada have completed similar work in conference papers and could be cited (Dalton, B; Sabuda D, Bresee L et al. External Validation of Estimates of Antibacterial Dispensing in the IMS Brogan Xponent® Database in a Canadian Province. IDWeek 2013 https://idsa.confex.com/idsa/2013/webprogram/Paper41244.html) & Chong M, Dutil L, Bhatia T, Marra F, Patrick DM. Assessing antimicrobial consumption

using two different methodologies in British Columbia. Can J Infect Dis Med Microbiol

2007; 18(1): 35 Abstract A3.

Response: References added to background

In general, the authors have conducted a careful analysis using appropriate methods, and this manuscript is likely deserving of publication. I have a few suggestions to help make small improvements in clarity.

The goal of this study was to reliably predict the top 25 percentile of prescribers of antimicrobials to patients 65 and older, by identifying them in a public database of all pharmacies and comparing to the IQVIA database. I think the authors could do a better job describing this objective in the title as one does not reflexively think that quantification and validation would be performed at a physician level. I think some discussion of how these data would relate to the exponent database derived data on population level antimicrobial utilization would be appropriate (or if not at all, please state) and the conclusion "that the xponent database is validated for patients 65 and older " is actually inaccurate considering the principle finding of validating the identification of high prescibers of antibiotics in a population of 65 and older. "

Response: This point is well taken and correct. We have modified the title to: “Validating a Popular Outpatient Antibiotic Database to Reliably Identify High Prescribing Physicians”. We also modified the language in the abstract, objectives, and conclusions to more clearly reflect this (Lines 25, 43, 75, 264)

The definition of physician antibiotic prescribing rate is unclear. Is this based on number of prescriptions, days of therapy or DDD etc.

Response: We have clarified this in the methods line 99: “number of antibiotic prescriptions/number of total prescriptions by that physician”

Physicians were excluded for "prescribing less than 9 antibiotics". Does this mean <9 antibiotic prescriptions over the time period? Can you explain this number?

Response: Correct. We have clarified this in line 122 in the methods: “We excluded physicians who prescribed <9 antibiotics to patients ≥65 years of age during the study year in either Xponent or ODB databases. We used this cut off, which removes 5% of all antibiotic prescriptions, in order to exclude infrequent antibiotic prescribers.”

Could you define "antibiotics" in the study methods better? eg "systemic antibacterials" rather than just referring reader to the supplement.

Response: Modified to (line 96): “Antibiotics were defined as WHO ATC J01 class antibacterials for systemic use limited to those taken orally and dispensed from an outpatient pharmacy (S1 Table).”

The methods and results for agreement are confusing. In the methods it is stated the agreement of quartile groupings was evaluated, so one expects results in terms of categorical analysis. However in the text of results and table 3 mean number of of antibiotics prescriptions per physicians is reported.

Response: We have clarified in the methods what was done (line 142). We feel using the mean numbers helps with the clinical meaning of the kappas which are sometimes hard to interpret: “We descriptively compared the mean number of antibiotics prescribed per physician. Then we used the quartiles and evaluated agreement of the quartile groupings using weighted kappa statistics with 95% CIs for total antibiotics as well as subclasses of antibiotics.”

The agreement on antibiotic prescribing per 100 prescriptions was assessed by Bland Altman plots. I am unclear of the relevance of antibiotic prescription per 100 prescriptions. This should be discussed in background

and discussion.

Response: We have clarified in the methods line 156: “IQVIA data does not contain numbers of patient visits. Therefore, we also assessed the correlation of the antibiotic prescribing rate (number of antibiotic prescriptions per 100 of total prescriptions) to number of antibiotic prescriptions per 100 patient visits using Spearman correlation coefficients.” We have also modified line 242 in the discussion to state: “We demonstrated that the number of antibiotic prescriptions per 100 total prescriptions is imperfect, but is highly correlated with the number of antibiotic prescriptions per 100 patient visits for primary care physicians in patients 65 years of age or older. This is a potentially useful metric from pharmacy data to indirectly account for differences in patient volume, when patient visits are unavailable.”

If there are a significant number of non physician prescribers in Ontario, are they captured in ODB and exponent? Even if their prescribing rates are lower than that of physicians, it would be useful to note if agreement is similar with non physicians.

Response: Non-physician prescribers are not captured in either database. We clarified this in the methods (lines 93 and 113) and added this to the limitations line 261. We do not have any data source in Ontario currently that captures non-physicians prescribers.

In figures 5 and S3 one can observe correlation but there appears to be bias with discounting of the xponent values. This is not commented upon. Can the slope of the regression line be provided? It would also provide easier interpretation if the x and y axes were given the same scales and number values.

Response: Figures modified as suggested

27 Aug 2019

[EXSCINDED]

PONE-D-19-17515R1

Validating a Popular Outpatient Antibiotic Database to Reliably Identify High Prescribing Physicians

PLOS ONE

Dear Dr. Schwartz,

Thank you for submitting your manuscript to PLOS ONE and for addressing  the peer reviewer and editorial comments. However, there remain some additional issues which need to be addressed. Consequently, the manuscript does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

Thank you for the title change as per the reviewer comments - please add " for patients 65 years of age and older" as well to make this point even more explicit. Also mention this point in the sentence in the first paragraph of the Discussion.

You have added the additional databases available in the Introduction as requested  but the references do not  reflect the actual databases - please apply more direct references and limit the your selection of self citations which are both  indirect and excessive.

There are an excessive number of references cited in the Introduction about the point of both the CDC and Health  Canada  using the IQVIA data ie 17-39 and not all are necessary - please reduce the number by at least 50%.

The sentence containing "  and the validity of IQVIA data has been assessed in some jurisdictions. [15] [16]" provides examples from within Canada and for broader populations than 65 and older and deserves to have  this point added. Alternatively it could be added to the  second paragraph of the Discussion when you speak about the use of this data in the US and Canada and add it as part of the academic discussion on the use of these databases.

There is a stray legend in italics on lines 204-205 just near the end of the Results - please correct this.

Despite an explicit request, the Reference section continues to have multiple errors - incorrect format not in the standard style for date ( sometimes at the front of the reference in brackets and other times elsewhere and ), non use if urls where they should be present,  case errors everywhere and non use of italics for Latin terms etc.  - again too numerous to count. . Please correct  (do not rely on Mendeley which does not pick up all the formatting issues) or the manuscript will be returned again .

==============================

We would appreciate receiving your revised manuscript by Sept 16 2019 . When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

John Conly, MD

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

27 Aug 2019

August 27 2019,

Dear Dr. Conly,

Thank you for these comments. Each addressed below.

Sincerely,

Dr. Kevin Schwartz

On behalf of the co-authors

Comments:

• Thank you for the title change as per the reviewer comments - please add " for patients 65 years of age and older" as well to make this point even more explicit. Also mention this point in the sentence in the first paragraph of the Discussion.

Response: Added suggested line to title and line 215 in the discussion

• You have added the additional databases available in the Introduction as requested but the references do not reflect the actual databases - please apply more direct references and limit the your selection of self citations which are both indirect and excessive.

Response: The references were selected as examples for the aforementioned databases. Reference 7 used IQVIAs GPM database; We have added reference 8, which used CompuScript; reference 9 is an example of the ODB database; reference 10 used IQVIAs Xponent database; reference 11 used EMRALD (an administrative electronic medical record database). We have removed one of the EMRALD references.

• There are an excessive number of references cited in the Introduction about the point of both the CDC and Health Canada using the IQVIA data ie 17-39 and not all are necessary - please reduce the number by at least 50%.

Response: Reduced as requested

• The sentence containing " and the validity of IQVIA data has been assessed in some jurisdictions. [15] [16]" provides examples from within Canada and for broader populations than 65 and older and deserves to have this point added. Alternatively it could be added to the second paragraph of the Discussion when you speak about the use of this data in the US and Canada and add it as part of the academic discussion on the use of these databases.

Response: Moved this point and references to Discussion paragraph 2, line 225 as suggested

• There is a stray legend in italics on lines 204-205 just near the end of the Results - please correct this.

Response: subheading removed as requested

• Despite an explicit request, the Reference section continues to have multiple errors - incorrect format not in the standard style for date ( sometimes at the front of the reference in brackets and other times elsewhere and ), non use if urls where they should be present, case errors everywhere and non use of italics for Latin terms etc. - again too numerous to count. . Please correct (do not rely on Mendeley which does not pick up all the formatting issues) or the manuscript will be returned again .

Response: References edited to specifications

6 Sep 2019

[EXSCINDED]

PONE-D-19-17515R2

Validating a Popular Outpatient Antibiotic Database to Reliably Identify High Prescribing Physicians for Patients 65 Years of Age and Older

PLOS ONE

Dear Dr. Schwartz,

Thank you for submitting your re-revised  manuscript to PLOS ONE and addressing most of the requested changes . There remain a couple of outstanding items that must be addressed.  Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR:

1. The sentence containing "and the validity of IQVIA data has been assessed in other  jurisdictions...…" should include the fact that  these assessments were done within Canada in patients of all ages....  which had been previously requested but  was not added  .Its placement in the Discussion is fine .

2. The references remain incomplete with several case and alignment errors. Please correct them as previously  requested..

We would appreciate receiving your revised manuscript by Oct 21 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

John Conly, MD

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

11 Sep 2019

1. The sentence containing "and the validity of IQVIA data has been assessed in other jurisdictions...…" should include the fact that these assessments were done within Canada in patients of all ages.... which had been previously requested but was not added .Its placement in the Discussion is fine .

Response: Sentence modified as requested. To better reflect the references; it now reads: “Abstracts validating IQVIA antibiotic databases in other Canadian jurisdictions, performed with patients of all ages, have been previously presented.[32,33]”

2. The references remain incomplete with several case and alignment errors. Please correct them as previously requested..

Response: Two people have independently reviewed the references for errors and they have been corrected

16 Sep 2019

Validating a Popular Outpatient Antibiotic Database to Reliably Identify High Prescribing Physicians for Patients 65 Years of Age and Older

PONE-D-19-17515R3

Dear Dr. Schwartz,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

John Conly, MD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

18 Sep 2019

PONE-D-19-17515R3

Validating a Popular Outpatient Antibiotic Database to Reliably Identify High Prescribing Physicians for Patients 65 Years of Age and Older

Dear Dr. Schwartz:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr John Conly

Academic Editor

PLOS ONE