SOBRINA Spanish study-analysing the frequency, cost and adverse events associated with overuse in primary care: protocol for a retrospective cohort study.

INTRODUCTION: Several institutions and quality national agencies have fostered the creation of recommendations on what not to do to reduce overuse in clinical practice. In primary care, their impact has hardly been studied. The frequency of adverse events (AEs) associated with doing what must not be done has not been analysed, either. The aim of this study is to measure the frequency of overuse and AEs associated with doing what must not be done (commission errors) in primary care and their cost. METHODS AND ANALYSIS: A coordinated, multicentric, national project. A retrospective cohort study using computerised databases of primary care medical records from national agencies and regional health services will be conducted to analyse the frequency of the overuse due to ignore the do-not-do recommendations, and immediately afterwards, depending on their frequency, a representative random sample of medical records will be reviewed with algorithms (triggers) that determine the frequency of AEs associated with these recommendations. Cost will determine by summation of the direct costs due to the consultation, pharmacy, laboratory and imaging activities according to the cases. ETHICS AND DISSEMINATION: The study protocol has been approved by the Ethics Committee of Primary Care Research of the Valencian Community. We aim to disseminate the findings through international peer-reviewed journals and on the website (http://www.nohacer.es/). Outcomes will be used to incorporate algorithms into the electronic history to assist in making clinical decisions. TRIAL REGISTRATION NUMBER: NCT03482232; Pre-results. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is an overuse study conducted in the primary care setting in a country with a national health system.

This study develops a methodology for measuring the harm caused by ineffective practices.

We use data from real clinical encounters.

This approach identifies a part of the overuse related to safety and sustainability; however, the do-not-do recommendations (DNDs) may not always be appropriate for all cases of overuse.

The magnitude of harm dues to overuse by ignoring DNDs could vary from mild to serious, introducing a detection bias.

Introduction

Overdiagnosis, overuse, overmedicalisation and low-value care are terms that are being used to represent the provision of medical services for which the potential for harm exceeds the potential for benefit.1 This overuse of diagnostic and therapeutic resources poses problems for the health of individuals and for health systems everywhere.2–4 No medical specialty, not even primary care,5–7 is immune from this overuse. According to the sources of information used (claims by patients; clinical histories; and surveys with professionals, patients and health managers), the extra costs from such overuse vary between 10% and 30% of the total health expenditures in highly developed countries.8 9

The immediate causes of this overuse include10–14 insufficient updating of knowledge by professionals, decisions based on the fear of facing future litigation (defensive medicine), the custom of continuing to do things that they have been doing until now, lack of time during consultation, inadequate incentives, influence by the pharmaceutical industry, inadequate communication with patients and pressure from patients who request diagnostic tests or treatments based on inadequate sources of information such as the internet.

Do-not-do recommendations

Sometimes, the determination of overuse is simple, but in several services, the cases of overused are not clear. In recent years, the so-called ‘Less is More Medicine’ movement has spread owing to the actions of institutions and scientific societies.15 16 This movement has sought to identify ‘what must not be done’17 and it has led to informative and educational campaigns, including, for example, the ‘Choosing Wisely’ campaign,8 18 which is intended to raise awareness about the need to reduce requests for tests or for carrying out medical procedures thought to be unnecessary.

In 2007, the National Institute for Health and Care Excellence (NICE) created a database of a series of recommendations on relatively frequent practices that are considered inadequate based on sufficient evidence. These recommendations then became part of a ‘do-not-do’ list. Currently, NICE19 lists about 850 practices with scientific evidence about their low effectiveness. In 2009, the American Board of Internal Medicine organised the ‘Choosing Wisely’ campaign,18 which focused on avoiding the inappropriate use of diagnostic and therapeutic resources, directed towards both clinicians and patients. Its recommendations included 70 practices chosen from about 400 practices that were not shown to benefit certain profiles of patients. The ‘Do No Harm Project’ in Colorado (USA),20 ‘Slow Medicine’ in Italy,21 ‘Smarter Medicine’ in Switzerland22 and ‘Commitment to Quality by Scientific Societies’ by the Spanish Ministry of Health in collaboration with 39 scientific societies23 are other examples of this commitment to attain better healthcare, intending to banish practices with poor or doubtful effectiveness.

Since 2010, the Archives of Internal Medicine journal (renamed JAMA Internal Medicine in 2013) of the American Medical Association has been publishing research under the title ‘Less is More’.24 This section of the journal includes studies on practices that either provide no benefits to patients or entail elevated risks without any offsetting sufficient benefit, with the goal of eradicating such practices. The British Medical Journal heads the ‘Too Much Medicine’25 campaign with the goal of drawing attention to problems caused by overdiagnosing and wasting health resources. Lancet edits the series called ‘Right Care’,26 which includes studies on the frequency, causes, consequences and measures for reducing overuse.

Up to now, the results from these campaigns have hardly been conclusive. In the USA, 3 years after publication of the do-not-do recommendations (DNDs), Rosenberg et al9 found that the frequency of imaging for headaches and imaging techniques in cardiology had decreased, but that the use of nonsteroidal anti-inflammatories in patients with hypertension, heart failure or chronic kidney disease, in addition to the human papillomavirus test in women below 30 years of age, had increased. The usage frequency for antibiotics for sinusitis, as well as that for taking X-rays for low back pain and prior to chest surgeries, remained stable. Those authors concluded that greater efforts were necessary to achieve more widespread implementation of these recommendations given that, however successful that they were, hardly any change had been achieved.

When doing what must not be done causes harm

Overuse is related with low-value care, and the worst thing about it lies in obtaining a negative result when something that should not be done is done. One example of this is prescribing an unnecessary test or procedure included on the DNDs, which causes the patient harm (adverse event, AE). Although the frequency of doing what must not be done has been analysed in some studies outside Europe,3 5 6 12 14 the occurrence of AEs related with this overuse has not been studied systematically until now.

Overuse in primary care

In the context of primary care, overuse has been studied much less. In 2011, the National Physicians Alliance carried out the ‘Promoting Good Stewardship in Clinical Practice’27 project, whose objective was to identify a list of top-5 practices that it would recommend to be eliminated. In Spain in 2014, the Spanish Society of Family and Community Medicine produced, by consensus, an initial list of 15 DNDs in daily clinical practice, 10 of which were related to treatments and five to diagnostic tests.28 The Spanish Paediatric Association prepared five DNDs with applications in primary care.29 Despite these efforts, however, the frequency with which these practices considered inappropriate persist in primary care clinics has not been determined. Furthermore, neither have their economic consequences nor the harm caused to patients have been examined.

Study objectives

Primary objective

To estimate the frequency of AEs directly related with diagnostic and/or therapeutic indications to the patient for which consensus exists based on scientific evidence or agreement among experts (DNDs for primary care) that they should not have been indicated (commission errors) and their cost.

Secondary objectives

To measure the frequency of overuse and determine the extra costs incurred by indicating diagnostic tests and procedures and/or therapeutic included in the DNDs for primary care.

Methods and analysis

This coordinated, multicentric and national research project is being conducted in the primary care setting in Spain in collaboration with investigators from Andalusia, Aragón, Castilla La Mancha, Catalonia, Valencian Community, Madrid, Murcia, Navarre and the Basque Country. The research includes the steps outlined in figure 1. This project was approved by the National Research Agency in Spain, in November 2016. The research team started work in March 2017 and aim to complete it by December 2019.

Figure 1

Steps in the design, review and approval of this study.

A retrospective cohort study, an analysis of the frequency and cost of doing what must not be done, in addition to an analysis of the frequency and cost of AEs associated with these practices will be conducted.

Definitions

AE is unintentional harm the patient suffers throughout the course of the healthcare that is related to the care received and not the patient’s underlying illness.30 31

Overuse is the utilisation of tests, procedures or treatments in which the potential harm exceeds the possible benefits.1 In this study, continuing to do what must not be done (ignoring DNDs) was used as a measure of overuse.

DNDs are practices included on the lists of consensus-based recommendations from governmental institutions or scientific societies to avoid inappropriate overuse, and it forms part of the ‘Less is More Medicine’ movement (in Spain, scientific societies call this the ‘Commitment to Quality’).32

Patient safety and overuse

Aim

To identify the frequency and severity of AEs related with the frequency of overuse related to DNDs in primary care, and the unnecessary cost incurred from such overuse. Primary care in Spain includes the practices of general practitioners and paediatricians.

Information sources

Computerised databases of primary care medical records from national agencies and regional health services, electronic clinical history, prescription records, cost information of Spanish hospitals and the primary care system, and agreements on DNDs for primary care.

Data collection

To determine the overuse related to DNDs, seven DNDs from medicine and three from paediatrics, that fulfil the criteria of presenting relatively frequently in practice, can be reliably identified and can also be the cause of an AE, have been chosen based on the prioritisation recommended in a previous study using the Delphi technique with 100 healthcare professionals (box 1).33

Prescribe benzodiazepines to treat insomnia, agitation or delirium in people older than 65 years.

Prescribe non-steroidal anti-inflammatory drugs for patients with cardiovascular disease, chronic kidney disease, hypertension, heart failure or liver cirrhosis.

Recommend taking a dose of 1 g of paracetamol for more than 3 days.

Order antibiotic treatment for acute bronchitis when the patient does not suffer from chronic obstructive pulmonary disease, heart failure, diabetes or kidney disease, or undergoing active chemotherapy.

Prescribe lipid-lowering drugs in patients older than 75 years without previous cardiovascular events.

Perform prostate cancer screening in asymptomatic patients.

Order image tests in nonspecific lumbago.

Prescribe antibiotics for infants with pharyngitis.

Prescribe mucolytic drugs, antitussives or antibiotics for upper respiratory infections in infants.

Combine treatment between ibuprofen and paracetamol in infants.

A two-stage methodology will be used to identify patients who suffered avoidable AEs related to DNDs. Figure 2 illustrates the pathway through the SOBRINA (Grupo de Investigación en SOBReutilización INnecesariA; in English, "Research Group on Overuse") study. This study assumes that there is a higher number of AEs when healthcare professionals ignore the DNDs.

Figure 2

Flow diagram for the SOBRINA study. AEs, adverse events.

In tier 1, the frequency of procedures or treatments related to DNDs that persist in the daily practice in primary care over the last 3 years will be determined with algorithms similar to the indicators of overuse proposed by Segal et al.8 These indicators will be applied to determine the frequency of the DNDs listed in box 1, from available electronic information systems (electronic history, prescription, etc).

A population database cohort of approximately 8 294 900 of individuals attended by the public health system (data from national agencies and nine Spanish regional health services) will be used to determine the frequency of procedures or treatments related to DNDs during their use of primary care service (figure 3). The frequency of DNDs will estimate considering as numerator the total number of cases with overuse due to ignoring DNDs and denominator the total number of patients under a specific condition (see supplementary material). These figures will guide the calculation of the extra cost that they represent for the entire health system.

Figure 3

Overview of methods, data sources, sampling and measures of SOBRINA study. DNDs, do-not-do recommendations; AEs, adverse events.

In tier 2, we will conduct electronic searches of clinical computer systems to identify patients with avoidable AEs (figure 3). Their frequency will be calculated as the number of patients with at least one contact or episode of event or circumstance that resulted in unintentional harm, per 1000 listed patients. The frequencies calculated in the first phase (tier 1) will determine the minimum number of cases to be included in the study to determine potential cases of AE associated with the overuse. In the study conducted by Rosenberg et al,9 an assessment of seven low-value services revealed that the frequency of overuse due to ignoring DNDs ranged from 8% to 85% depending on each service. Preliminary data on the overuse of services listed in box 1, obtained using the algorithms and procedures designed for this study, yield a 11.3% rate of inadequate prostate cancer screening (one case of inadequate decision per nine patients) or 69.4‰ of inadequate prescriptions of benzodiazepines to treat insomnia, agitation or delirium in elderly persons (one case of inadequate prescription per 14 patients). The estimated samples in each case ranged from 2110 to 5296, depending on the statistical power applied (table 1).

Table 1

Number of clinical histories to be reviewed considering different levels of statistical power

Statistical power (%)	N*	N†
80	2110	3248
85	2400	3696
90	2800	4304
95	3450	5296

*Sample estimates for the inadequate screening of prostate cancer.

†Sample estimates for the inadequate prescriptions of benzodiazepines.

A random sample chosen from clinical histories of patients attended over the past 5 years by professionals at care centres where medical or nursing attention is provided will be used to calculate the frequency of AEs for each DND. This period was defined after conducting a preliminary trial for extracting data to assure the feasibility of the procedure and to obtain reliable data. The care centres participating in this study belong to nine out of the 17 currently operating regional health services in Spain. In this sample of cases, the potential cases of AEs associated with the DNDs will be identified first using the trigger tool and reviewing the primary care centre’s and hospital’s medical records. Medical charts will be retrieved and categorised into the following three groups: no AE, AE not likely and AE likely. Once these cases are classified, health personnel from the centre where the patient was attended will review them in depth to confirm whether an AE occurred, will rate its severity and will identify its relationship with ignoring DNDs. The APEAS (Spanish National Study of Adverse Events in Primary Care)34 questionnaire and taxonomy will be used for this. This questionnaire was used in the Spanish national study to identify frequency, nature, causes and consequences of AEs in primary care. This instrument was designed from the questionnaire used in the Harvard Medical Practice Study35 36 in its project on patient safety, and it was adapted following the experience from the ENEAS (Spanish National Study of Adverse Events in Public Hospitals)37 study also conducted in Spain to determine the frequency of AEs in hospitals. The APEAS questionnaire is complemented by a computer application for managing the data called System for Monitoring and Controlling AE in Primary Care (the SIVCEA AP 1.0 database).38 The reviewers will complete the form each time a case with an AE is identified. They will be classified into the following categories: events related to the processes of healthcare (including those due to ignoring the DNDs), communication gaps, medication, treatment (non-medication) and related to the skills of health professionals. The harm severity of each AE will be assessed on a 1–8 scale. This scale has been adapted from the harm scale used previously by Woods et al39 (box 2). The information will be introduced into the SIVCEA (Sistema de Información para la Vigilancia y Control de Efectos Adversos; in English, “Information System for the Surveillance and Control of Adverse Effects”) computer application and the forms will be safeguarded for an eventual quality review, ensuring confidentiality by a blind registration system developed for this study. Reviewers will assess if the AE is related to the DNDs using a scale from 0 (no evidence) to 7 (virtually certain). Scores higher or equal to 4 are deemed to indicate a positive relation between the AE and ignoring the DND. Reviewers of the forms from each centre will be trained to correctly apply this protocol. A degree of agreement is expected between reviewers after conducting this training which included example cases. The inter-rater agreement, calculated by Kappa coefficient, is expected to be greater than 0.80. Moreover, this information will permit the estimation of the cost involved in repairing the harm associated with the DNDs by summing of the direct costs due to the consultation, pharmacy, laboratory and imaging activities according to the cases. Figure 3 illustrates methods, data sources, sample and expected outcomes of this study.

No harm.

Emotional harm.

Insignificant harm.

Minor temporary harm.

Major temporary harm.

Minor permanent harm.

Major permanent harm.

Death.

Study variables

Frequency of overuse related to DNDs, cost of inappropriate overuse, frequency, and severity and cost of AEs related to DNDs.

Data analysis

The characteristics of the case of overuse and the trigger tool will be described by using absolute frequencies and relative frequencies (percentages) for qualitative variables. The quantitative variables will be expressed using means and SDs (in the case of a normal distribution) or by using the medians and interquartile ranges (in the case of a non-normal distribution). Cumulative rates of AEs will be calculated after complete review of the patient’s admission, with a 95% CI, and the association between the outcome variables (presence/absence of AEs) and the remaining variables collected will be measured via a bivariate analysis.

For the bivariate analysis, either the χ2 test or Fisher’s exact test will be used for the qualitative variables, and either Student’s t-test or the Mann-Whitney U Test will be used for the quantitative variables (according to whether or not they fulfil the normality criteria). Additionally, a variance analysis will be conducted to compare various means. Statistical analysis of the sample size effect will be performed. To control the confusion and/or interaction of the explanatory variables related to the severity and preventability of AE, a logistic regression will be used. It will be conducted using the stepwise forward selection method by likelihood ratio. Statistical significance will be set at p<0.05 for all the tests used, and the analyses will be conducted using Statistical Package for the Social Sciences (SPSS) V. 24.0.

Limitations and potential sources of bias

This study aims to identify part of the overuse related to safety and sustainability and does not explore the causes of overuse related to these DNDs. DNDs may not always be clear-cut cases of overuse. The design of this study does not permit stablish that doing what must not be done by general practitioners or paediatricians is a direct cause for AE’s occurrence. Our analysis is based on patients’ clinical records; however, the quality of the registered information can vary among physicians. The magnitude of harm among DNDs could vary from mild to serious, thus introducing a detection bias. Lastly, this study does not examine patient-level safety risk differences and potential regional variations.

Numerators

Overuse in cases determined as stemming from ignoring DNDs will be considered (see online supplementary material). These frequency data will be divided based on age groups (1–3, 4–9, 10–14, 16-44, 45-55, 56–64, 65–75 and >75 years), gender, professional profile, type of health centre (urban or rural) and year. Possible factors related with the frequency of overuse due to ignoring DNDs (dependent variable) will be identified with a logistic regression analysis.

PATIENT and public involvement

Patients and or public are not involved.

Ethics

The electronic information available will be reviewed in a retrospective manner, taking the case volume and study design into account; exemption of informed consent from patients has been authorised. Access to information will be granted in accordance with the request procedures, treatment and transfer of health data from the information services of the health services participating in this study. In no case will the project investigators gain access to the clinical information of patients.

To ensure confidentiality of the study data, the only persons or groups with access to it will be the principal investigator (PI) and his/her team of collaborators (from the work centre), the promoter or person that the PI designates, the ethics committees of the investigation for primary care, the relevant health authorities and those responsible for the analysis of the said data.

The content of the documents generated throughout the study (questionnaire for identifying AEs according to the APEAS study) will be safeguarded from unauthorised use by individuals external to the investigation and, therefore, will be considered strictly confidential and will not be revealed to third parties. The PI will ensure anonymity of the patients, as well as the protection of their identities from unauthorised persons. Patients will be identified by a code and not by their given name. The PI will be responsible for safeguarding the study documentation until it is finished.

The project has been drafted in a manner that ensures respect for the principles contained in the Declaration of Helsinki, the Council of Europe Convention (Oviedo), and the UNESCO Universal Declaration. No ethical aspects of the investigation contrast the rights of Spanish patients contained in Law 41/2002. In this study, regulations and ethical commitments with respect to the analysis of data that correspond to minors will be observed carefully. None of the participating investigators have commitments with industrial or commercial sectors that could interfere with the project. Data codification will be based on a masking system that respects the confidentiality of the professionals. Access to clinical data of the electronic histories will only be gained by professionals from the corresponding health centres that are authorised for such access, and no identification data of the patients or the professional/s involved in the care will be codified.

Dissemination

First, the documentation of the impact of ignoring DNDs in the present study will be doing include in residency training programmes for general and community practitioners, paediatricians and nursing training.

Second, the findings will be used to develop a guide that demonstrates the impact of ignoring DNDs and that proposes changes in the algorithms to assist in clinical decisions-making that can be incorporated into the electronic history based on the results obtained. In this manner, primary care physicians could rely on assistance when prescribing and indicating tests and procedures adjusted to the results of this investigation. This guide will be available on the website (http://www.nohacer.es/). Third, this study will extract results and lessons from the study that must be incorporated into Spain’s National Strategy on Patient Safety.

Moreover, direct dissemination of the results is planned via the project’s website, social networks, mass media, scientific societies, as well as in specialised journals and at national and international scientific workshops.

Discussion

The literature differentiates between underuse, misuse and overuse.40 Until now, most research has focused on the first two because they have been easier to address.

Inappropriate overuse is present at all care levels, all specialties and all health systems.3 This justifies the recent increase in research on this issue,41 despite the fact that the frequency data on the frequency of this overuse and its impact have not been conclusive due to different methodologies employed to gather them.2

Consensus-based techniques have brought advances in producing lists with recommendations about what not to do, an approach that has permitted overcoming difficulties when defining the overuse for certain medical procedures and tests. There are fewer studies in primary care, but epidemiological studies do exist that include examples of overuse.27 28

Until now, most published research has been conducted in the USA. As such, for example, bronchodilator overuse has been estimated to be between 3% and 81%, Prostate-Specific Antigen test overuse from 16% to 80%, overuse of antibiotics for upper respiratory tract infections from 2% to 89% and overuse of tumour markers from 7% to 80%.6 Based on claims by Medicare patients, Segal et al8 estimated annual extra costs for unnecessary laminectomies of $796 million, $686 million for unnecessary contrast abdominal CT scans and $516 million for conducting positron emission tomography (PET)/TC scans on patients with low risk for prostate cancer. In Spain, more than 20% of all knee replacements are inappropriate,42 and it has been calculated that between 20% and 50% of imaging tests could have been avoided; a significant portion of this percentage was due to requests by the patients themselves or their relatives. The role of the patient in overuse has hardly been examined until now.

In several countries, frequency estimations have been followed up by information campaigns43–49 targeting clinicians and, in some cases, patients, to reduce this overuse. Despite their modest results, they seem to point in the right direction although greater engagement by all stakeholders involved is necessary to achieve better results.50

In DNDs, which have spread to all countries,51 the study on inappropriate overuse has a new approach based on practices for which broad consensus exists that they are inadequate or even harmful to patients. These approaches are also gradually spreading in all health systems.

The present protocol draws from these considerations and exploits the consensus reached in Spain on certain DNDs in primary care to establish the frequency of overuse of some of these practices that are unnecessary or harmful to patients. Furthermore, it also aims to estimate their costs for the health system. This is the first study with these characteristics to be carried out in Spain, a country with a public national health system, where the primary care physician (or the paediatrician in case of children) is the gatekeeper for the entire system. However, in the case of primary care, and as far as we know, inappropriate overuse has not been studied systematically, and this opens opportunities for new research in other countries.

However, this study also argues that doing what must not be done can occasionally harm the patient (causing AEs). This aspect has been analysed very little until now,2 and it is one of the classic problems in patient safety when focusing on clearly avoidable AEs.52 This low-value care represents an unnecessary risk for the patients that health systems must eradicate.

This project contributes to improving patient safety in primary care, calling attention to risks that patients are subjected to by clinical practices for which ample consensus and evidence exist that they should not occur. Furthermore, the project provides a measure of the economic impact of not following certain DNDs in primary care in the Spanish health system.

The project develops a methodology for analysing this phenomenon that could then be employed in other countries to carry out studies on the frequency of the low-value care associated with this overuse.

This study quantified the overuse in primary care and developed a methodology for measuring the harm caused by these ineffective practices that should have been eradicated. This was done to quantifying the magnitude of the problem of inappropriate overuse in primary care and to understand some of its causes, and to thus lay the basis for the development of actions aimed at mitigating this problem.