Understanding the epidemiology of avoidable significant harm in primary care: protocol for a retrospective cross-sectional study.

INTRODUCTION: Most patient safety research has focused on specialist-care settings where there is an appreciation of the frequency and causes of medical errors, and the resulting burden of adverse events. There have, however, been few large-scale robust studies that have investigated the extent and severity of avoidable harm in primary care. To address this, we will conduct a 12-month retrospective cross-sectional study involving case note review of primary care patients. METHODS AND ANALYSIS: We will conduct electronic searches of general practice (GP) clinical computer systems to identify patients with avoidable significant harm. Up to 16 general practices from 3 areas of England (East Midlands, London and the North West) will be recruited based on practice size, to obtain a sample of around 100 000 patients. Our investigations will include an 'enhanced sample' of patients with the highest risk of avoidable significant harm. We will estimate the incidence of avoidable significant harm and express this as 'per 100 000 patients per year'. Univariate and multivariate analysis will be conducted to identify the factors associated with avoidable significant harm. ETHICS/DISSEMINATION: The decision regarding participation by general practices in the study is entirely voluntary; the consent to participate may be withdrawn at any time. We will not seek individual patient consent for the retrospective case note review, but if patients respond to publicity about the project and say they do not wish their records to be included, we will follow these instructions. We will produce a report for the Department of Health's Policy Research Programme and several high-quality peer-reviewed publications in scientific journals. The study has been granted a favourable opinion by the East Midlands Nottingham 2 Research Ethics Committee (reference 15/EM/0411) and Confidentiality Advisory Group approval for access to medical records without consent under section 251 of the NHS Act 2006 (reference 15/CAG/0182). Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Having a random sample of practices stratified by list size will reduce bias.

The study will take place in England, which limits generalisability to other countries.

We may miss some patients with avoidable significant harm, although we will be able to estimate how frequently this occurs.

Bias may be introduced if more than a small minority of patients objects to a review of their records.

Training and ongoing feedback will help ensure that the general practices involved in data collection adhere to consistent methods for classifying patients.

Introduction

Background/rationale

Most patient safety research has focused on specialist-care settings where there is now considerable appreciation of the frequency and causes of medical errors, and the resulting burden of adverse events.1–4 Epidemiological studies have shown that 3.2–16.6% of all hospital patients experience an error,3 5–10 and many of these result in substantial avoidable patient morbidity and considerable costs to health systems. Unfortunately, the pace of patient safety research in primary care has in comparison with secondary care settings been slow,11 and only recently has the World Health Organization (WHO) focused on safer primary care.12

Members of our team recently completed a WHO-commissioned systematic review13 to determine the frequency, burden and preventability of patient safety incidents in primary care, including 9 systematic reviews and 100 primary studies. Their review highlighted that there have been few large-scale robust studies of the seriousness and avoidability of harm in primary care. Based on existing studies, errors occur in around 2–3% of primary care encounters, and around 1 in 25 of these errors results in serious harm that has a substantial impact on patients' well-being. An exception, that has informed the sample size calculations for this study, is an investigation of over 96 000 primary care patient encounters from Spain which showed that around 67 per 100 000 consultations (0.067%) were associated with serious avoidable harm.14

Even though the UK has been an exemplar of high-quality primary healthcare,15 it is important to develop an understanding of the frequency and causes of avoidable harm in this setting to reduce the risks. The current study has been designed to address this serious gap in the literature. Once we have a better understanding of the epidemiology of avoidable significant harm in primary care, we will have the necessary evidence to design and evaluate interventions that will reduce the burden of avoidable harm.

We will use retrospective case note review, which has been used to good effect in hospital settings to estimate the incidence of avoidable harm1 3 7 and deaths.16 We use the term ‘harm’ to refer to ‘an outcome that negatively affects a patient's health or quality of life’.17 This is sometimes also referred to as a ‘healthcare adverse event’.18 In our study, we will concentrate on the most ‘significant’ harms such as those causing loss of function of a body organ and/or hospital admission.

In keeping with previous studies, we will focus our investigations on an ‘enhanced sample’ of patients with the highest risk of avoidable significant harm,1 7 in order to make most efficient use of the time of reviewers involved in data collection. We will also use a well-established scale for assessing the avoidability of harm.14 16 Our definitions of significant harm and avoidability are shown in box 1.

Definitions of significant harm and avoidability Significant harm: A patient outcome is symptomatic, which required more intensive intervention than might otherwise have been required (eg, additional operative procedure) and resulted in an escalation of care (eg, hospital admission), or death. This caused a loss of function of at least one bodily organ, which may have been a temporary or permanent loss of its function. Avoidable: An error of omission (failing to do the right thing) or commission (doing something wrong) in healthcare management that reflects a failure to follow acceptable practice at an individual or system level

Objectives

The objectives of the study are to:

Estimate the incidence of avoidable significant harm in primary care in England,

Quantify, describe and classify the patient safety incidents that result in avoidable significant harm and their severity,

Identify ameliorable factors that, if addressed, could help reduce the incidence of avoidable significant harm in primary care, and

Based on the findings of the research (ie, objectives 1–3 above):

- Advise on the development of new measures at national and local levels, aimed at ensuring that the NHS Outcomes Framework includes primary care in relation to risks of avoidable harm.

- Advise on the extent to which future assessments of avoidable harm in primary care could be made more efficient through interrogation of electronic health records.

- Advise on interventions that might help reduce the incidence of avoidable harm.

Methods

Study design

We will undertake a retrospective cross-sectional study involving case note review of a cohort of primary care patients that attended the GP clinic within a 12-month window prior to the beginning of the study. We will conduct electronic searches of clinical computer systems in general practices (for 12 months from the start of the retrospective cohort) to identify patients at increased risk of avoidable significant harm.

Setting

We plan to recruit general practices from three geographically dispersed areas of England (East Midlands, London and the North West).

Eligibility of general practices for entering the study

Inclusion criteria

General practices will be eligible to participate if:

They provide written informed consent.

They deliver NHS services.

They have electronic health records (this is the case for almost all general practices in England) and use one of the three main GP computer systems in England: EMIS Web (EMIS Health 2016), SystmOne (The Phoenix Partnership 2016) or INPS Vision (In Practice Systems, 2016).

Their electronic health records can reliably identify patients who have been admitted to secondary care.

Exclusion criteria

General practices will be excluded from the study if they are involved in a major reorganisation (such as a merger with another practice) since this would make it difficult to identify the practice list size for the retrospective case note review.

Recruitment of general practices and data collectors

We will use a stratified random sampling technique to approach the general practices. We plan to recruit up to 16 general practices stratified (in quartiles) by list size. Up to 18 independent general practitioners will be recruited and trained to undertake data collection in the general practices.

We will email and/or write to general practices (via the practice manager and general practitioners within the practices) inviting participation. We will use a range of approaches to encourage participation, including prior publicity about the study, engaging local opinion leaders and providing reassurance about data confidentiality. We will encourage retention of the general practices in the study by ensuring that the data collection procedures are not disruptive to the workings of the practice, and that the GP reviewers involved in data collection build good working relationships with the practices. Practices will be reimbursed for their time, and GP reviewers will be paid for their work on the project.

Participants

All patients in these practices will be considered potentially eligible for inclusion in the study, although we will identify an ‘enhanced sample’ in each practice of patients who are likely to have higher than average risk of avoidable significant harm (see section below on sampling for more details).

Patients will be excluded if they have a computer code in their clinical records, indicating that they do not wish to be included in research studies. Patients will also be excluded if (as a result of publicity about the project) they state that they do not wish for their records to be included in the retrospective case note review.

Sampling of patient records

Our approach to sampling of patient records will be a three-stage process:

Stage 1: Identify total population of the practices at the start of the retrospective cohort

Stage 2: Identify patients at increased risk of avoidable harm (the ‘enhanced sample’)

Stage 3: Identify those from stage 2 who have experienced a significant new heath problem during the 12-month retrospective review period

The population for stage 1 will be those patients registered with the general practices at the start of the retrospective cohort (which will be 12 months before the date that sampling begins in each practice). We anticipate a total population of around 100 000 (based on 16 practices with a median list size of 6250). Figure 1 shows the stages in our study.

Figure 1

Flow chart showing how patient records are selected.

In order to identify patients at increased risk of avoidable harm (stage 2), we will draw on suggestions made by the commissioners of the research (the Department of Health), the literature on avoidable harm in primary care (see references in the bullet list below) and our own experience of inductively analysing reports of harm associated with primary care in the National Reporting and Learning System.19 Search strategies will be developed and tested for each of the aforementioned medical record systems. Searches of records at each practice (for 12 months from the start of the retrospective cohort) will identify patients who may be at increased risk of having experienced avoidable significant harm. We plan to include the following categories of patient:

Died16

Admitted to hospital or a mental health facility20

Certified unfit for work (long-term sick leave)

Resident in a care home21

Undergone an invasive procedure in general practice such as a minor operation or joint injection22

Major polypharmacy (10 or more repeat medications)23 24

Four or more major significant morbidities

In addition, we will include patients identified by the participating general practices as having experienced avoidable harm (eg, from significant event audits25). Based on pilot work, and running an algorithm inclusive of these categories in EMIS in an urban GP surgery with a practice list size of 13 000, we anticipate that this ‘at risk’ enhanced sample could constitute around 10% of the average general practice population, for example, 10 000 patients in total across all of the participating practices.

In each of the participating general practices, a spreadsheet will be generated listing those patients at increased risk of avoidable harm. The spreadsheet will contain patient demographic information (name, date of birth and NHS number) and a unique ‘Patient Study Number’ will be generated for each patient. The spreadsheet will be password-protected and stored on the GP practice's clinical system (with a back-up file elsewhere on the system). A hard copy will be printed out and used by the GP reviewers to identify patients in subsequent stages of the sampling. This hard copy will remain in the practice at all times.

The next stage of sampling (stage 3) will identify patients with significant health problems (irrespective of whether they are avoidable or not). It will involve one of the GP reviewers screening the electronic health records of patients to identify any new significant health problems, including accidents, experienced by patients over the 12 months of the study, irrespective of the cause (this will include all deaths). Our pilot work has shown that this is the most efficient way of identifying patients with potentially avoidable harm; ∼25% of the ‘Stage 2 sample’ are likely to be highlighted, that is, 2500 patients in total across all of the practices. The research team will provide the GP reviewers with comprehensive guidance on the health problems we want to screen for; this will include all new major physical and psychiatric morbidities, and accidents. Detailed record review will be performed on this final sample of patients to identify the extent to which failures in primary healthcare contribute to any of these significant health problems.

For a subsequent sensitivity analysis in which potentially avoidable harm that might have been ‘missed’ as a result of the above sampling process is accounted for in the estimation of the incidence of avoidable significant harm, the GP reviewers will screen the following:

2.5% random sample of ‘Stage 1’ patients not included in the ‘Stage 2’ sample (2250 patients in total across all of the practices);

10% random sample of the ‘Stage 2’ patients will be screened again by a second GP reviewer (around 1000 patients in total across all of the practices);

For any patients identified (by these two processes) as potentially having avoidable significant harm, the GP reviewers will undertake retrospective records review (as described below).

Undertaking the retrospective record reviews

In each practice, two of the GP reviewers will independently undertake a retrospective record review to identify cases of avoidable significant harm. Where potentially avoidable significant harm is detected, the retrospective case note reviews will go back to the root of the problem, for example, the failure to act on ‘red flag’ symptoms 5 years before a delayed diagnosis of inoperable cancer. To help in the identification of cases, the GP reviewers will be encouraged to identify ‘triggers’ indicating potential errors,26 for example, significantly out-of-range laboratory test results. In addition, the GP reviewers will have access to other primary healthcare professionals on our study team if they need advice on the avoidability of harm in primary care outside general practice. In keeping with a number of previous studies, the GP reviewers will judge the ‘avoidability’ of harm on a six-point scale (see table 1).14 16

Table 1

Six-point avoidability scale

Rating	Category	Description
1	Totally unavoidable	Virtually no evidence of avoidability
2	Unavoidable	Slight to modest evidence of avoidability
3	Possibly avoidable	Possibly avoidable, <50–50, but close call
4	Probably avoidable	Probably avoidable, more than 50–50, but close call
5	Probably avoidable	Strong evidence of avoidability
6	Totally avoidable	Virtually certain evidence of avoidability

Data collection

For those cases of harm considered to be at least ‘possibly avoidable’, one of the GP reviewers will record partially anonymised data onto a case report form to be retained in a study site file in the participating general practices. The case report forms will be stored in a secure location in the general practice, for example, locked cabinet or locked office and separate to the hard copy of the spreadsheet.

The GP reviewers will also record pseudonymised data relating to their retrospective case note reviews onto a specially designed data collection system on a tablet computer for synchronous transfer to a secure database server at Cardiff University (no information will be retained on the tablet computer itself). All devices will be encrypted, remotely managed and password-protected. The pseudonymised data (based on the spreadsheet generated for stage 2 of the sampling process) will consist of the unique ‘Patient Study Number’, age and sex of each patient, but no information that could allow the patient to be identified from outside the practice.

For stage 3, the GP reviewers will record whether they judge the patients to have experienced a significant health problem during the 12-month retrospective data collection period. Then, as part of the retrospective case note review, the reviewers will record their judgements of the avoidability of the health problems detected. For those patients judged to have experienced avoidable significant harm, the GP reviewers will record the nature of the avoidable harm using the comprehensive patient safety classification system developed in the PISA study.27 The multiaxial PISA classification system will permit reviewers to choose codes to describe the type of safety incident (eg, administration, medication), identify contributory factors (eg, patient comorbidity, staff work load) and report patient harm severity as described by the WHO International Classification for Patient Safety.27 The recursive model for incident analysis27 28 will be applied using a series of rules to enable consistent coding between reviewers, an applied example is provided in figure 2.

Figure 2

The recursive model for incident analysis. Illustrative case: a man aged 67 years presents to see a locum GP, with a 6-week history of fatigue, which he only mentions while leaving at the end of a consultation where four other problems were dealt with, including hypertension, psoriasis, knee pain and a medication review. As fatigue was presented right at the very end, the locum documents this and arranges for some blood tests and asks the patient to return in 2 weeks for review. No systems review or examination is documented. The patient has his bloods taken. The full blood count sample is reported as ‘not labelled’ and no results are given; the urea and electrolytes and liver function tests were all within the reference ranges. When another GP in the practice looks at the results—she marks them as normal. The patient calls for the results and to make an appointment, but as the bloods are reported as normal, he decides not to make one. Six months later, he presents back to his usual GP who notices he is very pale and has lost weight. Systems review identifies an 8-month history of loose motions, and examination reveals a large mass in his left iliac fossa. Haemoglobin is 70 g/L and ferritin is 3 µg/L. Further investigation reveals inoperable adenocarcinoma of the sigmoid colon.

Ensuring consistency of judgements regarding avoidable significant harm

A subgroup of the study team will meet regularly during the data collection period and will review all of the information provided by the GP reviewers. They will provide feedback to the reviewers on the quality of their reports, and consistency of their coding, in order to help ensure improvements where necessary. They will also discuss the judgements of the reviewers concerning the presence of avoidable significant harm and the classification of this. To ensure consistency, the study team will make the final judgement in terms of the classification of avoidable significant harm, particularly if the paired GP reviewers disagree on their classification. If further information is needed in order for the subgroup to make an informed judgement, we will ask one of the GP reviewers to go back and obtain this. Also, if the perspective of a particular member of the primary healthcare team is required to help make a judgement, the subgroup will discuss the case with a relevant member of the wider study team.

Bias

Sampling of practices and patients

To reduce bias, we will recruit a stratified random sample of practices in the East Midlands, London and Greater Manchester. All patients within a practice will be potentially eligible for inclusion, although bias may be introduced if patients with more serious conditions object to a review of their records. As noted above, to assess the possibility that patients with avoidable significant harm will not be included in the enhanced sample, a 2.5% random sample of ‘Stage 1’ patients not included in the ‘Stage 2’ sample will be reviewed. To assess the possibility that the GP reviewers may ‘miss’ patients with significant harm when reviewing the stage 2 sample, a 10% random sample of these patients will be screened by a second GP reviewer.

Training of reviewers

The main issue is the extent to which the GP reviewers involved in the retrospective case note review adhere to consistent methods for identifying and classifying patients with avoidable significant harm. To reduce reviewer bias, e-learning and face-to-face training will be provided to GP reviewers on human factors in healthcare, principles of patient safety incident analysis and patient safety incident coding using the PISA classification system. All identified cases of avoidable significant harm will be double-coded by GPs.

The findings of each pair of GPs will be compared to determine the extent to which the GP reviewers agree on the identification and classification of avoidable significant harm. Disagreements will be monitored in real time and discussed in regular teleconference calls with a view to ensuring correct interpretation of codes and their definitions. These calls will also allow GPs to bring difficult cases for peer discussion, and to agree to any developments in classification (ie, new codes, changes in definitions) and/or coding rules.

Study size

On the basis of our pilot study, our suggested sample size of 100 000 patients is the largest we can manage within the funding available while also conducting the study to the highest standards of rigour. The aim of our calculations is to identify the precision for which a population incidence of avoidable harm can be estimated for a given sample size of 100 000 patients. The estimates are shown in table 2 for a range of incidences of avoidable significant harm (the figure of 80/100 000 patients per year is highlighted as we think this is at the lower boundary of the likely incidence).14

Table 2

Degree of precision of estimates of the incidence of avoidable significant harm

Estimated incidence of avoidable significant harm (per 100 000 patients per year)	Precision (width of 95% CI)	95% CIs based on a sample size of 100 000 patients
20	9	11 to 29
40	12	28 to 52
80	18	62 to 98
100	20	82 to 120
200	28	172 to 228

Bold values are the lower bound of the likely incidence.

Analyses

Descriptive analysis

We will describe the demographics of the study population (in terms of age, sex, ethnicity and deprivation) and compare this informally with: (1) data for England from the Office of National Statistics and English Indices of Deprivation; (2) any practices that were approached but did not agree to take part. We will also describe patient demographics (the age distribution and sex) of those with potentially avoidable harm and for the subsamples identified in stages 2–3. In addition, we will describe the characteristics of the GPs, including GP training status and workload (based on consultation rates).

We will estimate the incidence of avoidable significant harm and express this as ‘per 100 000 patients per year’ accompanied by 95% confidence intervals (CIs). We will do this initially on the ‘assumption’ that our sampling process has identified all the avoidable significant harms in the overall population. We will also do a sensitivity analysis to take account of the proportion of harms that might have been missed by our sampling processes. We will estimate the incidence of:

Significant harm judged at least possibly avoidable

Significant harm judged at least probably avoidable (this will be our principal outcome)

Patterns of avoidable significant harm will be summarised, focusing particularly on the types of safety event, such as administration of medication, the setting (eg, general practice, community pharmacy, community nursing, etc), the types of staff involved and outcomes including severity of harm.

Statistical analysis

Univariate analysis, followed by multivariable analysis adjusting for confounders, will be conducted to identify the factors (relating to patients and general practices) associated with avoidable significant harm, for example, risk of avoidable harm could be higher in practices with the greatest workload.

We will assess inter-rater reliability of judgements made by paired GP reviewers using the Kappa statistic (with 95% CI). These assessments will be undertaken on the following judgements:

Identification of patients with significant new health problems (significant harm) from a 10% random sample of the ‘Stage 2’ sample.

Identification of patients with significant harm judged at least possibly avoidable.

Identification of patients with significant harm judged at least probably unavoidable.

Ethics

The decision regarding participation by general practices in the study is entirely voluntary. The investigator (who will be a member of the study team) will explain the details of the study and provide a practice information sheet, ensuring that the practice team have sufficient time to consider participating or not. The investigator or their nominee shall emphasise to general practices that consent regarding study participation may be withdrawn at any time without having to give a reason for this and without penalty. The investigator (usually one of the GP reviewers) will inform the senior representative of the general practice of any relevant information that becomes available during the course of the study (such as serious safety incidents), and will discuss with them whether they wish to continue with the study.

We do not plan to seek individual patient consent for the retrospective case note review. If (as a result of local publicity about the project) patients state that they do not wish their records to be included in the retrospective case note review, the GP reviewers involved in the data collection will respect these requests and not include the patients. Patients will be able to make their request for their records not to be included in the study by contacting their practice and a withdrawal form will be completed. To allow patients the opportunity to dissent, consented practices will be asked to display the study information 6 weeks before the data collection period begins. Patients will be excluded if they have a computer code in their clinical records, indicating that they do not wish to be included in research studies.

The PPI representatives will be members of the project management group, which will convene at least once monthly, and they will actively contribute to ongoing discussions about the conduct of the study. They will also contribute to at least one of the early meetings with GPs who will be involved in data collection so that they can provide their views on the operationalisation of definitions of avoidable significant harm in the study.

Dissemination

We will meet with relevant colleagues within NHS England towards the end of the project to appraise them of the key findings. We will specifically discuss potential new measures for the NHS Outcomes Framework, and the conduct of future assessments of the scale and nature of avoidable harm in primary care. We will explore methods of further dissemination with the Primary Care Patient Safety Expert Group (NHS England) including holding an event and running webinars. Similarly, we will engage with the network of NHS England Patient Safety Collaboratives and advise on interventions aimed at reducing the incidence of avoidable significant harm in primary care.

We expect to produce the following outputs from our research:

The production of robust generalisable information on the frequency, nature and seriousness of avoidable significant harm in primary care.

A report for the Department of Health's Policy Research Programme.

Several high-quality peer-reviewed publications in scientific journals, which we will report using STROBE and RECORD guidelines.

Presentations at meetings and conferences for service users, healthcare professionals, policymakers and academics.

A range of materials for healthcare professionals to help them with strategies for analysing and reducing the incidence and impact of avoidable significant harm in their contexts (these will be performed after the project has been completed, and without additional charge to the Department of Health).

Materials for service users to help them to recognise the potential for avoidable harm in primary healthcare, and how they might work with healthcare professionals to avoid these harms, for example, by voicing concerns.

The PPI representatives will run patient stakeholder events to provide emerging findings to service users in order to gauge likely responses to the publication of our report.