Clinical audit-ESR perspective.

This paper provides a comprehensive outline of the audit process advocated for clinical radiologists and clinical radiology departments. The philosophy discussed is equally appropriate for interventional and diagnostic radiologists.

Introduction

Within Europe there is wide variation in the understanding and implementation of clinical audit. Interpretation of the term clinical audit and its differentiation from regulation, quality assurance, accreditation and research also differs across Europe. This document attempts to define and establish the scope of clinical audit in a way that is applicable across member states and radiological organisations. Participation in clinical audit has many benefits which include the demonstration of a commitment to the delivery of a high quality service. It may also indicate areas of the service where further investment is required

Definition

Clinical audit is a tool designed to improve the quality of patient care, experience and outcome through formal review of systems, pathways and outcome of care against defined standards, and the implementation of change based on the results. Audit uses specific methodology in which performance is compared with a preselected standard. If the standard is not achieved, reasons for this are explored, change is implemented and a re-audit is carried out to ensure improvement [1]. This methodology is often described in terms of the audit cycle, illustrated in Fig. 1.

Fig. 1

The audit cycle.

Responsibility for audit

Departmental audit

Those who use, pay for or manage radiology departments or services will wish to ensure that these are of the highest possible standard, but clinical audit, as implied in the term, is a professionally led activity, which is designed and carried out by appropriately trained health professionals rather than managers or professional auditors. Health professionals, including radiologists together with other professional staff such as radiographic/nursing/technical staff and physicists who are directly involved in service delivery are often best placed to know those areas which are either particularly important in the delivery of a safe service, or where improvement may be required. Equally, they are best placed to suggest specific improvement strategies where necessary. Although the audit is professionally led, all staff within the team which has responsibility for the part of the service under review should be able to contribute to the process.

Audit can be described as internal or external. Internal audit, which is more commonly carried out, refers to audit carried out within a department or institution and external audit refers to audit performed by professionals from outside the department or institution. Whether internal or external, audit should not be carried out without the knowledge of those involved in the delivery of the service and should be a planned, scheduled process. All audit, whether within the unit in which one works, or audit involving others’ work is dependent on professional honesty and integrity. Appropriate confidentiality should be observed as the aim is improvement, not blame.

When organisations invite teams external to the organisation to carry out audit, the considerations outlined above related to professional leadership, confidentiality and no-blame/improvement culture still apply.

Personal or individual audit

This is related to departmental audit but is a more difficult and contentious issue as standards for reporting accuracy have not been established. It is a professional duty of all radiologists to examine both the quality of their work and the systems within which that work is carried out [2], and self audit is a valuable learning tool as well as ultimately being beneficial to patient care. Further guidance on how individuals can monitor their performance will be formulated by the ESR. Audit has an important role in continuing professional development and education for both individuals and departments. Improvements in the quality of the delivery of radiological services should be focused on self-improvement aided by identifying areas where further investment in services is required. The results of audit should be used within a positive, constructive and forward-looking framework and not used in a non-statistically valid way to judge individual performance. Working within a department which invests time and effort in clinically relevant audit and which looks at team and individual performance in the context of the overall improvement of services to patients is the most valuable form of individual audit.

Scope of audit

It is possible to audit every aspect of a radiology department and how it functions, and every stage of the patient journey from receipt of request to the radiology report reaching the referrer. Audits can be comprehensive and look at a large number of factors or processes simultaneously or can be tailored to very specific areas of service delivery.

Types of audit

All aspects of a radiological organisation and its performance are amenable to audit. Audit can be divided into three categories.

Structure audit. Examination of the systems within which we work, for example the management structure, accommodation, equipment, staffing and training.

Process audit. Examination of the processes involved in the delivery of care from initial referral to delivery of a radiological report including for example quality management of the processes, justification, waiting times and examination practices and protocols [3, 4].

Outcome audit. Examination of the outcome or results of the delivery of care, which may include medical outcome and patient satisfaction [5, 6].

Standards

Audit cannot be carried out without a preset standard against which performance can be assessed. These are not necessarily widely available. There is a particular lack of validated patient outcome or accuracy standards. This factor, together with a general difficulty in measuring patient outcomes, results in this type of audit being the most difficult to carry out.

Sources of standards

Standards against which local performance can be measured can be found from a variety of sources

Local, European or international legislation. Compliance with these standards is compulsory [7].

Peer-reviewed research. These will provide benchmark standards but may have to be interpreted in the light of local facilities and expertise.

Recommendations or consensus statements from learned or national societies and organisations. These will usually have been developed to be applicable in routine practice [8, 9].

Where no published or recommended standards are available, these may have to be established by local agreement or consensus before the relevant audit is undertaken. Under these circumstances, locally sourced data from comparative investigations, pathology, surgical findings, peer group review or clinical follow-up may allow the setting of local standards for outcome audits.

High or low standards?

Standards, other than those governed by legislation, are not necessarily pass or fail. A very high or aspirational standard may only be achieved by the very few but could serve to encourage maximum improvement. If the selected target standard is based on the average expected performance, then initially, 50% will be expected to fall below it. A low or minimum target standard may be regarded as the minimum acceptable level of performance. The level of the standard selected should be taken into account in interpretation of results.

Indicators

The indicator or indicators are measurable variables related to the standard. An indicator or series of indicators should be identified at the start of the project to decide what data will need to be collected to calculate the value of the indicator and hence to decide if the chosen standard has been met. Examples of indicators in radiology include the examination volume per modality as a productivity indicator, the report turnaround time as a reporting efficiency indicator, access to an imaging modality as an access indicator and the expenditure on contrast media as a financial indicator.

Data collection

Data to be collected may include items such as observations or measurements. Collection should aim to ensure that the data are complete, accurate and representative so that valid conclusions can be reached. The ease of data collection may be affected by the locally available data storage methods.

Prospective versus retrospective audit

Data may be collected prospectively over a period of time, for a predetermined number of cases, or retrospectively from existing information sources. Prospective collection is more likely to ensure completeness of information, but the process of collection may influence the behaviour of participants and therefore the outcome of audit. It may also take longer to gather the data. Retrospective collection from records may however result in incomplete data being available. A not uncommon scenario would be a retrospective audit which shows areas for improvement, followed by prospective re-audit after the appropriate changes have been made.

Examples of audits

Structure audit

Type: staff training.

Standard: 100% of department staff should have completed training in cardiopulmonary resuscitation.

Indicator to be measured: percentage of staff who have completed training within the time frame specified in local rules.

Data to be collected: the total number of staff and the number who have undergone training.

Suggested number of staff to be sampled: all staff.

Process audit

Type: patient consent [10].

Standard: for 100% of interventional vascular radiology procedures, there is documented evidence that a discussion of the procedure by a suitably qualified member of staff has taken place and there is a written record of patient consent.

Indicator to be measured: the percentage of patients for whom there is evidence that consent procedures have been completed.

Data to be collected: consecutive patient records examined for written evidence of pre-procedure discussion, the name of the doctor and the patient’s written consent.

Suggested number of patient records to be sampled: 30 consecutive interventional procedures.

Outcome audit

Type: procedure complication rate [1].

Standard: fewer than 20% of lung biopsies should result in pneumothorax and fewer than 8% of patients should require chest drain insertion.

Indicator to be measured: the percentage of patients who suffered a pneumothorax and the percentage requiring a chest drain.

Data to be collected: consecutive lung biopsies, patient identifier, name of doctor, needle size used, presence or absence of pneumothorax, chest drain required or not.

Suggested number of procedures to be sampled: all lung biopsies carried out in 1 year.

Accuracy of audit

Audit is a sampling process and unlike research is not primarily designed to be statistically robust since it is carried for the purpose of improving local quality of care rather than influencing others’ practice. When audit data are interpreted, this potential, although not inevitable statistical weakness, should be taken into account. Various statistical methods can be employed to increase confidence in the statistical validity of the results [11].

Analysis of audit results

When the chosen standard is attained, this can be taken as affirmation of the quality of the service and reassurance that no change is necessary. Audit is primarily a quality improvement tool, and in those cases where the chosen standard is not reached, the results should be interpreted in a culture which does not seek to blame individuals. Analysis of the results should examine all the possible reasons for the results not meeting the standard, including the target level chosen, system, process, and technical reasons. Only then can system changes be introduced to address any measured shortcomings. Consideration should also be given to possible sampling bias accounting for the underperformance. A checklist of suggested changes to improve performance should be then be drawn up and implemented.

Re-audit

When change has been implemented, it is mandatory to repeat the same audit process to ensure that the changes introduced have led to the expected improvement. This ‘closes the loop’.

Time/resources required

Professional input into the design and standards chosen for audit is mandatory, but data collection and analysis can be delegated to suitably trained staff. Audit is potentially time consuming and needs to be allocated sufficient time and financial resources.

Conclusion

As part of clinical governance, healthcare organisations are accountable for continually improving the quality of their services [12]. Clinical audit, correctly and professionally conducted, is a powerful tool to improve patient care, experience and outcome.