A patient-centred model to quality assure outputs from an echocardiography department: consensus guidance from the British Society of Echocardiography.

Background Quality assurance (QA) of echocardiographic studies is vital to ensure that clinicians can act on findings of high quality to deliver excellent patient care. To date, there is a paucity of published guidance on how to perform this QA. The British Society of Echocardiography (BSE) has previously produced an Echocardiography Quality Framework (EQF) to assist departments with their QA processes. This article expands on the EQF with a structured yet versatile approach on how to analyse echocardiographic departments to ensure high-quality standards are met. In addition, a process is detailed for departments that are seeking to demonstrate to external bodies adherence to a robust QA process. Methods The EQF consists of four domains. These include assessment of Echo Quality (including study acquisition and report generation); Reproducibility & Consistency (including analysis of individual variability when compared to the group and focused clinical audit), Education & Training (for all providers and service users) and Customer & Staff Satisfaction (of both service users and patients/their carers). Examples of what could be done in each of these areas are presented. Furthermore, evidence of participation in each domain is categorised against a red, amber or green rating: with an amber or green rating signifying that a quantifiable level of engagement in that aspect of QA has been achieved. Conclusion The proposed EQF is a powerful tool that focuses the limited time available for departmental QA on areas of practice where a change in patient experience or outcome is most likely to occur.

‘Quality involves the totality of a patient’s experience’

Background

Echocardiography is an important clinical investigation which is frequently incorporated into a patient’s journey across the full spectrum of medical specialties. Its use is routinely recommended in several patient cohorts due to its ability to provide data that when acted upon can significantly improve outcomes and reduce mortality (2, 3). However, it is a predominantly physiological test with an inherent conflict between the subjective nature of image analysis and interpretation and the binary nature of the accompanying report. In addition, studies have demonstrated that the intra-observer, inter-observer and inter-study variability of echocardiography is often significant (4). This variation and the overall accuracy of studies and their accompanying reports can potentially be improved by a robust quality assurance (QA) process within echocardiographic departments. However, despite this logical concept, there is a paucity of published documentation outlining a specific, practical, methodology of how a complete and holistic QA process can be setup.

Both the European Association of Cardiovascular Imaging (EACVI) and the American Society of Echocardiography (ASE) have previously published outline quality initiatives for echocardiography departments (5, 6). The BSE has also recently produced an EQF to assist echocardiographic departments with their QA processes (7). This article further builds on this QA framework and by doing so aims to provide a practical guide on how to undertake QA.

Previously published methodologies for echocardiography QA often focus on the blind re-reading of an arbitrary number of cases (8). This suggested toolbox borrows heavily from the field of clinical chemistry (9); a discipline which crucially has stable control materials readily available to benchmark against. Thus, whilst this mechanism of QA is suited to linear analysis systems to ensure consistency of machine processes, it is ill-fitting to the more nuanced nature of echocardiography where variability, human judgement and error play a far more central role. In addition, there is a challenge produced by applying the quantitative and heavily statistical methodology used in this specialty to the frequently more qualitative process of echocardiography, resulting in a degree of mal-alignment.

Furthermore, the process of blind re-reading of a set number of randomly selected cases may reveal interesting or important discrepancies but is more likely to simply identify unimportant omissions of measurements which seldom alter the conclusions of a test. Worse still, this arbitrary process can be perceived or used as a tool to assert authority in a hierarchical manner; a manifestly undesirable approach to team-working and leadership which can have disastrous consequences when the stakes are high (10).

In reality, the ‘dry and blind’ approach ignores how humans learn best. Fewer take-home messages are created, and it is harder to identify and incorporate subsequent practice improvement measures into a busy scanning schedule. Human nature is to learn through curiosity and subsequent explanation of the facts through a human narrative (11). This is best illustrated in medicine through the enduring power of clinical case-studies, which continue to be a popular feature in major journals.

To reconcile the conflicts outlined above, the BSE produced the EQF. This is a patient-centred QA process which switches the focus away from a contained, inward-looking, analysis of echo acquisition and report generation and instead steers the review process with an over-riding emphasis of improving patient care. In order to achieve this, the QA process comprises four domains: Echo Quality, Reproducibility & Consistency, Education & Training, and Customer & Staff Satisfaction (Fig. 1).

Figure 1

The EQF.

Placing the EQF into practice

Exacting standards in echocardiography are best achieved through a continuous cycle of improvements. In truth, the simple act of engaging in some form of QA of a department’s output will bring benefits; often in areas not initially anticipated. However, whilst the EQF is a useful guide to apply to QA, we recognise that it only forms part of a process which also requires feedback and interpretation. To capture the various levels to which departments can utilise the EQF a traffic light system for each domain is proposed: initially red (no evidence of involvement), then amber (engagement in the process) and finally green (excelling in the process).

We describe in Table 1 and the next section our interpretation of what would be considered sufficient to be engaging in the process of QA (amber status). This list is deliberately prescriptive, and the numbers proposed are derived from an expert consensus of what would be reasonable and achievable for a department. Alongside this list we provide examples of best practice (i.e. green status). To excel at QA to this level does not require a greater degree of granular analysis of studies (i.e. reviewing more and more scans in greater detail). Instead, an excellent QA process is one where feedback and reflective practice are embedded within to ensure that all lines of enquiry are intelligent and clinically relevant.

Table 1

Summary of amber and green requirements for each EQF domain.

1 Echo Quality
1A Echo studies	Amber: Quarterly departmental meetings with a minimum of five cases reviewed per meeting
	Green: Departments must also provide evidence that the whole team is involved in the process, and provide evidence of feedback and quality improvement
1B Echo reports	Amber: Quarterly departmental meetings with a minimum of five cases reviewed per meeting
	Green: Departments must also provide evidence that the whole team is involved in the process, and provide evidence of feedback and quality improvement
2 Reproducibility & Consistency
2A Variability	Amber: One key output variable per year rotating over a 5-year cycle (e.g. LV ejection fraction, aortic valve area assessed by continuity equation, severity of MR, estimation of right heart pressures via TR velocity, aortic root dimensions)
	Green: Departments must also provide evidence that the whole team is involved in the process, and provide evidence of feedback and quality improvement
2B Audit	Amber: Two audits per year (these could be automated through a department’s image-archive and simply involve discussion of the results at the departmental meeting)
	Green: Departments must also provide evidence that the whole team is involved in the process, and provide evidence of feedback and quality improvement
3 Education & Training
3A Training	Amber: Evidence of implementation of a structured echo training programme
	Green: Assessment framework and evidence of successful completion e.g. BSE/EACVI/ASE Accreditation
3B Teaching	Amber: Case reviews within departmental meeting; 20 h per year
	Green: Evidence of topic teaching (this could be involvement in a wider regional meetings)
4 Customer & Staff Satisfaction
4A Service users	Amber: One service user survey and an action plan within a 3-year cycle
	Green: Two-yearly rolling programme
4B Patients and carers	Amber: One customer satisfaction survey and an action plan within a 3-year cycle
	Green: Two-yearly rolling programme

‘Health is all about people. Beyond the glittering surface of modern technology, the core space of every health care system is occupied by the unique encounter between one set of people who need services and another who have been entrusted to deliver them’

Methods

Central to the entire process of the EQF is dedicated time for a departmental meeting. The frequency and duration of these meetings is not mandated but we envisage that the achievement of an amber rating in all four domains of the EQF should be possible with a weekly 1-hour meeting for key individuals. However, it is recognised that varied work practices may mean that alternative provisions are more appropriate; for example, a 4-hour monthly meeting. A list of people attending each departmental meeting should be kept and an attendance rate of >75% for each practitioner would be considered good practice. In addition to frequent meetings of key individuals, variable in frequency in response to the specific features of each unit, we consider it appropriate for the entire department to meet at least quarterly.

1 Echo Quality

Are we constantly improving the quality of our echo pictures we produce? Do our reports help clinicians provide better patient care?

Echo studies should be selected at random from those recently placed in a department’s image archive. One method is to randomly select a day in the previous month and choose every 7th study recorded; if a sonographer is chosen more than once, then the scan immediately preceding this one is used. Interesting cases should also be included. Cases may be anonymised depending on departmental preference.

1A Echo studies

Group re-reading of randomly selected studies during a departmental meeting; it is suggested that this process is supported by use of a structured score sheet (Supplementary Data 1, see section on supplementary data given at the end of this article). The study is reviewed and marked by the group, facilitated by a senior echocardiographer (cardiologist or physiologist). Results should be discussed during the meeting. This process will likely lead to further enquiry or identification of learning-goals as required. The score sheets and comments should be retained and recorded.

Amber: Quarterly departmental meetings with a minimum of five cases reviewed per meeting.

Green: Departments must also provide evidence that the whole team is involved in the process and provide evidence of feedback and quality improvement.

1B Echo reports

Group review of randomly selected studies is undertaken during the departmental meeting. Focus is directed on the accuracy and interpretation of the findings recorded and the usefulness of the echo report produced (see Supplementary Data 2 for suggested structured report score-sheet). Again, the results are discussed by the group, facilitated by a senior echocardiographer.

Amber: Quarterly departmental meetings with a minimum of five cases reviewed per meeting.

Green: Departments must also provide evidence that the whole team is involved in the process, and provide evidence of feedback and quality improvement.

2 Reproducibility & Consistency

Are high standards achieved for every patient in every situation?

2A Variability

Simple statistical tools are used to assess the range in reporting of key output variables (e.g. left ventricular ejection fraction (LVEF)). Five studies that focus on the chosen key variable(s) are chosen at random. These cases are then anonymised in the echo archive and details sent to each sonographer to enable individual, targeted, offline analysis. Sonographers then send their results back to a nominated senior echocardiographer for collating. A graph can then be produced with the anonymised sonographer results plotted on the y axis against each of the five cases on the x axis (Fig. 2). From this graph, the nominated senior echocardiographer can identify any practitioner who is an outlier and further explore the reasons behind this. Time in the departmental meeting is then used to review all cases as a group and explore the collated results. Focus can then be brought back to the initial reason which prompted this detailed review to see if further action is required.

Figure 2

Example of assessing LVEF within an echocardiography department. Each dot represents a single sonographer’s calculated LVEF (y axis). Five separate cases were analysed (x axis).

Amber: One key variable per year rotating over a 5-year cycle (e.g. LVEF, aortic valve area assessed by continuity equation, severity of mitral regurgitation, estimation of right heart pressures via various methodologies and aortic root dimensions).

Green: Departments must also provide evidence that the whole team is involved in the process and provide evidence of feedback and quality improvement.

2B Audit

In many ways the entire EQF could be described as an audit process. However, within this specific audit domain, there is a specific line of enquiry that focuses on the ‘process of patient care’. Importantly, each described enquiry must be well circumscribed and quantitative to allow for meaningful interrogation. Suggested topics are:

Appropriateness of echo referrals with reference to published guidelines (13).

Waiting times of echo reports to be finalised and despatched.

Adherence to published guidelines of trans-thoracic echocardiography minimum datasets (14).

LVEF measured by echo vs LVEF measured by cardiac magnetic resonance imaging.

How often is the make/size of valve replacement known.

Some topics will lend themselves to automated analysis using machine language and echo database mining that will increase the efficiency of the audit process. As industry produces ever more sophisticated electronic image and report archives it should even become possible to use database mining tools to provide real-time oversight of a department. Such uses will be benefited by sufficient technical support, for instance, to create queries in SQL (Standard Query Language, the standard language for storing, manipulating and querying computer databases). However, given the large amount of data produced by all departments, clinical oversight of this process is critical so that key questions can be asked which are known to link directly with deficiencies in patient care. Some examples of database queries include:

Indexing values. Is the height and weight recorded for a minimum of 90% of outpatient cases?

How many cases of aortic stenosis, mitral regurgitation or LV systolic dysfunction are reported as ‘moderate to severe’.

Assessment of conflicted measurements in aortic stenosis. Less than 5% of cases reported as not having severe aortic stenosis where the AVA indexed to BSA (<0.6 cm /m ) suggests severe aortic stenosis.

Reporting of aorta dimensions in aortic regurgitation. All cases reporting mild or greater aortic regurgitation should have a complete set of measurements for the aortic root and ascending aorta.

Amber: Two audits per year (this could be automated through a department’s image-archive and simply involve discussion of the results at the departmental meeting).

Green: Departments must also provide evidence that the whole team is involved in the process, and provide evidence of feedback and quality improvement.

3 Education & Training

How do we improve patient care through education of all providers and users of echo?

Wherever possible clinical research should be embedded into departmental practice. This may be work specific to the techniques of echocardiography or the use of echocardiography as an analytical tool within a separate line of enquiry. Embracing this natural link will help to nurture a culture of education and training which is advantageous to both the department and the wider echocardiography community.

3A Training

Any department with non-accredited sonographers will already have a structured training and supervision programme in place. This process may also extend to other echo stakeholders within the host institution (e.g. the intensive care, acute medical and emergency departments). Consequently, it is possible for an echo department to demonstrate achievement of this domain simply by providing the names of accredited alumni who have received training there and the outline programmes of courses that staff have contributed to.

Amber: Evidence of implementation of a structured echo training programme.

Green: Assessment framework and evidence of successful completion e.g. BSE/EACVI/ASE Accreditation.

3B Teaching

This may be unstructured through the review of interesting cases in the departmental meeting. However, specific programmes are encouraged (Table 2 for example subjects) and may be undertaken as part of regional and national educational meetings. It is also appropriate to provide education to service users, potentially through local primary healthcare forums or national physician meetings.

Table 2

Suggested teaching subjects.

• Fundamentals of ultrasound physics and haemodynamic calculations

• Ventricular size and systolic function• Diastolic dysfunction• Ischaemic heart disease • Coronary artery anatomy and disease • Role of echocardiography in IHD • Complications of myocardial infarction• Cardiomyopathies• Aortic valve disease • Assessing and quantifying aortic stenosis/regurgitation • Limitations of the continuity principle• Mitral valve disease • Anatomy of the mitral valve • Operative and percutaneous treatment• Tricuspid and pulmonary valve disease (congenital heart disease) • Echo imaging • Tricuspid/pulmonary stenosis • Tricuspid/pulmonary regurgitation.• Prosthetic heart disease • Role of echocardiography • Haemodynamic assessment of prosthetic heart valves • Complications of prosthetic valves• Diseases of the aorta • Anatomy and structure of the aorta • 2D imaging and measurements of the aorta • Aortic aneurysms • Aortic dissections• Pericardial disease • Anatomy and function of the pericardium • Pericarditis • Pericardial effusions • Cardiac tamponade • Constrictive pericarditis• Infective endocarditis and cardiac masses• Congenital heart disease • Shunt lesions • Obstructive lesions • Cyanotic lesions • Foetal circulation• Practical teaching suggestions • Image optimisation • Technical tips for Doppler imaging • Imaging the aorta • Accurate continuity equation measurement • PISA measurements • Planimetry • Scanning tips for congenital heart disease

Amber: Case review within departmental meeting; 20 h per year.

Green: Evidence of topic teaching (this could be involvement in a wider regional meetings).

4 Customer & Staff Satisfaction

What do people who use our service say about us? Are we kind to our patients?

4A Service users

An example questionnaire for health care professionals who request echocardiograms is available in Supplementary Data 3. It covers a range of subjects: the perception of waiting time for scans, ease of access to timely reports and the fluidity of interpretation of the results presented into clinical practice. It is anticipated that discussion of the results of this questionnaire will act as catalyst for enquiry elsewhere in feedback loop.

Amber: One service user survey and an action plan within a 3-year cycle.

Green: Two yearly rolling programme.

4B Patients and carers

An example questionnaire for patients is available in Supplementary Data 4. It is designed to explore the whole patient journey: ease of finding the department, the attitude of staff, perception of dignity during the scan and understanding of how the results will be conveyed. Practical experience suggests that the highest completion rate is when paper copies of the questionnaire are available for patients to fill-in immediately after their scan.

Amber: One customer satisfaction survey and an action plan within a 3-year cycle.

Green: Two yearly rolling programme.

Excelling at QATo excel at QA is to embed it within the wider systems and processes of a department.

Examination of the domains outlined previously is most powerful when incorporated within a broader feedback mechanism. When undertaken in its entirety, this process forms an analysis of a departments' outputs which is driven by, and responsive to, local demands. In doing so, attention is focused on areas of review likely to result in a positive impact on patient care.

Initially, various feedback mechanisms can be used to ascertain where deficiencies are present in the service that warrant further review. Any tool of reflective practice is suitable: the surveys described in domain four, review of an interesting case, root-cause analysis of a critical event or discussion resulting from interesting journal article. The service deficiency identified can then be mapped into the domains of the EQF framework to focus subsequent review on tangible mechanisms to interrogate and address them. Through this process, the opportunity to identify where shortfalls are present and what the solutions are available to enhance patient care are maximised. A suggested approach would consist of the following stages (also outlined in Fig. 3):

Figure 3

Flow chart of potential methods to embed the EQF into clinical practice.

Input: Review of a critical case, journal article or patient feedback survey.

Identify deficiency: Using the EQF domains as a guide.

Intervention: This could be a statistical analysis of departmental variability in the assessment of a key output variable (e.g. mitral regurgitation) or a training update performed during the departmental meeting.

Discussion: Group review of the intervention.

Output: Specific points that enable practice improvement are agreed upon (e.g. changing hospital signage, adjusting protocols to allow for CT scanning of a dilated aorta the same day as the echo or adjusting the content of reports).

Three examples of this process in a real-world environment are outlined next. In addition, Supplementary Data 5 demonstrates how following this process generates evidence, which can be used in support of an application for BSE Departmental Accreditation.

Example 1: Responding to the requirements of different patient groups.

Input: A customer satisfaction survey identified a high level of anxiety among patients undergoing regular echo monitoring whilst receiving herceptin chemotherapy.

Deficiency: This was a felt to be a reproducibility issue compounded by patient fixation of LVEF as an exact number. Furthermore, there was an occasional decision by oncology colleagues to halt chemotherapy following a 10% reduction in LVEF where both measurements remained in the normal range (i.e. 68–56%).

Intervention: Five sets of anonymised LV-only images were created by drawing random cases from the echo archive. Before the departmental echo meeting all physiologists performed an offline analysis of LVEF (using Simpsons Biplane method) for each case and email their results to a nominated senior echocardiographer. These results were collated (Fig. 2) and discussed as a group during the departmental meeting.

Discussion: It was clearly demonstrated that all sonographers were reporting LVEF to within a 10% range. Importantly, no outlying individual sonographers were present in need of further training. However, this process identified the limit of LVEF as a measure from our department and that our previous reporting to an assumed accuracy of ±1% was inappropriate.

Output: Herceptin monitoring study reports were adjusted to only report LVEF ranges (i.e. >55, 45–55, 35–45, <35%).

Example 2: Revisiting how to assess mitral regurgitation.

Input: Review of a complicated case of mitral regurgitation (MR) during a departmental meeting.

Deficiency: There was general confusion regarding the appropriateness and value of the multiple different echo variables available to assess MR.

Intervention: As described in example 1, five anonymised cases of MR were reviewed by each practitioner using all currently recommended BSE echo variables.

Discussion: The group identified that if LVEF was poor then assessment of MR using physiological variables (i.e. MR velocity–time integral/regurgitant volume) underestimated the severity of MR. However, in this setting, the more anatomical variables (i.e. vena-contracta assessment: measurement of the MR orifice) remained accurate.

Output: This was felt to be a very empowering process for all staff as the above caveat in the assessment of MR was determined from first principles whilst discussing the inconsistency of our results. As a result, all practitioners now have a greater understanding of how to approach a case where different MR echo variables produce conflicting results.

Example 3: Serious case review – evaluating the aortic root.

Input: Consultant colleague feedback to the department. A case was reported of a 55-year-old man, with a 15-year-old mechanical aortic valve replacement, who attended for an outpatient CT thoracic aorta that identified a type A aortic dissection. The aortic root on a recent trans-thoracic echocardiogram was reported as 63 mm in diameter (performed one month beforehand; the result had prompted the CT request).

Deficiency: Is echocardiography a sufficiently accurate tool to predict similar cases in our hands? We would not wish to flood our CT department with urgent cases, but can a threshold be established for more expedited CT imaging of the aorta.

Intervention: Known CT cases of dilatation of the ascending aorta were identified by radiology colleagues. An inter-observer analysis of the associated echo-derived aortic root diameters was performed by each echo practitioner and results compared between modalities.

Discussion: Very similar values were obtained both between each member of our department measuring each echo case and the echo-mean aortic root diameter compared to the CT-derived aortic root diameter (i.e. ±2 mm).

Output: An education need was identified; the utility of obtaining a parasternal long-axis window one rib-space above normal to assess the aortic root was emphasised as were echo-features of aortic dissection (e.g. the presence of a small pericardial effusion). In addition, departmental protocol was altered so that everyone with an aortic root diameter of greater than 55 mm now must have a same day CT thoracic aorta (this diameter is the conventional threshold for offering prophylactic aneurysm-repair surgery (15)). This adjustment in departmental policy was only felt to be practical given the close correlation between our echo and CT-derived aortic dimensions.

Conclusion

QA of an echo department’s output will always rely on a collective quantitative and statistical analysis of the key-variables produced. Furthermore, the re-reading of studies is unavoidable. However, to maximise engagement in this process and focus the limited time allocated most effectively, it is important to be guided by supporting feedback which re-focuses the emphasis on improving patient care. The proposed EQF is a robust tool that facilitates departments with this aim and helps to ensure that high standards are maintained for all patients at all times.

Declaration of interest

The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of this guideline.

Funding

This work did not receive any specific grant from any funding agency in the public, commercial or not-for-profit sector.