Clinical management issues vary by specialty in the Victorian Audit of Surgical Mortality: a retrospective observational study.

OBJECTIVE: Clinical management issues are contributory factors to mortality. The aim of this study was to use data from the Victorian Audit of Surgical Mortality (VASM), an educational peer-review process for surgeons, to discover differences in the incidence of these issues between surgical specialties in order to focus attention to areas of care that might be improved.
DESIGN: This study used retrospectively analysed observational data from VASM. Clinical management issues between eight specialties were assessed using χ(2) analysis. DATA SOURCES: VASM data were reported by participating public and private health services, the Coroner and self-reporting surgeons across Victoria.
RESULTS: A total of 2946 specific clinical issues as deficiencies of care were reported. 15% of cases had significant issues of care. The most common clinical management issue was the delay in delivery of treatment. Other clinical issues included the quality of communication and documentation, preoperative and postoperative care, adverse events and protocol issues. There were significant differences in issues between specialties.
CONCLUSIONS: The clinical management issues presented across surgical specialties were similar; however, five issues of clinical care differed significantly in frequency across surgical specialties. The three main issues varying among specialties were complications after operation, communication and postoperative care. Addressing these clinical management issues via the peer-review process may impact positively on patient care. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

The different specialties have not been previously studied in this fashion although these differences might be expected.

Because of the limitations of the data set, in-depth analysis of the reasons for these differences is not apparent. Small numbers reduce the impact of differences in some specialties and exclude a few from analysis completely.

The mortality audit is confined to deceased patients and clinical details are not available for predictive analysis.

Introduction

Surgical audits and registries are established to measure, monitor and identify outcomes through the peer-review process, thus providing feedback to the healthcare providers. Consequently, this type of monitoring system can improve patient care.1–4 The Victorian Audit of Surgical Mortality (VASM) is a retrospective observational clinical audit that follows the operating principles of Australian clinical quality registries as specified in the Australian Commission on Safety and Quality in Health Care.5

VASM is part of the Australian and New Zealand Audit of Surgical Mortality (ANZASM), a binational network of regionally based audits of surgical mortality that aims to ensure the highest standards of safe and comprehensive surgical care. VASM was established in 2008 in collaboration with the Victorian Surgical Consultative Council (VSCC) and is funded by the Victorian Department of Health with infrastructure provided by the Royal Australasian College of Surgeons where it is responsible to the Professional Development and Standards Board via the Research, Audit and Academic Surgery Board.

VASM's peer-review component is similar to its interstate counterparts (figure 1). All deaths that occur in the hospital either postsurgery or while in the hospital under the care of a surgeon are peer reviewed. To appraise the appropriateness of the clinical care provided in each case, all cases undergo first-line assessment (FLA) and a minority of cases also undergo second-line assessment (SLA). During peer review, clinical management issues would be identified.

Figure 1

Victorian Audit of Surgical Mortality (VASM) audit process.

Currently, evaluating outcomes for many audit and registry activities focus on specialty-specific clinical management issues.6 The aim of this study was to examine VASM data for differences in clinical management issues across specialties. It was hypothesised that clinical management issues would be similar across the specialties but that the degree of severity would vary.

Methods

This study utilised observational data, retrospectively analysed. It examined ‘clinical management issues’ as deficiencies of care per specialty. Data from the audit period 1 January 2008 to 30 June 2012 were analysed by a mixed method approach using quantitative and qualitative data. Ethics approval was granted by the Victorian Department of Health Ethics Committee at inception of the audit to ensure compliance with privacy legislation. Participation of the surgeons was covered under the Commonwealth Qualified Privilege Scheme, protecting the confidentiality of information created as part of the audit.

The VASM data were reported by participating public and private health services, the Coroner and self-reporting surgeons across Victoria, which had a population of 5 623 500 as in June 2012. Clinical data were collected on 2 400 542 patients who underwent surgery in 250 Victorian public and private hospitals, including surgeons who treated the patients and their specialty. Of the 5585 surgical deaths reported from 79 hospitals since 2008, only 2862 (51.24%) cases completed the full audit process, as described in figure 1, which involved an FLA and an SLA. From this data pool, 559 (19.53%) were referred to a second-line peer review. The remaining 2723 (48.75%) of 5585 cases not included in the analysis were pending completion or excluded as terminal care or incorrectly attributed to surgery.

There were 1082 surgeons in Victoria of whom 957 (88.45%) were recruited into participation in VASM. The audit data represented 13 specialty-craft groups. Data were analysed for eight of the specialties: cardiothoracic, general, neurosurgery, orthopaedic, otolaryngology/head and neck (ENT), plastic, urology and vascular surgery. Ophthalmology, gynaecology, paediatric, maxillofacial and other (trauma, transplant and oncology) specialties (33 cases) were excluded from this analysis due to low numbers of clinical issues (<35).

Clinical information such as diagnosis, cause of death, course to death, risk status, operative profile, clinical management issues and reviewers’ comments were collected, entered and coded into the custom-built audit database. SLA involved a rigorous review of the de-identified surgical case record form, medical notes and the first-line assessor's comments. If a clinical management issue was identified during peer review, it was classified as an area for consideration, area of concern or adverse event, according to its importance in contributing to mortality. There were two possible outcomes: either death was a direct outcome of the disease process and clinical management had no impact on the outcome, or there was a perception that aspects of patient management may have contributed to the death of the patient. The outcome from the peer review was passed onto the treating surgeon to close the educational loop of the audit.7

Quantitative analysis was conducted using the StatsDirect statistical package8 and Microsoft Excel (2010). Categorical variables have been compared for the eight specialties with more than 35 clinical management issues using r×c χ2 analysis. The Fisher-Freeman-Halton exact test output from StatsDirect was chosen to determine nominal independence.

Data were also analysed, both as an aggregate and per specialty, using a thematic analysis approach. Thematic analysis is a qualitative method that provides a structure for reviewing the descriptive data to generate themes.9 10 This involved Read-Coded data being clustered and reclustered in accordance to its clinical relevance to establish clinical management issues data for each specialty. Categorical variables were the outcome of this analysis and descriptions of the main clinical management issues and categories are listed in online supplementary appendix 1.

Results

The average age of patients who underwent surgery was 80 years, with a range from 1 day to 102 years. The majority of surgical deaths occurred in the 75–85-year age group. The presence of comorbidities increased the risk of death. Presentation to the emergency department with an acute life-threatening condition often required surgical intervention. The emergency admissions usually had an American Society of Anesthetists (ASA)11 grade of 4, 5 or 6. Surgical cases with an ASA status over 4 were more likely to be referred for an SLA.12 Table 1 outlines the categories of clinical management issues. The largest category is ‘delay in definitive treatment’ with 638 of the 2946 specific issues cited. This category has been further subdivided; data not shown are of the following subcategories: delay in diagnosis, 153 (5.2% of all issues); delay in transfer to surgical unit, 148 (5%); delay in surgery where earlier operation was desirable, 129 (4.4%); delay in recognising complications, 49 (1.7%); delay in starting medical treatment, 34 (1.2%); delay in transfer to tertiary hospital, 27 (1%); delays in patient care, 11 (0.4%); delay in fully investigating the patient, 11 (0.4%); delay in patient presenting, 11 (0.4%); and delay in operation caused by missed diagnosis, 11 (0.4%).

Table 1

Clinical management issues per specialty

Specialty	Cardiothoracic		ENT		General		Neurosurgery		Orthopaedic		Plastic surgery		Urology		Vascular surgery		Fisher exact
Clinical issues	n	%	n	%	n	%	n	%	n	%	n	%	n	%	n	%	p Value
Adverse events (AL, bleed, PE, injury)	11	3	1	3	58	4	5	2	17	5	2	4	1	1	6	2	0.329
Anaesthesia related	3	1	0	0	10	1	0	0	4	1	2	4	1	1	2	1	0.258
Communication issues or poor documentation	14	4	1	3	72	5	9	4	12	3	2	4	6	5	19	7	0.44
Critical care issues	6	2	0	0	23	2	3	1	10	3	0	0	1	1	3	1	0.824
Delay in definitive treatment	74	19	6	16	356	24	42	20	65	18	7	15	16	12	69	26	0.001*
Complications after operation	40	10	10	26	113	8	17	8	25	7	7	15	16	12	24	9	0.005*
Operation inappropriate	89	22	7	18	293	20	49	23	54	15	3	7	47	36	53	20	<0.001*
Management or protocol issues	59	15	5	13	162	11	36	17	54	15	5	11	14	11	34	13	0.166
Preoperative care issues	48	12	4	11	184	13	17	8	43	12	6	13	13	10	27	10	0.667
Intraoperative complication	11	3	1	3	4	0	4	2	1	0	0	0	2	2	2	1	<0.001*
Postoperative care issues	38	10	3	8	141	10	28	13	69	19	10	22	10	8	22	8	<0.001*
Septicaemia and wound	4	1	0	0	25	2	1	0	11	3	2	4	1	1	1	0	0.08
Transfer problems	0	0	0	0	18	1	1	0	3	1	0	0	1	1	2	1	0.407
Total	397	100	38	100	1459	100	212	100	368	100	46	100	129	100	264	100	–

Total n=2913.

*p<0.05.

Table 2 shows that the majority of the 2862 audited deaths—2400 (84%)—had no, or only minor, clinical management issues. In 453 cases (16%), major issues of care were identified (areas of concern and adverse events). Relevant data were missing for nine cases. Multiple clinical management issues could be identified per case. A total of 2926 clinical management issues were identified from the peer-review process. Clinical management issues which were perceived as having ‘probably contributed to death’ accounted for 182 events in 146 (5%) cases and, in some instances, issues were considered to be definitely preventable (179 cases, 6%).

Table 2

Frequency of clinical management issues

	Total occurrences	Patients affected by clinical issues (n=2862)
Degree of criticism of patient management
No issues identified	1840	1840 (64%)
Area of consideration	1286	560 (20%)
Area of concern	562	271 (9%)
Area of adverse event	230	182 (6%)
Missing data	36	9 (<1%)
Total	3954	2862 (100%)
Perceived impact on patient outcome
No issues of management identified	1840	1840 (64%)
Did not affect clinical outcome	489	243 (9%)
May have contributed to death	1345	598 (21%)
Probably contributed to death	182	146 (5%)
Missing data	98	35 (1%)
Total	3954	2862 (100%)
Perceived preventability of clinical issues
No issues identified	1840	1840 (64%)
Definitely preventable	238	179 (6%)
Probably preventable	836	378 (13%)
Probably not preventable	739	360 (13%)
Definitely not preventable	82	49 (2%)
Missing data	219	56 (2%)
Total	3954	2862 (100%)

Multiple clinical management issues could be identified per case.

Figure 2 shows that the proportion of cases with clinical management issues has progressively decreased throughout the audit period. There were no clinical management issues identified for 53% of patients in 2007–2008; by the following audit period (2008–2009), 61% of patients received surgical care with no clinical management issues. Clinical management issues continued to decrease in subsequent years so that by 2011–2012, 70% of patients had no issues. This was statistically significant (p<0.001) and highlights the benefit of peer review for surgical patient care.

Figure 2

Spectrum of clinical management issues across the audit period. Explanatory note under figure: Total n=2862. Missing data n=10 (<1%).

In the eight specialties with more than 35 issues of clinical management, these issues could be categorised into 13 categories, as shown in figure 3. The top six categories were delay in definitive treatment (635, 22%), inappropriate operation (595, 20%), protocol issues (369, 13%), preoperative care issues (342, 12%), postoperative care issues (321, 11%) and complications after operation (252, 9%).

Figure 3

Aggregate clinical management issues. Explanatory note under figure: Total n=2913.

The five categories of clinical management issue that showed statistically significant differences in frequency across specialties are shown in figure 4. ‘Complications after operation’ was one of the five most frequently identified issues in urology, cardiothoracic, ENT and plastic surgery, while ‘operation inappropriate’ was a predominant issue in neurosurgery, cardiothoracic, urology and vascular surgery. Under the umbrella of the ‘operation inappropriate’ category the decision to operate at all was the issue in 60% of these four specialties while the type of procedure was felt to be incorrect in 40%.

Figure 4

Statistically significant clinical management issues per specialty. Explanatory note under figure: Total n=2913. ENT=otolaryngology head and neck surgery.

Discussion

The establishment of VASM was part of a national initiative by the fellows of the Royal Australasian College of Surgeons, in collaboration with the VSCC. The independent peer-review component of the audit has been regarded as a forum for sharing clinical knowledge and learning outcomes. It could be perceived as subjective; however, this study reveals that the severity and frequency of specific clinical management issues identified as part of the audit peer review varied between specialties. The most common deficiencies of care were ‘delay in definitive treatment’, ‘inappropriate operation’ and ‘management and protocol issues’ while the three main issues varying among specialties were ‘complications after operation’, ‘communication issues or poor documentation’ and ‘postoperative care’. ‘Delay in definitive treatment’ was identified as the most severe clinical management issue from this study and from the national report data.13 In orthopaedic clinical studies on hip fracture patients, delay in definitive treatment was shown to be attributable to patient factors such as age and comorbidities, as well as waiting times to theatre.14 15 In the aggregate results, ‘communication issues or poor documentation’ was in the top seven clinical management issues. Communication problems have been recognised as the main issue in clinical care, contributing nearly 70% of sentinel events.16

In general, clinical management issues were similar across surgical specialties; however, some issues of clinical care were unique to a specialty as indicated in figure 4. ‘Operation inappropriate’ rated highest in neurosurgery and urology, ‘delay in definitive treatment’ was highest in general and vascular surgery and ‘postoperative care’ highest in orthopaedic and plastic surgery.

VASM addressed these emerging adverse trends through its educational component. The learning outcomes from cases that underwent SLAs form the basis of the yearly publication of clinical case studies through case note review booklets which are freely available.17 These case studies have also formed topics for workshops and seminars which have focused on improving communication among the surgical community, the deteriorating patient, transfer issues, conducting peer-review assessment and profiling the national accreditation advantages of VASM to the Victorian hospitals. Benefit extends beyond the surgical community and other clinical teams (nursing, critical care, emergency and anaesthetics) have been included in our audience and presenters. We recognise that the surgeon is only one contributor to the care of a surgical patient, and survival depends on quality of care at all stages of an episode of care. To close the education loop, the audit findings were also provided to its stakeholders: the hospital and healthcare services, the surgeons, the VASM Management Committee, VSCC and the Victorian Department of Health, to ensure that appropriate changes were implemented.

To validate the progress of VASM and its educational process, VASM was externally audited and produced two reports on the inter-reliability of its peer-review process. The findings indicated that data collected through VASM could be translated into improvements in the quality and safety of patient care which had attained a robust and mature status. A concordance study confirmed that there was good agreement between first and second line assessors, as well as detecting a higher detection rate of clinical management issues by either assessor compared with the treating surgeon, thus validating the audit processes.18 19

Furthermore, VASM and its ANZASM counterparts began collecting feedback from surgeons and hospitals through its annual evaluation survey. From the VASM survey results, positive feedback on the value of the case note review booklets, as well as a perceived improvement in the local level of care was reported.20 Various examples of improvements from learning outcomes in other clinical audits and registries to which VASM could refer included the observed changes in surgical practices in Western Australia,21 the development of risk scores specific to Australian patients who underwent cardiovascular surgery,22 the surgical outliers process embedded into the vascular audit system,23 providing training programmes within the health setting,24 25 improving communication between medical and surgical teams,26–28 ensuring surgeons in Australia meet high standards of clinical practice and patient care29 and the New Zealand perioperative mortality review committee activity.30 A recent review by Watters et al31 has shown that there is value in establishing a predictive model to reduce perioperative mortality, but acknowledges that unless mortality audits examine the circumstances surrounding deaths improvements will not occur. This has been well demonstrated in the original Scottish Audit of Surgical Mortality, which formed the basis for the establishment of ANZASM.32 The Scottish audit has examined 9 years of data collection and found that adverse events were more commonly due to failures in system processes than to individual clinical errors. No other mortality audit has published a comprehensive analysis of differences between specialties but North et al33 did analyse delay in diagnosis and found that this factor was highest in general, cardiothoracic and vascular surgery. This correlates well with our data which found that delay in treatment was highest in general than vascular surgery.

The main strength of this audit is its independent blinded peer review. The quality and depth of feedback to stakeholders is extensive. Limitations remain data quality, but this will improve when an electronic-only data capture method is adopted in 2014. Also only aggregate data can be provided to hospitals because of quality privilege legislation restriction. Although VASM is restricted to the analysis of deaths under the care of a surgeon and does not include morbidity or near misses, dissemination of outcomes of data collected and analysed is associated with a reduction in the number of clinical management issues.

We have highlighted the differences in clinical management issues between specialties and demonstrated that the audit is associated with a reduction in these issues over successive audit periods. Focusing on specific areas of clinical management issues by specialties can be expected to continue this trend, with a positive impact on patient care.