Prospective validation of classification of intraoperative adverse events (ClassIntra): international, multicentre cohort study.

OBJECTIVE: To prospectively assess the construct and criterion validity of ClassIntra version 1.0, a newly developed classification for assessing intraoperative adverse events.
DESIGN: International, multicentre cohort study.
SETTING: 18 secondary and tertiary centres from 12 countries in Europe, Oceania, and North America. PARTICIPANTS: The cohort study included a representative sample of 2520 patients in hospital having any type of surgery, followed up until discharge. A follow-up to assess mortality at 30 days was performed in 2372 patients (94%). A survey was sent to a representative sample of 163 surgeons and anaesthetists from participating centres. MAIN OUTCOME MEASURES: Intraoperative complications were assessed according to ClassIntra. Postoperative complications were assessed daily until discharge from hospital with the Clavien-Dindo classification. The primary endpoint was construct validity by investigating the risk adjusted association between the most severe intraoperative and postoperative complications, measured in a multivariable hierarchical proportional odds model. For criterion validity, inter-rater reliability was evaluated in a survey of 10 fictitious case scenarios describing intraoperative complications.
RESULTS: Of 2520 patients enrolled, 610 (24%) experienced at least one intraoperative adverse event and 838 (33%) at least one postoperative complication. Multivariable analysis showed a gradual increase in risk for a more severe postoperative complication with increasing grade of ClassIntra: ClassIntra grade I versus grade 0, odds ratio 0.99 (95% confidence interval 0.69 to 1.42); grade II versus grade 0, 1.39 (0.97 to 2.00); grade III versus grade 0, 2.62 (1.31 to 5.26); and grade IV versus grade 0, 3.81 (1.19 to 12.2). ClassIntra showed high criterion validity with an intraclass correlation coefficient of 0.76 (95% confidence interval 0.59 to 0.91) in the survey (response rate 83%).
CONCLUSIONS: ClassIntra is the first prospectively validated classification for assessing intraoperative adverse events in a standardised way, linking them to postoperative complications with the well established Clavien-Dindo classification. ClassIntra can be incorporated into routine practice in perioperative surgical safety checklists, or used as a monitoring and outcome reporting tool for different surgical disciplines. Future studies should investigate whether the tool is useful to stratify patients to the appropriate postoperative care, to enhance the quality of surgical interventions, and to improve long term outcomes of surgical patients. TRIAL REGISTRATION: ClinicalTrials.gov NCT03009929. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Over 310 million surgeries are conducted worldwide each year.1 One in six patients having elective surgery experiences at least one postoperative adverse event.2 The growing annual surgical volume1 3 and the increasing complexity of comorbid conditions in patients having surgery4 5 has caused a rise in adverse events related to surgery. Intraoperative adverse events are associated with adverse outcomes,6 7 8 9 10 but in contrast with postoperative complications,11 12 13 a prospectively validated instrument for standardised and transparent reporting of intraoperative adverse events is lacking. Uniform definitions of perioperative outcomes, however, are required for programmes to improve the safety of surgery, for benchmarking, and to adequately compare results from clinical research.14

In 2015, our group developed a CLASSification of Intraoperative Complications (ClassIntra version 1.0, formerly known as CLASSIC, supplementary table S1) in a Delphi process involving international, interdisciplinary experts, which was retrospectively validated.15 This tool defines intraoperative adverse events as any deviation from the ideal intraoperative course between skin incision and skin closure, and includes events related to surgery and anaesthesia. Similar to the widely used Clavien-Dindo classification for reporting postoperative complications (supplementary table S2),11 we sought to develop a generic classification, broadly applicable to all surgical disciplines, and including anaesthesia. To align with the validated Clavien-Dindo classification,16 17 CLASSIC was updated by including five severity grades, maintaining the decision criterion on the need for treatment and severity of symptoms.

The aim of this international, multicentre, prospective cohort study was to investigate the validity and practicability of the updated classification in many different surgical disciplines.

Methods

Study oversight

This investigator initiated validation study of ClassIntra version 1.0 consisted of two parts. An international, multicentre, prospective cohort study was conducted in 18 centres in 12 countries in patients having any type of surgery requiring hospital admission for at least one night. The objective was to assess construct validity. Because no gold standard exists for the severity of intraoperative adverse events, we assessed the relation between the ClassIntra grades and other indirect measures that we expected to be related.18 In particular, we investigated the stepwise increasing effect of the severity grades of ClassIntra and relevant postoperative outcomes, such as postoperative complications, length of hospital stay, and length of surgery. The second part involved a survey of 10 fictitious case scenarios describing intraoperative adverse events. The survey was sent at the beginning of the prospective cohort study to physicians from the same international centres to grade severity. The aim was to assess criterion validity and reliability by comparing severity ratings by physicians with benchmark ratings set by the core team.

Across all centres, collaborators were trained in standardised data collection. Local study investigators confirmed the validity of the data from their centres. The results are reported according to STROBE (strengthening the reporting of observational studies in epidemiology).19

Patients and study procedures (prospective cohort study)

The centres prospectively included consecutive patients or a representative group of patients having any type of surgery from any surgical discipline. Excluded were day case surgeries, procedures not taking place in the operating room, procedures with no anaesthesia, and follow-up procedures.

The surgical and anaesthesia team prospectively recorded the severity grade and details of intraoperative adverse events that occurred between skin incision and skin closure, according to ClassIntra version 1.0 (table 1), during team sign-out20 at the end of surgery. Disagreement was resolved by consensus. All patients were followed up until discharged from hospital and were assessed daily by the postoperative care team to identify and grade postoperative complications according to the Clavien-Dindo classification.11 Because this was a pragmatic cohort study, performed during daily clinical practice, the postoperative team could not be blinded to the intraoperative adverse events, but was not aware of the ClassIntra grade. Postoperative complications after discharge, including mortality within 30 days, were assessed during structured follow-up visits or telephone calls to patients or their relatives.

Table 1

ClassIntra version 1.0 classification of intraoperative adverse events. The classification defines intraoperative adverse events as any deviation from the ideal intraoperative course occurring between skin incision and skin closure. Any event related to surgery and anaesthesia during the index surgery must be considered and should be rated directly after surgery.* A requirement is that the indication for surgery and the interventions conform to current guidelines

Grade	Definition	Examples
Grade 0	No deviation from the ideal intraoperative course	—
Grade I	Any deviation from the ideal intraoperative course:• Without the need for any additional treatment or intervention• Patient with no or mild symptoms	• Bleeding: bleeding above average from small calibre vessel, self-limiting or definitively manageable without additional treatment than routine coagulation• Injury: minimal serosal intestinal lesion, not requiring any additional treatment• Cautery: small burn of the skin, no treatment necessary• Arrhythmia: arrhythmia (eg, extrasystoles) without relevance
Grade II	Any deviation from the ideal intraoperative course:• With the need for any additional minor treatment or intervention• Patient with moderate symptoms, not life threatening, and not leading to permanent disability	• Bleeding: bleeding from medium calibre artery or vein, ligation; use of tranexamic acid• Injury: non-transmural intestinal lesion requiring suture(s)• Cautery: moderate burn requiring non-invasive wound care• Arrhythmia: arrhythmia requiring administration of antiarrhythmic drug, no haemodynamic effect
Grade III	Any deviation from the ideal intraoperative course:• With the need for any additional moderate treatment or intervention• Patient with severe symptoms, potentially life threatening or potentially leading to permanent disability	• Bleeding: bleeding from large calibre artery or vein with transient haemodynamic instability, ligation or suture; blood transfusion• Injury: transmural intestinal lesion requiring segmental resection• Cautery: severe burn requiring surgical debridement• Arrhythmia: arrhythmia requiring administration of antiarrhythmic drug, transient haemodynamic effect
Grade IV	Any deviation from the ideal intraoperative course: • With the need for any additional major and urgent treatment or intervention• Patient with life threatening symptoms or leading to permanent disability	• Bleeding: life threatening bleeding with splenectomy; massive blood transfusion; stay at intensive care unit• Injury: injury of central artery or vein requiring extended intestinal resection• Cautery: life threatening burn injury by cautery leading to fire requiring intensive care treatment• Arrhythmia: arrhythmia requiring electroconversion, defibrillation, or admission to intensive care
Grade V	Any deviation from the ideal intraoperative course with intraoperative death of the patient	—

These events were not defined as intraoperative adverse events: sequelae, failures of cure, events related to the underlying disease, incorrect site or incorrect patient surgery, or errors in indication.

Data describing the patients enrolled in the study were anonymised before entry into a secure web based electronic database. The database had automated checks for plausibility, consistency, and completeness of the data entries. Further consistency checks were performed after recruitment was finished.

ClassIntra

ClassIntra version 1.0 was derived from CLASSIC,15 which had four severity grades, but the same definition depending on the intervention required to treat the adverse event and the severity of symptoms (supplementary table S1). The only difference in the updated ClassIntra is that grade II has been divided into two grades. The additional grade (new grade III) describes a moderate deviation from the ideal course, which requires moderate treatment and results in severe symptoms. The rationale for updating the classification was to align with the number of grades in the Clavien-Dindo classification to avoid misclassification in grading intraoperative or postoperative death (supplementary table S2).11 A retrospective (unpublished) pilot study showed high reliability for ClassIntra.

We secured the rights to the ClassIntra classification by filing a trademark with the Swiss Federal Institute of Intellectual Property. The trademark was extended to relevant countries for our activities by international registration at the World Intellectual Property Organisation.

Outcomes

The primary outcome was the association of the most severe intraoperative adverse events, according to ClassIntra, with the most severe postoperative complications, according to the Clavien-Dindo classification.11 Secondary outcomes were the association of the most severe intraoperative adverse events with the sum of all postoperative complications, calculated by the comprehensive complication index,12 postoperative length of hospital stay, length of surgery, and 30 day mortality. Thirty day mortality was defined as death within 30 days of surgery, either in hospital or after discharge.

All models for the primary and secondary outcomes were adjusted for confounders: American Society of Anesthesiologists physical status,21 as a summary measure of perioperatively relevant comorbidities; age; length of surgery; complexity of surgery, graded as one of five categories according to the British United Provident Association (BUPA)22 23; urgency of the procedure (planned or emergency); experience of the surgical and anaesthesia team (for a definition, see the statistical analysis plan in the supplementary file); and wound class.24 All confounders were defined according to biological plausibility and scientific rationale. Type of operation and complexity were recorded according to the BUPA classification.22 23 BUPA categorises all commonly performed surgical procedures into five complexity grades (minor, intermediate, major, major plus, and complex major; supplementary table S15) and has been used previously for scientific purposes.25 26

Inter-rater reliability and practicability of ClassIntra

An online anonymous survey (SurveyMonkey, San Mateo, CA, USA) of 10 fictitious case scenarios (supplementary table S3) describing intraoperative adverse events was sent to a representative sample of physicians from the participating centres. The scenarios were designed to cover a range of severity grades and medical specialties. The benchmark rating for each case scenario was determined by consensus of the core team. The physicians also rated the practicability of the instrument on a nine point scale, with end anchors of “very practical” and “not practical at all.”

Statistical analysis

A multivariable hierarchical proportional odds model was used to investigate the association between the most severe intraoperative and postoperative adverse event considering clustering within study centres. The proportional odds assumption was mainly assessed with the Brant test. Because of the sparseness of the data for the higher severity grades for intraoperative and postoperative adverse events, and breach of the proportionality assumption for the five categories of the Clavien-Dindo system, a three category classification (Clavien-Dindo grade 0, grades I-II, and grades III-V) was used. The rationale for these cut-offs was to distinguish an uneventful course from a minor or major deviation, and was in line with cut-offs used in most publications combining categories.16 The consistency of the coefficients was checked by performing an internal validation with bootstrapping methods, with 100 replications of samples from 2520 patients. A model with a binary endpoint (any v no postoperative complication) was used as a sensitivity analysis.

The association of the most severe intraoperative adverse event and the weighted sum of all postoperative complications (comprehensive complication index)12 was explored with a multivariable two level mixed effect regression model accounting for correlation between patients from the same study centre. Associations between the most severe intraoperative adverse event and the secondary endpoints of postoperative length of hospital stay and length of surgery were investigated in a multivariable two level mixed effect log linear regression model. The coefficients of the log linear regression models were back transformed to improve the interpretability of the results showing percentage changes27 in hospital days or duration of surgery for one unit increase in the corresponding covariate.

In the survey, the average agreement between the ratings of the physicians and the core team was calculated. Inter-rater reliability was assessed with the intraclass correlation coefficient based on the mean absolute agreement, estimated in a two way random effects model.28

The participating hospitals were a voluntary convenience sample. The planned sample size of 2500 patients allowed for robust estimation in the multivariable models, assuming at least one postoperative complication in 10% of the patients (that is, at least 250 events; for details, see the statistical analysis plan in the supplementary file).

Analyses were performed with Stata 14 (StataCorp, College Station, TX). Further details on data management and statistical analysis are in the statistical analysis plan (supplementary file).

Missing data

When an undefined complexity grade in the BUPA classification system (in 4% of procedures) was found, a grade corresponding to a similar procedure was used. This analysis was then compared with a complete case analysis. If no information was available on the presence of the anaesthesia resident or nurse, or whether they were in training or had graduated, we assumed that these team members were not present or were in training. There were no other missing values for outcomes or covariates, except for data on 30 day mortality in patients lost to follow-up.

ClassIntra study group

The ClassIntra study group consisted of a multidisciplinary, international team of anaesthetists, surgeons with a range of surgical specialties, epidemiologists, and statisticians with extensive expertise in perioperative care.

Patient and public involvement

This research was a multicentre collaboration to evaluate a newly developed classification for intraoperative adverse events. The data and software were not publicly available for analysis by patients or members of the public. Our protocol did not include patient and public involvement in the planning or implementation of this research. No patients were asked to advise on interpretation or writing up of the results. However, future research will evaluate the effect of intraoperative adverse events measured with ClassIntra on patient reported outcomes.

Results

Patient and procedural characteristics

Of 2640 patients screened, 2520 were enrolled in 18 secondary and tertiary centres in North America, Europe, and Oceania, representing 12 countries. All patients were recruited between 14 February 2017 and 27 July 2018. The median number of patients enrolled per site was 130 (interquartile range 100-176). The main reason for exclusion was day surgery (fig 1). Patients had a range of surgical procedures from different surgical disciplines, except ophthalmic surgery (table 2), with 44% (n=1090) of surgeries corresponding to the two highest complexity levels. Patient and procedural characteristics are described in table 2 (examples of the most frequent types of surgical procedures are in supplementary table S15). Baseline characteristics were similar in patients with and without intraoperative adverse events. The American Society of Anesthesiologists’ physical status classes, and the complexity of the surgical procedure, however, were higher in patients with an intraoperative adverse event.

Fig 1

Flow diagram of patients in the validation study of ClassIntra

Table 2

Patient and procedural characteristics for the whole study population and for subgroups with and with no intraoperative adverse events

	All patients (n=2520)	Patients with no intraoperative adverse events (n=1910)	Patients with intraoperative adverse events (n=610)	Missing data(No (%))
American Society of Anesthesiologists (ASA) physical status
ASA I	503 (20)	431 (23)	72 (12)	—
ASA II	1118 (44)	852 (45)	266 (44)	—
ASA III	805 (32)	565 (30)	240 (39)	—
ASA IV	92 (3.7)	62 (3.3)	30 (4.9)	—
ASA V	2 (0.1)	—	2 (0.3)	—
Age (median (IQR; range))
In adults (age ≥18; n=2340)	61 (46-72; 18-97)	60 (45-71; 18-97)	64 (49-74; 18-93)	—
In children (age <18; n=180)	9 (5-14; 0-17)	9 (5-14; 0-17)	8 (3-13; 0-16)	—
Sex
Male	1382 (55)	1038 (54)	344 (56)	—
Female	1138 (45)	872 (46)	266 (44)	—
Body mass index in adults (median (IQR); n=2340)	26 (23-30)	26 (23-30)	26 (23-30)	14 (1)
Weight in children (kg; median (IQR, range); n=180)	32 (19-53; 4-137)	33 (19-54; 4-137)	26 (17-37; 7.9-85)	—
Surgical discipline
Gastrointestinal surgery	1437 (57)	1085 (57)	352 (58)	—
Orthopaedic surgery and traumatology	297 (12)	260 (14)	37 (6.1)	—
Vascular surgery	169 (6.7)	121 (6.3)	48 (7.9)	—
Urology	134 (5.3)	109 (5.7)	25 (4.1)	—
Ear, nose, throat, and maxillofacial surgery	122 (4.8)	99 (5.2)	23 (3.8)	—
Neurosurgery and spine surgery	96 (3.8)	53 (2.8)	43 (7.1)	—
Cardiac surgery	73 (2.9)	41 (2.2)	32 (5.3)	—
Paediatric surgery	54 (2.1)	48 (2.5)	6 (1.0)	—
Gynaecology	46 (1.8)	29 (1.5)	17 (2.8)	—
Obstetrics	44 (1.8)	31 (1.6)	13 (2.1)	—
Reconstructive and hand surgery	26 (1.0)	21 (1.1)	5 (0.8)	—
Thoracic surgery	22 (0.9)	13 (0.7)	9 (1.5)	—
Urgency of the procedure
Planned	2153 (85)	1627 (85)	526 (86)	—
Emergency	367 (15)	283 (15)	84 (14)	—
Complexity of surgical procedure (original)
Minor	105 (4.2)	94 (4.9)	11 (1.8)	98 (4)
Intermediate	437 (17)	383 (20)	54 (8.9)
Major	790 (31)	613 (32)	177 (29)
Major plus	442 (18)	323 (17)	119 (20)
Complex major operation	648 (26)	431 (23)	217 (36)
Additional procedure	406 (16)	266 (14)	140 (23)
Wound class
Clean	1341 (53)	1033 (54)	308 (50)	—
Clean contaminated	1001 (40)	746 (39)	255 (42)	—
Contaminated	147 (5.8)	103 (5.4)	44 (7.2)	—
Dirty	31 (1.2)	28 (1.5)	3 (0.5)	—
Operating surgeon
Senior consultant	1662 (66)	1239 (65)	423 (69)	—
Junior consultant	544 (22)	427 (22)	117 (19)	—
Resident	314 (12)	244 (13)	70 (11)	—
Assisting surgeon	2371 (94)	1781 (93)	590 (97)	—
Senior consultant	555/2371 (23)	385/1781 (22)	170/590 (29)	—
Junior consultant	578/2371 (24)	432/1781 (24)	146/590 (25)	—
Resident	1238/2371 (52)	964/1781 (54)	274/590 (46)	—
Anaesthesia consultant present	2311 (92)	1746 (91)	565 (93)	—
Senior consultant	1481/2311 (64)	1112/1746 (64)	369/565 (65)	—
Junior consultant	830/2311 (36)	634/1746 (36)	196/565 (35)	—
Anaesthesia resident present	1443 (57)	1017 (53)	426 (70)	1 (0.0)
Anaesthesia nurse present	1866 (74)	1405 (74)	461 (76)	2 (0.1)
Proportion who graduated	1699 (91)	1268 (90)	431 (93)	3 (0.2)
Anaesthesia technique
General anaesthesia	2035 (81)	1558 (82)	477 (78)	—
Regional anaesthesia	198 (7.9)	158 (8.3)	40 (6.6)	—
Combined techniques	265 (11)	176 (9.2)	89 (15)	—
Monitored anaesthesia care	22 (0.9)	18 (0.9)	4 (0.7)	—

Data are number (%) unless indicated otherwise. IQR=interquartile range.

Intraoperative adverse events according to ClassIntra

Of 2520 patients enrolled, 610 (24%) experienced 778 intraoperative adverse events (table 3). The most severe intraoperative adverse event per patient was grade I in 161 patients (6.4%), grade II in 309 (12%), grade III in 122 (4.8%), and grade IV in 18 patients (0.7%). No intraoperative deaths occurred (table 3).

Table 3

Outcomes for the whole study population and for subgroups with and with no intraoperative adverse events

	All patients (n=2520)	Patients with no intraoperative adverse events (n=1910)	Patients with intraoperative adverse events (n=610)	Missing data(No (%))
All iAE (several events per patient possible)
0	1910 (71)	1910 (100)	—	—
I	198 (7.4)	—	198 (25)	—
II	417 (16)	—	417 (54)	—
III	142 (5.3)	—	142 (18)	—
IV	21 (0.8)	—	21 (3.0)	—
V	—	—	—	—
Total No of iAE per patient
0	1910 (76)	1910 (100)	—	—
1	485 (19)	—	485 (80)	—
2	92 (3.7)	—	92 (15)	—
3	27 (1.1)	—	27 (4.4)	—
4	3 (0.1)	—	3 (0.5)	—
5	2 (0.1)	—	2 (0.3)	—
6	1 (0.0)	—	1 (0.2)	—
Transfer after operation	677 (27)	403 (21)	274 (45)	—
Intermediate care unit	347/677 (51)	244/403 (61)	103/274 (38)	—
Intensive care unit	330/677 (49)	159/403 (39)	171/274 (62)	—
Unplanned	45/677 (6.7)	6/403 (1.5)	39/274 (14)	—
Length of surgery (minutes; median (IQR, range))	100 (60-170, 4-760)	90 (55-147, 4-760)	151 (93-240, 12-673)	—
Most severe postoperative complication per patient
0	1682 (67)	1367 (72)	315 (52)	—
I	349 (14)	257 (13)	92 (15)	—
II	277 (11)	162 (8.5)	115 (19)	—
IIIa	72 (2.9)	45 (2.4)	27 (4.4)	—
IIIb	55 (2.2)	40 (2.1)	15 (2.5)	—
IVa	53 (2.1)	23 (1.2)	30 (4.9)	—
IVb	7 (0.3)	3 (0.2)	4 (0.7)	—
V	25 (1.0)	13 (0.7)	12 (2.0)	—
All postoperative complications (several events per patient possible)
I	1106 (44)	747 (48)	359 (37)	—
II	983 (39)	557 (36)	426 (44)	—
IIIa	160 (6.4)	94 (6.1)	66 (6.8)	—
IIIb	105 (4.2)	72 (4.7)	33 (3.4)	—
IVa	110 (4.4)	51 (3.3)	59 (6.1)	—
IVb	20 (0.8)	9 (0.6)	11 (1.1)	—
V	25 (1.0)	13 (0.8)	12 (1.2)	—
Total No of postoperative complications per patient
None	1682 (67)	1367 (72)	315 (52)	—
1	412 (16)	288 (15)	124 (20)	—
2-5	329 (13)	198 (10)	131 (21)	—
>5	97 (3.9)	57 (3.0)	40 (6.6)	—
Reoperation
No	2421 (96)	1844 (97)	577 (95)	—
Any reoperation	99 (3.9)	66 (3.5)	33 (5.4)	—
Total No of reoperations
1	68 (69)	46 (70)	22 (67)	—
2	16 (16)	11 (17)	5 (15)	—
3	6 (6.1)	4 (6.1)	2 (6.1)	—
4	8 (8.1)	5 (7.6)	3 (9.1)	—
5	1 (1.0)	··	1 (3.0)	—
Postoperative length of hospital stay (days; median (IQR, range))	3 (2-6, 0-191)	3 (1-5, 0-106)	6 (3-9, 1-191)	—
Total length of hospital stay (days; median (IQR, range))	4 (2-8, 1-193)	3 (2-7, 1-113)	7 (3-11, 1-193)	—
Stay during postoperative course	68 (2.7)	40 (2.1)	28 (4.6)	—
Intermediate care unit	18 (26)	15 (38)	3 (11)	—
Intensive care unit	50 (74)	25 (63)	25 (89)	—
30 day mortality	26 (1.1)	13 (0.7)	13 (2.1)	148 (5.9)

Data are number (%) unless stated otherwise. IQR=interquartile range; iAE= intraoperative adverse events.

Postoperative complications

During the postoperative course, 2509 complications were seen in 838 patients (33%). Table 3 gives details of the severity grades and number of postoperative complications.

Multivariable analysis showed a gradual increase in the odds of a more severe postoperative complication with an increasing grade of the most severe intraoperative adverse event (fig 2; supplementary table S5). Consistently, a stepwise increase in the sum of all postoperative complications (comprehensive complication index) with an increasing grade of ClassIntra was seen (supplementary table S6).

Fig 2

Multivariable hierarchical proportional odds model for the most severe postoperative complications according to the Clavien-Dindo classification (three categories for outcome: no postoperative complications, grades I-II, and grades III-V), adjusted for the most relevant confounders. ASA=American Society of Anesthesiologists’ physical status level

More secondary outcomes

Median postoperative length of hospital stay was 3 days (interquartile range 2-6) in all patients, and was doubled in patients with at least one intraoperative adverse event (median 6 days; interquartile range 3-9 days) compared with patients with no intraoperative adverse events (3; 1-5 days) (table 3). In the multivariable analysis, median postoperative length of hospital stay increased with each ClassIntra grade by 6.1% (95% confidence interval 2.8 to 9.3%; supplementary table S7).

The median length of surgery was 100 minutes (interquartile range 60-170) with a maximum length of 760 minutes (table 3). The length of surgery was longer in patients experiencing an intraoperative adverse event, with an increase of 16% in median length of surgery with each grade increase in ClassIntra (95% confidence interval 13% to 18%; supplementary table S8). Thirty day follow-up was available for 2372 (94%) patients (fig 1). Mortality at 30 days was 1.1% (n=26), with a higher incidence in patients with an intraoperative event (table 3 and table S9).

Survey

The response rate to the survey was 83% (136/163 physicians). About a third of respondents were anaesthetists and two thirds were surgeons from various surgical disciplines with a range of years’ experience (supplementary table S4). Reliability among physicians showed an intraclass correlation coefficient of 0.76 (95% confidence interval 0.59 to 0.91). Overall agreement with the ratings of the core team was 61% (interquartile range 42-71%; supplementary figure S1), with deviation usually explained by physicians rating one grade lower than the core team. Physicians rated the practicability of ClassIntra as 6 out of 9 (interquartile range 5-7).

Discussion

Principal findings

This international multicentre study showed that ClassIntra is a valid and standardised tool for assessing intraoperative adverse events, with high construct validity, given the increasing effects of severity grades on important patient outcomes. ClassIntra also showed good criterion validity and practicability across different surgical disciplines.

The association between severe intraoperative and postoperative complications, well established in abdominal surgery,6 8 was replicated in a heterogeneous population of surgical patients from a range of surgical disciplines. The association of intraoperative adverse events and postoperative length of hospital stay might have been mediated by postoperative complications (that is, more postoperative complications resulted in a longer stay in hospital after operation). The association between intraoperative adverse events and length of surgery was most likely related (that is, more adverse events might have prolonged surgery, which in turn could have led to more intraoperative adverse events).

Strengths and limitations of the study

ClassIntra is a comprehensive classification, with a patient centred perspective, because it records any event related to anaesthesia or surgery, and grades the outcomes according to those relevant to the patient, the symptoms caused by the event, and the measures taken to treat the event. ClassIntra was derived from CLASSIC, which was developed by an interdisciplinary, international group of anaesthetists, surgeons, epidemiologists, and statisticians.15 ClassIntra aligns with the Clavien-Dindo classification of postoperative complications,11 allowing the reporting of intraoperative and postoperative adverse events.

Our robust study design, based on a large sample size involving a range of surgical disciplines, and on prospective multicentre data collection, implies good generalisability. Good generalisability always introduces some heterogeneity, which was accounted for by adjusting the analyses for the complexity of the surgical procedure and patient related factors. Whether the results are mainly driven by the largest subgroup of patients having an intra-abdominal procedure will be investigated in a secondary analysis comparing the results in several subgroups. The primary advantage of a broad classification is that it increases uniformity in reporting intraoperative adverse events and allows comparison of results between different centres and times. In our study, details of each intraoperative adverse event were collected. Our group is currently investigating how the additional information can be used to support quality improvement in the future. We therefore encourage adding details about the events and intervention specific intraoperative outcomes of interest to the ClassIntra grade, whenever possible within a framework of standardised core outcomes.29

To keep it practical and simple, the classification uses generic definitions of the five severity grades; individual judgment is necessary for interpretation. Similarly, defining an event as deviation from the ideal intraoperative course requires personal judgment. Inter-rater reliability in the survey was strong and disagreement was mostly by one grade. We found the inter-rater reliability results reassuring, given that the physicians had various backgrounds and were grading intraoperative adverse events in different surgical disciplines and in anaesthesia. In clinical practice and in our prospective cohort study, joint assessment by surgeons and anaesthetists during sign-out20 should lower this variability and the risk of under-reporting and downgrading of intraoperative adverse events. As clinicians become more familiar with ClassIntra, we believe the system will be easier to use, similar to the experience with the Clavien-Dindo classification after its introduction into routine clinical practice. Generally, different professional backgrounds, levels of experience, and cultural diversity among physicians might have explained the practicability ratings of less than five in 25% of the survey participants and ratings greater than seven in another 25%.

Because our project was conducted as part of daily clinical practice, having an independent person to record intraoperative and postoperative events would not have been feasible, and blinding of those assessing postoperative outcomes would have been unethical. The direction of the potential bias is unclear, however, as under-reporting and over-reporting could, theoretically, occur. Under-reporting and downgrading of complications is a problem in healthcare settings and should be handled with a positive, non-discriminative learning culture. A classification for complications should be sensitive enough to monitor and detect rating variability caused by downgrading.30 31

Our secondary endpoints included length of hospital stay. We acknowledge that this outcome might be affected by the variables included in the model (that is, the most severe intraoperative adverse event and the confounding variables) and also by cultural norms, logistical issues, and economic considerations. This analysis should be viewed as supporting evidence as it showed consistent stepwise increasing association in alignment with all other postoperative endpoints.

Findings in relation to other studies

Other classification systems for intraoperative adverse events have been published.32 Kaafarani and colleagues8 classified intraoperative adverse events into six grades. Grades I-IV depend on the intervention required to treat the intraoperative adverse events and the deviation from the planned procedure. Grade V refers to an intraoperative injury missed during surgery and requiring re-operation within seven days. Grade VI refers to intraoperative death. This classification was based on a retrospective analysis of routinely collected data in selected patients with accidental puncture or laceration. Although their classification approach differs considerably from ClassIntra, the authors confirmed an association between major intraoperative and postoperative adverse events.8 Our study was based on prospective data in unselected patients from a range of surgical interventions and considered any event related to surgery or anaesthesia. We did not include re-operation because of the difficulty in determining if an intraoperative injury or a postoperative complication was responsible for the re-operation.

A classification of surgical intraoperative adverse events for laparoscopic surgery was proposed, with five grades similar to ClassIntra, but excluded adverse events related to anaesthesia.33 Others used readily available information, such as quality indicators of the intraoperative period, to predict postoperative outcome. Coulson and colleagues estimated the acute risk change in the probability of death before and after operation based on two validated risk scores.34 35 The 10 point Apgar score for surgery was developed in patients having a colectomy and was evaluated in a broader surgical population.36 37 Prediction of postoperative complications was based on estimated blood loss, lowest mean arterial pressure, and heart rate. Although both scores showed good predictive ability, they provided only a surrogate marker for adverse intraoperative events.

Bothner and colleagues developed an incidents, events, and complications scale related to anaesthesia, based on a set of 63 events with five ascending severity grades depending on the effect on postoperative care and outcome.38 The scale was implemented for quality assurance and covered the whole period of anaesthesia, from induction to discharge to the ward or to the intensive care unit. Their reported rate of 22% adverse events was relatively higher than our findings where events related to anaesthesia and surgery were taken together, which could partly be explained by the restricted period covered in our study.

Implications for clinicians or policy makers

Patient safety in the perioperative period has gained considerable interest and successful interventions, such as the reduction in morbidity and mortality after introduction of a surgical safety checklist, have been seen.20 ClassIntra covers a major unmet need, as the first prospectively validated classification system that is simple and generally applicable to all surgical disciplines, to our knowledge. It takes a holistic view by integrating any adverse event, regardless of origin. Also, because innovative surgical techniques are increasingly complex and implemented quickly, preventing morbidity during the implementation period is important.39 ClassIntra should be included in routine practice to alert the postoperative care team to patients with intraoperative adverse events and at risk of further postoperative complications.

Future research

We acknowledge that our definition of intraoperative adverse events included only the time between skin incision and closure. Induction of and emergence from anaesthesia were not covered, which can have risks for more adverse events. Assessing the construct validity of a new classification is an ongoing process, which requires confirmation by more convergent and discriminative validity assessments to increase credibility. In future research, we will evaluate the properties of ClassIntra extending the assessment period to include the periods related to anaesthesia (that is, induction and emergence), focusing on patient reported outcomes, thus expanding the hypothetical construct for assessing validity.

ClassIntra can be used in clinical research as a defined core outcomes measure as it provides standard definitions, allowing accurate communication of research results and comparisons across research studies.11 40 ClassIntra can help evaluate the origin of the complications during the surgical procedure, providing more granularity to the complication profile for each procedure or surgical unit. Also, ClassIntra can be used in critical incidence reporting systems,41 for measuring individual performance in educational settings and system performance for benchmarking by institutions, and for outcome research.Future implementation studies should evaluate this tool in daily clinical practice for risk stratification of postoperative care or for planning resources and hospital capacities, in view of the increase in postoperative length of stay with the increasing severity of the ClassIntra grade. For day surgeries, the risk of perioperative complications is minimal. Future research should investigate whether assessing intraoperative complications is associated with early visits to the general practitioner or readmission.

Conclusion

ClassIntra version 1.0 is a validated and standardised tool to quantify and qualify intraoperative adverse events in clinical practice and research. It showed good reliability and practicability for all surgical disciplines and for anaesthesia. We reproduced the well established association between the severity grade for intraoperative adverse events and important postoperative outcomes. Hence ClassIntra is a useful tool for recording intraoperative adverse events and should be incorporated into the perioperative surgical safety checklist to highlight recurrent and potentially avoidable complications. ClassIntra could improve the technical aspects of intraoperative care and provide clarity in audits of practice.

Intraoperative adverse events are a concern for patients, anaesthetists, and surgeons because of their association with adverse postoperative outcomes

In contrast with postoperative complications, prospectively validated systems for reporting intraoperative adverse events are lacking

Uniform definitions and transparent reporting of intraoperative adverse events are required for programmes to improve the safety of surgery and to compare results from clinical research

ClassIntra is the first prospectively validated classification system with good reliability and practicability that includes intraoperative adverse events related to surgery and anaesthesia

ClassIntra covers a major unmet need as it provides a standardised instrument to quantify and qualify intraoperative adverse events in clinical practice and clinical research for a range of surgical disciplines and for anaesthesia