Application of the Boston Technical Performance Score to intraoperative echocardiography.

BACKGROUND: The Technical Performance Score (TPS) developed by Boston Children's Hospital showed surgical outcomes correlate with adequacy of technical repair when implemented on pre-discharge echocardiograms. We applied this scoring system to intraoperative imaging in a tertiary UK congenital heart surgical centre.
METHODS: After a period of training, intraoperative TPS (epicardial and/or transesophageal echocardiography) was instituted. TPS was used to inform intraoperative discussions and recorded on a custom-made database using the previously published scoring system. After a year, we reviewed the feasibility, results and relationship between the TPS and mortality, extubation time and length of stay.
RESULTS: From 01 September 2015 to 04 July 2016, there were 272 TPS procedures in 251 operations with 208 TPS recorded. Seven patients had surgery with no documented TPS, three had operations with no current TPS score template available. Patients left the operating theatre with TPS optimal in 156 (75%), adequate 34 (16%) and inadequate 18 (9%). Of those with an optimal score on leaving theatre, ten had more than one period of cardiopulmonary bypass. All four deaths <30 days after surgery (1.9%) had optimal TPS. There was a statistically significant difference in extubation times in the RACHS category 4 patients (3 days vs 5 days, P < 0.05) and in PICU and total length of stay in the RACHS category three patients (2 and 8 days vs 12.5 and 21.5 days respectively) if leaving theatre with an inadequate result.
CONCLUSIONS: Application of intraoperative TPS is feasible and provides a way of objectively recording intraoperative imaging assessment of surgery. An 'inadequate' TPS did not predict mortality but correlated with a longer ventilation time and longer length of stay compared to those with optimal or adequate scores.

Introduction

An integral component of any congenital cardiac surgical unit is analysis of patient outcomes both in terms of morbidity and mortality. In the United Kingdom, all congenital heart surgery results are reported centrally and externally audited to gauge both centre-specific and national outcome. As mortality rates have fallen over the past several decades, quality markers and assessment of morbidity have assumed greater importance. This is a complex task which must take account of the cardiac lesion, comorbidities and pre- and postoperative care. Given that many factors can impact on patient outcome, objective assessment of each component of a patient’s care is vital. One important aspect is an analysis of the ‘technical’ result of the surgical procedure.

The Technical Performance Score (TPS) was developed at Boston Children’s Hospital to assess the adequacy of surgical repair (1, 2). It is based on objective measures of key components of the repair to try and minimise inter-operator variation and focus the postoperative assessment on key areas. Each surgical procedure is divided into several components, and each component is scored as either optimal, adequate or inadequate. The overall score of the operation is the worst score in any individual component. Thus, an operation with some ‘optimal’ components but one ‘inadequate’ component is scored as inadequate overall. This score has been extensively reported to have a strong association with outcome (1, 3, 4, 5, 6, 7, 8) but all the key reports have come from the single centre where the score was developed, and predominantly performed on postoperative echocardiograms prior to the patient’s discharge from hospital. Ideally, however, it might be better to apply such a scoring system intraoperatively to inform adequacy of surgery at a time when the surgeon can immediately revise the repair.

We report our initial experience of introducing the TPS during intraoperative echocardiography at a designated tertiary congenital cardiac surgical centre in the United Kingdom. We aimed to investigate the feasibility of implementing the TPS on intraoperative imaging rather than pre-discharge imaging. Furthermore, we examined the range of TPS scores across a wide range of cardiac lesions and impact of TPS score on patient outcome in terms of mortality, time to extubation and length of stay. Cardiac lesions were stratified according to Risk Adjustment in Congenital Heart Surgery (RACHS-1) scores to assess the predictive value of the TPS across a range of severity.

Methods

Institutional ethical approval was obtained. The TPS was introduced in January 2015 and applied with no modifications from the Boston Children’s Hospital TPS other than it was performed and calculated intraoperatively based on transoesophageal or epicardial echocardiography. At our centre, there are around 400 cases performed each year (300 on cardiopulmonary bypass) (https://nicor4.nicor.org.uk/CHD/an_paeds.nsf/vwContent/home). Intraoperative imaging is offered as a consultant-led service and the type of imaging, for example, transesophageal versus epicardial was a decision between the surgical and cardiology team based on the patient’s size, cardiac lesion and clinical status. All consultant paediatric cardiologists undertake intraoperative imaging at our centre.

There was a run-in period where a database was designed and trialled, and staff familiarised themselves with the system. Intraoperative echocardiography was offered for all cases. From 01 September 2015 completion of TPS became mandatory for all surgical procedures within the paediatric cardiology department. If intraoperative imaging was not requested by surgeons, the database was completed to document this. Completion of the database was in addition to contemporaneous formal reporting of all echocardiograms and decision making. There were monthly reviews of the completion of the data with any outstanding entries into the database completed from the formal reports and echocardiographic review.

A list of all operations performed in the institution was reviewed against the TPS database. Cases excluded from TPS were pexy of the aorta or other vessels or division of vascular ring (because there is no echocardiographic means of assessment) and isolated patent arterial duct (PDA) ligation as these were almost universally extremely pre-term infants.

With respect to the TPS, only areas intervened on are scored, and any iatrogenic damage downgrades the surgical procedure to ‘inadequate’. However, the repair of the damaged area can itself be ‘optimal’. If no intraoperative imaging was requested, the reason was documented if known. The patient demographics (including age and weight), operations (and RACHS-1 score) (9), re-interventions and life status were recorded, as was the extubation date, total length of stay and stay on the paediatric intensive care unit (PICU).

Descriptive statistics are used with median and range stated. For comparison between groups, the Student t-test and Mann–Whitney tests were used.

Results

A total of 251 operations were performed at our unit in the study period. In 15 cases (6%) more than one run of cardiopulmonary bypass was undertaken. In 33 cases (13.1%), surgeons did not request intraoperative imaging and no scores were recorded. Those cases are shown in Supplementary Table 1 (see section on supplementary data given at the end of this article). Only one of these children has required reintervention (1.8 kg coarctation via thoracotomy needing an aortic arch balloon dilation 149 days postoperatively). There were three deaths in this group, all from haemodynamic collapse (not blockage) after arterial shunt. Based on intention-to-record TPS, this was achieved in 208 cases (96.7%). In seven cases (3.3%), there was a failure to record a TPS despite intraoperative imaging being undertaken.

Four patients had procedures that currently have no TPS criteria (aortopulmonary window in two patients, comprehensive Norwood stage 2 and ventricular septal defect (VSD) enlargement) so no score could be entered. One hundred patients had epicardial echocardiography, 103 had transoesophageal echocardiography and five patients had both. Patients who underwent epicardial echocardiography had a median (range) weight of 4.4 kg (2.2–15.7 kg). For transoesophageal echocardiography, the median (range) weight was 14.7 kg (4.7–70 kg). The weight of patients having epicardial echocardiography was significantly lower than those having transoesophageal echocardiography (P < 0.05).

Of the 208 completed TPS scores, 156 (75%) were given optimal, 34 (16%) adequate and 18 (9%) inadequate. There were four 30-day deaths (1.9%), all had a TPS of optimal. Table 1 shows the TPS scores by anatomical lesion. This table further indicates in which groups any deaths or late reintervention was observed. For atrial septal defect (ASD), Norwood and superior or total cavopulmonary completion, the TPS was uniformly optimal, but late reoperation or death occurred in the latter two groups despite an apparently good result from surgery. In the case of two ventricular septal defects (VSD) judged ‘inadequate’ on the basis of the diameter of the colour flow jet across a residual VSD, oxygen saturation runs performed intraoperatively confirmed a Qp:Qs of <1.5:1. These repairs were accepted and neither patient required reoperation.

Table 1

Operation and TPS score for key procedures.

	Optimal, n (%)	Adequate, n (%)	Inadequate, n (%)
ASD (11)	11 (100)	0 (0)	0 (0)
VSD (33)	29 (88)	2 (6)	2 (6)
AVSD (24)	19 (79)	2 (8)	3 (13)b
TOF (18)	10 (56)	7 (39)	1 (6)
ASO/ASO+ (11)	10 (91)	0 (0)	1 (9)
TAPVD/PAPVD (9)	5 (56)	0 (0)	4 (44)b
LVOTO (12)	7 (58)	3 (25)	2 (17)
Ross (10)	3 (30)	7 (70)	0 (0)
RVOTO/PVR (12)	8 (67)	4 (33)	0 (0)
Norwood/shunt (7)	7 (100)b	0 (0)	0 (0)
HF/Glenn/TCPC (22)	22 (100)a	0 (0)	0 (0)
Complex/other (39)	25 (64)aaab	9 (23)b	5 (13)

ASD, atrial septal defect; ASO, arterial switch operation; ASO+, arterial switch operation plus other abnormalities; AVSD, atrioventricular septal defect; CoA, coarctation of the aorta; HF, hemi-Fontan; LVOTO, left ventricular outflow tract obstruction; PAPVD, partial anomalous pulmonary venous drainage; PVR, pulmonary valve replacement; RVOTO, right ventricular outflow tract obstruction; TAPVD, total anomalous pulmonary venous drainage; TCPC, total cavopulmonary connection; TOF, tetralogy of Fallot; VSD, ventricular septal defect.

a30-day death; b30-day reintervention on the operated area (excluding pacemaker).

The highest incidence of an ‘inadequate’ score was following total anomalous pulmonary venous drainage (TAPVD) repair. This score was based on the mean Doppler gradient across the surgical anastomosis immediately after repair. In only one case was reoperation required and, in the others, the mean gradient fell with time. The majority of Ross operations were judged only ‘adequate’ as even mild aortic valve regurgitation downgrades the repair from optimal to adequate.

TPS inadequate

A total of 28 (13.5%) patients had an initially inadequate TPS score. In 15 of these cases, the surgeon revised the repair during the same surgery leading to an improvement in the TPS in 10/15 cases. The procedure was changed in three patients, in one no further TPS was measured and in the final case there was no changes in the TPS and so the result was accepted. In the remaining 13 cases, no further surgery was performed at that time and the patient returned to the ICU with an ‘inadequate’ TPS score (Fig. 1 and Table 2).

Figure 1

Patients who had an initial Technical Performance Score (TPS) of inadequate aortic stenosis (AS), ventricular septal defect (VSD).

Table 2

Patients leaving theatre with TPS ‘inadequate’.

Age at op (days)	Weight (kg)	Operation	Imaging	Comment	Reintervention
41	4.0	AVSD	Epicardial	Residual VSD	Redo VSD closure D14
1	3.1	TAPVD*	Epicardial	PV mean 9 mmHg, less at d/c	–
1	3.7	TAPVD	Epicardial	–	Redo to find confluence D1
202	5.7	PA-plasty	Epicardial	Residual branch PS – accepted	–
4680	36.0	AoV repair	TEE	Proceeded to AVR (AVR TPS 1)	–
121	5.4	AVSD	Epicardial	Residual VSD but low Qp:Qs on saturations run	–
6	3.3	ASO/VSD/arch	Epicardial	RPA 45 mmHg	–
4856	34.0	TV replacement	3D TEE	CHB requiring PPM	PPM in theatre D0
4805	52.0	PAPVD/ASD	3D TEE	Lower baffle >4 mmHg	–
1447	17.8	Sub AS	2D TEE	3× bypass – accepted	–
910	12.7	Double switch	2D TEE	Failed DS, failed BV so band	PPM D26
20	2.4	CoA (thorac)	Epicardial	Redo as sternotomy, no repeat TPS	–
787	7.4	LVOTO	2D TEE	–	PPM D30
90	3.6	AVSD	Epicardial	–	PPM D22
132	4.0	VSD	Epicardial	Low Qp:Qs and not seen post-op	–
125	6.3	TOF	Epicardial	No repeat echo done after redo	–
309	5.1	MV repair	2D TEE	–	–
20	3.5	TAPVD	Epicardial	–	–

AVR, aortic valve replacement; AS, aortic stenosis; ASD, atrial septal defect; ASO, arterial switch operation; AVSD, atrioventricular septal defect; CoA, coarctation of the aorta; LAVV, left atrioventricular valve; LVOTO, left ventricular outflow tract obstruction; MV, mitral valve; PA, pulmonary artery; PAPVD, partial anomalous pulmonary venous drainage; TAPVD, total anomalous pulmonary venous drainage; TOF, tetralogy of Fallot; TV, tricuspid valve; VSD, ventricular septal defect.

Two patients who left theatre with an ‘inadequate’ score required reintervention related to their operation (AVSD repair with residual VSD; TAPVD repair). Complete heart block requiring permanent pacing was seen in four patients (tricuspid valve replacement in HLHS; failed double switch with subsequent VSD enlargement and pulmonary artery band; extensive left ventricular outflow tract obstruction resection; complete atrioventricular septal defect). One patient had a high mean pressure gradient across the pulmonary venous confluence following TAPVD repair which resolved prior to discharge.

In the remaining patients, although scored as inadequate, the surgical result was accepted as the best that was felt to be technically achievable. None of these patients died within 30 days and all survived to hospital discharge.

Re-interventions <30 days on operated area

In patients who left theatre with a TPS of optimal or adequate, there were three re-interventions (Table 3):

Table 3

Re-interventions <30 days on the operated area in patients leaving theatre with ‘optimal’ or ‘adequate’.

Age at op (days)	Weight (kg)	Operation	Imaging	Reintervention
273	6.8	Nikaidoh	Epicardial (1)	Redo RCA implantation D1
29	3.6	MBTS	Epicardial (1)	Shunt redo (blocked) D1
64	3.5	Supra AS after CAT	Epicardial (2)	Dissection and freeing of LMB D24

(1) TPS optimal; (2) TPS adequate; (3) TPS inadequate.

AS, aortic stenosis; CAT, common arterial trunk; LMB, left main bronchus; MBTS, modified Blalock-Taussig shunt; RCA, right coronary artery; TPS, Technical Performance Score.

Right coronary artery re-implantation after Nikaidoh.

Shunt replacement/upsizing after an acute shunt blockage.

Surgical re-exploration to release adhesions causing left main bronchus obstruction following relief of supravalvar aortic stenosis in repaired common arterial trunk.

Impact of RACHS-1 score

There was no significant difference in RACHS-1 scores between the optimal, adequate and inadequate groups. When comparing extubation times between those leaving theatre with an optimal or adequate score with those leaving with an inadequate score, there was a significant difference in extubation times in the RACHS-1 category 4 patients. RACHS-1 category 3 patients had significantly longer PICU and overall length of stay if they left theatre with an inadequate score compared to an optimal or adequate score (Table 4).

Table 4

Extubation time, PICU and overall length of stay by RACHS-1 and TPS.

	Extubation time (days)	PICU LOS (days)	Total LOS (days)
Risk category 1
TPS optimal/adequate (12)	0 (0–2)	1 (1–7)	4.5 (3–10)
TPS inadequate (1)	0	1	3
P value	0.773	0.642	0.219
Risk category 2
TPS optimal/adequate (82)	1 (0–7)	2 (0–23)	5 (2–62)
TPS inadequate (6)	0.5 (0–4)	2.5 (1–5)	6 (4–23)
P value	0.850	0.231	0.476
Risk category 3
TPS optimal/adequate (71)	1 (0–26)	2 (0–63)a	8 (3–80)a
TPS inadequate (6)	3 (0–7)	12.5 (1–20)a	21.5 (6–39)a
P value	0.070	0.023a	0.011a
Risk category 4
TPS optimal/adequate (17)	3 (0–12)a	5 (1–15)	9 (3–25)
TPS inadequate (5)	5 (3–13)a	6 (1–16)	12 (8–50)
P value	0.024a	0.214	0.288
Risk category 6
TPS optimal/adequate (5)	6 (3–27)	10 (4–38)	17 (5–122)
TPS inadequate (0)	–	–	–
P value	–	–	–

Values given as median (range).

aStatistically significant.

LOS, length of stay; RACHS-1, risk adjustment for congenital heart surgery; TPS, Technical Performance Score.

Mortality

In the year prior to the study, the 30-day mortality rate was 2.6%. All four deaths <30 days after surgery (1.9%) had optimal TPS. In those that died, the median RACHS-1 risk category was 3. A review of the causes of death shows that only one was likely related to a technical issue: right coronary artery obstruction after Nikaidoh operation which became apparent over the first 24 h (the TPS score was ‘optimal’ in theatre but the patient developed myocardial ischaemia and coronary angiogram findings led to re-implantation of the right coronary artery the following day) (Table 5).

Table 5

Postoperative deaths.

Age at op (days)	Weight (kg)	Operation	Imaging (TPS)	Death post-op (days)	Mode of death
189	6.3	HF/Glenn	Epicardial (1)	3	Tachyarrhythmia
356	6.3	PA/VSD repair	Epicardial (1)	22	Sepsis
6	2.8	CAT	Epicardial (1)	8	PHT/intrinsic lung disease (ECMO)
273	6.8	Nikaidoh	Epicardial (1)	4	Coronrary issue leading to reoperation, ECMO and then brain injury

(1) TPS optimal; (2) TPS adequate; (3) TPS inadequate.

Common arterial trunk (CAT), extracorporeal membrane oxygenation (ECMO), hemi-Fontan (HF), pulmonary atresia with a ventricular septal defect (PA/VSD), pulmonary hypertension (PHT), Technical Performance Score (TPS).

Discussion

The TPS designed by Boston Children’s Hospital was readily able to be applied in our centre intraoperatively. Ninety-six point seven percent of cases assessed by operative echocardiography had a TPS score recorded. Development of TPS scores for operations not currently included would increase the implementation rate still further. Ninety-one percent of scores were optimal or adequate. The finding of an inadequate score, prompted surgical revision in 15/28 (53.6%) of cases resulting in improved score in 10/15 (66.7%). Our initiation of a structured system of reporting provided by TPS allowed imaging operators to become familiar with a specific menu of criteria covering most, but not all, congenital cardiac procedures. The approach has been well received by both surgical and cardiology teams to provide a degree of consistency in the metrics applied to assess each operation. At the outset, the introduction was discussed with both cardiology and surgical teams who were receptive to the approach. The use of the score provided a helpful guide at the end of surgical procedures and assisted intraoperative dialogue when residual lesions were identified as well as being useful when training junior cardiologists in intraoperative assessment.

The Boston approach was to use the score on pre-discharge echocardiograms, thus not allowing for further runs of bypass if problems were found. We have shown that intraoperative identification allowed further bypass and improvement of TPS. Additionally, pre-discharge echocardiography will not capture patients who have not survived to discharge, and so the true surgical result may not be known.

When key cardiac procedures were broken down the incidence of optimal result ranged from 30 to 100% (Table 1). Fifteen patients left the operating theatre with an ‘inadequate’ TPS score because it was agreed that the best technical result achievable had been obtained despite the residual lesions (Table 2). None of the patients in this group died within 30 days of surgery and all survived to hospital discharge.

Looked at from a different perspective, four patients died postoperatively during the study period (Table 5). All the patients who died left the operating theatre with a TPS score of optimal. Causes of death were arrhythmia, sepsis and underlying lung disease, with only one death from a neurological injury from ECMO which was required due to reoperation for coronary obstruction. Recognition of the limitations of echocardiography, on which the TPS is based, may inform decision-making to use alternative imaging modalities including CT, MRI or angiography in patients at high risk of shunt-related complications. This has been supported by the recent publication from the Boston group in Norwood patients (10).

With respect to the TPS scores obtained for each procedure, some operations notably ASD closure, arterial switch operation, Norwood operation and hemi-Fontan/Fontan procedures had uniformly optimal results. In contrast, other surgeries such as tetralogy of Fallot repair, Ross operation and repair of TAPVD had much worse scores. In tetralogy of Fallot most ‘adequate’ rather than ‘optimal’ scores were due to the right ventricular outflow tract Doppler gradient. However, our institutional practice was to accept intraoperative Doppler gradients of 2–3 m/s to avoid the need for transannular patch and reduce the longer-term incidence of pulmonary valve regurgitation. Currently none of these patients have required reintervention on their right ventricular outflow tract. The TPS for the Ross procedure does not allow for any intraoperative regurgitation in the autograft if the result were to be scored as optimal. Even trivial regurgitation which would be unlikely to be of significance would place the patient in the ‘adequate’ category. In repair of TAPVD patients often had high mean anastamotic Doppler gradients in theatre, which moved the repair to the ‘inadequate’ category but such gradients reduced in the first postoperative days. This finding may relate to the significant decompression of pulmonary congestion seen in theatre which typically settles during the postoperative course. As the TPS was developed on pre-discharge scans, it is likely not representative of the values seen in theatre for both regurgitation in Ross and pulmonary veins in TAPVD. These differences could also be compounded by the differing sensitivity for detection of very minor regurgitation/residual shunts on epicardial or transoesophageal echocardiography compared to transthoracic echocardiography.

When judging the significance of residual VSD – the TPS criteria are based on measurements of colour flow jet width. Currently if a TPS score is inadequate, we will routinely perform a paired oxygen saturation run to calculate the systemic to pulmonary blood flow ratio to aid in the decision making.

The use of the TPS score based on intraoperative echocardiographic findings did not predict 30-day mortality, but when risk stratified, there was a longer ventilation time in RACHS-1 risk category 4 patient and longer length of stay in RACHS-1 risk category 3 patients, consistent with reports from Boston (2). Length of stay has limitations as there may be other factors which prolong stay, such as ward bed availability and co-existent non-cardiac comorbidities. The approach however does provide a well-defined structure to intraoperative assessment but in its current form is not predictive of outcome, as this is not solely based on the technical surgical result alone.

Looking ahead we are aiming to potentially adjusting scores based on intraoperative values compared to those seen postoperatively on which the TPS was originally based and we will create scores for the procedures not already covered. Additionally, the score may be expanded to consider the rhythm in which the patient leaves theatre, as rhythms such as junctional ectopic tachycardia can have a significant impact on morbidity and mortality. Further prospective studies are required to analyse subgroups and institute new scores.

In summary, application of the TPS system is feasible intraoperatively in a UK setting. The system is still in evolution and we aim to perform future reviews. Through the institution of TPS, the intraoperative imaging service has further developed and the relationship with the surgical team and intraoperative dialogue grown stronger.

Declaration of interest

The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.

Funding

The authors (J M S and K P) acknowledge support by the Department of Health through the National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s & St Thomas’ National Health Service Foundation Trust in partnership with King’s College London and King’s College Hospital National Health Service Foundation Trust. The Division of Imaging Sciences receives also support as the Centre of Excellence in Medical Engineering (funded by the Wellcome Trust and Engineering and Physical Sciences Research Council (EPSRC); grant number WT 088641/Z/09/Z) as well as the British Heart Foundation Centre of Excellence (BHF award RE/08/03).