Outcome Reporting in Cardiac Surgery Trials: Systematic Review and Critical Appraisal.

BACKGROUND: There is currently no accepted standard for reporting outcomes following cardiac surgery. The objective of this paper was to systematically review the literature to evaluate the current use and definition of perioperative outcomes reported in cardiac surgery trials. METHODS AND
RESULTS: We reviewed 5 prominent medical and surgical journals on Medline from January 1, 2010, to June 30, 2014, for randomized controlled trials involving coronary artery bypass grafting and/or valve surgery. We identified 34 trials meeting inclusion criteria. Sample sizes ranged from 57 to 4752 participants (median 351). Composite end points were used as a primary outcome in 56% (n=19) of the randomized controlled trials and as a secondary outcome in 12% (n=4). There were 14 different composite end points. Mortality at any time (all-cause and/or cardiovascular) was reported as an individual end point or as part of a combined end point in 82% (n=28), myocardial infarction was reported in 68% (n=23), and bleeding was reported in 24% (n=8). Patient-centered outcomes, such as quality of life and functional classification, were reported in 29% (n=10). Definition of clinical events such as myocardial infarction, stroke, renal failure, and bleeding varied considerably among trials, particularly for postoperative myocardial infarction and bleeding, for which 8 different definitions were used for each.
CONCLUSIONS: Outcome reporting in the cardiac surgery literature is heterogeneous, and efforts should be made to standardize the outcomes reported and the definitions used to ascertain them. The development of standardizing outcome reporting is an essential step toward strengthening the process of evidence-based care in cardiac surgery.

Outcome measures vary between trials, particularly in the field of cardiac surgery, for which there is currently no widely accepted standard for reporting postoperative adverse events. The lack of uniformity has hindered the ability to compare and synthesize findings across different trials. Efforts to perform meta-analyses in cardiac surgery have often been impeded by the heterogeneity of pooled outcomes. Furthermore, generic composite outcomes traditionally used in cardiology such as major adverse cardiac events (MACE; death, myocardial infarction [MI] with or without revascularization) and major adverse cardiovascular and cerebrovascular events, or MACCE (death, MI, stroke), are neither designed nor adapted for cardiac surgery because they do not reflect other important postsurgical complications. The Society of Thoracic Surgeons (STS) has provided definitions of postsurgical complications and a surgery-specific composite outcome consisting of in-hospital death, stroke, prolonged ventilation, acute kidney injury, deep sternal wound infection, or reoperation1; however, the extent to which this composite outcome has been adopted for use in clinical trials is unknown. The appropriateness of combining adverse events with differing clinical impacts (eg, stroke and wound infection) has also been questioned.2

There is a pressing need to improve and homogenize outcome reporting in cardiac surgery, similar to what was successfully achieved in other fields such as the RIFLE classification3, the Bleeding Academic Research Consortium4 and the Transcatheter Valve Academic Research Consortium (VARC).5 The objective of this paper was to review the selection and definition of individual and composite outcomes reported in the recent body of randomized controlled trials in cardiac surgery. With this knowledge, stakeholders would be better equipped to develop recommendations for standardized outcome reporting adapted to cardiac surgery.

Methods

Search Strategy

The PubMed search string “Cardiac Surgical Procedures” (MeSH) AND “Randomized Controlled Trial” (Publication Type) was used to identify cardiac surgery randomized controlled trials published between January 1, 2010, and June 30, 2014. Our search was limited to 5 high-impact journals in general medicine, cardiovascular medicine, and cardiothoracic surgery6: New England Journal of Medicine, Circulation, Journal of the American College of Cardiology, Annals of Thoracic Surgery, and The Journal of Thoracic and Cardiovascular Surgery. Reference lists from retrieved manuscripts were hand searched to supplement the PubMed search.

Selection Criteria

Studies were included if the study design was randomized and the study population was undergoing coronary artery bypass grafting and/or heart valve repair or replacement surgery. Studies were excluded if the primary outcome was not a clinical event (eg, the primary outcome was a biomarker) or if the study population was pediatric or focused on congenital heart disease. Case reports, case series, editorials, reviews, and post hoc analyses of randomized controlled trial data were also excluded.

Data Extraction

For each article meeting inclusion criteria, the following variables were extracted: first author, journal, year of publication, trial name and registration, sample size, study population, intervention, control, primary outcomes, secondary outcomes, and duration of follow-up. In addition, the operational definitions used to ascertain MI, stroke, prolonged ventilation, acute renal injury, and bleeding were extracted. Patient-centered outcomes included postoperative pain, quality of life, mood, neurocognitive function, and New York Heart Association functional class. Health care resource utilization included hospital and intensive care unit length of stay and cost analyses.

Analysis

Studies were reviewed, and data were extracted in duplicate by 2 independent observers (M.G., L.D.); disagreements were resolved by consensus. The primary and secondary outcomes were represented in tabular format and summarized according to the number and proportion of randomized controlled trials reporting each outcome measure. The Stata 13 software package (StataCorp) was used to organize the extracted data and to prepare summary statistics.

Results

Of 190 potentially relevant trials, 34 met the selection criteria and were included in our systematic review (Figure 1). Included trials were evenly distributed among the journals searched (Table 1). Sample sizes ranged from 57 to 4752 participants (median 351; quartiles 1 to 3: 198 to 699). Overall, 26 trials involved coronary artery bypass grafting only, 5 involved valve repair or replacement only, and 3 involved a combination. The maximum duration of follow-up for outcome surveillance ranged from 5 to 14 days (median 7.5 days) in 6 trials, from 30 days to 1 year (median 365 days) in 19 trials, and was >1 year (median 1825 days) in 9 trials.

Figure 1

Flow diagram for search strategy. CABG indicates coronary artery bypass grafting.

Table 1

Summary of Trials Meeting Inclusion Criteria

First Author	Trial Name or Identifier	Journal and Year	N	Intervention	Control	Primary Outcomes
Adams et al 201411	NCT01240902	NEJM 2014	795	TAVR	SAVR	Mortality (1 year)
Morice et al 201312	SYNTAX	Circ 2014	1800	CABG	PCI	Composite: mortality, MI, CVA, revasc. (5 years)
Chocron et al 201313	MOTIV CABG	ATS 2013	361	Escitalopram after CABG	Placebo after CABG	Composite: mortality, MI, low CO syndrome, ventilation >24 hours, reintubation, brain injury, delirium, AKI, pneumonia, sepsis, DSWI, heart failure, hospitalization, reoperation (1 year)
Diegeler et al 201314	GOPCABE	NEJM 2013	2539	Off-pump CABG	On-pump CABG	Composite: mortality, MI, CVA, AKI requiring dialysis, revasc. (30 days, 1 year)
Kamalesh et al 201315	VA CARDS	JACC 2013	198	PCI	CABG	Composite: mortality, MI (2 years)
Karkouti et al 201316	NCT00914589	JTCS 2013	409	Recombinant factor XIII after cardiac surgery	Placebo after cardiac surgery	Blood transfusions (7 days)
Lamy et al 201317	CORONARY	NEJM 2013	4752	Off-pump CABG	On-pump CABG	Composite: mortality, MI, CVA, AKI requiring dialysis (30 days); Composite with revasc. (5 years)
Sezai et al 201318	UMIN000004537	ATS 2013	367	Carperitide during CABG	Placebo during CABG	Composite: mortality, MI, CVA, revasc., heart failure, hospitalization (1 year)
Shi et al 201319	NCT01596738	ATS 2013	120	Tranexamic acid during CABG	Placebo during CABG	Blood transfusions (perioperative)
Bokesch et al 201220	CONSERV-2	JTCS 2012	218	Tranexamic acid during CABG	Ecallantide during CABG	Composite: MI, blood transfusions, chest tube drainage, creatinine change (30 days)
Deja et al 201221	—	JTCS 2012	390	Aspirin before CABG	Placebo before CABG	Blood loss and chest tube drainage (12 hours)
Desai et al 201222	—	JTCS 2012	189	Strict glucose control after CABG	Liberal glucose control after CABG	Composite: mortality, AKI, DSWI, ventilation >24 hours, pneumonia, length of stay, AF; time to target glucose range (30 days)
Farkouh et al 201223	FREEDOM	NEJM 2012	1900	PCI	CABG	Composite: mortality, MI, CVA (30 days, 1 year)
Houlind et al 201224	DOORS	Circ 2012	900	Off-pump CABG	On-pump CABG	Composite: mortality, MI, CVA (30 days)
Kang et al 201225	EASE	NEJM 2012	57	Early surgery for endocarditis (<48 hours)	Usual care for endocarditis	Composite: mortality, clinical embolic event (6 weeks)
Lemma et al 201226	On-Off	JTCS 2012	411	Off-pump CABG	On-pump CABG	Composite: mortality, MI, CVA, AKI, ARDS, reoperation for bleeding (30 days)
Mannacio et al 201227	—	ATS 2012	230	IABP for 12 hours before CABG	IABP for 2 hours before CABG	Mortality (in hospital)
Sezai et al 201228	UMIN000002489	JTCS 2012	105	Landiolol IV with or without oral bisoprolol after CABG	No beta blocker after CABG	AF (1 week)
Torina et al 201229	—	JTCS 2012	60	Modified ultrafiltration after CABG	Usual care after CABG	Blood transfusions, chest tube drainage, hospital and critical care length of stay (48 hours)
Bonow et al 201130	STITCH	NEJM 2011	601	CABG	Medical therapy	Mortality (1 year)
Feldman et al 201131	EVEREST II	NEJM 2011	279	Percutaneous MV repair	Surgical MV repair or replacement	Composite: mortality, MI, CVA, AKI, DSWI, ventilation >48 hours, reoperation GI complication, AF, sepsis, transfusion ≥2 units (30 days) Composite: mortality, reoperation, severe MR (30 days)
Kourliouros et al 201132	ISRCTN41309956	JTCS 2011	104	Atorvastatin high dose after CABG or SAVR	Atorvastatin low-dose after CABG or SAVR	AF (in hospital)
Mehta et al 201133	PREVENT-IV	Circ 2011	1034	Edifoligide before treatment to venous grafts	Placebo before treatment to venous grafts	Composite: mortality, MI, revasc. (5 years) Venous graft failure (1 year)
Sezai et al 201134	NU-HIT for CRF	JACC 2011	303	Carperitide during CABG	Placebo during CABG	AKI (1 year)
Smith et al 201135	PARTNER	NEJM 2011	699	TAVR	SAVR	Mortality (1 year)
Wimmer-Greinecker et al 201136	NCT00985634	ATS 2011	110	CABG with thermosensitive polymer	CABG with conventional vessel loops	Composite: mortality, MI, graft occlusion, low CO syndrome (30 days)
Grossi et al 201037	RESTOR-MV	JACC 2010	165	CABG with ventricular reshaping	CABG with or without MV repair	Composite: mortality, MI, CVA, reoperation, device failure (2 years)
Hueb et al 201038	MASSII	Circ 2010	611	CABG or PCI	Medical therapy	Composite: mortality, MI, revasc. (10 years); individual end points
Hueb et al 201039	MASSIII	Circ 2010	308	Off-pump CABG	On-pump CABG	Composite: mortality, MI, CVA, revasc. (5 years)
Kapur et al 201040	CARDia	JACC 2010	510	PCI	CABG	Composite: mortality, MI, CVA (1 year)
Moller et al 201041	Best Bypass Surgery	Circ 2010	341	Off-pump CABG	On-pump CABG	Composite: mortality, MI, CVA, cardiac arrest, low CO syndrome, revasc. (30 days)
Omran et al 201042	—	JTCS 2010	220	CABG with ventral cardiac denervation	CABG	AF (in hospital)
Veeger et al 201043	CABADAS	ATS 2010	726	Aspirin/dipyridamole or warfarin after CABG	Aspirin after CABG	Composite: mortality CV, MI, revasc. (14 years)
Weltert et al 201044	—	JTCS 2010	320	Erythropoietin before CABG	Usual care	Blood transfusions (in hospital)

AF indicates atrial fibrillation; AKI, acute kidney injury; ARDS, acute respiratory distress syndrome; ATS, Annals of Thoracic Surgery; CABG, coronary artery bypass grafting; Circ, Circulation; CO, cardiac output; CVA, cerebrovascular accident; DSWI, deep sternal wound infection; GI, gastrointestinal; IABP, intra-aortic balloon pump; IV, intravenous; JACC, Journal of American College of Cardiology; JTCS, Journal of Thoracic and Cardiovascular Surgery; MI, myocardial infarction; MR, mitral regurgitation; MV, mitral valve; NEJM, New England Journal of Medicine; PCI, percutaneous coronary intervention; revasc., repeat revascularization; SAVR, surgical aortic valve replacement; TAVR, transcutaneous aortic valve replacement.

Mortality (all-cause and/or cardiovascular) was reported as an individual end point or as part of a composite end point in 28 trials (82%), MI was reported in 23 trials (68%), need for repeat revascularization or reoperation was reported in 22 trials (65%), stroke or transient ischemic attack was reported in 18 trials (53%), acute kidney injury was reported in 11 trials (32%), and bleeding complications were reported in 8 trials (24%) (Table 2). Patient-centered outcomes were reported in 10 trials (29%). Health care resource utilization was reported in 12 trials (35%).

Table 2

Graphical Representation of Primary and Secondary Outcomes in Included Trials With Overview of Commonly Used Combined Endpoints in Cardiovascular Research

Composite end points were used as the primary outcome measure in 19 trials and as a secondary outcome measure in 4 trials. Overall, 14 different composite end points were used, of which 6 were variants of the MACCE composite, 3 were variants of the MACE composite, and none were based on the STS composite. Eight of 9 trials using the MACE or MACCE composite incorporated repeated revascularization procedures.

The operational definitions of individual end points were equally variable. MI was defined based on World Health Organization criteria in 2 studies, European Society of Cardiology and/or American Heart Association criteria in 4 studies, VARC criteria in 1 study, creatinine kinase elevation greater than the upper limit of normal in 2 studies, creatinine kinase or troponin elevation >3 times the upper limit of normal in 1 study, creatinine kinase or troponin elevation >5 times the upper limit of normal in 4 studies, and creatinine kinase or troponin rise to various levels depending on time after surgery in 3 studies. No diagnostic criteria for MI were provided in 5 trials.

Stroke was defined based on focal neurological deficit with imaging findings in 3 trials and on acute focal neurological deficit lasting ≥24 hours with or without confirmatory imaging in 11 trials; no diagnostic criteria were provided in 6 trials.

Acute kidney injury was defined based on need for renal replacement therapy in 5 trials, need for renal replacement therapy or prespecified elevation in creatinine (each with different thresholds, ranging from 221 mmol/L [2.5 mg/dL] to 309 mmol/L [3.5 mg/dL]) in 3 trials, and prespecified elevation of twice the preoperative creatinine level with or without oliguria in 2 trials; no diagnostic criteria were provided in 2 trials. Prolonged ventilation or intubation was defined as >24 hours in 2 trials and >48 hours in 1 trial. The definition of postoperative bleeding differed in each of the 8 trials in which it was reported.

Discussion

To our knowledge, this review of adult cardiac surgery trials is the first to examine the current state of outcome reporting. We found that mortality and MI were most frequently reported as individual or composite end points and, conversely, that the STS composite was not used as an outcome measure. One of the most striking findings was the heterogeneity of composite end point reporting. This was apparent at 3 different levels. First, the decision to use or not use a composite as the primary outcome measure was evenly split between trials. Second, the choice of events included in composites was highly variable. Third, the operational definitions of events were ill defined, particularly the thresholds used to dichotomize continuous metrics such as troponin rise for MI or ventilation duration. MACE and MACCE also had varied definitions in the trials, similar to prior reports in general cardiology.7

Heterogeneity in cardiac surgery outcome reporting limits the ability to synthesize and meta-analyze results across trials to generate guidelines with the highest level of evidence.8 This is relevant, given the shift toward evidence-based practice derived from randomized controlled trials in cardiac surgery and other surgical subspecialties.9 In addition, nonstandardized outcome measures limit the ability to directly compare the effectiveness of various surgical techniques, perioperative interventions, and providers.10 As new competing techniques and technologies emerge, it is increasingly important that surgical outcome reporting be comparable among trials. Due to the lack of consensus outcome measures in cardiac surgery, investigators often default to using mortality as the end point of choice, despite being grossly underpowered to do so (as was apparent in the many of the trials reviewed and in an even greater proportion of observational studies). Similarly, with the exception of the STS models, most risk scores use mortality as the sole end point, neglecting the importance of other complications and patient-centered outcomes.

The use of composite end points in cardiac surgery may be beneficial for several reasons. Composite end points avoid the arbitrary choice of a single outcome when several may be of clinical importance for the cardiac surgery patient45 and allow for estimation of the net clinical benefit of the intervention when risks and benefits are both considered.46 Composite end points encompass postoperative complications, which are important determinants of functional recovery and quality of life.47 Patient recruitment in cardiac surgery trials has historically been difficult48 and continues to be challenging despite the emergence of collaborative research networks.49 Composite end points yield an increased number of events and a smaller required sample size, resulting in improved statistical efficiency and precision.50 This is especially true if event rates are low and efforts to analyze individual outcomes or to perform meta-analyses lead to false-negative and false-positive conclusions.51 Presenting a clear sample size calculation matched to the primary outcome of interest, as was done in most reviewed trials, is critical in this regard.

Choosing the proper events to include in a cardiac surgery–specific composite end point is of vital importance. The breadth of adverse events encountered after cardiac surgery is not captured by generic composite outcome measures traditionally used in cardiology, such as MACE or MACCE. Acute kidney injury and deep sternal wound infections, for example, are postoperative events that are associated with considerable morbidity. MI, the usual driver of MACE-type end points in cardiology trials, has different connotations and prognostic impact in the postoperative setting and concerns pertaining to measurement errors if ascertaining MI soon after surgery. The use of composites may be justified only if each component is of similar importance to the patient,2 whereas it may be questionable if components are empirically different in impact (eg, deep sternal wound infection and stroke).2 Assigning weights to the components may circumvent this caveat and help increase the validity of conclusions.52

Quality of life, physical performance, cognitive function, and dependency for activities of daily living have been broadly categorized as patient-centered outcomes because they reflect domains that are crucial to the patient but are extrinsic to traditional domains of mortality and pathophysiology that are emphasized by physicians and researchers. There is increasing awareness in the cardiovascular community that these data should be collected and reported, particularly when studying elderly populations in which the priority of care may have shifted from longevity to quality of life. Postoperative length of stay, stroke, and readmission have been identified as important indicators of quality of care,53 strengthening the rationale for also reporting these end points.54

For a criterion to be a useful part of a composite end point, it should be clinically important and reliably ascertainable. In the cardiology literature, consensus efforts have been made to standardize adjudication of MI,55,56 renal injury,3 and bleeding.4 These consensus documents are not necessarily portable to the specific context of cardiac surgery, for which the mechanism, magnitude, and clinical implication of certain events are fundamentally different. Consider the difference between medical versus surgical bleeding and ambulatory versus perioperative troponin rise. The STS composite does not include perioperative MI, which is in part due to its low ascertainment reliability.1 Recommended definitions of perioperative MI vary considerably, from a highly restrictive approach requiring evidence of an acute coronary embolus57 to a multifaceted approach incorporating clinical and biomarker criteria.55 Other potentially important cardiac surgery outcomes, such as prolonged postoperative mechanical ventilation, have not been uniformly defined or adopted for use.

VARC is a context-specific consensus document focused on transcatheter aortic valve replacement; it provides standardized end points with clearly defined criteria for reporting.5 A meta-analysis showed that the VARC end points were frequently being implemented to report clinical outcomes.58 Although there has been an initial attempt to develop a similar document focused on pediatric cardiac surgery,59 there has yet to be an attempt in adult cardiac surgery.

Limitations

Because our search was limited to 5 scientific journals (for feasibility purposes), we did not capture trials published in other journals. The selected journals represent the highest ranked impact factors in their respective subspecialties of cardiothoracic surgery, cardiology, and general medicine, and we expect that the heterogeneity of outcome reporting could have been more pronounced if we had included smaller lower ranked studies. Conversely, the selected trials encompassed a wide variety of interventions and comparators (surgery versus surgery, surgery versus transcatheter procedure, surgery plus adjunctive medical therapy versus surgery alone), such that the heterogeneity of outcome reporting could have been less pronounced if we restricted our selection criteria to one of these types of trials. We excluded trials that did not report a clinically driven primary outcome, and this also led to underrepresentation of smaller studies that were underpowered to assess clinical events. We chose to focus on clinical events because these will likely form the basis of future efforts to develop standardized guidelines for reporting outcomes.

Conclusion

Outcome reporting in the cardiac surgery literature is heterogeneous, and efforts should be made to standardize the outcomes reported and the definitions used to ascertain them. Measures of functional status and resource utilization are currently underreported and should be integrated in standardized reporting schema. The development of standardizing outcome reporting is an essential step toward strengthening the process of evidence-based care in cardiac surgery.

Sources of Funding

Dr Afilalo is supported by a Research Scholar award from the FRQ-S.

Disclosures

None.