Therapeutic hypothermia and Type II errors: Do not throw out the baby with the ice water.

After initial enthusiasm for mild therapeutic hypothermia (TH) treatment after brain injuries, including global cerebral ischemia after cardiac arrest, subsequent trials suggested similar benefit using only targeted temperature management (TTM), with fewer side effects. Globally, effective treatment of brain ischemia with TH has declined. Recent data suggest, however, that TH to 33°C may be superior to TTM. We review the background and rationale underlying TH and TTM. We present previously published data from our own laboratory that confirms TH to 33°C provides superior brain cytoprotection, compared to 35°C or 37°C, over a range of delays to treatment and several durations of TH. We illustrate that the treatment effect size of either or 35 is superior to 37, but the effect size difference between 33 and 35, although significant, is small. We estimate that to demonstrate the superiority of TTM over TH, a clinical trial would need between 3,000 and 9,000 patients depending on the desired treatment effect size. Our review and our own data suggest that TH to 33°C is superior to TTM to 36°C, but an extremely large clinical trial would be needed to demonstrate the difference. Copyright:

Introduction

Therapeutic hypothermia (TH) is the most potent neuroprotective therapy ever studied in experimental cerebral ischemia. Cooling the brain as little as 1°C significantly alters brain responses to ischemia.[12] TH exerts multiple effects at multiple stages of the ischemic cascade, many of which involve temperature-dependent mechanisms [Figure 1].[34] Today, brain temperature must be rigidly controlled in all experimental cerebral ischemia studies to avoid confounding effects.[5] Brain metabolism – utilization of oxygen and glucose – drops significantly at lower temperature, thus conserving resources and prolonging penumbral survival, but regional cerebral blood flow is preserved.[67] Hypothermia probably works by inhibiting temperature-dependent enzymes such as proteases, caspases, endonucleases, lipases, and metalloproteinases, although further work is required. In addition, hypothermia inhibits inflammatory responses such as leukocyte migration/lymphocyte activation and minimizes free radical generation.[89]

Figure 1

Putative mechanisms underlying the protective effect of therapeutic hypothermia. Therapeutic hypothermia appears to inhibit a variety of processes thought to be important in mediating cell damage after ischemia (reprinted[4] with permission from Wu et al., Lancet Neurol 12:275-284, 2013)

No putative neuroprotectant has been studied over the broad and deep range of animal cerebral ischemia models as has hypothermia. A large meta-analysis by an independent group rated the rigor and quality of preclinical hypothermia studies as excellent.[1011] Of all putative neuroprotectants studied, hypothermia ranks among those studies with the highest rigor, meaning there are an adequate number of high-quality studies to predict eventual clinical success. A number of preclinical studies meet the RIGOR guidelines.[12] There is a clear and consistent benefit of TH demonstrated prominently in higher quality papers that included randomization, blinding, and both histological and functional outcomes.

The potent protective effect of TH has proven problematic in translation to patients. Powerful protection with TH has been clearly documented after accidental neonatal hypoxic–ischemic injury.[131415] Studies of TH for head trauma have routinely failed to show significant benefit.[161718]

In 2002, two well-recognized studies demonstrated the benefit of TH in survivors of cardiac arrest.[1920] Both trials included victims of ventricular fibrillation/tachycardia with prompt return of spontaneous circulation (ROSC), adequate blood pressure, and rapid transport times. In both trials, a study done in Melbourne, Australia, and another in Europe, TH was induced using surface cooling with ice packs, but cooling began in the field during the Melbourne trial, so these patients reached target temperature within 2 h, on average.[19] Comparatively, in the European trial, target temperature was reached within 8 h [Figure 2].[20] In both trials, the normothermia groups received unspecified “routine care” although core body temperatures appear to have been maintained around 37°C. After TH, the patients were slowly and carefully rewarmed to avoid overshoot hyperthermia in both trials. The graphic data provided for the European study, reprinted in Figure 2, indicate some variation, however. Temperature variation in the control groups may have allowed for hyperthermia in enough numbers to account for the demonstrated therapeutic benefit.

Figure 2

Time course of therapeutic hypothermia in the European trial of therapeutic hypothermia for cardiac arrest patients with return of spontaneous cerebral circulation. Mean temperatures differed between the groups early after treatment onset. Some degree of hyperthermia occurred in the “normothermia” group (reprinted[20] with permission, NEJM 346:549-556, 2002)

National and international guidelines soon agreed to recommend TH for selected survivors of cardiac arrest, with profound benefits seen anecdotally.

The targeted temperature management (TTM) investigators conducted a trial without a normothermia group to determine whether moderate hypothermia to 33°C is no better than controlled normothermia to 36°C.[21] The study attempted to carefully control core body temperature to either 33°C or 36°C and avoided fever in the 36°C group. Otherwise, inclusion criteria resembled the prior trials in some ways, but the TTM investigators broadened the inclusion criteria to all survivors of cardiac arrest due to any cause. No significant differences between the groups could be detected in the TTM trial, a result publicized globally causing clinicians to widely and abruptly abandon difficult cooling protocols in favor of normothermia. Such actions clearly overemphasize the significance of one negative trial and miss the obvious truth that the TTM trial compared two levels of carefully titrated TH.[2223]

The TTM trial differed from the prior European and Melbourne trials in critical design features. Patients with less recoverable injuries were included in the TTM trial: asystole (12%) and pulseless electrical activity (8%). The rate of ST-segment elevation myocardial infarction of 41% in TTM was likely higher than in either of the preceding trials.

As shown in Figure 3, cooling proceeded slowly in the TTM trial. The two treatment groups do not differ significantly for 9 or 10 h after cooling began. In the group assigned to 33°C, the target temperature was not reached until around 8 h after cooling start, perhaps as long as 10 h after ROSC. While the exact mechanism of neuroprotection with TH is not known, most observers suggest that cooling should begin urgently after ischemia.[24] If the onset to cooling is delayed significantly, there may be a benefit, but much longer cooling durations are required.[25] Thus, we suggest there is a real probability that TH in the TTM trial started too late to yield a statistically significant difference between the two study groups.

Figure 3

Time course of therapeutic hypothermia in the targeted temperature management trial. The mean temperatures did not differ between the groups for many hours after treatment onset. Little hyperthermia could be observed in the normothermia group (reprinted[21] with permission, NEJM 369:2197-2206, 2013)

To determine whether different levels of TH differ in their therapeutic efficacy, we used a novel in vitro model. We cultured the individual cell types comprising the neurovascular unit (NVU). The description of the NVU – comprising neurons, astrocytes, endothelial cells, and pericytes –suggested to us the importance of testing the individual elements.[26] Previously, we have shown differential effects of various neuroprotectants on endothelial cells, astrocytes, and neurons,[25] but all prior clinical trials treated the NVU as homogenous and assumed that all cell types respond similarly. We cultured neurons, astrocytes, or endothelial cells from rats and applied standard oxygen–glucose deprivation (OGD) as an in vitro model of ischemia. We predicted neurons would be selectively vulnerable, followed by astrocytes, followed by endothelial cells. In contrast to our prediction, astrocytes showed the greatest resistance to OGD [Figure 4]. We required 10 h of OGD to kill 80% of astrocytes compared to 2 h for neurons and 6 h for endothelial cells.

Figure 4

Cell type-specific response to oxygen–glucose deprivation. In monocultures, the effect of oxygen–glucose deprivation differs according to cell type. Neurons are the most vulnerable, followed by endothelial cells, with astrocytes the most resistant to injury. In Panel a, cell viability was measured using the conversion of MTT (3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide, a tetrazole); in Panel b, cell death was measured using the lactate dehydrogenase (LDH) assay

We then designed an experiment to replicate the essential relationship among target temperature–depth, delay to treatment, and duration. We varied the duration of OGD, the target temperature, and the delay time after OGD before starting TH. Previously, data suggested that after relatively longer delays to treatment, both deeper TH and longer treatment durations were needed.[272829] Our results in primary cultures essentially confirm that longer delays require longer durations of treatment [Figure 5]. An exciting, novel finding in our data is that different elements of the NVU – neurons, astrocytes, and endothelial cells – all exhibit the delay–duration relationship but on very different time scales. The mechanism of the different time scales is not known and there is a compelling need for detailed studies of the differential effects of target depth, delay, and duration on these different elements of the NVU. Nevertheless, all NVU cell types exhibit the same delay–duration relationship seen by other investigators: the longer the delay to treatment, the longer duration cooling is needed to show benefit.

Figure 5

Relationship among target temperature–depth, delay to treatment, and duration in neurons and astrocytes subject to oxygen–glucose deprivation. Cell viability was assessed with the 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide assay after oxygen–glucose deprivation.[25] Results from neuron cultures are shown in Panels A and B, whereas astrocytes are shown in Panels C and D. In Panels A and C, there is no delay: hypothermia began at the end of 2 h oxygen–glucose deprivation and continued for 2, 4, or 24 h. In Panels B and D, there was a 90-min delay between the end of oxygen–glucose deprivation and the start of cooling. Using the 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide viability assay (A and C), hypothermia protected neurons and astrocytes after no delay. After 90 min (Panels B and D), only duration 6 or 24 h at 33°C was effective. After zero delay, hypothermia at 33 or 35 protected neurons and astrocytes regardless of the treatment duration. After 90 min delay (Panel B and D), only 33°C protected neurons at durations of 6 or 24 h. Regardless of depth or duration, all hypothermia was superior to normothermia (Panel A and B) after no delay (ANOVA, Dunnett's, *P < 0.05, **P < 0.01, ***P < 0.001 comparing 33°C to 35°C. P <0.05 for *: 33°C vs. 35°C, #: 37°C vs. 33°C and Φ: 37°C vs. 33°C) (graphs redrawn from data previously published[25])

Furthermore, we showed that cooling to a target depth of 33°C was superior to 35°C in all three cell types in the NVU and at any combination of delay/duration [Figure 5a and b]. These novel data require replication, but eventually, the data could influence the design of clinical trials of TH for both cardiac arrest and stroke.[23] On more careful inspection of the data, however, another insight can be gleaned. Comparing TH with either 33°C or 35°C, there is a powerful and statistically significant difference from normothermia at 37°C. For example, in Figure 5a, cell viability is about 20% (cell death about 80%) after OGD under normothermic conditions. By comparison, two levels of TH, 33°C and 35°C, both preserve cell viability around 60% (cell death about 40%) for a combined relative treatment effect to about 50% (40% cell death compared to 80%). On the other hand, the difference between the two levels of TH is quite subtle. Again looking at Figure 5a, the relative treatment effect comparing 33°C to 35°C is about 20% (67%–54% =13%/67%). With longer delays to treatment, the differences are present but less dramatically. Nevertheless, at most treatment times in all three cell types, the treatment effect of the TH (either level) compared to 37°C is about 3 times greater than the relative treatment effect of 33°C compared to 36°C. As our data are based on monocellular cultures, our results should be replicated in double or triple-cell type cultures in which elements of the NVU are subject to OGD together.

A Type II Error in Clinical Practice

Our in vitro data above suggest that two levels of TH differ in efficacy. Whether the elements of the NVU respond similarly in the whole brain remains an open question. Our data predict that survivors of cardiac arrest would respond to 33°C with greater benefit than 35°C, and thus, the different results among European, Melbourne, and TTM are easy to understand. In the trials in which TH was compared to normothermia alone, efficacy could be easily demonstrated due to quicker cooling times and deeper target temperatures. In contrast, the TTM trial assigned patients to two TH target temperatures; further, the time to reach target temperature was quite long. Given this, the TTM trial can be viewed as significantly underpowered, such that negative findings between the two study groups were virtually guaranteed. Although the TTM trial was larger than either of the preceding two trials, nevertheless there were not nearly enough patients to differentiate between two levels of controlled TH.

Combining prior results with the present in vitro data, we can estimate the number of patients who need to minimize the likelihood of a Type 2 error in a clinical trial comparing two levels of TH, such as 33°C and 35°C. Tables 1 and 2 summarize the rates of “good outcomes” and mortality across the three clinical trials. Good outcome was defined slightly differently among the three trials, but for the purpose of estimating needed sample sizes, these differences were ignored. As shown in Table 1, the absolute difference between hypothermia and control was 16% in the European and 22% in Melbourne, corresponding to relative differences of 41% and 84%, respectively. The mortality differences, summarized in Table 2, similarly show a relative effect size of 25% for the European and 17% for Melbourne. As shown in Table 3, the relative treatment effect sizes seen in our in vitro studies, comparing either temperature against normothermia, were large and comparable to that seen in the European and Melbourne trials. Comparing 33°C versus 35°C, however, the treatment effect sizes ranged over 7.7 ± 8.4% (neurons), 2.1 ± 1.3 (astrocytes), and 9.6 ± 7.2 (endothelial cells) depending on delay to and duration of treatment. We, therefore, estimated sample sizes for a putative comparison of 33°C versus 36°C in humans assuming a power of 80% and significance of 5% to detect a clinically meaningful reduction of mortality from assumed 52% in one group and either 49% (3% absolute; 6% relative) or 47% (5% absolute; 11% relative) in the other group. For the 6% relative effect size, a properly sized trial would require 4,359 patients per group or a total of 8,718. For an 11% relative treatment effect size, a trial would require 1,579 patients per group, or 3,138 patients total, calculated using G-Power two-tailed z-test for difference between independent proportions.

Table 1

Summary of “good outcomes” in three trials

Trial	Target temperature (°C)	Randomized (n)	Modified ITT (n)	Patients with follow-up	CPC 1 or 2, n (%)	Delta
TTM	36	474	466	464	222 (48)	2 (NS)
	33	476	473	469	218 (46)
HACA	32-34	137		136	75 (55)	16 (P=0.009)
	Normo	138		137	54 (39)
Melbourne	33	43		43	21 (48)	22 (P<0.05)
	Normo	34		34	9 (26)

Three major trials of TH after cardiac arrest used slightly different definitions of “good outcome” and slightly different control groups. The TTM trial compared two levels of TH, whereas the HACA and Melbourne trials compared attempted hypothermia to no active TTM. A number of patients randomized, eligible for the ITT analysis, and with follow-up are listed. The TTM and the HACA trials used the clinical performance rating scale (CPC) at follow-up, whereas the Melbourne trial used postdischarge destination to evaluate the effects of TH on outcomes. The delta listed are the absolute difference (in percentage favorable outcome) comparing the treated to control groups. Statistical significance was tested differently in each trial. ITT: Intention-to-treat, CPC: Cerebral performance category, TTM: Targeted temperature management, HACA: Hypothermia after cardiac arrest, NS: Not significant, TH: Therapeutic hypothermia

Table 2

Mortality in three trials

Trial	Target temperature	Randomized (n)	Mortality	Mortality, n (%)	Delta
TTM			End of trial	180 days
	36	474	235	226 (48)	2 (NS)
	33	476	225	220 (46)
HACA				6 months
	32-34	137		56 (41)	14 (P=0.02)
	Normo	138		76 (55)
Melbourne				3 months
	33	43		22 (51)	11 (NS)
	Normo	34		23 (62)

In the same trials summarized in Table 1, mortality rates were assessed at different time points. Mortality is shown as the n (%) rates in each group. The delta is the absolute difference (in percentage) between the study groups. Statistical significance was tested differently in each trial. TTM: Targeted temperature management, HACA: Hypothermia after cardiac arrest, NS: Not significant

Table 3

Absolute treatment effect sizes

Cell type	Delay	Duration	33 versus normo	35 versus normo	33 versus 35
Neurons	0	2	44.38	36.27	8.11
	0	6	50.97	43.13	7.84
	0	24	57.90	56.08	1.83
	30	2	29.15	27.65	1.50
	30	6	34.53	31.52	3.01
	30	24	39.87	34.16	5.71
	60	2	34.48	22.98	11.50
	60	6	43.63	25.27	18.36
	60	24	46.24	34.02	12.23
	90	2	−1.48	9.05	−10.53
	90	6	29.15	18.00	11.15
	90	24	40.45	18.98	21.47
Mean±SD			37.4±14.9	29.8±12.4	7.7±8.4
Ratio			4.8	3.9	1.0
Astrocytes	0	2	41.55	0.19	41.36
	0	6	45.97	20.72	25.25
	0	24	56.33	14.12	42.21
	30	2	22.23	14.39	7.84
	30	6	22.64	22.67	−0.03
	30	24	39.60	26.30	13.30
	60	2	20.94	11.74	9.20
	60	6	24.79	12.02	12.77
	60	24	39.87	16.36	23.51
	90	2	28.36	8.23	20.13
	90	6	38.73	16.01	22.72
	90	24	50.36	18.62	31.73
Mean±SD			35.9±11.9	15.11±6.9	2.1±1.3
Ratio			18.0	7.2	1.0
Endothelial cells	0	2	−93.41	−87.99	6.17
	0	6	−92.54	−62.59	47.85
	0	24	−92.52	−91.75	0.84
	30	2	−90.42	−70.02	29.14
	30	6	−79.60	−74.18	7.31
	30	24	−90.50	−86.13	5.08
	60	2	−86.16	−77.27	11.51
	60	6	−92.22	−71.99	28.10
	60	24	−91.52	−85.28	7.31
	90	2	−89.12	−69.65	27.97
	90	6	−86.72	−68.82	26.00
	90	24	−93.81	−87.77	6.89
Mean±SD			−71.4±3.2	−61.9±7.6	9.6±7.2
Ratio			7.4	6.4	1.0

These data were compiled to generate the plots shown in Figure 5. Monocellular cultures of neurons, astrocytes, or endothelial cells subject to OGD were treated with TH to 33°C or 35°C for varying durations after varying delays. Each row shows the difference in survival comparing the three treatment groups. In the column labeled “33 versus normo,” the numbers indicate the survival of cells treated with 33°C in excess of those treated with 37°C, similarly for the remaining two columns. The rows marked “ratio” indicate the ratio of the mean treatment effect size versus that of “33 versus 35.” The ratios indicate that TH with 33°C gives a treatment effect size that is 4.8 times larger than that of 33 versus 35. The ratios for 33°C are larger than those for 35°C for all three cell types. SD: Standard deviation, OGD: Oxygen-glucose deprivation, TH: Therapeutic hypothermia

The achieved power (post hoc) comparing 52% and 47% survival rates for a two-tailed test is much different than the standard 80%. In addition to the 5% effect size being small, part of the reduced power is associated with the increased variance seen when the estimated proportion is around 0.5(chance), where the sample variance is maximized [Figure 6]; the sample variance is maximized at p^ =0.5. A larger sample variance leads to a reduced z-statistic and increased P values. This means that a 5-pt difference is easier to detect for a 90%–95% change than for a 47%–52% change, holding constant the sample sizes in both analyses.

Figure 6

The variance of the binomial distribution is maximized around chance (P = 0.5), leading to a decrease in statistical power to detect changes in this region. This means that, for equivalent sample size, it is easier to detect a change between 90% to 95%, than a change between 47% to 52%

Clinical Implications

What then are we to do at the bedside, given these data? First of all, and as suggested by the TTM authors, we definitely should not abandon TH for survivors of cardiac arrest due to a shockable rhythm. Across the United States and in other countries, many groups have announced their intention to abandon TH after cardiac arrest, and this clearly would be a mistake and a disservice to patients.[2230] Recently, a survey of door-to-target temperature shows increasing variation among the US sites.[24] In contrast to the European and Melbourne trials – which excluded patients with nonshockable rhythm – a recent French study showed a significant survival benefit in patients treated with TH after cardiac arrest due to asystole and PEA.[31] Although TH is technically complex, so are many things we do to patients with far less basic science justification. Second, it will be critical to actively apply TH to appropriate patients. Benefit was seen in two trials that used rapid cooling to 33°C, compared to no treatment, and that seems like the most rational target given the data. Our data also support the conclusion that 33°C is superior to 35°C or 36°C. Thus, on a purely evidence-driven basis, the rational conclusion would be to preserve the goal of ultra-rapid cooling to 33°C. Third, it can be reasonably argued that since cooling to 36°C has not yet been demonstrated to be superior to 33°C – even if that failure represents a Type II error – then a policy targeting 36°C would be considered clinically acceptable. Although we disagree, we must admit such a conclusion is rational and defensible in given the present situation. Replication of our results in multiple cell cultures, combining two or more elements of the NVU subject to OGD, would strongly argue against mild TTM to 36°C.

The most critical issue facing our patients now is the spreading nihilism among clinicians who have not examined all the available data carefully. A single experiment should never move a field dramatically, yet this is exactly what happened following publication of the TTM study. The totality of the data suggests that deeper levels of TH are better, but certainly, all would agree some hypothermia is better than none. We must refocus on the larger dataset and consider the implications of overemphasizing one study at the expense of a very large body of knowledge.

Open Questions

It is not clear how long after cerebral ischemia effective therapy can wait (delay); nor is it clear for how long TH must be continued (duration); nor can we be sure of the best temperature to which to cool patients (depth). From considerable literature, it is fundamentally clear that TH protects through multiple mechanisms, making it a highly desirable neuroprotective therapy.[4] The mechanism of increasing harm with longer delay to treatment deserves further study, but almost no work explains the mechanism of the “duration effect:” longer durations of TH are needed to overcome prolonged delays to TH onset. However, previous in vitro studies have not shown or linked a relationship between duration, delay, and depth of hypothermia. Considerable further research is needed to explore the fundamental relationships among cooling depth, cooling delay, and cooling duration.

Most importantly, relative benefit of deeper TH to 33°C must be confirmed in a properly sized clinical trial that includes enough patients to detect a benefit compared to 36°C.

Financial support and sponsorship

This work was supported by NINDS R01 NS075930, the Lippman Family Foundation, and the Louis and Carmen Warschaw Chair in Neurology Research.

Conflicts of interest

There are no conflicts of interest.