Using machine learning to uncover heterogeneity of beta blocker response in heart failure.

A recent study by Karwath et al.1 in The Lancet applied machine learning-based cluster analysis to pooled data from nine double-blind, randomized controlled trials of beta blockers, identifying subgroups of efficacy in patients with sinus rhythm and atrial fibrillation.

Main text

Machine learning has received substantial attention in recent years, including in the scientific and medical literature., From the perspective of the technical advances enabled by machine learning for data analysis, data processing, and modeling, this attention has been largely justified. For example, certain types of analyses that previously were not possible or at a minimum posed substantial challenges—such as analyzing raw formats of very high dimensional data—have been made tractable through machine learning approaches. More complex machine learning methods, such as deep neural networks and derivatives, can both discover and leverage very high degrees of interaction within data to improve performance, allowing them to consistently surpass traditional statistical methods for certain problems. Nonetheless, machine learning methods are simply tools. And at a minimum, two considerations are needed to enable these tools to yield their greatest value (Figure 1). First, an appropriate scientific question is critical to guide the decision of what analytic approach (e.g., machine learning or traditional statistical or both) and which specific method(s) therein to employ. Second, careful attention must be paid to the nature of the data being used to investigate this question, including understanding and/or mitigating biases inherent therein. There are many circumstances in which machine learning adds little or may be inappropriate to use compared with traditional statistical methods. For all the excitement surrounding machine learning, the familiar adage of “garbage in garbage out” remains true, perhaps even more so in the face of increasingly complex and less widely understood analytic tools.

Figure 1

Determining whether machine learning analysis can add value to a scientific study

Two important considerations can help determine the added value that machine learning analysis may provide. The specific scientific question and the nature of the data available to analyze can help determine whether traditional statistical analysis, machine learning or a combination of both can best accomplish the intended goal. Created with BioRender.com.

Karwath et al. in The Lancet investigated the question of whether there are clusters of heart failure patients in both sinus rhythm and atrial fibrillation that exhibit differential benefit from beta blocker medications. Among patients who have heart failure with reduced ejection fraction (HFrEF), beta blockers are a cornerstone medical therapy supported by substantial prior literature. However, beta blockers have not demonstrated benefit in HFrEF patients with atrial fibrillation, a common comorbid condition. To investigate this question, Karwath et al. used previously collected data considered to be among the highest quality for this purpose: individual-level data from nine randomized controlled trials of beta blockers. They obtained a uniform set of clinical variables from 15,659 HFrEF participants from the nine pooled trials. Then, a neural network-based method, called a variational autoencoder, was used to process this input data into a smaller set of derived-variables, simultaneously retaining critical elements of the original variables while also distilling important high-level interactions between them. While Karwath et al. applied this to relatively few original input variables, this neural network-enabled analysis step could be extrapolated to much higher dimensional input data, such as genomic data, effectively processing it in order to be tractable for subsequent analysis. The derived-variables output from this step were then analyzed using clustering algorithms to identify distinct patient clusters within strata of atrial fibrillation and sinus rhythm.

Using this approach, this study’s primary findings identified a cluster of patients in atrial fibrillation that had lower all-cause mortality with beta blockers (odds ratio = 0.57, 95% CI 0.35–0.93; p = 0.023), and several clusters of patients in sinus rhythm that did not benefit from beta blockers. These clusters were identified in the context of overall pooled results from the nine trials showing that HFrEF patients in sinus rhythm exhibited lower mortality with beta blockers whereas those in atrial fibrillation did not, which is consistent with prior literature. While Karwath et al. implemented a validation protocol that aimed to verify clustering robustness, which they reported as “confirmed”, whether the overall approach employed or the clusters identified will truly generalize to prospectively enrolled data remains to be determined. The validation protocol they performed was still only a proxy for true external validation, akin to extrapolating in vivo results from an in vitro experiment, and thus warrants appropriate caution. One possible analysis that could help support the reported findings using existing data is whether a less complex dimensionality reduction approach, such as principal components analysis or a shallower autoencoder, can also identify atrial fibrillation patients with mortality benefit, especially given the low-dimensionality of the original variables.

If this overall machine learning-enabled clustering approach is ultimately validated by prospective studies, then the true potential of similar neural network-supported approaches for clinical/biomedical data extends far beyond what was done here. Because neural networks can accept high-dimensional raw data of nearly any type, they make it possible to process truly large-scale data in ways not previously possible; the primary limitation presently is computational power/memory, which increases yearly. For example, in the clinical domain, raw imaging data (e.g., radiologic studies, ultrasound videos, radioisotope imaging), continuous diagnostic recordings (e.g., ECG/telemetry, EEG), or genetic panels could be used, whereas in biology, genetic/genomic, proteomic, pharmacogenomic, epigenomic, microbiomic data, or similar could be used in their raw or near-raw formats. Ultimately, machine learning approaches draw their power from the high-level interactions they are able to leverage, which almost by definition makes them harder to understand—the “black-box effect” Karwath et al. describe. While much active research in the machine learning and computer science fields aims to address this, they almost certainly will not attain the degree of interpretability of standard linear models (assuming all assumptions are met). Thus, this will remain a trade-off when choosing to use machine learning approaches.

Properly applied, the large and continuously growing library of machine learning methods can serve to expand researchers’ capabilities with regard to the type of data that can be analyzed and the range of analyses that can be performed. Ultimately, though, it is likely through a combination of insights gained from novel approaches, like machine learning, validated by well-established study designs, such as prospective randomized trials, that the most clinically relevant advances in biomedicine will be achieved.