Translating Data Analytics Into Improved Spine Surgery Outcomes: A Roadmap for Biomedical Informatics Research in 2021.

STUDY
DESIGN: Narrative review.
OBJECTIVES: There is growing interest in the use of biomedical informatics and data analytics tools in spine surgery. Yet despite the rapid growth in research on these topics, few analytic tools have been implemented in routine spine practice. The purpose of this review is to provide a health information technology (HIT) roadmap to help translate data assets and analytics tools into measurable advances in spine surgical care.
METHODS: We conducted a narrative review of PubMed and Google Scholar to identify publications discussing data assets, analytical approaches, and implementation strategies relevant to spine surgery practice.
RESULTS: A variety of data assets are available for spine research, ranging from commonly used datasets, such as administrative billing data, to emerging resources, such as mobile health and biobanks. Both regression and machine learning techniques are valuable for analyzing these assets, and researchers should recognize the particular strengths and weaknesses of each approach. Few studies have focused on the implementation of HIT, and a variety of methods exist to help translate analytic tools into clinically useful interventions. Finally, a number of HIT-related challenges must be recognized and addressed, including stakeholder acceptance, regulatory oversight, and ethical considerations.
CONCLUSIONS: Biomedical informatics has the potential to support the development of new HIT that can improve spine surgery quality and outcomes. By understanding the development life-cycle that includes identifying an appropriate data asset, selecting an analytic approach, and leveraging an effective implementation strategy, spine researchers can translate this potential into measurable advances in patient care.

Introduction

Spine surgery has a proud history of applying rigorous research and technological innovations to advance the care of patients with complex spine disease. Historically, many technological advances came from biomedical engineering, which has contributed to improved imaging modalities, implant technology, and fusion biologics. However, there is increasing acknowledgment that spine surgeons still face substantial uncertainty related to basic treatment questions, such as the likelihood of surgical success and chance of postoperative complications. This recognition has increased the focus on using data science to transform spine surgery practice. Supporting this mission, there has been explosive growth in healthcare data – an estimated 16,000 exabytes in 2018. The increased availability of data assets has expanded opportunities to use biomedical informatics tools to improve virtually all aspects of spine care, including: diagnosis and imaging classification; treatment selection and risk prediction; perioperative management; and administrative tasks.[3,4]

Nonetheless, data science has not yet transformed spine surgery in the way it has some areas of medicine and society. Despite the increasing availability of large data assets and advanced computing power, we remain far from the goal set by the Institute of Medicine in 2007 to have 90% of clinical decisions supported by accurate, timely clinical information by the year 2020. To narrow this divide, spine surgeons must understand how key decisions related to dataset selection, analytic techniques, and implementation strategy influence the clinical impact of health information technology (HIT). Recognizing these important considerations (Figure 1), this review will provide a HIT roadmap to help realize the potential of data assets and biomedical informatics tools to improve spine surgery practice.

Figure 1.

A diagram depicting the process of developing and implementing new health information technology in spine surgery. EHR indicates electronic health record.

Types of Data Assets

An overview of data assets available for spine surgery informatics research is shown in Table 1.

Table 1.

A summary of the Strengths, Limitations, and Ideal Uses for Data Assets used in Spine Surgery Biomedical Informatics Research.

Data asset	Strengths	Limitations	Uses
Administrative (claims) data	▪ Large sample size▪ Inexpensive▪ Clinical and cost data▪ Population coverage▪ Structured data	▪ Unreliable data accuracy▪ Limited breadth of data (e.g. no lab or imaging data)▪ Delayed availability (due to coding and processing)	▪ Population-level outcome trends 7 ▪ Health policy and cost analysis 8 ▪ Linkage with clinical registries 9
Spine surgery registries	▪ Relatively large sample size▪ High quality, “real world” data▪ Condition-specific data collection (e.g. patient-reported outcomes, imaging)▪ Structured data	▪ Expensive to establish and maintain▪ Narrow clinical focus and data collection▪ Delayed data availability	▪ Quality improvement programs 10 ▪ Comparative effectiveness studies 11 ▪ Hypothesis generation for clinical trials 12
Electronic Health Records	▪ Real-time data acquisition▪ Wide breadth of data (e.g. clinical, imaging, free-text)▪ Inexpensive to access▪ Large sample size	▪ Inconsistent data quality▪ Often lack patient-reported outcomes▪ May lack generalizability▪ Unstructured data	▪ Real-time safety alerts 13 ▪ Integration with clinical registries 14 ▪ Quality and outcomes research 15
Mobile data	▪ Real-time, real-world data collection▪ Detailed physical function data▪ Rapid data acquisition	▪ Limited availability▪ Socioeconomic barriers to use▪ Uncertain patient acceptance▪ Interpreting clinical importance can be challenging▪ Interoperability and data storage	▪ Physical function assessments 16 ▪ Real-world data acquisition 17 ▪ Physiologic data collection▪ Low patient burden
Biobanks	▪ Individualized▪ Biological detail	▪ Expensive▪ Limited availability▪ Patient privacy concerns	▪ Precision medicine (e.g. risk prediction, drug targeting) 18 ▪ Linkage with EHR/registry data 19

Administrative Datasets

Administrative datasets based on billing claims have been used frequently in spine surgery research, likely due to their widespread availability, relatively low cost, structured data, and population-level coverage.[20-22] These datasets have provided important insights into the effectiveness of policy interventions, surgical costs, and population-level trends.[8,7] However, diagnoses from billing codes are often imprecise and lack imaging data, limiting the ability to evaluate clinical outcomes. For example, billing data have limited ability to distinguish key surgical variables, such as the number of levels treated or the use of minimally invasive techniques, confounding comparative effectiveness research efforts. Although technically complex, linking administrative and clinical registry data can help overcome some of these limitations and broaden potential applications.

Spine Surgical Registries

Spine surgery registries are experiencing increasing growth and attention. A 2015 systematic review identified 25 registries representing 14 countries. Among the most recent, the American Spine Registry has emerged as a successor to the Quality Outcomes Database with the goal of unifying neurosurgery and orthopedic registries efforts.[25,26] Other registries, such as the International Spine Study Group and European Spine Study group, have focused on particular spine populations, such as deformity.[27-29] These registries offer advantages over claims data, including data quality control, detailed patient characteristics, and inclusion of patient-reported outcomes.[25,10,30] These attributes have generated substantial enthusiasm among both surgeons and hospital administrators. Nonetheless, few registries capture imaging data, and standards for processing and storing these data are lacking. Additionally, considering maintenance fees and the need for a full-time employee for data review, establishing a multicenter registry can cost millions of dollars.[30,32,33] Finally, most registries are not designed to collect real-time patient data. Linking registries with electronic health records (EHR) and mobile health data offers an opportunity to decrease their cost and expand potential uses.[34,35]

Electronic Health Records

EHRs represent an expansive and underutilized source of spine surgery data. Currently, at least 98% of hospitals have adopted an EHR system, creating vast quantities of patient data, updated in real time. The EHR offers spine surgeons valuable opportunities to both develop and implement informatics tools. While many surgeons are familiar with using the EHR for simple research tasks (e.g. identifying patients by procedure code), its full potential has largely been untapped. For example, automated workflows are capable of populating quality improvement registries,[35,37] though such pipelines are not routine. Additionally, multidimensional EHR data can be used in real-time to support evidence-based decision-making. For example, a model predicting surgical complications evaluated 285 clinical, demographic, administrative, and laboratory variables to develop a prediction tool that processes EHR data in real-time to provide risk predictions at the point-of-care. In spine surgery specifically, the use of real-time EHR analytics to support decision-making has been less common, though there have been notable successes, such as clinical decision-support for guiding appropriate spine imaging.[39,40] Challenges to leveraging insights from the EHR include the frequent use of unstructured data (e.g. clinic notes), non-random missing data, and inconsistent data quality.[41,42] Additionally, generating multicenter datasets is often challenging because many EHRs, even from major vendors, store data in unique, institution-specific ways. Nonetheless, with continued efforts in areas such as natural language processing, opportunities to replace manual chart abstraction with sophisticated EHR queries continue to expand and are likely to assume a growing role in spine surgery research and quality improvement. Likewise, broad adherence to interoperability standards will facilitate the implementation of analytic pathways and clinical decision support across health systems.

Mobile Data

Mobile health (mHealth) is at the vanguard of biomedical informatics, with both researchers and “Big Tech” companies vying to capitalize on the increasing use of smartphones and wearable technology. By its nature, mHealth removes many barriers of having patients complete outcomes questionnaires, and in this way, might pave the way for seamlessly collecting population-based physical outcomes data. Indeed, there is expanding evidence for the role of mHealth in postoperative monitoring after spine surgery, [17,46,47] and there are increasingly available commercial applications intended to aid post-discharge patient surveillance.[48-50] Particularly notable, one study used a mobile application to aid postoperative monitoring for over 1,600 enhanced recovery after surgery patients. More generally, mHealth used in spine surgery has shown success in collecting patient-reported outcome measures, decreasing surgical cancellations, monitoring postoperative recovery,[17,46] and guiding postoperative rehabilitation. In other fields, mHealth applications have also been used to support behavioral modification related to factors that may impact spine surgery outcomes (e.g. cardiovascular disease, medication compliance).[54,55] Despite such promises, there remain important obstacles to more widespread use of mHealth. Several studies have shown that only a minority of patients use such applications regularly,[17,51] and despite promising reports, rigorous evidence demonstrating improved outcomes or decreased cost is lacking. For example, despite increasing use of mobile sensor data to study activity measures, such as step count,[57,58] there is sparse evidence demonstrating the extent to which such real-time measures reliably capture physical function or quality of life. Additional barriers to expanding mHealth include patient reservations related to privacy protection and technology familiarity, socioeconomic disparities in access,[59,60] and uncertainties related to data and evidence quality. Finally, there remains an ongoing need to integrate mHealth technology with existing EHR systems, which is often a complicated and costly endeavor. As these barriers are overcome, spine surgery practice will benefit from new efficiencies and care pathways, while researchers will derive new insights from high-frequency, real-world data collection.

Biobanks

Genomic, proteomic, and metabolomic (i.e., ‘omic’) data assets serve an essential role in tailoring treatment selection and outcome prediction to individual patient characteristics. Biobanks have been slow to take hold in spine surgery. Current spine-related biobanks focus on tumor samples, such as the Chordoma foundation biobank, and spinal cord injury.[63-65] However, novel insights regarding osteoarthritis from the UK Biobank demonstrate that other areas of spine surgery, particularly degenerative disease, could benefit from these pooled resources. To maximize their impact, ‘omic’ data should be integrated with more complete clinical information. Given the substantial resources required, more widespread adoption of spine surgery biobanks will require support from funding bodies and innovative solutions from data scientists, such as linking biochemical data with clinical EHR platforms.

Analytical Techniques

As important as selecting an appropriate dataset is the analytical approach used to investigate those data. While some authors describe a continuum between fully human-guided and machine-guided statistical techniques, we will distinguish traditional regression techniques from newer machine-guided approaches. Each of these analytical techniques contains multiple nuances and variations, including approaches to handling clustered and longitudinal data. Detailed reviews are available on such topics.[4,69-71] Our goal is to provide an overview of the key advantages and weaknesses of each approach, along with the applications each is best suited to address (Table 2).

Table 2.

Strengths, Weaknesses, and Ideal Use of Regression Versus Machine Learning Techniques.

	Regression models	Machine learning
Strengths	Familiar to researchers and clinical spine surgeons High model transparency Established techniques to test statistical significance of observed differences	Able to model complex patterns and unstructured data Not bound by pre-existing assumptions Superior predictive power (in some circumstances)
Weaknesses	Assumptions of linearity and additivity Difficulty modeling unstructured data Decreased predictive power (in some circumstances)	Decreased model transparency Higher sample size requirements Less familiar to spine surgeon researchers
Ideal Use	Risk models using structured data Conducting inference related to treatment outcome and cost Evaluating policy interventions	Modeling high volumes of unstructured data (e.g. real-time EHR output, mobile health data) Interpreting imaging data, mobile activity sensors

Regression Models

Regression models – including linear, logistic, and proportional hazards regression – are the traditional workhorse of statistical modeling. Regression models are generally designed to evaluate categorical and linear predictors, but techniques also exist for modeling non-linearity, including restricted splines and fractional polynomials. While several approaches exist to help automate variable selection and prevent overfitting,[73,74] variable selection and other modeling choices – such as interaction testing – remain heavily influenced by expert knowledge. While regression models are effective at risk prediction, they are particularly valuable for testing the statistical significance of observed variations, including surgical costs, clinical outcome, and health policy interventions.[11,75,76] Finally, regression results are generally easy to interpret, facilitating the identification of clinically relevant relationships and possibly enhancing surgeon acceptance of risk predictions.

Machine Learning

Machine learning refers to the intersection of statistics and computer science dedicated to using computing power to make predictions by recognizing patterns within data. Most applications of machine learning familiar to spine surgeons would be categorized as supervised learning, which involves training a model to predict a known outcome (e.g., postoperative complications, a fracture on CT).[71,77] By comparison, unsupervised learning involves using computers to detect new patterns in data, such as defining disease categories without preexisting constraints. Due to their advantages detecting novel classifications within high dimensional data, unsupervised approaches are likely to assume a dominant role in the future, though at present remain relatively uncommon in spine surgery and clinical medicine.

While variable interactions and spline transformations can extend regression techniques, they are largely bound by assumptions related to linearity and additivity (i.e., predictor variables have an additive effect on the outcome). By comparison, machine learning can accommodate much more complex patterns and unstructured data that may more accurately reflect spine surgery practice. A variety of machine learning techniques, including random forests, support vector machines, and convolutional neural networks have been developed for this purpose. Yet machine learning approaches have important shortcomings, including a lack of interpretability (i.e., the “black box” problem) or clinical applicability, and higher sample size requirements.[79,80] Advances in “interpretable machine learning” have helped address some of these shortcomings but still do not fully replicate an inherently interpretable modeling structure.

Selecting an Analytical Approach

Overall, regression techniques are better suited to making inferences (e.g. are outcomes from fusion better than decompression), given their greater transparency and well-defined approaches for determining statistical significance. Machine learning may offer advantages when engaging in prediction, though such gains are far from certain. Benefits of machine learning are likely to be most pronounced when dealing with complex datasets, and large sample sizes (e.g. thousands of cases) are often needed to yield stable predictions.[68,80] These limitations, combined with the relatively simple nature of many clinical datasets, likely explain the fact that machine learning approaches have often shown modest if any advantages compared to regression in many spine clinical prediction studies.[83-86] Consequently, investigations using machine learning for clinical predictions should demonstrate sufficient improvements in predictive performance to justify the loss of interpretability.

By comparison, machine learning has shown greater success when dealing with complex data assets, such as high volume EHR data, mobile sensors, and imaging analysis.[4,87] For example, machine learning approaches have been used to aid preoperative planning in deformity surgery,[88,89] and also to classify gait abnormalities based on mobile sensor data.[90,91] Likewise, machine learning approaches have proven effective at analyzing high-volumes of EHR data in real-time to aid postoperative risk predictions at the point-of-care. Other innovative efforts, such as integrating high-volume clinical and imaging data with expert opinion to improve patient classification in spondylolisthesis, are ongoing.

Future Perspectives

While regression techniques remain a mainstay in spine surgery research, there are a variety of approaches that have received scant attention and may open new analytic opportunities in the future. For example, multilevel models are well-suited to modeling hierarchical data (e.g. distinguishing patient vs. surgeon effects) as well as longitudinal trends (e.g. postoperative recovery trajectory).[94,95] Likewise, spine surgeons should consider making use of emerging techniques like generalized additive models, which allow substantial flexibility in modeling complex relationships while preserving interpretability. Finally, as large data assets continue to expand, so too will the role for machine learning techniques, particularly unsupervised approaches that may identify novel phenotypes of complex disease. Therefore, the emerging challenge for spine researchers is learning how best to deploy these powerful resources.

Implementation and Evaluation

Rigorous analytics applied to appropriate data assets serve as the foundation for effective HIT, such as clinical decision support predicting postoperative complications or tools to help select osteotomy sites for planning deformity correction. However, to effectively impact spine surgical practice, new HIT must be adopted by diverse stakeholders within complex healthcare systems. These challenges may be particularly prominent in spine surgery, where surgeon preference and institutional traditions remain important influences on management practices. Many of the concepts relevant to implementing HIT may be unfamiliar to spine surgeons, but identifying how and when such approaches can be used is key to moving biomedical informatics from the research setting into clinical practice.

Human-centered Design

Human-centered design (HCD) and evaluation refers to an iterative process that involves users throughout the design lifecycle to ensure that new HIT meets the needs and preferences of end-users.[98,99] After an initial HIT prototype is developed based on user-specified requirements, formal usability and usefulness testing should be completed in a simulated environment prior to clinical implementation. A number of mixed methods approaches can be employed to assess usability, such as the think-aloud technique, which elicits users’ thoughts and feelings as they use the new technology. This think-aloud approach has been used to evaluate a virtual reality vertebroplasty simulator and a novel outcome assessment tool for spine trauma, identifying potential problems and suggestions for improvement.[102,103] Another approach, cognitive walkthroughs, involves a trained evaluator analyzing the cognitive processes required to use new HIT, thereby identifying potential discrepancies between designers’ and users’ understanding of a task. This technique was used to evaluate a dashboard for presenting predicted patient-reported outcomes to spine surgery patients. Alternatively, heuristic evaluation uses human-computer interaction experts to identify usability problems based on established heuristic principles that may be missed with user testing.[104,105] This approach was used in combination with cognitive walkthrough to optimize the patient-reported outcome dashboard noted above. After completing these types of evaluations, field testing in clinical settings can reveal real-world problems not identified in a laboratory environment.[107,108] An exhaustive discussion of the HCD process is beyond the scope of this review, and many of the approaches involved, particularly the mixed methods techniques, may be unfamiliar to most spine surgeons. Consequently, surgeons seeking to implement new HIT should seek out methods experts to assist in this process.

Sociotechnical Analysis

Sociotechnical analysis provides a conceptual framework to evaluate the interconnected organizational, human, and technical elements impacting the adoption of HIT. Sociotechnical analysis focuses on the following aspects of implementation: the hardware and computing infrastructure; clinical content; human-computer interface; people; clinical workflow and communication; organizational policies, procedures, and culture; and system measurement and monitoring after implementation. In doing so, this approach provides a foundation for studying key implementation measures, such as barriers and context. Sociotechnical analysis is typically pursued through qualitative interviews with stakeholders, though surveys and EHR interrogation can also be used. This approach has rarely been used in spine research, though one study conducted a sociotechnical analysis to evaluate clinical video telehealth for spinal cord injury patients. There have also been limited successes using this approach to inform the implementation of clinical decision support in other surgical populations, such as patients with traumatic brain injury and patients requiring orthopedic imaging.[114,115] Spine surgeons developing new HIT should consider conducting a sociotechnical analysis to improve the likelihood that their intervention will be successfully integrated into clinical practice.

EHR Log Analysis

Traditional approaches to understanding how clinicians interact with HIT include interviews, surveys, and direct observation. While informative, such methods are labor and resource intensive and may not capture the full variability in care processes. Addressing these short-comings, EHR log analysis evaluates the time users spend performing different EHR-related tasks. This technique can be used to assess usage behaviors and clinical workflow, describe HIT demands, and evaluate the impact of HIT on care processes. This technique has been used to study time demands by surgical residents and currently represents an untapped opportunity for spine surgeons to collaborate with informatics experts to evaluate clinical practices and new HIT interventions.[117,118]

Implementation Trials

The most rigorous approach for evaluating new HIT is an implementation trial, which typically assumes a cluster-randomized design. These studies are often designed to evaluate effectiveness outcomes, such as a trial for a machine learning-derived early warning system for intraoperative hypotension. However, focusing only on effectiveness creates a missed opportunity to study key implementation outcomes, such as context, barriers, and facilitators. Implementation trials for spine disease have evaluated the role of mobile phone-based postoperative rehabilitation and an online application for managing low back pain, providing high-level evidence of the effectiveness of these interventions.[53,121] While labor and resource intensive, for high-stakes HIT interventions—including those that may warrant reimbursement from payers—implementation trials remain the gold standard for demonstrating an impact on health outcomes and care delivery.

Challenges and the Path Forward

Realizing the potential of biomedical informatics to transform spine surgery will involve navigating a variety of challenges and considerations, which are summarized in Figure 2. Among the most important challenges spine surgeons should consider are:

Figure 2.

A summary of the key challenges and considerations in developing, implementing, and maintaining health information technology.

Click Fatigue

With the increasing adoption of EHRs, spine surgeons, like most physicians, are inundated with alerts, more than half of which are overridden. To reduce click fatigue, researchers should focus on identifying when data analytics tools are most likely to impact clinical outcomes.[123,124] Likewise, adoption of HIT interventions will be enhanced by focusing on design strategies that reduce the cognitive workload demanded of busy spine surgeons.

Model Maintenance

Like any medical device, successful predictive models must be maintained over time and across different healthcare settings, adding to their long-term costs. Counterintuitively, the more effectively a model impacts practice and improves outcomes, the more its performance may diminish over time with changing conditions. Similarly, changing practice patterns and patient characteristics often lead to a decay in model performance over time. Furthermore, many predictive models suffer from poor portability across institutions, as was found in a model predicting infections after spine surgery. More efficient systems for sharing, testing, and updating prediction models across institutions are needed in spine surgery and medicine more broadly, particularly to make these tools accessible to smaller institutions with limited information technology resources.

Regulation and Oversight

As HIT interventions assume increasingly prominent roles in spine surgery practice, the role of government regulation must be defined. A recent review found that nearly half of healthcare applications did not describe their content source, and several popular healthcare applications have been removed for poor clinical accuracy. Given the high-risk nature of spine surgery, surgeons seeking to broadly implement new HIT (e.g. to guide patient selection) should proactively consider engaging with regulatory bodies to preserve innovation while ensuring the rigor of HIT interventions.

Stakeholder Acceptance

To increase acceptance of HIT among spine surgeons, researchers must address doubts related to the quality of their underlying evidence and how these interventions interact with existing clinical practices. Soliciting diverse surgeon feedback early in HIT development is therefore key to decreasing conflict between established practices and new interventions. Finally, data analytics tools should augment rather than replace clinical experience, and explicitly incorporating surgeon judgment into predictive models may enhance stakeholder acceptance.

Ethical Challenges

Relying on purely data-driven, particularly machine-based predictions to guide spine surgery decision-making has the potential to accentuate disparities based on race and socioeconomic status. Specifically, models built to mimic human decision-making may reinforce known disparities in treatment access and outcomes. Furthermore, data assets may not contain adequate representations of minority groups, leading to decreased predictive performance in those populations.[133,134] Recognizing these potential challenges will allow spine surgeons to maximize the ethical use of HIT.

Conclusions

The growth in HIT has provided access to data and computing resources previously unattainable in spine surgery, which has contributed to a rapid rise in informatics research. Like nearly all technology, biomedical informatics in spine surgery is subject to the “hype cycle model” described by Gartner Inc., summarizing the path toward sustained use of new innovations. At present, we are likely experiencing the peak of inflated expectations. To truncate the trough of disillusionment associated with unmet expectations, spine surgery researchers should recognize the strengths and limitations of diverse data assets and analytic tools, while also leveraging effective HIT implementation strategies. Through navigating these complex considerations, spine research may move toward a plateau of productivity, where new HIT innovations produce meaningful advances in spine surgery quality and outcomes.