Response to: Letter to the Editor, Minimally Invasive versus Open Surgery for Spinal Metastasis: A Systematic Review and Meta-Analysis.

Dear Editor & Drs. Ali and Tariq,

We appreciate your interest in our publication and the time taken to read and thoughtfully critique our work [1]. We believe in constructive criticism and cherish the opportunity to both address concerns raised by Drs. Ali and Tariq and further improve our work.

On the first point raised by Drs. Ali and Tariq concerning Prospective Register of Systematic Reviews (PROSPERO) registration: we routinely register planned reviews on PROSPERO and are fully aware of its intended purpose of transparency and reducing study duplication [2-5]. However, for countries other than the United Kingdom, the waiting time prior to the pandemic was at least 3 months, and in our experience, even longer. Due to current world affairs, PROSPERO is not actively reviewing submissions and is uploading them “as-is” prior to initial screening if unrelated to coronavirus disease 2019, thus its capacity as a quality-control filter is currently limited. Additionally, in our experience, we have found that full compliance with registration does not prevent other authors, from performing the same or similar studies [6-8].

Due to the aforementioned factors, we have decided not to pursue registration when we anticipate a study will take us less than a month from inception to submission, as we estimate publication would take an approximately similar time to the registration certification and upload. This defeats its use in preventing study duplication and our methods are transparent enough to allow and encourage this kind of discussion. Such is the case of this manuscript, which as detailed in methods, was initiated in November and submitted in December. Additionally, in this study’s specific case, we had already performed a small version of the analysis for in-house outcomes grand rounds session, which, per PROSPERO rules, would have made it ineligible for registration [9].

A recent study in vaccine research found that registration was one of the least complied factors within guidelines, finding only 26% of compliance [10]. Similar studies on ophthalmology reported again that registration was the least complied guideline at 9%, 19.4% in maxillo-craniofacial surgery, and 14% in neuroendovascular reviews [11-13]. Prior iterations of similar studies have also been successfully performed without registration [14,15]. Thus, given the time implication, that it does not actively prevent other studies from being performed, and the current relaxed standards, we believe registration shouldn’t be a barrier to performing and publishing reviews and currently reported compliance seems to reflect similar sentiments.

Regarding the second point mentioned on heterogeneity, we discuss the matter in the closing statements of the discussion section; we believe heterogeneity comes from unaccounted variations in concurrent treatment and staging from patients across different studies as well as from differences in indications for surgical intervention, which is difficult to control for and we even encouraged future dedicated studies. However, and again as we state various times in the manuscript, this is a difficult population to study and this should not preclude current attempts of analyzing available evidence.

We typically perform stepwise addition analysis of studies on I2 % to assess which studies induce heterogeneity and might impact results and determine if further stricter reanalysis or subgrouping is needed as recommended by Cochrane and have done so successfully in the past [3]. In this paper’s case, we found that heterogeneity changed effect size but not significance or direction in any analyzed variable’s result except for operative time, in which we opted for a more conservative conclusion. We typically perform sensitivity analysis to a degree in all of our work; however, the most accurate way to explore this heterogeneity is a meta-regression. In either case, Review Manager ver. 5.3 (RevMan; Cochrane, London, UK) does not feature dedicated functions for these processes [16,17].

Thirdly, on publication bias, other than a funnel plot, Review Manager ver. 5.3 (Cochrane) does not offer additional tools for publication bias such as Egger’s test. While we routinely assess funnel plots for symmetry, we find it to be too arbitrary to give any credible weight and attempt to correlate our findings with prior published literature. Funnel plots alone have been found to have limited use and studies have found researchers’ limited ability to adequately interpret funnel plots [18,19].

Lastly, we find McGrath’s method to be interesting and will undoubtedly consider it for future studies. However, it is important to notice that while the publication of McGrath et al. [20]~ is around 2 years, it has only been cited approximately 40 times compared to the publication of Wan et al. [21] (2,765), suggesting it’s yet to be more widely adopted and therefore we believe it to be an unfair assumption to expect it to be the default method for data estimation. Additionally, we specifically stated in our limitations section that data estimation was a limitation of our study but will look to incorporate this possible more accurate data estimation tool in future studies.

Once again, we appreciate the feedback and hope to have provided adequate responses to Drs. Ali and Tariq’s concerns.