The presence of spin in systematic reviews focused on diabetic neuropathy: A cross-sectional analysis.

BACKGROUND: Spin-the misrepresentation of a study's actual results-has the potential to alter a clinician's interpretation of the study's findings and therefore could affect patient care. Studies have shown spin frequently occurs in abstracts of systematic reviews from a variety of other medical disorders and specialties. AIMS: Our primary aim was to evaluate whether the nine most severe types of spin occurred in systematic review abstracts' concerning diabetic neuropathy treatments. Secondly, we aimed to determine whether spin presence was associated with the methodological quality of a systematic review.
METHODS: A search of MEDLINE and Embase collected 1297 articles focused on diabetic neuropathy treatments, of which we included 114 systematic reviews for spin assessment. Each included study was evaluated for the nine most severe types of spin as defined by Yachitz et al. The methodological quality of a systematic review was determined by using the AMSTAR-2 instrument. All screening and data extraction were conducted in a masked, duplicate fashion. Since the final sample size of 114 was not sufficiently powered to do multivariable logistic regression, we calculated unadjusted odds ratios which evaluated relationships between spin presence within abstracts and study characteristics.
RESULTS: From the 114 articles reviewed, spin was present in 7.9% of the studies (9/114), with spin type 5: "conclusion claims the beneficial effect of the experimental treatment despite the high risk of bias in the included primary studies" as the most frequent in our study. Spin types 1, 2, 6, and 8 were not identified. No association was observed between the study characteristics and spin presence, including the methodological quality of a systematic review.
CONCLUSIONS: Overall, spin is infrequently observed in abstracts of systematic reviews covering diabetic neuropathy treatments. When comparing our results to other fields of medicine, the field of diabetic neuropathy research publishes systematic reviews whose abstracts mostly portray the findings of the review's full-text to reflect the results adequately.

Introduction

Nearly 10% of the United States adult population is living with diagnosed diabetes (either type 1 or 2), with an additional 3% estimated to be living with undiagnosed diabetes [1]. Among those living with diabetes, diabetic neuropathy is one of the most common clinical complications, present in over 30% of these individuals [2,3]. Apart from strict glucose control, few therapeutic agents are FDA approved to control the symptoms of diabetic neuropathy [3]. In order to provide the appropriate care to patients with diabetic neuropathy, physicians must stay updated through research and various scientific literature. Systematic reviews and meta-analyses serve as an excellent method for physicians to keep their medical knowledge current.

Systematic reviews are widely recognized as the gold standard for a high level of research evidence and typically represent the most thorough summaries of current literature [4]. Systematic reviews use explicit systematic methods to reduce the risk of bias in the review process, therefore they are considered the highest level of evidence [5]. Given this high standing, they are supposed to report findings free of spin. Yachitz et al. classify spin based on three overarching categories: misleading reporting, misleading interpretation, and inappropriate extrapolation [6]. While the entirety of a scientific work should maintain a high professional standard, there is one section that should be more thoroughly scrutinized—the abstract. As the portion most commonly read by clinicians, the abstract has the greatest potential to alter the course of patient treatment and outcomes [7].

Spin has been identified as a broadly used form of bias in randomized controlled trials within various fields, as well as, specific disease states [8,9]. However, discrepancies have been found between specialties. For example, 70% of otolaryngology abstracts contain spin in comparison to 44.3% of emergency medicine abstracts, and 27.3% of cardiology abstracts [9-The Journal of the American Osteopathic Association. U.S. National Library of Medicine. 2020 ">11]. Due to the variability in results of past research, further exploration is warranted to examine the presence of spin in additional fields. One unexplored topic is diabetic neuropathy and the presence of spin within treatment options. As diabetic neuropathy is highly prevalent in the US, the evaluation and maintenance of scientific integrity are vital.

The purpose of this study was to analyze the nine most severe types of spin as defined by Yachitz et al. [6] within abstracts of systematic reviews focused on diabetic neuropathy treatment options. Our secondary objective was to determine whether spin was associated with particular characteristics of systematic reviews, including their methodologic quality.

Methods

Protocol and reporting

To ensure our study’s reproducibility and transparency, this study’s protocol and data analysis scripts can be found in Open Science Framework (https://osf.io/7e5nd/) [12]. Additional studies examining the presence of spin in systematic reviews in different medical fields were conducted simultaneously with this study. Additional statisticians repeated analysis to further ensure the reproducibility of our work. We used Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [13], as well as Murad and Wang’s [14] guidelines for the composition of the manuscript.

Search strategy and screening

One of us (DW), a systematic review librarian, systematically searched Embase and MEDLINE to locate systematic reviews limited to the management of diabetic neuropathy treatment (Fig 1). The librarian’s searches were completed on June 2, 2020, with the search returns uploaded to Rayyan [15], a systematic review screening platform. After removing duplicates, in a masked, duplicate fashion, investigators (AK and HR) screened remaining articles by titles and abstracts to determine eligibility. Any discrepancies were noted and resolved by the two researchers through a consensus meeting.

Fig 1

The search strategy to obtain systematic reviews.

Eligibility criteria

For an article to be included in this study, the following must have been met: (1) the article must be a systematic review with or without a meta-analysis, (2) the review is focused on the management of diabetic neuropathy, (3) the article is retrievable in English, and (4) the article includes human participants. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) definition was used to define a systematic review [16].

Training

An online training course for systematic reviews and meta-analyses by Li and Dickersin [17] was completed by investigators before screening commenced. The investigators also participated in four days of training focused on the definitions and identification of the nine most severe spin types that occur in systematic review abstracts as described by Yavchitz et al. [6] Lastly, investigators completed training over A MeaSurement Tool to Assess Systematic Reviews (AMSTAR-2) [18], an appraisal instrument used to determine the methodological quality of each included systematic review and meta-analysis. A comprehensive outline of author training can be found in our protocol published on Open Science Framework [12].

Spin data extraction

The investigators (AK and HR) individually assessed every included systematic review for the presence of the nine most severe types of spin using a pilot-tested Google form for all data extraction—used previously by our team in another spin study [19]. The definition of these nine spin types can be found in Table 1. Following extraction, the two investigators (AK and HR) reviewed and discussed any inconsistencies within their results. When an agreement could not be reached, an additional author (RO) was available for adjudication.

Table 1

Spin types and frequencies (%) in abstracts.

Nine most severe types of spin	No. (%), containing spin
1) Conclusion contains recommendations for clinical practice not supported by the findings.	0 (0)
2) Title claims or suggests a beneficial effect of the experimental intervention not supported by the findings.	0 (0)
3) Selective reporting or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention.	2 (1.8)
4) Conclusion claims safety based on non-statistically significant results with a wide confidence interval.	1 (1)*
5) Conclusion claims the beneficial effect of the experimental treatment despite the high risk of bias in the included primary studies.	5 (4.4)
6) Selective reporting or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention.	0 (0)
7) Conclusion extrapolates the review’s findings to a different intervention (i.e., claiming efficacy of one specific intervention although the review covers a class of several interventions).	1 (0.9)
8) Conclusion extrapolates the review’s findings from a surrogate marker or a specific outcome to the global improvement of the disease.	0 (0)
9) Conclusion claims the beneficial effect of the experimental treatment despite reporting bias.	3 (2.6)

*9 did not assess safety, therefore n = 105.

Data extraction of general characteristics

Additionally, investigators (AK and HR) extracted the following study characteristics from each systematic review: intervention type (pharmacological or non-pharmacological), whether a review discussed compliance with PRISMA [20], whether the associated journal required the compliance with PRISMA, the journal’s 5-year impact factor, the study’s funding source (not funded, industry, not mentioned, private, or public), and date the journals received the systematic reviews. Again, independent data extraction occurred with authors masked to each other’s responses. Following data extraction, authors were unmasked and discrepancies were resolved by group discussion.

AMSTAR-2 rating

The methodological quality of each article was assessed via AMSTAR-2 (https://amstar.ca/), which rates systematic reviews from ‘high’ to ‘critically low’ based on a list of 16 criteria. The investigators (AK and HR) independently evaluated every included article utilizing the AMSTAR-2 tool, entering all data into a Google form. Articles that received different scores from the investigators were reexamined. All conflicts were resolved. The AMSTAR-2 has high reliability and validity compared to other systematic review appraisal tools, such as the original AMSTAR and Sach’s Instrument [18].

Statistical analysis

The study characteristics, spin frequency, and AMSTAR-2 ratings were reported as percentages and counts. In the protocol, we clarified the possibility of a binary logistic regression and assessed a power analysis to determine sample size from a previous study by our team [19]. Since the final sample size of 114 was not sufficiently powered to do multivariable logistic regression, we calculated unadjusted odds ratios which evaluated relationships between spin presence within abstracts and study characteristics. StataCorp 16.1 was used for the analyses [21].

Results

Characteristics

Our searches returned 1,297 articles, with 258 excluded as duplicates. During screening an additional 758 were excluded during abstract screening—578 because they were unrelated to diabetic neuropathy, 96 because they were ineligible by automation tools, 17 because they used animal subjects, and 5 because they were not related to diabetic neuropathy treatment. During the full-text review we excluded an additional 167 articles—68 due to wrong study design, 38 due to no relation to treatment, 18 due to the articles being only published abstracts, 16 due to being unavailable in English, 13 due to no full text being available, 7 due to articles only being protocols, 3 due to using animal studies, 3 due to articles being withdrawn, and 1 due to article being an erratum—resulting in a final sample size of 114 systematic reviews. The exclusion rationale and screening process are illustrated in Fig 2. The primary type of intervention was pharmacological therapies (77/114, 67.5%), followed by non-pharmacological interventions (37/114, 32.5%), and surgical interventions (0/114, 0%). The majority of the included studies did not mention adherence to PRISMA (81/114, 71.1%). Additionally, 74.6% of the journals did not recommend authors adhere to PRISMA guidelines (85/114). Of the 114 systematic reviews, 22 studies did not include a funding statement (21/114, 18.4%). Of the remaining, 77 received funding (67.5%), with public funding being the most common form of funding (44/114, 38.6%). Twenty-one studies did not include a funding statement (21/114, 18.4%), and 16 reported no funding (16/114, 14.04%). The dates the publishing journal received the systematic reviews ranged from 1993 to 2019 (Table 2).

Fig 2

Flow diagram of study selection.

Table 2

General characteristics of systematic reviews and meta-analyses.

Characteristics	No. (%) of Articles (n = 114)
	Total (%)	Abstract Without Spin	Abstract With Spin	Odds Ratio (95% CI)
Intervention type
Non-pharmacologic	37 (32.5)	33 (28.9)	4 (3.5)	1 [Ref]
Pharmacologic	77 (67.5)	72 (63.2)	5 (4.4)	0.57 (0.14–2.27)
Article mentions adherence to PRISMA
No	81 (71.1)	75 (65.8)	6 (5.3)	1 [Ref]
Yes	33 (28.9)	30 (26.3)	3 (2.6)	1.25 (0.29–5.32)
Publishing journal recommends adherence to PRISMA
No	85 (74.6)	77 (67.5)	8 (7.02)	1 [Ref]
Yes	29 (25.4)	28 (24.6)	1 (0.9)	0.34 (0.04–2.87)
Funding source
Not Funded	16 (14.04)	15 (13.2)	1 (0.9)	1 [Ref]
Industry	22 (19.3)	20 (17.5)	2 (1.7)	1.5 (0.12–18.13)
Not Mentioned	21 (18.4)	20 (17.5)	1 (0.9)	0.75 (0.04–12.99)
Private	11 (9.6)	10 (8.8)	1 (0.9)	1.5 (0.08–26.86)
Public	44 (38.6)	40 (35.1)	4 (3.5)	1.5 (0.15–14.52)
AMSTAR-2 Rating
High	9 (7.9)	8 (7.02)	1 (0.9)	1 [Ref]
Moderate	31 (27.2)	27 (23.7)	4 (3.5)	1.19 (0.12–12.17)
Low	39 (34.2)	37 (32.5)	2 (1.7)	0.43 (0.03–5.37)
Critically Low	35 (30.7)	33 (28.9)	2 (1.7)	0.48 (0.39–6.04)
Journal Impact Factor, M (SD)	4.77 (4.37)	4.93 (4.47)	2.82 (2.10)	0.75 (0.50–1.12)

Spin in abstracts

Spin was identified in 9 of the 114 systematic review abstracts pertaining to diabetic neuropathy treatment (7.9%) (Table 1). However, certain abstracts contained multiple types of spin; thus, a total of 12 occurrences of spin were found. The most prevalent spin type was type 5—“conclusion claims the beneficial effect of the experimental treatment despite the high risk of bias in the included primary studies”—which occurred in 5 abstracts (5/114, 4.4%). Spin type 9—“conclusion claims the beneficial effect of the experimental treatment despite reporting bias”—was the second most frequent spin type, present in 3 abstracts (3/114, 2.6%). Nine studies did not mention safety measures in their abstract conclusion; therefore, spin type 4 was only observed in 1 of 105 studies (<1%). Spin types 1, 2, 6, and 8 did not occur in any abstracts. Our logistic regression models showed no significant associations between presence of spin within abstracts and a study’s adherence to PRISMA, intervention type, funding source, nor if the publishing journal recommends PRISMA adherence or its 5-year impact factor (Table 2).

AMSTAR-2 rated 7.9% of studies as ‘high’ quality (9/114), 27.2% as ‘moderate’ quality (31/114), 34.2% as ‘low’ quality (39/114), and 30.7% as ‘critically low’ quality (35/114) (Table 2). Approximately 99.1% of the systematic reviews developed their research question based on the Population, Intervention, Comparator group, Outcome (PICO) method (113/114) (Table 3). Additionally, 86.8% of authors explained the study selection design for inclusion (99/114). In contrast, 7% (7/114) of the systematic reviews used a comprehensive literature search strategy, and 14% (16/114) reported on the sources of funding for their analyzed reviews.

Table 3

AMSTAR-2 items and frequency of responses.

AMSTAR-2 Item	Response, n (%)
	Yes	No	Partial Yes
1) Did the research questions and inclusion criteria for the review include the elements of PICO?	113 (99.1)	1 (0.9)	-
2) Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol?	51 (44.7)	37 (32.5)	26 (22.8)
3) Did the review authors explain their selection of the study designs for inclusion in the review?	99 (86.8)	15(13.2)	-
4) Did the review authors use a comprehensive literature search strategy?	8 (7.02)	27 (23.7)	79 (69.3)
5) Did the review authors perform study selection in duplicate?	91 (79.8)	23 (20.2)	-
6) Did the review authors perform data extraction in duplicate?	93 (81.6)	21 (18.4)	-
7) Did the review authors provide a list of excluded studies and justify the exclusions?	30 (26.3)	79 (69.3)	5 (4.4)
8) Did the review authors describe the included studies in adequate detail?	56 (49.1)	16 (14.04)	42 (36.8)
9) Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?	59 (51.8)	34 (29.8)	19 (16.7)
10) Did the review authors report on the sources of funding for the studies included in the review?	16 (14.04)	98 (86.0)	-
11) If meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results?*	64 (56.1)	17 (14.9)	-
12) If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?*	58 (50.9)	23 (20.2)	-
13) Did the review authors account for RoB in primary studies when interpreting/discussing the results of the review?	78 (68.4)	36 (31.6)	-
14) Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?	65 (57.02)	49 (43.0)	-
15) If they performed quantitative synthesis, did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?*	31 (27.2)	50 (43.9)	-
16) Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review	87 (76.3)	27 (23.7)	-

*33 systematic reviews did not perform a meta-analysis.

- Partial Yes was not applicable.

Discussion

General findings

Our study identified five of the nine most severe types of spin in systematic review abstracts involving diabetic neuropathy treatments. The most frequent type of spin was type 5: “conclusion claims the beneficial effect of the experimental treatment despite the high risk of bias in the included primary studies.” One example of spin type 5 is from the abstract of a systematic review covering the efficacy of duloxetine in treating painful neuropathy, chronic pain, or fibromyalgia [22]. In the abstract, the authors claim a daily dose of 60 mg and 120 mg of duloxetine was an effective treatment for individuals with diabetic neuropathic pain. However, an examination of the article’s full text revealed the majority of the primary studies included in this review had either a high risk of bias or the risk of bias was unclear. Omitting the risk of bias assessments from the abstract, as seen in this study, is problematic as studies with a high risk of bias can change the magnitude and direction of the study’s outcomes–studies with a high risk of bias yield greater, and often incorrect, effect sizes as compared to studies with low-risk of bias [23,24]. As healthcare professionals often solely rely on the abstract [7], it becomes apparent a clinician’s interpretation could be distorted. Therefore, we recommend authors of systematic reviews declare whether their included primary studies are at high risk of bias so clinicians can be cautious when interpreting the study’s results.

Investigating spin in medical research has been of growing interest and has been well studied in randomized controlled trials. When comparing our results to other spin studies, spin occurs at a much lower frequency in systematic reviews focused on the management of diabetic neuropathy. The results of this study are encouraging as research over diabetic neuropathy seems to be at lower risk of containing spin compared to other fields. With these encouraging findings, the goal for diabetic neuropathy research should be focused on further eliminating the small amount of spin which occurs and preventing spin from occurring in future studies.

Recommendations

Studies have shown misleading results in the abstract often go unrecognized by peer reviewers. For example, one study found reviewers failing to identify spin in nearly 80% of the studies reviewed [25]. In fact, this same study found 15% of reviewers asked the authors to include some form of spin in their abstract’s conclusion [25]. Based on these findings, we first recommend spin training be provided to reviewers and editors as this would likely increase spin recognition and ultimately lead to its prevention. Additionally, we recommend the establishment of reporting guidelines which directly address spin and its subtypes as current guidelines fail to address spin. By having these guidelines, authors would be able to eliminate spin as it may be unintentionally included in the abstract. Lastly, we recommend clinicians using caution when interpreting the results of a study from only the abstract. Education covering spin may be beneficial to physicians and physicians in training.

Strengths and limitations

Our study’s strengths lie in its transparency, reproducibility, and measures to mitigate the effect of bias. All study materials were uploaded online prior to beginning our study, and any potential deviations from our protocol were recorded in an openly available update on OSF. Additionally, literature was reviewed in a double-masked fashion, the method deemed as the “gold standard” by the Cochrane Collaboration [26]. Limitations of our study include its cross-sectional nature, which reduces its generalizability, and the use of only MEDLINE and Embase to conduct our search, which may have lessened our sample size. Identifying spin is an inherently subjective process. However, to mitigate this subjectivity, investigators underwent intensive training to better characterize and identify spin.

Conclusion

Overall, spin is infrequently observed in abstracts of systematic reviews covering diabetic neuropathy treatments. When comparing our results to other fields of medicine, systematic reviews related to diabetic neuropathy more often accurately portray the findings of the articles’ full-text. Given that nearly 8% of these systematic reviews did contain spin, further reducing spin would lessen potential bias within these articles, which may be beneficial to other researchers, healthcare providers, and their patients.

28 Feb 2022

The Presence of Spin in Systematic Reviews Focused on Diabetic Neuropathy: A Cross-Sectional Analysis

PLOS ONE

Dear Dr. Khan,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

A premise for acceptance is that the authors contributions become crystal clear.

The study protocol should be submited as addtional material.

Please submit your revised manuscript by Apr 08 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Tim Mathes

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/fileid=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf".

2. Thank you for stating the following financial disclosure:

“MV received the award.

The development of our protocol was funded by the Oklahoma State University Center for Health Sciences Presidential Mentor-Mentee Research Fellowship Grant.”

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.""

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in the Funding Section of your manuscript:

“The development of our protocol was funded by the Oklahoma State University Center for Health Sciences Presidential Mentor-Mentee Research Fellowship Grant.”

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

“MV received the award.

The development of our protocol was funded by the Oklahoma State University Center for Health Sciences Presidential Mentor-Mentee Research Fellowship Grant.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

5. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Khan and colleagues performed a systematic review to evaluate the presence of nine of the most severe types of spin (as identified by Yachitz et al.) in the abstracts of systematic reviews on diabetic neuropathy. This work builds upon a number of similar evaluations of spin of a variety of disciplines by their group, and therefore the results can be compared to other research topics, which they have done to an extent in the discussion. The methods appear to be rigorously executed from what is provided, although not all materials are able to be evaluated by the reviewer to make a final determination (see first comments).

Major comments

"All relevant data are within the manuscript and/or Supporting Information files"

- Unless I missed it, I do not see raw data available to review.

Line 86: "To ensure our study’s reproducibility and transparency, this study’s protocol, extraction forms, and data analysis scripts can be found in Open Science Framework (OSF) [link]."

- This link is not active (permission error) for the reviewer to check these items. Please allow public access or attach the items for review if there is a reason for these items not to be public before publication.

PRISMA flow diagram

- Please provide enough information so a reader can reproduce your process of using automation tools that were used to screen records, and so readers can assess the reliability of doing this.

Do the following statements contradict one another?

- "Since the final sample size of 114 was not sufficiently powered to do multivariable logistic regression, we calculated unadjusted odds ratios which evaluated relationships between spin presence within abstracts and study characteristics."

- "Our logistic regression model found no significant association between abstracts with spin and a study’s PRISMA adherence, intervention type, if the publishing journal recommends PRISMA adherence, funding source, or the 5-year impact factor of the publishing journal (Table 2)."

Minor comments

Abstract

- Conclusion: I agree that compared to other fields, review abstracts on diabetic neuropathy seem to contain less spin. Yet, 8% of studies still seems unacceptable, and therefore I would not use the language that they "accurately portray the findings of the review's full-text". Your conclusion (line 211) seems to agree that this is encouraging but we should not be complacent.

Line 69: "physicians must stay updated through research and various learning platforms".

- Do you mean by reading the scientific literature?

70: "Comprehensive study analyses which comprise multiple studies" -> change to "Systematic reviews and meta-analyses", or rephrase another way.

Language

- Please edit the manuscript carefully for grammar and spelling, including the abstract.

Reviewer #2: Thank you for the opportunity to review the manuscript entitled “The Presence of Spin in Systematic Reviews Focused on Diabetic Neuropathy: A Cross-Sectional Analysis”.

The authors have performed a meta-research study that consisted of a systematic search for systematic reviews on diabetic neuropathy treatment and an analysis of the prevalence of different types of spin. They also extracted review characteristics and assessed the included review’s methodological quality using AMSTAR 2, and then evaluated whether there was an association between these characteristics or quality and the prevalence of spin.

While this is an important field of study and valuable research question, there are several aspects concerning the manuscript that should be addressed:

Major:

1. Apart from AK, HR, DW and MV, it is currently unclear how the co-authors were involved in the study process. Please provide an author statement using the CRediT Taxonomy.

2. Introduction: Argumentation:

a. I cannot entirely follow the argumentation around systematic reviews, e.g. “Because systematic reviews are the gold standard it is important that they are conducted without bias or misleading findings”. I would consider it the other way around: By definition, systematic reviews use explicit, systematic methods to reduce the risk of bias in the review process (bias cannot be eliminated), therefore they are considered the highest level of evidence (for most questions). Given this high standing, they are supposed report findings in a way that is not misleading. However, while spin has long been known to occur in RCTs, there is now evidence for spin in systematic reviews, too (citations of such studies).

b. The concept of spin should be elaborated more in the argumentation, e.g. introduce the classification by Yachitz before the aims & objective statement.

c. The point about physicians often only reading abstracts should also be moved up to the argumentation, i.e. before the aims & objective statement.

3. Introduction: Choice of references: It is unclear why spin in anesthesiology and emergency medicine RCTs is mentioned here; I would suggest citing studies of spin in RCTs on diabetic neuropathy treatment, if available, or an evaluation of spin in RCTs that is not focused on a specific field. If there is a reason to believe that the medical field influences the amount of spin, it would be interesting to cite such evidence in the introduction. Regarding spin in abstracts of systematic reviews: it would be interesting to present a brief overview of the findings of the other meta-research studies conducted by the team around MV (at least 18 of which have already been published according to a PubMed search).

4. Methods: Protocol and Reporting: The files in OSF do not appear to be public (when I tried it (as a logged-in OSF user), it said “You Need Permission”). Please make your files public, otherwise the reproducibility and transparency of your study (which you name as a strength) is not given.

5. Methods: Protocol and Reporting: Please provide more information on the third-party analytical team.

6. Methods: Protocol and Reporting: While I greatly encourage the use of reporting guidelines, it is unclear to me why PRISMA 2020 was used for this meta-research study. It also appears to me that there were some misunderstandings, i.e., in the PRISMA checklist it says “Yes/Table 3” for the items 13e (Describe any methods used to explore possible causes of heterogeneity among study results (e.g. subgroup analysis, meta-regression)) and 14, neither of which was applicable in this study. Furthermore, line numbers need to be updated if use of PRISMA 2020 is continued (or changed to page numbers may be more feasible).

7. Methods: Data Extraction: The column in Table 1 that contains results must be reported in the results section please. The remaining table would benefit from examples and should contain the citation of Yachitz et al.

8. Methods: Data Extraction: The description of AMSTAR-2 assessment should be an extra paragraph containing an explanation of how the overall assessment was derived at. Furthermore, the tool needs to be cited correctly (https://www.bmj.com/content/358/bmj.j4008) and it is currently unclear what you mean by “high construct validity coefficients based on the original AMSTAR-2 instrument (r = 0.91) and the (r = 0.84).[16]”.

9. Methods: Data Extraction: Please provide the categorization system for each characteristic and add that you also extracted the date the journals received the systematic reviews.

10. Results: Characteristics: Please provide a list of included reviews as well as a list of exclusions with reasons.

11. Results: AMSTAR-2 Rating: Please provide the results per individual review in a supplemental file.

12. Discussion: Please discuss potential reasons why systematic reviews focused on the management of diabetic neuropathy tend to have less spin than systematic reviews from other fields.

13. Discussion: It is unclear why Ottwell et al. and Heigle et al. are being cited. It would be more informative to report the range of spin across the systematic review abstracts of various clinical disciplines that the team around MV have investigated.

Minor:

14. To better differentiate between studies as in other meta-research studies and the included articles, I would suggest to re-word them to “included reviews”, “analysed reviews”, etc.

15. Abstract: Methods: “…, of which included 114 systematic reviews for spin assessment.” Suggest to insert “we” in front of “included”.

16. Abstract: Methods: Please state that you mean the nine most severe spin forms as defined by Yachitz et al. 2016.

17. Abstract: Methods: Please include a brief description of the analyses performed.

18. Abstract: Results: “No association exists”, please reword to “was observed”.

19. Abstract: Conclusions: Please change to “whose abstracts mostly portray the findings of the review’s full-text accurately.” to reflect the results adequately.

20. Methods: Search Strategy: Please rename section to “search strategy and screening” and describe if there were any limits to the search and how you used the automation function of Rayyan.

21. Methods: Fig 2 is mentioned before Fig 1, please correct.

22. Results: Figure 1: It should “diagram” not “diaphragm”. Please provide a legend for the asterisks and correct the number of records identified from databases (= MEDLINE and Embase) to 1297 and the number of records identified from registers to 0.

23. Results: Characteristics: “The primary type of intervention was pharmacological therapies (77/114, 67.5%), followed by non-pharmacological interventions (37/114, 32.5%), and nonsurgical interventions (0/114, 0%).” Do you mean “surgical interventions”? Otherwise, the categories are not mutually exclusive.

24. Results: Characteristics: “77 received funding (67.5%)” this value is not completely correct, as 21 did not mention funding but may have been funded. Suggest to re-organize to: “Twenty-one studies did not include a funding statement (21/114, 18.4%). Of the remaining, 77 received funding (67.5%), with public funding …”

25. Results: Table 2: Journal Impact Factor, M (SD): What is the value in the fourth column? If it is an odds ratio, what was the reference?

26. Results: Table 3: In the right column, please indicate the items where partial yes was not applicable.

27. Conclusion: Please change to “whose abstracts mostly portray the findings of the review’s full-text accurately.” to reflect the results adequately.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Colby Vorland

Reviewer #2: Yes: Tanja Rombey

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

22 Apr 2022

All comments are provided in our documentation titled “Response to Reviewers” which is attached to the manuscript submission. We are truly grateful for all the feedback received from the reviewers, as it truly allowed our manuscript to improve.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

23 May 2022

15 Aug 2022

All of the responses for each of the reviewer and editor comments are viewable in the Second Rebuttal Letter. We thank the reviewers for allowing us the opportunity to improve our manuscript.

Submitted filename: Response to the Reviewers- The Second Rebuttal Letter.docx

Click here for additional data file.

6 Sep 2022

The Presence of Spin in Systematic Reviews Focused on Diabetic Neuropathy: A Cross-Sectional Analysis

PONE-D-21-21584R2

Dear Dr. Khan,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Tim Mathes

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: Thank you for addressing all my previous comments!

I do not have any further comments other than I believe it would be beneficial for the reader if the explanation for the interpretation of the OR for the JIF would be included in a legend below the table. However, I leave this decision to the authors.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Colby Vorland

Reviewer #2: Yes: Tanja Rombey

**********

11 Sep 2022

PONE-D-21-21584R2

The Presence of Spin in Systematic Reviews Focused on Diabetic Neuropathy: A Cross-Sectional Analysis

Dear Dr. Khan:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Tim Mathes

Academic Editor

PLOS ONE