A systematic review protocol for reporting deficiencies within surgical case series.

INTRODUCTION: Case series are an important and common study type in surgical literature. There is evidence that key data are excluded from published case series, and currently no reporting guideline exists for case series. There is, therefore, the potential to change practices and improve the reporting of case series. Reporting guidelines have been shown to be efficacious in raising the bar for reporting quality. We present our protocol for the first stage of guideline development--a systematic review of previously identified deficiencies in how surgical case series are reported. METHODS AND ANALYSIS: Electronic searches will be conducted on MEDLINE, EMBASE, Cochrane Methods Register, Science Citation Index and Conference Proceedings Citation Index, from the start of indexing until 5 November 2014. The electronic search strategy was developed with an information specialist. Two independent researchers will identify articles for inclusion, specifically those that describe reporting deficiencies within surgical case series. Data will be extracted to specifically focus on the deficiencies of reporting. These will be categorised according to their type, and other identified issues will also be presented. Data will be presented with descriptive statistics to determine frequently missing types of data, and the commonest reporting issues tabulated. ETHICS AND DISSEMINATION: The authors hope to disseminate the findings as widely as possible, irrespective of results, as these will add to the wider corpora of information on this subject. The systematic review will be published in a peer-reviewed journal and will be presented at a wide range of national and international conferences. Ultimately, this will inform a Delphi process for the development of a surgical case series reporting guideline. PROTOCOL REGISTRATION: CRD42015016145. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

A case series is an uncontrolled study that samples participants—with both a specific outcome and a specific intervention (exposure), or one that samples participants with a specific outcome of interest, regardless of their exposure status.1 These are commonly a retrospective review of a string of interesting cases with a unifying feature—which may be exposure, intervention, treatment or outcome. It is unclear whether the definition implies that the cases should be consecutive. These are frequent within the medical literature but are also present within social sciences and the humanities.1 As with case reports, their value has been debated.2 3 In the age of evidence-based medicine and the randomised controlled trial as the criterion standard to show the efficacy of a particular treatment, what is the role of a case series study?

In the summer of 1999, the use of a case series in the recognition of a new disease was exemplified by the epidemic of West Nile encephalitis in New York.4 Historically, case series were important in identifying the impact of maternal drinking and pregnancy outcome, and the role of vitamin C in preventing scurvy.5 6 More recently, a study by Albrecht et al7 of case series published in the Lancet found that a high proportion went on to have follow-up trials and that they were useful in establishing an early evidence base for treatments of rare diseases in which trials would not be feasible. For some specialties, establishing control groups may be difficult, for example, accident and emergency medicine. In the social sciences, many social psychology studies have been case series, for example, Yale Psychologist Milgram's seminal work on obedience to authority figures.8

In a 2005 report, Dalziel et al9 outlined that case series are used in 30% of health technology assessments—assessments used in the provision and suitability of care. However, poor reporting in the case series included in their study severely constrained their analysis and investigation of the hypothesis that findings in case series may be affected by methodological characteristics.9 Currently, no standardised reporting criteria developed within a robust methodological framework exist for case series. In the ongoing drive to improve the evidence base for clinical practice, a number of tools have been developed to improve the quality of reporting research. For example, publication of CONSORT (Consolidated Reporting Standards of Randomised controlled Trials) has seen the quality of articles in some fields improve significantly.10 11 The CONSORT statement has also been used to highlight and raise awareness of poor compliance in some fields.12–16 More recently, our group has shown the deficiency of observational studies in plastic surgery using the guideline STROBE (Strengthening the Reporting of Observational Studies in Epidemiology).17

A wide variety of reporting guidelines are now available across different research study types, except for case series. Problems in the reporting of surgical case series, in particular, have been highlighted to us from our recent experience in conducting a systematic review of autologous fat grafting for breast reconstruction.18 In this study, 25 of the 31 included studies were case series, yet 20% did not mention the age of the participants and 48% did not mention whether the participants had been treated with radiotherapy, an important prognostic factor. Surgery has the additional complexity of involving learning curves. The technique selected is not the sole factor affecting outcome. Patients need to be carefully selected appropriately worked-up the technique has to be meticulously worked out and implemented in an appropriate setting, with an appropriate postoperative regimen.

Readers need complete, transparent information and failure to provide this will short circuit critical appraisal, assessment of external validity and impact decisions on whether, for instance, a surgeon should change their practice. We aim to close this gap and help produce a reporting guideline for case series that is methodologically robust, easy to use and accepted internationally across a broad range of specialties and disciplines. According to Moher et al's19 guidance on guideline development, the early steps in this process require an analysis of previous literature to identify previous guidance (if any), and to analyse relevant evidence on the quality of reporting of published research articles within the domain of interest. We report our protocol for such a systematic review.

Objective

To conduct a systematic review exploring the reporting deficiencies within surgical case series that have been identified in the existing literature.

Methodology

This systematic review will be conducted according to the recommendations outlined in the Cochrane Handbook for reviews and reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.

Criteria for selecting studies

The following search criteria were specifically devised to locate studies specifically pertaining to the reporting quality of case series and to provide evidence for the objectives previously stated.

Types of studies/material

Research articles and systematic reviews which highlight reporting deficiencies in case series.

Types of participants

Human participants undergoing surgery.

Types of interventions

Any surgical intervention.

Types of comparator

Typically case series will have no comparator or control group. We did not specify anything here within our search criteria.

Outcomes

Specified reporting deficiencies identified within the articles relating to case series.

Search methods for identification of studies

Electronic searches

The following electronic databases will be searched from their inception to 5 November 2014: MEDLINE, EMBASE, Cochrane Methods Register, Science Citation Index and Conference Proceedings Citation Index.

Search terms and keywords

The search strategy has been developed through consultation with an information specialist based at the Bodleian Library, University of Oxford. Its aim is to locate papers related specifically to the reporting quality of case series. This search will utilise the English language keywords combined with Boolean logical operators. The search will be restricted to the English Language and tailored to the idiosyncrasies of each individual aforementioned database.

An example of a search strategy for the MEDLINE database is shown below (table 1).

Table 1

Search strategy for the MEDLINE database

#	Searches	Results
1	case series.mp.	39 714
2	"series of cases.”mp.	2580
3	"series of case reports.” mp.	547
4	1 or 2 or 3	42 346
5	Research Design/st [Standards]	9548
6	Research design/ and Quality Control/	1034
7	Research design/ and “Reproducibility of Results"/	6230
8	Research design/ and Data interpretation, Statistical/	4912
9	*Research design/	24 740
10	(quality adj5 (reporting or criteria or characteristic? or feature? or standard? or aspect?)).ti,ab.	29 385
11	(reporting adj5 (selection or recruit* or eligibility or study size or study design? or outcome?)).ti,ab.	4466
12	(reporting adj5 ((loss adj2 follow up) or dropout? or drop out? or attrition or retention)).ti,ab.	153
13	(reporting adj5 (missing data or missing value* or incomplete data or incomplete value*)).ti,ab.	64
14	(methodolog* adj5 (reporting or criteria or characteristic? or feature? or standard? or aspect? or quality)).ti,ab.	19 999
15	(reporting adj2 (scor* or system*)).ti,ab.	6476
16	strobe.ti,ab.	560
17	5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16	91 558
18	4 and 17	591
19	(case series adj5 (methodologic* or reporting)).ti,ab.	171
20	(case series adj5 (quality or bias or heterogen* or rigor* or rigour* or robust* or generalisab* or valid*)).ti,ab.	181
21	(case series adj5 (missing data or missing value* or incomplete data or incomplete value*)).ti,ab.	0
22	(case series adj5 ((loss adj2 follow up) or dropout? or drop out? or attrition or retention)).ti,ab.	6
23	(case series and (methodologic* or reporting)).ti.	14
24	(case series and (quality or bias or heterogen* or rigor* or rigour* or robust* or generalisab*)).ti.	38
25	(case series and (missing data or missing value* or incomplete data or incomplete value*)).ti.	0
26	(case series and ((loss adj2 follow up) or dropout? or drop out? or attrition or retention)).ti.	6
27	18 or 19 or 20 or 21 or 22 or 23 or 24 or 25 or 26	873
28	limit 27 to english language	842

Identification and selection of articles

Studies identified by the electronic search strategy will be listed. Results, including citation, title and abstracts, will be populated into Microsoft Excel Database (Microsoft, Redmond, Washington, USA) and duplicates removed. Titles and abstracts will be screened independently by two teams of authors (S-YL/KJLJ and BG/HS/KW) for issues relating to the reporting quality of case series. Any conflicts not resolvable between the two teams will be referred to the lead author (RA) for resolution. Articles selected after title and abstract screening will have full-text downloaded and a further assessment made of their eligibility. Once articles have been selected for inclusion, data extraction will take place.

Data extraction and management

Data will be extracted independently by two teams of authors (S-YL/KJLJ and BG/HS/KW) utilising standard extraction fields, where relevant data for each study will be collated. As this is a systematic review exploring the currently identified problems with case series reporting, the standardisation of extraction fields will be challenging. As such, reporting quality issues will be broadly grouped under the headings of: failure to use standardised definitions, missing or selective data, transparency or complete reporting and other. ‘Other’ will allow us to expand on any issues that are not part of this core set of reporting quality problems. We will also record whether the use of alternative study designs was considered. Any conflict of extraction will be resolved by discussion; where resolution of this is not possible, the lead author (RA) will have final say. This data will then be entered into a Microsoft Excel2011 database (Microsoft). Data collected will then be grouped into themes in which reporting deficiencies are occurring.

Data synthesis and statistical analysis

Outcomes will be tabulated, with descriptive statistics performed as appropriate to determine frequently missing types of data within reports of case series.

Sensitivity analysis

A sensitivity analysis will be performed whereby results from those studies whose primary aim was to assess the reporting quality of multiple case series will be looked at separately from those articles which may mention an issue in passing in their discussion.

Dissemination

The authors hope to disseminate the findings as widely as possible, irrespective of results, as these add to the wider corpora of information on this subject. The systematic review will be published in a peer-reviewed journal and will be presented at a wide range of national and international conferences.

Conclusion

This systematic review will inform us as to what types of data are missing and how reporting could be improved. It is an important first step along the path towards a guideline for reporting case series. The findings of our systematic review will be taken forward to the next step—generating further possible items through inviting experts to contribute thoughts. This will help prepare for a Delphi process involving a variety of experts who will inform the development of the reporting guideline.