Evidence map of pancreatic surgery: protocol for a living systematic review and meta-analysis.

INTRODUCTION: Pancreatic surgery is a large and complex field of research. Several evidence gaps exist for specific diseases or surgical procedures. An overview on existing knowledge is needed to plan and prioritise future research. The aim of this project is to create a systematic and living evidence map of pancreatic surgery. METHODS AND ANALYSIS: A systematic literature search in MEDLINE (via PubMed), Web of Science and Cochrane Central Register of Controlled Trials will be performed searching for all randomised controlled trials (RCT) and systematic reviews (SR) on pancreatic surgery. RCT and SR will be grouped in research topics. Baseline and outcome data from RCT will be extracted, presented and effect sizes meta-analysed. Data from SR will be used to identify evidence gaps. A freely accessible web-based evidence map in the format of a mind map will be created. The evidence map and meta-analyses will be updated periodically. DISSEMINATION: After completion of the project, a permanently updated evidence map of pancreatic surgery will be available to patients, physicians, researchers and funding bodies via www.evidencemap.surgery. Its use will allow clinical decision-making based on primary data and prioritisation of future research endeavours. PROSPERO REGISTRATION NUMBER: CRD42019133444. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Through a comprehensive search and selection of high-quality articles, the best available evidence for pancreatic surgery will be gathered.

Contrary to medical databases, the evidence map in the form of a mind map will present randomised controlled trials and systematic reviews ordered by research topics in an intuitive fashion.

The evidence map of pancreatic surgery will strengthen the visibility of primary research results in pancreatic surgery.

Background

Quantity and quality of randomised controlled trials (RCT) for pancreatic surgery is increasing, however, there are still blind spots regarding specific operations and diseases.1 Socioeconomic pressure demands for prioritisation of relevant research projects in the field of pancreatic surgery. Since pancreatic diseases are devastating for patients and highly impair their quality of life,2 3 there is an urgent need for the best treatment, which should be based on the best available evidence. Consequently, patients undergoing pancreatic surgery should be included in prospective trials whenever evidence is lacking. Therefore, pancreatic surgery research should be performed according to an objective priority setting.

The two main surgically treated diseases of the pancreas are tumours and chronic pancreatitis.1 For both entities, surgery remains the only chance of cure or long-term increase of quality of life, respectively.2 3 Therefore, all patients bear the burden of a severe disease in need of major surgery, and must carry the risk of postoperative morbidity which is as high as 73%.4 Therefore, one of the major research interests is to find the most effective and safe way to operate patients. Since perioperative mortality in specialised centres is low nowadays,4 the focus lies on reduction of pancreas-specific complications like postoperative pancreatic fistula (POPF),5 delayed gastric emptying6 or post-pancreatectomy haemorrhage.7

To systematically investigate the field of pancreatic surgery, two innovative methods of evidence-based medicine are combined: the living systematic review (SR) and evidence mapping.

Living SR follow the established methods of an SR. However, they overcome the difficulty that normal SR are soon outdated or redundant after their publication.8 Living SR are assumed to achieve a greater validity with increased benefits for physicians and patients at lower spending of resources over time.9 Some experts even think that living SR should become the flagship of synoptic evidence and the research community should have a strong interest to establish living SR in their fields.10

Evidence mapping is also an emerging approach to systematic assessment of quantitative and qualitative aspects.11 Although there is no universally applied definition of evidence mapping yet, its aim is usually to summarise evidence and identify gaps in the body of knowledge regarding a specific area of research. In times of scarcity of health system resources and overload of information, this approach may enable researchers and funding bodies to prioritise future research questions.12

The combination of the methods of living SR and evidence mapping applied on pancreatic surgery will result in an intuitive and permanently up-to-date map of available evidence including living meta-analyses (MA). Through visualisation of available evidence, healthcare professionals, patients and funding bodies gain direct access to highly relevant data.

Aim

The major problem of evidence management is that most research activities are not harmonised with clinical and political relevance. This results in production of waste-evidence, rather than needed evidence by prioritisation. The first step in priority setting would be an up-to-date characterisation of existing knowledge, lack of knowledge and research questions. Thus, the aim of this project is to create a systematic and living evidence map of pancreatic surgery.

Methods/design

The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guideline was followed.13 Furthermore, the living systematic review network guidelines on how living SR should be published,10 how living MA should be updated14 and how living recommendations should be formed15 will be followed wherever applicable. The project was prospectively registered and for full transparency the protocol is herewith published open access.

Systematic literature search

A systematic literature search in all major electronic bibliographic databases with relevance for surgical literature will be searched16: MEDLINE (via PubMed), Web of Science and Cochrane Central Register of Controlled Trials (CENTRAL). No restrictions will be applied regarding language or publication date. The full search strategy for MEDLINE (via PubMed) will be:

‘((pancreas[MeSH terms] OR pancreas[tiab] OR pancreatic[tiab] OR pancreato*[tiab]) AND (resection* [tiab] OR removal [tiab] OR surger* [tiab] OR surgical [tiab] OR laparotom*[tiab] OR enucleation* [tiab] OR operation* [tiab] OR operated [tiab] OR ‘surgical procedures, operative’[MeSH terms] OR ‘general surgery’[MeSH terms])) OR (pancreaticoduodenectom*[tiab] OR pancreatoduodenectom*[tiab] OR pan-creatoduodenectom*[tiab] OR duodenopancreatectom*[tiab] OR pancreatectom*[tiab] OR Whipple[tiab] OR Kausch-Whipple[tiab] OR ppWhipple[tiab] OR dpphr[tiab] OR PPPD[tiab] OR pancreaticoduodenectomy[MeSH] OR pancreatectomy[MeSH] OR ‘Pancreas/surgery’[Mesh] OR ‘Pancreatic Diseases/surgery’[Mesh] AND (randomized controlled trial [pt] OR random*[tw] OR RCT [tw] OR ‘Randomized Controlled Trials as Topic’[Mesh] OR ‘Controlled Clinical Trial’ [pt] OR systematic review [pt] OR meta-analysis [pt] OR re-view [pt] OR meta-analysis [tw] OR review [tw])’. The full search strategy for Web of Science and CENTRAL is displayed in online supplementary appendix 1.

By a preliminary literature search in MEDLINE (via PubMed), Web of Sciences and CENTRAL, >30 000 potentially eligible articles were identified. It is expected that the first version of the evidence map will contain >250 RCT and 400 SR/MA.

Study selection

The PICO question is shown in box 1. Following the recommendations of the Cochrane Collaboration,17 titles, abstracts and full texts of identified articles will be screened independently by two reviewers. If there is a disagreement between the two reviewers, this will be resolved by a third reviewer. The screening process will be done with the bibliographic software EndNote X9 (Clarivate Analytics).

Inclusion: patients with any kind of pancreatic disease that requires surgery.

Exclusion: patients with pancreatic diseases that does not require surgery.

Inclusion: all kinds of interventions will be included as long as they are aimed to affect the surgical outcome, that is, medical devices, perioperative management, surgical strategy, drug and nutrition.

Exclusion: endoscopic retrograde cholangiopancreatography, radiologically guided punctures or similar interventions.

Systemic cancer therapies and pancreatic transplantation.

Any other kind of control compared with the above-mentioned intervention including endoscopic retrograde cholangiopancreatography, radiologically guided punctures or similar interventions.

Mortality, postoperative pancreatic fistula, delayed gastric emptying, postpancreatectomy haemorrhage, bile leak, chyle leak, intra-abdominal fluid collection/abscess, overall morbidity, overall survival, length of hospital stay and operation time.

Eligible study designs to be included will be RCT and SR with or without MA. SR will only be eligible if they meet minimal quality requirements, that is, SR must search at least two established literature databases and provide a critical appraisal with validated tools like the Cochrane Collaboration tool for assessing risk of bias18 for RCT or like the risk of bias in non-randomized studies of interventions (ROBINS-I) for non-randomised studies.19

The focus of this evidence map is pancreatic surgery. Interventions to be included should aim to affect the surgical outcome, that is, medical devices (eg, stapler vs scalpel resection in distal pancreatectomy), perioperative management (eg, prehabilitation of patients, or intraoperative fluid management), surgical strategy (eg, open vs laparoscopic access to the abdominal cavity), drug (eg, somatostatin analogues to influence POPF) and nutrition (eg, immunonutrition to avoid complications). Interventions like endoscopic retrograde cholangiopancreatography, radiologically guided punctures or similar interventions will only be assessed as control groups to the above-mentioned interventions. Moreover, studies on neo-adjuvant/adjuvant systemic treatment, or pancreatic transplantation will be excluded.

Data extraction

All stages of data extraction and quality assessment will be carried out independently by two reviewers using predefined items. Any disagreement will be resolved by consensus, or by consultation with a third reviewer. The items are directly extracted to a user interface (Microsoft.NET framework, Windows Forms) with automated plausibility checks. The data will be saved in a database (Microsoft SQL Server 2017 Express) tailored for this project. The database will allow saving resources during data extraction and making data usable for presentation on the evidence map and for statistical analysis. After validation of the extracted data, the relational database will be able to export the extracted data in the exact form needed for presentation on the evidence map. Furthermore, the database will have an interface to the statistical programme to export data needed for the MA. All extracted items for RCT and SR within the user interface are shown in online supplementary appendix 2.

Furthermore, the methodological quality of included RCT will be assessed using the newly suggested Cochrane Collaboration tool for assessing risk of bias 2.0.20 The tool includes five standard domains of bias: bias arising from the randomisation process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in measurement of the outcome and bias in selecting of the reported result. These domains will be rated as ‘high risk’, ‘low risk’ of bias, or some concerns. Finally, an overall risk of bias judgement will be made. As recently recommended for surgical trials, detailed information on blinding will be recorded and reported.21 Furthermore, industrial funding will be considered as another potential threat to validity.22

Data synthesis for creation of the evidence map

All included RCT and SR will be clustered according to the type of operation, the type of disease and the type of interventions. Consequently, studies on the same research topics will be grouped, for example, pylorus-resecting versus pylorus-preserving (intervention: surgical strategy) in partial pancreatoduodenectomy (operation) for tumours or chronic pancreatitis (disease).

Information on existing SR will be shown within the evidence map and will be used for identification of evidence gaps in the research topics, that is, missing RCT. Therefore, no quantitative data will be extracted and no critical appraisal of SR will be performed. Including SR in the evidence map is preferred to the inclusion of all other primary study types like non-randomised prospective trials or retrospective studies.

Information on existing RCT will also be shown within the evidence map and the extracted data will be used for pooling in MA. For each research topic, the following set of outcomes will be reported in the MA: mortality, postoperative pancreatic fistula (graded as biochemical leak, B, C if the International Study Group of Pancreatic Surgery (ISGPS) definition5 is used), delayed gastric emptying (graded as A, B, C if the ISGPS definition6 is used), postpancreatectomy haemorrhage (graded as A, B, C if the ISGPS definition7 is used), bile leak (graded as A, B, C if the International Study Group of Liver Surgery definition23 is used), chyle leak (graded as A, B, C if the ISGPS definition24 is used), intra-abdominal fluid collection/abscess, overall morbidity (if available according to the Clavien-Dindo classification25), overall survival (as 1–5 years survival rate and median overall survival), length of hospital stay and operation time. Protocols of ongoing RCT will be displayed within the evidence map until the final results are available.

Furthermore, for each outcome the certainty of the evidence will be rated using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) system.26 27 This includes limitations in the design from the risk of bias assessment (see above), indirectness of evidence, unexplained heterogeneity or inconsistency of results, imprecision of results and publication bias. Thus, the certainty of the evidence will be rated to be very low, low, moderate or high for each outcome.

Statistical analysis

If more than three RCT investigate the same research topic, for example, pylorus-resection versus pylorus-preservation in pancreaticoduodenectomy, the above-mentioned outcomes of these RCT will be pooled in living MA.

Statistical analyses will be performed with R.28 Dichotomous data (mortality, postoperative pancreatic fistula, delayed gastric emptying, postpancreatectomy haemorrhage, bile leak, chyle leak, intra-abdominal fluid collection/abscess, overall morbidity, survival rate) will be pooled in a Mantel-Haenszel model to estimate ORs and associated 95% CIs. For complications defined by the ISGPS, the MA will discriminate grade A complications from clinically relevant B/C complications. For continuous data (mean overall survival, length of hospital stay, operation time), mean differences and associated 95% CIs will be calculated using an inverse-variance model. A two-sided level of significance below 5% will be considered statistically significant. Continuous values reported as median with range will be converted to mean and SD.29 For dichotomous and continuous data, a prediction interval will be calculated. Statistical heterogeneity among trials will be evaluated by means of the I2 statistic. We will consider I2<25% to indicate low statistical heterogeneity and I2>75% to indicate high statistical heterogeneity. A random-effects rather than a fixed-effects model will be used for MA when clinical heterogeneity is assumed and at least five RCT are available.

If more than two interventions are compared within a research topic, a state-of-the art Bayesian network MA will be performed. Either linear or logistic random-effects models will be applied. Pooled effect estimates obtained in the network MA (adjusted mean differences or log ORs) will be provided with 95% CIs. Furthermore, a treatment ranking based on the probability of being the most efficient arm will be carried out.

To evaluate the risk of publication bias, funnel plots will be created and tested for asymmetry using the test by Harbord et al,30 if >10 trials are available for a living MA.

Creating the evidence map

The evidence map of pancreatic surgery will be freely accessible for everyone via the internet. An example how the structure of the evidence map, its instructions and information on a research topic (eg, pylorus resection vs pylorus preservation in pancreaticoduodenectomy) might look is accessible on www.evidencemap.surgery

The quantitative and qualitative analyses are only one part of the added value by the evidence map. The evidence map will be configured as a mind map leading its reader from the centre (pancreatic surgery) to a research topic, for example, pylorus resection versus pylorus preservation in pancreaticoduodenectomy (figure 1). In the centre of the map the icon behind the map version, Pancreatic surgery V0 in the example, will provide a summary of the evidence map including a PRISMA flow chart of the actual version. Furthermore, for every type of operation a pooled estimate of mean with 99% CI and median with IQRs from all RCT for the outcomes will be calculated and presented for bench marking purposes. Furthermore, two bubble plots will be created, mapping all RCT by types of operation to types of intervention and types of disease to types of intervention. Within the bubble plots sample size of the trials will be expressed by bubble size and the geographical region by a colour code. This will allow concluding on overall evidence gaps in pancreatic surgery and differences between geographical regions.

Figure 1

Example of the possible structure of the evidence map (www.evidencemap.surgery). GRADE, Grading of Recommendations, Assessment, Development and Evaluations; MA, meta-analyses; RCT, randomised controlled trials; SR, systematic reviews.

Through logical connections the reader will be guided to research topics. These are marked with symbols indicating the presence or absence of RCT and SR (a tick means that RCT are existing, a cross means that RCT on the research topic are missing; a star means that SR are existing and an exclamation mark means that SR are missing). In this example, the symbols mean that there is at least one RCT and SR/MA available for the research topic. In this fashion, the evidence map gives an intuitive presentation of available evidence and evidence gaps become visible.

For every research topic the reader can look at the existing RCT and SR (figure 2).

Figure 2

Example of existing literature (RCT and SR) and living meta-analysis for a research topic (www.evidencemap.surgery). GRADE, Grading of Recommendations, Assessment, Development and Evaluations; MA, meta-analyses; PPPR, pylorus preservation vs plyorus resection; RCT, randomised controlled trials; SR, systematic reviews.

For RCT and SR the name of the first author and the year of publication are displayed. Behind the year of publication three icons are shown. The first icon gives the original conclusion of the article and the full reference. The second icon is a link to the article on the journal homepage or if the manuscript is published open access the full text is directly downloadable. The third icon is available for RCT only and contains the extracted data as an exportable and processable file (.xlsx).

Finally, from the ‘Living MA and GRADE’ field a summary of findings table (GRADE), the forest plots and the funnel plots for all outcomes of a research topic will be downloadable from the evidence map.

Additionally, the evidence map will have a comment function and will allow physicians, researchers and patients to interact with the evidence map by adding comments. In this way, researchers can report their new research directly or patients can comment on the importance of future research within research topics. There will be an administrator answering comments and additionally reacting on important subjects via social media.

Living systematic review and meta-analyses

After its induction, a periodically update including the steps of literature search, screening and extraction is planned at least every 6 months. If new RCT and SR are available on these searches, they will be added to the research topics and the MA will be renewed resulting in living MA. Version numbers and date of last updates will be displayed on the map itself and on every research topic.

Patient involvement

In order to adequately incorporate patients, a priority setting partnership (PSP) for pancreatic cancer treatments in Germany (www.europaeisches-pankreaszentrum.de/extrainfo/psp-pankreaskarzinom/) is performed. The objective of this project is to involve patients, their families, caregivers, specialists, nurses and other stakeholders to identify and prioritise unanswered scientific questions in the treatment of pancreatic cancer. From these responses unidentified research topics may emerge. In a second step, patients as well as experts will be asked to rank the existing research topics for priority. Results of the PSP in conjunction with the living evidence map would allow a transparent, objective and patient-centred identification of the most urgent future research topics in pancreatic cancer.

Moreover, national and international patient representative organisations will be involved during the beta-test phase of the evidence map to invite them for their comments, especially on importance of the research topics presented. Furthermore, these organisations will be invited to link the map on their internet presences.

Dissemination

The aim of this project is to develop and maintain an evidence map of pancreatic surgery, that is, a living SR with MA and mapping of the evidence. The map will contain all existing evidence from RCTs and SRs on pancreatic surgery plotted as an intuitive and interactive mind map. The presented evidence is based on a comprehensive systematic literature search and comprehensibly selection of literature. Through www.evidencemap.surgery, a permanently updated evidence map of pancreatic surgery will be disseminated to patients, physicians, researchers and funding bodies.

The living evidence map of pancreatic surgery will serve different purposes for researches, clinicians, patients and funding bodies. For researchers, the evidence map in pancreatic surgery will be a help to get a quick overview about existing research questions. Notably, this is not an attempt to substitute single SR on a specific subject. Much more, the intention is to provide a strong reference as a comparator. Moreover, it will speed up and harmonise the conduct of future SR as researchers can rely on the performed literature search, on the extracted data and critical appraisal. This map would be highly relevant to patient care and the healthcare system because it would show ‘what works’ and ‘what is missing’ at a glance and in an intuitive fashion. Clinicians could use the map to inform their patients on benefits and harms of different pancreatic surgery interventions based on up-to-date high-quality data. The difference to follow a guideline is that clinicians can interpret the primary literature from RCT and SR for their individual patients instead of applying recommendations from guidelines. In the same manner, patients will have access to primary data sorted by logical connections, which will allow them to find evidence appropriate for their cases. Moreover, researchers, clinicians and patients will be able to comment on research topics and interact with the pancreatic surgery community. Finally, such an evidence map should be of interest for funding bodies because an objective assessment of which research project is most pressing to be funded becomes possible.

The first version will be presented at the World Pancreas Forum (Bern, Switzerland; www.worldpancreasforum.com) on 6 February 2020. The first citable version, that is, a version of which cornerstone data will be published in a peer-reviewed journal is planned for the end of 2020. After this the online version will be updated every 6 months and a new citable version is planned after 2 and 4 years. Thereafter, the impact on literature and research of the evidence map will be re-evaluated. As social media become more and more important in the dissemination of scientific results, the evidence map will be promoted on Facebook and Twitter.31 Therefore, updates and living MA will be blogged and tweets/re-tweets will be done to surgeons and surgical journals. Finally, to our knowledge the proposed evidence map would be the first of its kind. Therefore, this project would also inspire other researchers to follow and create such maps in their medical fields.