Prognostic accuracy of different disseminated intravascular coagulation criteria in critically ill adult patients: a protocol for a systematic review and meta-analysis.

INTRODUCTION: Disseminated intravascular coagulation (DIC) is a common and serious condition that can lead to poor outcomes in critically ill patients. To make a correct diagnosis and improve the outcome of patients with DIC, several organisations have put forward DIC scoring systems. However, which criteria is the best to use for diagnosing DIC remains a continuing controversy even though many studies have been conducted to validate the diagnostic accuracy of each DIC scoring system. METHODS AND ANALYSIS: We will conduct a systematic review and meta-analysis of the diagnostic accuracy of DIC criteria for the prediction of mortality in critically ill adult patients. The primary objective is to assess the predictive values of the DIC criteria of Japanese Association for Acute Medicine, International Society on Thrombosis and Haemostasis, Japanese Ministry of Health and Welfare, Korean Society on Thrombosis and Hemostasis and Chinese Diagnostic Scoring System for 28-day mortality. We will search electronic bibliographic databases such as MEDLINE, EMBASE and Cochrane Central Register of Controlled Trials. Two reviewers will independently screen titles and abstracts, perform full article reviews and extract study data. We will report study characteristics and assess methodological quality using the Quality of Diagnostic Accuracy Studies-II tool. If pooling is appropriate, we will compute parameter estimates using bivariate random-effects and hierarchical summary receiver operating characteristic models to produce summary receiver operating curves, summary operating points (pooled sensitivity and specificity) and 95% confidence regions around the summary operating points. Clinical and methodological subgroup and sensitivity analyses will be performed to explore heterogeneity. ETHICS AND DISSEMINATION: This systematic review will help physicians diagnose DIC accurately and improve their clinical practice in critically ill settings. Approval from an ethics committee is not required. The findings will be disseminated through publication in a peer-reviewed journal. PROSPERO REGISTRATION NUMBER: CRD42017079350. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This systematic review will be the first report to answer the clinical question of which disseminated intravascular coagulation (DIC) criteria can most accurately predict mortality and be useful in overall populations of critically ill patients and patients with sepsis.

The supposed major limitation in this review will be that we only assess the prognostic values and not the diagnostic values of the DIC criteria, due to the lack of a gold standard for DIC diagnosis.

We will carry out a systematic review using appropriate methodologies and quality assessment tools according to the Grading of Recommendations Assessment, Development and Evaluation approach.

We believe that the results will be directly useful in helping clinicians to better select DIC criteria based on the best available evidence.

Introduction

Disseminated intravascular coagulation (DIC) is a disorder frequently seen in critically ill patients, especially with severe sepsis or multiple trauma1 that may lead to severe bleeding and organ dysfunction. Because mortality is higher in patients with DIC than in those without DIC,1 several organisations have put forward DIC scoring systems with the aim of improving the outcome of patients with DIC. The Japanese Ministry of Health and Welfare (JMHW) proposed a criteria for the diagnosis of DIC in 1976.2 The criteria involved the evaluation of global coagulation tests, underlying diseases and clinical symptoms. Thereafter, the subcommittee of the International Society on Thrombosis and Haemostasis (ISTH) proposed a scoring system for overt and non-overt DIC in 2001.3 However, it was easy for the diagnosis to be made too late based on the JMHW or the ISTH criteria in settings of critical illness and especially for sepsis patients. Thus, the Japanese Association for Acute Medicine (JAAM) proposed another DIC scoring system that was aimed at making an early diagnosis of DIC for patients on acute medicine.4 In addition, other criteria for DIC were proposed, those of the Korean Society on Thrombosis and Hemostasis (KSTH) and the Chinese DIC Scoring System (CDSS).5 6

The controversy over which criteria are the best to use for diagnosing DIC continues even though many studies have been conducted over the years to validate the diagnostic accuracy of each DIC scoring system.5 7–12 Japanese and Italian clinical practice guidelines for DIC management recommend the use of either the JMHW, ISTH or the JAAM criteria while the British guideline recommends the use of the ISTH criteria.13–15 Although the guidelines do not offer consistent recommendations about diagnosing DIC, no meta-analysis has been conducted so far to compare the prognostic performance among the several available DIC criteria. Thus, we will conduct a systematic review and meta-analysis of the prognostic accuracy of several DIC scoring systems in critically ill adult patients.

Methods and analysis

Protocol

This review adopts recommendations on the conduct and reporting of systematic reviews and meta-analyses outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement,16–18 the Meta-Analysis of Observational Studies in Epidemiology proposal19 and the Cochrane Diagnostic Test Accuracy Working Group.20

Focused review questions

Primary Objective To determine and compare the prognostic values of ISTH overt DIC criteria and non-overt DIC criteria, JMHW DIC criteria, JAAM DIC criteria, KSTH DIC criteria and CDSS DIC criteria for mortality in critically ill adult patients.

Secondary Objective To determine which criteria are superior for the prediction of mortality, especially focused on adult patients with sepsis diagnosed by Sepsis-1, Sepsis-2 or Sepsis-3.21–23

Types of studies

We will include all studies that assess the diagnostic accuracy of any DIC criteria noted above in adult critically ill patients. Diagnostic accuracy studies are typically of a delayed cross-sectional design. However, we will also include randomised controlled trials, cohort studies and case-control studies. Included studies should have sufficient information to build a 2×2 contingency table (true and false, positive and negative). Articles with experimental animals, narrative reviews, correspondence, case reports, expert opinions and editorials will be excluded.

Types of participants

We will include studies evaluating critically ill adult patients aged 18 years or older. Since ‘critical illness’ is somewhat poorly defined we will include critical illnesses whose definitions are generally accepted, such as acute respiratory distress syndrome, sepsis and systemic inflammatory response syndrome in this review. These will include participants from different clinical settings, such as emergency departments, hospital wards and intensive care units. We will exclude all studies investigating animals, those predominantly comprising neonates or postcardiac surgical, heart failure or perioperative patients and those comprising healthy participants as controls.

Studied tests

The index tests will be the ISTH overt DIC criteria3 and non-overt DIC criteria,3 JMHW DIC criteria,2 JAAM DIC criteria,4 KSTH DIC criteria5 and CDSS DIC criteria.6 A summary of DIC criteria is shown in table 1. In case DIC criteria are assessed more than once, the data evaluated at the earliest time point will be used as the index test. Cut-off values of fibrin-related markers, antithrombin, protein C and thrombin-antithrombin complex in the ISTH overt/non-overt DIC criteria will be defined according to the individual studies.

Table 1

Summary of DIC scoring systems

	Points	ISTH overt-DIC	ISTH non-overt DIC	JMHW DIC (without leukaemia)	JAAM DIC	KSTH DIC	CDSS DIC
Platelet counts	3	–	–	≤50×109/L	<80×109/L or >50% decrease/24 hours	–	–
	2	<50×109/L	–	>50, ≤80×109/L	–	–	<80×109/L
	1	≥50, <100×109/L	≤100×109/L Rising= −1, stable=0, falling=+1	>80, ≤120×109/L	≥80, <120×109/L or 30%–50% decrease/24 hours	<100×109/L	≥80, <100×109/L or ≥50% decrease/24 hours
FDP	3	Strong increase	–	≥40 µg/mL	≥25 µg/mL		≥9 µg/mL*
	2	Moderate increase	–	≥20, <40 µg/mL	–	–	≥5, <9 µg/mL*
	1	–	Increase Rising= −1, stable=0, falling=+1	≥10, <20 µg/mL	≥10, <25 µg/mL	Increase	–
Prothrombin time	2	≥6 s	–	≥1.67†	–	–	≥6 s
	1	≥3, <6 s	>3 s Rising=−1, stable=0, falling=+1	≥1.25, <1.67†	≥1.2†	>3 s or aPTT >5 s	≥3, <6 s or aPTT ≥10 s
Fibrinogen	2	–	–	≤1 g/L	–	–
	1	<1 g/L	–	>1, ≤1.5 g/L	–	<1.5 g/L	<1 g/L
Antithrombin	–	–	Normal= −1, low=+1	–	–	–	–
Protein C	–	–	Normal= −1, low=+1	–	–	–	–
TAT complexes	–	–	Normal=−1, high=+1	–	–	–	–
SIRS score	1	–	–	–	≥3	–	–
Underlying disease	–	–	Two points	One point	–	–	Two points
Bleeding symptom	–	–	–	One point	–	–	Meet anyone One point
Organ failure	–	–	–	One point	–	–	Meet anyone One point
Shock or microcirculatory disorder	–	–	–	–	–	–	Meet anyone One point
Required points for criteria-positive		Five points	Five points	Seven points	Four points	Three points	Six points

*D-dimer

†Prothrombin time (PT) ratio.

aPTT, activated partial thromboplastin time; CDSS, Chinese Diagnostic Scoring System; DIC, disseminated intravascular coagulation; FDP, fibrin/fibrinogen degradation products; ISTH, International Society on Thrombosis and Haemostasis; JAAM, Japanese Association for Acute Medicine; JMHW, Japanese Ministry of Health and Welfare; KSTH, Korean Society on Thrombosis and Hemostasis; SIRS, Systemic Inflammatory Response Syndrome; TAT, thrombin-antithrombin.

Reference standards

The reference standard will be 28-day mortality, which is one of the standard outcome measures in critical care settings. In case multiple mortality outcomes are reported or 28-day mortality is not available, the mortality outcome closest to 28-day will be handled as the reference standard.

Exclusion criteria

We will exclude studies from which we cannot obtain or calculate the true and false and positive and negative rates from the text, appendices or after contacting the main authors; abstracts that do not provide enough information for the assessment of the methodological quality; and duplicates or subcohorts of already published cohorts.

Search strategy

We will search the following databases for relevant studies: MEDLINE (via PubMed), EMBASE and the Cochrane Central Register of Controlled Trials. We have developed a search strategy using a combination of keywords and medical subject headings (MeSH)/EMTREE terms, which are ‘(‘disseminated intravascular coagulation’ OR DIC) AND (‘International Society on Thrombosis and Haemostasis’ OR ISTH OR ‘Japanese Ministry of Health and Welfare’ OR JMHW OR ‘Japanese Association for Acute Medicine’ OR JAAM OR ‘Korean Society on Thrombosis and Hemostasis’ OR KSTH OR ‘Chinese Diagnostic Scoring System’ OR CDSS OR overt OR non-overt)’. We will screen the reference lists of all relevant papers for additional studies. We will also contact relevant authors for further details and information on ongoing/unpublished trials. Our MEDLINE search strategy will be adapted appropriately for searches in the other two databases. No language restrictions will be applied. Only article beyond 1976 will be reviewed because none of the scoring systems became available before 1976.

Citation management and screening

Citations will be stored and duplicates will be removed using EndNote software (Thomson Reuters, Toronto, Ontario, Canada). Studies will be screened initially according to title and abstract by two authors (JY, MK) independently and those not meeting the criteria will be discarded. Disagreements will be resolved by discussion and referral to a third author (KY), if necessary. After this initial stage, the full text of all remaining studies will be reviewed by two authors (JY, MK) independently for inclusion or exclusion in the final study. Disagreements will be resolved in the same way as the initial screening. We will use Covidence software (Covidence; Melbourne, Australia) in this screening process. We will document the study selection process in a PRISMA flow diagram.

Data extraction

Two authors (JY, AK) will extract the study characteristics from each included study, including data on the assessment of quality and the investigation of heterogeneity and transfer that information into a study-specific format. Adjudication by a third author (KY) will be used if necessary. We will cross-tabulate the numerical information from the index test results (positive or negative) in 2×2 tables against the target disorder (positive or negative) and we will present the results as tables. Efforts will be made to contact the authors of primary studies to provide missing data where necessary.

Assessment of risk of bias

The quality of the included studies will be independently assessed by two authors (JY, AK) and verified by a third (KY), if necessary. The study quality of each article will be reported according to the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool.24 The presence of spectrum, threshold, disease progression and verification bias (partial or differential) will be specifically assessed. For each domain, we will assign a judgement regarding the risk of bias as high, low or unclear. We will attempt to contact the trial corresponding author for clarification when insufficient detail is reported to assess the risk of bias.

Data synthesis

We will present the results graphically by plotting estimates of sensitivities and specificities (both with 95% CI) in a Forest plot and in a receiver operating characteristic (ROC) space to visually assess the between-study variability. We will use Review Manager software, RevMan 5.3 (Nordic Cochrane Centre, Cochrane Collaboration) to document the descriptive analyses.

We will pool studies only if they share a common threshold, are conducted in the same or similar setting and show sufficient clinical homogeneity. For the meta-analysis, we will fit a summary ROC curve using a bivariate random-effects model and derive summary accuracy indices (sensitivity and specificity) and corresponding likelihood ratios using the MIDAS module for STATA software, V.14.0. Also, we will estimate positive predictive value and negative predictive value which are more useful clinically. We will plot the 95% confidence ellipse and prediction region around averaged accuracy estimates in the ROC space. We will generate a nomogram, which is a user-friendly graphical depiction of positive predictive value and negative predictive value by prevalence.

Assessment of heterogeneity

We will investigate heterogeneity initially by visual examination of Forest plots of sensitivities and specificities and through visual examination of individual study results in the ROC space. Statistical heterogeneity will be evaluated informally from Forest plots of the study estimates and more formally using the χ2 test (p value <0.1, significant heterogeneity) and I2 statistic (I2 >50%= significant heterogeneity).

Assessment of publication biases

If a sufficient number of studies are identified, we will investigate publication bias by Deeks’ funnel plot. We will interpret this analysis with caution given the lack of statistical power of this test and the absence of consensus about adequate methods to detect publication bias in reviews of diagnostic test accuracy.

Sensitivity and subgroup analysis

We will examine the robustness of the meta-analyses by conducting sensitivity analyses and excluding studies according to different components of the QUADAS-2 tool for assessing the risk of bias. Our primary analysis will include all studies, and sensitivity analysis will exclude those studies with high risk of bias or with important concerns about potential applicability.

If sufficient studies are available, we will undertake subgroup analyses to explore the sources of potential heterogeneity in sensitivity and specificity. Meta-regression analysis and subgroup analysis will be performed using the following as covariates: sepsis or not, trauma or not, year of publication, country, prevalence of DIC (<50% or ≥50%), baseline mortality risk, sample size (<100 or ≥100), types of fibrin-related markers, and cut-off values of fibrin-related markers.

Interpretation and summary of findings

A major focus of reviews of diagnostic test accuracy is to obtain summary estimates of test accuracy. However, knowing that a test has a high sensitivity, for instance, does not tell us whether the test will have much effect on the patient, nor does it tell us that using this test in practice will be beneficial for the patient or cost-effective. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for diagnostic tests has now been developed, which provides guidance on how to translate accuracy data into a recommendation involving patient-important outcomes.25 We will apply the GRADE approach to rate the quality of the evidence.

Patient and public involvement

Patients and the public were not involved in the design of this protocol.

Discussion

DIC is a common and serious condition that can lead to poor outcomes in critically ill patients.1 Although several guidelines for DIC management have been published to improve its clinical practice,13–15 the evidence for diagnosing DIC accurately is limited. Therefore, physicians still have difficulty diagnosing and managing DIC in clinical settings and there is a need for properly designed, well-conducted reports to determine which criteria should be used to assess DIC.

There will be potential major limitations in this review. First, we will only assess the prognostic values and not the diagnostic values of the DIC criteria. Initially, the efficacy of the diagnostic test accuracy should be evaluated on sensitivity and specificity for the presence of the disease. Because of the lack of a gold standard for DIC diagnosis, we cannot properly assess the diagnostic rates of the DIC criteria. Alternatively, we will compare the prognostic values of each of the DIC criteria in this review. Second, several interventions, such as transfusion and anticoagulant therapy, can affect coagulation parameters and subsequently, the DIC score. As there may be no predefined protocol for diagnostic time points, we will use the data for assessment of DIC criteria that was evaluated at the earliest time point as the index test to minimise the influence of clinical exposure in this review. Third, the variety of study populations can influence the heterogeneity of the predictive value of the DIC criteria. Thus, we will perform subgroup analyses focused on sepsis and trauma, which are the major causes of DIC in critically ill patients.

This meta-analysis will be the first report to reveal the characteristics of several DIC criteria for the prediction of mortality and may be useful in assessing overall populations of critically ill patients and sepsis patients. We will carry out a systematic review of the diagnostic test accuracy of several DIC criteria for predicting mortality using appropriate methodologies and quality assessment tools according to the GRADE approach. Thus, we believe that the results will be directly useful in helping clinicians to better select DIC criteria based on the best available evidence.

Ethics and dissemination

Approval from an ethics committee is not required since this systematic review will use publicly available data without directly involving human participants. Our findings will be presented at relevant scientific conferences and disseminated through publication in a peer-reviewed journal.