Guidelines for reporting case studies on extracorporeal treatments in poisonings: methodology.

A literature review performed by the EXtracorporeal TReatments In Poisoning (EXTRIP) workgroup highlighted deficiencies in the existing literature, especially the reporting of case studies. Although general reporting guidelines exist for case studies, there are none in the specific field of extracorporeal treatments in toxicology. Our goal was to construct and propose a checklist that systematically outlines the minimum essential items to be reported in a case study of poisoned patients undergoing extracorporeal treatments. Through a modified two-round Delphi technique, panelists (mostly chosen from the EXTRIP workgroup) were asked to vote on the pertinence of a set of items to identify those considered minimally essential for reporting complete and accurate case reports. Furthermore, independent raters validated the clarity of each selected items between each round of voting. All case reports containing data on extracorporeal treatments in poisoning published in Medline in 2011 were reviewed during the external validation rounds. Twenty-one panelists (20 from the EXTRIP workgroup and an invited expert on pharmacology reporting guidelines) participated in the modified Delphi technique. This group included journal editors and experts in nephrology, clinical toxicology, critical care medicine, emergency medicine, and clinical pharmacology. Three independent raters participated in the validation rounds. Panelists voted on a total of 144 items in the first round and 137 items in the second round, with response rates of 96.3% and 98.3%, respectively. Twenty case reports were evaluated at each validation round and the independent raters' response rate was 99.6% and 98.8% per validation round. The final checklist consists of 114 items considered essential for case study reporting. This methodology of alternate voting and external validation rounds was useful in developing the first reporting guideline for case studies in the field of extracorporeal treatments in poisoning. We believe that this guideline will improve the completeness and transparency of published case reports and that the systematic aggregation of information from case reports may provide early signals of effectiveness and/or harm, thereby improving healthcare decision-making.

Case reports are an integral segment of the medical literature. The detailed descriptions of a clinical problem, the reasoning process, and the deducted therapeutic interventions, as well as their evaluation, have been central to medical education. Even in the era of evidence-based medicine, case studies can alert clinicians to the possible adverse effects of therapies, or highlight new diagnostic or therapeutic approaches. Despite their limitations in establishing causality and efficacy, case reports are essential for teaching purposes and represent a scientific description of personalized medicine. If compelling, case reports are often followed by more robust scientific investigations, best epitomized by prospective controlled trials. However, the latter are not universally achieved in certain specialties, like clinical toxicology and emergency medicine 1. Due to the highly varied nature of acutely poisoned patients, case reports remain the most common type of publication in the toxicology literature 1 and this is particularly true concerning cases of extracorporeal treatments (ECTRs) for poisoning.

For the purposes of the following discussion an ECTR comprises any procedure occurring outside the body that can enhance the elimination of a poison and includes hemodialysis, hemofiltration, hemoperfusion, as well as others. For these types of publications to have any external validity, it is essential that the reporting of cases be comprehensive. There is, therefore, major interest in improving the quality of case reports.

The EXtracorporeal TReatments In Poisoning (EXTRIP) workgroup is a group of diverse stakeholders developing consensus guidelines on the role of ECTR in the treatment of poisoned patients. The methodology includes a detailed literature review together with expert opinion 2. Unfortunately, during the clinical guidelines development, EXTRIP members observed much variability in the quality of case studies: the data provided were inconsistently presented, important elements were frequently omitted, and/or calculations were flawed, thus diminishing the utility of most published cases.

To enhance the quality and reliability of medical research, various groups have developed and proposed guidelines on how to report transparent, accurate, and complete studies. Such reporting guidelines already exist for randomized controlled trials 3, observational studies 4, systematic reviews and meta-analyses 5, diagnoses 6, and health economic evaluations 7. Reporting guidelines for case reports for adverse events were first published in 2007 8,9.

Consensus-based guidelines for the reporting of case studies were developed in 2013; the CAse REporting (CARE) checklist is structured to contain key components of a case report and capture useful clinical information 10. The proposed checklist provides a generic framework of essential items to satisfy the need for completeness and transparency for published case reports while providing a balance between adequate detail and concise writing. However, the CARE guidelines were not designed to describe toxicology case reports and so cannot be directly extrapolated to assess the quality of a case report of a poisoned patient undergoing extracorporeal treatment for removal of toxins (ECTRTOX). This stems from the specific particularities of ECTRTOX reports relating to complicated interventions, highly variable case presentations, quantitative toxicokinetic (TK) data, and the need for calculations. Ideally, ECTRTOX reports should provide reliable estimate of the amount of poison removed so that this can be related to clinical outcomes. This may permit a better understanding of the mechanisms behind a clinical outcome and reproduce (or avoid) a clinical intervention. Similar to the CARE guidelines, an ECTRTOX report should allow the reader to understand the details of the poisoning exposure, clinical presentation and evolution, treatments, and measured outcomes 11. Important limitations to the case report should also be clearly presented 12,13.

Our primary objective was to develop reporting guidelines for case studies where extracorporeal treatments are performed for poison removal, by using the CARE checklist as a template, adding other essential components that are specific to ECTR, and then selecting a set of items considered minimally essential for reporting through a consensus-based process. Secondary objectives were to propose tools for prospective data collection, TK calculations, and reporting so that subsequent case reports of ECTR in poisoned patients will be accurate, complete, reliable, and transparent.

Methods

Research Design

Our methodology was inspired from the Guidance for Developers of Health Research Reporting Guidelines 14, but was modified to account for the fact that the literature review and its appraisal were already performed prior to the start of this reporting guideline development.

This consensus-based process consisted of three phases:

Preliminary evaluation by the EXTRIP workgroup: extensive review of literature, face-to-face meeting and discussion on the quality of the literature, and item generation for a case reporting checklist 2

Selection of minimally essential items: alternate rounds of voting (group of selected panelists through a modified Delphi technique) and of external validation (three independent raters)

Writing of reporting guideline and Explanation & Elaboration document: rationale of each decision was summarized from the aggregate comments from the voting steps as well as from the email communications between panelists.

PHASE 1: Preliminary evaluation by the EXTRIP workgroup

For the purpose of drafting clinical recommendations, the EXTRIP initiative reviewed the total body of the international literature from multiple databases for 16 predetermined poisons (medicines, chemicals, toxins). There were no limitations on language (articles were translated as required) and the publications covered every article published since 1913 2. This literature review yielded over 7500 articles, most of which were reviews, commentaries, or editorials. There were 2908 articles that presented original data, 90% of which were case reports, while observational studies and randomized controlled trials were rare and the remainder were in vitro or animal experiments. For each selected poison, all case reports were thoroughly analyzed. First, data were reviewed within the small subgroups responsible for each specific poison. In this analysis, all pertinent data were extracted and the quality of the studies accessed via multiple discussions by email or telephone. Thereafter, a face-to-face 5-day meeting was held in 2012, regrouping 28 of the 29 EXTRIP participants, which permitted extensive discussion concerning the deficiencies, errors, limitations, and biases of the included case reports. The general consensus was that the bulk of these case reports were of very poor quality because crucial information was often missing and/or calculations were erroneous 2. Based on this work, the EXTRIP group generated a preliminary draft of items that should be reported in an ECTRTOX case report.

PHASE 2: Selection of Minimally Essential Items

Selection of Participants

All 29 members of the EXTRIP workgroup were invited to participate as panelists to the reporting guidelines. These panelists were initially identified, in 2010, as potential stakeholders for the EXTRIP workgroup because of their content expertise from diverse backgrounds: nephrology, emergency medicine, clinical toxicology, critical care, pediatrics, and clinical pharmacokinetics. Furthermore, each had experience publishing in the biomedical literature, and/or reviewing and appraising these types of case reports. In addition, a specialist in pharmacology, having already participated in the development of reporting guidelines for pharmacokinetic studies, was specially invited to participate in our reporting guideline process. Panelists were expected to complete the item generation, to vote on the pertinence of the item of the checklist, and to comment on each proposed item to explain the rationale underlying their selection.

Then, three experts, well acquainted with the EXTRIP goals and objectives (participants in ad hoc groups for EXTRIP) and experienced in publication and peer reviewing, were invited as raters for the external validation rounds. Raters were expected to complete the item generation, to validate the clarity of each item using a set of predetermined case reports.

Literature Search

A Medline search for all case reports published in English between January 1st to December 31th of 2011 was performed to complete the item generation and to proceed to the two rounds of external validation. A case report was defined as an original description of one or more patients. Only manuscripts that describe individual patients who underwent ECTR for poison removal were included. Manuscripts that presented aggregate or grouped data were excluded. We also excluded conference abstracts and letters to the editor, as they were not deemed complete enough for evaluation (although these were all considered for evaluation in the clinical guidelines development).

The search strategy was: (Toxicity OR poison* OR intoxication OR overdos*) AND (Hemoperfusion OR haemoperfusion OR hemofiltration OR haemofiltration OR hemodialysis OR haemodialysis OR hemodiafiltration OR haemodiafiltration OR dialysis OR plasmapheresis OR plasmaphaeresis OR plasma exchange OR exchange transfusion OR CRRT OR renal replacement therapy OR extracorporeal therapy) (Fig.1).

Figure 1

Flow diagram for the 2011 literature search. ECTR, extracorporeal treatment

All selected case reports were organized in alphabetical order of first author name and divided into three equal parts: the first for completing the item generation, the second for the first round of validation, and the last for the second round of validation.

Item Generation

As mentioned above, the EXTRIP workgroup provided a preliminary checklist. This checklist was originally intended to include all the possible items that could be deemed valuable in an ECTRTOX report. All items included in the CARE guidelines were added to this preliminary checklist. Furthermore, the raters evaluated a set of preselected ECTRTOX reports published in 2011 to complete the list for any potentially missing items.

Selection of Minimally Essential Items

After the completion of the exhaustive preliminary checklist, which included all possible pertinent items that would need to be reported, a voting round, followed by a validation round, was then performed twice. The rationale for doing so was to allow an external verification at each step of the process to rapidly identify any problem concerning the phraseology or the clarity of each retained item 9 (Fig.2).

Figure 2

Item selection.

Voting procedure. For the voting procedure, a modified Delphi technique was used for selecting items from the proposed checklist. At each round, the checklist was emailed to all panelists with the preliminary draft of the rationale for each item. The panelists were asked to vote on the importance of each item on a Likert scale (1–3 = superfluous, 4–6 = desirable, 7–9 = essential). The voting procedure was performed anonymously. Medians and disagreement indexes were calculated for every item in each round. Panelists were asked to provide comments to justify keeping or excluding each item in the checklist. They had the opportunity to suggest additional items or to request that items be rephrased. At the end of each voting round, the voting results and the panelists' comments were sent back to all panelists. Discussion was encouraged and disagreement resolved online or by telephone. Nonresponders were sent two reminders.

Two iterative rounds were planned, and each round served a different purpose that was relayed to the panelists. The first voting round was intended to be as inclusive as possible so that all criteria that scored a median vote ≥4 were kept and admissible for the second round. The second round of voting was intended to be more selective, so panelists were asked to indicate if an item should be included in the final list or excluded because judged to be nonessential. This time, only items that scored a median vote ≥7 with a disagreement index ≤1 were selected for the final checklist.

A third round of voting could be deemed necessary if there were any outstanding issues, such as signs of clear disagreement or misunderstanding. In this situation, the round would be performed as previously described in the second voting round, but would necessitate a more formal discussion (conference call) before proceeding to the vote.

External validation procedure. The goal of the external validation procedure was to assess the clarity of each item on the checklist. In other words, this step would assess if a group of individuals without extensive knowledge of the rationale for each item could understand adequately and consistently what was required to be reported. Thus, three independent raters, blinded to the panelists' votes and discussions, evaluated all items from the selected checklist for a set of preselected case reports published in 2011. Each item was scored as “reported”, “incomplete”, or “not reported”. Inter-rater agreement between raters was measured for each item for all the articles.

Two rounds of external validation were planned after each round of voting. If inter-rater agreement was low (if intra-class correlation was ≤0.40) 15,16 for a specific item, the item was reevaluated for precision and rephrased.

Statistical Analysis

For the item selection, medians and disagreement indexes were calculated for each item and interpreted as previously mentioned. The disagreement index, defined as the Interpercentile Range divided by the Interpercentile Range Adjusted for Symmetry, describes the dispersion of ratings more effectively than the mean absolute deviation from the median. Index values less than or equal to 1 indicate agreement between panelists 17. For the external validation, rater's inter-agreement was performed with the use of intra-class correlation coefficients (ICC) and their 95% confidence intervals (CI), using two-way analysis of variance (ANOVA) techniques on the total scores, as suggested by Shrout and Fleiss 18. This technique measures reliability between raters using a fully crossed (rater × paper), two-way ANOVA design in which paper and rater are separate effects. We considered rater as a random effect and the raters in the study a random sample from a population of potential raters. Statistical analyses were performed with IBM SPSS Statistics 21 for Windows (IBM Corp., Armonk, NY, USA).

PHASE 3: Writing of Reporting Guideline and Explanation & Elaboration Document

The rationale of each included item was drafted based on the aggregate comments of panelists during both voting rounds. All panelists were offered three opportunities to review the manuscript. Development of an evaluation and elaboration (E&E) document was circulated between all panelists to share comments on the presented items and concepts. The final draft needed to be approved unanimously by all participants prior to submission. The manuscript was also externally reviewed by a member of the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network and a member of the CARE guidelines.

Results

Selection of Participants

Twenty of the 29 EXTRIP members and the invited expert participated as panelists. The three invited raters accepted to participate in the external validation rounds. The panelists' primary expertise was as follows: nephrology (n = 5), emergency medicine (n = 3), clinical toxicology (n = 8), critical care medicine (n = 1), and clinical pharmacology (n = 4). The three raters were nephrologists. A methodologist supervised the entire process to ensure raters' blinding and to compile anonymous votes and comments.

Item Generation

The checklist of items was divided into four subsections (see Tables S1–S4): (i) Format of the manuscript, (ii) Clinical data regarding the case, (iii) Poison measurements, (iv) TK calculations.

Selection of Minimally Essential Items

First round of voting. A total number of 143 items were voted on by the panelists in the four sections: 14, 98, 15, and 16, respectively. The panelists' response rate was 100% and only 3.7% of the votes were missing (112/3024). During this first and inclusive round, no item was excluded. Nevertheless, after consulting the panelists' comments, some items were modified or added.

First round of external validation. Raters evaluated the first set of articles; missing response represented only 0.4% (32 missing out of 7539 total possible responses). The ICC was always over 0.4, meaning that there was a good correlation between raters. After reviewing the items with an ICC between 0.4 and 0.7 and related comments, some were modified to improve clarity and precision.

Second round of voting. After restructuration, merging, and rephrasing a few items, a total number of 137 items were voted on by the panelists in the four sections: 16, 93, 11, and 17, respectively. The panelists' response rate was 100% and only 1.7% of the votes were missing (49/2898). At this stage, 23 items were excluded.

Second round of external validation. Raters evaluated the second set of articles; missing responses were only 1.2% (64/5494). The ICC was always over 0.4, meaning there was a good correlation between raters. Fewer items had an ICC between 0.4 and 0.7 and none were modified further.

No further rounds were necessary as there were no comments from panelists or the raters suggesting dissent. Furthermore, there was no remaining disagreement or inconsistency that could have warranted a supplementary round. The final checklist consisted of 114 items, which were further combined and simplified for the elaboration and explanation document (Table1).

Table 1

Final simplified checklist

Section/Topic	Item No	Checklist item
Title and keywords	1
Abstract	2
Introduction	3
Subject characteristics	4a	Age and gender
	4b	Body weight and height
	4c	Concurrent diseases and medical conditions including baseline creatinine with estimated kidney function
	4d	List of regular co-medications (including over-the-counter and traditional medicines)
	4e	If the drug/poison was previously taken therapeutically: indication, length of use, dosing regimen and time of last dose
Poisoning description	5a	Identification/name of poison, route of exposure, formulation, quantity of exposure and timing/duration of exposure
	5b	Context
	5c	Co-ingestions, including alcohol and recreational drugs
	5d	History of spontaneous emesis
	5e	Delay to presentation following exposure or last dose
	5f	Source providing the history of the poisoning (patient, friends and family, paramedics)
Poisoning presentation	6a	Main toxic symptoms (chief complaints) during admission
	6b	Clinical findings (relevant physical examination findings) related to the poisoning during admission
	6c	Investigations relevant to the poisoning during admission including results of other pertinent toxicology laboratory testing
	6d	Prognostic characteristics (staging): relevant criteria utilized to determine the severity of poisoning
	6e	Risk assessment: conditions and clinical features (present or anticipated) that prompted ECTR
	6f	Diagnostic challenges and diagnostic reasoning including other diagnoses considered
Treatments other than ECTR	7	Type of intervention, administration of intervention and modification in intervention
Studied treatment (ECTR)	8a	Indication for ECTR initiation and suspected contraindications to a specific ECTR in this case
	8b	Timing of ECTR initiation (relative to exposure and admission)
	8c	Pertinent technical characteristics of ECTR, which may include: type/modality of ECTR, machine type, membrane (Kuf, surface area, material, brand, number used), duration, interruptions, number of treatments and anticoagulation, blood flow, dialysate flow, ultrafiltration, dialysate composition
	8d	Indication for ECTR cessation
	8e	Tolerability of the intervention and adverse or unanticipated events
Poison sampling	9a	Description of sampling methods  Measurement in vascular compartment: origin of sampling (plasma, serum, whole blood or red blood cells), measurement of free (unbound) poison if significantly bound to protein and rebound assessment  Measurement/collection in urine if significant renal elimination  Measurement/collection in effluent
	9b	Description of methods of sample testing
	9c	Access to all raw data
Toxicokinetic calculations	10a	Description of the calculations used
	10b	Calculations and results: Protein binding / sieving coefficient, Vd, extraction ratio, ECTR clearance, quantity removed (by ECTR and by endogenous pathways),% recovered during ECTR, fractional removal, half-life (pre-ECTR, per-ECTR and post-ECTR)
Follow-up and Clinical outcomes	11a	Clinician and patient assessed outcomes including improvement with ECTR and description of the temporal improvement in relationship to ECTR
	11b	Relevant clinical findings and investigations following ECTR including description of improvement after ECTR (sustained or not)
	11c	Length of ICU stay and of hospital stay
Timeline	12	Timeline of significant events and result of sampling on table or graph
Discussion	13a	Molecular characteristics (MW, PB, Vd, endogenous clearance) and normal/therapeutic range
	13b	Information about denominator
	13c	Relevant medical literature: Review of previous similar cases and data on the natural evolution of a similar case, in the absence of ECTR
	13d	Rationale for conclusions
	13e	Strengths and limitations of the management of this case
	13f	Generalizability / Applicability
Conclusion	14	Main take-home lessons

ECTR, extracorporeal therapy; ICU, intensive care unit; Kuf, ultrafiltration coefficient of a dialyzer; MW, molecular weight; PB, protein binding; Vd, volume of distribution.

Discussion

Despite their potential for stimulating research or informing clinical practice, published case reports often have compromised external validity due to omissions or lack of transparent reporting. The recently published CARE guidelines provide a formal framework for reporting intended to be applicable to most types of case reports. However, the complexity and level of detail required for ECTRTOX reports necessitate additional considerations for transparent reporting. Specifically, for this purpose, ECTRTOX reports not only require consistent clinical data but also reliable and consistent calculations enabling quantification of poison elimination; the latter point relies on proper poison sampling from the patient. All three of these components are essential to ensure completeness and ultimately required to assess the causal relationship between exposure and toxic symptoms, as well as between poison removal and clinical improvement. Furthermore, clinical imprecisions that are inherent to toxicology, such as timing, duration, and quantity of the exposure, need to be addressed and estimated. The various ECTR modalities available for poison elimination have technical specifications, which also need to be detailed.

We found, in our original literature review, that many published case reports, irrespective of the year of publication, omit critical details about patient demographics, treatments provided, and outcomes. Sample measurements are often rare or absent and toxicokinetic calculations are sometimes flawed by incorrect assumptions. These imprecisions impede the assessment of causality and generalizability. As a result, the information may be misleading to providers, and the clinical applications may be detrimental to patient care.

During the development of these reporting guidelines, most of the items from the CARE checklist, aside from patient perspective and informed consent, were included, as they were considered useful. Informed consent from the patient was not retained in the final list, as it was thought to be a requirement that depended more on the authors' responsibility and journal regulations, and did not alter the potential value or quality of the case report.

There are limitations to our methodology; the group of panelists that were chosen ultimately dictated which final criteria were selected, which can be biased if certain specialities are overrepresented. However, the individuals were chosen for their content expertise and came from diverse backgrounds. All panelists had extensive prior experience in reviewing and criticizing case reports for the EXTRIP clinical guideline development; many were also experienced in other peer review activities. Furthermore, several conference calls and in-house meetings permitted group discussions on the ideal constituents of a robust case report. Adherence of these reporting guidelines will need to be prospectively evaluated over time and modified accordingly, if necessary.

The sequential steps undertaken by our group differed from those proposed by the Guidance for Developers of Health Research Reporting Guidelines 14, due to the previous work performed by the EXTRIP workgroup. For example, significant discussion had already taken place prior to the Delphi rounds; there may therefore have been a priori consensus, which could theoretically have impeded critical thought, and this may be suggested by the relatively few Delphi rounds needed to obtain consensus 2. The addition of two external validation rounds permitted the reporting guideline development to remain dynamic and ascertain that all items remained clear and understandable without extensive explanations for an end user.

Because case reports are likely to remain a significant proportion of the available literature in many acute care specialties, such as emergency medicine, and clinical toxicology 1,19, the standardized reporting of case studies will enable better consistency between publications, aiding both authors and reviewers. Furthermore, clinicians who are assessing a patient poisoned with a rare or uncommon xenobiotic will have easy-to-follow steps for data and sampling acquisition as well as tools to facilitate and encourage dissemination of these cases.