A systemic review of toxic death in clinical oncology trials: an Achilles' heel in safety reporting revisited.

BACKGROUND: Toxic death is defined as study treatment-related mortality and as such is considered as an iatrogenic death. This belongs to unnatural death where an autopsy is advised. Until now, conventional autopsy is the gold standard to discriminate between pre- and post-mortem discrepancies.
METHODS: The consequences of lack of systematically performing an autopsy will be explored in the setting of oncological clinical trials.
RESULTS: During more than one decade, 6428 Serious Adverse Events have been registered in the EORTC Safety database on a total of 34 734 subjects. The number of deaths were 764 (mortality rate of 2.2%) whereof 255 (rate of 0.7%) toxic deaths. In 89.8% of these toxic deaths, no autopsy has been done; in 25.1% (64 cases) an inconsistent cause of death was found based on studying of the medical narrative. The autopsy rate was only 10.2% (26 out of 255) and, in 46.2% of the performed autopsies, there was a clinical pathological discrepancy.
CONCLUSION: When no autopsy is performed, there is a high risk for a wrong diagnosis in case of suspected toxic death. The high discrepancy rate, possibly due to a low autopsy rate, shows that toxic death is an Achilles' heel in iatrogenic mortality.

Toxic death is far from being a common understanding in the medical world. It has been used over many decades in oncology and, more specifically, in oncological clinical trials. The reason is that, historically, and still today, a substantial proportion of the subjects die in these studies. Main reasons for this mortality are the disease itself and nocive effects of the therapies which are often the last possible treatment options in a cancer population. Roberts defined ‘Toxic Death’ as treatment-related death, specified as fatal toxicity due to trial participation. One can define toxic death as a fatal Serious Adverse Event (SAE) related to study treatment.

As such, toxic death falls under the denominator of iatrogenic death (unnatural cause) and has to be distinguished from mortality due to the disease (natural death) itself. This brings toxic death in the domain of forensic medicine, where an autopsy could be necessary to give a definitive answer on the cause of death (Saukko and Knight, 2004; Madea and Saukko, 2008; Van de Voorde, 2010). When autopsy is mentioned throughout the text, referral is made to forensic or medicolegal autopsy.

The conduct of a clinical trial in Europe is regulated by Clinical Trial Directives and a major part of this legislation is the legal translation of the guidelines on Good Clinical Practice (GCP). Despite these directives, which include procedures when a subject dies in a clinical trial, the obligation to perform an autopsy for final diagnosis is missing. To note is the huge variation of autopsy rates in the world ranging from 4 to 49% (Saukko and Pollak, 2000).

In parallel there is a European Recommendation with criteria for performing a medicolegal autopsy, which applies to unexpected and unnatural deaths (Anonymous, 2000). However, iatrogenic death and as such toxic death is not specifically mentioned in this document.

Conventional autopsy as an instrument to reveal diagnostic discrepancies has been empirically proven throughout the years with first publications since 1912 (Shojania ; Roulson ). The classification of discrepancies between pre- and post-mortem findings according to Goldman can be considered as the reference (Goldman ; Battle ).

The goal of this work is to highlight the problems that may arise from lack of a correct diagnostic approach when a subject dies in an oncological clinical trial.

Materials and methods

Database

The SAE from a series of oncological clinical trials conducted within the European Organisation for Research and Treatment of Cancer (EORTC) is stored in a safety database, called SAfE (on site acronym for SAE Database). These clinical trials have been carried out with ethical committee approvals.

Assessment of data

Data extraction has been performed by one author (BP) for the period of 01 January 1999 until 28 January 2011 for the total number of SAE, deaths, and toxic deaths. In case of classification as toxic death, additional information is collected from the pharmacovigilance narrative and an autopsy report request is done, as standard procedure.

A first analysis was done independently by two authors (MK and BP): When an autopsy was performed, then the clinical-pathological diagnostic discrepancy was assessed using the criteria from Goldman (Goldman ). These criteria are shown in Table 1. Furthermore, the clinical reported cause of death diagnosis was reviewed to check its consistency in relation to the pharmacovigilance narrative. As such, we classified the cause of death diagnosis as consistent or inconsistent (and/or undetermined) in relation to the pharmacovigilance narrative. When no autopsy was performed, or no information of such autopsy was available, only the latter consistency review was done. For all the cases, the diagnostic term accuracy was verified using the Preferred Term of the Medical Dictionary for Regulatory Activities (MedDRA) terminology.

Table 1

Goldman classification for clinical pathological discrepancies

Discrepancy	Class	Description
Major	Class I	Missed diagnoses with potential adverse impact on survival and detection before death would have probably changed the treatment decisions
	Class II	Missed diagnoses with no potential impact on survival when management would have changed
Minor	Class III	Missed diagnoses related to the terminal disease but not related to the cause of death
	Class IV	Missed diagnoses not related to terminal disease or cause of death
Not assessable		No satisfactory diagnoses are found clinically or on the autopsy report to explain the cause of death

The further classification was done according to the country where the SAE was experienced; type of oncological disease; gender, and age category (<18, 18–45, 45–65, ⩾65). The amount of recruited subjects in the concerned studies for the same time period was extracted from the clinical database.

The corresponding author had complete access to the data in these studies and takes full responsibility for data integrity and correctness of its analyses.

An important remark is that most of the study protocols have exemption rules for SAE. The most important is presumed disease progression, described earlier (Roberts ). Presumed disease progression would as such not appear in the safety database, although it can be a potential toxic death.

Statistical methods

No formal statistics was used and the analyses of the data were purely descriptive.

Results

During more than one decade, 6428 SAE were registered within SAfE. These SAE were reported from 92 clinical trials. The number of deaths was 764 cases (mortality rate of 2.2%) whereof 255 (rate of 0.7%) toxic deaths and 509 cases of non-treatment related deaths. To put these numbers in relation to recruitment and geographical distribution, the accrual numbers during the same period are displayed in the graphs below (Figure 1A and B). Worthwhile to mention, the recruitment number of 34 734 subjects in these 92 studies was approximately half of the total accrual number of EORTC subjects (67 003) during that period (data from the EORTC Annual report 2011–2012). The reason is that only studies were taken into account with reported deaths and where EORTC is the owner of the safety database.

Figure 1

(A) Recruitment density from 92 EORTC studies between 1999 and 2011. (B) Toxic death density from 92 EORTC studies between 1999 and 2011.

Clinical versus post-mortem diagnosis

In the majority of the 255 study treatment-related deaths, no autopsy was performed, that is, 229 cases (89.8%). From these 229 deaths without autopsy, the reported cause of death was undetermined and/or inconsistent with the described narrative in 64 cases (25.1% from toxic deaths). An autopsy was only done in a minority of the cases (26 cases or 10.2%), and within 12 of these (46.2% of the autopsies), there was a clinical–pathological discrepancy. One case could not be assessed because of the fact that the cause of death remained uncertain after autopsy. To note is that autopsy in this case was limited to macroscopy only without any histo-pathological examination. This is summarised in Table 2.

Table 2

Toxic death in relation to autopsy status and diagnosis at death

255 toxic deaths (764 deaths)		Number	% toxic death
Autopsy performed	Concordanta	13	5.1
	Discrepanta	12	4.7
	Not Assessablea	1	0.4
No autopsy performed	Consistentb	165	64.7
	Inconsistentb	64	25.1

Diagnostic discrepancy between pre- and post-mortem findings according to Goldman.

Relation between clinical cause of death diagnosis and medical case narrative (Inconsistent includes undetermined cause of death diagnosis).

Potential consequence of not systematically performing an autopsy

Despite the fact that in 8 of the 12 autopsies, the clinical diagnosis was consistent with the narrative, discrepant post-mortem findings were discovered. When we extrapolate this to the 165 toxic deaths without autopsy and a cause of death diagnosis consistent with the narrative, there is a potential wrong cause of death reported in 110 cases. Adding the 64 cases with an inconsistent or undetermined cause of death, the risk for a wrong cause of death diagnosis increases to 68.2% (174 out of 255 cases) from the toxic deaths due to negligence of performing an autopsy.

Details about the 12 clinical-pathological discrepancies are displayed in Table 3. All these cases had major discrepancies, and in 7 cases (26.9% of the autopsies), a Class I discrepancy was withheld.

Table 3

Classification from twelve discrepant cases between pre- and post-mortem findings according to Goldman

Case	Clinical diagnosis	Autopsy findings
Class I (missed primary diagnoses with a potential impact on survival and treatment strategy)
1	Veno-occlusive disease of liver	Candidia albicans sepsis
2	Escherichia coli sepsis	Aspergillus pneumopathy
3	Aspergillus infection	Bacterial sepsis
4	Opportunistic bacterial infection	Disseminated fungal infection
5	Cerebral toxoplasmosis	Cerebral fungal infection
6	Death of unknown cause	Pulmonary edema/ Pneumonia/ Abdominal venostasis
7	Pneumonia	Deep venous thrombosis and Pulmonary embolism
Class II (missed primary diagnoses without impact on survival and treatment strategy)
8	Chemotherapy hepatoxicity	Pulmonary aspergillosis
9	Mycotic brain abcess	Haemorrhagic left brain damage
10	Fungal pneumonia	Non-fungal bronchopneumonia
11	Bleomycin-induced pneumopathy or Pneumocystosis	CMV pneumonia
12	Pancytopenia	Sepsis

Geographical spread

The median accrual in these 92 studies was 1465 subjects (min–max: 5–9214) with 4 studies where more than 1000 subjects were recruited. When the safety data were analysed in relation to accrual and geographical origin, the following could be noted. The rate of toxic death on all deaths was almost constant (33.4%) in the different countries, with an exception for Spain where this rate was much lower with 3 toxic deaths for 31 mortalities (9.7%) on a total of 1816 recruited subjects. The cause of death diagnosis was inconsistent with the narratives in France in half of the toxic deaths. The above is described in Table 4. Eastern European countries tend to underreport SAE as can be seen from Figure 1.

Table 4

Toxic death in relation to its geographic origin

Country		Toxic death			No autopsy
	Mortality number (rate %)	Number (rate %)	% of mortality	Autopsy number	Cons.	Incons.
Austria	15 (4.7)	4 (1.3)	26.7	2	2
Belgium	126 (3.2)	35 (0.9)	27.8	3	23	9
Canada	11 (1.4)	3 (0.4)	27.3		3
East-Europea	32 (2.5)	17 (1.3)	53.1	3	11	3
France	80 (1.1)	24 (0.3)	30.0	1	12	11
Germany	120 (3.8)	33 (1.0)	27.5		26	7
Italy	82 (3.2)	41 (1.6)	50.0	7	27	7
Portugal	7 (3.4)	4 (1.9)	57.1		3	1
RoWb	41 (3.4)	10 (0.8)	24.4		5	5
Scandinaviac	12 (1.5)	4 (0.5)	33.3	1	3
Spain	31 (1.7)	3 (0.2)	9.7		2	1
Switzerland	10 (1.5)	3 (0.5)	30.0		2	1
The Netherlands	102 (1.4)	34 (0.5)	33.3	7	21	6
Turkey	7 (1.8)	2 (0.5)	28.6		1	1
UK	88 (2.8)	38 (1.2)	43.2	2	24	12
Total	764 (2.2)	255 (0.7)	33.4	26	165	64

Abbreviations: Cons./Incons.=Consistent/Inconsistent cause of death diagnosis compared with the medical narrative.

East Europe: Bulgaria, Croatia, Czech Republic, Hungary, Poland, Serbia, Slovak Republic, Slovenia.

RoW: Rest of the World: Argentina, Australia, Brazil, Ecuador, Egypt, Hong-Kong, India, Israel, Japan, Pakistan, Peru, Russia, South Africa, Taiwan.

Scandinavia: Denmark, Norway, Sweden

Tumour disease type

A subdivision according to oncological disease type has been done in Table 5. Most of the mortalities and toxic deaths were seen in leukaemia trials with, respectively, 192 cases and 94 cases. There was a huge variance (3.8–100.0%) of the toxic deaths in relation to the deaths among the different disease groups.

Table 5

Toxic death in relation to oncological disease group

Disease group		Toxic death			No autopsy
(Number of subjects)	Mortality number	Number	% of mortality	Autopsy number	Cons.	Incons.
Chronotherapy (199)	6	3	50.0	1	1	1
Leukaemia (1023)	192	94	49.0	11	66	17
Lung (1442)	34	21	61.8	2	13	6
Breast (11 996)	41	12	29.3		6	6
NDD (430)	53	2	3.8	1		1
Melanoma (2823)	83	10	12.0		8	2
Lymphoma (2904)	37	29	78.4	5	16	8
Cutaneous lymphoma (49)	4	3	75.0		1	2
Radiation oncology (2609)	23	3	13.0	1		2
Head and neck (178)	5	4	80.0		2	2
Brain (1166)	19	5	26.3	1	3	1
Genito-urinary (2221)	83	23	27.7	1	16	6
Gastro-intestinal (1192)	22	10	45.5		9	1
Gynaecology (2875)	29	12	41.4	1	7	4
Children’s leukaemia (25)	2	2	100.0		2
STBS (2907)	59	21	35.6	2	15	4
Infectious diseases (628)	65
NOCI (76)	7	1	14.3			1
Total (34 743)	764	255	33.4	26	165	64

Abbreviations: NDD=new drug development; STBS=soft tissue and bone sarcoma; NOCI=Network Of Core Institutions;

Cons./Incons.=Consistent/Inconsistent cause of death diagnosis compared with the medical narrative

Demographic data

The division of the data according to gender and age category is represented in Table 6. There was no distinction between the two sexes related to number of autopsies performed. A low number of autopsies were observed within subjects younger than 45 years old.

Table 6

Toxic death in relation to gender and age category

				No autopsy
Gender	Mortality number	Toxic death number	Autopsy number	Cons.	Incons.
Male	461	143	15	91	37
Female	303	112	11	74	27
Age category
<18 year	2	2		2
18–45 year	97	18	3	11	4
45–65 year	293	95	16	55	24
⩾65 year	372	140	7	97	36

Abbreviations: Cons./Incons.=Consistent/inconsistent cause of death diagnosis compared with the medical narrative.

Discussion

Autopsy advocacy

Our manuscript is built upon the assumption that autopsy is the gold standard for correct diagnosis definition, but how far is that true? Published data mention that a technically and adequately performed conventional autopsy fails to determine the correct cause of death in 1–7% of the cases (Rilke, 1991; Shojania ; Van de Voorde, 2010). On the opposite, when no autopsy has been performed, data shows a wrong cause of death diagnosis in about 30% of the cases (Pritt ; Cambridge and Cina, 2010). The diagnostic problems for fatal sepsis, multi-organ failure, and systemic inflammatory response syndrome are highlighted in a recent publication (Pastores ). Addition of a few biomarkers could ameliorate the diagnostic value of autopsy in case of sepsis (Tsokos, 2007). Another problem with toxic death could be interpretation difficulties from post-mortem toxicological results, such as inter-individual differences in metabolisation of drugs (Musshoff ; Sajantila ).

In our study, the cause of death remained undetermined in one case, probably owing to the fact that the autopsy did not include a microscopical investigation. Not only the correct diagnosis is important but as well the causal relationship with study treatment. In clinical trials where efficacy and safety from study treatment is investigated, it is paramount to know whether the mortality was due to study treatment or could be attributed to another reason. Autopsy is still unbeaten in both excluding and confirming the causal relationship with drugs (Ebbesen ; Madea ; Madea and Preuss, 2009).

Until today, other techniques such as minimally invasive (needle biopsy and endoscopic) and non-invasive (post-mortem total body CT or MRI) autopsies have not been widely accepted to replace conventional autopsy (Ermenc, 2000; Bisset ; Burton and Underwood, 2007; Thomsen ; Fan ; Roberts ).

Mortality and toxic death rate

This study reveals a mortality rate of 2.2% and a toxic death rate of 0.7% (34 734 subjects) that can be considered, respectively, at the lower limit and upper limit compared with data in the literature for mortality rate (2.1–20.2%) and toxic death rate (0.1–0.95%) (Lazarou : Ebbesen ; Roberts ; Pastores ). To place toxic death rate in perspective, in one study they found that fatal SAE are double as high compared with the number of fatal traffic accidents (Madea and Preuss, 2009). In another study, fatal SAE counted for between the fourth and sixth cause of death in US hospitals (Lazarou ).

The extreme mortality number of 20.2% (658 deaths on 3257 patients) in one study can be explained by the fact that this was observed in an oncological intensive ward (Pastores ). Our study deals with an oncological population with an inherent increased risk for dying. However our mortality rate is on the lower end. This can only be partially explained by underreporting of the number of SAE, such as in a Swedish study where they mentioned that 86% of adverse events are not reported (Backstrom ). In most of the EORTC trials, monitoring is foreseen and possible non-reported cases would have been detected. A more plausible explanation could be that, in a lot of trial protocols, cancer progression is excluded to be reported as an SAE. Consequently, when such subject dies, disease progression may not be the cause of death and a potential toxic death as such is not reported. Another explanation is selection bias, because according to GCP guidelines, it is the investigator’s responsibility to assess the causal relationship between death and study treatment. Some investigators have the tendency to declare the death preferentially due to the disease or its complications (Pastores ; Burgesser ). Without an autopsy, it is far from evident to declare a case as study treatment related mortality or not. Thus, in our study with 509 cases reported as non-treatment-related death, an unknown percentage, in fact, could be a treatment-related mortality.

Autopsy rate and diagnostic discrepancy

It is important to learn from ante-mortem errors, but also to know whether the death is due to study treatment or not (Rilke, 1991; Madea and Preuss, 2009; Lehrnbecher ). Probably one out of three death certificates are wrong in the absence of autopsy (Roulson ).

The autopsy rate in this analysis was 10.2% (26 out of 255). In the literature, this varied from 3.7% to 100% (Goldman ; Battle ; Saukko and Pollak, 2000; Ebbesen ; Shojania ; Roulson ; Aalten ; Burton and Underwood, 2007; Pastores ; Shojania and Burton, 2008; Van de Voorde, 2010), the most recent data (last 20 years) ranging between 3.7% and 28.4% (Shojania ; Roulson ; Aalten ; Burton and Underwood, 2007; Shojania and Burton, 2008). One exception is the prospective study from Ebbesen with an autopsy rate of 78.1% (Ebbesen ). This is interesting as the goal of this study was to look at incident rates of toxic deaths in the hospital. Accordingly, our autopsy rate is low in determining pre- and post-mortem discrepancies.

Moreover, we found in our study a diagnostic discrepancy rate of 46.2% (in 12 autopsies on a total of 26 autopsies or 4.7% from the toxic deaths). All these discrepancies were major, whereof Class I discrepancies compromised 26.9% of the autopsies. Published data varies between 23.5% and 39.0% for major discrepancies and from 9.0 to 25.5% for Class I discrepancies (Goldman ; Battle ; Di ; Shojania ; Roulson ; Aalten ; Pastores ; Burgesser ). The high rates both for major and Class I discrepancies are not so surprising as they are correlated with low autopsy rates. It has been shown that major and Class I discrepancies decrease with, respectively, 12.4% and 17.4% for each increase of autopsy rate by 10% (Shojania ; Van de Voorde, 2010).

Most of the discrepancies in this study were fungal infections (mainly pulmonary) followed by sepsis and thromboembolic events. In the study from Burgesser , these were mainly respiratory infections and acute myocardial infarction. The author stated that 15% from the cases were transplantation patients and with the use of immunosuppressive drugs, making them more susceptible for infections. This is also the case in an oncological population where a lot of bone marrow suppressive agents make these patients vulnerable for infections. Embolic events in cancer patients are not exceptional, because the tumour and therapy are both risk factors for developing emboli. Lung emboli and opportunistic infections (viral, bacterial and fungal) are often a source of clinical–pathological discrepancy in the literature (Goldman ; Battle ; Rilke, 1991; Roulson ; Aalten ; Pastores ; Lehrnbecher ; Burgesser ).

Geographical differences

In most countries, the number of toxic deaths was approximately one-third of the mortality numbers, except for Spain where this was only 9.7%. We cannot explain this effect, because these studies were run internationally and the same monitoring was applied among these countries. We did not observe higher autopsy rates in contrary of what is described in literature for regions such as the USA, Canada, UK, Hungary, Sweden and Switzerland (Burton and Underwood, 2007; Van de Voorde, 2010). In these countries no or only a few autopsies were performed despite the quite high recruitment numbers.

In 25.1% of the toxic deaths, the cause of death diagnosis was inconsistent with the narrative, and no autopsy was performed to verify the correct diagnosis. The reason for, in France, this even happened in half of the cases, remains unclear. It is certainly not the only country with a low autopsy rate, but autopsy could have helped to decrease the inconsistency rate.

Tumour and demographic characteristics

A high mortality and toxic death rate were reported for leukaemia trials. A possible explanation for this effect is that the majority of the subjects were older patients with acute myeloid leukaemia and as such represented a black and white matrix compared with non-haematological tumours. Together with a highly toxic therapy resulting in a lot of infectious complications, it may not be surprising that this disease is still characterised with a high mortality. The high variance of the toxic death rate among the different tumour types was remarkable and probably due to the high heterogeneity of the population and its treatment strategy.

An interesting observation was the low number of autopsies within subjects younger than 45 years old. In forensic medicine, dying at young age and iatrogenic death (including toxic death) are considered unnatural deaths and autopsy is advised. Despite the presence of both conditions, it did not result in a higher autopsy rate for this subpopulation.

Shortcomings of this study

Firstly this is a retrospective study that generates bias for available information. It is possible that only when there was doubt about the clinical cause of death diagnosis, an autopsy was performed.

Some of these deaths may have occurred at home or in smaller district hospitals where the investigator has little chance to influence requesting an autopsy.

Most of autopsy rates in publications are based on hospitalised cohorts and not as in this study where both subjects in ambulant and hospital environment were included. Cautious comparison is therefore needed with these publications.

At last, the safety database is constructed in such way to report to regulatory bodies. The disadvantage is that in case there is no relationship between death and study treatment, available information is very limited. Only when a death is classified as toxic death (by the investigator), a narrative is generated and stored in the database, and completed with an autopsy report when available. It may not be a surprise that, even with consideration of the results of the study here, this is a potential source of bias.

Conclusion

This study with 34 734 subjects recruited during more than one decade in 92 oncological studies has a mortality rate of 2.2% and a toxic death rate of 0.7%. The low autopsy rate of 10.2% (26 autopsies on 255 toxic deaths) severely disrupts the quality and correctness of the diagnoses, translating in a high diagnostic discrepancy rate of 46.2% (or 4.7% of all toxic deaths). The majority of diagnostic discrepancies were fungal infections, which were discovered by autopsy. Because of negligence of performing an autopsy, the risk for a wrong cause of death diagnosis peaks at 68.2%. It is justified to systematically perform an autopsy in case of suspected toxic death, not only for the value of autopsy as a diagnostic test, but also to identify similar cases with a Class I discrepancy and where correct ante-mortem knowledge might have influenced the survival. This paper should serve as a stimulus for further investigation in this matter and as a recommendation to perform an autopsy at a higher percentage of death events in clinical trials.