Quality evaluation of investigator-initiated trials using post-approval cancer drugs in Japan.

Investigator-initiated trials (IIT) are important aspects of medical research and have contributed substantially to modern oncology. IIT using post-approval drugs have been conducted by domestic institutions in Japan. Data from the present study were obtained by all IIT registered clinical trials for five cancers (lung, colorectal cancer, gastric cancer, liver cancer, and breast cancer) using drugs approved from 1999 to 2009 in Japan. Kaplan-Meier method, analysis of variance (anova), and Kruskal-Wallis test were used to estimate time to enrolment completion (TTEC) and time to enrolment per patient (TTEP). Of 1222 trials eligible for analysis, 465 trials (38%) completed enrolment to the studies, and 203 trials (17%) published results. In the distribution according to trial phase, 98 (8%) were phase I, 1058 (87%) were phase I/II + II, and 66 (5%) were phase II/III + III. Accrual achievement and publication rates were higher in late-phase than in early-phase trials. Median TTEC was 1387 days (95% confidence interval [CI], 1302-1472). Median TTEP was 38.5 days (95% CI, 34.5-42.5). The median TTEC and TTEP were significantly different in each trial phase (P < 0.01), funding source (P < 0.01), and publication status (median TTEC published trials versus unpublished trial; 720 days vs 1672 days, median TTEP; 16 days vs 55.8 days; P < 0.001). Many IIT using approved cancer drugs have been conducted; however, the quality of the clinical trials was low in terms of accrual achievement, publication rate, and time to publication of trial results.

Clinical trials have helped improve cancer care and define the standards for optimal cancer treatment. Although not without intrinsic risk, clinical research is necessary to advance treatment. Clinical trials consider and try to improve on the best available therapies in addition to providing structure and rigor to treatment plans.1 Clinical trials generate safety and efficacy data through phase I–III trials. Implementation of clinical trials is classified in terms of initiatives and funding sources, whether the initiatives are industry‐initiated or investigator‐initiated trials (IIT), and whether the trials are funded by academic or public funds, or by pharmaceutical, biotechnology, or medical device companies.

IIT are important aspects of medical research in academic institutions and have contributed substantially to modern oncology.2 The central aims of an IIT include collecting additional safety data and data that could be used to support new indications with limited or no commercial potential. However, planning an IIT is a major challenge for Japanese investigators because of limited advice from experts in protocol writing, regulation of trials, and statistical analysis. Furthermore, there are limited government funds available for new treatment development, including clinical trials. Therefore, investigators who wish to conduct clinical trials that might lead to new drug development must obtain funding from public and private sources. Implementing a clinical trial with an investigational new drug (IND) is more expensive than implementing a clinical trial with an established drug. IIT using post‐approval drugs have been conducted by domestic institutions in Japan.3 Meanwhile, IIT without industry collaboration have often been carried out without certain guidelines, such as a clinical trial directive, good clinical practice (ICH‐GCP), and the standard operating procedures (SOP) of their institutions.

The quality of clinical trials is evaluated by several factors, including registration to a trial database,4 publication rate, time to publication of trial results,5 funding source‐related reporting,6 reporting trial results,7 trials conducted by a secure cooperative group, and accomplishment of trial enrolment.8 Moreover, generating predictable results and moving onto the next phase of a trial demonstrates a well‐planned trial. The objectives of the present study were to reveal the current condition of IIT using approved cancer drugs in Japan, and to evaluate the actual quality of IIT data from a clinical trial registry.

Materials and Methods

Data sources

We used data from Japanese registries (University Hospital Medical Information Network [UMIN], Japanese Pharmaceutical Information Center Clinical Trials Information [Japic‐CTI], and Japanese Medical Association Center for Clinical Trials [JMACCT]). IIT are supported by numerous companies within the pharmaceutical and biotechnology industry and are referred to as investigator‐initiated studies (IIS) or investigator‐sponsored trials (IST).

Identification of eligible studies

The aim of the present study considers the current circumstances of clinical IIT. The eligibility criteria for trials to be included in the present study were: (i) being enrolled to a clinical trial registry; (ii) being an IIT, including investigator‐initiated, pharmaceutical company‐sponsored trials; (iii) treating one of the top five causes of cancer‐related death (lung, gastric, colorectal, breast, and liver cancer); (iv) prospectively using drugs that were approved in Japan from 1999 to 2009 (Table S1); and (v) beginning before October 2014. The exclusion criteria were: (i) industry‐initiated trials; (ii) pre‐initiation trials; (iii) prospective observational clinical trials, including phase IV trials; and (iv) biomarker studies.

Definitions

In the present study, an IIT was defined as independent research conducted by an investigator or institution (academic, private, or governmental), regardless of funding source. Three reviewers (SK, HH, and JH) independently evaluated trials enrolled with clinical trial registries as IIT.

The study phases were defined as follows: phase I is testing the safety, side‐effects, best dose, and timing of a new treatment; phase II is a study that tests whether a new treatment works for a certain type of cancer; phase I/II also tests how well a certain type of cancer responds to a new treatment, and in the phase II part of the trial, patients usually receive the highest dose of treatment that does not cause harmful side‐effects based on the phase I part of the clinical trial; phase III is a study that tests the safety and how well a new treatment works compared with a standard treatment, and combining phases II and III might allow research questions to be answered more quickly and with fewer patients.

We calculated the time to enrolment completion (TTEC) as the number of days from the initiation date of a trial until its primary completion date as reported to the trial registry or to publication. The primary completion date was defined as the date on which the final participant was examined or received an intervention for the purpose of final data collection for the primary outcome as censored data. The time to enrolment per patient (TTEP) was defined as the average time required to enrol a patient in the trial (Fig. 1).

Figure 1

Flowchart of the present study.

Funding sources were classified as government (ministries and other agencies), other public (non‐profit organization or academic institution), industry (pharmaceutical, biotechnology, or medical device company), and veiled (no precise funding source).

To determine the publication and publication rate for clinical trials, three reviewers (SK, HH, and JH) independently searched PubMed, MEDLINE, and EMBASE. We used seven criteria to identify matching publications: author name, registration number, study design, indication, intervention, primary outcomes, and intention‐to‐treat enrolment. We further refined the list by matching additional characteristics from the list of each registry.

Statistical analysis

To aggregate trials into groups, we examined investigator‐reported baseline characteristics, including phases of trials, planned sample size, inclusion criteria, features of the clinical trial design, and source of funding. We used chi‐squared (χ2) or Fisher's exact test to compare cohorts and to characterize the data. The Kaplan–Meier method was used to estimate TTEC, censoring data based on the last update of each registry. Differences between each arm were assessed using a log–rank test. Analysis of variance (anova) and Kruskal–Wallis test were used to estimate statistically. Estimates were stratified according to funding source, trial phase, type of cancer, and trial status (terminated, completed, or published). All statistical analyses were done using IBM SPSS Statistics 18 software (IBM Corp., Somers, NY, USA).

Results

Study population

From 1999 to 2009, 10 drugs were newly approved for the treatment of lung cancer, 5 for colorectal cancer, 3 for gastric cancer, 3 for liver cancer, and 9 for breast cancer (Table S1).

Of 1222 trials eligible for analysis during the study period, 465 trials (38%) completed study enrolment, and 203 (17%) published results (Fig. 1). In the distribution according to trial phase, 98 (8%) were phase I trials, 1058 (87%) were phase I/II + II, and 66 (5%) were phase II/III + III.

According to type of cancer, lung cancer was treated in 457 trials (37%); colorectal cancer, 329 trials (27%); gastric cancer, 157 trials (13%); liver cancer, 84 trials (7%); and breast cancer, 195 trials (16%). Distribution of trial phase did not differ between cancer types. Distribution of funding sources included government sources (6%), industry (3%), other public funding (8%), and veiled funding (83%). The government funding rate of late‐phase trials (phase II/III + III) was higher than that of early‐phase trials. Funding from government sources (20%) was higher for liver cancer trials than for other types of cancer. Other public funding was higher for breast cancer trials than for other cancer types (Table S2). Moreover, the rates of enrolment completion and publication were higher in late‐phase trials (Table 1).

Table 1

Distribution of clinical trials by trial phase

	Total	Phase I	Phase I/II, II	Phase II/III + III
	1222	98	1058	66
Type of cancer
Lung cancer	457	37	401	19
Colorectal cancer	329	23	287	19
Gastric cancer	157	15	131	11
Liver cancer	84	12	69	3
Breast cancer	195	11	170	14
No. trial arms
1	967	98	869	0
2	247	0	184	63
3	6	0	4	2
4	2	0	1	1
Funding source, n (%)
Government funding	75	8	54	13
Industry funding	37	2	26	9
Other public funding	97	6	81	10
Veiled funding	1012	84	901	27
Enrolment completed	465 (38%)	47 (48%)	378 (36%)	40 (61%)
Published	203 (17%)	15 (15%)	173 (16%)	15 (23%)

Time to enrolment completion

Median TTEC was 1387 days (95% confidence interval [CI], 1302–1472). By trial phase, the median TTEC was 1002 days (95% CI, 797–1207) for phase I trials, 1403 days (95% CI, 1297–1509) for phases I/II and II, and 1290 days (95% CI, 751–1828) for phases II/III and III. TTEC did not differ by type of cancer (log–rank, P = 0.18). By funding source, the median TTEC was 1255 days (95% CI, 1069–1441) for government funded trials, 1093 days (95% CI, 1027–1159) for industry funded trials, 1110 days (95% CI, 908–1312) for other public funded trials, and 1435 days (95% CI, 1330–1540) for funding from veiled sources (Table 2; Fig. S1).

Table 2

Time to enrolment for each factor according to type of cancer, trial phase, funding source, and publication

		Period of enrolment, days (95% CI)
Type of cancer	Lung cancer	1320 days (1165–1475)	P = 0.18
	Colorectal cancer	1303 days (1166–1440)
	Gastric cancer	1302 days (1114–1490)
	Liver cancer	Not reached
	Breast cancer	1503 days (1342–1664)
Trial Phase	Phase I	1002 days (797–1207)	P = 0.009
	Phase I/II, II	1403 days (1309–1497)
	Phase II/III, III	1290 days (751–1829)
No. trial arms	1	1358 days (1265–1451)
	2	1530 days (1222–1837)	P = 0.02
	3	726 days (713–739)
	4	1858 days (–)
Funding source	Government funding	1255 days (1069–1441)	P = 0.004
	Industry funding	1093 days (1027–1159)
	Other public funding	1110 days (908–1312)
	Veiled funding	1435 days (1330–1540)
Journal published	Published	720 days (673–767)	P < 0.001
	Unpublished	1672 days (1539–1805)

The median TTEP was 23.3 days (95% CI, 20.7–25.8). By type of cancer, the median TTEP was 20.0 days (95% CI, 13.8–26.1) for lung cancer, 26.0 days (95% CI, 22.0–30.0) for colorectal cancer, 24.0 days (95% CI, 18.3–30.0) for gastric cancer, 29.4 days (95% CI, 21.4–37.3) for liver cancer, and 17.3 days (95% CI, 10.8–23.8) for breast cancer (P < 0.001). By trial phase, the median TTEP was 47.5 (95% CI, 34.7–58.2) for phase I trials, 22.4 days (95% CI, 19.9–24.9) for phase I/II and II trials, and 5.2 days (95% CI, 1.9–8.4) for phase II/III and III trials (P < 0.001). By funding source, the median TTEP for veiled funding sources was longer than for unveiled trials (P < 0.001) (Fig. 2).

Figure 2

Subgroup analyses of time to enrolment per patient of clinical trials. *Kruskal–Wallis test.

Reporting results to medical journals

During the analysis period, 203 (17%) trials had been reported in medical journals. The publication rate of trials for gastric cancer (26%) was higher than that for other cancers. Trials funded by a government source had a higher publication rate than did other funding sources (Table 3). The median TTEC was 720 days (95% CI, 673–767) for published trials and 1672 days (95% CI, 1539–1805) for unpublished trials (Fig. 3).

Table 3

Distribution of clinical trials for publication

		Published	Unpublished
	Total	203 (17%)	1019 (83%)
Type of cancer	Lung cancer	90 (20%)	367 (80%)
	Colorectal cancer	32 (10%)	297 (90%)
	Gastric cancer	41 (26%)	116 (74%)
	Liver cancer	8 (10%)	76 (90%)
	Breast cancer	32 (16%)	163 (84%)
Study Phase	Phase I	15 (15%)	83 (85%)
	Phase I/II, II	173 (16%)	885 (84%)
	Phase II/III, III	15 (23%)	51 (77%)
No. trial arms	1	167 (17%)	800 (83%)
	2	32 (13%)	215 (87%)
	3	3 (50%)	3 (50%)
	4	1 (50%)	1 (50%)
Funding source	Government funding	21 (26%)	59 (74%)
	Industry funding	3 (9%)	31 (91%)
	Other public	16 (16%)	81 (84%)
	Veiled funding	163 (16%)	848 (84%)

Figure 3

Time to complete enrolment of clinical trials according to publication in medical journals.

Duplicative trials

We categorized trials that targeted advanced cancer, perioperative cancer, and elderly patients with cancer. Several regimens overlapped regarding the type of cancer and the subject of the trial. Seventeen trials (5% of trials for colorectal cancer) using bevacizumab + XELOX were conducted for advanced colorectal cancer, 30 (7% of trials for lung cancer) using erlotinib monotherapy were conducted for advanced lung cancer, and 13 (8% of trials for gastric cancer) using S‐1 plus cisplatin for perioperative gastric cancer were conducted (Fig. S2).

Discussion

The present study revealed the actual implementation status of IIT using post‐approval drugs. Investigators have conducted many duplicate studies using the same regimens and the same types of subject. TTEC and TTEP, indicating promising trial enrolment, were prolonged in early phase trials. The trials reported to medical journals had a shorter TTEC. To conduct a high‐quality IIT using post‐approval drugs, researchers should carefully consider before planning IIT.

It is important to reduce the number of low‐enrolling clinical trials while improving the number of high‐priority trials that successfully achieve their intended accrual goals.9 Approximately 38% of Cancer Therapy Evaluation Program (CTEP)‐supported oncology trials fail to attain the originally specified minimum accrual goal, with phase III trials more frequently falling short of achieving their accrual goals.10 These clinical trials not only are unable to achieve the patient enrolment necessary to evaluate the proposed scientific hypotheses, but also remain open longer than planned, resulting in unanticipated costs from additional administrative and clinical resources.11, 12, 13 Although phase III trials may have a greater proportion of trials that reach a scientific endpoint without achieving the originally intended accrual goal than early‐phase trials, early‐phase trials take a longer time to accrue than phase III trials.14 TTEC in the present study revealed that early‐phase trials, except phase I trials, needed a longer time to accrue than late‐phase trials. Moreover, the time needed to accrue each patient, TTEP, was influenced by trial phase. These results are supported by the findings of previous studies.11, 12, 13, 14

The funding source of clinical trials affects several aspects of achievement, and one report showed that industry‐funded trials are more often favorable to the sponsor's products than non‐industry‐funded trials.15 However, industry‐funded trials adhere to legal obligations more often than trials funded by government or academic sources in Clinicaltrials.gov.6 Funding sources have not been made abundantly clear in the Japanese clinical trial registry; however, the present study showed that clarification of the funding source leads to steady accrual to the trials, and the funding source affects the publication rate. Efforts at fundraising are required to allow trials to reach their accrual goals, accomplish their objective, publish the trial results, and conduct high‐quality clinical trials.

In 2004, the International Committee of Medical Journal Editors (ICMJE) announced its strategy to make the registration of clinical trials a prerequisite for publication consideration.4 The World Health Organization (WHO) facilitates international collaboration in setting standards for clinical trial registration.16 The Reporting Food and Drug Administration Amendments act (FDAAA) of 2007 reflects the ethical obligation of researchers and sponsors to report and publish the results of trials and to respect human trial participants through fidelity to commitments made explicit to contribute to generalizable knowledge.17 In the present study, objectives were extracted from samples of the clinical trial registry. However, for those trials that were completed, most have not been reported in the registry and few have been published in medical journals. Regardless of the FDAAA mandate, the reporting rate was similarly low in ClinicalTrials.gov.6 Reporting, including journal publication, reflects the ethical obligation of researchers to respect human trial participants through fidelity to commitments made explicit in the informed consent. In IIT, principal investigators who initiated the trials have been more responsive to take action on the FDAAA mandate.

Our study had several limitations. Our analysis included registered Japanese clinical trials only, and these registrations might have been missed or not regularly updated. We could not consult each trial protocol directly. Moreover, the present study targeted only those oncology drugs that have received regulatory approval and clinical trials for the top five causes of cancer‐related deaths. Finally, because trial characteristics are submitted by investigators, we cannot independently verify their accuracy.

In conclusion, multiple IIT using approved cancer drugs have been conducted, but the quality of clinical trials was not high grade in terms of publication rate, time to publication for trial results, and accrual achievement. We propose that researchers deliberate carefully and conduct clinical trials with attention to the following: (i) a rigorous review process of the trial idea; (ii) fundraising for a particular clinical trial; (iii) disclosure of the funding source; (iv) implementation according to certain guidelines, such as the clinical trials directive, ICH‐GCP, and the SOP of their institutions; (v) building a secure cooperative study group; (vi) accurate estimation of accrual prediction and early judgment of trial termination; and (vii) reporting along FDAAA guidelines because clinical trials must be conducted for public spiritedness and in an ethical manner.

Disclosure Statement

Shunsuke Kondo has received research funding from AstraZeneca, Eli Lilly Japan K.K., and Bayer Yakuhin, Ltd. The other authors declare no conflict of interest.

Fig. S1. Time to complete enrolment of clinical trials according to funding source.

Click here for additional data file.

Fig. S2. Duplicative trials and classification according to advanced cancer, perioperative cancer, and elderly patients with cancer.

Click here for additional data file.

Table S1. List of drugs and the dates of approval.

Click here for additional data file.

Table S2. Distribution of clinical trials by cancer type.

Click here for additional data file.