The quality of registration of clinical trials.

BACKGROUND: Lack of transparency in clinical trial conduct, publication bias and selective reporting bias are still important problems in medical research. Through clinical trials registration, it should be possible to take steps towards resolving some of these problems. However, previous evaluations of registered records of clinical trials have shown that registered information is often incomplete and non-meaningful. If these studies are accurate, this negates the possible benefits of registration of clinical trials. METHODS AND
FINDINGS: A 5% sample of records of clinical trials that were registered between 17 June 2008 and 17 June 2009 was taken from the International Clinical Trials Registry Platform (ICTRP) database and assessed for the presence of contact information, the presence of intervention specifics in drug trials and the quality of primary and secondary outcome reporting. 731 records were included. More than half of the records were registered after recruitment of the first participant. The name of a contact person was available in 94.4% of records from non-industry funded trials and 53.7% of records from industry funded trials. Either an email address or a phone number was present in 76.5% of non-industry funded trial records and in 56.5% of industry funded trial records. Although a drug name or company serial number was almost always provided, other drug intervention specifics were often omitted from registration. Of 3643 reported outcomes, 34.9% were specific measures with a meaningful time frame.
CONCLUSIONS: Clinical trials registration has the potential to contribute substantially to improving clinical trial transparency and reducing publication bias and selective reporting. These potential benefits are currently undermined by deficiencies in the provision of information in key areas of registered records.

Introduction

Many instances of unethical research conduct by clinical trial sponsors and investigators have come to light over the past decade. The types of misconduct vary and include not obtaining approval from research ethics committees, not obtaining informed consent from trial participants and the fabrication of data [1]–[5]. Despite the ethical obligation to accurately report the results of research in humans [6], some trial sponsors have deliberately withheld negative outcome information when publishing the findings of clinical trials, and when making the trial findings available to regulatory authorities [7]–[9]. Such behaviour is particularly concerning when the misconduct involves trials recruiting participants in low and middle income countries with deficient oversight mechanisms [5].

Prospectively registering clinical trials can potentially prevent at least some of this misconduct from occurring, specifically selective reporting, by putting key protocol information about each trial in the public domain, ideally before the first participant is recruited to the study. Five years have now passed since the International Committee of Medical Journal Editors (ICMJE) first published its statement requiring registration as a precondition of publication [10], and the World Health Assembly approved the establishment of the International Clinical Trials Registry Platform (ICTRP) by the World Health Organization (WHO). Today the ICTRP provides free access via a single web portal to more than 120,000 records of registered trials made available by clinical trial registries around the world [11]. In the intervening years the number of countries and agencies that have created and implemented their own policies on trial registration has increased, including the World Medical Association which now explicitly states in the Declaration of Helsinki that prospective registration is an ethical requirement [6], [12]–[24].

Prospective registration can only contribute to the more ethical conduct of clinical trials however, if all of the key information about the trial is registered, and the registered data are meaningful. The ICMJE agrees that quality is important and states that missing or uninformative entries in any of the fields required by the WHO 20-item Trial Registration Data Set is inadequate [25], [26]. The quality of registered data has been called into question of late, with particular concerns regarding the quality of contact information [27]–[31], intervention details, [27], [30]–[33] and the outcomes (and outcome measures) being used [27], [31], [33]–[36]. Poor data quality raises doubt on the ability of trial registration to meet the challenge of achieving research transparency, including the ability to adequately address publication bias and selective reporting, and reducing the amount of wasted research [37]–[39].

The objective of this study was therefore to determine whether registered records of clinical trials contained complete and meaningful data for key items in the WHO Trial Registration Data Set [25]. Given the particular concern regarding the quality of contact information, intervention details, and outcome information, it was agreed that these data items would be the focus of the study.

Methods

A random 5% sample of all clinical trial records of trials registered as interventional between 17 June 2008 and 17 June 2009 was taken from the ICTRP database. Records of trials that were registered as observational, records that pertained to US Food and Drug Administration (FDA) lockbox device trials [40] and records that were duplicate records (due to registration of a trial in more than one register) were not eligible for the sample [41]. For trials with multiple records the record with the earliest registration date was considered eligible. At the time the sample was taken the database included trials registered in nine different registries.

About the data

The ICTRP Search Portal imports the WHO Trial Registration Data Set from registries that meet WHO criteria, including ClinicalTrials.gov. As the format of each data item differs across registries, data is currently imported into the portal as text. The ICTRP publishes a hyperlink to the record in the source registry (i.e. the registry that provided the data) so users can view additional information, if required.

Data extraction

Registry name, trial ID, target sample size, recruitment status, date of registration, date of first enrolment and the public and scientific title for each record were downloaded from the ICTRP database and imported into Excel on 17 June 2009.

During manual searching of records, it became clear that several records of trials that were registered as interventional were in fact records of observational trials, diagnostic accuracy trials or treatment protocols for continuation of treatment after inclusion in a study protocol. These records were excluded from further data extraction.

Descriptive information on study phase, study design, randomization status and inclusion criteria for gender and age of participants was extracted manually from the complete registered record in the source registry. Data on interventions and sponsorship was also extracted manually and was then coded. The system used to code interventions was adapted from the codes used for intervention types on ClinicalTrials.gov [42]. Primary sponsors were coded as being foundation, government, industry, university/hospital, or other. Trials were coded as being industry funded (primary sponsor was industry), partially industry funded (primary sponsor was non-industry, but secondary sponsor or source of monetary or material support was industry) or non-industry funded.

Contact information

The presence or absence of the following contact details was evaluated: name of a contact person (investigator or other), email address and telephone number. The WHO 20-item Trial Registration Data Set requires registration of separate scientific and public contact details [25]. There was however variation in registration formats for contact details between different registries. Some registries had one field for contact details, others had two separate fields for public and scientific contact details and others multiple contact fields. For records with only one contact field the presence of contact information was extracted from that field. For records with multiple contact fields, if the contact details were present in any of the fields, the information was denoted to be present.

Interventions

Given the considerable variability in the types of interventions evaluated in trials, comparison of registration quality between different intervention categories is difficult. It was therefore decided to limit the evaluation of the quality of registered intervention data to trials that investigated drugs, biologicals or vaccines, including active comparators. Placebo comparators were not evaluated. For each intervention and active comparator the presence or absence of the following five intervention specifics was collected: name, dose, duration of the intervention, frequency of administration and route of administration. All intervention arms were assessed separately. Name was denoted to be present if a company serial number or a drug name was provided. Only interventions and active comparators mentioned in the intervention field were assessed. Other texts in the record were scanned for additional information on mentioned interventions.

Outcome measures

The number of primary and secondary outcomes per record was collected. Each primary and secondary outcome was evaluated for specificity, using a classification system adapted from the system used by Zarin et al in their assessment of quality of outcomes [33]. If a record contained multiple outcomes, all were assessed separately. Outcomes were classified as being a specific measure, a domain, vague, an unexplained abbreviation, or a part of safety monitoring.

Besides assessing the specificity of each outcome, the presence or absence of a time frame was collected for every outcome. Some outcomes assessed the duration of an event, the time to an event or were safety monitoring outcomes. For these outcomes, reporting a time frame is not possible, and the timeframe was therefore denoted as irrelevant. Time frames were denoted to be not meaningful when they did not specify a point in time when the outcome was to be measured.

Only outcomes mentioned in the outcome fields were assessed. Other texts in the record were scanned for additional information on mentioned outcomes.

Pilot

Before starting data extraction a small pilot project was carried out on 25 random records from the ICTRP database to test the assessment framework. Results of the pilot were discussed by DG and RV and the framework was subsequently adapted.

Assessment rules

All records were assessed for eligibility by RV who then extracted and coded the data. During eligibility assessment and data extraction trial records that were not covered by the framework, or where that was ambiguous, were further assessed by DG. Conflicts were resolved by mutual agreement.

A more detailed overview of the rules used in all assessments is provided in supporting information file S1.

Analysis

Odds ratios and Pearson's chi-squares were calculated to assess the relationship between sources of funding and the presence of contact details. For this purpose, partially industry funded trials and non-industry funded trials were grouped together.

Completeness of registration of intervention specifics was analysed according to funding source and trial phase. A binary outcome variable was used that could be incomplete versus complete registration of the intervention. Complete registration entailed the reporting of drug name, dose, duration, frequency and route. Funding source was categorized as in the analysis of contact details. Trial phase was categorized to be Phase 0 or I versus other (some trials were registered as being Phase I & II; these were categorized as other). Regression analysis with robust estimation of variance for clustered samples was used to assess whether these variables influenced completeness of registration of intervention specifics [43].

Quality of registration of primary outcomes was analysed according to funding source, sample size category, trial phase and intervention category. A binary outcome variable was used that could be registration of a specific measure with a meaningful time frame present or for which a time frame was irrelevant, versus any other outcome. Funding source was categorized as in the analysis of contact details. Trial phase was categorized as in the analysis of intervention specifics. Sample size was categorized as being <100 participants versus 100 or more participants. Interventions were categorized to being either drug, biological or vaccine versus other interventions. Regression analysis with robust estimation of variance for clustered samples was used to assess whether these variables influenced the quality of registration of primary outcomes [43].

Statistical analyses were performed using SPSS version 15.0.1 and STATA version 11.1.

Results

There were 754 records in our 5% sample. One record was withdrawn by the registry and could not be assessed. 22 records were excluded from data extraction because the corresponding trials were of an observational or diagnostic accuracy study design or were a treatment protocol for continuation of treatment after inclusion in a study protocol. A total of 731 records were included for data extraction, of which 439 investigated drugs, biologicals or vaccines (Figure 1).

Figure 1

Flowchart.

All information that had to be extracted manually from the registered records was collected between 17 June 2009 and 11 August 2009. Baseline data on registry name, primary sponsor category, intervention type, study phase, study design, randomization status and inclusion criteria for gender of participants are presented in Table 1.

Table 1

General descriptive information.

Category	Number of records	Percentage of records (%)
Registry name 1
ANZCTR	26	3.6
ChiCTR	11	1.5
ClinicalTrials.gov	628	85.9
CTRI	4	0.5
DRKS	2	0.3
IRCT	4	0.5
ISRCTN	39	5.3
NTR	16	2.2
SLCTR	1	0.1
Primary sponsor
Foundation	10	1.4
Government	39	5.3
Industry	246	33.7
University/hospital	398	54.4
Other2	37	5.1
Not specified	1	0.1
Intervention type 3
Drug	385	52.7
Biological/vaccine	82	11.2
Device	49	6.7
Procedure/surgery	69	9.4
Radiation	23	3.1
Behavioural	76	10.4
Genetic4	14	1.9
Dietary supplements	53	7.3
Physical therapy	23	3.1
Organizational	21	2.9
Diagnostic	9	1.2
Other	16	2.2
Study phase 5
0	10	1.4
I	106	14.5
I & II	38	5.2
II	122	16.7
II & III	16	2.2
III	101	13.8
IV	85	11.6
Not specified	253	34.6
Study design
Single arm	162	22.2
Controlled	458	62.7
Crossover	79	10.8
Not specified	32	4.4
Randomization 6
Randomized	518	70.9
Non-randomized	23	3.1
Not specified	29	4.0
Not applicable	161	22.0
Gender
M	39	5.3
F	79	10.8
Both	599	81.9
Not specified	14	1.9
Total per category	731	100

Registry acronyms stand for: Australian New Zealand Clinical Trials Registry (ANZCTR), Chinese Clinical Trial Register (ChiCTR), Clinical Trials Registry - India (CTRI), German Clinical Trials Register (DRKS), Iranian Registry of Clinical Trials (IRCT), International Standard Randomized Controlled Trial Number Register (ISRCTN), The Netherlands National Trial Register (NTR), Sri Lanka Clinical Trials Registry (SLCTR).

Other sponsors consisted of persons that were registered as primary sponsor, non-governmental organizations, collaborative research institutions and clinical research organizations.

Overlap was possible, total in this category is greater than 731.

Genetic interventions consisted of gene transfer therapy and somatic cell transplants.

The presence of study phase in records was analysed separately for trials in drugs, biologicals or vaccines. Of 439 trials researching these types of interventions, study phase was reported in 370 records (84.3%).

For single arm trials, randomization is not applicable. However, one single arm trial was registered as being randomized.

Records were additionally checked for the presence of entries in the fields for recruitment status, date of first enrolment and the public and scientific title. The former three were present in all records, the latter was reported in 700 records (95.8%). Furthermore, information was collected on sample size and age of participants. Sample size was reported in 721 records (98.6%). The median target sample size for these records was 68 [IQR 30–200]. Age of participants was reported in 700 records (95.8%). 89 records (12.2%) mentioned inclusion of participants <18 years of age. Finally, registration dates and dates of first enrolment were compared. The majority of records in our sample did not provide a day for the date of first enrolment but only a month and a year, which limited this analysis to comparing the month in which trials were registered to the month in which the first participant was recruited. The registration date was in a later month than the date of first enrolment in 53.4% of records (median: 10 months). This difference was more than one month in 43.6% of records. Registration date and date of first enrolment were in the same month in 20.7% of records. The registration date was in an earlier month than the date of first enrolment in 26.0% of records (median: 2 months).

Quality of registration of contact information

Overall, 81.0% of records reported a name of a contact person (n = 592). 59.4% of records provided an email address (n = 434) and 64.2% of records a telephone number (n = 469). 68.7% of records provided either an email address or a telephone number (Table 2).

Table 2

Presence of contact details by funding source.

		Name	Email	Telephone nr.	Email or tel. nr.
Industry (N = 246)	N	132	96	115	139
	%	53.7	39.0	46.7	56.5
Partially industry (N = 76)	N	74	48	50	50
	%	97.4	63.2	65.8	65.8
Non-industry (N = 408)	N	385	289	303	312
	%	94.4	70.8	74.3	76.5
Overall (N = 731) 1	N	592	434	469	502
	%	81.0	59.4	64.2	68.7

For one trial, no primary sponsor was registered.

Industry funded trials were less likely to mention a name in their registered records than partially industry funded trials or non-industry funded trials (OR = 15.9, 95% CI: 9.9–25.5, p<0.001). Industry funded trials were also less likely to mention an email address in their registered records (OR = 3.6, 95% CI: 2.6–4.9, p<0.001) or to mention a telephone number (OR = 3.1, 95% CI: 2.2–4.2, p<0.001). There were no differences in the presence of contact details between partially industry funded trials and non-industry funded trials (p = 0.28, p = 0.18 and p = 0.13 respectively).

Quality of registration of interventions involving drugs, biological or vaccines

There were 439 records of trials that investigated drugs, biologicals or vaccines. Intervention specifics were recorded for 726 experimental or active comparator arms. A name was reported in 713 arms (98.2%). For dose, duration of the intervention, frequency of administration and route of administration, information was present in 512 (70.5%), 508 (70.0%), 550 (75.8%) and 535 (73.7%) arms respectively. 321 arms (44.2%) were complete in registering intervention specifics.

Multiple logistic regression analysis showed that funding source was not a significant predictor of completeness of registration of intervention specifics (p = 0.39), but that study phase was (p<0.001). Additional univariate analyses were performed, which confirmed that funding source was not a significant predictor of intervention registration quality (p = 0.34) and that trials that were Phase 0 or I were more likely to be complete in reporting intervention specifics than other trials (OR = 2.7, 95% CI: 1.5–4.9, p<0.001).

Quality of registration of outcome measures

The 731 included trial records reported 1271 primary outcomes and 2372 secondary outcomes. 66.2% of records reported one primary outcome, 17.5% reported two, 6.0% reported three and 9.2% reported four or more. The maximum number of primary outcomes reported in one record was 24. Eight records (1.1%) reported no primary outcome at all, and 149 records reported no secondary outcomes (20.4%).

The degree of specificity of reported outcomes was assessed (Table 3). 38.2% of primary outcomes, 33.2% of secondary outcomes and 34.9% of primary and secondary outcomes combined were specific measures, for which a time frame was irrelevant or for which a meaningful time frame was present.

Table 3

Degree of specificity of primary and secondary outcomes.

	Primary outcomes (N = 1271)	Secondary outcomes (N = 2372)	Primary and secondary outcomes (N = 3643)
Classification				Examples
Specific measure (%)	47.1	42.5	44.1	All-cause mortality, quality of life by SF-36, pulmonary functioning by FEV-1
Domain (%)	36.7	38.7	38.0	Freedom from progression, quality of life, pulmonary functioning
Vague (%)	5.4	6.5	6.1	Efficacy, symptoms, laboratory parameters
Unexplained abbreviation (%)	3.5	4.6	4.2	Any unexplained abbreviation
Safety monitoring (%)	7.3	7.8	7.6	Adverse event monitoring, drug toxicities, complications
Time
Time present (%)	65.9	62.7	63.8	Mortality at one year
Time present, not meaningful (%)	10.8	13.7	12.7	ECG twice a year, social impact throughout study
Time absent (%)	7.7	9.2	8.7
Time irrelevant (%)	15.6	14.4	14.8	Duration of stay in ICU, time to progression

Multiple logistic regression analysis showed that funding source (p = 0.30), target sample size (p = 0.93), intervention category (p = 0.39) and study phase (p = 0.70) were all not significant as predictors for the reporting of specific measures with a meaningful time frame present (or irrelevant). Additional univariate analyses were performed, which confirmed that none of the dependent variables were significant predictors of outcome registration quality (p = 0.24, p = 0.33, p = 0.49 and p = 0.46 respectively).

Discussion

To be able to fulfil the promise of clinical trials registration, it is of paramount importance that registration is comprehensive, complete and accurate. That is, that all trials in all countries are registered, that meaningful data are registered for every item in the WHO Trial Registration Data Set [25], and that registered data are correct and up-to-date. This study confirms the findings of similar studies that have shown that the quality of registered trial data is a significant problem and that it needs to be improved.

There should be clearly assigned responsibility to a named Principal Investigator in all registered records of clinical trials to facilitate investigator accountability and transparency [29]. By Principal Investigator (PI) we mean “the individual who is responsible and accountable for conducting the clinical trial” [25]. In 2008 Sekeres et al examined 1388 clinical trial register entries and found that all 440 registered trial records with recruitment status “in progress” that were either non- or partially funded by industry named the scientific leadership of the trial, compared with 49% (111/226) of those funded by industry; findings confirmed by the current study.

There are well-established ethical, scientific and legal obligations associated with being a clinical trial investigator. International research standards such as the International Conference on Harmonization (ICH) Topic E6 require investigators to have appropriate qualifications and experience, to ensure compliance with the trial protocol, to obtain and document informed consent, and to be responsible for the medical care of trial subjects and for the integrity of the research data and results [44]. Although key international standards (ICH E6, Declaration of Helsinki) do not specifically require trials to have named scientific leadership, it seems reasonable to ask for them to be publicly named and accountable for the trials onto which they recruit participants, considering their responsibilities both to the participants they recruit, and to future patients who may benefit from the results of the study [6]. Similarly, it is important that investigators be contactable should the publication of the results of their research be delayed (or not achieved, despite increasing public and legal pressure to do so), to enable the results of studies to be made available to investigators of similar studies and meta-analysts [45]. The PI is also ultimately responsible for registering the trial and hence for the quality of the registered data. Some of the problems identified with the quality of registered data may therefore be solved by having a named PI in the registry record.

Arguably the two most important pieces of information about a clinical trial that need to be registered are the description of the interventions being compared, and the outcomes upon which any conclusion about the safety and effectiveness of the interventions will be made. As demonstrated by this and previous studies, the quality of this information, as it has been registered to date, has been poor [27], [30]–[36].

In 2005, nine to ten percent of registered trial records on ClinicalTrials.gov provided an incomplete or nonspecific description of the intervention name [30], [33]. Although subsequent studies suggest that this has improved to less than two or three percent [33], [34], [46], more information is required about the intervention than the name. There should also be a description that is detailed enough for it to be possible to distinguish between the arms of the study. For trials of drugs, biologicals and vaccines this means information on the dose, frequency, route of administration and duration of treatment [25], [47]. In the current study less than half of the intervention arms where this information was relevant provided it. That there is room for improvement is confirmed by the fact that records of some trials (Phase 0 and I) describe interventions in greater detail, perhaps due to a greater focus on the specifics of the intervention in these trials.

Similarly, more is required when registering trial outcomes than the name. To be complete the record should contain the name of the outcome, information on the instrument that is being used to measure it (when applicable), and the time points at which it will be measured. Primary outcomes with a specific measure and a meaningful time frame were registered in only 31% of records evaluated by Zarin et al in 2005, and in 38% of records in the current study [33]. Given the critical importance of the primary outcome to the scientific integrity of the study it is of enormous concern that this key information is still not being made public in a way that is meaningful or informative. Since the primary outcome is the one that the study should be designed to evaluate, and hence used to calculate the sample size, it is also concerning that so many trials claim to have multiple primary outcomes with almost one in ten trials claiming four or more. The combined problem of multiple primary outcomes, lack of specification of the instrument being used to measure the outcome, and non-reporting of time frames leaves the door open for fishing expeditions and will not solve the problem of selective reporting bias.

The trial records in this study were registered between June 2008 and June 2009 on any one of the nine registries that provided data to the ICTRP Search Portal, including ClinicalTrials.gov. Although the latter is the most established and clearly the largest registry, 14% of the records in this study were provided by the other registries. As more countries seek to improve the transparency of clinical trial research involving nationals of that country, to be more accountable to the individuals who consent to participate in clinical research, to better oversee and monitor that research, and to make information accessible in the languages spoken by the nationals of each country, it is inevitable that the number of trial registries will increase [48]. Since the start of this study, the number of registries that provide data to the ICTRP has already risen from nine to twelve.

Prospective registration is defined by the ICMJE and WHO as registration of a clinical trial before recruitment of the first participant. Even allowing amnesty for trials registered in compliance with national laws (such as the Food and Drug Administration Amendments Act in the US), more than 40% of the records in our sample were registered one month or more after recruitment of the first participant, with a median time to registration of 10 months for retrospectively registered trials. Data from the Australian New Zealand Clinical Trials Registry (ANZCTR) confirm these findings and show no improvement for 2010 (Personal communication, L. Askie, 29 June 2010). This delay is clearly not acceptable, particularly as many trials could feasibly complete recruitment in such a time frame and could potentially then retrospectively register the trial in a way that could favour a particular result. It is for this reason that some registries refuse to retrospectively register trials. Adoption and enforcement of the ICMJE policy on prospective registration by more journal editors could make an important difference, and some key journals are playing a leading role in this regard [49]. By emphasizing the importance of informative entries and specifically underlining the consequences of omitting information, journal editors could contribute even more to the attainment of high quality registration.

It is important to note that any study of the quality of registered records is not the same as a study of the quality of the design or conduct of clinical trials. It is just as possible that the trials that have not been adequately registered are of high quality as low quality. However, just as the quality of a trial and its results can usually only be assessed against the quality of the publication reporting those results, in the absence of the complete protocol we have no other choice than to judge the quality of a trial's design against the information entered into a trial registry.

It has now been five years since the ICMJE and the World Health Assembly put their crucial support behind the need to prospectively register clinical trials. In the time that has passed the number of registered trials has increased from less than 10,000 to more than 120,000, but a significant proportion of the information that has been registered remains deficient. In an attempt to improve the quality of registered data the WHO ICTRP has introduced a number of measures. One is to improve the explanatory text for the Trial Registration Data Set to make the requirements for registration clearer, particularly for contact, intervention and outcome information [25]. Another is the establishment of International Standards for Clinical Trial Registries, the aim of which is to improve the quality of registered data by establishing a clear minimum requirement for quality control processes performed and data recording practices used by individual clinical trial registries. It is our intention to repeat this study following the introduction of the standards and continue to monitor the quality of registered data. If successful, these measures could improve the meaningfulness and usefulness of registered data, and hence ensure its scientific, ethical and moral integrity.

Assessment rules. These contain a more detailed explanation of the methods used to assess the quality of registered records on the ICTRP database.

(0.07 MB DOC)

Click here for additional data file.