Clinical trial registration, and publication in acupuncture studies: A systematic review.

BACKGROUND: To research 1) how many acupuncture clinical trials are registered with the WHO International Clinical Trial Registry Platform (ICTRP) and what patterns they demonstrate, 2) publication of the articles of acupuncture clinical trials which were registered with ICTRP.
METHODS: The search strategy using the ICTRP: Intervention: acupuncture; Recruitment status: All; Date of registration; from 1 Jan 1990 to 31 Dec 2018. We searched the indexed articles in PubMed using trial IDs on 25 Feb 2019. When the paper was published, we indicated the number of weeks from the date of registration with ICTRP to the date of publication in order to define time till the publication. We divided the whole period we analyzed into 6 periods of every 3 years and measured the proportion of publication and the time from the date of registration of each trial till its publication in each period by the Kaplan-Meier method.
RESULTS: Forty-three countries/areas conducted at least one acupuncture clinical trial. The total number of registrations was 1758. China, the USA, and the Republic of Korea accounted for 61% of those registrations. The proportion of publication was 178/1758 10% for the fully published papers and 141/1758 8% for the protocol papers.
CONCLUSIONS: The substantial increase of registrations by China, the Republic of Korea, Iran, Brazil, Japan was observed which may be attributed to improved awareness of the CONSORT statement. However, the fully published papers rate is low at 10%. The publication of results of acupuncture clinical trials should also be rigorously mandated.

Introduction

It is crucial to search, screen, and critically appraise each article and/or document by the methods used in systematic reviews (SRs) to answer research questions. However, these searches naturally do not include unpublished research results. Authors tend to submit papers which resulted in positive rather than negative findings, and editors tend to accept such positive papers rather than negative papers. In other words, only positive results are likely to be obtained in search, and incorrect conclusions are reached. This is a manifestation of publication bias, which is defined as "the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings" by Dickers in 1990. Sterling, T.D. first suggested the existence of publication bias in 1959. Furthermore, he reported in 1995 that there had been no change in the practices leading to publication bias for 30 years.

In the acupuncture research field, Vickers first pointed out in 1998 the tendency to publish papers that showed positive results in some specific countries, namely, China, Japan, Russia/USSR, and Taiwan. Additionally, Tang demonstrated the existence of publication bias in 49 randomized controlled trials of acupuncture on stroke using funnel plots. In this way, the publication bias has been noted in the acupuncture clinical trial literature in the late 1990s, i.e., in the early days of when the publication bias issue began to attract attention in the medical field.

Awareness of publication bias and the need for trial registry have been advocated by three international organizations. Concerns of the publication bias issue were first introduced in item 19 of the Declaration of Helsinki (2000, Edinburgh). The 2008 Seoul version followed with much elaboration. The current version (2013, Fortaleza, Brazil) has items 35 and 36, titled “Research Registration and Publication and Dissemination of Results.” In September 2004, the members of the International Committee of Medical Journal Editors (ICMJE) published a joint editorial aimed at promoting registration of all clinical trials. This movement was followed by the WHO Technical Consultation on Clinical Trial Registration Standards Meeting in 2005, and the WHO International Clinical Trial Registry Platform (ICTRP) was launched on 19 May 2006, and its website formally launched on 4 May 2007.

Currently, the ICTRP is the platform integrating information from primary registries in 17 countries/areas. This is free to access, and anyone can collect information on clinical trials all over the world. By utilizing such a system of clinical trial registration, a reduction in publication bias is expected.

The authors have two research questions. First, what is the state of the numbers and patterns of the registered acupuncture clinical trials? A survey on the registration of acupuncture clinical trials using the ICTRP was reported in 2013 and using the individual registries in the ICTRP in 2014. However, the number and proportion of countries where trials have been conducted have not been reported. Second, what is the current status of the publication of papers from the registered acupuncture clinical trials within the ICTRP; that is, what is the current state of publication bias? There is no research on the publication of acupuncture clinical trials registered with the ICTRP.

Methods

State of registration with ICTRP

Search strategies

The search was performed on 25 Feb 2019. The search strategy used the Advanced Search function of the ICTRP: Intervention: acupuncture; Recruitment status: All; Date of registration: from 1 Jan 1990 to 31 Dec 2018. All 199 countries/areas in the ICTRP search portal were searched as recruiting countries.

Eligibility criteria

Acupuncture clinical trials were defined as trials using acupuncture as an intervention. We included all clinical trials that used acupuncture as an intervention. All acupuncture clinical trials were included regardless of progress.

Data analysis

The clinical trials for each of the 199 countries/areas for each year and the grand total of all years were summed up. The cumulative absolute number of registered trials and its proportion in the top 10 countries/areas were calculated for each year. We divided the dates of registration into 6 periods of every 3 years, i.e., Period 1: before 2004, Period 2: 2004–2006, Period 3: 2007–2009, Period 4: 2010–2012, Period 5: 2013–2015, and Period 6: 2016–2018, and calculated the top 3 countries/areas for the center year in each period.

Proportion of trials with publications and time until publication

Publications based on clinical trials on acupuncture extracted from the ICTRP were searched for. We searched papers indexed in PubMed by using their trial IDs in the ICTRP on 18 Mar 2019.

The definition of a publication included either a fully published paper containing the results of a trial or a protocol papers with only a methodology. If the paper was published in PubMed, there was no restriction on the language. When multiple papers were found with the same trial ID, the earliest published paper was included. When both protocol papers and fully published papers were found, the protocol papers were excluded, and the full-published papers were included.

We calculated the number of weeks from the date of registration with the ICTRP to the date of publication to define the time until publication. We used the number of years to indicate the time until publication in the table. The time from the registration of each trial until its publication was analyzed from Period 1 to Period 5. Because it has been 18 years since ClinicalTrials.gov was established and the clinical trials registration launched in the United States in 2000, each country’s registration rate may differ as the patterns of utilizing these resources have been changing. Periods 5 and 6 have a limitations in their data, such that there are 5 years for potential publication of a trial registered in 2013, 4 years since 2014, 3 years since 2015, 2 years since 2016, and 1 year since 2017; however, Period 5 is presented in the figure to analyze the recent trend. We estimated the proportion of publications and the time until publication from with each period within 5 years after registration by the Kaplan-Meier method. The Kaplan-Meier survival curves are presented by using GraphPad Prism 6.0.

Results

State of registration with the ICTRP

The number of registered clinical trials and registration patterns for each of the top 10 countries/areas arranged by total number through the end of 2018 is listed in Table 1. In 2004, the total registered number was 59, i.e., clinical trial in the USA 37, the UK 19, Australia 1, Germany 1, and others 1 registered. The number in each registry was as follows: ClinicalTrials.gov 37, ISRCTN.org 21, and ANZCTR 1.

Table 1

Top ten countries/area for the number of registered acupuncture trials with the WHO-ICTRP.

No.	Countries/area	2004	2005	2006	2007	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017	2018	Total
1	China	0	0	1	9	22	12	22	31	34	47	54	81	88	124	113	638
2	USA	37	15	15	16	18	18	13	20	15	17	20	17	20	15	13	269
3	Korea, Republic of	0	0	2	1	2	6	7	10	9	25	12	28	24	26	15	167
4	Iran	0	0	0	0	0	1	2	9	10	6	9	17	6	22	20	102
5	Brazil	0	0	3	0	3	2	4	8	5	5	3	7	8	13	19	80
6	Japan	0	0	0	1	3	3	3	5	2	5	4	13	12	10	6	67
7	UK	19	4	10	3	2	4	2	1	3	0	4	2	1	1	2	58
8	Taiwan	0	2	2	1	2	2	3	1	8	4	5	4	6	10	7	57
9	Germany	1	5	2	3	4	3	3	2	4	2	7	6	4	6	4	56
10	Australia	1	4	0	3	4	11	9	5	6	2	1	4	1	2	2	55
11	others	1	5	15	8	13	12	16	12	12	14	19	25	12	21	24	209
	Total	59	35	50	45	73	74	84	104	108	127	138	204	182	250	225	1758

The cumulative number of registered acupuncture clinical trials for each country/area is shown in Fig. 1A. The cumulative total number of registered acupuncture clinical trials among these countries/areas was 1758. It shows that in 43 countries/areas among the 199 countries/areas included in the ICTRP, at least one acupuncture clinical trial was conducted. The top 10 countries/areas were China 638, the USA 269, the Republic of Korea ROK 167, Iran 102, Brazil 80, Japan 67, the UK 58, Taiwan 57, Germany 56, and Australia 55 Supplement 1.

Fig. 1

(A) Cumulative number and (B) Cumulative proportion of registered acupuncture clinical trials with the WHO-ICTRP.

The cumulative proportion of registered acupuncture clinical trials from 2004 to 2018 is shown in Fig. 1B. The USA had the most registrations until 2011; however, the USA has had the second most registrations since 2014. The UK had the second most registrations in 2005, the third most in 2008, and has been the fourth or below since 2011. China became the country with the second most registrations in 2008 and 2011 and has had the most since 2014. The ROK had the third most registrations in 2014 and 2017.

Since 2005, the USA has consistently registered between 13 and 20 clinical trials each year. No significant changes in patterns have been observed for Australia, the UK, and Germany. On the other hand, there was a sharp increase in the initiation of clinical trial registrations observed from 2008 to 2012 in Asian countries/areas such as China, the ROK, Iran, Japan, Taiwan, and Brazil. The relative proportion of studies from the USA, the UK, Germany, and Australia has been constantly decreasing.

Current status of publication in PubMed

Fig. 2 shows the Kaplan-Meier survival curve of the proportion of registered trials with a publication and time until publication for all 1758 acupuncture clinical trials that were registered from 2004 to the end of 2018. This figure shows both fully published papers and protocol publications and includes the results with 95% confidential intervals.

Fig. 2

Cumulative percentage of articles indexed in PubMed in the 5 years after registration in the primary registry or data provider.

A total of 178 10% acupuncture clinical trials among the 1758 trials that were registered by the end of 2018 were published as an article and were found in PubMed. A total of 141 8% acupuncture clinical trials were published as a protocol.

We divided the proportion of registered trials with a publication and time until publication was divided into 3-year periods.

Fig. 3 shows the Kaplan-Meier survival curve, which is divided into 6 periods. The proportion of trials with a fully published paper increased year by year and reached 21% during Period 3 and Period 4. During Period 3, the number of fully published papers rapidly increased at approximately year 3 and showed a tendency to reach a plateau after that. On the other hand, a constant increase was observed during Period 4.

Fig. 3

Cumulative percentage of articles indexed in PubMed in the 5 years after registration in the primary registry or data provider for each period.

The publication of protocol papers emerged during Period 2 and rapidly increased to 8 in Period 3, to 21 in Period 4 and to 74 in Period 5.

The median (interquartile range) time until full publication was as follows:

Period 1 (before 2004): 5 papers/41 trials (12%), 175 (140–243) weeks;

Period 2 (2004–2006): 11 papers/104 trials (11%), 162 (137–227) weeks;

Period 3 (2007–2009): 40 papers/192 trials (21%), 167 (115–228) weeks;

Period 4 (2010–2012): 61 papers/29 trials (21%), 149 (59–197) weeks; and

Period 5 (2013–2015): 38 papers/469 trials (8%), 138 (95–170) weeks.

Discussion

The aim of this study is to research the current state of acupuncture clinical trial registration with the WHO (ICTRP) and the current status of the publication of articles based on acupuncture clinical trials that were registered with the ICTRP. The number of clinical trials for acupuncture registered by 31 March 2018 was 1758, with a sharp increase in registrations in Asia from 2008 to 2012. The top 3 countries/areas, China, the USA and the ROK, accounted for approximately 60% of the total. Publications based on the registered clinical trials were searched for by trial ID, and 178 (10%) were found as fully published papers in PubMed. Furthermore, 141 (8%) were published as a protocol article in PubMed. The number of clinical trials and time until publication after registration were investigated across 6 periods of time. The trends observed in recent years showed an increase in the number of publications and a decrease in the time until publication. This study revealed the state of acupuncture clinical trial registration with the ICTRP, the publication rate of articles, and the publication speed in each period within 5 years of registration.

Acupuncture clinical trial registration with the WHO-ICTRP

The trial registration system plays a role in preventing publication bias by publishing information about prospective clinical trials. Using the ICTRP, Gu investigated the registration state of acupuncture clinical trials and reported that 740 clinical trials were registered during the period from 1999 through 2012. In our study, we reported that 1758 acupuncture clinical trials were registered by 2018, and the number of registered acupuncture clinical trials drastically increased in the 5 years since 2012. The increase in Asian countries, especially in China and the ROK, is outstanding. This tendency have been observed not only in the acupuncture field but also in all fields with registered clinical trials in the ICTRP. Before the ICTRP was established, the USA had been running Clinicaltrials.gov since 2000, and it was mandatory to register clinical trials.

Nakamura reported in 2009 that mainly the USA and European countries registered with Clinicaltrials.gov, but few Asian countries did. The idea of a clinical trials registry was not well known among Asian countries before the ICTRP was established. There were registry systems equivalent to Clinicaltrials.gov, such as the UMIN-CTR in Japan (2005), Chinese Clinical Trial Registry (ChiCTR) in China (2005), and Clinical Research Information Service (CRiS) in the ROK (2010); however, there is a possibility that those systems were not well known in the early years after their launch. Subsequently, the ICTRP has gradually become better recognized by the publication of CONSORT for abstract in 2008 and CONSORT 2010, those request registration number and name of trial register. The number of registered acupuncture clinical trials in Asia has increased, and now 2 of the top 3 countries are Asian countries. Henceforth, the publication bias in Asia, which was pointed out by Vickers, will decrease. It is expected that systematic reviews draw conclusions without the influence of publication bias.

Publication status in PubMed

Ten percent of all registered acupuncture clinical trials were indexed in PubMed within 5 years of registration. Both the fully published papers and the protocol papers showed the same tendency regarding rate of publication and increased year by year, and the time until publication shortened. Particularly in Period 3 among the 6 periods, the publication of fully published papers rapidly increased in the 3rd and 4th years.

Ross reported the proportion of clinical trials that were registered in Clinicaltrials.gov and published on Medline within 2 years from the completion date of the trial. The percentage was 61% among those who were completed before 2004, 52% among those who were completed in 2004, and 42% among those who were completed in 2005. He also reported 294 (46%) papers among 635 clinical trials that were completed by 31 December 2008 were indexed in Medline within 30 months. The WHO mentions on their website that the publication rate of the results is 50%. As this shows, there were many registered clinical trials that did not publish the results. However, the percentage of acupuncture clinical trials that published the results has been much lower.

In the USA, which is one of the leading countries with ICTRP registration, the Food and Drug Administration Amendments Act (FDAAA) of 2007 mandated that the results of clinical trials be published. On the other hand, regulatory agencies in leading Asian countries/areas such as China, the ROK, Iran, Japan, and Taiwan have not mandated that results get published, and this may have an impact on papers being unpublished or delayed in publication. It is crucial to have papers published to prevent publication bias. Furthermore, Su investigated the registration information of the acupuncture randomized controlled trials that were registered in the ICTRP and the published paper contents and indicated the insufficient reporting and the significant selectivity in reporting of outcomes. Henceforth, the publication of research results from acupuncture clinical trials should be rigorously evaluated.

Limitations

Abstracts without the trial ID may have not been found even when these articles were indexed in PubMed. We hope the trial IDs will be mentioned in abstracts to comply with CONSORT for abstract guidelines and, as noted in CONSORT2010 Ib, to realize the reliable identification of the trials. Additionally, correspondence of the clinical trial registry information, which can be obtained by text mining of the investigator’s names, trial titles, and disease names, and the published papers should be confirmed.

It is usual to research the publication status of registered trials with calculations based on the completion date of the study, and we think it is appropriate to do so.20, 21 A shift in the time until publication occurs if the calculation is based on the registration date rather than the completion date. However, much of the registration information lacked input to the primary registry or data provider; therefore, it was difficult to identify the completion date. The completion date was added to the WHO Trial Registration Data Set (Version 1.3) as a new item in 2017. This publicizes the last date when the last data were collected for a clinical study (last subject, last visit). Henceforward, these analyses using the completion date will become possible.

As an initial step in the research, the search was confined to the PubMed database only. However, papers that are not indexed in PubMed cannot be extracted. Henceforth, it is necessary to search Google Scholar and Embase as well as each country’s bibliographic database, for instance, the China National Knowledge Infrastructure (CNKI), Chinese Biomedical Literature (CBP), and Chinese VIP Information in China; the KoreaMed Korean Studies Information Service System, Korea Citation Index, and Korea Science Citation Index in the ROK; and the Ichushi Web in Japan. In addition, investigators on trials in which publications could not be found on PubMed would have to be manually contacted by e-mail or phone call.

In conclusion, the number of registrations of acupuncture clinical trials with the ICTRP is increasing. In the publication status survey using PubMed, the overall publication rate was low, but when assessed over periods of time, an increase in the publication rate and a shortened time to publication was revealed. Since clinical trial registration is expected to reduce publication bias, being compliant with clinical trial registration is essential in the future. For further research, it is necessary to investigate the publication status using multiple databases.

Authors’ contributions

KT and YM contributed to conceptualization. YM contributed data collection and wrote the original draft of the manuscript. YM and NTW contributed to formal analysis and interpretation of data and assisted in the preparation of the manuscript. KT and TS critically reviewed & edited the manuscript. All authors approved the final version of the manuscript, and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflict of interest

There are no conflicts of interest to report.

Funding

There is no external funding to report.

Ethical statement

No ethical approval was required for this manuscript.

Data availability

Data will be made available on request