Publication status of completed registered studies in paediatric appendicitis: a cross-sectional analysis.

OBJECTIVE: Appendicitis is considered the most frequent surgical emergency in children. While the management of paediatric appendicitis is evolving, the precise amount of unpublished completed trials, potentially introducing bias into meta-analyses, is unknown. Controversial issues include the appropriate choice of surgical procedures, criteria for diagnosis of appendicitis, the role of antibiotic treatment and pain management. Selective reporting may introduce bias into evidence-based clinical decision-making, and the current, precise extent of unpublished results in paediatric appendicitis is unknown. We therefore assessed the publication status of completed clinical studies involving children registered on ClinicalTrials.gov.
DESIGN: Cross sectional analysis. STrengthening the Reporting of OBservational studies in Epidemiology criteria were applied for design and analysis. SETTING AND PARTICIPANTS: ClinicalTrials.gov was queried for completed studies which were matched to publications on ClinicalTrials.gov, PubMed or Google Scholar. If no publication could be identified, principal investigators were contacted. INTERVENTIONS/EXPOSURE: Observational analysis. PRIMARY AND SECONDARY OUTCOME MEASURES: The proportion of published and unpublished studies was calculated. Subgroup analysis included studies on surgical procedures, diagnosis, antibiotic treatment and pain management.
RESULTS: Out of n=52 completed clinical studies involving children with appendicitis, n=33 (63%) were published and n=19 (37%) were unpublished. Eighty-three per cent (n=43/52) of clinical trials assessed the above-listed controversial issues. Diagnostic studies were most rigorously published (91% of trials reported), data on surgical procedures, antibiotic and pain management were less transparent. Sixty-six per cent of interventional studies and 60% of randomised studies were published. Median time-to-publication, for example, the delay between completion of the trial until public availability of the results was 24 (IQR 12-36), range 2-92 months.
CONCLUSION: Despite the importance of appendicitis in clinical practice for the paediatric surgeon, there remains scientific uncertainty due to unpublished clinical trial results with room for improvement in the future. These data are helpful in framing the shifting paradigms in paediatric appendicitis because it adds transparency to the debate. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study analysing reporting transparency in clinical research of paediatric appendicitis.

Clinical trial registration databases other than ClinicalTrials.gov were not analysed.

Unregistered clinical studies were not captured by the present study method.

Introduction

Appendicitis is considered the most frequent surgical emergency in children with an incidence of 86 cases per 100 000 people.1 2 Efforts are increasing to standardise diagnosis and management, nevertheless controversies continue to exist and challenges remain.3 Although a variety of scoring systems have been developed,4–6 there is still no unequivocal consensus on clinical, laboratory and imaging criteria for diagnosing appendicitis. After the diagnosis is made, usually surgical intervention follows. Recent studies have demonstrated that non-operative management for carefully selected children with acute appendicitis is possible.7–9 Different surgical approaches exist: over the years, laparoscopic appendectomies have widely replaced open traditional procedure.10 Many surveys compare different minimally invasive techniques finding no relevant differences in outcome between three-port or single-incision appendectomies.11 12Optimisation of pain management in children with appendicitis has recently become the centre of several investigations.13 14

Selective reporting of clinical trial results introduces bias into evidence-based clinical decision-making.15–17 The precise extent of bias in paediatric appendicitis is unknown. We therefore assessed the public availability of study results of completed clinical studies involving children with appendicitis registered in the major clinical trial database. We drew particular attention on studies focusing on important controversial issues, that is, surgical procedures, diagnosis of appendicitis, antibiotic treatment and pain management. The aim of this study is to render the current publication status of completed, registered, clinical trials in appendicitis involving children transparent.

Methods

We determined the proportion of published and unpublished results of studies on paediatric appendicitis that were registered and reported as ‘completed’ on ClinicalTrials.gov.

ClinicalTrials.gov database query www.clinicaltrials.gov was accessed through the internet. The detailed search criteria were: keyword ‘appendicitis’ and ClinicalTrials.gov query selection parameters ‘completed studies’ and ‘child’. Data were downloaded.

Search for publications of completed studies

ClinicalTrials.gov, PubMed and Google Scholar were searched for publications related to the completed registered paediatric studies on appendicitis identified as described above. Keywords for literature research included the NCT number, study title as listed in ClinicalTrials.gov or semantic keywords generated from study title as listed in ClinicalTrials.gov, place of study or principal investigator. If no publication could be found on PubMed or Google Scholar as a next step principal investigators or sponsors were contacted directly and asked to provide the publication of the study to make sure that no published study is missed (see the flow sheet in figure 1). All investigators of unpublished studies (n=19) listed in ClinicalTrials.gov were contacted by email, n=5 replied, none provided published study results. Close of database for the search in each repository was 3 May 2016.

Figure 1

Study flow diagram: identification of published and unpublished clinical trials registered on ClinicalTrials.gov involving children with appendicitis.

Statistical analysis

The following continuous or categorical variables were considered: NCT number, study title, gender and age of participants, study type, study design, condition, intervention, recruitment status, completion date, availability of study results, publication date, sponsor/collaborator and country of sponsor/collaborator. The purpose of clinical studies or the intervention was analysed and trials were categorised into five groups according to their major research topic: (1) surgical procedures, (2) diagnostic criteria, (3) antibiotic treatment, (4) pain management and (5) other.

Time-to-publication was calculated as the difference in months between publication date and completion date.

Standard methods of descriptive statistics were applied. Missing data were not imputed. All calculations were performed with SAS Enterprise Guide V.5.1.

STrengthening the Reporting of OBservational studies in Epidemiology criteria were applied for design and analysis of this cross sectional study.18 Close of database was 3 May 2016. A study flow sheet is provided in figure 1.

Patient involvement

Patients were not involved in this research project.

Results

Publication status of studies and trial participants

Overall, we identified n=52 completed clinical studies on appendicitis involving children registered on ClinicalTrials.gov. Out of those, n=33 (63%) studies were published and n=19 (37%) studies were unpublished (figure 2, table 1, tables 2A,B). Published trials contained data from n=11 997 study participants. The unpublished trials embody information from n=98 673 patients (figure 3). Median size of published trials was 150 (IQR 73–360), range 21–4000 patients whereas median size of unpublished studies was n=184 (IQR 82–500), range 2–40 000 participants. Three unpublished studies were outliers and had 15 000 (one study) and 40 000 participants each (two studies). Year of completion ranges from 2005 to 2016. Out of n=23 observational studies, n=14 (39%) were published and out of n=29 interventional studies, n=19 (66%) were published. n=25 studies were randomised trials. Out of those, n=15 (60%) were published and n=10 (40%) remained unpublished. The published randomised studies contained data of n=2461 patients, the unpublished studies recruited 1411 patients. All studies involved both genders. The difference in publication rates by country of sponsor/collaborator is shown in table 3.

Figure 2

Published and unpublished paediatric appendicitis studies: number of trials by year of completion.

Table 1

Publication status of studies registered as completed on ClinicalTrials.gov involving children with appendicitis

Issue	Overall number of studies	Number and percentage of published studies	Number and percentage of published randomised studies	Number of patients enrolled in unpublished studies	Number of patients enrolled in unpublished randomised studies
Surgical procedure	16	9 (56%)	5 (56%)	1479	786
Diagnostic criteria	11	10 (91%)	2 (100%)	500	0
Antibiotic treatment	11	7 (64%)	4 (57%)	15 275*	275
Pain management	5	2 (40%)	2 (66%)	1119	50
Other	9	5 (56%)	2 (59%)	80 300†	300

Outliers:

*Study NCT02311452 was registered as completed on ClinicalTrials.gov and having enrolled 15 000 patients.

†Studies NCT02018016 and NCT02017951 were reported as completed and having enrolled 40 000 patients each.

Table 2A

Characteristics of unpublished clinical trials (n=19)

Study identifier	Study issue	Study design	Enrolment	Completion date
NCT01054417	Surgical procedure	Observational	142	July 2011
NCT01678365	Antibiotic treatment	Interventional	43	October 2009
NCT01115153	Antibiotic treatment	Interventional	150	February 2010
NCT02730585	Diagnostic criteria	Observational	500	December 2013
NCT01067937	Pain management	Interventional	891	April 2011
NCT02724410	Antibiotic treatment	Interventional	82	November 2013
NCT02673528	Surgical procedure	Observational	451	January 2016
NCT01967745	Surgical procedure	Observational	100	September 2013
NCT02625987	Surgical procedure	Interventional	200	September 2015
NCT02580487	Pain management	Observational	178	October 2015
NCT00554008	Surgical procedure	Interventional	400	January 2011
NCT02311452	Antibiotic treatment	Observational	15 000	Not specified
NCT02352519	Pain management	Interventional	50	June 2015
NCT02714023	Other	Interventional	240	September 2015
NCT01424631	Surgical procedure	Interventional	2	Not specified
NCT02018016	Other	Observational	40 000	August 2013
NCT02017951	Other	Observational	40 000	August 2013
NCT02687217	Other	Interventional	60	May 2013
NCT01515293	Surgical procedure	Interventional	184	Not specified

Close of database 3 May 2016.

Figure 3

Published and unpublished paediatric appendicitis studies: number of patients by year of completion (log scale). Three studies were outliers (see tables 1 and 2A).

Table 2B

Characteristics of published clinical trials (n=33)

Study identifier	Study issue	Study design	Enrolment	Completion date
NCT01718171	Other	Observational	183	July 2012
NCT00528138	Diagnostic criteria	Observational	132	September 2008
NCT01697059	Antibiotic treatment	Interventional	73	May 2015
NCT00716703	Diagnostic criteria	Interventional	250	October 2005
NCT00888888	Surgical procedure	Observational	87	June 2010
NCT02137603	Surgical procedure	Interventional	36	December 2014
NCT01096927	Antibiotic treatment	Interventional	160	February 2011
NCT00435032	Antibiotic treatment	Interventional	128	Not specified
NCT00630071	Diagnostic criteria	Observational	103	August 2008
NCT00723788	Diagnostic criteria	Interventional	21	April 2010
NCT00854815	Diagnostic criteria	Interventional	220	June 2012
NCT00195923	Antibiotic treatment	Observational	100	January 2007
NCT00462020	Antibiotic treatment	Interventional	150	November 2008
NCT00783016	Pain management	Interventional	234	May 2011
NCT01698099	Diagnostic criteria	Observational	500	September 2012
NCT00677417	Diagnostic criteria	Observational	538	May 2008
NCT00477061	Pain management	Interventional	71	March 2005
NCT01652170	Diagnostic criteria	Observational	2201	February 2014
NCT00414375	Antibiotic treatment	Interventional	30	March 2009
NCT00908804	Surgical procedure	Interventional	Not specified	Not specified
NCT02304653	Diagnostic criteria	Observational	226	November 2013
NCT01572558	Antibiotic treatment	Interventional	51	October 2012
NCT00913380	Diagnostic criteria	Interventional	891	April 2011
NCT00981136	Surgical procedure	Interventional	360	December 2011
NCT01002365	Other	Interventional	Not specified	Not specified
NCT01738750	Surgical procedure	Interventional	100	December 2013
NCT00413855	Other	Interventional	40	June 2005
NCT01260064	Surgical procedure	Interventional	150	May 2011
NCT01348464	Surgical procedure	Observational	150	December 2011
NCT01734837	Surgical procedure	Observational	390	August 2013
NCT02047786	Other	Observational	4000	August 2013
NCT00925145	Surgical procedure	Observational	32	December 2010
NCT01657565	Other	Observational	390	January 2011

Close of database 3 May 2016.

Time to public availability of results

Median time-to-publication, that is, the delay from completion of the trial until public availability of the data was 24 (IQR 12–36) range 2–92 months. More recent studies tended to be published faster than older studies (figure 4).

Figure 4

Time-to-publication of paediatric appendicitis studies. ‘FDAAA’ indicates the time line mandated by the Food and Drug Administration Amendments Act of 2007.20

Six studies were completed less than 1 year before close of database. Of those, only one study was published (tables 2A,B). This study, a comparison of surgical versus antibiotic therapy, for appendicitis had positive results and was published within 8 months after completion.

Study sponsors

Three studies were sponsored or cosponsored by the industry. All these studies were published. All other studies were sponsored by academia.

Discussion

In order to render clinical research transparency, the AllTrials initiative (www.alltrials.net) called for registration and publication of all results of all clinical trials. In addition, publication of clinical research data is considered an ethical imperative.19 In 2007, the prospective registration and mandatory publication of applicable clinical trials within 1 year of completion became federal law in the USA with the Food and Drug Administration Amendments Act (FDAAA).20

Our analysis demonstrated that 37% of registered completed studies in appendicitis involving children remain unpublished. Results for the majority of interventional studies (66%) and randomised studies (60%) were publicly available. In contrast, most observational studies remained unpublished. Unpublished studies account for data of 98 673 patients overall. However, three large retrospective studies were outliers in this group: two studies were designed to compare outcomes of appendectomies between hospitals in 40 000 patients each (studies NCT02018016 and NCT02017951). The third study included data from 15 000 children to compare oral and intravenous treatment in appendicitis and other inflammatory conditions (study NCT02311452). Without these three studies, unpublished studies contained data from 3673 patients.

While the majority of trials, that is, n=43/52 (83%) assessed the four important controversial key issues, the availability of answers to these questions varied: diagnostic studies were most rigorously published, with 91% of all trials published, data on surgical procedures, antibiotic and pain management were less transparent. The lack of transparency may lead to an ongoing ambiguity in the field. All industry-sponsored or cosponsored studies were published which may indicate that industry compliance with FDAAA—which mandates publication within 1 year after trial completion— is high.20 Publication rates varied between countries of sponsor/collaborator (table 3).

Without overwhelming evidence for or against an intervention, most surgeons may remain faithful to their successfully proven routine; especially, regarding new surgical techniques or alternative treatment options like antibiotic treatment. New concepts in diagnostic criteria and pain management may be more easily adopted. We speculate that professional mentality may play a role, too, and that it might be possible that less traditional attendings may be more open for new developments and try to adopt new surgical techniques, diagnostic criteria and treatment options like antibiotics instead of surgery in selected patients. In general, scientific uncertainty and ambiguity may explain different approaches in surgery. In order to adopt new evidence, it may be appropriate to update and disseminate internal guidelines regularly.

Median time-to-publication in the present study was 24 months. This was twice as long as the deadline of 12 months after completion mandated by FDAAA. In 2007, Hopewell et al reviewed time-to-publication as time between start of trial and time of publication and found that results of positive studies were available after 4–5 years and negative studies after 6–8 years.21

There was a trend towards improvement in terms of timely public availability of results. Similar issues exist in other challenging areas of paediatric medicine, such as epilepsy, autism and liver transplantation.15–17 22

Limitations and directions for future research

This study has several limitations. Clinical trial registration databases other than ClinicalTrials.gov were not analysed. Unregistered clinical studies were not captured by the present study method. This present analysis relies on accurate data entry into ClinicalTrials.gov.20 We made all efforts to avoid a study being classified as unpublished by searching the two major medical literature databases, PubMed and Google Scholar, and by contacting investigators directly. Reasons why studies remain unpublished were reviewed by Song et al who identified non-submission of study results due to lack of time or low priority and fear of being rejected by journals as the predominant issue.23 In particular, researchers may encounter difficulties to publish their results, such as lack of interest of the journal or multiple rejections by different journals. While multiple unsuccessful attempts to publish a manuscript in high-impact journals may lead to publication delay,24 the ClinicalTrials.gov webpage allowing timely posting of topline clinical trial results is easily accessible and should be helpful in early dissemination of research findings. Six studies were completed less than a year before close of database, which may be too short to publish in a peer-reviewed journal, however posting the results on ClinicalTrials.gov would have been possible.

The present data emphasise the necessity to sensitise paediatric and paediatric surgery residents towards awareness of possible publication bias in paediatric appendicitis. In addition, these data serve as baseline for future publication monitoring.25 We strongly encourage publication of all trial results. In particular, negative data are important in order to prevent subjects being exposed to unnecessary, repeated research. In addition, negative data can be helpful in identifying and correcting perpetual scientific misconception in medical practice, exemplified by the case of fluid replacement with hydroxyethyl starch in critical care patients where data in meta-analyses of clinical trials were unable to further support a positive risk-benefit-ratio for this previously widely used intervention.26 27 The present data serve as quantitative baseline for data transparency in paediatric appendicitis, and it would be of high interest to analyse progress on this issue in the future.

Conclusion

These data raise awareness that despite the importance of appendicitis in clinical practice for the paediatric surgeon, there remains a certain degree of scientific uncertainty due to unpublished clinical trial results with room for improvement in the future. Therefore, biases may exist in the current literature. These data are helpful in framing the shifting paradigms in paediatric appendicitis because they add transparency to the debate.

Table 3

Published (n=33) and unpublished (n=19) completed studies on paediatric appendicitis by country

Countries	Published studies (N)	Unpublished studies (N)
Chile	3	0
Croatia	0	1
Denmark	2	0
Egypt	0	1
Finland	1	1
France	0	1
Germany	3	0
India	0	1
Iran	1	0
Israel	1	1
Italy	1	0
Mexico	0	1
Netherlands	1	1
Scotland	1	2
South Korea	3	0
Spain	0	2
Sweden	1	0
Tunisia	0	1
Turkey	1	1
USA	13	5
USA/Germany	1	0