Plain Language Summaries of Clinical Trial Results: A Preliminary Study to Assess Availability of Easy-to-Understand Summaries and Approaches to Improving Public Engagement.

BACKGROUND: Easy-to-understand, stand-alone factual summaries of clinical trial results have the potential to improve public understanding of and engagement with pharmaceutical research. The European Clinical Trial Regulation (EU) No. 536/2014 is a major regulatory initiative that will result in a large number of such plain language summaries (PLSs) posted in the public domain. Today, however, little is known about the extent to which PLSs are written and are available to the general public.
OBJECTIVES: This preliminary study assessed (i) 20 top pharmaceutical companies' positions on improving transparency and commitment to disclosing trial result summaries in an easy-to-understand format and (ii) the availability of such summaries in the public domain and the ease of locating them via general web searches.
METHODS: The availability of PLSs in the public domain was estimated based on the number of EudraCT technical result summaries in four disease areas: chronic obstructive pulmonary disease, asthma, meningitis, and influenza. The likelihood of PLSs being easy to find through internet search engine queries by members of the public was assessed using Google.
RESULTS: All 20 sponsors had committed to improve clinical trial transparency, 17 committed to sharing PLSs with trial participants, and 14 had at least one PLS available in the public domain. A total of 99 clinical studies in these four disease areas had technical summaries posted on EudraCT between 1 January 2017 and 30 June 2020. Of these 99, 14 studies had PLSs in the public domain. A total of 12 of 14 PLSs were directly captured by search engine. However, the sponsor trial identifier or EudraCT number had to be included in the search term to locate them. Generic search terms resulted in large volumes of non-relevant results.
CONCLUSION: Despite the progressive movement towards clinical trial transparency, easily accessible PLSs on clinical trials are currently scarce. The provision of a European mandate and framework for non-technical result summaries by Regulation (EU) 536/2014 will be a major step to bring about positive change.

Key Points

Digital Features

To view digital features for this article go to 10.6084/m9.figshare.12997565.

Introduction

Patients, advocacy groups, and the general public are demanding greater transparency of information on clinical studies [1, 2]. Sharing objective, easily accessible, unbiased research information in a timely manner complements other initiatives to  build trust, partnership, and engagement throughout the clinical study process [2]. Ultimately, improvement in public health literacy and trust between the pharmaceutical industry and the public will help develop new medicines [1] and vaccines. Clearly, the industry still has some way to go before relevant information on every clinical trial becomes ubiquitously available to all. Reviews of posting requirements for technical research summaries on EudraCT and clinicaltrials.gov have revealed gaps in compliance [3-5]. Nonetheless, the clinical trial transparency movement has progressively evolved [6]. Initiatives promoting trial transparency have been announced by the FDA (the FDA Amendments Act of 2007), the European Medicines Agency (Directive [EU]  2001/20/EC), and journal editors [7]. Posting requirements from national authorities have increased, with some now requiring disclosure of the full protocols and clinical study reports [8, 9].

Initiatives on disclosure of technical clinical trial material have undoubtedly increased the availability of information to those working in clinical research. However, the information they present is typically complex and not easily understood by people who do not have a scientific or medical background [10]. In addition, these postings may not be easy to find without prior knowledge of the existence of specialized hosting portals. The utility of these technical documents for patients and the general public may therefore be limited.

Consequently, there has been a push to establish policies and practices to facilitate patient and public involvement and participation in clinical research. In the European Union, Clinical Trial Regulation (Regulation [EU] No 536/2014) is a major step towards improving the transparency of clinical research, thereby potentially increasing engagement with patients and the public [11]. This regulation creates the Clinical Trials Information System (CTIS) for studies conducted within The European Union. It adds the requirement for posting easy-to-understand summaries (known as plain language summaries, patient lay summaries, simple summaries, and trial results summaries [collectively referred to as PLSs here]) of all clinical trials in addition to the currently required technical results summaries. In the US, the Public Health Service Act (2007) also requires a summary of the clinical trial and its results in non-technical, understandable language, although this is yet to be implemented due to challenges in establishing statutory standards. We expect other regulators to follow.

There is increasing evidence [12-14] that study participants want to learn about their disease, the effectiveness and safety profile of the drug they received, and whether the study will contribute towards improved public health. Shalowitz and Miller [14], for example, found that 90% of respondents wanted to receive study results. Although we know that study participants who receive a patient-focused summary value them [12], less is known about the level of interest of the general public in clinical trials. Preliminary evidence from our own unpublished data (in preparation) on vaccines and the accounts of others [15] suggest that clinical research is seen as important for the development of new medicines and vaccines. Despite growing public interest in clinical research and the pending European clinical trial regulation, little is known about the availability and accessibility of such documents in the public domain [2].

This preliminary study assessed (i) 20 top pharmaceutical companies’ positions on improving transparency and commitment to disclosing trial result summaries in an easy-to-understand format and (ii) the availability of such summaries in the public domain and the ease of locating them via general web search.

Methods

To estimate how many of the large commercial sponsors of clinical trials are currently preparing PLSs, we identified the 20 top pharmaceutical companies by 2019 revenue [16]. Next, we searched the websites of these companies for public positions on both improving transparency and committing to writing PLSs for patients or the general public. Clinical trials sponsor websites and various other portals (e.g., trialsummaries.com [17], CISCRP.org [18]) were analyzed to identity how many of these companies had posted at least one PLS of a clinical study in the public domain.

The percentage of studies with publicly available PLSs was calculated for two areas of traditional pharma research (chronic obstructive pulmonary disease [COPD] and asthma) and two prominent areas of interest for vaccines (meningitis and influenza). To establish a baseline, the total number of technical result summaries posted on EudraCT in each of these four disease areas between 1 January 2017 and 30 June 2020 was determined. As of 31 August 2020, the number of these studies with PLSs in the public domain was determined as described above. The ‘findability’ of these PLSs by the public was assessed by simulating potential search patterns for clinical trial results using Google; the search engine that accounts for 87% of market share [19]. Search terms were selected based on guidance by ClinicalTrials.gov and EudraCT (e.g., [20]). Search strings contained terms such as ‘clinical trial’, disease (e.g., COPD, chronic obstructive pulmonary disease) and the drug name (e.g., influenza vaccine or glycopyrronium/formoterol fumarate). Sponsor trial identifier and the EudraCT trial identifier, both with and without the disease name, were also used in the search queryies. Other disease areas were also included in search terms to estimate how easy it was to identify any patient-focused summary of clinical trial results through a search engine. This assessment was not limited to PLSs of studies with technical result summaries posted on EudraCT. The search queries went from general (e.g., lay summaries of vaccine research, plain language results of vaccine research studies) to more specific search strings. Relevancy was ascertained by checking the top 50 hits for any content on patient-focused clinical trial information.

Results

Sponsor Policies on Transparency and Sharing of Clinical Trial Results

All 20 of the top commercial sponsors committed to improving data transparency. In total, 17 of the 20 sponsors also committed to sharing plain language information on clinical trials with trial participants and 9 indicated their intent to also share these summaries with the public. At least one PLS on any disease area had been publicly posted by 14 of the 20 sponsors.

Plain Language Summary (PLS) Availability

Publicly posted PLSs were identified for 14 of 99 (14%) completed trials on the four selected diseases with technical result summaries posted on EudraCT (Table 1). We could not find a PLS for the remaining 85 studies.

Table 1

The number of studies with EudraCT result postings and PLSs

	Technical summary posted	PLS in the public domain
Asthma	59	05
COPD	22	07
Influenza	11	01
Meningitis	07	01
Total	99	14

COPD chronic obstructive pulmonary disease, PLSs plain language summaries

Retrieving Selected PLSs by Internet Search Engine

When used in the same search, the terms ‘clinical trial’ and the disease (e.g., asthma, COPD, influenza, or meningitis) returned tens of millions of hits, although none of the 14 PLSs were identified in the top 50. Combining the term ‘clinical trial’ with the drug name (e.g., formoterol fumarate, influenza vaccine) also failed to identify any of the 14 PLSs in the top 50. Using the sponsor’s unique trial identifier alone resulted in a large variation in the number of hits depending on the complexity of the identifier (e.g., HZA113719, 205724, D5970C00002). Repeat searches produced different numbers of hits, although they were in the same order of magnitude. Most searches using the sponsor’s unique trial identifier alone did not directly identify the PLS (Table 2) in the top 50 hits. A total of 12 of the 14 PLSs were identified when including the disease with the sponsor’s unique trial identifier in the same search. Searching against the EudraCT number alone resulted in direct first page hits for 10 of 14 corresponding PLSs. The results remained the same when including the disease in the search string (Table 2). Four of the missing PLSs did not have the EudraCT number referenced within the document. Two of them also omitted a reference to the sponsor’s unique trial identifier.

Table 2

Finding plain language summaries with technical result summaries on EudraCT through Google

	Search by “clinical trial” and disease	Search by drug/vaccine name	Sponsor’s trial identifier		EudraCT identifier
			Alone	With disease	Alone	With disease
Asthma	0	0	01	05	04	04
COPD	0	0	01	05	04	04
Influenza	0	0	0	01	01	01
Meningitis	0	0	0	01	01	01
Total	0	0	2	12	10	10

Trials with EudraCT technical summary postings on four major categories of diseases between 1 January 2017 and 30 June 2020

COPD chronic obstructive pulmonary disease

Retrieving ANY Patient-Focused Clinical Trial Result Summaries Through Search Strategies

When the search was extended to any patient-focused summary on individual clinical trials, a large volume of search hits was retrieved (Table 3). None of these searches yielded a clinical trial patient-focused summary within the top 50 hits. However, links were found to some sponsor websites or general discussions about legislation, transparency, and engagement with patients and the public. Adding the disease or the name of the drug or vaccine studied to the search string also did not result in direct hits. Table 3 contains searches for PLSs on meningitis trials and is illustrative of search results in other disease areas (not shown). When the disease, and the terms ‘study’ and ‘plain language summary’ were included in the search query, the hits often included a link to the site hosting the PLS. The respective PLS was the only hit when including the sponsor’s unique trial identifier in the search string.

Table 3

Search for any plain language summary (PLS) through Google

Search term	Number of results	Specificity
“Lay summaries of vaccine research”	7.6 million	Provides links to some company sites where information is shared with patients, many links to related general discussions
“Plain language results of vaccine research studies”	4.46 million	Provides links to some company sites describing their position on disclosing data to patients, and many links to related general discussions
Plain language results of meningitis vaccine research studies	602,000	Many more specific results on meningitis, meningitis vaccines, meningitis vaccine side effects, relevant journal articles, no PLS
Meningitis, Haemophilus influenzae type b study result “plain language summary”	913	Most results were directed toward a published journal article or overview (such as the Cochrane Library). However, the site where the PLS of this specific study is published (e.g., the sponsor’s study register) was not included
Meningitis, Haemophilus influenzae type b Study 112931 result “plain language summary”	1	When the sponsor’s study number was included, the website containing the published PLS was the only result

“ ” were used to denote an exact search term

Discussion

Understandable, easy-to-read summaries of clinical research have been recognized as providing greater transparency to those interested in learning about clinical trial results [2]. As such, the PLS may become an important component in the movement to increase public awareness and subsequently stimulate engagement in clinical research. This movement is certainly gaining momentum among clinical trials sponsors. All 20 of the top pharmaceutical companies in this exploratory investigation, and 85% of pharmaceutical companies in general [21], have made public commitments to improving transparency and patient engagement. While the commitment is clear, implementation of these company positions may be at an early stage, as sponsors face challenges such as funding and building capability [1, 22]. We do not know how many PLSs have been written and distributed to patients to date, but their availability in the public domain is limited. However, the fact that 14 of the 20 companies we looked at had publicly posted one or more PLS is an encouraging signal, especially since the European Regulation (EU) No 536/2014 mandating PLSs is still not in force. Nonetheless, there is still a long way to go.

Our finding that relatively few studies have a publicly available PLS is broadly consistent with previous accounts that production of PLSs remains scant. Getz and Farides-Mitchell [1] suggested that < 2% of all clinical trials completed or terminated in a 3-year period had returned a PLS to study participants. The higher percentage (14%) of studies with PLSs reported here reflects our focus on publicly posted PLSs (rather than those directly returned to participants) for studies that had technical results postings on EudraCT. Goldacre et al. [4], reported that half of all trials were non-compliant with this European Commission requirement for posting technical summaries.

The utility of the PLS, however, goes beyond simply producing them in a timely manner and their subsequent posting on a publicly accessible website. Better public engagement predicates that they are also easy to find. And, while 12 of the 14 PLSs were indexed by Google, we failed to identify relevant hits when simulating general public search patterns. When we broadened our search for any PLS of clinical trials, regardless of disease or whether a technical summary had been posted on EudraCT, we again found the results to be disappointing. As expected, we achieved more relevant hits as the search terms became more specific. Direct hits were achieved only when using the sponsor’s unique trial identifier within the search query. It therefore appears that PLSs would not be easy to find by curious members of the public attempting to learn more about specific clinical drug, vaccine or disease trials.

While our finding on the sparsity of PLSs is consistent with published accounts by others [1], it should be noted that this is a pilot study. It is based on a limited sample size. The estimation of frequency of PLSs was confined to four major disease areas with technical result summaries posted on EudraCT. We did not search the ClinicalTrials.gov website. The authors accept that there may be a favorable bias as study sponsors prepare for implementation of the European regulation (EU) No 536/2014. In the absence of comprehensive published accounts, we can only guess how the public will search for information. Even then, search hits can vary among individuals for a given query because ever-evolving search engine algorithms personalize user experience. However, this is unlikely to be beneficial if the material cannot be easily found in the first place.

Conclusion

Overall, these results illustrate that PLSs are generally not publicly available and when they are, they are not easy to retrieve through search engines. We can expect the numbers of PLSs to increase as the European Commission Regulation (EU) No 536/2014 approaches implementation. Nonetheless, the utility of this regulation will not reach full potential until PLSs containing digestible information relevant to patients and the public are easy to locate through indexing and result-ranking of search engines. Considering the potential of the PLS to improve the clinical research process, and based on the outcome of this preliminary research, we are conducting a larger international investigation probing the potential of these documents to engage the public together with what the public would most like to learn from such trials.

The top 20 pharmaceutical companies have public positions committing to clinical trial transparency; 17 of them committed to sharing simple patient-focused summaries with trial participants, 9 indicated they would share such summaries with the public, and 14 of them had posted at least one of these in the public domain.

Relatively few completed studies (14/99) with technical summaries posted in EudraCT in four major disease areas had publicly available patient-focused summaries.

The few plain language summaries in the public domain are not easily detected by internet search engine, thereby limiting their utility as a vehicle to improve public health literacy on clinical trials.