Quality of reporting on thoracic radiotherapy technique in prospective lung cancer trials: A systematic review.

BACKGROUND: The aim of this study is to assess the quality of reporting of thoracic (T) RT technique for curative intent treatment in prospective lung cancer trials.
METHODS: We searched MEDLINE for eligible trials published from 1996 to 2016. We assessed the included trials' reports on whether they reported the RT dose prescription method; RT dose-planning procedures; algorithm for tissue inhomogeneity dose corrections; organs at risk dose constraints; target volume definition, simulation and/or motion management procedures; treatment verification procedures; total RT dose; fractionation schedule; conduct of quality assurance as well as presence or absence of deviations in RT treatment planning and delivery adequately. We performed univariable and multivariable logistic regression to determine the factors that may influence the quality of reporting.
RESULTS: We found 85 eligible trial reports. Target volume definition, total RT dose, and fractionation schedules were reported adequately in more than 90% of the included trials. Algorithm for tissue inhomogeneity dose corrections, simulation and verification procedures, presence or absence of deviations in RT treatment planning and delivery were reported adequately in less than 20% of the included trials. Twenty-three trials (27%) reported 7 criteria or more adequately. Both univariable and multivariable logistic regression showed that trials with RT focused research question were more likely to have adequate quality in reporting (judged as adequate reporting in 7 criteria or more) than trials with non-RT focused question (odds ratio 4.11, 95% confidence interval 1.10 to 15.43, P value = .04).
CONCLUSION: There is significant variability in the quality of reporting on thoracic radiotherapy treatment in prospective lung cancer trials. Future research should focus on developing consensus guidelines to standardize the reporting of radiotherapy technique in clinical trials.

Introduction

The complexity of thoracic radiation therapy techniques has increased over the last 2 decades. The use of 3-dimensional (3D) computed tomography (CT) based simulation for conformal treatment planning has jumped from 2% in 1994 to 77% in 2005 in United States.[ CT simulation allows the radiation oncologists to better anatomically define the target lesions and to calculate the dose to the tumour and normal tissues more precisely. The introduction of intensity modulated radiation therapy (IMRT) technique further complicates the planning and delivering of thoracic radiation therapy when compared with 3D conformal radiation therapy technique as it involves shaping the radiation dose to conform to the target volumes more precisely, thus creating much sharper radiation dose gradients between tumor and normal tissues.[

The reports of trials involving the use of thoracic radiation therapy should contain sufficient details on how radiation therapy was planned and delivered to the trial participants. This is important for several reasons. First, the planning and delivery of high dose curative intent thoracic radiation therapy can be complex and the readers need to have a clear understanding of exactly what was done for the trial participants. The radiation team including radiation oncologists, radiation therapists, medical dosimetrists and medical physicists can learn how to treat the patients better in real world by reproducing the same radiation therapy treatment employed in these trials accurately. This is crucial as a meta-analysis of several randomized trials including 2 cooperative groups trials on lung cancer showed that patients receiving radiation therapy which contained major deviations from the protocol stated dosimetric parameters were associated with lower overall survival outcomes.[ Second, the readers can fully evaluate the reliability and relevance of trial results for his or her clinical practice if they have complete information on the radiation therapy intervention details. Third, trialists can help plan future similar trials if they have sufficient information on the treatment details.

Bekelman and colleagues evaluated the quality of radiotherapy reporting in 61 randomized trials of Hodgkin's and non-Hodgkin's lymphoma and found that there was serious deficiency in the quality of radiotherapy reporting.[ They have proposed that consensus standards for radiotherapy reporting should be developed and integrated into the peer review process as the interpretation, replication and application of the randomized trials results depend on the adequate description and quality assurance of radiotherapy interventions. Although the CONSORT statement for non-pharmacological trials has been developed to standardize the reporting of non-pharmacological interventions, it was not developed specifically for reporting of radiotherapy treatment.[ The CONSORT statement did not mention any specific radiotherapy treatment criterion such as target volume definition, dose constraints for organ at risks. Although similar guidelines have been proposed independently by different research groups to standardize the reporting of radiotherapy technique in clinical trials, it is not known if these guidelines have been adopted in research practice.[

Currently, the quality of thoracic radiotherapy reporting in prospective lung cancer trials is unclear. Hence, we performed this study to determine the quality of thoracic radiation therapy reporting in prospective lung cancer trials and the possible factors that may influence the quality of reporting.

Patients and methods

Study criteria

This study incorporated prospective designed single or multi-arm trials including radiotherapy-naïve patients with histologically or cytologically proven non-small cell or small cell lung cancer. One of the intervention arms needs to include thoracic radiotherapy delivered with curative intent. The included trials need to report either efficacy or toxicity in their treatment outcomes. The sample size of the included trials must be 100 or more as we judged that trials with smaller sample size are less likely to influence clinical practice.

Search strategy

Trials were identified by searching MEDLINE via Pubmed from 1996 to 2016. The search strategy included the medical subject headings of “lung neoplasms” and “radiotherapy”. The results were then hand searched for eligible trials. In addition, the reference lists of selected trials were scanned for any other relevant trials.

Selection of studies and Data Collection

Three reviewers (YYS, THT, JCST) independently assessed the eligibility of abstracts identified by the search. YYS and JCST are certified specialists in radiation oncology. THT is an advanced specialist trainee in radiation oncology. The full-text article of any trial that appeared to meet the inclusion criteria was retrieved for closer examination. Disagreements were resolved by consensus. The same reviewers extracted the data independently using standardized data collection form. Data retrieved from the reports include publication details, radiotherapy treatment details, and trial characteristics such as sample size, and outcome measures.

In a situation when the trials have multiple reports, the initial trial report will be selected for assessment. The trial protocol will be selected for assessment if they were included as a supplementary material or referenced in the trial report or published on the cooperative group trials’ websites.

Quality of thoracic radiotherapy technique assessment

We assessed the quality assessment was based on the reporting of the following 11 criteria (Table 1)[: radiotherapy dose, prescription method, radiotherapy dose planning procedures, algorithm for tissue inhomogeneity dose corrections, at least 1 organ at risk dose constraints, target volume definition, simulation and/or motion management procedures, treatment verification procedures, total radiation dose, fractionation schedule, conduct of quality assurance and deviation in the radiation treatment planning and delivery. These criteria were selected as they were important parameters to ensure that radiation therapy treatment was delivered consistently and accurately during the conduct of trials. An adequacy score based on the total number of criterion assessed to be adequately reported was calculated for each trial. Trials with adequacy scores in the top 25 percentile are considered to have adequate quality in the reporting of thoracic radiotherapy.

Table 1

Adequacy definition of radiotherapy reporting criterion.

Statistical analysis

The descriptive statistics were presented as percentages. Potential predictors of adequate quality of reporting were assessed first using univariable logistic regression. Variables with P value less than .2 in the univariable logistic regression were included in the multivariable logistic regression. Variables with P value less than .05 in the multivariable logistic regression were considered statistically significant. Continuous variables such as year of publication, sample size, and impact factor were reclassified as nominal variable into various categories determined a priori. All statistical analysis was performed using STATA (version 15.1, StataCorp).

Ethical review

Ethical review is not necessary for this study as it does not involve individual patient data.

Results

Results of search strategy

We identified 85 eligible trials using the search strategy summarized in Figure 1. We screened 1523 articles and excluded 1436 articles as they did not meet the inclusion criteria. There were 2 articles which we were unable to retrieve as full article and hence excluded as well.

Figure 1

Results of search strategy.

Characteristics of included studies

The characteristics of the 85 included trials were summarized in Table 2. Seventy-four trials (87%) were of randomized design. Seventy-two trials (85%) included patients with non-small cell lung cancer. Forty-three trials (51% published in year 2006 to 2016. Forty-seven trials (55%) were cooperative group trials. Thirty-one trials (36%) were conducted in North America. Sixty trials (71%) used overall survival as primary endpoint. Sixty-eight trials (71%) were sponsored by industry. Sixty-one trials (72%) had sample size of at least 300 patients. Sixteen trials (19%) were published in radiotherapy focused journal defined as journals related to various radiation oncology societies such as the America Society for Radiation Oncology, European Society for Radiotherapy and Oncology, Royal College of Radiologists and Royal Australian and New Zealand College of Radiologists. Thirty-one trials (36%) had research questions that were radiotherapy focused. Fifty-six trials (66%) employed 3-dimensional conformal or intensity modulated or arc therapy radiotherapy techniques. Seventeen trials (20%) reported their trial registry number. Sixty-six trials (78%) were published in journals with impact factor 15 or less (we used the impact factor of the journal that corresponds to the year of publication of the trial).

Table 2

Characteristics of included studies.

Quality of thoracic radiotherapy technique reporting

There was significant variability in quality of thoracic radiotherapy technique reporting among the included trials (Table 3 and Fig. 2). Twenty-nine trials (34%) reported the radiotherapy dose prescription method adequately. Sixty-nine trials (81%) reported radiotherapy dose planning procedures adequately. Seven trials (8%) reported the algorithm used for tissue inhomogeneity dose corrections. Sixty-five trials (76%) reported organ at risk dose constraints adequately. Seventy-nine trials (93%) reported the target volume definition adequately. Twelve trials (14%) reported the simulation and/or motion management procedures adequately. Fifteen trials (18%) reported treatment verification procedures adequately. All trials reported the total radiation dose adequately. Eighty-three trials (98%) reported the fractionation schedule adequately. Twenty-nine trials (34%) reported the conduct of quality assurance adequately. Thirteen trials (15%) reported the presence or absence of deviation in radiation treatment planning and delivery adequately. Twenty-three trials (27%) reported 7 or more criteria adequately, that is, these trials were considered to have adequate quality in reporting of lung radiotherapy technique.

Table 3

Quality of thoracic radiotherapy technique reporting (number of trials that reported each criterion adequately).

Figure 2

Quality of thoracic radiotherapy technique reporting (number of trials categorized according to the total number of criterions that were reported adequately).

Predictors of adequate thoracic radiotherapy technique reporting

Univariable logistic regression showed that variables including study design, year of publication, types of primary endpoints, types of trial question, types of RT technique used and listing in trial registries have P value less than .2 and were included in the multivariable logistic regression (Table 4). Multivariable logistic regression showed that trials that had a radiotherapy focused question were 4 times more likely than trials with non-radiotherapy focused question to have adequate quality in the reporting of thoracic radiotherapy technique (odds ratio 4.11, 95% confidence interval 1.10 to 15.43, P value = .04) (Table 5). Other factors including study design, year of publication, types of primary endpoints, type of radiotherapy technique used and listed in trial registry did not have a statistically significant impact on quality of thoracic radiotherapy technique reporting based on the multivariable analysis.

Table 4

Univariate logistic regression.

Table 5

Multivariable logistic regression.

Discussion

This study showed that the quality of reporting of curative intent thoracic radiotherapy technique in prospective lung cancer trials was variable. Trials with a radiotherapy focused research question were more likely to have adequate quality reporting than trials with non-radiotherapy focused research question.

The findings of this study were like previous studies.[ This study showed that only 34% of the included trials reported the dose prescription method adequately. In 1988, an editorial published in the International Journal of Radiation Oncology, Biology and Physics highlighted that the reporting of dose prescription was adequate in less than one-third of the clinical papers.[ A review of 200 articles published in Radiotherapy and Oncology and International Journal of Radiation Oncology, Biology and Physics before 1993 showed that only 36% of the articles were judged to have acceptable reporting for dose specification.[

This study also showed that only 27% of the included trials reported at least 7 criteria adequately. Bekelman and colleagues evaluated the quality of radiotherapy reporting in 61 Hodgkin and Non-Hodgkin's lymphoma RCTs in 6 domains: target volume, radiation dose, fractionation, radiation prescription, quality assurance, and adherence to quality assurance.[ They showed that there is serious inconsistency in the reporting of radiotherapy technique in the 6 domains. Similarly, in veterinary radiation oncology, Keyerleber and colleagues evaluated 46 manuscripts for completeness of reporting of radiation therapy treatment planning, dose, delivery and quality assurance using 50 checklist items.[ They showed that only 9 out of the 50 checklist items were reported adequately in at least 80% of the manuscripts. A recent review of 454 randomized phase III trials in radiation oncology showed that nearly 40% of the included radiation treatment arms did not describe the radiation techniques used, demonstrating a significant variation in the quality of radiotherapy treatment reporting in published trial reports.[

One possible reason for the incompleteness or inconsistency in reporting the key parameters of radiotherapy technique in clinical trials maybe the lack of guidelines specifically formulated for trials involving the use of radiation therapy. Although Bentzen has suggested several checklist items relating to radiotherapy treatment planning and delivery to be included in The CONSORT statement in 1998,[ it is unfortunate that his suggestions were not incorporated. In 2010, a global quality assurance of radiation therapy clinical trials harmonization group was formed to homogenize the radiation therapy quality assurance standards in various clinical trial groups.[ However, this group has yet to publish any guideline to standardize the reporting of radiotherapy technique in clinical trials. It is important for the clinical trialists of radiation oncology community to work together if we hope to improve the quality of reporting for radiation oncology trials.

The complexity of radiation therapy technique has increased over the last 20 years. Approximately one-third of the included trials used 2-dimensional radiation therapy techniques while the remaining two-thirds used 3-dimensional or intensity modulated radiation therapy techniques. Despite the varying complexity of radiation therapy techniques used by the included trials, we believe that it is reasonable to evaluate the reporting of radiation therapy technique using the same 10 criteria as the focus is on whether the investigators reported these criteria and not on whether the radiation therapy technical details reported by the investigators are correct

It is not surprising to find that trials with a radiotherapy focused question to have better quality in the reporting of radiotherapy technique as their research questions were more likely focusing on comparing the effects of different radiation treatment parameters such as dose, treatment volumes on the patients’ clinical outcomes. It was surprising to observe that the quality of reporting has not improved over the years and the articles published in the radiation therapy focused journals did not have better quality of reporting. One possible explanation is that most of the clinical trialists, peer-reviewers and editors are not sure what constitutes to be adequate quality for reporting of radiotherapy technique in clinical trials.

The strengths of this study are first we adopted published tools to evaluate the quality of reporting.[ Second, this study focused specifically on the reporting of curative intent thoracic radiotherapy technique in prospective lung cancer trials. Third the results of this study complement previous studies as mentioned earlier.[

The limitations of this study are first, the sample size is small, hence making it difficult for us to conclude the findings definitively at this stage. However, the consistency of our results with other published studies lends strength to this study's conclusions. Second, we included trials with a sample size of at least 100 patients, as we felt that these trials were more likely to have an impact on practice. It is possible that the overall quality of reporting may change with inclusion of trials with smaller sample size. Third, the definition of adequate quality of reporting was decided based on the top 25 percentiles of the adequacy score. We acknowledged that this decision is made arbitrarily. We felt that if the trial report can report at least 7 criteria adequately, it should have sufficient information for the readers to understand how the radiation was delivered to the subjects in the trial.

The implications of this study are first we need to have a guideline for radiotherapy technique reporting to be uniformly adopted by the radiation oncology community so that the readers can evaluate and apply the study results appropriately. Second, we acknowledge that these results do not suggest that the quality of the study design is inadequate as the lack of reporting may be due to gaps in writing and not due to inappropriate conduct of the study.[ Nevertheless, omission in pertinent details of radiotherapy treatment could affect the reader's judgment of the validity and relevance of the trial findings.

In summary, the quality of reporting of curative intent thoracic radiation therapy technique in prospective lung cancer trials was variable. Trials with a radiotherapy focused question were more likely to have adequate quality reporting than trials with non-radiotherapy focused question. Future research should focus on developing consensus guidelines to standardize the reporting of radiotherapy technique in clinical trials.

Author contributions

Conceptualization: Yu Yang Soon, Teng Hwee Tan, Jeremy Chee Seong Tey.

Data curation: Yu Yang Soon, Desiree Chen, Teng Hwee Tan, Jeremy Chee Seong Tey.

Formal analysis: Yu Yang Soon, Desiree Chen.

Methodology: Yu Yang Soon, Desiree Chen, Teng Hwee Tan, Jeremy Chee Seong Tey.

Software: Desiree Chen.

Writing – original draft: Yu Yang Soon, Teng Hwee Tan, Jeremy Chee Seong Tey.

Writing – review & editing: Yu Yang Soon, Desiree Chen, Teng Hwee Tan, Jeremy Chee Seong Tey.

Yu Yang Soon orcid: 0000-0002-1995-2170.