Framing of research question using the PICOT format in randomised controlled trials of venous ulcer disease: a protocol for a systematic survey of the literature.

INTRODUCTION: Although venous ulcers have a great social and economic impact, there is a lack of evidence from randomised controlled trials (RCTs) to support appropriate management for this disease. Framing the research question using the Population; Intervention; Comparator; Outcome; Time frame (PICOT) format in RCTs can improve the quality of the research design.
OBJECTIVES: To evaluate how the PICOT format is used to frame a research question in reports of RCTs of venous ulcer disease and to determine the factors associated with better adherence to the PICOT format in framing the research question. METHODS AND ANALYSES: We will conduct a systematic survey of RCTs on venous ulcers published in the National Institute of Health, PubMed database between January 2009 and May 2016. We will include all RCTs addressing therapeutic intervention for venous ulcer disease involving human subjects, and published in the English language. The selection process will be carried out in duplicate by two independent investigators. First, titles and abstracts will be screened, then full-text articles. We will examine whether the five elements of the PICOT format are used in formulating the research question and give a score between 0 and 5. The primary outcome will be the proportion of studies that have adequately reported all five PICOT elements. DISSEMINATION: This will be the first survey to assess how the PICOT format is used to frame research questions on the management of venous ulcers in reports of RCTs. On completion, this review will be submitted to a peer-reviewed biomedical journal for publication and the findings will also be presented at scientific conferences. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

This will be the first review to assess how the Population; Intervention; Comparator; Outcome; Time frame (PICOT) format is used to frame research questions in reports of randomised controlled trials (RCTs) on venous ulcers

Selected articles will be reviewed independently and in duplicate to evaluate the possible factors associated with better adherence to the PICOT format

Limitations: Only RCTs published in English will be considered. Only published information will be used. Our findings will only be generalisable to the field of venous ulcer disease.

Introduction

Among the chronic skin ulcers, venous ulcers (VU) are the most common. It is the advanced stage of chronic venous disease and is more common in Western countries.1 The prevalence of VU varies from 0.06% to 2%, occurring in ∼4% of people over 65 years.2 The prevalence of VUs varies substantially between studies due to differences in diagnostic methods, epidemiological characteristics of the patients and whether or not foot ulcers are included.3 4 The primary risk factors for VU include old age, phlebitis and deep venous thrombosis.1 5

Complications arising from VU are diverse, primarily related to its chronicity. The main complications are critical colonisation and contact dermatitis.6 7 More severe complications include cellulitis, osteomyelitis and malignant change.8 9 High rates of relapse after healing is other problem with annual recurrence rates varying between 26% and 69%.10–12 In addition, VU comes with physical, social, economic and emotional burden.13 14 The overall cost of VU treatment in most Western countries is ∼1% of their entire healthcare budgets.15

Although VU has a great social and economic impact, there is a lack of evidence from randomised controlled trials (RCTs) to support the management of patients with VU. A recent systematic review of the trials on comparative effectiveness of advanced wound dressings, antibiotics and surgical management of chronic VU concluded that many of the trials had serious methodological flaws; they were small trials with limited statistical power, which in turn limited their ability to provide conclusive results.16 Furthermore, a critical assessment of the evidence in this review revealed that more trials are needed to evaluate the effects of advanced wound dressings, systemic antibiotics and surgical interventions, compared with one another or to a mandatory compression system established as the standard of care for treatment or management of patients with VU.17

Well-designed, properly conducted and appropriately reported RCTs summarised in a systematic review are considered the gold standard for providing the best evidence on the benefits and harms of different treatments for a particular disease.18 The first step in designing an RCT is the research question (RQ). The success of any research process relies, in part, on how well the investigators are able to turn a clinical problem into an RQ. This is not a simple task.19 The RQ determines the research architecture, strategy and methodology.20 A clear and focused RQ will help to determine the appropriate study design and the most appropriate methods of statistical analysis and sample size.19 A well formulated research question helps to minimise error, measure input and output variables appropriately, consider external and internal validities, limit bias and also address clinical as well as statistical relevance.21

Posing a well formulated RQ will help the practitioner focus on the problem that is most important.22 The PICO format strategy for framing RQs, which was first introduced in 199523 and later expanded to the acronym PICOT in 2006,24 contains the following five elements: Population or sample participants that the researcher wishes to recruit into the study, Intervention of interest, Comparator intervention or a control group to compare with the intervention of interest, Outcomes or results that will measure the effectiveness of intervention and the Time frame over which the outcomes, are assessed.19 25

In other fields of medicine, the use of the PICOT framework is suboptimal and often associated with poor reporting of key methodological issues.26 27

Hypotheses

We hypothesise the following: (1) <25% of the RCTs on VU use all the five PICOT elements in their RQ,26 (2) the RQ is more likely to follow the PICOT format if the RCT is published in a journal that endorses the CONSORT statement,28–32 or has a high impact factor,30 33 is a multicentre study,34 has a large sample size (>100 participants),35–37 reports a statistically significant result for the primary outcome38 and whether the trial is industry funded.35 39 These six factors were chosen based on evidence from previous research that show them to be related to better overall reporting quality.40 Furthermore, the framing of the RQ using the PICOT format has also been shown to be independently associated with better reporting quality.26

Objectives

The purpose of this review is to generate knowledge on how researchers are framing RQs in trials on the management of VU and to provide recommendations for improvement. The specific objectives of this study are: (1) to assess how the PICOT format is used to frame RQs in reports of RCTs published on VU and (2) to determine the factors associated with better adherence to the PICOT format in the framing of the RQ.

Methods

Study design

This study will be a systematic survey of the RCTs on VU published in the National Institute of Health, PubMed database between January 2009 and May 2016. This time frame and library were chosen primarily based on feasibility considerations. This time frame is also part of the period during which there have been several published articles addressing the completeness of reporting or adherence to various reporting guidelines.40 41

Inclusion criteria: RCTs of a therapeutic intervention for VU, involving human participants and written in English.

Exclusion criteria: Non-randomised studies, study protocols of RCTs and abstracts.

A study will be defined as an RCT if the assignment of participants to interventions was described by phrases such as ‘randomly allocated’, ‘assigned at random’ or ‘allocated by randomisation’, and if a control group is present. The control group could be placebo, another treatment, a different dose of the same treatment, usual care or just no treatment.35

The search strategy will include terms for RCTs (Randomized Controlled Trial(ptyp)), venous ulcers (venous ulcers, stasis ulcers, varicose ulcers, venous stasis ulcers, venous hypertension ulcers) and exclusions for other types of articles (study protocol, review, systematic review, meta-analysis, cohort, case–control, case series, guideline and editorial) and limits set for the specific time periods of interest (1 January 2009 to 31 May 2016). The strategy that we will adopt for the search is described in online supplementary appendix 1. The selection process will be carried out by two independent investigators (LPFA and MW) in two screening phases: title and abstract screening and full-text review. The investigators will resolve any discrepancies through consensus. First, the studies will be evaluated by reading the abstracts and only those which fulfil the inclusion and exclusion criteria will be selected for further screening.

Figure 1 is a flow diagram showing the study selection procedures.

Figure 1

Flow diagram showing the study selection procedure. RCTs, randomised controlled trials; VU, venous ulcers.

Rating the framing of the RQ

We will use the same methodology applied in a previous study.26 One paragraph from the introduction or methods section in the full text that best describes the primary RQ, hypothesis or objective will be chosen. In that paragraph, we will evaluate the framing of the RQ, regardless of whether it was formulated as an RQ, hypothesis or objective. We will examine whether the five elements of the PICOT format are used. The five elements are the type of patients or population relevant to the question (P), the intervention (I), the comparative intervention (C), the outcome of interest (O) and the time horizon for measuring the outcome (T). We will score each element as 1 if it was present and as 0 if it was absent. Thus, we will create a PICOT score ranging from 0 to 5. The score will represent a measure of completeness of the description of the primary RQ.

Data abstraction

We will use a Microsoft Excel standardised data abstraction form to extract data from each article. To explore the possible factors that influence the framing of the RQ, additional details will also be obtained: endorsement of the CONSORT statement, journal impact factor, whether the study was conducted at a single center or multicentre, total number of patients recruited in the study, whether the study is industry sponsored, and whether the study reported a statistically significant result for the primary outcome.

Two reviewers (LPFA and MW) will independently abstract the data and any disagreement will be resolved through consensus. If consensus cannot be reached, a third author (LT) will be contacted.

Statistical analysis

The primary outcome will be the percentage of studies that have adequately reported all five PICOT elements and the secondary outcomes will be the percentage of reporting each PICOT item.

We will calculate the percentage of trials that clearly stated each PICOT element and associated 95% CI. We will report descriptive statistics on categorical data as numbers (percentages) and PICOT score as the median (IQR).

The PICOT score will be computed as the sum of the five individual elements and will range from 0 to 5, as described previously.26 For the elements that have a ‘zero’ count or ‘full’ counts, for instance, when none of or all of the included trials reported that PICOT element, the 95% CI will be calculated by adopting the rule of three.42 The rule of three states that: if none of the n individuals reports the item of interest, we can be 95% confident that the chance of this event occurring is at most three in n. For the other PICOT elements, the 95% CI of the count will be calculated by assuming a binomial distribution. The probability that an RCT had clearly stated the element will set to be the observed probability in the sample. We will use Cohen's κ statistic to calculate chance-adjusted inter-rater agreements. Agreement will be interpreted as poor if κ≤0.2, fair if 0.21≤κ≤0.4, moderate if 0.41≤κ≤0.6, substantial if 0.61≤κ≤0.8 and good if κ>0.8.43

We will use generalised estimating equations44 to determine the factors associated with adequate question formulation. The dependent variable will be the PICOT score, ranging from 0 to 5. Adjustments will be made for (1) whether or not the journal endorses the CONSORT statement, (2) journal impact factor, (3) number of centres (multicentre versus single centre), (4) sample size (≤100 vs >100), (5) results of trial (if the primary outcome was statistically significant) and (6) funding status (industry funded versus other sources of funding).

Data will be analysed using Statistical Package for Social Sciences (SPSS) V.16.0 (SPSS, 2009, Chicago, Illinois, USA).

Sample size calculation

The sample size was determined based on the primary objective: estimating the proportion of studies that adequately report all five PICOT elements using a 95% CI. On the basis of a prior estimate of the proportion of RCTs reporting five PICOT elements of 0.20 from a similar study,26 we would require at least 61 RCTs to obtain an estimate of the 95% CI with a margin of error of 0.10. Table 1 provides a summary of the sample size estimates for different values of the margin of error and prior estimates of the proportion of RCTs with an RQ that adequately included five PICOT elements.

Table 1

Sample size (N) estimates

	p
E	0.15	0.20	0.30
0.05	196	246	323
0.10	49	61	81
0.15	22	27	36

P, prior estimate of proportion with all five PICOT elements; E, margin of error.

N=1.962 p(1-p)/E2.

PICOT, Population; Intervention; Comparator; Outcome; Time frame.

The summary of objectives, outcomes, hypotheses and methods of analysis is depicted in table 2.

Table 2

Summary of objectives, outcomes, hypotheses and methods analysis

Objectives	Outcomes	Explanatory variables	Hypotheses	Methods of analyses
Primary: To assess how the PICOT format is used to frame RQ, objectives or hypotheses based on reports of RCTs published on VU	Primary: Proportion of studies that have adequately reported all 5 PICOT elements		1. There are <25% RCTs about VU that adequately report all 5 PICOT elements in the RQ	Percentage of trials that clearly state each PICOT element and associated 95% CI
Secondary: To determine the factors associated with better adherence to the PICOT format in framing the RQ		CONSORT endorsement by the journal Journal impact factor Number of centres (multiple centres vs a single centre) Sample size (≤100 vs >100) Results of trial (if the primary outcome was statistically significant) Funding status (industry funded vs other)	2. The use of the PICOT format will be better if the journals in which it is published endorses the CONSORT statement, has a high impact factor, is a multicentre study, has a sample size >100, reports statistically significant results for the primary outcome and is industry funded	Multivariable regression* (GEE)

*This analysis will be adjusted for the number of centres, sample size, significance of results for primary outcome and source of funding.

PICOT, Population; Intervention; Comparator; Outcome; Time frame; RCT, randomised controlled trial; RQ, research question; VU, venous ulcers.

Discussion and dissemination

RCTs on VUs are the best source of high quality evidence to provide information on the relative merits of various treatments. However, these answers are threatened by the lack of well-designed and reported clinical trials.17 Asking a good RQ is the first step in conducting a good study and improves the overall quality of the study.20 The question should be sufficiently clear, concise and directed to the heart of research to be developed. The success of a scientific study depends in part on the researcher’s ability to transform the clinical problem in the central RQ.19

The results of this analysis will help to elucidate the extent to which the PICOT format is used to frame RQ in RCTs published on VU. In the event that use of the PICOT framework is suboptimal, this analysis will reinforce the importance of framing a good RQ.

Ethics committee approval was not sought for this review since we are dealing with published data. On completion, this review will be submitted to a peer-reviewed biomedical journal for publication and the findings will also be presented at scientific conferences.