The practice of pilot/feasibility studies in informing the conduct of HIV related clinical trials in sub-Saharan Africa: A scoping review.

Introduction: Pilot/feasibility studies represent a fundamental phase of the research process and play a vital role in the preliminary planning of a full size HIV clinical trial. Published HIV clinical trial protocols were reviewed to establish the extent to which the proposed HIV clinical trials are informed by a prior pilot/feasibility study.
Methods: The JBI methodology for scoping reviews was followed. Six databases were systematically searched to identify articles for inclusion.
Results: Thirty two (32) published HIV study protocols were included. Articles were in the English language and were published in the past 10 years (2011-2020). The review results showed that the majority of HIV-related clinical trials in sub-Saharan Africa were not informed by pilot/feasibility studies. The results further indicated that the number of HIV clinical trials informed by a pilot/feasibility study have been on the increase in the 8 years' period since 2012, a trend that indicates positive uptake of pilot studies in HIV related studies. A few select countries (South Africa, Uganda, Zimbabwe, Malawi and Kenya) comprised more than 70% of all clinical trials that were informed by a pilot/feasibility study, conducted in sub Saharan Africa. Conclusions: Although there is an increasing interest among researchers to integrate pilot/feasibility studies in HIV related research, limited countries in sub-Saharan Africa appear to have embraced this trend. Strategies that can motivate researchers to engage in a culture of incorporating pilot/feasibility studies in HIV related research should be implemented.

Introduction

Pilot/feasibility studies represent a fundamental phase of the research process and are largely a research methodological requirement [1,2]. Feasibility studies are pieces of research done before a main study and are used to estimate important parameters that are needed to design the main study, while pilot studies are a smaller version of the main study used to test whether the components of the main study can all work together [1,[3], [4], [5], [6]]. Though representing slightly different objectives, pilot and feasibility studies are essential in assessing the feasibility, acceptability, safety of treatment or interventions, recruitment potential, randomization and blinding processes, and provide estimates for sample size calculation [3,5,7].

Through their role, pilot/feasibility studies therefore contribute to the determination of the most appropriate trial design and help to prevent extensions or unintended closure of trials as a result of failure to recruit sufficient numbers [5]. Pilot/feasibility studies also contribute to the safety of larger trials in general by revealing unforeseen individual or group characteristics that could expose them to adverse events during or after the trial. In addition, they can contribute to reduction in costs by foreseeing unnecessary expenditures that can be avoided. Indeed, it has been argued that incorporating pilot/feasibility studies in clinical trial conduct can result into improvements in the quality of research conducted and reduces waste in research [5,7].

It has been reported that while many pilot or feasibility studies aim to inform future research, it is possible that many do not reach their intended goal. A review by Arain and colleagues that aimed to ascertain the practice of pilot/feasibility studies in informing clinical trial design and conduct reported that only 8 out of 90 pilot studies led to subsequent main studies [8], while Blatch-Jones and colleagues [5], in their cross-sectional study to ascertain the role of feasibility and pilot studies in randomised clinical trials, reported that even though many (81%) of the studies suggested the need for further research, it was not clear if these resulted into full-sized randomised clinical trials. Adequate reporting of endpoints of pilot/feasibility studies could be helpful in ascertaining if, and to what extent, pilot studies contribute to the conduct of larger clinical trials, to provide a rationale for such to be supported and funded.

Despite the likely benefits of conducting pilot/feasibility trials as part of larger HIV clinical trials, the practice of undertaking these as a pre-requisite for conducting HIV clinical trials in sub Saharan Africa is not well documented, yet as stated by In [3], pilot studies are justified because evidence from their conduct informs whether or not the main study is feasible. Such information can be used to modify the clinical trial protocol hence improving the quality and efficiency of the main study. Thus, the conduct of pilot studies is not only ethical, but helps to reduce waste of efforts by researchers and study participants and provides clues on how research resources can be better spent [2].

We aimed to conduct a scoping review of published HIV clinical trial protocols, to establish how the intended trials have been informed by a prior pilot/feasibility study. We focused on pilots for HIV related clinical trials because there was scarcity of data on this topic, but also, worldwide, the sub Saharan Africa region has the highest HIV prevalence hence many HIV-related clinical trials do take place in this region. The review had the following specific objectives.

To estimate the proportion of clinical trial protocols whose proposed clinical trials are informed by a pilot/feasibility study

To characterise protocols whose proposed clinical trials are informed by a pilot/feasibility study by time, person and place.

Methods

This was a scoping review of protocols of HIV related clinical trials to be undertaken in sub-Saharan Africa. We defined HIV-related clinical trials as any clinical trials that are conducted to find better ways to prevent, detect or treat HIV/AIDS or health states that arise due to HIV infection or AIDS. We defined a protocol of HIV related clinical trials as a document that describes how a clinical trial will be conducted (the objective(s), design, methodology, statistical considerations and organization of a clinical trial,) and ensures the safety of the trial subjects and integrity of the data collected [9]. Being a relatively new area of study, a scoping review (the purpose of which is to provide an overview rather than a synthesis of the available research evidence) will provide initial insights into the area of study and a direction for further research [10]. The scoping review was rigorously conducted following the JBI approach and is reported using the PRISMA-ScR reporting guideline and checklist [[11], [12], [13]].

Inclusion criteria

Types of participants/population: The review included published and/or un-published study protocols that were designed for conducting human based HIV clinical trials. In this review, the clinical trial was eligible for inclusion if it was HIV related.

Context: We included all protocols whose proposed HIV related clinical trials were/would be undertaken in sub-Saharan Africa. Studies that indicated multiple settings but which also included sub-Saharan Africa were included. We excluded study protocols whose settings were not indicated or were not very clear, or were conducted outside sub-Saharan Africa.

Types of studies: Sources of data included published/unpublished protocols for HIV related clinical trials. We only included individual protocols and not review articles. Additionally, protocols for pilot studies (and not for full clinical trials) were excluded. Protocols that reported ongoing or completed clinical trials were included.

Search strategy

The search aimed to identify both published and unpublished (Gray or difficult to locate) primary sources of evidence that reported on using pilot/feasibility studies to inform their HIV clinical trials. A three-step search strategy was utilized. First we carried out an initial limited search of two databases including: Medline (Ovid) and CINAHL. The initial search was then followed by an analysis of the text words contained in the titles and abstracts of retrieved papers, and of the index terms used to describe the articles. A second search using all identified keywords and index terms was then undertaken across all included databases: MEDLINE (OVID), CINAHL, EMBASE, Web of Science, Cochrane Central Register of Controlled Trials (CENTRAL) databases, and African Index Medicus (AIM). Thirdly, the reference list of identified articles were searched for additional sources. Where necessary, we contacted authors of primary sources for further information or clarification. Gray literature was searched from Google, Google Scholar, ClinicalTrials.gov, Protocol Exchange, and UK Clinical Research Network (UKCRN) Portfolio Database. We included all HIV clinical trial protocols published in the previous 10 years (2011–2020). We theorised that there would be many clinical trials in sub Saharan Africa during the last 10 year period to provide sufficient evidence for the review. Because of time and resource constraints, only studies published in the English language were considered for inclusion in this review. Keywords that were used to search for articles included: pilot*, feasibility, clinical trial*, study, HIV, Human Immunodeficiency Virus, AIDS, protocol, proposal, sub-Saharan Africa, Africa, low income setting and low income countr*. A search done from CINAHL is provided as Appendix I.

Study selection

All articles retrieved through the systematic search and from Gray literature were imported into Endnote reference management software for screening. Selection of documents was performed by two independent reviewers. Any disagreements that arose were resolved by consensus. The selection of articles was done at three levels. First articles were screened at title level, followed by abstract review and finally full text documents were retrieved and screened for eligibility. The article screening process and reporting of results were aligned to the PRISMA flow diagram from the PRISMA-ScR statement (Appendix II) [14]. A list of excluded sources at full text review with reasons for exclusion plus details about eligible articles are presented in appendices III and IV respectively.

Data extraction

Data was extracted using a structured tool adapted for this study from the JBI scoping review methodology guideline [11] (Appendix V). Extracted data included: Author(s), Year of publication, clinical trial characteristics, year published, country(s) hosting the trial, population, sample size, methodology/methods, intervention (and comparator), duration of the intervention, and funding agency. Finally data related to the main study outcome, by indicating yes or no to the question of whether the proposed trial was informed or not by a pilot or feasibility study, was extracted. The extraction process was carried out by two independent reviewers, after which the entire team reviewed the extracted results until consensus was reached.

Data analysis

Data analysis was done in the Microsoft excel software. Following data extraction, we computed the proportion of the protocols that were informed by a pilot/feasibility study. We also compared how other variables such as study setting, year of publication and trials duration were associated with the primary outcome. Data were analysed and interpreted using simple descriptive statistics, illustrated in figures and tables, and summarized in a narrative [15] using excel and Statistical Package for the Social Sciences version 25 (SPSS) software.

Results

A total of 286 articles were identified from the database searches and an additional five (5) were identified from references of studies. A total of 36 duplicates were automatically removed from the retrieved references. After reviewing the titles and abstracts of 291 papers, a full text review of 46 articles was done, of which 14 of them were excluded, leaving a total of 32 papers that were included in the review (Fig. 1).

Fig. 1

Flow diagram for the scoping review process adapted from the PRISMA statement by Moher and colleagues (2009).

Main review findings

This scoping review aimed to establish how pilot/feasibility studies are integrated in HIV related clinical trial research by informing the conduct of larger clinical trials. The review findings are presented below as per review objectives.

To estimate the proportion of clinical trial protocols whose proposed clinical trials are informed by a pilot/feasibility study

A total of 32 clinical trial protocols met the selection criteria. Of these, 44% (14/32) [[16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29]]were informed by a pilot/feasibility study while the majority [56% (18/32)] [[30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47]] were not informed by a pilot/feasibility study (Fig. 2).

Fig. 2

Proportion of scoping review protocols that were informed by pilot as a function of the total protocols which met the inclusion criteria, March 2021.

Whereas the aggregate data above shows that fewer clinical trials were informed by a pilot/feasibility, stratifying the clinical trials by year revealed that the number of articles that met the selection criteria increased from 1 in 2012 to 8 in 2020. There was an increase in the number of clinical trials that were informed by a pilot/feasibility from 1 in 2012 to 5 in 2020 (Table 1).

Table 1

Protocols which met the inclusion criteria per year, March 2021.

Year of article	Total number of articles that met criteria	Number of RCTs informed by pilot	Number of RCTs not informed by pilot
2020	8	5	3
2019	3	0	3
2018	5	2	3
2017	6	2	4
2016	3	1	2
2015	2	1	1
2014	3	1	2
2013	1	1	0
2012	1	1	0
Total	32	14	18

To characterise protocols whose proposed clinical trials are informed by a pilot/feasibility study by time, person and place.

To address this objective, we used the CONSORT checklist of information to include when reporting a pilot trial, which among others, includes “results of any other analyses performed including subgroup analyses” [48], Two dimensions of time were determined. The first related to the year in which a protocol was published while the second related to the length of the proposed clinical trial. The ‘person’ characteristic related to the participants in the proposed clinical trial disaggregated by gender, while ‘place’ related to the countries in sub Saharan Africa to host the proposed clinical trial.

Clinical trial participants by gender

The 32 clinical trials targeted all gender categories. In 72% (23/32), the trials involved both males and females, followed by females alone [25% (8/32)] and lastly males alone [3% (1/32)]. There was no significant association of gender to the primary outcome (Table 2).

Table 2

Clinical trial participants by gender among 32 protocols that met criteria.

Gender	Frequency		Percent
Female	8	25
Male	1	3
Male and Female	23	72
Total	32	100

All the 14 clinical trials that were informed by a pilot came from 5 countries; 8 from South Africa, 3 from Uganda, and 1 from Zimbabwe, Malawi and Kenya. Noteworthy, these 5 countries contributed 72% (25/32) of all the protocols that met the selection criteria (Table 3).

Table 3

Number of protocols that met the selection criteria by country.

Country	Informed by pilot	Not informed by pilot	Total
South Africa	8	3	11
Uganda	3	0	3
Zimbabwe	1	0	1
Malawi	1	2	3
Kenya	1	3	4
Uganda, South Africa and Zimbabwe	0	1	1
Tanzania	0	1	1
Swaziland	0	1	1
Nigeria	0	2	2
Mozambique	0	1	1
Malawi and Zambia	0	1	1
Malawi and South Africa	0	1	1
Kenya and Swaziland	0	1	1
Cameroon	0	1	1
Total	14	18	32

It was revealed that 6 of the 7 community trials were informed by a pilot followed by 5 of the 16 clinical sites based trials. The other 3 clinical trials that were informed by a pilot were those conducted at ANC primary health care clinics, designated project area and midwife obstetric unit each contributing 1 trial (Table 4).

Table 4

Distribution of clinical trials by settings.

Context	Informed by pilot	Not informed by pilot	Total trials that met selection criteria
Community trial	6	1	7
Clinical site	5	16	21
Antenatal clinics at primary healthcare clinics	1	0	1
Designated project area	1	0	1
Midwife Obstetric Unit	1	0	1
Safe Male circumcision Clinic	0	1	1
Total	14	18	32

Trials (journal) scored the highest articles published and informed by a pilot and this could be attributed to the fact that the journal is a focus for many trials and trial protocols, also shown to have the highest protocols that were not informed by a pilot.

The majority of trials informed by a pilot study were those that lasted for 3 years (5), followed by those that lasted for 2 years (3). Trials that lasted for a 1 year, 4 and 5 years contributed 2 eligible protocols. None of the trials that lasted for less than 1 year was informed by a pilot study (Table 5).

Table 5

Duration of Clinical Trial trials in relation to being informed by a pilot/feasibility study.

Duration of intervention (in years)	Trial informed by pilot	Trial not informed by pilot	Total
5	2	0	2
4	2	0	2
3	5	4	9
2	3	7	10
1	2	4	6
0	0	3	3
Total	14	18	32

It is however not well understood if the trials with no documented pilot study were actually not informed by pilot study. This could not be verified during our study period and call for an expanded study in the area.

Discussion

This scoping review aimed to establish how pilot/feasibility studies are integrated in the conduct of HIV related clinical trials in SSA. The review results showed that the majority of proposed HIV related clinical trials in SSA were not informed by pilot/feasibility studies. These results are in tandem with recent studies in the field which have suggested that while many pilot or feasibility studies aim to inform future research, it is possible that many do not reach their intended goal. In a review by Arain and colleagues that aimed to ascertain the practice of pilot/feasibility studies in informing clinical trial design and conduct, it was reported that only 8 out of 90 pilot studies led to subsequent main studies [8]. Similarly, Blatch-Jones and colleagues [5], in their cross-sectional study to ascertain the role of feasibility and pilot studies in randomised clinical trials, reported that even though many (81%) of the pilot studies suggested the need for further research, it was not clear if these resulted into full-sized randomised clinical trials. In contrast to the above authors [5,8] whose focus was on the end points of pilot/feasibility studies, we think that by reviewing of full RCT protocols (instead of pilot/feasibility studies in themselves), our study provided better certainty that a particular pilot/feasibility study informed further research (rather than only providing recommendations which may not be implemented). Our review thus provides more reliable evidence on the extent to which pilot/feasibility studies may inform full sized clinical trials. Additionally, the two studies [5,8] assessed broader contexts both geographically and clinically. Our review provides more contextualised data on how pilot/feasibility studies inform HIV related clinical trials undertaken in sub-Saharan Africa, which can guide focused follow up/interventions.

Reasons for low utilisation of pilot/feasibility studies in informing the conduct of larger clinical trials are not well documented but could imply an underreporting of how thre respective pilot/feasibility studies inform the conduct of a subsequent clinical trial. Arain et al. [8], asserted that the low reporting in their study could be as a result of the outcomes of the respective pilot/feasibility which could have suggested that further/larger clinical trials may not be useful if they are not likely to be feasible, cost effective, safe or necessary, if tangible results have already been achieved. These insights further emphasise the need for more focused research on actual outcomes of HIV related pilot/feasibility studies, which we feel will be understood better through reviewing of clinical trial protocols or actual clinical trials informed by the pilot/feasibility study. We also recommend further research on the factors that hinder the integration of pilot/feasibility studies in the conduct of HIV related clinical trials. Further still, low utilisation of pilot/feasibility studies in research could be risky to study participants, hence we recommend research to evaluate human subject's protection for clinical trials that are not informed by pilot/feasibility studies.

Our review revealed that the number of HIV related clinical trials that were informed by a pilot/feasibility study have been on the increase in 8 years’ period since 2012. This trend highlights an increasing interest among researchers to incorporate pilot/feasibility studies in HIV research undertaken in SSA. The increased adoption of implementation research in recent years to assess feasibility of interventions has elevated interest in the use of pilot/feasibility studies [49] and could be responsible for the increase observed in the current review. However, due to sparse research in the area, we could not compare our results with other literature and we recommend that more research be conducted in the area.

Only five countries (South Africa, Uganda, Zimbabwe, Malawi and Kenya) contributed to the more than 70% of all proposed clinical trials informed by a pilot/feasibility study. These results correlate with the prevalence of HIV in sub-Saharan Africa, with Southern (e.g. South Africa and Zimbabwe) and Eastern (e.g. Kenya and Uganda) African countries ranking high in HIV prevalence [50].

Strengths and limitations

This is the first review that has assessed how pilot/feasibility studies inform the conduct of HIV related clinical trials in SSA by reviewing actual clinical trial protocols. Our review therefore provides novel insights in the field of HIV related trial conduct. Being a new research area though, we were unable to relate adequately our results with other literature, which limits comparison of the conclusions made. This review reported low utilisation of pilot/feasibility studies in HIV related clinical trials. It is however not well understood if the trial protocols with no documented pilot/feasibility studies were actually not informed by these. We were unable in the current review to expose such information. We also acknowledge that we could have missed out unpublished protocols that would fit our inclusion criteria. This bias affects the credibility of our results and limits their ability to inform improvements in clinical trial practice. Further still, it would be necessary to understand the final outcome of the proposed trials in the reviewed protocols and derive more understanding of any differences between those that are informed by a pilot/feasibility study and those that are not. We were unable to achieve this in the current study. We recommend more expanded research to address the above identified gaps. A mixed methods empirical study to interact with actual researchers and assess uptake and factors associated with use or non-use of pilot/feasibility studies in the conduct of HIV related clinical trials would be enlightening.

Conclusions

This scoping review is the first to assess how proposed HIV related clinical trials are informed by pilot/feasibility studies and provides novel insights in the field. Despite the likely benefits associated with use of pilot/feasibility studies, the review revealed very minimal uptake of these among SSA researchers. The review further revealed that although there is an increasing interest among researches to integrate pilot/feasibility studies in HIV related research, limited countries appear to have embraced this trend. Strategies that can motivate researchers to engage in a culture of 9incorporating pilot/feasibility studies in HIV related research should be embraced. Policies should focus at routinizing the integration of pilot/feasibility studies in HIV related clinical trials, as a way of reaping the numerous likely benefits of pilot/feasibility studies. Further research, preferably using an empirical mixed methods approach can uncover more insights on factors facilitating and hindering use of pilot/feasibility studies in the conduct of HIV related clinical trials in SSA.

Ethics approval and consent to participate

Not applicable as this is a review article.

Consent for publication

Not applicable.

Funding

The current review was funded by the , and was administered by the University of Liverpool.

Authors' contributions

SN conceived the research idea and wrote the research concept, searched for articles, participated in the data analysis and spearheaded writing of the manuscript. LOO contributed to the concept development, data analysis and writing of the manuscript. BAP contributed to literature searching and writing of the manuscript. CE contributed to the analysis of the data and writing of the manuscript. JBMJ contributed to development of the concept, spearheaded the data analysis process and contributed to writing of the manuscript. All authors read and approved the final manuscript.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Search ID#	Search Terms	Actions
S18	S3 AND S6 AND S10 AND S13 AND S17	(40)
S17	S14 OR S15 OR S16	(88,228)
S16	(MH “Low and Middle Income Countries”) OR “low income countr*"	(4197)
S15	(MH “Low and Middle Income Countries”) OR “low income setting"	(1679)
S14	(MH “Africa+") OR “sub-Saharan Africa"	(85,089)
S13	S11 OR S12	(119,148)
S12	“proposal"	(9834)
S11	“protocol” OR (MH “Protocols+") OR (MH “Research Protocols")	(109,671)
S10	S7 OR S8 OR S9	(150,888)
S9	“AIDS"	(68,063)
S8	(MH “Human Immunodeficiency Virus+") OR “Human Immunodeficiency Virus"	(23,312)
S7	“HIV"	(114,598)
S6	S4 OR S5	(2,027,528)
S5	“study” OR (MH “Pilot Studies")	(1,842,631)
S4	(MH “Clinical Trials+") OR “clinical trial*"	(370,985)
S3	S1 OR S2	(139,482)
S2	(MH “Pilot Studies”) OR “feasibility"	(111,433)
S1	“pilot*"	(111,250)

#	Article	Reason for Exclusion
1.	Dzinamarira, T. and T. P. Mashamba-Thompson (2020). “Adaptation of a Health Education Program for Improving the Uptake of HIV Self-Testing by Men in Rwanda: a Study Protocol.” Medicina (Kaunas, Lithuania) 56(4).	Protocol for a pilot and not a full RCT
2.	Grarup, J. et al. (2015). “Challenges, successes and patterns of enrolment in the INSIGHT Strategic Timing of AntiRetroviral Treatment (START) trial.” HIV Medicine 16(S1): 14–23.	Not RCT
3.	Grover, S. et al. (2019). “Building research capacity through programme development and research implementation in resource-limited settings - the Ipabalele study protocol: observational cohort studies determining the effect of HIV on the natural history of cervical cancer in Botswana.” BMJ open 9(12): e031103.	Not an RCT
4.	Reynolds, N. R. et al. (2016). “MAHILA: a protocol for evaluating a nurse-delivered mHealth intervention for women with HIV and psychosocial risk factors in India.” Bmc Health Services Research16(a): 352.	Not an RCT
5.	Sued, O. et al. (2018). “Physician-delivered motivational interviewing to improve adherence and retention in care among challenging HIV-infected patients in Argentina (COPA2): study protocol for a cluster randomized controlled trial.” Trials19(1).	Not an RCT
6.	Kamal, A. K. et al. (2015). “Improving medication adherence in stroke patients through Short Text Messages (SMS4Stroke)-study protocol for a randomized, controlled trial.” BMC neurology 15: 157.	Not HIV related
7.	Kane, J. C. et al. (2020). “Common Elements Treatment Approach (CETA) for unhealthy alcohol use among persons with HIV in Zambia: Study protocol of the ZCAP randomized controlled trial.” Addictive Behaviors Reports 12: 100,278.	Protocol for a pilot and not a full RCT
8.	Vaccher, S. et al. (2016). “Protocol for an open-label, single-arm trial of HIV pre-exposure prophylaxis (PrEP) among people at high risk of HIV infection: the NSW Demonstration Project PRELUDE.” Bmj Open6(6): e012179.	Not an RCT
9.	Fan, X. et al. (2020). “Evaluation of smartphone APP-based case-management services among antiretroviral treatment-naïve HIV-positive men who have sex with men: a randomized controlled trial protocol.” BMC Public Health20(1): 85.	Not an RCT
10.	Warren, C. E. et al. (2012). “Study protocol for the Integra Initiative to assess the benefits and costs of integrating sexual and reproductive health and HIV services in Kenya and Swaziland.” BMC Public Health 12(1): 973-973.	Non RCT, Quasi experimental study
11.	Weibel, D. et al. (2013). “Need for collaborative international vaccine benefit-risk studies in low-income countries: A pilot project.” Pharmacoepidemiology and Drug Safety 22(SUPPL. 1): 262–263.	Not RCT, not HIV related
12.	Yassi, A. et al. (2014). “Considerations for preparing a randomized population health intervention trial: lessons from a South African-Canadian partnership to improve the health of health workers.” Global health action 7: 23,594.	Not RCT, not HIV related
13.	Zunza, M. et al. (2017). “Interactive weekly mobile phone text messaging plus motivational interviewing in promotion of breastfeeding among women living with HIV in South Africa: study protocol for a randomized controlled trial.” Trials 18(1): 331.	Not an RCT
14.	Zurcher, K. et al. (2020). “Novel approach to estimate tuberculosis transmission in primary care clinics in sub-Saharan Africa: protocol of a prospective study.” BMJ open 10(8).	Non RCT (Prospective study but informed by pilot)