Strengthening the quality of clinical trials of acupuncture: a guideline protocol.

INTRODUCTION: Acupuncture has been accepted in many Western countries and clinical trials have been increasing recently. However, the problems of insufficient and low-quality evidence remain, and substantially hinder the development of acupuncture clinical trials. We therefore aim to develop a guideline to strengthen the quality of acupuncture clinical trials, in accordance with WHO handbook for guideline development and the Reporting Items for practice Guidelines in HealThcare. The guideline will help to improve the quality of acupuncture clinical trials. METHODS AND ANALYSIS: We will search for studies on the quality of acupuncture clinical trials using PubMed, EMBASE, Web of Science, China National Knowledge Infrastructure, Wanfang Data, China Science and Technology Journal Database, The Cochrane Library, the WHO and Health Technology Assessment websites, and other sources. We will also check reference lists and contact experts in the field. We will systematically evaluate the quality of acupuncture clinical trials, and extract and summarise the quality problems and countermeasures of such trials. We will also systematically review clinical trial quality control manuals and systems and formulate research questions on quality control in acupuncture clinical trials. Finally, we will develop the guideline and establish a comprehensive quality control system to ensure high quality acupuncture clinical trials. We will also evaluate the guideline and will update the guideline to reflect new scientific evidence. ETHICS AND DISSEMINATION: Ethics committee approval and informed consent are not required for developing guideline because only published data will be used, however, we will interview the patients, the ethics committee approval has been got from West China Hospital of Sichuan Unversity (Number: 2021-1188). We will publish all manuscripts arising from this research and present the findings at conferences. GUIDELINE REGISTRATION NUMBER: IPGRP-2021CN093. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A key strength of this research is the diversity of our study team which will facilitate the development of the guideline.

We will then set up a dynamic feedback mechanism to evaluate and update the guideline as needed.

To the best of our knowledge, this is the first study to design an comprehensive quality control system for high-quality clinical trials of acupuncture.

Our study design is limited by the fact that available relevant literature, and it is difficult to obtain sufficient relevant information to ascertain overall patterns and predict the quality problems of acupuncture research.

Another limitation is the inclusion of only acupuncture clinical trials published in Chinese and English.

Introduction

Acupuncture is a widely recognised traditional Chinese therapy1 and has been gradually accepted in many Western countries.1 2 Over the years, acupuncture has been incorporated into the health insurance policies of several countries.3 4 Studies have indicated that there have been many recommendations for the use of acupuncture in many clinical practice guidelines published worldwide. Both the number of guidelines recommending acupuncture5 and the number of acupuncture randomised controlled trials (RCTs) are increasing.6 7 However, the problems of insufficient and low quality evidence remain, and the methodological reporting quality of acupuncture trials requires improvement.8 9 These problems have substantially hindered the development of acupuncture clinical trials, and there is much debate about the credibility and specific therapeutic applications for acupuncture in the medical community. Therefore, improving the quality of acupuncture clinical trials is very important to enhance the international status of acupuncture.

The rapid development of evidence-based medicine has greatly boosted the development of trials and health decision-making models, and improved the quality of acupuncture clinical trials.10 Several new standards and methods have been developed to improve the quality of clinical trials of acupuncture. For example, based on the Consolidated Standards of Reporting Trials (CONSORT) statement,11 the STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA)12 has been published. The STRICTA is an extended version of the CONSORT statement. These measures are intended to promote the development and improve the quality of clinical trials of acupuncture. However, recent evidence demonstrates that the quality of acupuncture clinical trials still requires improvement.13–16 This may be because the current international standards and quality control system for acupuncture clinical trials address only some aspects of the process of such trials. For example, reporting standards focus on the quality of reporting and acupuncture clinical trial registration standards focus on trial registration. Some aspects of clinical trials and the quality of research data may be neglected, and there is a lack of systematic and comprehensive monitoring of the quality of acupuncture clinical trials. Therefore, there is an urgent need to establish a comprehensive quality control system for acupuncture clinical trials, to improve trial quality. A guideline is needed to help establish a quality control system. This paper describes a protocol to develop such a guideline.

Methods

This guideline will refer to the processes and methods of the WHO formulation manual (2014 edition),17 and will be formulated in accordance with the relevant requirements of the Appraisal of Guidelines for Research and Evaluation (AGREE II)18 and the Reporting Items for practice Guidelines in HealThcare (RIGHT).19 A visual overview of our approach, highlighting key components of the process, is shown in figure 1. The project will comprise four parts: (1) Systematic review to identify all quality problems in acupuncture clinical trials; (2) Systematic review of manuals on quality control in clinical trials; (3) Establishment of a guideline for high-quality acupuncture clinical trials and (4) Updating and re-evaluation of the guideline. The time of the guideline development will be from January 2021 to December 2023.

Figure 1

Flow chart diagram of guideline development.

An explanation of the terms used in this protocol

Acupuncture

Acupuncture is a clinical discipline that is an important part of traditional Chinese medicine (TCM). Acupuncture is used to prevent and treat diseases using the framework of TCM theory.

Clinical trial

This term in the current study refers to any study of acupuncture intervention in humans (patients or healthy volunteers) to confirm the effects of acupuncture. The aim of a clinical trial in current study is to determine the efficacy and safety of the acupuncture intervention.

Agency of the Guideline

This guideline will be jointly initiated by members of editorial boardof the Chinese Journal of Evidence-Based Medicine and members of the editorial board of Journal of Evidence-based Medicine. We will also invite the professional committee for evidence-based medicine of the Chinese Medical Doctor Association, and the Evidence-Based Acupuncture and Moxibustion Professional Committee of the Chinese Association of Acupuncture–Moxibustion to be involved in. The West China Hospital of Sichuan University and the Chengdu University of Traditional Chinese Medicine were involved in formulating parts of the guideline.

Ethics and dissemination

Ethics committee approval and informed consent are not required for developing the guideline because only published data will be used, however, because we will interview the stakeholders including patients, the ethics committee approval has been got from West China Hospital of Sichuan University(Number: 2021-1188). We will publish all manuscripts arising from this guideline and present the findings at conferences. We will also encourage user feedback to inform future updates of the guideline, as needed.

Multidisciplinary teams

The guideline project team will comprise four parts: the guideline steering committee, the guideline expert group, the guideline secretariat group, and the external review group. The responsibilities of each group will be consistent with previously published proposal documents.19

The steering committee will be planned to be consist of ten experienced guide-making experts: seven clinical acupuncturists and three methodologists.

Taking into account regionality, authority and multidisciplinary collaboration,20 we plan to invite 26 guide-making experts, including senior clinical acupuncturists, senior acupuncture researchers, and experts and editors in evidence-based acupuncture. They will participate in three Delphi survey rounds to determine the quality issues the guideline will cover. The guideline expert group will not initially include economists and management experts. If relevant issues arise in the process of formulating the guideline, face-to-face interviews will be conducted and we will feed back the results of the consultation to the expert group.

The secretariat group is composed of several people who are clinical acupuncturists or evidence-based physicians with experience in developing guidelines. To ensure that consistent standards are used during the development process, all study investigators will receive standardised training.

The members of the external review expert group will include acupuncturists, policy researchers, and medical journal editors. They will not be directly involved in the formulation of the guideline, but will be responsible for reviewing the full guideline and provide valuable feedback as and when the recommendations are developed.

Guideline scope

The preliminary name of the guideline is ‘Comprehensively strengthening the quality of clinical trials of acupuncture’. The guideline will be relevant for medical institutions that carry out clinical studies on acupuncture, such as TCM hospitals, integrated traditional Chinese and Western medicine hospitals, integrated TCM and Western medicine departments of general hospitals, and institutions engaged in acupuncture research. The users will include clinical acupuncturists, acupuncture researchers and medical journal editors. The guideline will focus on the scientific and rational design of high quality acupuncture clinical trials, the stringency and standardisation of implementation, the accuracy and transparency of results reporting, and the standardisation and comprehensiveness of trial reports.

Search strategy

To assess the quality problems of acupuncture clinical trials, we will develop a search strategy based on the results of expert consultation. The following sources will be searched: PubMed, EMbase, Web of Science, China National Knowledge Infrastructure (CNKI), WanFang Data, China Science and Technology Journal Database, The Cochrane Library, the WHO and Health Technology Assessment (HTA) websites and other sources. All relevant studies evaluating the quality of acupuncture clinical trials will be comprised; there will be no limitation on publication year. Different combinations of subject terms, free terms and keywords will be used for different databases. In addition, the quality summarised from guideline, systematic review will be also included. The search terms are shown in table 1.

Table 1

Strategies for retrieval of relevant studies evaluating the quality of acupuncture clinical trials

	Search terms
Strategy 1
#1	Acupuncture OR filiform acupuncture OR electroacupuncture OR moxibustion OR acupuncture and moxibustion OR ear acupuncture OR transcutaneous electrical stimulation OR acupoint application OR acupoint catgut embedding OR heat-sensitive moxibustion OR fire acupuncture OR acupoint injection OR warm needle OR small needle knife OR thumbtack needling OR skin needle OR intradermal needle OR scalp acupuncture OR eye needle OR body acupuncture OR hand twist needle OR triangular needle OR auriculotherapy
#2	Quality OR quality evaluation OR quality control OR methodology quality
#3	#1 AND #2
Strategy 2
#1	Acupuncture OR filiform acupuncture OR electroacupuncture OR moxibustion OR acupuncture and moxibustion OR ear acupuncture OR transcutaneous electrical stimulation OR acupoint application OR acupoint catgut embedding OR heat-sensitive moxibustion OR fire acupuncture OR acupoint injection OR warm needle OR small needle knife OR thumbtack needling OR skin needle OR intradermal needle OR scalp acupuncture OR eye needle OR body acupuncture OR hand twist needle OR triangular needle OR auriculotherapy
#2	Quality OR quality evaluation OR quality control OR methodology quality
#3	Systematic review OR Meta-Analysis OR overview OR clinical guidelines OR expert consensus OR reporting guideline
#4	#1 AND #2 AND #3

To develop the quality control guideline, we will search specifications, consensus, and handbooks using PubMed, EMbase, Web of Science, CNKI, WanFang Data, China Science and Technology Journal Database, The Cochrane Library and the WHO and HTA websites. Additionally, references from relevant literature will be manually searched. We will also check the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network to identify any reporting guidelines that are published or under development. The retrieval scheme is shown in table 2. The retrieval strategy will be adjusted according to the different databases.

Table 2

Strategies for retrieval the consensuses and handbooks

Step	Search terms
#1	quality control OR quality evaluation OR quality assurance OR trial
#2	manual OR handbooks OR specifications OR consensus
#3	#1 AND #2

Inclusion and exclusion criteria

The inclusion criteria will comprise (1) All studies on the quality of clinical trials of acupuncture, including experimental studies (RCTs, non-RCTs and non-randomised concurrent controlled trials) and observational studies (cohort studies, case control studies, cross-sectional studies); (2) Clinical guidelines and expert consensuses and (3) Secondary literature on the quality of acupuncture clinical trials (systematic reviews, meta-analyses and overviews). Only published final reports of studies will be eligible. We will exclude articles not written in Chinese or English; articles with incomplete or missing research data; articles for which we are unable to obtain original data; and duplicate articles.

Data extraction and management

Prior to formal screening, we will conduct tests to maintain consistency in data extraction. The data form will be uniformly developed by an expert, and the literature data extraction and management will be completed independently by two researchers. Any discrepancies between reviewers will be resolved through discussion with a third party. Should any new information that is of interest arise during the full-text screening or data extraction, we will update the data collection form to incorporate this information. Any modifications to the present protocol will be reported in the final published review.

Systematic review

Two reviewers will evaluate the quality of non-RCTs using the Methodological Index for Non-Randomised Studies.21 For RCTs, the risk of bias22will be assessed using the Cochrane Collaboration risk of bias assessment tool (Cochrane Handbook for Systematic Reviews of Interventions V.5.1.0). The Newcastle-Ottawa Scal23 will be used for cohort, case–control and cross-sectional studies. The AGREE II tool18 will be used for clinical research guidelines. We will assess the methodological quality of comprised systematic reviews and meta-analyses using the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews) criteria.24 We will refer to the Cochrane Collaboration systematic review evaluation handbook during the entire process of developing this guideline.25 Each article will be reviewed by two independent investigators trained in research methodology and statistics. We will examine the level of inter-reviewer agreement using Cohen’s kappa statistic; any discrepancies between reviewers will be resolved through discussion. The secretariat group will summarise and clearly present the recommendations.

Generation of a list of candidate items

The generation of a list of candidate items will be informed by two sources. First, a list of items will be compiled based on the guideline secretariat group’s report of the studies comprised in the methodological review and interview of the stakeholders. Second, this list will be presented to the guideline expert group alongside the methodological studies report. The expert group will then participate in three Delphi survey rounds to determine the quality issues the guideline will cover. A 5-point Likert scale will be used to rate the importance of each question, ranging from not important to extremely important. Using statistical analysis of the average score and coefficient of variation of each topic, the final decision about the quality of clinical trials comprised in the guideline will be made by the steering committee according to specific consensus rules. The quality issues set out in the guideline proposal may be fine-tuned in this section, based on the feedback of the guideline expert group. The guideline will be developed according to the guidance for developers of health research reporting guidelines.26 Table 3 shows the complete guideline development workflow.

Table 3

Overview of the complete guideline development workflow

Project	Content
A. Preparatory work
	1 Determine the requirements of the guideline
	1.1 Develop new guideline
	1.2 Extend existing guideline
	1.3 Use existing guidelines directly
	2 Review the literature
	2.1 Identify existing relevant guideline
	2.2 Find evidence to show the quality of the published research report
	2.3 Identify potential biased information in the current study
	3 Obtain sponsorship for the development of reporting guideline
B. Before the meeting
	4 Identify participants
	5 Delphi survey
	6 Develop a list of items for face-to-face meeting discussion
	7 Prepare for a face-to-face meeting
	7.1 Determine the size and time required for face-to-face meeting
	7.2 Arrange the logistics of the meeting
	7.3 Set the agenda of the meeting
	7.3.1 Report the relevant background of the project
	7.3.2 Share the results of the Delphi survey with participants
	7.3.3 Invite the chairman of the meeting
	7.4 Prepare materials for participants
	7.5 Prepare the meeting record
C. Face-to-face consensus Meeting
	8 The expert group will discuss the relevant evidence and results prepared before the presentation by the members of the research group
	8.1 Discuss the basic principles of determining the items to be comprised
	8.2 Discuss the making of flow chart
	8.3 Discuss the relevant document production strategy; determine the personnel that should be comprised in each link and the author of the final research results
	8.4 Discuss the knowledge transformation strategy of the report guide
D. After the meeting
	9 Disseminate the reporting guideline
	10 Develop explanatory document
	11 Determine the publishing strategy
	11.1 We will consider publishing simultaneously in a journal
E. Evaluation
	12 Submit the guideline to the National Health Commission of the People’s Republic of China
	13 The guidelines will be translated into multiple languages and recommend it to national clinical trial management agencies
	14 Recommend the guidelines to the China Clinical Trials Registration Platform and WHO Clinical Trials Registration Institution
	15 The guideline will be presented at conferences and published in journal
	16 Establish a dynamic feedback mechanism and evaluate the guideline

Development the Guideline

In this stage, we will develop and refine the guideline. We will invite acupuncture clinical experts, evidence-based methodology experts, reporting guideline methodological experts, journal editors and statistical experts from China and other countries to face-to-face meetings. Based on the results of these meetings, the steering committee will decide on the recommendations. We will use a face-to-face meeting to obtain consensus on those recommendations identified as requiring further consultation. The expert group will form a list of items according to the consensus. The face validity and clarity of this list will be tested with end users, and will receive additional fine-tuning as needed prior to publication. We will return the draft document to the steering group and stakeholders to obtain additional feedback. The guideline secretariat group will then summarise the feedback on the revision of the guideline and write the first edition of the guideline.

External review

We will circulate the guideline to external review group (eg, in terms of approval, clarity and feasibility) and optimise its content according to feedback. The secretariat group will revise the recommendations in the light of feedback, the expert group will review the final draft of the recommendations, and the steering committee will eventually review and approve the full text of the final guideline.

Updates and evaluation of the guideline

We will evaluate the science, rationality, flexibility, and stability of the whole quality control system for high-quality acupuncture clinical trials, and establish a mechanism of dynamic monitoring and feedback. The guideline will be translated into multiple languages and recommended to national clinical trial management agencies. We will also recommend the guideline to the China Clinical Trials Registration Platform and the WHO Clinical Trials Registration Institution, and recommend the guideline to the National Health Commission of the People’s Republic of China. We will incorporate feedback on the content, format and usefulness of the guideline from researchers of these institutions/organisations. We will continue to contact journal editors to update the guideline.

Stakeholders’ involvement

Referring to the 2015 edition of the patient guideline development manual published by the International Guide Collaboration Network,27 we will invite stakeholders to participate in the development of the guideline. Stakeholders will comprise patients, clinical acupuncturists, acupuncture researchers, methodological study authors, caregivers, nurses, methodologists, statisticians, health policy-makers, medical legal workers and journal editors. We will seek recommendations from investigators regarding members of the general public and patients who could be recruited for this project. First, we will collect information using cross-sectional surveys, focus interviews, individual interviews, online consultations and other methods. Second, some stakeholders will be able to participate in the guideline expert group to make recommendations about the quality of clinical trials of acupuncture. The responsibilities of stakeholders will mainly comprise: (1) Participating in the investigation and determination of clinical quality problems of acupuncture in the early stage; (2) Formulating recommendations according to evidence from the literature and from clinical research and (3) Feeding back on the guideline recommendations from the perspective of stakeholders. Informed consent will be obtained from all study participants before the project starts.

Patient and public involvement of the protocol

This is a protocol of a guideline, and the patient and public involvement is not needed for the protocol.

Discussion

The aim of this protocol is to provide the protocol for developing the guideline which will help to improve the quality of acupuncture clinical trials by (1) Analysing the quality problems in acupuncture clinical trials and developing a guideline, as proposed by the EQUATOR network; (2) Permitting researchers to review the clinical trial quality control manuals and systems and appraise acupuncture clinical trials based on their adherence to proposed guidelines; (3) Establishing a comprehensive quality control system to ensure high quality acupuncture clinical trials, and updating and evaluating the published guideline to ensure that it is useful and widely disseminated; and in doing so (4) Allowing end users of acupuncture clinical trials to better evaluate and identify issues with study design and reporting, enabling them to improve the quality of acupuncture clinical trials in their research.

Strengths of the future guideline

The guideline will have several strengths. First, a key strength of the guideline is the diversity of our study team. We will invite a large and multidisciplinary team with expertise in consensus activities, guideline development and research methodology and synthesis. We will also invite international experts to participate. This diversity will facilitate the development of the guideline. We will also encourage stakeholders’ feedback. The inclusion of journal representatives in our core guideline team will aid the wide dissemination of the guideline, and we will continue to contact journal editors for their endorsement. We will then set up a dynamic feedback mechanism to evaluate and update the guideline as needed. Finally, there is currently no comprehensive guideline for the reporting of quality control systems for high quality clinical trials of acupuncture. Such a guideline in needed with the increase in the number of acupuncture clinical trials.

Limitation of the protocol

This protocol has some shortcomings. First, the low number of high quality acupuncture clinical trials is a problem. Available relevant literature is limited, and it is difficult to obtain sufficient relevant information to ascertain overall patterns and predict the problems of acupuncture research. This will prolong the literature review and hinder the overall progress of future work. To mitigate this limitation, we will consult extensively with librarians on optimal approaches to identify the maximum number of studies. We will also design all screening and data extraction prompts to ensure consistency and replicability of our work. Second, literature about the quality of acupuncture clinical trials is also insufficient. Another limitation is the inclusion of only acupuncture clinical trials published in Chinese and English. In future studies, we plan to incorporate relevant literature published in other languages, such as French and Spanish.

Conclusions

This protocol describes in detail the steps of developing a guideline for a comprehensive quality control system for high-quality clinical trials of acupuncture. The guideline will be developed in strict accordance with the methodology and standards of evidence-based guidelines, and will also take into account the characteristics of acupuncture clinical trials. The results will be used as a context for analysing general strengths and gaps in the current quality of evidence in acupuncture clinical trials.