Developing a core outcome set on traditional Chinese medicine (COS-TCM) for chronic heart failure (CHF): a study protocol.

INTRODUCTION: Chronic heart failure (CHF) is a serious and advanced stage of various cardiovascular diseases and portends poor prognosis. An increase in clinical studies has reported the effectiveness of traditional Chinese medicine (TCM). For example, intravenous Chinese medicine can significantly improve cardiac function and biomarkers in patients with CHF. However, there exists inconsistency, lack of practicality and unclear reporting of outcomes in these clinical trials causing difficulty in the comparison of results across similar studies during data synthesis. A core outcome set (COS) can help in the standardisation of outcomes reported across studies from the same healthcare area. The aim of this study is to develop a COS on TCM for CHF (COS-TCM-CHF) to reduce heterogeneity in reporting and improve quality assessment in clinical trials to support data synthesis in addressing the effectiveness of TCM treatment. METHODS AND ANALYSIS: This study will include constructing an outcome pool which will identify potential outcomes through systematic reviews of TCM randomised clinical trials, two clinical registry databases, semi-structured interviews of patients and the clinicians' questionnaire. According to the characteristics of TCM and a taxonomy recommended by the Core Outcome Measures in Effectiveness Trials (COMET) initiative, all outcomes in the outcome pool will be classified into different domains. A preliminary list of outcomes which will then be used in the Delphi survey is generated using a certain criteria based on the length of the pool. The Delphi survey will include two rounds with seven key stakeholder groups to select candidate items for a consensus meeting. A final COS-TCM-CHF will be developed at a face-to-face consensus meeting involving representatives from the different stakeholders. ETHICS AND DISSEMINATION: Ethical approval of this study has been granted by Evidence-based Medicine Centre of Tianjin University of Traditional Chinese Medicine Research Ethics Committee (TJUTCMEC201200002). We will disseminate our research findings of the final COS on the website of Chinese Clinical Trials for Core Outcome Set, with open access publications and present at international conferences to reach a wide range of knowledge users. TRIAL REGISTRATION NUMBER: http://www.comet-initiative.org/studies/details/1486. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Systematic reviews for randomised clinical trials in chronic heart failure on traditional Chinese medicine and identifying additional potential outcomes from two clinical trial registry databases (an international-based and a China-based).

Semi-structured interviews will involve both patients, as well as clinicians, serving as a qualitative method in the core outcome set (COS) development.

Limiting the preliminary checklist of outcomes to a minimum of 100 items to avoid low response efficiency in the Delphi survey.

The Delphi survey and consensus meeting will include key stakeholders such as patient representatives, healthcare professionals and COS users.

TCM is mainly used in China. Hence, the geographical spread of the representatives of stakeholders will be a limitation but able to address the perspective of Chinese population.

Introduction

Background

Chronic heart failure (CHF), caused by changes in cardiac structure and function, is a serious condition in the advanced stages of various cardiovascular diseases, with a poor prognosis.1 The prevalence of CHF is rising with an ageing population and changes lifestyle habits. It is anticipated that the prevalence of heart failure (HF) will increase 46% from 2012 to 2030, with an estimate of more than 8 million adults with HF in USA.2 China has an incidence rate of approximately 0.9% in HF,3 with 500 000 new patients with HF being diagnosed every year and 5-year mortality rate as high as 60%–80%.4 Currently, renin-angiotensin-aldosterone system inhibition, β-receptor blockers, ACE inhibitors and angiotensin receptor blocker have been the mainstream of medical therapy for CHF.5 However, prolonged uses of these drugs may cause severe side effects and could result in declining quality of life for patients with CHF. Traditional Chinese medicine (TCM) is commonly used in combination with conventional therapy to treat CHF in China, improving clinical symptoms and health status on the premise of long-term survival.

Syndrome differentiation is used as a treatment principle for TCM, whereby the TCM syndrome exhibits the severity and stages of the disease combining with the patient’s constitution. It is determined by using the four methods of diagnosis: tongue examination, history taking (inquiry), listening and smelling examination, palpation (pulse taking, abdominal examination, etc). Based on the different syndromes of a disease, a combination of herbs or a TCM formula (containing complex compounds in specific ratios and doses) will be used for the treatment. Early intervention of TCM may reduce the adverse effects due to long-term usage of conventional drugs, as well as improving the immune and physical function of the patients to minimise re-hospitalisation caused by acute attacks of CHF.6 Clinical studies have reported the effectiveness of TCM such as Qishen Yiqi dripping pills, Shenfu or Shengmai injection can also significantly improve the cardiac function and biomarkers in patients with CHF.7 However, these results should be viewed with caution due to a lack of methodological quality for TCM-related clinical studies. There is inconsistency in the outcomes collected and reported across these studies making it difficult to combine and compare results during systematic reviews and meta-analyses.8 In addition, these outcomes may lack relevance to patients and clinicians, leading to research waste.

A way to address this problem of heterogeneity in outcomes reporting is to standardise outcomes reported across CHF-TCM studies, so that outcomes can be meaningful for patients, healthcare professionals and other relevant stakeholders. This could be achieved by the development and implementation of a core outcome set (COS). A COS is an agreed standardised set of outcomes, which should be measured and reported, as a minimum, in all trials for the same healthcare area.9 The role of developing a COS is to improve the consistency in outcome reporting and to measure appropriate and important outcomes in healthcare trials. A significant increase in registered COS studies on Core Outcome Measures in Effectiveness Trials (COMET) website reflected the increasing awareness of the unwarranted variation in outcome collection and reporting.10 However, it is worthy to note that the purpose of a standardised outcome set, or COS is not to create new outcomes, but to select those outcomes which matter the most to the various stakeholders such as clinicians, researchers, policymakers and patients.

In China, there is increasing emphasis on the development of TCM for chronic disease care and prevention, with an increased usage of TCM-related interventions on chronic diseases, particularly in heart disease, stroke, cancer and diabetes.11 This is reflected by the uniqueness of TCM in its ability to treat patients by altering their ‘inner environment’ or simply, to regulate as a whole in the form of improving the patient’s physical function, self-adaptive ability and immune function, which is beneficial for chronic diseases management and prevention. With more research on TCM being carried out in China, guidance to improve the value of trials and provide strong evidence to evaluate the true effectiveness of TCM treatments is urgently required.

At present, there are two related COSs for HF recorded in the COMET database. The Heart Failure Association of the European Society of Cardiology developed a consensus for HF endpoints in clinical trials published in 2013, with a 1-year follow-up on the defined outcomes across different regions between May 2011 and April 2013, reflecting on the higher rate for all-cause mortality in acute HF than CHF across the 1-year span.12 13 Another was the 17-item standardised set of outcome measurement for HF established by The International Consortium for Health Outcomes Measurement on March 2020, which selectively addressed functional, psychosocial, burden of care and survival outcome domains.14 Though the stakeholders involved in the COS development for HF represented an international perspective and included patient-reported outcomes, the standardised outcome set in both COSs may be biased toward Western patient populations, lacking engagement in the Chinese patient population and Chinese healthcare professionals, and does not involve any outcomes related to TCM syndromes or its characteristics. Therefore, it is important to develop a COS that specifically address the need of CHF studies using TCM treatment, which can represent all stakeholders for this specific intervention. The COS will include outcomes with TCM characteristics and achieve consensus in Chinese research or clinical experts and its patient population.

Objective

Currently, there is no study on COS development for CHF in TCM-related clinical studies. The objective of this study is to develop a COS-TCM for CHF (COS-TCM-CHF) and this study protocol is written with reference to the Core Outcome Set-STAndards for Reporting (COS-STAR),15 the Core Outcome Set-STAndards for Development (COS-STAD)16 and the Core Outcome Set-STAndardised Protocol Items (COS-STAP) statement.17

Scope

The health condition for this study is on CHF. Patients with CHF aged 18 and above will be included. This COS will cover all TCM-related interventions, including herbal medicine decoction, Chinese patent medicine, extracts of herbal medicine, intravenous Chinese medicine, acupuncture, cupping, Tuina, moxibustion and other rehabilitation therapy of TCM. The COS-TCM-CHF will be implemented in all future studies that examine outcomes of TCM-related interventions for CHF.

Methods and analysis

Steering committee membership

The steering committee will include 10 members, who are experts from different research fields. They include three TCM and three Western medicine clinical experts in the field of cardiology, two methodologists, one journal editor, one COS developer and one patient representative with CHF. The steering committee will review this protocol and provide guidance at each stage of the study process, as well as resolving any disagreements during the process. The committee will also attend and facilitate the consensus meeting as well as to engage COS uptake in the post development stage. All members of the steering committee membership will be coauthors of the COS-TCM-CHF.

Working group

The working group will be made up of 15 members, including TCM and Western medicine clinicians, methodologists, as well as professors and postgraduates from Chinese Clinical Trials COS Research Centre (ChiCOS), Tianjin University of Traditional Chinese Medicine (TJUTCM), China. The role of the working group will include organising regular meetings, facilitating communication and hold discussion meetings to seek advice from the steering committee if there are any differences to be resolved.

Stakeholders involvement

We will invite seven groups of stakeholders to participate in the development of COS-TCM-CHF, which include healthcare professionals, patients or their representatives, COS developers, COS users (clinical trialists, systematic reviewers and clinical guideline developers), methodologists, policymakers and journal editors. The recruitment of stakeholders is as follows:

Western medicine and TCM healthcare professionals specialising in cardiology, with at least 5 years of clinical experience and academic qualifications of postgraduate and above. They will be selected from the China Association of Chinese Medicine (CACM) and Chinese Society of Cardiology.

Patients diagnosed with CHF possessing a moderate level of literacy and communication from inpatient wards and outpatient clinics of four TJUTCM affiliated hospitals.

COS developers, especially with relevant TCM experience via searching published or ongoing COS studies registered on COMET website.

Methodologists, COS users and journal editors will be invited by sampling with the help of the Chinese Cochrane Centre and TJUTCM Evidence-Based Medicine Research Centre.

Policy makers in public health decision will be selected by CACM.

Patient and public involvement

Patients or their representatives will be involved in the semi-structured interviews, two rounds of Delphi survey and the final consensus meeting. We will recruit patients or their representatives with adequate communication skills. Addressing outcomes that are important to patients is crucial for COS-TCM-CHF development, hence patients’ experiences in the management of CHF will contribute to a large extent in the development process.

Design

The study design for COS-TCM-CHF has been informed by COMET Initiative Handbook, with reference to the guidelines from COS-STAD recommendations and COS-STAP, which will be conducted in five stages (figure 1):

Figure 1

Flow chart of the study design for COS-TCM-CHF. COS, core outcome set; COS-TCM-CHF, COS on TCM for chronic heart failure; RCTs, randomised clinical trials; TCM, traditional Chinese medicine.

Identifying potential outcomes and constructing an outcome pool (Stage 1)

Merging outcomes and grouping under outcome domains (Stage 2)

Generating a preliminary list of outcomes (Stage 3)

Conducting a two-round Delphi survey to select candidate items for consensus meeting (Stage 4)

Hold a consensus meeting to determine the final COS-TCM-CHF (Stage 5).

Stage 1: Identifying potential outcomes and constructing an outcome pool

The purpose of COS development is not to create new outcomes, but to identify the most important outcomes to healthcare professionals, researchers, patients and public health decision-makers in a specific healthcare field or disease, through a consensus process. We will identify potential outcomes by constructing an outcome pool for COS-TCM-CHF development through the four steps:

Conducting systematic review to collect currently reported outcomes from published literature in TCM-related clinical studies for CHF.

Identifying potential outcomes reported in international and Chinese clinical trial registry databases.

Patients’ semi-structured interviews to collect outcomes which matter most to the patients or their representatives.

Clinicians’ questionnaire survey to collect outcomes which are of interest to the healthcare professionals.

Step 1: Systematic review of published literature

Search strategy

We will conduct a comprehensive search strategy from nine English and Chinese databases. Four English databases include PubMed, the Cochrane Library, Embase and Web of Science and five Chinese databases include China National Knowledge Infrastructure, WanFang Database, SinoMed, Chinese Scientific Journal Database and TCM Clinical Evidence Database System. Randomised clinical trials (RCTs) of CHF with TCM-related interventions published in from 2015 to 2020 will be included. The search strategy for English database is shown in online supplemental file 1.

Eligibility criteria

The inclusion and exclusion criteria for the systematic review of published literature is shown in table 1.

Table 1

Inclusion and exclusion criteria of the systematic review for published literature

	Inclusion criteria	Exclusion criteria
Types of literature	Randomised controlled trials.	Unable to retrieve full texts.
Participant	Patients diagnosed with chronic heart failure*, above 18 years of age.	Patients with other comorbidities or in a more critical condition.
Intervention	Intervention in treatment group: Traditional Chinese medicine-related treatments which include oral or intravenous herbal medication, Chinese patent medicine, acupuncture, Tuina and acupoint application. Intervention in control group: No limitation.	None.
Outcomes	No restriction.	No reported outcomes.
Language	Chinese and English.	Published in Chinese non-core journals.

*According to ‘Guidelines for the diagnosis and treatment of heart failure in China (2018)’ or ‘American College of Cardiology/American Heart Association guidelines (2016)’.

Literature screening and data extraction

Two reviewers will independently screen the literature, extract data and cross-check them according to the inclusion and exclusion criteria. For the missing information of some studies, we will try our best to contact the relevant authors to provide them. If not possible, the study will be excluded. Any disagreements will be resolved through discussion after a thorough reading of the paper or by consulting with the third researcher. An extraction table will be designed by the working group to collect relevant information on the study design, setting, demographics of participants, types of intervention and outcomes. Data extracted on the outcomes reported will include the name, definition, measurement time points and measurement instruments or methods. Four principles of data extraction are as follows:

Data extracted will be performed by two reviewers which will then be compared and cross-checked. Any disagreements will be resolved by a senior researcher.

Outcomes extracted will be reported verbatim to ensure authenticity and traceability from the original data.

The free-text field is placed in the extraction table to record special circumstances at any time.

Any alteration of the data extracted during the process will be recorded.

Furthermore, we will assess the reporting quality of outcomes reported in the included studies. The method of this assessment is shown in table 2 with reference to other COS studies.18–20

Table 2

Seven items assessment of the reporting quality of outcome measures

No.	Criterion	Yes	No
1	Whether the outcome was clearly stated as primary or secondary outcome.	1 point	0 point
2	Whether the outcome was defined or not. Outcomes were considered defined if text of their meaning or a citation was provided.	1 point	0 point
3	It was clearly described how the outcomes are measured or the outcome measurement (indicators and/or tools used, if relevant).	1 point	0 point
4	It was clearly described by whom the outcomes are measured.	1 point	0 point
5	It was clearly described the time points and time period at or during which outcome was measured.	1 point	0 point
6	Are methods used to enhance the quality of outcome measurement (eg, repeated measurement, training) if appropriate?	1 point	0 point
7	The reporting of outcomes was consistent throughout the article. There is no unambiguous reporting that makes it confusing for the reader to assess what has been done.	1 point	0 point

Step 2: International and Chinese clinical trial registry databases

We will search Chinese Clinical Trial Registry and ClinicalTrials.gov registry using the key word ‘heart failure’ from inception to 2020. The protocols registered of CHF RCTs with TCM interventions will be included with reference to the same inclusion and exclusion criteria listed in table 1. For literature screening and data extraction of clinical trial registration protocols, similar technique will be applied as per the above-mentioned systematic review of published literature. Subsequently, data extracted will be analysed and outcomes will be aggregated with the list of potential outcomes derived from systematic review of published literature, patients’ semi-structured interviews and clinicians’ questionnaire surveys (as described in Methods 3 and 4 later). Additionally, the reporting quality of outcome measures from registered clinical studies will also be assessed using the quality assessment score in table 2.

Step 3: Patients’ semi-structured interview

Participant selection and recruitment

Patients ≥18 years of age, diagnosed with CHF (according to the clinical guidelines for the diagnosis and treatment of CHF in China or ‘American College of Cardiology/American Heart Association guidelines 2016’) previously or currently under TCM treatment as well as their family members or caregivers will be invited to participate in the semi-structured interview. Potential participants will be approached at the cardiology department of inpatient wards and outpatient clinics in Grade 3A TJUTCM affiliated First Hospital and TJUTCM affiliated Second Hospital, Grade 2A TJUTCM affiliated Fourth Hospital, as well as TJUTCM affiliated BaoKang Community Hospital. Patients or their representatives should possess a moderate level of literacy and communication skills to ensure effective communication. Patients who are cognitively impaired or have serious comorbidities will be excluded from the study. Informed consent is necessary to recruit participants into this study. All potential participants will be informed by a member of the research team, that they are under no obligation to take part and are free to withdraw from the study at any time without their medical care or legal rights being affected. In the event whereby the recruited patients have a language or communication barrier, they will be represented by their caregivers.

Sampling size

Patients’ perspective is important and essential throughout the development process. Sampling for qualitative interviews should aim for a diversity of participants.21 We will consider the gender, age, CHF classification and treatment history of the potential participants. As there is no guideline or consensus on the sample size of this qualitative research, and with past experience of semi-structured interviews done in COS studies,22 23 we aim to recruit a sample size of 50 patients.

Data collection

The semi-structured interview will be conducted by qualified and trained investigator to explain the study to the participants. Study information sheets will be given to participants and informed consent will be collected before interviews are commenced. A questionnaire in Chinese language will be designed in advance for this semi-structured interview. A medical history and additional socioeconomic and demographic information on all participants will be collected. The investigator will ask the participants what outcomes are of importance to them in a face-to-face interview. In the event where a participant is not able to answer, the trained investigator will provide the list of outcomes collected from the systematic review of published literature and registered studies as a guide to inform participants on outcomes. In addition, an open-ended question will be asked to allow participants provide any other outcome which they consider important. The interviewer will then populate the questionnaire with the participant’s answer. All interviews will be audio-recorded to aid detailed data analysis.

As the educational level of participants may vary, the questionnaires will be written using plain English or spoken Chinese for ease of understanding. The working group will carry out pilot-testing to establish if the questionnaire is user friendly. Where possible, professional jargon will be changed into plain English or spoken Chinese to ease comprehension for participants. The outline of the semi-structured interview is as listed below:

‘What kind of result from the CHF treatment do you think is the most important to you?’

‘What is the change in your CHF management, symptoms or daily lifestyle do you consider to be the most important in helping you to determine the effectiveness of the treatment?’

‘Which aspect of CHF treatment is of most concern to you or which aspects do you hope to get better improvement?’

‘Are there any changes after the treatment of CHF? If yes, what changes have made you feel better or even worse than before?’

Step 4: Clinicians’ questionnaire survey

Healthcare professionals, including TCM and western practitioners specialising in cardiology with ≥5 years of clinical experience and a master’s degree or above will be invited to participate in the questionnaire surveys. Potential candidates will be approached at hospital wards and outpatient clinics of TJUTCM affiliated First Hospital, TJUTCM affiliated Second Hospital, TJUTCM affiliated Fourth Hospital and TJUTCM affiliated BaoKang Community Hospital. Participation will be on a voluntary basis and informed consent is required if they agreed to participate in the survey.

Though the outcomes collected will be more comprehensive if the sample size is larger, we considered the feasibility and representativeness of the surveys. Generally, sampling in qualitative studies is purposive, therefore the clinicians who will participate should range from medical officers to senior experts in the field of cardiology. There is currently no guideline for the sampling size for qualitative studies, hence based on our past experience and previous COS studies, we will recruit a sample size of 80 participants. In the event when less than 80 surveys are completed, there will be a further recruitment of clinicians from national cardiology academic conferences until the sample size is reached.

Data collection

A predesigned questionnaire in paper form will be distributed to the healthcare professionals or clinicians. Demographics information, educational and academic status of the clinicians will be collected on agreement to complete the questionnaire survey. Questionnaires are being designed in an open-ended format and clinicians will have the freedom to fill in outcomes which are of importance to them. There will be a limit to a maximum of five outcomes listed in order to achieve the selection of outcomes which are of high importance to the clinicians. The list of outcomes collected from the questionnaires will be analysed by the working group.

Data analysis

Data analysis will be performed after collecting data from the four different methods. Outcomes collected from literature or clinical registry databases as well as qualitative research (patients’ semi-structured interviews and clinicians’ questionnaire surveys) will be imported into an Excel table. These outcomes collected will form the outcome pool. The outcomes will be labelled with numbers corresponding to the original studies or qualitative surveys for easy reference and traceability. On completion of outcome labelling, the outcomes will then be cross-checked and standardised. The methods of standardising outcomes are as follows:

The outcomes are sorted according to their similarities.

Extracted verbatim of the outcomes will be standardised according to their original description or definition, in order to address problems such as abbreviations, nicknames and composite outcomes. The composite outcome will be separated into individual outcomes.

Duplicates will be removed from the outcome pool. Similar and overlapping outcomes will be merged into their standardised terminologies.

Two researchers of the working group will record the corresponding number assigned to the outcomes in the list as well as record the frequency of each outcome.

Finally, an outcome pool is constructed after the outcomes are standardised. The steering committee will be consulted if there were any discrepancies in the process of outcome standardisation. And if there is no consensus reached on an outcome, the outcome will be excluded.

Stage 2: Merging outcomes and grouping under outcome domains

After an outcome pool is constructed, outcomes will be merged and grouped under their respective outcome domains recommended in previous COS studies.24 Since COS-CHF-TCM focuses on developing a set of outcomes to be implemented in TCM-related studies, the outcome classification with TCM characteristics will be added such as TCM syndrome scoring scale. The process will be carried out by two researchers independently. After which the classification of outcomes will be cross-checked. Any discrepancies will be resolved by a third researcher from the steering committee.

Stage 3: Generating a preliminary list of outcomes

The outcome pool can be used as a preliminary list of outcomes in the questionnaire of the Delphi process. However, this list of outcomes can be a long list. If the list of outcomes from the outcome pool is directly taken as the preliminary list of outcomes, it may result in a low response during Delphi survey. Based on previous experience in developing a COS, participants involved in the Delphi survey were less willing to respond when there was a large number of outcomes in the preliminary list, for instance needing more than 10 min to complete the questionnaire. Furthermore, the scores of outcomes are likely to be too concentrated, making it difficult to reach a consensus on the ranking of importance. As a result, multiple rounds of Delphi survey will be required which lead to additional time and resource wastage.

Therefore, it is important to shorten the list of outcome pool to establish a preliminary list of outcomes, which of a suitable length to be used in the Delphi survey later (Stage 4). The criteria for retaining or dropping the outcomes are as follows:

If the number of outcomes collected in the outcome pool is less than or equal to 100, all outcomes will be retained in the preliminary list.

If the number of outcomes collected in the outcome pool is more than 100, the working group will conduct an internal vote to drop the items in the outcome pool under the guidance of the steering committee. If 90% of the members in the working group think that an item is unnecessary to enter the preliminary list, the item will be removed. The remaining items will then be included to form the preliminary list of outcomes.

Stage 4: Two-round Delphi survey

Software

The two-round electronic Delphi survey will be conducted using a Delphi software developed by ChiCOS, similar to the DelphiManager software implemented by COMET working group for COS research. The Delphi software is programmed using Chinese language to cater to the stakeholder groups in China. According to the preliminary list of outcomes, it can automatically generate questionnaire for the Delphi survey and to display results of outcome scoring from the participants and comparison over different stakeholder after each round of survey is completed. The Delphi software is also equipped with a database containing the information of pre-existing Delphi participants or experts shared by ChiCOS working group.

Selection of Delphi participants and sampling size

Seven stakeholder groups will be invited to participate in the Delphi survey, which include clinicians, CHF patients or their representatives, experienced COS developers, methodologists, potential COS users (clinical trialists, systematic reviewers, clinical guideline developers, etc), policymakers and journal editors. The three key stakeholder groups as mentioned in COS-STAD, namely the clinicians, patients or their representatives and COS users are essential in COS development. Hence, we will invite at least 40 participants for each of the three key stakeholder groups. For the group of clinicians, a 1:1 ratio will be fixed for Western medicine doctors and TCM doctors. At least 15 participants will be invited for each of the remaining stakeholder groups. We plan to invite approximately 210 participants for the Delphi survey. It is beneficial for more participants to represent each stakeholder group in order to convince future patients or other stakeholders of its value. Hence, there is no need to specify an upper limit for the number of participants. The selection criteria for participants from the different stakeholders was previously stated in the ‘Stakeholders involvement’ section. Relevant information of the selected participants (excluding patients) will be recorded in the Delphi experts’ database system on ChiCOS website. It is noteworthy to identify the pre-existing Delphi participants name list available on Delphi database system on ChiCOS website shared by members of ChiCOS working group as well as other COS developers. This set of name list can be used to supplement the Delphi participants for this study.

Delphi scoring

The Delphi process will require each participant to consider the importance of each outcome from the four aspects, namely importance on clinical value, recognisability of outcomes in related studies, able to reflect the advantages of TCM effectiveness, as well as being consistent and measurable. Participants will be asked to score each of the outcome items in the preliminary checklist using a scoring scale of 1–9, with 1–3 labelled as ‘not important’, 4–6 labelled as ‘important but not critical’ and 7–9 labelled as ‘critical’.25 Participants will have an option of ‘unsure’ if they are unable to assess the importance of the outcome items.

Delphi round 1

The preliminary list of outcomes generated in Stage 3 will be used in the first round of Delphi survey. For the questionnaire of Delphi round 1, a brief summary of the COS study will be presented to the participants. Participants will need to select which stakeholder group they belong to and be asked to score all the outcome items. At the end of the questionnaire, there is an additional open question, ‘What other outcomes do you think are important but not included in this questionnaire?’, to allow participants to add new outcomes. Besides, participants will have the option to provide their suggestion in a free-text field at the end of each item. On agreeing to participate in the Delphi survey, participants are required to score all outcome items. If the list of questions in the survey is not completed, there will be a reminder to avoid any missing data. Participants will not be able to submit any incomplete questionnaire so as to ensure the integrity of the survey. In order to ensure the comprehensibility of the questionnaire, we will invite a language expert to assist in revising the outcome terminology into plain language. The questionnaire will be pilot-tested on a small group of participants before disseminating to all participants.

Participants (excluding patients) will be sent a personalised email outlining the study along with a link to the online questionnaire of Delphi round 1. And they will be asked to complete the survey within 3 weeks. A reminder email will be sent out at the end of week 2 to prompt completion of the survey. An inspection to assess the number of participants will be done near the end of week 2 by the working group. If the response rate of the Delphi survey (number of respondents/number of invited participants) is lower than 70%, the survey will be extended for two more weeks and a reminder email will be sent to the respective participants who did not respond. For the stakeholder group of patients or their representatives, the survey will be conducted face-to-face in inpatient wards or outpatient clinics by a trained investigator to complete the paper questionnaire. Both the online and paper questionnaires are of the same version. All participants are asked to score each item according to their importance. After completing the rating score, they will have the opportunity to add any other item which is not in the list but important to them.

The working group will collect all completed questionnaires and calculate the overall participant response rate for Delphi round 1. For each outcome item, the average score, the score distribution across different stakeholder groups as well as the number of participants from different stakeholders who have scored it will be summarised. If ≥70% of the participants in any stakeholder group considered the item to be ‘important’ or ‘critical’ (ie, score of 4–9 points), the item will be included in the next round of Delphi. At the same time, the working group will inspect and determine whether the newly added items by the participants are duplicates in the preliminary list and only new and unique outcome will be retained to be included in the next round for scoring. Duplicates or overlapping outcomes will be removed and not entered in the next round.

Delphi round 2

Participants who completed the first round will be invited to participate in Delphi round 2. Response rate, the distribution of scores and participants’ own score for each outcome are being presented to all participants. After considering the feedback from the first round, participants will be asked to re-score the retained outcomes which have met the requirement to enter the second round and score the newly added outcome items from the first round. If the scores between the two rounds change drastically, for example, a participant considered an item in round 1 as ‘not important’ (1–3 points) and re-scores the item as ‘critical’ (7–9 points) in round 2, he or she will need to provide the reason for the change in scores in the free-text field. Participants are also able to provide any suggestions for each item in the survey.

Delphi round 2 survey will also be sent out electronically to all participants (excluding patients or their representatives) by the Delphi software developed by ChiCOS. The survey will be printed into hard copy similar to Delphi round 1 for patients to complete in the second round. The survey should be completed within 3 weeks and a reminder email will be sent to prompt completion of the survey at the end of week 2. If the response rate is lower than 70%, the survey will be extended for one more week and a reminder email will be sent to the respective participants who did not respond.

At the end of Delphi round 2, the participants’ response rate, the average score as well as the score distribution for each outcome item are calculated. Participants’ score changes between rounds 1 and 2 will be examined and verified, as well as summarising the reasons for the change in scores. Together with the reasons, the average score for each outcome in round 2 is compared with that in round 1, so that we can assess attrition bias. There will not be any source for missing data as we have made it compulsory for participants to complete all the questions in the Delphi survey before submission.

Stage 5: Consensus meeting

The final phase of the developing COS-TCM-CHF is to hold a face-to-face consensus meeting after the Delphi process. The consensus meeting will be a 1-day event and 35 participants will be invited to confirm the final outcomes for COS-TCM-CHF. The consensus meeting will be held in Tianjin, China, for the convenience of most participants. In the event where there are other major events such as the resurgence of Coronavirus outbreak, the meeting will be changed to online video conference.

Recruitment

We will invite participants from the different stakeholder groups who have completed the two rounds of Delphi survey to join the consensus meeting. Experts of the steering committee and members of the working group will also attend the meeting. We will draw lots to select representatives of the participants from each stakeholder group, including patient participants. If a representative from a stakeholder group is not available to attend the meeting, he or she will be replaced by another participant from the same stakeholder group.

Consensus definition

Consensus criteria will be specified a priori. The definition of the consensus is shown in table 3, which was made according to the consensus previously used by other COSs developed and COMET recommendations.9 21 26 Outcomes are classified into three categories: ‘consensus in’, ‘consensus out’ and ‘no consensus’.

Table 3

Consensus definition

Consensus classification	Description	Definition
Consensus in	Consensus that outcome should be included in COS	>70% of participants scoring 7–9 and <15% of participants scoring 1–3
Consensus out	Consensus that outcome should not be included in the COS	>70% of participants scoring 1–3 and <15% of participants scoring 7–9
No consensus	Uncertainty about importance of the outcome	Anything else

COS, core outcome set.

Consensus process and final decisions

After a short review of the COS-TCM-CHF study, the scoring results of Delphi round 2 will be shown to all participants in the meeting. According to the consensus definition, outcomes of ‘consensus in’ will be prioritised to be included in the final COS and outcomes of ‘consensus out’ will be excluded. The participants will vote anonymously for the outcomes rated as ‘no consensus’. After the voting, results will be calculated and outcomes achieving a consensus will enter the final COS-TCM-CHF. In the process, all participants have the opportunity to discuss any outcome. If there are any disputes, it will be settled by the steering committee through the nominal group technique.27 The final COS-TCM-CHF will be developed which will include 4–10 core outcomes according to the COMET recommendations.21

Ethics and dissemination

The study has obtained ethical approval through the Ethics Committee of the Evidence-based Medicine Centre, TJUTCM, Ref: TJUTCMEC201200002. Informed consent will be obtained from all participants who participate in the semi-structured interviews, questionnaire survey and two rounds of the Delphi survey.

When the development of the COS is completed, it will be reported based on the items of the COS-STAR. The findings will be submitted for publication in peer-reviewed and open access journals and will be presented at national and international conferences on CHF. In addition, the results will be disseminated on the website of ChiCOS (www.Chicos.org.cn). In addition, with the help of ChiCOS and the China Association of Chinese Medicine, we hope to promote awareness of the COS results and we intend to send the publication of the COS to all participants via emails or express delivery to improve COS-TCM implementation.