Evaluating traditional Chinese medicine diagnostic instruments for functional dyspepsia: systematic review on measurement properties.

BACKGROUND: Pattern diagnosis-guided treatments in Traditional Chinese Medicine (TCM) has been recognised by the eleventh revision of the International Classification of Diseases (ICD-11). Accurate pattern diagnosis requires reliable and valid diagnostic instruments that guide the collection of TCM clinical data without bias. This study synthesised the existing TCM diagnostic instruments for functional dyspepsia (FD) and appraised their quality regarding their development process and measurement properties.
METHODS: Seven electronic databases were searched for validation studies on TCM diagnostic instruments for FD. Synthesis and appraisal of the included studies were performed following the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) Initiative guidelines adapted for TCM diagnostic instruments. Risk of bias assessment was conducted using the COSMIN Risk of Bias Checklist.
RESULTS: Five studies were included, with five unique TCM diagnostic instruments for FD identified. All five diagnostic instruments were of inadequate quality in terms of their development process, implying a shortcoming in their relevance, comprehensibility, and comprehensiveness. Only the criterion validity of Stomach Qi Deficiency Pattern Assessment Scale was of sufficient quality and had no risk of bias in its validation.
CONCLUSION: The quality of TCM diagnostic instruments for FD warrants urgent improvements. None of them was considered reliable or valid for guiding TCM pattern diagnosis. To support the evidence base of the standardization of TCM patterns in ICD-11, TCM diagnostic instruments should be developed and validated rigorously under the COSMIN guidelines. Amendments should be made on the guidelines to accommodate the features and uniqueness of TCM diagnostic process.

Introduction

Pattern diagnosis-guided treatment is an essential component in the practice of Traditional Chinese Medicine (TCM). It refers to the individualization of prescriptions according to patients’ TCM pattern diagnosed from an overall analysis of complex clinical information gathered via “Four Diagnostic Methods”. To facilitate the incorporation of TCM into primary healthcare systems, the World Health Organization (WHO) has been working on the standardization of TCM terminologies and classifications since 1981.1, 2, 3 The WHO program of International Classification of Traditional Medicine (ICTM) aims to standardize disease names, patterns, symptoms, signs, indications for treatment, and interventions within the TCM system. A consensus was reached in 2005 after two informal consultations and one formal meeting involving international Traditional Medicine experts from eight Member States. Results of this standardization were published as the WHO International Standard Terminologies on Traditional Medicine in the Western Pacific Region, and this have been included in the eleventh revision of the International Classification of Diseases (ICD-11) as a dedicated supplementary chapter., This is considered as the first attempt to incorporate of Traditional Medicine into the ICD-based structure, with an aim of facilitating interprofessional communication and collaboration between traditional and conventional clinicians., standardizing TCM patterns for conventional diagnoses is expected to improve the inter-rater agreement among TCM practitioners in pattern diagnosis and encourage the incorporation of TCM pattern diagnosis into clinical trials.8, 9, 10, 11 Research on the standardization of pattern diagnosis process resembles research on diagnostic evaluation without gold standard., In this process, reliable and valid measurement instruments are vital to collecting essential patient data, allowing repeatable diagnosis of TCM patterns.14, 15, 16 Such TCM diagnostic instruments have been developed and validated for various conditions, including mental stress, lateral elbow pain, ulcerative colitis, and coronary heart disease.

These instruments are developed to facilitate accurate TCM pattern diagnosis, such that treatments can be better personalised for patients. For instance, electroacupuncture and herbal medications are found to be effective for managing symptoms of functional dyspepsia (FD),21, 22, 23 yet the current evidence base is not established under the principle of pattern diagnosis-guided treatment. Although it is expected that personalised treatments guided by pattern diagnosis are superior to standardised prescriptions in terms of effectiveness, it is unclear which TCM diagnostic instruments are of sufficient reliability and validity for FD.

This systematic review aimed to identify a reliable and valid instrument for facilitating TCM pattern diagnosis of FD. We synthesised the existing TCM diagnostic instruments for FD and appraised their development process and measurement properties using evidence-based guidelines published by the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) Initiative.,

Methods

Eligibility criteria

We expected to include both published and unpublished studies in Chinese or English, without date limitation, on the validation of TCM diagnostic instruments for FD. We focused on TCM diagnostic instruments which were designed to help collect clinical data via “Four Diagnostic Methods” (inspection, listening and smelling, inquiry, and palpation). Studies on TCM diagnostic instruments for functional gastrointestinal disorders were also included, as long as dyspeptic patients were sampled. All included studies must attach or properly cite at least one accessible full text of the instrument per se. Moreover, they must report the results of at least one measurement property validated. When duplicate studies were identified, only the most recent versions were selected.

Search strategy

Seven electronic databases were searched from their inception to October 16 2019,. Five of them were Chinese: Wanfang Data, China Academic Journals Full-text Database, China Academic Journal Network Publishing Database, SinoMed, and Index to Taiwan Periodical Literature System. Two were English: MEDLINE and EMBASE. Validated search filters with high sensitivity for measurement instruments were applied on MEDLINE and EMBASE., Details of the search strategies are illustrated in Appendix 1. We also conducted manual searches for potentially eligible studies from reference lists of included studies.

Study selection

References from the seven electronic databases were imported into EndNote X9 (Clarivate Analytics, Philadelphia, United States). After deduplication, titles and abstracts were screened, and full text of the potentially eligible citations was retrieved for further assessment and confirmation. Two independent reviewers (Ho and Chung) were responsible for study selection. Disagreements and discrepancies were resolved by consensus between reviewers or by arbitration by a third reviewer (Wong).

Data extraction

Two independent reviewers (Ho and Chung) extracted information on the characteristics of included TCM diagnostic instruments, including their year of development, origin and source language, construct(s), target population, purpose(s), mode of administration, recall period, number of items, and response options. We also extracted relevant information on the characteristics of the study populations, including their sample size, average age, gender, disease(s) in concern, average disease duration, disease severity, source of participants, geographical location of instrument administration, and response rate.

Data synthesis

Due to heterogeneity across included studies, the development process and measurement properties of each included TCM diagnostic instrument were summarised narratively.

Appraisal of instrument development process

A well-developed TCM diagnostic instrument should have its items be relevant for the construct of interest within a specific population and context of use (relevance to the TCM patterns), be clearly understood by that specific population (comprehensibility among patients), and be covering all key aspects of the construct (comprehensiveness of aspects related to the TCM patterns).

The quality of the development process of each included instrument was evaluated by two independent reviewers (Ho and Chung) via a three-step procedure [Appendix 2] guided by a 37-item questionnaire customised for the evaluation of TCM diagnostic instrument development [Appendix 3]. Designed based on the COSMIN items for evaluating the development quality of patient-reported outcome instruments, our newly adapted questionnaire includes key items regarding (i) cognitive interview study or pilot test with patients for diagnostic instrument comprehensibility; and (ii) cognitive interview study with TCM experts for diagnostic instrument comprehensiveness. Rating discrepancies were resolved by consensus through discussions or by arbitration under a third reviewer (Wong).

Appraisal of measurement properties

The degree of reliability and validity of a TCM diagnostic instrument is reflected by the quality of measurement properties. Reliability is defined as the degree to which measurement is free from measurement error. This encompasses the measurement properties of internal consistency, test-retest reliability, and measurement error. Validity is defined as the degree to which a measurement instrument measures the constructs it purports to measure. This consists of the measurement properties of content validity, construct validity, structural validity, cross‐cultural validity, and criterion validity.

The quality of the measurement properties of each included TCM diagnostic instrument was rated against relevant COSMIN criteria., Each measurement property was judged to be either of sufficient quality or insufficient quality., For instance, the test-retest reliability of an instrument would be of sufficient quality if its intraclass correlation coefficient or Cohen's kappa coefficient was shown to be ≥ 0.70 in the validation study., Assessments were performed independently by two reviewers (Ho and Chung) in duplicate. Rating discrepancies were resolved by consensus through discussions or by arbitration under a third reviewer (Wong).

Risk of bias assessment

Risk of bias assessment was performed by two reviewers (Ho and Chung) independently following the procedures recommended by the COSMIN Initiative., The risk of bias of each included validation study was rated against criteria listed in the COSMIN Risk of Bias Checklist on a four-point scale: no risk of bias, serious risk of bias, very serious risk of bias, or extremely serious risk of bias. Rating discrepancies were resolved by consensus through discussions or by arbitration under a third reviewer (Wong).

Results

The search yielded 654 citations [Fig. 1]. Two additional references were identified through manual searches on reference lists. After deduplication, 430 studies proceeded to the screening process. After title and abstract screening, 415 studies were excluded and only fifteen studies were qualified for full text assessment. Eventually, five validation studies completely satisfied the inclusion criteria and were included in this review.

Fig. 1

Flow of literature search.

Characteristics of instruments

Five TCM diagnostic instruments for FD were evaluated in these five validation studies published between 2010 and 2018 [Table 1]. Three of the five diagnostic instruments were developed in China and having Mandarin as their source language.31, 32, 33 The remaining two were developed in South Korea and Hong Kong and having Korean and Cantonese, respectively, as their source language., Three of the five diagnostic instruments were developed for diagnosing TCM patterns among patients with FD or dyspepsia.33, 34, 35 The other two were made for serving a similar purpose among patients with functional gastrointestinal disorders., The mode of administration among the five varied. Two were self-administered questionnaires with about fifty items,, two required a face-to-face interview with a physical assessment performed by TCM practitioners,, and one administered via phone-based interviews by trained personnel.

Table 1

Characteristics of included Traditional Chinese Medicine diagnostic instruments.

Instrument	Year of development	Origin(source language)	Construct(s)	Target population	Purpose(s)	Mode of administration	Recall period	Number of items	Response options
Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale31	2010	China(Mandarin)	TCM signs and symptoms of FGIDs	Patients with FGIDs	Diagnose TCM patterns	Self-administered	2 weeks	44	Never1; Not that much2; Moderate3; Frequently4; All the time5
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders32	2011	China(Mandarin)	TCM signs and symptoms of FGIDs	Patients with FGIDs	Diagnose TCM patterns	Self-administered	2 weeks	54	Five-point Likert scale:1–5 (1=Never, 5=All the time)
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (Amended)35	2015	Hong Kong(Cantonese)	TCM signs and symptoms of FD	Patients with FD	Diagnose TCM patterns	Phone-based interview administered by trained research assistants	2 weeks	24	Five-point Likert scale:1–5 (1=Never, 5=All the time)
Traditional Chinese Medicine Functional Dyspepsia Scale33	2015	China(Mandarin)	TCM signs and symptoms of FD	Patients with FD	Diagnose TCM patterns	Interview-based with physical assessmentadministered by TCM practitioners	2 weeks	33(24 answered by patients; 9 answered by practitioners)	Never1; Not that much2; Moderate3; Frequently4; All the time5
StomachQi Deficiency Pattern Assessment Scale34	2018	South Korea(Korean)	TCM signs and symptoms of SQD	Patients with dyspepsia	Diagnose TCM patterns	Interview-based with physical assessmentadministered by TCM practitioners	Not reported	12(9 by patients; 3 by TCM practitioners)	Never1; Not that much2; Moderate3; Frequently4; All the time5

TCM: Traditional Chinese Medicine; FGIDs: Functional gastrointestinal diseases; FD: Functional dyspepsia; SQD: Stomach qi deficiency

Quality of instrument development process

All five included TCM diagnostic instruments performed well in describing clear constructs and their origins, explaining the target population, and clarifying the context of use. However, due to the lack of concept elicitation process involving TCM experts, all five instruments were judged to have an inadequate design quality [Table 2]. Only the development of Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale included cognitive interview studies with a representative sample of patients and TCM experts. Apart from the described limitations, comprehensibility and comprehensiveness of the included instruments were not satisfactory either. As the lowest rating across the 37 items decides the overall rating, all included TCM diagnostic instruments were graded as “inadequate” in terms of the quality of their development process.

Table 2

Quality of development process among included traditional Chinese medicine diagnostic instruments.

	FGIDS31	FGDDI32	Amended-FGDDI35	CMFDS33	SSQD34
Diagnostic instrument design
Clear construct
(item 1 in Appendix 3)	Very good	Very good	Very good	Very good	Very good
Clear origin of construct (item 2 in Appendix 3)	Very good	Very good	Very good	Very good	Very good
Clear target population for which the instrument was developed (item 3 in Appendix 3)	Very good	Very good	Very good	Very good	Very good
Clear context of use(item 4 in Appendix 3)	Very good	Very good	Very good	Very good	Very good
Instrument development involved a sample from TCM expert community (item 5 in Appendix 3)	Inadequate	Inadequate	Inadequate	Inadequate	Inadequate
Concept elicitation (item 6–13 in Appendix 3)	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
Overall quality of diagnostic instrument design	Inadequate*	Inadequate*	Inadequate*	Inadequate*	Inadequate*
Cognitive interview study or pilot test (patients)
Study or test performed among the target population (item 14 in Appendix 3)	Very good	Inadequate	Inadequate	Inadequate	Inadequate
Study or test performed in a sample representing the target population (item 15 in Appendix 3)	Doubtful	Not applicable	Not applicable	Not applicable	Not applicable
Comprehensibility assessment (item 16–25 in Appendix 3)	Doubtful#	Not applicable	Not applicable	Not applicable	Not applicable
Overall quality of cognitive interview study or pilot test (patients)	Doubtful*	Inadequate*	Inadequate*	Inadequate*	Inadequate*
Cognitive interview study (TCM experts)
Study performed among the TCM expert community (item 26 in Appendix 3)	Very good	Inadequate	Inadequate	Inadequate	Inadequate
Study performed in a sample representing the TCM expert community (item 27 in Appendix 3)	Very good	Not applicable	Not applicable	Not applicable	Not applicable
Comprehensiveness assessment (item 28–37 in Appendix 3)	Doubtful#	Not applicable	Not applicable	Not applicable	Not applicable
Overall quality of cognitive interview (TCM experts)	Doubtful*	Inadequate*	Inadequate*	Inadequate*	Inadequate*
Overall quality of instrument development process	Inadequate@	Inadequate@	Inadequate@	Inadequate@	Inadequate@

FGIDS: Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale; FGDDI: Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders; Amended-FGDDI: Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (Amended); CMFDS: Traditional Chinese Medicine Functional Dyspepsia Scale; SSQD: Stomach Qi Deficiency Pattern Assessment Scale; TCM: Traditional Chinese Medicine

The lowest rating across items 1–13, 14–25, and 26–37 in Appendix 3 was taken as the overall qualities of diagnostic instrument design, cognitive interview study or pilot test (patients), and cognitive interview (TCM experts), respectively.

The lowest ratings across items 16–25 and 28–37 in Appendix 3 were taken as the ratings of comprehensibility assessment and comprehensiveness assessment, respectively.

The lowest rating across items 1–37 in Appendix 3 was taken as the rating of the overall quality of instrument development process.

Quality of measurement properties

Traditional chinese medicine functional gastrointestinal disorders scale

Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale (FGIDS) was validated among 100 healthy subjects and 198 patients with FD or irritable bowel syndrome (IBS) [Table 3]. The validation study took place in hospitals and specialist outpatient clinics in Guangzhou, Shenzhen, and Maoming, China. Its structural validity was assessed by examining whether the proposed nine-factor model could be jusitified by the data obtained using the diagnostic instrument [Table 4]. The hypothesised factors proposed four possible TCM patterns for FD (stagnation of liver qi, spleen-stomach qi deficiency, liver qi invading the stomach, and stomach dampness-heat); and five possible TCM patterns for IBS (liver qi depression, liver qi invading the spleen, spleen-stomach weakness, cold-heat complex, and large intestinal dryness accumulation). Only one TCM pattern for FD and four TCM patterns for IBS could be revealed from the data, with an unsatisfactory comparative fit index (0.87) and root mean square error of approximation (0.064) [Table 5]. The structural validity of FGIDS was therefore considered to be of insufficient quality.

Table 3

Characteristics of populations sampled among included validation studies.

Instrument	Sample			Disease characteristics			Source of participants	Location	Response rate
	N	Average age(SD, range)	Gender% female	Disease(s)	Average disease duration	Disease severity
Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale31	300(Healthy: 100Patients: 200)	15–25 (31.9%)26–45 (46.6%)46–60 (13.8%)≥60 (7.7%)	47%	50% FD50% IBS	Not mentioned	Not mentioned	Hospital & Specialist outpatient clinic	Guangzhou, Shenzhen, and Maoming, China	99.3%
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders32	500(Healthy subjects& patients)	15–25 (23.6%)26–35 (36%)36–45 (27%)46–55 (8.4%)56–70 (5%)	49.5%	FGIDs	Not mentioned	Not mentioned	Specialist outpatient clinic	Guangzhou, China	72.4%
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (Amended)35	270	50.6(12.8, 18–80)	78.6%	FD	Not mentioned	Not mentioned	Specialist outpatient clinic	Hong Kong	83.7%
Traditional Chinese Medicine Functional Dyspepsia Scale33	91	40.3(11.95)	46.1%	FD	Not mentioned	Not mentioned	Specialist outpatient clinic	Guangzhou and Shanghai, China	Not mentioned
StomachQi Deficiency Pattern Assessment Scale34	60(Healthy: 30SQD: 30)	Healthy: 37.5 (11.1, 19–74)SQD: 40.47(10.3, 19–74)	Healthy 63%SQD: 40%	Dyspepsia	Not mentioned	Not mentioned	Hospital	Seoul, South Korea	Not mentioned

FD: Functional dyspepsia; IBS: Irritable bowel syndrome; FGIDs: Functional gastrointestinal diseases; SQD: Stomach qi deficiency

Table 4

Validation of measurement properties among included Traditional Chinese Medicine diagnostic instruments.

	FGIDS31	FGDDI32	Amended-FGDDI35	CMFDS33	SSQD34
Content validity	Not validated	Not validated	Not validated	Not validated	Not validated
Structural validity	Validated	Validated	Validated	Not validated	Not validated
Internal consistency	Not validated	Not validated	Not validated	Not validated	Validated
Cross-cultural validity	Not validated	Not validated	Not validated	Not validated	Not validated
Measurement invariance	Not validated	Not validated	Not validated	Not validated	Not validated
Test-retest reliability	Not validated	Not validated	Not validated	Validated	Not validated
Measurement error	Not validated	Not validated	Not validated	Not validated	Not validated
Criterion validity	Not validated	Not validated	Not validated	Validated	Validated
Construct validity	Not validated	Not validated	Not validated	Not validated	Not validated
Responsiveness	Not validated	Not validated	Not validated	Not validated	Not validated

FGIDS: Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale; FGDDI: Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders; Amended-FGDDI: Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (Amended); CMFDS: Traditional Chinese Medicine Functional Dyspepsia Scale; SSQD: Stomach Qi Deficiency Pattern Assessment Scale

Table 5

Summary of measurement property validations.

Instrument	Structural validity	Internal consistency	Test-retest reliability	Criterion validity
	Result[Quality of measurement property]*	Result[Quality of measurement property]#	Result[Quality of measurement property]@	Result[Quality of measurement property]+
Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale31	• Total CFI and RMSEA: 0.87; 0.064 • CFI and RMSEA of each identified factor (TCM pattern):○ Stomach dampness-heat (pattern of FD) – 0.52; 0.15 ○ Liver qi depression (pattern of IBS) – 0.72; not reported ○ Liver qi invading the spleen (pattern of IBS) – 0.69; 0.17 ○ Spleen-stomach weakness (pattern of IBS) – 0.56; 0.16 ○ Cold-heat complex (pattern of IBS) – 0.81; 0.20 • [Insufficient]	Not validated	Not validated	Not validated
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders32	• Total CFI and RMSEA: 0.83; 0.052 • [Sufficient]	Not validated	Not validated	Not validated
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (Amended)35	• KMO: 0.777 • Bartlett's sphericity test: p<0.001 • EFA:○ 8 factors with eigenvalue >1, contributed to 59.4% of total variance ○ Factors were not related to TCM patterns • [Insufficient]	Not validated	Not validated	Not validated
Traditional Chinese Medicine Functional Dyspepsia Scale33	Not validated	Not validated	• Cohen's kappa of each identified factor (TCM pattern):○ Spleen-stomach qi deficiency – 0.65 ○ Liver qi depression– 0.82 ○ Heat – 0.74 ○ Stagnation of phlegm-dampness – 0.76 ○ Qi stagnation – 0.68 • [Insufficient]	• Gold standard: Diagnosis made by Chinese medicine experts • Sensitivity and specificity of each factor (TCM pattern):○ Spleen-stomach qi deficiency – 0.73; 0.95 ○ Liver qi depression – 0.88; 0.94 ○ Heat – 0.90; 0.90 ○ Stagnation of phlegm-dampness – 0.90; 0.89 ○ Qi stagnation – 0.76; 0.93 • [Indeterminate]
StomachQi Deficiency Pattern Assessment Scale34	Not validated	• Cronbach's alpha: 0.87 • [Indeterminate]	Not validated	• Gold standard: Diagnosis made by Korean medical doctors • Sensitivity: 93.33% • Specificity: 86.67% • AUC: 0.916 • [Sufficient]

CFI: Comparative fit index; TCM: Traditional Chinese Medicine; FD: Functional dyspepsia; IBS: Irritable bowel syndrome; RMSEA: Root mean square error of approximation; KMO: Kaiser-Meyer-Olkin Test; EFA: Exploratory factor analysis; AUC: Area under the Receiver Operator Curve

Sufficient: CFI >0.95 OR RMSEA <0.06; indeterminate: not all information for “sufficient” reported; insufficient: criteria for “sufficient” not met

Sufficient: Sufficient structural validity AND Cronbach's alpha(s) ≥0.70 for each unidimensional scale or subscale; indeterminate: criteria for “sufficient structural validity” not met; insufficient: sufficient structural validity AND Cronbach's alpha(s) <0.70 for each unidimensional scale or subscale

Sufficient: Cohen's kappa ≥0.70; indeterminate: Cohen's kappa not reported; insufficient: Cohen's kappa <0.70

Sufficient: Correlation with gold standard ≥0.70 OR AUC ≥0.70; indeterminate: not all information for “sufficient” reported; insufficient: correlation with gold standard <0.70 OR AUC <0.70

Traditional chinese medicine syndrome differentiation instrument for functional gastrointestinal disorders

Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (FGDDI) was validated among 500 subjects, including healthy participants and patients with functional gastrointestinal disorders. The setting was in specialist outpatient clinics in Guangzhou, China [Table 3]. Its structural validity was assessed by investigating whether the proposed four-factor model could be justified by the data obtained using the instrument [Table 4]. The four factors corresponded to the four TCM patterns of FD, including liver qi depression, spleen deficiency with dampness encumbrance, spleen-stomach qi deficiency, and qi stagnation. As the root mean square error of approximation of the model was smaller than 0.06, the structural validity of FGDDI was judged to be of sufficient quality [Table 5].

Traditional chinese medicine syndrome differentiation instrument for functional gastrointestinal disorders (Amended)

Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (Amended) (Amended-FGDDI) was validated among 270 FD patients recruited from specialist outpatient clinics in Hong Kong [Table 3]. Its structural validity was evaluated via exploring replicability of the TCM patterns contained in FGDDI using data collected in Hong Kong [Table 4]. The structural validity of Amended-FGDDI was considered to be of insufficient quality as only exploratory factor analysis were conducted [Table 5].

Traditional chinese medicine functional dyspepsia scale

Traditional Chinese Medicine Functional Dyspepsia Scale (CMFDS) was validated among ninety-one FD patients recruited from specialist outpatient clinics in Guangzhou and Shanghai, China [Table 3]. CMFDS proposed a five-factor model, which corresponded to TCM patterns of spleen-stomach qi deficiency, liver qi depression, heat, stagnation of phlegm-dampness, and qi stagnation. Its test-retest reliability and criterion validity were evaluated [Table 4]. Since Cohen's kappa of at least 0.70 was not achieved by all factors, the test-retest reliability of CMFDS was rated to be of insufficient quality [Table 5]. The criterion validity was of indeterminate quality [Table 5], as the factors’ correlation with the gold standard (diagnosis made by TCM practitioners) or area under the receiver operator curve results were not reported.

Stomach qi deficiency pattern assessment scale

Stomach Qi Deficiency Pattern Assessment Scale (SSQD) was validated among thirty healthy subjects and thirty patients with dyspepsia in hospitals located in Seoul, South Korea [Table 3]. Its internal consistency and criterion validity were evaluated [Table 4]. Due to a lack of sufficient structural validity, its internal consistency was judged as indeterminate [Table 5]. Nevertheless, with an area under the receiver operator curve value of 0.916, its criterion validity was rated as of sufficient quality [Table 5].

Results of the risk of bias assessment are illustrated in Table 6. Criterion validity studies of CMFDS and SSDQ were at no risk of bias. The structural validity study of Amended-FGDDI and the internal consistency study of SSDQ were at no risk of bias as well. However, structural validity studies of FGIDS and FGDDI, as well as the test-retest reliability study of CMFDS were at serious risk of bias.

Table 6

. Risk of bias among measurement property validation studies.

Instrument	Structural validity	Internal consistency	Test-retest reliability	Criterion validity
Traditional Chinese Medicine Functional Gastrointestinal Disorders Scale31	Serious risk of bias	Not validated	Not validated	Not validated
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders32	Serious risk of bias	Not validated	Not validated	Not validated
Traditional Chinese Medicine Syndrome Differentiation Instrument for Functional Gastrointestinal Disorders (Amended)35	No risk of bias	Not validated	Not validated	Not validated
Traditional Chinese Medicine Functional Dyspepsia Scale33	Not validated	Not validated	Serious risk of bias	No risk of bias
StomachQi Deficiency Pattern Assessment Scale34	Not validated	No risk of bias	Not validated	No risk of bias

Discussion

Summary of main results

A total of five unique TCM diagnostic instruments were included, of which their validation studies evaluated seven measurement properties conducted among five populations. Among these results, the structural validity of FGDDI and the criterion validity of SSDQ were of sufficient quality. Only the validation study of SSDQ was at no risk of bias. Apart from the inadequate quality of measurement properties, the development process of these instruments failed to demonstrate sufficient quality. Therefore, according to the COSMIN guidelines, none of them could be recommended as a reliable and valid instrument for facilitating TCM pattern diagnosis of FD.

Implications for practice

The quality of measurement properties is crucial for ensuring reliability and validity of measurement instruments, including TCM diagnostic instruments. Three measurement properties are commonly validated by the developers of TCM diagnostic instruments: (i) content validity – it refers to whether the instrument covers a broad scope of accepted and relevant TCM diagnostic concepts, contains comprehensible diagnostic items, and demonstrates a high consistency between its diagnostic concepts and diagnostic questionnaire items;, 36, 37, 38 (ii) structural validity – it assesses the strength of statistical relationships between TCM patterns, and the relevant diagnostic questionnaire items;39, 40, 41 and (iii) test-retest reliability – it indicates the consistency of diagnostic results when the instrument is administered twice on the same patient over a while, assuming that the condition would stay stable during the period., 42, 43, 44 Yet, despite the significance of these measurement properties to pattern diagnosis, they were not rigorously validated among the existing TCM diagnostic instruments for FD.

Implications for research

Future researchers should also conduct critical appraisals on the relevance, comprehensibility, and comprehensiveness of TCM diagnostic instruments with the amended COSMIN items proposed by this study, of which the operation procedure is reported in Appendix 2. Assessment criteria for measurement properties in the COSMIN guidelines should also be amended to accommodate the appraisal of TCM diagnostic instruments. Criterion validity demonstrates the degree of which assessment results from a TCM diagnostic instrument is an adequate reflection of result from the “gold standard” test. Nevertheless, considering the absence of a gold standard test in TCM pattern diagnosis,45, 46, 47, 48 as well as the poor inter-rater agreement among TCM practitioners in pattern diagnosis, it is doubtful whether diagnoses made by TCM practitioners should serve as the gold standard for criterion validity assessments of TCM diagnostic instruments. Latent class analysis is commonly performed nowadays to explore TCM patterns, with examples from studies conducted among patients with systemic lupus erythematosus, cardiovascular disease, depression, persistent insomnia, and vascular mild cognitive impairment. Since the construct of TCM patterns revealed by latent class analysis may not be a linear combination of relevant TCM signs and symptoms,, , factor analysis may not be suitable for evaluating the structural validity of those diagnostic instruments. As TCM pattern is a summary of pathological changes in a patient at a specific time point and is constantly changing, such dynamic feature may not favor standard evaluation of test-retest reliability which requires patients to be stable for at least two weeks., To understand the dynamic feature of TCM pattern, prognostic tests may be conducted to study the possible changes of pattern diagnosis along the course of the disease. For example, empirical evidence illustrated that the progression from impaired glucose tolerance to diabetes mellitus resembles the progression from spleen deficiency with dampness encumbrance to spleen-stomach dampness-heat, then eventually to yin and qi deficiency.

Recently, many TCM diagnostic instruments have been developed to enable an systematic and reproducible measurement of TCM diagnostic patterns.17, 18, 19, 20, , 40, 41, 42, , 56, 57, 58 Unlike the disease specific instruments included in this review, some of those instruments measure specific TCM diagnostic construct directly without considering the disease entity per se. The Traditional Chinese Medical Diagnostic Descriptor was developed under this approach, with a goal of measuring TCM patterns regardless of the disease defined under Chinese or conventional medicine. Another example is the Blood Stasis Questionnaire, which measures the presence of blood stasis generically. Comparative diagnostic research should be conducted to compare how different measurement approaches would influence prescription decision, and subsequently, patient outcomes. Last but not least, multisite cross-sectional studies involving patients from countries could be conducted to explore potential geographic variations on pattern diagnosis between populations.

Strengths and limitations

One of the strengths of this study is that searches were performed on seven Chinese and English electronic databases to ensure the comprehensiveness of the literature search. Second, the sensitivity of the literature search was strengthened by the use of validated search filters for measurement instruments. Third, this systematic review was formulated under the COSMIN guidelines with novel amendments made to accommodate the assessment of the development process of TCM diagnostic instruments.

One of the main limitations is that criteria for appraising measurement properties proposed by COSMIN guidelines may not be entirely applicable to TCM diagnostic instruments, as the COSMIN guidelines were mainly developed for the evaluation of patient-reported outcome instruments. That said, systematic reviews on measurement properties of instruments other than patient-reported outcome have been conducted using the COSMIN guidelines.59, 60, 61 Further amendments on the guidelines should be made for appraising TCM diagnostic instruments.

Conclusions

This systematic review revealed that the quality of existing TCM diagnostic instruments for FD is unsatisfactory. We were not able to identify any reliable and valid instruments for TCM pattern diagnosis of FD. Future researchers are recommended to develop and evaluate TCM diagnostic instruments following the guidance of the COSMIN Initiative, with relevant amendments to accommodate the features and uniqueness of TCM diagnostic process. Such research initiative will establish the evidence base of TCM pattern diagnosis, supporting further standardization of diagnostic practice under the auspice of ICD-11.