Randomised clinical trials on acupuncture in the Korean literature: bibliometric analysis and methodological quality.

OBJECTIVE: Acupuncture systematic reviewers have increasingly searched Chinese databases and journals to identify eligible randomised clinical trials (RCTs). However, reviewers have infrequently searched for eligible RCTs in Korean databases and journals. This study aimed to identify difficult to locate acupuncture RCTs in Korean databases and journals and to assess the characteristics and quality of the identified RCTs.
METHODS: Eleven electronic databases and seven journals were searched up to December 2012. All RCTs using needle acupuncture were considered for inclusion. Key study characteristics were extracted and risk of bias was assessed using the Cochrane Collaboration tool.
RESULTS: One hundred and forty-three publications met our inclusion criteria. Acupuncture RCTs in the Korean literature emerged in the mid-1990s and increased in the mid-2000s. Diverse methods of acupuncture were used, including some methods unique to Korea (eg, Saam acupuncture). The largest proportion of trials evaluated acupuncture for musculoskeletal conditions (27.3%). The mean sample size was 44.3±25.3 per trial. Random sequence generation methods were reported in 44.8% of the RCTs, whereas only 11.9% reported methods of allocation concealment. A low proportion of trials reported participant blinding (32.9%) and outcome assessment blinding (18.9%).
CONCLUSIONS: Korean acupuncture trials, many of which evaluate acupuncture styles unique to Korea, are typically omitted from systematic reviews of acupuncture, resulting in the potential for language bias. The development of this database of difficult to locate Korean trials, which includes English language translations of abstracts, will enable these trials of varying quality to be assessed for inclusion in future acupuncture systematic reviews.

Introduction

Acupuncture has an ancient history as a dominant therapeutic intervention in East Asian countries such as China, Korea and Japan.1 The therapy has been widely used for the treatment of various conditions. Acupuncture has recently been increasingly introduced and popularised in Western societies, thus increasing the demand for research on its efficacy. Currently, many clinical trials and reviews on acupuncture are listed in English-based medical databases such as MEDLINE and The Cochrane Library.

Systematic reviews are regarded as the strongest evidence among clinical studies in the field of evidence-based medicine because of the rigorous analytical methods used. The reviews use explicit systematic methods that are selected with a view to minimising bias.2 The identification of all the potentially eligible trials has been described as ‘the most fundamental challenge’ of preparing systematic reviews.3 It is therefore important to conduct an extensive literature search that covers all relevant articles to avoid publication, language and other possible biases.4

As acupuncture is still more prevalent in East Asian countries, a number of clinical trials on acupuncture have been conducted in the region and indexed only in local databases. Because these trials are often not included in MEDLINE, they may not be identified through literature searches and are therefore likely to be omitted from systematic reviews. In a previous study by Kim et al,5 75% of acupuncture-related Cochrane reviews included Chinese databases in their search, whereas only 4% of the reviews included Korean and Japanese databases. This lack of relevant database searches due to language barriers increases the risk of language bias.

To address this concern, there have been attempts to introduce information about Korean clinical studies on acupuncture into Western journals. A 2005 overview summarised ‘clinical case studies’ of acupuncture published in the Korean literature from 1983 to 2001.6 However, this overview included all type of clinical studies, did not report the numbers or characteristics of any randomised clinical trials (RCTs) identified and did not include assessments of the methodologies used in the identified studies. A more comprehensive review of RCTs on acupuncture published in Korea through May 2007 was conducted by our authors in 2009,7 with a more rigorous analytical method and systematic approach. However, the excessively rigid inclusion criteria for the identified articles prevented this review from covering many Korean RCTs. Indeed, because only 10 articles were included in that review, certain trends or tendencies in the publications could not be analysed.

For this reason, we aimed to update our previous work and to provide comprehensive information on Korean acupuncture RCTs to those who need this information. This paper is an extended sample of the Korean acupuncture RCTs from our previous work.7 The study aims to analyse the bibliometric characteristics and the quality of RCTs on acupuncture in Korean literature, with an effort to retrieve and include all of the relevant articles. English translations of the abstracts and PDFs of the full Korean articles of identified trials are contributed to the Cochrane Complementary Medicine Field specialised register of controlled trials, and trial citations are forwarded to The Cochrane Central Register of Controlled Trials (CENTRAL) to increase their accessibility to systematic reviewers. We systematically approached the literature search and selection process as well as the data extraction and quality assessment. Analysis of the characteristics of Korean acupuncture RCTs from this study will be of help to future researchers and will provide preliminary data for acupuncture researchers who are unable to access Korean literature directly.

Methods

Searches

Electronic bibliographic databases and journals

Eleven bibliographic databases were searched electronically and seven Korean journals were searched manually, all from their inception up to December 2012. Korean trials indexed in English-based databases were not considered. There could be studies which were indexed in English-based databases such as MEDLINE or EMBASE, but those sites were not searched. Key features of each database and journal searched are included in online supplementary appendix S1. The search terms used for the electronic databases were ‘(acupuncture OR acupuncture point OR meridian) AND (random OR control)’. Korean language terms relevant to acupuncture or clinical trials were also used.

Other sources

In addition to searching electronic sources, we searched our own paper files, data on clinical trial performances and information from clinical studies supported by the Korea Institute of Oriental Medicine (KIOM). To retrieve grey literature, dissertations and conference proceedings were searched. Finally, we checked the reference lists of relevant articles.

Study selection

Two authors (SK, JCK) conducted the literature search and retrieved citations of studies presumed to be potential RCTs. Full-text articles were obtained for all potentially eligible trials and two authors (SK, BCS) independently checked the eligibility criteria of the full-text articles.

Parallel or crossover RCTs that assessed the efficacy or physiological features of acupuncture were included, regardless of the participants’ medical conditions, language (Korean or English) of publication or publication year. Uncontrolled clinical trials and controlled but non-RCTs, including trials using a quasi-random method of allocation (eg, alternation, date of birth or case record number) were excluded. We predefined acupuncture as penetrating needling on acupuncture points, regardless of acupuncture methods or stimulation types; non-penetrative interventions such as acupressure, transcutaneous electrical nerve stimulator (TENS) or laser acupuncture were therefore excluded but reported separately for future research (see online supplementary appendix S2). We also excluded pharmacopuncture because the efficacy of needling was combined with injected pharmacologically active substances.8 Controls included sham/placebo acupuncture, other types of acupuncture, no treatment or other active intervention. We included RCTs in which co-interventions were combined with acupuncture in the experimental group if the co-interventions were also given equally to the control group(s). However, if the main purpose of the trial was to assess the efficacy of the co-intervention and not the acupuncture, the trial was excluded (see online supplementary appendix S3). All types of participants, even healthy subjects, were included because our research aim was to present all of the features of Korean acupuncture RCTs.

Data extraction and analysis

The characteristics of the included RCTs were analysed and extracted in the following categories: (1) publication year and journal for bibliometric analysis; (2) study design, acupuncture method (acupuncture point selection method), acupuncture stimulation type, structure of acupuncture experimental and control groups; (3) medical conditions of participants; (4) sample size; and (5) whether data on safety were reported. The medical conditions of participants were categorised according to a modification of the International Classification of Diseases 10.9

Risk of bias

Risk of bias was assessed independently by two authors (SK, JCK) using the Cochrane Collaboration's risk of bias tool which includes the following seven domains2: (1) random sequence generation; (2) allocation concealment; (3) blinding of participants and personnel; (4) blinding of outcome assessment; (5) incomplete outcome data; (6) selective reporting (if all prespecified outcomes in a prepublished protocol were reported, the risk of bias was judged low); and (7) other sources of bias. Discrepancies were resolved by consensus through discussion between the two reviewers based on the judgement criteria in Chapter 8 of the ‘Cochrane Handbook for Systematic Reviews of Interventions’2 and, if needed, by asking for further evaluation by a third reviewer (B-CS).

Results

Study description

As shown in online supplementary appendix S4, our searches identified 994 citations of which 283 full-text articles were evaluated in full; 140 full-text articles were excluded, leaving 143 trials that met our inclusion criteria (131 published in Korean and 12 in English, see online supplementary appendix S5). The literature search process is summarised in online supplementary appendix S4, following the PRISMA flow diagram.10

Bibliometric analysis

The first acupuncture RCT published in Korea was in 1989,11 and a second RCT was published in 1996. From then on, the number of publications increased, with the highest number of publications occurring in 2007 (23 RCTs). Since then, there has been a fluctuation in the number of publications per year, with 14 published in 2012 (figure 1).

Figure 1

Number of Korean acupuncture randomised clinical trials by publication year.

The 143 RCTs were published across 17 journals; 128 of the 143 records were published in journals and 15 studies were identified from dissertations. The journals publishing acupuncture RCTs were mostly related to Traditional Korean Medicine (TKM) (11 of 17 journals, 121 of 143 papers; table 1).

Table 1

Number of Korean acupuncture randomised clinical trials published by journal

Journal name	n (%)
Traditional Korean medicine
The Journal of Korean Acupuncture and Moxibustion Society	63 (44.1)
Journal of Oriental Rehabilitation Medicine	19 (13.3)
Journal of Meridian and Acupoint	10 (7.0)
Journal of Korean Oriental Medical Society	8 (5.6)
Journal of Oriental Neuropsychiatry	6 (4.2)
Journal of Korean Oriental Internal Medicine	5 (3.5)
The Journal of Oriental Obstetrics and Gynecology	4 (2.8)
The Journal of Korea Chuna Manual Medicine for Spine and Nerves	3 (2.1)
Korean Journal of Oriental Physiology & Pathology	1 (0.7)
Korean Journal of Oriental Medicine	1 (0.7)
Journal of Oriental Medicine	1 (0.7)
Physical therapy
The Korean Academy of Physical Therapy Science	2 (1.4)
Korean Academy of University Trained Physical Therapists	1 (0.7)
Western medicine
The Journal of Korean Academy of Craniomandibular Disorders	1 (0.7)
The Korean Pain Society	1 (0.7)
Dentistry
The Korean Journal of Thoracic and Cardiovascular Surgery	1 (0.7)
Medical engineering
Journal of Biomedical Engineering Research	1 (0.7)
Studies not published in journals
Master's thesis	11 (7.7)
PhD thesis	4 (2.8)
Total	143 (100.0)

Study design, types of interventions and controls

The study designs, acupuncture styles, acupuncture stimulation methods and types of controls are shown in table 2. Parallel designs (n=131; 91.7%) were more commonly used than crossover design RCTs (n=12; 8.3%), with the two-arm parallel design predominating (n=100; 69.9%).

Table 2

Study design and types of interventions in Korean acupuncture randomised clinical trials

Characteristics	n (%)
Study designs	143 (100.0)
Parallel, two-arm	100 (69.9)
Parallel, three-arm	25 (17.5)
Parallel, four-arm	4 (2.8)
Parallel, five-arm	1 (0.7)
Parallel, six-arm	1 (0.7)
Crossover	12 (8.4)
Acupuncture methods (acupuncture point selection)	143 (100.0)
Classical acupuncture	104 (72.7)
Saam acupuncture	17 (11.9)
Microsystem acupuncture
Auricular acupuncture	5 (3.5)
Scalp acupuncture	3 (2.1)
Foot acupuncture	1 (0.7)
Wrist-ankle acupuncture	1 (0.7)
Master Dong's acupuncture	7 (4.9)
Constitution acupuncture*	4 (2.8)
Pyung-Hyung (equilibrium) acupuncture	1 (0.7)
Acupuncture stimulation	143 (100.0)
Manual acupuncture	110 (76.9)
Electroacupuncture	25 (17.5)
Intradermal acupuncture (embedding in the skin)	4 (2.8)
Burning acupuncture (needle heated by flame)	1 (0.7)
Warm needling acupuncture (needle warmed by moxibustion)	1 (0.7)
Electromagnetic acupuncture	1 (0.7)
Aroma acupuncture	1 (0.7)
Acupuncture vs control design†
Acupuncture vs other acupuncture	58 (36.7%)
Acupuncture vs sham acupuncture	39 (24.7%)
Acupuncture vs no treatment	32 (20.3%)
Acupuncture vs other intervention‡	19 (12.0%)
Acupuncture+other intervention‡ vs other intervention‡	10 (6.3%)

*Constitution acupuncture includes Sasang constitution acupuncture (n=2), 8-constitution acupuncture (n=1) and Taegeuk acupuncture (n=1).

†Individual trials could use multiple types of control groups.

‡Other interventions include physical therapy, Western medication, herbal medicine, Kegel exercises, moxibustion, Chuna manipulation, transcutaneous electrical nerve stimulator, ultrasound, venesection, ICT, stellate ganglion block and analgesics.

Diverse acupuncture styles were used in the RCTs, reflecting the acupuncture trends in Korea. Although classical acupuncture was used in most of the studies (n=104; 72.7%), methods unique to Korea such as Saam acupuncture and constitution acupuncture12 were used in 21 trials (14.7%).

Manual stimulation of the needle was the most frequently used form of stimulation for acupuncture (n=110; 76.9%), followed by electroacupuncture (n=25; 17.5%), intradermal acupuncture (n=4; 2.8%) and other techniques. Intradermal acupuncture refers to a specially made short needle that is embedded in the skin or subcutaneously for a relatively long time (less than 1–3 days, twice a week is general in clinical practice).

Controls included other types of acupuncture (to compare the effects of different acupuncture methods) (n=58; 3.7%), sham acupuncture (n=39; 24.7%), no treatment (n=32; 20.3%) or other active interventions (n=19; 12.0%). There were 10 RCTs (6.3%) in which another intervention was applied to both groups while acupuncture was applied only to the experimental group (table 2).

Medical conditions

Of the included trials, 74.1% were classified into one of 12 medical condition categories with the greatest concentrations being in the categories musculoskeletal conditions, circulatory conditions and nervous system conditions. The remaining 25.9% of trials were conducted in healthy participants (table 3).

Table 3

Categories of medical conditions in Korean acupuncture randomised clinical trials

Medical conditions	n (%)
Musculoskeletal conditions	39 (27.3)
Low back pain (including disc disorder)	13
Shoulder pain	7
Sprain (ankle, wrist)	7
Knee osteoarthritis	6
Neck pain	2
Myalgia	2
Elbow joint pain	1
Axial apondyloarthritis	1
Circulatory conditions	17 (11.9)
Stroke	15
Essential hypertension	2
Nervous system conditions	14 (9.8)
Bell's palsy	5
Headache	5
Parkinson's disease	3
Dizziness	1
Genitourinary conditions	8 (5.6)
Primary dysmenorrhoea	4
Hot flush	2
Premenstrual tension syndrome	1
Stress incontinence	1
Mental and behavioural disorder	6 (4.2)
Unspecified mood disorder	4
Anxiety disorder	1
Tobacco use	1
Ear, nose and throat conditions	5 (3.5)
Temporomandibular joint disorder	2
Allergic rhinitis	2
Tinnitus	1
Digestive conditions	3 (2.1)
Dyspepsia	2
Constipation	1
Motor or non-motor vehicle accident	3 (2.1)
Endocrine, nutritional and metabolic conditions	3 (2.1)
Obesity
Postprocedural disorders	3 (2.1)
Nausea, vomiting, pain after surgery
Symptoms, signs and abnormal clinical and laboratory findings not elsewhere classified	3 (2.1)
Fatigue
Conditions of the skin and subcutaneous tissue	2 (1.4)
Acne vulgaris
Healthy participants*	37 (25.9)

*Healthy participants were examined for autonomic nervous system function (n=17), acupuncture sensation comparison (n=5), muscle function (n=4), stress (n=3), sensation threshold (n=2), recovery after exercise (n=2), signal activation in the brain (n=1), suppressed gastrointestinal motility (n=1), sweat control (n=1) and weight reduction (n=1).

Sample size

The total number of participants from the 143 included RCTs was 6332. Sample size per trial ranged from 8 to 175, with a mean ± SD of 44.3±25.3. Sample size per arm (total 330 arms) ranged from 4 to 116, with a mean ± SD of 19.3±11.9. Seven RCTs only stated the total sample size and did not specify the sample size of each group at the randomisation period. Thus, when calculating the mean and SD of the sample size per arm, we divided the total sample size by the number of arms for those seven trials.13–19 Statistical power analysis was conducted in only three studies (2.1%).20–22

The risk of bias of the included RCTs is shown in table 4. A random sequence was adequately generated in 44.8% (n=64) of the included RCTs. Many studies did not specify how the random sequence was generated and they were estimated as unclear risk of bias; 88.1% of the RCTs (n=126) did not describe allocation concealment. Examples of participant blinding were sham needle or superficial needling on non-acupuncture points. The proportions of trials in which participants were blinded (n=47; 32.9%), outcome assessors were blinded (n=27; 18.9%) and incomplete outcome data were addressed (n=42; 29.4%) were low. Almost none of the trials were free of selective outcome reporting; only three protocols were published, which were estimated to have low risk of bias. Almost none of the trials were free of other risk of bias either; only two trials clearly stated on every method that was applied to minimise biases whenever applicable.

Table 4

Methodological quality of Korean acupuncture randomised clinical trials according to Cochrane risk of bias

Bias	Low risk of bias	High/unclear risk of bias
	n (%)	n (%)
Random sequence generation	64 (44.8)	79 (55.2)
Allocation concealment	17 (11.9)	126 (88.1)
Blinding of participants and personnel	47 (32.9)	96 (67.1)
Blinding of outcome assessment	27 (18.9)	116 (81.1)
Incomplete outcome data	42 (29.4)	101 (70.6)
Selective reporting	3 (2.1)	140 (97.9)
Other sources of bias	2 (1.4)	141 (98.6)

Safety report

Only 23.8% (n=34) of the RCTs included descriptions of adverse events associated with acupuncture. Those that did report adverse events commented that events were mild or that there were no adverse events at all.

Discussion

This study is an analysis of acupuncture RCTs published in the Korean literature. The trials assessed the effectiveness of various types of acupuncture in a variety of medical conditions. RCTs of acupuncture in the Korean literature slowly emerged in the mid-1990s and increased markedly in the mid-2000s. From the literature search, we found that there were substantial numbers of clinical trials on acupuncture in Korean literature, even though Korean trials indexed in English-based databases were not considered.

Although acupuncture RCTs in Korea have strength in quantity, the high risk of bias remains their weakness. As mentioned above, a random sequence was adequately generated in 44.8% of the included RCTs whereas only 11.9% concealed allocation. The proportions reporting double blinding or blinding of outcome assessments were low, as were the proportions addressing incomplete outcome data. Almost none of the included RCTs were free of selective outcome reporting. Studies with a high risk of bias are likely to distort the outcomes and exaggerate the treatment effects.23

RCTs published in other East Asian countries such as China and Japan have also been shown to have high risks of bias. In a study to investigate the adequacy of randomisation in RCTs published in Chinese journals,24 only 21.6% of Traditional Chinese Medicine (TCM) RCTs conducted at medical universities and 12.4% of RCTs conducted at level 3 hospitals were considered authentic RCTs. Similar results were shown in an earlier study25 which reported only 15% of the RCTs to be blinded. Even lower methodological quality was reported in a study using the Consolidated Standards of Reporting Trials (CONSORT) checklist26 which identified only 7.9% of TCM RCTs published in China to have reported methods of generating a randomisation sequence. Studies evaluating acupuncture clinical trials in Japan show similar results to those in Korea.27 Among 33 trials described as randomised, seven of the trials have been identified as quasi-randomised.

Although it remains controversial whether non-English trials, which may have a higher risk of bias, should be included in systematic reviews, previous studies suggest that there is no significant difference between English trials and non-English trials in terms of methodological quality.28 Thus, it should be recommended that systematic reviews include studies regardless of publication language because a core component of systematic reviews is to conduct an extensive literature search that covers all relevant articles. To dispel the controversy, well-designed RCTs on acupuncture with rigorous methodology must be conducted by Korean or other East Asian authors.

From this study, characteristics of Korean acupuncture were identified. Korea has developed individualised acupuncture treatment methods such as Saam, Taegeuk, Sasang constitution or 8-constitution acupuncture. Various needle stimulation methods, such as burning acupuncture or warm needling acupuncture, also contribute to the diversity of Korean acupuncture. This diversity was reflected in the study designs, where the most frequently used control design in the included RCTs compared one type of acupuncture with another type of acupuncture.

A great number of Korean databases and relevant journals were searched for this study, allowing the largest number of RCTs on acupuncture in Korean literature to be evaluated. Categorised information from this study may be of help to future researchers. The number of articles related to intervention type or various medical conditions can be used for research. The search can also be used to indicate directions for further research—for example, if trials on commonly used acupuncture interventions or in medical conditions commonly treated with acupuncture are insufficient, this may imply that research resources should be assigned to these areas.

There are several limitations to this study. First, in spite of our effort to retrieve all relevant articles, we cannot be certain that our search was all-inclusive. No standard search filter exists in Korean databases, hindering an efficient and complete search for the records. Therefore, a researcher should review papers one by one; this study is a summary of Korean acupuncture that was completed via a manual search.

Another limitation is that we could not draw conclusions on the efficacy of acupuncture from this study. As mentioned above, the most frequently used control design in the included RCTs was comparing one type of acupuncture with another type of acupuncture. Thus, evaluating the clinical effectiveness of acupuncture in these cases was impossible or not applicable. Another control design in the trials compared acupuncture with another intervention. In many of these trials the efficacy of the intervention used as a control or the other type of acupuncture used as a control has not been evaluated by RCTs, which prevents us from demonstrating the efficacy of the targeted acupuncture as well. To overcome this limitation, another control group such as a sham acupuncture group should be added. Lastly, because more than two interventions were simultaneously performed in some trials, it was not possible for these trials to demonstrate the effectiveness of acupuncture independent of other treatments.

Conclusions

This review has identified and evaluated the largest number of difficult to locate acupuncture RCTs in the Korean literature to date. Acupuncture systematic reviewers who cannot access Korean databases and journals might review the compiled list of these Korean RCTs (see online supplementary appendix S5) to identify any that are potentially eligible for their reviews. Citations of potentially eligible trials can also be accessed through searching the CENTRAL database in The Cochrane Library. These RCTs on acupuncture in the Korean literature have assessed the effectiveness of various types of acupuncture on a variety of medical conditions, reflecting the diverse application of acupuncture in Korean clinical practice. Although these RCTs have increased in number, there is great room for improvement in their methodological quality.

Systematic reviews of acupuncture rarely include Korean literature.

We conducted an exhaustive search of databases.

We located 143 RCTs with 6332 participants; 73% used classical acupuncture.