An international expert survey on acupuncture in randomized controlled trials for low back pain and a validation of the low back pain acupuncture score.

BACKGROUND: Acupuncture is a promising treatment approach in patients with chronic low back pain (cLBP) but little is known about the quality of acupuncture in randomized controlled trials (RCT) of acupuncture cLBP. -
OBJECTIVE: To determine how international experts (IES) rate the quality of acupuncture in RCTs of cLBP; independent international validation of the Low Back Pain Acupuncture Score (LBPAS).
METHODOLOGY: Fifteen experts from 9 different countries outside China were surveyed (IES). They were asked to read anonymized excerpts of 24 RCTs of cLBP and answer a three-item questionnaire on how the method of acupuncture conformed to 1) Chinese textbook standards, 2) the expert?s personally preferred style, and 3) how acupuncture is performed in the expert?s country. Likert scale rating, calculation of the mode for each answer, and Spearman?s rank correlation coefficient between all three answers and the LBPAS were calculated.
RESULTS: On comparison with Chinese textbook standards (question 1), 6 RCTs received a good rating, 8 trials a fair and 10 trials a poor or very poor rating. 5 of the 6 trials rated good, received at least a good rating also in question 2 or 3. We found a high correlation of 0.85 (p<0.0001) between the IES and LBPAS ratings for question 1 and question 2, and a correlation of 0.66 (p<0.0001) for question 3.
CONCLUSION: The international expert survey (IES) revealed that only 6 out of 24 (25%) RCTs of acupuncture for cLBP were rated "good" in respect to Chinese textbook acupuncture standards. There were only small differences in how the acupuncture quality was rated in comparison to Chinese textbook acupuncture, personally preferred and local styles of acupuncture. The rating showed a high correlation with the Low Back Pain Acupuncture Score LBPAS.

Introduction

Acupuncture is a promising treatment approach in patients with low back pain (cLBP) with nearly the same or even better results compared to conventional therapy [10,31,3,35,40,18]. Since in fact various acupuncture strategies are used in clinical practice, research in order to define the best acupuncture technique is of clinical interest for many medical practitioners as well as for clinical researchers designing acupuncture trials [45,29,34]. Allready for Chinese acupuncture, representing the most widely used style, the specific manner in which acupuncture is applied (treatment regimens, point selection, needle techniques) appears to vary among, regions and countries [45,23]. Recommendations in the literature concerning point selection and needle techniques differ significantly [38,22,39,46,1,5,47,20,50,13,27,51,37]. The same holds true for randomized clinical trials (RCT) on cLBP, with point selection being fixed for all patients, or points being selected on an individual basis according to trigger points, Chinese meridians, syndromes, various microsystems to name a few [25,35]. In RCTs needle stimulation may range from non-existent to strong, needle insertion may be deep or shallow, the number of acupuncture treatments may vary from 1 to 15, and the stimulation of Ahshi points may be mandatory or forbidden. Despite the multiplicity of existing approaches to acupuncture, in a previous study we ascertained that there is a high degree of concurrence on certain aspects (minimal criteria) of Chinese acupuncture for the treatment of cLBP [36]. A systematic review by Yuan et al. arrived at a similar conclusion, while detecting differences between Chinese expert opinions, textbooks and acupuncture treatments in RCTs [49].

On the basis of a broad, international Delphi survey the aim of this study was to find out how leading acupuncture experts from non-Chinese countries would rate the acupuncture methods used in existing RCTs of acupuncture for cLBP. Furthermore we calculated the correlation between the results of this expert survey and with a Low Back Pain Acupuncture Score (LBPAS), derived from our recently published Acupuncture Questionnaire for Low Back Pain [36].

Method

Selection of international acupuncture experts

Acupuncture experts in a number of countries were selected on the basis of personal contact and scientific publications in the field of acupuncture. Additional criteria were clinical experience, a history of teaching, and textbook authorship. All experts were invited to recommend other experts to take part in the survey. Experts were contacted by email and telephone. Those who had already taken part in our previous acupuncture survey establishing minimal criteria for Chinese acupuncture in cLBP were excluded from selection [36].

Selection of randomized controlled trials (RCT) of acupuncture for cLBP

Twenty-five papers on RCTs of acupuncture for cLBP published between 1976 and 2007, were identified via a computerbased search of the Cochrane Complementary Field Trials Register, the Cochrane Controlled Trials Register, Medline, Embase and reference lists of articles. The acupuncture method for one of the 24 trials was published in two different papers (papers 2 and 10). We included both papers in order to explore the intra-rater differences [2,3]. For the purpose of this study we did not aim at completeness but sought to ensure that all major trials included in the recent meta-analyses of RCTs on cLBP were included [10,31]. From each trial, the paragraph describing the acupuncture method was extracted, and supplemented by a table summarizing all relevant acupuncture treatment data for that trial; a pdf document was created presenting this information for each trial in an anonymous way, thus minimizing the possibility that the surveyed experts would be able to identify the authors of the reported studies, which could be considered as a relevant source of bias.

Evaluation by International Expert Survey (IES)

The pdf document was sent to the selected experts, asking them to answer the following three questions about the acupuncture treatment method of each trial....

Please specify how you rate the acupuncture treatment for chronic low back pain in this trial....

1. ... according to your personal knowledge of Chinese acupuncture as you have studied it from textbooks on Chinese acupuncture.

2. ... according to your personal concept of individualized point selection (e.g. dry needling, trigger point stimulation, individual usage of Chinese point selection rules such as Ahshi points, channels, syndromes, pathogenic factors...)

3. ... according to the acupuncture commonly performed in your country (local style, not necessarily Chinese style).

Answers were to be given on a Likert scale [26,8]: 1 = very poor; 2 = poor; 3 = fair; 4 = good; 5 = very good

If the experts felt that the excerpt for a particular trial did not contain adequate information about the acupuncture treatment, they were asked to mark: "The trial does not show a sufficient amount of information for any further judgment. Cannot say".

Evaluation based on the Acupuncture Questionnaire for Low Back Pain/Low Back Pain Acupuncture Score (LBPAS)

Recently we published the 'Acupuncture Questionnaire for Low Back Pain' based on an international expert survey [36]. This survey revealed that a broad consensus exists with regard to the fundamental aspects of Chinese acupuncture for cLBP across different categories of practitioners and different countries. A typical treatment regime would consist of 11 sessions of 25 minutes each, given twice weekly. The practitioner would insert 12 needles, select points according to channels and syndromes, identify syndromes such as kidney deficiency (yin and yang), cold dampness, and Qi and blood stagnation. Preferred points besides Ahshi points are BL 23, BL 40, GB 34, BL 54, BL 60, GB 30 and Huatuo, although there is a high degree of variation concerning additional local and distal points. Needling depth is dependent on the point location or other individual patient factors, varying from 0.5 mm in very sensitive patients to 10 cm for points such as GB 30 or BL 54. Acupuncture should be combined with other techniques of Chinese medicine, the favourites being electrical stimulation and moxibustion. Based on the Acupuncture Questionnaire for Low Back Pain, a score was developed in such a way that the conformity of the acupuncture treatment applied in a given trial with the criteria derived from the Acupuncture Questionnaire for Low Back Pain could be expressed in a single number from 0 to 8, from 0 = very poor to 8 = very good (Table 1).

Table 1

Low Back Pain Acupuncture Score: LBPAS.

Item	1 point	0 point
Selected points mentioned	yes	no
Ahshi point used	yes	no
No. of needles inserted	≥ 12	< 12
Depth of needling mentioned	yes	no
DeQi required	yes	no or not mentioned
Needle retention time mentioned	yes	no
Treatments per week	2 or more	≤ 1 or not mentioned
No. of treatment sessions	≥ 10	< 10 or not mentioned

Previously published data from 18 experts from 10 different countries showed that a typical acupuncture treatment regimen or treatment protocol should at least meet the above criteria. These results were the basis for the development of the Low Back Pain Acupuncture Score [36]

This number is referred to as the Low Back Pain Acupuncture Score (LBPAS). For the present study, we (2nd author) calculated the LBPAS for all 25 publications and compared it to the IES score.

Statistics

For each trial, the frequencies of the ratings given by the 15 experts were counted. For each question, the most frequently selected rating (mode) was identified. The mode was defined as the true rating, since it had been selected by the majority of the experts.

For all 25 papers we then counted how often experts missed the mode for each question (false rating). Finally the number of false ratings per expert was counted. The number of false ratings per expert is presented with its measures of average and variability.

We calculated the Spearman's rank correlation coefficient of the expert's true rating for the three questions of the IES. The same statistical procedure was used to calculate the correlation between the IES and the LBPAS.

Results

Twenty-five publications on 24 RCTs were selected and evaluated by 15 out of 18 experts from the following countries: Australia (2), Austria (1), England (3), France (1), Germany (2), Korea (1), Norway (1), Spain (2), and US (2). Three declined to take part, two because lack of time (France, China), the other (China) reasoning that acupuncture treatment is a holistic approach and therefore information would have to be presented about every single patient's TCM diagnoses for all complaints, not merely cLBP. Since ultimately we could not include a Chinese expert in our survey, we restrict our results to non-Chinese experts and countries.

For each paper we determined the rating given by the majority of experts in each question. This rating was considered the true rating (mode). Conformity of the acupuncture method with Chinese textbook standards (question 1) was rated good (rating 4 or 5) in 6 trials, fair (rating 3) in 8 trials, and poor or very poor (rating 1 or 2) for 10 trials. Five of the 6 trials rated good [on question 1] received at least a good rating also in questions 2, asking the experts to grade the degree to which the method in the trial accorded with their personally preferred style, or - question 3 - the country-specific acupuncture style. Here too 6 of 10 trials that were rated poor or very poor received the same rating for all three questions. The results for each trial, differentiated by question, are presented in Table 2.

Table 2

Expert rating of all trials.

Trial No.	Mode Question 1	Mode Question 2	Mode Question 3	LBPAS	LBPAS Rating
1 [42]	2	3	2	5	fair
2 [3]	4	4	4	7	good
3 [19]	3	3	3	6	fair
4 [9]	2	1	2	4	poor
5 [14]	3	1	3	3	poor
6 [7]	1	1	2	2	poor
7 [11]	3	2	3	6	fair
8 [12]	1	1	1	1	poor
9 [4]	4	4	4	6	fair
10 [2]	4	4	4	7	good
11 [25]	4	3	3	7	good
12 [24]	2	1	1	1	poor
13 [48]	3	2	2	5	fair
14 [6]	2	2	2	3	poor
15 [35]	4	4	4	8	good
16 [28]	2	2	3	4	poor
17 [21]	2	2	3	3	poor
18 [17]	1	1	2	3	poor
19 [16]	2	2	3	2	poor
20 [15]	3	3	2	6	fair
21 [32]	3	2	4	4	poor
22 [33]	4	4	4	5	fair
23 [41]	3	4	3	6	fair
24 [44]	3	2	3	5	fair
25 [18]	4	4	5	8	good

Trial No. = number of trial. [] = reference number. Questions: "Please specify how you rate the quality of acupuncture treatment in this trial for chronic low back pain according to Question 1... Chinese acupuncture as you have studied it from textbooks on Chinese acupuncture? Question 2... according to your personal concept of individualized point selection? Question 3... according to the acupuncture commonly performed in your country? Answers were to be given on a five point Likert scale: 1 = very poor; 2 = poor; .3 = fair; 4 = good; 5 = very good. LBPAS = Low Back Pain Acupuncture Score, from minimum of 0 to a maximum of 8 points. LBPAS rating: 1 to 4 = poor; 5 and 6 = fair; 7 and 8 = good and very good.

We also counted how often each expert rated differently from the mode, which was defined as a false rating. The number of false ratings for all three questions was similar. In the median, the experts made 8 false ratings in question 1 and 9 false ratings in questions 2 and 3. That means that one third of all ratings were mistaken. The minimum number of false ratings was 4 for question 2, and 5 for questions 1 and 3. The maximum number of false ratings was 12 for question 2, and 13 for questions 1 and 3. The range of false ratings for all three questions was 8. The variability, as shown by the coefficient of variation, was low for all three questions and almost identical at nearly 24%.

We could not detect any intra rater variability. Regarding papers 2 and 10, the mode was 4 for each of the three questions in both papers.

There was a high correlation between the rating of question 1 (conformity with Chinese textbook acupuncture), question 2 (conformity with individual point selection) and question 3 (conformity with acupuncture as performed in the expert's country). For 8 trials (nos. 2, 3, 8, 9, 10, 14, 15 and 22), the same rating was chosen for questions one, two and three. For all other trials the difference between the ratings was one mark only (Table 2). The Spearmann's ranking correlation coefficient is 0.81 for question 1 to question 2, and 0.80 for question 1 to question 3. For question 2 to question 3 it is 0.72 (two-sided P value < 0.0001 for all correlations).

Rating with the Low Back Pain Acupuncture Score (LBPAS)

With the LBPAS, the conformity of a given acupuncture method with the criteria derived from the low back pain survey published earlier is expressed as a single number from 0 (very poor) to 8 (very good). Two trials had an LBPAS of 8 (15, 25), and two had an LBPAS of 7 (2/10, 11). The lowest LBPAS was 1, for two trials (8, 12) (Table 2).

Correlation of the International Expert survey (IES) and the LBPAS

A high correlation was found between the IES rating and the LBPAS for questions 1 and 2 (0.85, P < 0.0001), and a slightly lower correlation was found for question 3 (0.66, P < 0.0001).

Discussion

Using excerpts of 24 RCTs of chronic low back pain, we could show that there is a high correlation between how experts outside of China rate the acupuncture treatment compared to Chinese textbook acupuncture standards, personally preferred acupuncture style, and local acupuncture style of their respective country. Only a minority of 6 out of 24 trials (25%) were rated good according to Chinese acupuncture standards; 10 were rated poor or very poor and 8 were rated fair. On average each expert agreed with the majority rating in 16 out of 24 trials. Furthermore, the international expert evaluation correlates highly significantly with the LBPAS, based on our recently published Acupuncture Questionnaire for Low Back Pain [36], thereby providing an independent international validation of the LBPAS score.

The rating of all three questions correlates significantly. Nevertheless, the greatest difference in ratings was found between question 1 (Chinese acupuncture) and question 3 (acupuncture as performed in the respective country of the expert). This suggests that local styles do indeed differ from Chinese textbook acupuncture in some aspects but that the (non-significant) differences are smaller then expected. The expert ratings of the methods used in the various RCTs differed most strongly for RCTs where the treatment was not described in sufficient detail according to the STRICTA criteria, thus adding further evidence to the argument that a complete description of acupuncture treatment in RCTs is essential [30]. We also found that RCTs that made use of electro-acupuncture were generally rated higher than those that used only manual needle stimulation.

Yuan et al. recently published a systematic review of acupuncture treatment regimens used for low back pain, including nine Chinese expert opinions. For chronic unspecific low back pain, the main findings, which were in line with ours, were the frequent use of BL23, BL25 and BL40 as common acupuncture points, use of Ashi and trigger points, De Qi sensation, needling of about 10 points per treatment, with a needle retention time of about 20 minutes and a treatment number of 10. In general Chinese experts tended to use fewer points (median 5), treated a little longer (median 25.5 minutes) and would give up to 5 treatments per week [49].

Our data are also in line with other major systematic reviews and meta-analyses on acupuncture for cLBP, although in these reviews the rating either concerned only methodological characteristics of the study design, omitting an evaluation of the acupuncture treatment per se (van Tulder et al.), or judged the acupuncture treatment by three experienced acupuncturists, being coauthors, only (Furlan et al.). For example, Tulder et al. 1999 found, that only 2 of 11 studies met the level of "high quality" and Furlan et al. 2005 found 14 of 35 studies to be of "higher quality" [43,10].

It must be said that a high rating of the acupuncture treatment is one but not the only prerequisite of a high quality RCT. While this might be self-evident for the majority of readers, large, high-powered randomized controlled trials of high methodological quality are often based on the particular acupuncture treatment standards of the authors, which are not necessarily compatible with Chinese acupuncture itself. Results of these trials therefore do not apply to Chinese textbook acupuncture techniques, although this is not always clarified. Future investigations should determine whether ratings attributed by major meta-analyses and systematic reviews have introduced a bias in favour of meeting methodology against commonly accepted acupuncture standards in RCTs about cLBP.

Limitations

Our data is specific to Chinese acupuncture for cLBP. It was collected using a one-step standard Delphi survey of 15 acupuncture experts in nine different countries outside of China. It might be argued that 15 experts is not a sufficient number to be representative, that the respondents were not selected randomly, and that they were not equally distributed across the countries involved. While this is true, and we concede that other experts, e.g. from China, might come up with different ratings, we note that other systematic reviews or meta-analyses of acupuncture for cLPB have selected between 0 and 3 acupuncture experts only, drawn from the pool of authors or the local region. These limitations aside, we know of no other study to work with as many non-Chinese experts, drawn from as wide a range of different countries. Furthermore, the correlations among experts, professions and countries are so high for the ratings of most RCTs that it is unlikely that any other selection of experts would produce essentially different results.

Conclusion

This international expert survey (IES) showed that only in a minority of RCTs of acupuncture for low back pain the acupuncture quality was rated as good. Among international experts there is a high consistency in the rating of acupuncture in respect Chinese acupuncture textbook standards; rating differences with regard to personally preferred methods or local styles of acupuncture do exist but are not significant. The ratings showed a high correlation with the Low Back Pain Acupuncture Score (LBPAS), constituting an independent international validation of the score.

Appendix

We are thankful for the contributions of Weihong Li, David S. White (Australia), Eva Maria Wolkenstein (Austria), Hyangsook Lee (Korea), Jean-Marc Stephan (France), Michael Koch, Friedrich Molsberger (Germany), Panos Barlas, George Lewith, Adrian White (Great Britain), Morten Sørlie (Norway), Caridad Ortega Garcia, Jose Manuel Aranda Regules (Spain), Remy Coeytaux and N.N. (USA).