Evaluating the Characteristics, Reporting and Methodological Quality of Systematic Reviews of Acupuncture for Low Back Pain by Using the Veritas Plot.

OBJECTIVE: To evaluate systematic reviews (SRs) of acupuncture for low back pain (LBP) in terms of characteristics, reporting and methodological quality using a Veritas plot and to explore factors that may be associated with methodological quality and reporting quality. STUDY DESIGN AND
SETTING: We searched 8 electronic bibliographic databases to find all SRs, and we evaluated the SRs' quality in 6 dimensions, including publication year, design type, homogeneity, risk of publication bias, methodological quality by Assessment of Multiple Systematic Reviews (AMSTAR) 2 and reporting quality by Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA). Excel 2010 and Adobe Illustrator CC were used to draw and optimize Veritas plots. Exploratory analysis was done using SPSS software version 23.0 to explore factors related to AMSTAR-2 and PRISMA scores. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) evidence quality evaluation tool was used to grade all the outcome indicators in the included literature.
RESULTS: We included 19 SRs in the analysis. Literature quality rank scores ranged from 9.67 to 17.00, with an average score of 13.18 ± 2.35. The average score of AMSTAR-2 was 7.47, and the average score of PRISMA was 18.47. Overall, the main issues were research strategies, inclusion and exclusion criteria, publication bias, and registration in PROSPERO. The results of exploratory analysis showed that duplication of literature selected and appropriate tools to assess the risk of bias were related to the AMSTAR-2 score, and the summary of evidence was related to the PRISMA score. The GRADE quality evaluation results showed mainly low quality.
CONCLUSION: The quality of SRs on acupuncture for low back pain should be improved, mainly by strengthening the methodological quality and reporting quality. The Veritas plot is an effective graphical evaluation method that is worth popularizing.

Introduction

As a common and disabling symptom, low back pain (LBP) affected approximately 7.3% of people around the world in 2015.1,2 In the United States, the increase in personal and public health-care costs from 1996 to 2013 indicated an estimated spending of $87.6 billion on the management of lower back and neck pain, the third and fourth highest health-care costs in the disease category.3 Guidelines recommend using medication, imaging, and surgery prudently and suggest that clinicians should consider nonsteroidal anti-inflammatory drugs as first-line therapy.4 However, long-term use of nonsteroid anti-inflammatory drugs may cause gastritis, and high-dose aspirin use may cause tinnitus.5 Therefore, greater emphasis is now placed on education and self-care, physical and psychological therapies, and some forms of complementary medicine.6

In recent years, an increasing number of complementary and alternative therapies have been developed in clinical practice, among which acupuncture plays an important role.7 As a safe and acceptable form of acute analgesia, acupuncture can relieve the symptoms of LBP.8–10 The analgesic mechanism of acupuncture is to stimulate sensory nerve endings, leading to the release of endogenous opioid hormones and other nonopioids in the brain and spinal cord,11–13 which could block the transmission of nerve impulses and thereby relieve pain.14 In the American College of Physicians guideline, six related SRs and RCTs about acupuncture were included on noninvasive treatments for LBP, the oldest of which was published in 2002.15 The guideline still has some limitations because it has not systematically evaluated the SRs of acupuncture for LBP. Therefore, a scientific quality assessment of SRs is necessary to overcome the limitations of the guideline and to facilitate decision-making by clinicians. First used as an evidence-synthesis graphical tool in meta-analysis of cardiac surgery,16 Veritas plots are used to describe multiattribute data and to identify and interpret variability in meta-analyses,17 as well as to assess the key factors of meta-analysis quality such as heterogeneity, study design and publication bias. In addition to Veritas plots, we used the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach to assess the quality of evidence from the main findings.18 As a comprehensive graphical tool, Veritas plots could help researchers draw conclusions more intuitively from systematic reviews, so that decision-makers can more easily appraise the quality of meta-analyses, helping them apply high-quality evidence about acupuncture for LBP. Hence, this overview could provide a reference for evidence-based medical research on acupuncture for LBP.

Methods

Study Design of Eligible Studies

This study evaluated the SRs of acupuncture for LBP in six dimensions, and the study protocol has been registered on the PROSPERO platform; the registration number is CRD42019122610 (, CRD).

Search Strategy

We searched 8 electronic bibliographic databases: PubMed, Embase, Cochrane Library, Web of Science, China National Knowledge Infrastructure (CNKI), Chinese Biomedical Literature Database (CBM), Wanfang Data Knowledge Service Platform and Chinese Technical Periodicals (VIP) Database from inception to May 19th, 2019. The search strategy included only terms relating to or describing the intervention, and MeSH and free text terms were used to identify relevant literature. The search terms (acupuncture OR electro-Acupuncture OR electroacupuncture OR abdominal acupuncture OR auricular acupuncture OR scalp-acupuncture OR acupuncture treatment OR acupuncture points OR ear acupuncture OR abdominal acupuncture OR needle OR dry needle OR meridian acupoint OR jingluo OR zhenjiu OR zhenci OR dianzhen) AND (low back pain OR back pain OR lumbar near pain OR dorsalgia OR backache OR back disorder OR sciatica OR lumbago OR back disorder OR low discomfort OR lower back pain) AND (Meta-analysis OR systematic review) were used in English-language databases, while “jingluo”, “zhenjiu ”, “zhenci”, “dianzhen”, “xiayaotong”, “yaotong”, “beitong”, “xitongpingjia”, and “meta” were used in Chinese-language databases. The search details are presented in .

Inclusion and Exclusion Criteria

The inclusion criteria were the following: (1) population: patients with LBP; (2) interventions: acupuncture was evaluated as monotherapy or part of combination therapy, which included all kinds of acupuncture, regardless of its frequency or duration; (3) comparisons: pharmacotherapies, surgery, manipulation, placebo, or a different acupuncture treatment were considered; and (4) study design: SR. Studies meeting the following were excluded: (1) letters, editorials and expert opinions, case reports, abstracts only, and conference proceedings; (2) articles with no extractable data available; (3) articles published not in English or Chinese; and (4) articles unrelated to acupuncture or low back pain. Moreover, if the same author and/or institution was reported in more than one study, only the most recent study or largest population was included.

Data Extraction and Management

We used EndNote X9 software () for screening, to exclude duplicate studies, and to further screen after reading the information. Two researchers (F. H. and M. W. Q.) independently extracted data using a Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA) flowchart19 and Microsoft Excel 2010 (). The following were extracted from the included studies: author, country, condition, participants, interventions, methodological quality assessment tool and main conclusions. In case of disagreements, a third author (S. Y. Z.) participated in consensus conferences.

In order to analyze the factors related to Assessment of Multiple Systematic Reviews 2 (AMSTAR-2) and PRISMA scores and their effect sizes, we used multiple linear and ordinal regression analyses to model AMSTAR-2 and PRISMA scores as dependent variables. Only variables with p ≤ 0.10 on univariate analysis were included in the multivariate regression model to identify significant variables (p ≤ 0.05). Linear and ordinal regression analysis was performed using SPSS software version 23.0.

Quality Assessment

Two authors (F. H. and M. W. Q.) independently evaluated the methodological quality of the included studies using AMSTAR-2 and PRISMA. The AMSTAR-220 tool is an improved 16-item instrument that assesses key attributes of a well-conducted meta-analysis. Aspects such as prior design, literature search, data extraction, and data analysis were assessed by AMSTAR-2. We also used the PRISMA21 statement, a checklist with 27 items and 1 flowchart (4 stages), which contains the necessary entries for a transparent report on SRs to assess the risk of bias.

Scoring Methods

Veritas scores were determined for publication year, type of study, AMSTAR-2 score, PRISMA score, heterogeneity, and publication bias. Since disease and acupuncture techniques change over time, the year of publication is also an important factor in the study of heterogeneity.22 AMSTAR-2, updated in 2017, provides readers with a better assessment of research.23 Additionally, the PRISMA statement provides a standardized framework and allows authors to make full reports on SRs and assesses their quality.21 The heterogeneity of the study was included because it has a significant impact on the results of meta-analysis.24 Some studies are not published in index journals, and the negative results of some studies are not disclosed, so it is necessary to assess publication bias.25

The AMSTAR-2 scale comprises 16 items that are answered as “Yes” (item fully addressed), “No” (item not addressed), or “Partially satisfied” (item not fully addressed), resulting in scores from 0 to 16.20 Each item of the PRISMA scale is standardized and has a score of 1 for correctly used, 0.5 for insufficiently used, and 0 for unused or misused, with a full score of 27.21 The rank number of the remaining items is converted according to the medical statistics grade data processing method.26 In terms of the year of publication, the latest published article ranked the highest. When randomization is performed using mathematical techniques, the trial is characterized as a randomized controlled trial (RCT), such as using a random numbers table to assign patients to testing or controlled treatment. However, trials employing allocation methods such as coin flips, odd-even numbers, patient social security numbers, days of the week, medical record numbers, or other such pseudorandom processes are simply designated as controlled clinical trials (CCT). Among the included studies, RCTs are high-quality designs, while CCTs are low-quality. The heterogeneity score is the average of the heterogeneity score for clinical outcomes, which is assessed using the chi-square test and I2 statistic. More than half of the indexes in the literature on SRs showed that when p > 0.10, 0% ≤ I2 < 50%, the homogeneity indicates no heterogeneity, and the score is 3 points; 0.10 ≥ p ≥ 0.05, 50% ≤ I2 ≤ 75% is regarded as slightly significant heterogeneity with a score of 2 points, and 0.05 > p, I2 > 75% is considered significant heterogeneity with a score 1 point.24 If the publication bias is ignored, the risk of publication bias would be high.25

The scoring system was as> follows: in each project, the worst study gets a minimum score of 1 point, the second-to-last study gets 2 points, and so on. The best study is given the highest score n, where n= the number of studies. If there are two studies with the same score n, then the next included study will get n-2 points.27 The Veritas score was used as the final summary data. The score of each study was the average score in the six dimensions of quality.

The Drawing and Optimization of Veritas Plots

We used Excel 2010 and Adobe Illustrator CC () to draw and optimize the Veritas plots. The rank number of the above evaluation items in the overall literature was the value of each study in the coordinates of the Veritas plots. When drawing and optimizing the Veritas plots, Excel 2010 was used to generate the image, which was saved in a separate sheet. Then, we opened the image, exported the vector, calculated the diameter, created a polar coordinate grid, drew petals and did other optimization work in Adobe Illustrator CC to make the Veritas plots.

Results

Selection of Studies

The initial search detected 701 related publications, and 272 duplicated records were excluded by EndNote X9. After reading the titles and abstracts, 395 records were excluded from the preliminary screening, and after further screening, 15 studies were excluded. A total of 19 SRs were included for multivariate evaluation by Veritas plot.28–46 The literature retrieval and screening process is shown in Figure 1.

Figure 1

The literature retrieval and screening process.

General Characteristics

The years of publication of the included studies ranged from 1998 to 2018. There were 6 SRs28,31,37,39,43,44 on unrestricted types of LBP, 1 SR30 on acute LBP, 5 SRs32–35,40 on chronic LBP, 2 SRs36,38 on nonspecific LBP, 4 SRs29,41,42,45 on chronic nonspecific LBP, and 1 SR46 on acute, subacute or chronic nonspecific LBP.

As for the risk-of-bias tools, 10 SRs28–30,32,36,37,42,44–46 used Cochrane alone, 1 SR31 used the Jadad scale alone, 1 SR38 used the van Tulder scale alone, 1 SR39 used the 2003 CBRGC (Cochrane Back Review Group Criteria) and the Jadad scale, and 1 SR40 used CBRGC alone. No tools for risk assessment were used in the other 5 SRs.33–35,41,43 Fourteen SRs38 provided quality assessment, but every RCT was of low quality.

In the experimental group, 12 SRs30–36,38–42 used acupuncture alone as an intervention, and the remaining 7 SRs28,29,37,43–46 used acupuncture combined with other therapies. In the control group, 6 SRs29,31,33,34,36,37 were treated with placebo or sham acupuncture alone as interventions, and 13 SRs28,30,32,35,38–46 were treated with placebo or sham acupuncture combined with other therapies as interventions. The basic information included in SRs is shown in Table 1.

Table 1

Descriptive Characteristics of the Included Systematic Reviews

Author (Data)/Country	Patient Population	Number of RCTs Included (Patients)	Intervention Study		Quality of Original Studies Scale/Level	Main Results (Meta-Analysis)	Author’s Main Conclusions
			Intervention Group	Control Group
Furlan (2012)/Canada28	LBP	15 (3982)	SMT, Acupuncture, Massage, Mobilization	No treatment, Physical therapy (exercise and/or electrotherapy), Usual care immediately or at short-term follow-up	Cochrane risk of bias tool/Low	Acupuncture vs Inactive treatment1. Short-term posttreatment pain intensity [pooled VAS=−1.19 (95% CI −2.17, −0.21)] 3 RCTs	Subjects with chronic nonspecific LBP receiving acupuncture had statistically significantly better short-term posttreatment pain intensity and less immediate-term functional disabilitySubjects with acute/subacute nonspecific LBP, acupuncture did not significantly differ from placebo on pain or disability outcomes
						2. Immediate-posttreatment follow up pain intensity [pooled VAS=−0.59 (95% CI −0.9, −0.25)] 10 RCTs
Rubinstein (2010)/The Netherlands29	NSCLBP	35 (8298)	SMT, Acupuncture, Herbal medicine	Sham, Placebo, Passive Modalities	Cochrane risk of bias tool/Low	Acupuncture vs No treatment or Waiting list control	Acupuncture provides a short-term clinically relevant effect when compared with a waiting list control or when acupuncture is added to another intervention.
						1.1 Pain [MD=−24.10 (95% CI −31.52, −16.68)] 1 RCT
						1.2 Disability [SMD=−0.61 (95% CI −0.90, −0.33)] 1 RCT
						2. Acupuncture vs Sham/Placebo/Passive modalities
						2.1 Pain at 3 months [MD=−7.81 (95% CI −12.66, −1.89)] 4 RCTs
						2.2 Disability at 3months [SMD=−0.28 (95% CI −0.41, −0.16)]3 RCTs
						2.3 Recovery at 3 months [RR=3.53 (95% CI 0.91, 13.62)] 1 RCT
						3. Acupuncture plus an intervention vs Intervention alone
						3.1 Pain at 3 months [MD=−16.91 (95% CI −25.18, −8.61)] 3 RCTs
						3.2 Disability at 3months [SMD=−0.66 (95% CI −0.74, −0.58)] 4 RCTs
						3.3 Recovery at 3 months [RR=5.90 (95% CI 1.96, 17.70)] 1 RCT
						4. Acupuncture vs Any other intervention
						4.1 Pain at 3 months [MD=−9.40 (95% CI −12.13, −6.67)] 1 RCT
						4.2 Disability at 3months [SMD=−0.64 (95% CI −0.79, −0.49)] 1 RCT
Lee (2013)/Korea30	Acute LBP	11 (1139)	Acupuncture	Nonsteroidal anti inflammatory drugs, Medication, Sham acupuncture	Cochrane risk of bias tool/Low	Acupuncture vs Medication Overall improvement [RR=1.11 (95% CI 1.06, 1.16), p<0.001, I2=0%] 5 RCTs	Acupuncture may be more effective than medication for symptom improvement or relieve pain better than sham acupuncture in acute LBP.
						2. Acupuncture vs Sham acupuncture Pain intensity [MD=−9.38 (95% CI −17.00, −1.76), p=0.02, I2=27%] 2 RCTs
Ernst (1998)/England31	LBP	12 (472)	Acupuncture	Sham (placebo) acupuncture	Jadad scale/Low	Acupuncture vs Control intervention Pain intensity [OR=2.30 (95% CI 1.28, 4.13)] 9 RCTs	Acupuncture was shown to be superior to various control interventions, although there is insufficient evidence to state whether it is superior to placebo.
						2. Acupuncture vs Sham-controlled, Evaluator-blinded studies Pain intensity [OR=1.37 (95% CI 0.84, 2.25)] 4 RCTs
Xu (2013)/China32	CLBP	13 (2678)	Acupuncture	Sham acupuncture, Conventional care, Other alternative therapies	Cochrane risk of bias tool/Low	Acupuncture vs Blank treatments, Sham acupuncture and Other treatments Pain intensity [SMD=−0.43 (95% CI −0.61, −0.21), I2=85%] 18 RCTs	Acupuncture achieved better outcomes when compared with other treatments.
						2. Disability as a result of pain [SMD=−0.43 (95% CI −0.66, −0.21), I2=72.3%] 12 RCTs
						3. Spinal flexion [SMD=−0.15 (95% CI −0.63, 0.34), I2=77.4%] 5 RCTs
						4. Quality of life [SMD=0.47 (95% CI 0.15, 0.78), I2=85.1%] 5 RCTs
Vickers (2018)/America33	CLBP	39 (20827)	Acupuncture	Sham (placebo) acupuncture, No-acupuncture	None	Acupuncture vs Sham acupuncture Pain intensity [SD=0.30 (95% CI 0.21, 0.38)] 10 RCTs	Acupuncture is effective for the treatment of chronic pain, with treatment effects persisting over time.
						2. Acupuncture vs No-acupuncture control Pain intensity [SD=0.54 (95% CI 0.50, 0.57)] 12 RCTs
Vickers (2012)/America34	CLBP	31 (17922)	Acupuncture	Sham (placebo) acupuncture, No-acupuncture	None	Acupuncture vs Sham acupuncture Pain intensity [SD=0.37 (95% CI 0.27, 0.46)] 8 RCTs	Acupuncture is effective for the treatment of chronic pain.
						2. Acupuncture vs No-acupuncture control Pain intensity [SD=0.55 (95% CI 0.51, 0.58)] 7 RCTs
Trigkilidas (2010)/UK35	CLBP	4 (NR)	Acupuncture	Usual care treatment	None	Qualitative description only	Acupuncture could be effective in managing patients with LBP.
Machado (2009)/Australia36	NSLBP	76 (6865)	Acupuncture	Placebo treatment, Sham treatment	Cochrane risk of bias tool/Low	Qualitative description only	Acupuncture has a clinically relevant, persistent effect on chronic pain that is not completely explained by placebo effects.
Keller (2007)/Norway37	LBP	41 (NR)	TENS, Nonsteroidal anti inflammatory drugs, Manipulation, Acupuncture, Behavioral therapy, Exercise therapy	Placebo treatment, Sham treatment, No-treatment	Cochrane risk of bias tool/Low	Acupuncture vs Sham acupuncture or No treatmentPain intensity [SMD=0.61 (95% CI 0.41, 0.81), I2=0%] 7 RCTs	The effect of treatments for LBP is only small to moderate. there is a dire need for developing more effective interventions.
Yuan (2008)/Northern Ireland38	NSLBP	23 (6359)	Acupuncture (follow the STRICTA guidelines)	Therapy other than acupuncture	Van Tulder scale/Low	Qualitative description only	Acupuncture versus no treatment, and as an adjunct to conventional care, should be advocated.
Manheimer (2005)/USA39	LBP	33 (2214)	Acupuncture	Sham acupuncture, No treatment, Conventional therapy, Manipulation	1997 CBRGC and Jadad scale/Low	Acupuncture vs Sham acupuncture	Acupuncture effectively relieves chronic low back pain, but evidence about acupuncture’s effectiveness compared with other active treatments or for patients with acute back pain is inconclusive.
						Short-term effects of acupuncture on pain [SMD=0.58 (95% CI 0.36, 0.80)] 4 RCTs
						2. Long-term effects of acupuncture on pain [SMD=0.59 (95% CI −0.10, 1.29)] 2 RCTs
Ammendolia (2008)/Canada40	CLBP	19 (5001)	Acupuncture	Waiting list, Conventional therapy, Sham therapy	2003 CBRGC/Low	Qualitative description only	When compared with no treatment, there is evidence that acupuncture is effective in pain relief and functional improvement immediately after a series of treatment sessions and in the short-term follow-up.
Hutchinson (2012)/UK41	NSCLBP	7 (13874)	Acupuncture	TENS, Minimal (sham) acupuncture, Conventional treatment, Placebo No treatment	None	Qualitative description only	Acupuncture as more effective than no treatment.
Lam (2013)/Republic of Ireland42	NSCLBP	25 (5709)	Acupuncture	SMT, TENS, Medication, Physiotherapy, Exercise, A sham intervention	Cochrane risk of bias tool/Low	Acupuncture vs No treatment	Acupuncture may have a favorable effect on self -reported pain and functional limitations on NSCLBP.
						1.1 Pain Post-Intervention [SMD=−0.72 (95% CI −0.94, −0.49), I2=51%] 5 RCTs
						1.2 Disability Post-Intervention [SMD=−0.94 (95% CI −1.41, −0.47), I2=78%] 5 RCTs
						2. Acupuncture vs Medication (NSAIDs, muscle relaxants and analgesics)
						2.1 Pain Post-Intervention [MD=−10.56 (95% CI −20.34, −0.78), I2=0%] 3 RCTs
						2.2 Activity limitation[SMD=−0.36 (95% CI −0.67, −0.04), I2=7%] 3 RCTs
						3. Acupuncture vs Sham acupuncture
						3.1 Pain Post-intervention [MD=−16.76 (95% CI −33.33, −0.19), I2=90%] 4 RCTs
						3.2 Pain Follow-up [MD=−9.55 (95% CI −16.52, −2.58), I2=40%] 3 RCTs
						4. Acupuncture in addition to usual-care vs Usual-care
						4.1 Pain Post-intervention [MD=−13.99 (95% CI −20.48, −7.50), I2=34%] 4 RCTs
						4.2 Pain Follow-up [MD=−12.91 (95% CI −21.97, −3.85), I2=63%] 4 RCTs
						4.3 Disability Post-intervention [SMD=−0.87 (95% CI −1.61, −0.14), I2=71%] 3 RCTs
						4.4 Disability Follow-up [SMD=−0.51 (95% CI −0.91, −0.12), I2=0%] 2 RCTs
						5. Electro-acupuncture vs Self-care or Usual-care
						5.1 Pain Post-intervention [SMD=−1.39 (95% CI −2.37, −0.41), I2=92%] 5 RCTs
						5.2 Pain Follow-up [SMD=−0.66 (95% CI −1.17, −0.15), I2=66%] 4 RCTs
Johnston (2008)/Canada43	LBP	12 (NR)	SMT, Electro-acupuncture (High frequency), Moxibustion, Deep acupuncture, Electrical stimulation of auricular acupuncture points	TENS, SMT, Electro acupuncture (low frequency), Moxibustion	None	Qualitative description only	Investigators designing acupuncture or SMT trials should consider expertise-based randomization to increase the validity and feasibility of their efforts.
Liang (2016)/China44	LBP	10 (751)	Acupuncture, Needle warming moxibustion	Sham acupuncture, Traction therapy, Medication, Massage, Epidural injection	Cochrane risk of bias tool/Low	Acupuncture vs No-acupuncture control	Pure acupuncture may have a favorable effect on self reported pain and functional limitations in LBP patients.
						JOA [MD=2.83 (95% CI −1.17, −0.15), I2=0%] 3 RCTs
						2. ODI [MD=−5.07 (95% CI −7.50, −2.65), I2=0%] 2 RCTs
						3. VAS [MD=−1.32 (95% CI –1.41, −1.22), I2=0%] 8 RCTs
						4. RMDQ [MD=−2.80 (95% CI −3.49, −2.11), I2=0%] 2 RCTs
Xiang (2018)/China45	NSCLBP	7 (1768)	Acupuncture with or without electroacupuncture	Conventional treatment, Standard therapy, Routine care, Waiting list	Cochrane risk of bias tool/Low	Sham acupuncture/Placebo acupuncture vs Routine care or Waiting list	Sham acupuncture or placebo acupuncture was more efficacious for pain relief post-intervention. Concluding that SA or PA is appropriate for acupuncture research would be premature.
						1. Pain (VAS) [SMD=−0.36 (95% CI −0.52, −0.20), I2=16%] 6 RCTs
						2. Pain (CPGS) [SMD=−0.35 (95% CI −0.49, −0.20)] 1 RCT
						3. Disability (RMDQ) [SMD=0.11 (95% CI −0.78, 1.00), I2=94%] 3 RCTs
						4. Disability (PDI) [SMD=−0.42 (95% CI −0.90, 0.05), I2=66%] 2 RCTs
						5. Disability (ODI) [SMD=−0.30 (95% CI −0.69, 0.10), I2=16%] 1 RCT
Furlan (2005)/Canada46	Acute/subacute or chronic nonspecific LBP.	35 (2861)	Acupuncture or dry-needling	No treatment, Placebo/Sham acupuncture or Other sham procedure, and Other therapeutic interventions	Cochrane risk of bias tool/Low	Qualitative description only	Acupuncture and dry-needling may be useful adjuncts to other therapies for chronic low back pain.

Abbreviations: LBP, low back pain; CLBP, chronic low back pain; NSLBP, non-specific low back pain; NSCLBP, non-specific chronic low back pain; TENS, transcutaneous electrical nerve stimulation; SMT, spinal manipulative therapy; CBRGC, Cochrane Back Review Group Criteria; UK, the United Kingdom; USA, United States of America; SD, standard difference; MD, mean difference; SMD, standard mean difference; MWD, mean weighted difference; CPGS, Chronic Pain Grade Scale; ODI, Oswestry Disability Index; PDI, Pain Disability Index; RMDQ, Roland-Morris Disability Questionnaire; OR, the odds ratio; RR, the risk ratio; 95% CI, 95% confidence interval; NR, not recorded.

Evaluation Items

The results of the evaluation entries are shown in Table 2.

Table 2

Multivariate Evaluation and Rank Number in 6 Dimensions of the Included Literature

Study or Subgroup	Year of Publication	Type of Included Studies	AMSTAR 2 Score	PRISMA Score	Homogeneity	Publication Bias	Average Score of Rank Number
Furlan28	2012 [13]	RCT + CCT [1]	8.5 [11]	25.5 [19]	2.265[16]	Assessed [Egger’s test] [19]	13.17
Rubinstein29	2010 [10]	RCT [19]	9 [14]	17.5 [9]	0 [12]	Not assessed [16]	13.33
Lee30	2013 [16]	RCT [19]	9 [14]	20.5 [16]	3 [19]	Not assessed [16]	16.67
Ernst31	1998 [1]	RCT [19]	3.5 [2]	15.5 [5]	0 [12]	Assessed [Funnel plot] [19]	9.67
Xu32	2013 [16]	RCT [19]	10 [16]	20 [13]	2 [13]	Assessed [Begg’s test] [19]	16.00
Vickers33	2017 [18]	RCT [19]	5 [5]	14 [2]	0 [12]	Not assessed [16]	12.00
Vickers34	2012 [13]	RCT [19]	10.5 [17]	21.5 [17]	0 [12]	Not assessed [16]	15.67
Trigkilidas35	2010 [10]	RCT [19]	4.5 [4]	14 [2]	0 [12]	Not assessed [16]	10.50
Machado36	2009 [8]	RCT [19]	7.5 [10]	18.5 [10]	0 [12]	Not assessed [16]	12.50
Keller37	2007 [4]	RCT [19]	4 [3]	14.5 [3]	2.5 [17]	Not assessed [16]	10.33
Yuan38	2008 [7]	RCT [19]	9 [14]	16.5 [7]	0 [12]	Not assessed [16]	12.50
Manheimer39	2005 [3]	RCT [19]	6 [7]	23.5 [18]	0 [12]	Not assessed [16]	12.50
Ammendolia40	2008 [7]	RCT [19]	6.5 [8]	15 [4]	0 [12]	Not assessed [16]	11.00
Hutchinson41	2012 [13]	RCT [19]	3.5 [1]	20 [13]	0 [12]	Not assessed [16]	12.33
Lam42	2013 [16]	RCT [19]	11.5 [19]	20 [13]	2.09 [14]	Not assessed [16]	16.17
Johnston43	2008 [7]	RCT [19]	6 [7]	16 [6]	0 [12]	Not assessed [16]	11.17
Liang44	2016 [17]	RCT [19]	9.5 [15]	20.5 [16]	3 [19]	Not assessed [16]	17.00
Xiang45	2018 [19]	RCT [19]	9.5 [15]	20.5 [16]	2.25 [15]	Not assessed [16]	16.67
Furlan46	2005 [3]	RCT [19]	7 [9]	17.5 [9]	0 [12]	Not assessed [16]	11.33
Average score of rank number	10.58	18.05	10.05	10.42	13.53	16.47	13.18

Note: Square brackets indicate the score of Veritas.

Year of Publication

When the research literature on clinical issues is relatively new and involves a larger scope and time span, it is more meaningful to provide clinical guidance.46 In this study, the latest year of publication was 2018,33,45 and the earliest was 1998.31 Each of the years 2005,39,46 2007,37 2009,36 and 201644 contained 1 article. Each of the years 2008,38,40,43 2012,28,34,41 and 201330,32,42 contained 3 articles. Two articles were published in 2010.29,35

Types of Included Studies

Systematic reviews and meta-analyses based on high-quality randomized controlled trials are at the top of the pyramid of evidence recommendations, and the main source of evidence is high-quality randomized controlled trials.47,48 The 18 reports of RCTs29–46 and only 1 CCT28 included in this study reduced the risk of bias.

Methodological Quality

Of the 19 RCT articles, the highest score on AMSTAR-2 was 11.5,42,45 the lowest was 3.5,31,41 and the average score was 7.47. One article31 did not carry out a comprehensive literature search strategy, and only 1 article29 considered the literature completely, the others partially. Ten articles31,33,35–37,40–43,46 only provided the list of included literature but did not provide the list of excluded literature, which was unfavorable for the judgment of literature quality. Sixteen articles29,30,33–46 did not indicate whether there was publication bias. Ten articles31,32,34–36,40–42,44,46 did not describe conflicts of interest. The details of AMSTAR-2 of included studies are presented in .

Reporting Quality

The average PRISMA score included in our study was 18.47, with a maximum score of 25.528 and a minimum score of 14.33,35 All of the articles consisted of title, standard abstract, the current known theoretical basis of the study, detailed inclusion criteria, the number of preliminary screening articles, characteristics of the study, limitations and conclusions. Only 5 studies28,30,39,41,45 detailed the search strategy. None of the 1928–46 SRs included was registered on the PROSPERO platform. The details of AMSTAR-2 of included studies are presented in .

Homogeneity

As defined above, among the 19 articles, homogeneity was high except for the low heterogeneity of Rubinstein et al,29 Ernst et al,31 Vickers et al,33,34 Trigkilidas et al,35 Machado et al,36 Yuan et al,38 Manheimer et al,39 Ammendolia et al,40 Hutchinson et al,41 Johnston et al32 and Furlan et al.46 The details of Heterogeneity score of included studies are presented in .

Publication Bias

Only 3 articles28,31,32 reported publication bias. One was examined by Egger’s test,28 one by funnel plot,31 and one by Begg’s test.32 None of the other articles reported publication bias.

Veritas Plot Evaluation

Based on the evaluation of Veritas plots and the average rank numbers of all the studies, it was found that the study with the highest Veritas score was by Liang et al,44 with 17.00 points, while the lowest Veritas score was given to Ernst et al31 and Keller et al,37 with 9.67 points. There were 7 studies29,30,32,34,42,44,45 with Veritas scores ≥13.18 points, which was the average score. The Veritas plots are shown in Figure 2.

Figure 2

The Veritas plots.

Exploratory Analysis: Factors Associated with Methodological Quality and Reporting Quality

In univariate analysis, duplication of literature selected, the exclusion list and reasons for exclusion, appropriate tools to assess the risk of bias, appropriate statistical methods to combine results, assessment of the impact of bias risk on results, and reasonable analysis of bias risk were associated with an increase in AMSTAR-2 score (p≤0.10). After the above six variables were included in the multivariate linear regression model, the results showed that duplication of literature selected and appropriate tools to assess the risk of bias were still independent variables significantly affecting AMSTAR-2 score (p≤0.05). It was found that appropriate tools to assess the risk of bias had the largest impact on AMSTAR-2 score, reaching 2.262 (95% CI, 0.941~3.582; p = 0.003). (Table 3)

Table 3

Multivariate Linear Regression Analysis for Variables Associated with Better AMSTAR2 Score (n = 19)

Variables	B	S.E	t	p	95% CI
Constant	4.197	0.289	14.528	<0.001	3.568~4.826
Literature selected	1.483	0.544	2.729	0.018	0.299~2.668
Appropriate tools to assess the risk of bias	2.262	0.606	3.732	0.003	0.941~3.582

Abbreviation: S.E, standard error.

In univariate analysis, the title, eligibility criteria, retrieval, existing bias in a single study, research bias, the results of a single study, inter-research bias, and summary of evidence were associated with an increase in PRISMA score (p≤0.10). The multivariate linear regression model showed that the summary of evidence was still an independent and significant variable affecting PRISMA score (p≤0.05). Specifically, it was obvious that the summary of evidence had the largest impact on PRISMA score, reaching 6.993 (95% CI, 1.433~12.553; p <0.05) (Table 4).

Table 4

Multivariate Linear Regression Analysis for Variables Associated with Better PRISMA Score (n = 19)

Variables	B	S.E	t	p	95% CI
Constant	7.594	2.455	3.093	0.013	2.040~13.148
Summary of evidence	6.993	2.458	2.845	0.019	1.433~12.533

Abbreviation: S.E, standard error.

GRADE Grading of Evidence Quality

A total of 52 outcome indicators were included in the 19 SRs. The GRADE was used to evaluate the evidence intensity reports of outcome indicators of the SRs (Table 5). The results showed that a total of 33 outcome indicators were of low quality, 10 outcome indicators were of very low quality, 8 outcome indicators were of medium quality, and only 1 outcome indicator was of high quality.

Table 5

Quality of Evidence of the Included Systematic Reviews

First Author	Study Type	Outcome Indicator	Degradation Factors					Upgrade Factors	Quality of Evidence
			Risk of Bias	Inconsistency	Indirection	Inaccuracy	Publication Bias
Furlan28	RCT+CCT	VAS	−1	0	0	0	0	0	Low
		ODI	−1	0	0	−1	−1	0	Very Low
		RMDQ	−1	0	0	−1	−1	0	Very Low
Rubinstein29	RCT	RMDQ	0	0	0	0	−1	0	Moderate
		HFAQ	0	0	0	−1	−1	0	Low
		PDI	0	0	0	0	−1	0	Moderate
Lee30	RCT	VAS	0	0	0	0	0	0	High
		RMDQ	0	0	0	−1	0	0	Moderate
Ernst31	RCT	VAS	0	−1	0	0	−1	0	Low
Xu32	RCT	VAS	−1	−1	0	0	0	0	Low
		RMDQ	−1	0	0	0	0	0	Moderate
		PDI	−1	0	0	0	0	0	Moderate
		Aberdeen Low Back Pain Scale	−1	0	0	−1	0	0	Low
		ODI	−1	0	0	0	0	0	Moderate
		Fingertip-To-Floor Distance	−1	0	0	0	−1	0	Low
		Range of Movement of Spinal Flexion	−1	0	0	0	−1	0	Low
		SF-12	−1	−1	0	0	−1	0	Very Low
		SF-36	−1	−1	0	0	−1	0	Very Low
Vickers33	RCT	Time Course of Acupuncture Effects	−1	0	0	0	−1	0	Low
Vickers34	RCT	VAS	−1	0	0	0	−1	0	Low
Trigkilidas35	RCT	RMDQ	−1	0	0	0	−1	0	Low
		Bothersomeness	−1	0	0	0	−1	0	Low
		SF-36	−1	0	0	0	−1	0	Low
		VAS	−1	0	0	0	−1	0	Low
Machado36	RCT	Analgesic efficacy (100-point scale)	−1	0	0	0	−1	0	Low
Keller37	RCT	PDI	−1	0	0	0	−1	0	Low
Yuan38	RCT	PDI	0	0	0	0	−1	0	Moderate
Manheimer39	RCT	VAS	−1	0	0	0	−1	0	Low
		RMDQ	−1	0	0	0	−1	0	Low
Ammendolia40	RCT	PDI	−1	0	0	0	−1	0	Low
Hutchinson41	RCT	SF-36	−1	0	0	0	−1	0	Low
		HFAQ	−1	0	0	0	−1	0	Low
		Low Back Pain Rating Scale	−1	0	0	0	−1	0	Low
		Von Korff chronic pain scale	−1	0	0	0	−1	0	Low
		SF-12	−1	0	0	0	−1	0	Low
		RMDQ	−1	0	0	0	−1	0	Low
		Pain Disability Index	−1	0	0	0	−1	0	Low
		ODI	−1	0	0	0	−1	0	Low
		McGill Present Pain Index	−1	0	0	0	−1	0	Low
		Low Back Pain Rating Scale	−1	0	0	0	−1	0	Low
Lam42	RCT	VAS	−1	−1	0	0	−1	0	Very Low
		PDI	−1	−1	0	0	−1	0	Very Low
		RMDQ	−1	−1	0	0	−1	0	Very Low
Johnston43	RCT	None
Liang44	RCT	JOA	−1	0	0	0	−1	0	Low
		ODI	−1	0	0	0	−1	0	Low
		VAS	−1	0	0	0	−1	0	Low
		RMDQ	−1	0	0	0	−1	0	Low
Xiang45	RCT	VAS	−1	−1	0	0	0	0	Low
		RMDQ	−1	0	0	0	−1	0	Low
		ODI	−1	0	0	−1	−1	0	Very Low
		PDI	−1	−1	0	0	−1	0	Very Low
Furlan46	RCT	VAS	−1	0	0	−1	−1	0	Very Low

Abbreviations: VAS, Visual Analogue Scale; ODI, Oswestry Disability Index; RMDQ, Roland-Morris Disability Questionnaire; HFAQ, Hannover Functional Ability Questionnaire; PDI, Pain Disability Index; JOA, Japanese Orthopaedic Association Scores.

Discussion

With the help of the Veritas scores, we ultimately found that the mean Veritas plot scores of lack of publication year, type of study, AMSTAR-2, PRISMA, heterogeneity, and publication bias were 10.58, 18.05, 10.05, 10.42, 13.53, and 16.47, respectively. The study of Liang et al44 performed well in the year of publication, type of study, AMSTAR-2 score, PRISMA score and publication bias, earning it the highest Veritas score of 17 points. After close therewith is study of Lee30 and Xiang,45 with 16.67 points, respectively. They only have few difference in Year AMSTAR 2 score and homogeneity. As for AMSTAR-2, the outcomes of the review method, the assessment of the potential impact of risk of bias in individual studies on the results, the list of excluded studies, the funding sources, and the conflicts of interest were insufficient. In addition, in the case of PRISMA, the lack of a literature screening process, the lack of PROSPERO registration, and the absence of other analytical methods, such as sensitivity analysis and subgroup analysis, among others, were the main problems. Through observation of Veritas plots, we found that three principal problems of SRs were lack of publication bias, poor quality of reporting and methodology. The absence of publication bias assessment was an important problem in the SRs. Some SRs29,30,33–36 did not assess publication bias, and the absence of publication bias assessment may seriously affect the validity of the conclusions derived from the meta-analysis.34 Therefore, the clinical value of the results should be carefully considered. Moreover, the selection of acupuncture points and the treatment duration of each RCT included in the SRs may be different due to the specific situation of every patient. The skill level of the acupuncturists is a common problem in acupuncture therapy. Hence, subgroup analysis should be carried out, and acupoint selection should be explained, but our overview found that most of the SRs about this aspect were lacking.

By univariate analysis and multivariate linear regression, we found that duplication of literature selected and appropriate tools to assess the risk of bias were related to the AMSTAR-2 score, and summary of evidence was related to the PRISMA score. Best practice requires two authors to determine the eligibility of studies for inclusion in systematic reviews.49 Duplication of literature selected could ensure that as many qualified studies as possible are included, preventing omissions. The authors should choose the appropriate evaluation tool to evaluate the potential risk of bias in RCT intervention studies, which could objectively and correctly evaluate the risk of bias and analyze it to obtain more objective results.23 In the Discussion section, the authors should discuss the strength of the evidence of the relevant indicators in the study, because summary of evidence will lead different people to make different decisions.50 We used GRADE as an evidence quality evaluation tool.51 Only 1.9% high-quality evidence, 15.3% medium-quality evidence, and 82.6% low- to very low-quality evidence were found. Table 5 also shows that the factors leading to the degradation of evidence quality are mainly a lack of description of the randomization method as well as allocation and concealment methods.

SRs of evidence-based medicine are the top of the evidence tower and the best evidence to guide clinical practice.52 A high methodological and reporting quality of a SR means that the study design and implementation specifications are rigorous, and the results are repeatable, accurate, and clinically recommended.53 In our study, a Veritas plot was used to conduct a multidimensional analysis on the relevant indicators of SRs for acupuncture of low back pain. The overall quality of SRs and the data difference between a given SR and the average score were found, yielding intuitive and accurate evidence, demonstrating their advantages and disadvantages, and providing reference for clinical application. During the search process, we found that one article published in 2018 had a similar methodology as this study.54 Our study had more perspectives and higher accuracy because items such as the publication year, type of study, heterogeneity, and publication bias were added, and the AMSTAR was updated to AMSTAR-2.

This overview also has some limitations. One limitation is that due to language barriers, we only searched for manuscripts published in Chinese and English journals. Moreover, we observed the quality of the articles directly by the Veritas plot, a simple two-dimensional tool, which provides a visual mode of observation without much description. However, due to its subjectivity, authors in other fields may dispute the emphasis we have placed on the attributes in our Veritas plot. Of equal importance is the fact that it is difficult to compare the comprehensive strength between the objects when there are many objects involved in the evaluation. This tool cannot compare the quality of studies from different clinical areas at present.

In view of the above shortcomings, future appraisers who use meta-analyses need to assess the validity, reliability and perceived utility of the Veritas plot to quickly obtain the rankings for comprehensive evaluation and ensure its fairness and accuracy.

Conclusion

In conclusion, our study indicates that the methodology and reporting quality of SRs for acupuncture treatment of LBP still need to be improved. Future studies should make full use of the AMSTAR-2 tool and PRISMA statement to publish articles on this topic. The Veritas plot is an intuitive visualization tool for observing the quality of articles and is worthy of clinical application and promotion.