The Role of Acupuncture in Treating Perimenopausal Insomnia: An Overview and Quality Assessment of Systematic Reviews and Meta-Analyses.

OBJECTIVE: To summarize and critically assess the reliability of the methodological quality and outcome measures from systematic reviews (SRs)/meta-analyses (MAs) and provide an overall verdict about the therapeutic value of acupuncture for perimenopausal insomnia (PMI).
METHODS: We conducted a comprehensive literature search for SRs/MAs of seven major databases (English and Chinese). For each included review, the methodological quality was appraised according to the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the evidence quality was classified on the basis of the Grading of Recommendations, Assessment, Development and Evaluation (GRADE), and reporting quality was evaluated complying with Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009 (PRISMA-2009). Veritas plots were used to quantify the quality of included SRs/MAs.
RESULTS: Nine SRs/MAs were deemed eligible for the present overview. Considering the assessment of results from the AMSTAR-2 checklist, the methodological quality of one SR/MA was considered low, and the remaining eight were critically low. Major methodological deficiencies were concentrated on item 2 (the lack of protocol and/or registration information), item 7 (the lack of a list of excluded studies), and item 10 (the lack of reports on funding sources for individual studies included in the SRs/MAs). For the GRADE system, of the 25 outcomes, only three (12%) were rated as moderate-quality, while the remaining 22 were rated between low- and very low-quality. The PRISMA-2009 statement indicated three major reporting quality limitations in most SRs/MAs, namely: 1) only search terms without specific retrieval strategy; 2) incomplete descriptions for study characteristics, particularly the specific dosage and frequency of interventions in treatment/control groups; and 3) inadequate investigation and explanation of the source of high heterogeneity among original randomized control trials included. According to Veritas plots, quality rank scores of included SRs/MAs ranged from 3.3 to 8.3, with an average score of 6.4 ± 1.7.
CONCLUSION: Acupuncture appears to be beneficial for PMI management, but the quality of evidence is weakened by the unsatisfactory quality of both SRs/MAs and original trials included.

Background

Many women find perimenopause, the stage encompassing with the years of menstrual irregularities and the 12-month period after the occurrence of amenorrhea, a particularly challenging phase of their lives for preserving satisfying sleep.1,2 Women are more likely to struggle with insomnia across the lifespan than men are.1,3 The risk of sleep problems increases following the onset of perimenopause,3,4 with prevalence ranging between 39% and 47%, compared to premenopausal women with rates ranging between 16% and 42%.1,5 As a frequent and often bothersome menopausal symptom,3,6,7 perimenopausal insomnia (PMI) creates a substantial burden, and may have profound impacts on women’s physical and mental health.3,6 Insomnia has been associated with a variety of negative medical and psychiatric consequences, including mental distress, severe daytime fatigue and sleepiness, elevated risk of accidents, memory troubles, alcohol and drug abuse/dependence, increased cardiovascular and metabolic disorders (eg, stroke, coronary heart disease, diabetes, etc.), and even suicide.6,8,9 In addition to a lower quality of life, women with vasomotor symptoms accompanied by insomnia often have more emergency room visits compared with those without sleep disturbance.6 PMI is commonly intertwined with other coexistent medical conditions such as hot flashes, depressive disorder, frequent nocturnal enuresis, weight loss/gain and/or changes in appetite, which may further exacerbate negative outcomes of insomnia.10 It is hence imperative for women in midlife to access effective insomnia management, because the menopausal transition itself is also a plausible determinant of worsening sleep.1 Conventional therapies for PMI include hormone replacement therapy (HRT), psychotropic substances, particularly hypnotic/sedative and some antidepressants that have a hypnotic effect (eg, Mirtazapine, Trazodone, etc.), and cognitive behavioral therapy for insomnia (CBT-I).7,10–13 Despite the rapid development of pharmacotherapy12 and continuous progress in CBT-I,13 the usage of complementary and alternative therapy (CAM) as an independent option or in combination with conventional therapies in the treatment of insomnia is becoming more common.12 CAM with minimal side effects is desirable and is being constantly explored because of concerns about adverse events (AEs) and/or drug dependence stemming from conventional therapies (eg, HRT, sedatives/hypnotics, etc.).7

Originating from ancient China, acupuncture, is one of the most popular forms of CAM with simple maneuverability and a high degree of safety,7,9,11 and historically it has been utilized for insomnia and other mental illness in East Asian countries (eg, China, Korea, Japan, etc.).14 Acupuncture refers to the procedures entailing the stimulation of acupoints on the body surface (eg, limbs, trunk, scalp, ear, wrists, and ankles, etc.) through techniques such as sterile, metallic, hair-thin needles with or without electrical stimulation, heat (eg, moxibustion), and pressure.15 Specifically, acupuncture includes traditional needle acupuncture (TNA, including manual acupuncture and electroacupuncture) and modified modalities of acupuncture (they are listed in “Notes” of according to the acupuncture textbook16 compiled by the Ministry of Education of China). For this review, acupuncture is defined as the above-mentioned forms unless a specific form of acupuncture is noted. There has been a growing number of randomized control trials (RCTs) investigating the efficacy and safety of acupuncture for PMI over the past decade,11 indicating that the promising potential of PMI management with acupuncture has attracted considerable research interest. However, evidence regarding the treatment of PMI with acupuncture is not yet completely adequate.7 The results of the aforementioned RCTs are not consistent. Systematic reviews (SRs) and/or meta-analyses (MAs) have been conducted to provide a more objective evidence-based reference. However, the evidence quality of these SRs/MAs has not undergone critical appraisal which is an essential step before treatment recommendations can be presented and applied confidently.17 The aim of the current overview is to fill these gaps in the literature, investigate the general characteristics, and assess the methodological quality and the reliability of the conclusions of previously published SRs/MAs regarding acupuncture for PMI in order to provide quality reference for clinically sound, evidence-based decision-making.

Materials and Methods

Study Registration

The protocol for this overview was registered in the Prospective Register of Systematic Reviews (PROSPERO): No. CRD42021261845.

Eligibility Criteria

The SRs/MAs of clinical RCTs or quasi-RCTs about acupuncture for PMI were included. Participants included were required to be women with PMI. SRs/MAs focused on insomnia in pre- or post-menopausal status, both PMI and pre-/post-menopausal insomnia, or PMI comorbid with other physical or psychological disorders were excluded. Interventions were restricted to acupuncture, or acupuncture combined with conventional therapies for PMI including Western medication [HRT and psychotropic substances (eg, hypnotic, sedative, antidepressant, etc.)], CBT-I, or Chinese herbal medicine (CHM). In a recent real-world retrospective cross-sectional study conducted, oryzanol was found to be one of the top three drugs used to relieve insomnia in the Chinese population.18 Some SRs/MAs also demonstrated that oryzanol was often used as co-therapy in combination with those aforementioned conventional therapies for PMI15 or perimenopausal syndromes.19 Most original clinical trials of acupuncture are carried out in China, and the subjects of SRs/MAs are mainly Chinese women with PMI. Hence, SRs/MAs including original trials with acupuncture combined with oryzanol as intervention were also included in this overview. Although Tui-na/massage, Gua-sha (scraping therapy), and cupping therapy are also related to meridians and collaterals in traditional Chinese medicine (TCM) theory, they are not acupuncture therapies. Therefore, SRs/MAs were excluded if they included original studies using these therapies or acupuncture combined with these therapies as interventions. Comparator interventions (in control group) were restricted to waitlist control, placebo-/sham-acupuncture, CBT-I, or drugs (HRT/psychotropic substances/CHM/oryzanol). Acupuncture was not allowed in the control group. The SRs/MAs were excluded if they included original studies using acupuncture or acupuncture combined with other therapies in control groups. The primary outcome was subjective and/or objective indices related to sleep, and/or total clinical effectiveness rate. Secondary outcomes included scales/questionnaires related to mood, climacteric symptoms, and quality of life, reproductive hormone levels, as well as AEs ().

Search Strategy and Methods

Four Chinese and three English electronic databases – Wanfang database, China National Knowledge Infrastructure (CNKI), China biomedical literature service system (SinoMed), and Chongqing VIP database (CQVIP), and Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (via PubMed), and EMBASE – without limitations of publication year and language, were retrieved from their inception date until June 2021. The following grouped terminology were used for searching: (“insomnia” OR “sleep disorder” OR “dyssomnias” OR “sleep initiation and maintenance disorder”) combined with (“perimenopause” OR “climacteric”) combined with (“acupuncture” OR “needling” OR “auricular” OR “acupoint” OR “moxibustion” OR “catgut implantation”) combined with (“systematic review” OR “meta-analysis” OR “meta-analyses”). The fully detailed search strategy for database is shown in .

Data Extraction

One researcher (QQ-F) generated the search strategy, searched the potential databases, and drew up a list of records. Retrieved results were imported into the EndNote software (Version X9) and the duplications were excluded. Two evaluators (WJ-Z and FY-Z) independently screened titles or abstracts based on the inclusion and exclusion criteria. Potential studies that met the inclusion criteria were independently assessed and determined for eligibility with full text, and then cross-checked. Finally, two reviewers (QQ-F and FY-Z) independently extracted the data and proofread the information. Disagreements in the study selection and data extraction were resolved by discussion and consensus between QQ-F and FY-Z. Any unresolved discrepancies were settled through a consensus process, that is, discussion by referral to an expert (Z-Z). EndNote was used to manage literature, including storing the results of searches and removing duplications.

For each SR/MA, the following data were extracted: the last name of the first author, publication year, country, databases retrieved from, design, number of included original trials with total sample size, interventions in both treatment and control groups, quality assessment methods, outcomes, and major findings.

Quality Assessment Methods

Two assessors (WJ-Z and FY-Z) carried out independent evaluations on the methodological quality of SRs/MAs by using the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2).20 The checklist consists of 16 items answerable by three options (“Yes”, “Partial Yes” and “No”). After interpreting weaknesses detected in critical and noncritical items, the overall methodological quality of each SR/MA was categorized into four levels: “High”, “Moderate”, “Low”, or “Critically low”.

The evidence quality for each outcome generated from SRs/MAs was appraised to adhere to the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE)21 in accordance with the following five aspects: limitations (that is, a risk of bias of original RCTs), inconsistencies, indirectness, inaccuracy, and publication bias. After considering these aspects that can potentially affect the reliability of the given effect estimate, the quality of evidence was classified into “High”, “Moderate”, “Low”, or “Very Low” from a starting point determined by study design.

Reporting quality was evaluated on the basis of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. The PRISMA-202022 released in March 2021. However, considering most of the included SRs/MAs were completed before then, we still used the PRISMA-200923 for assessing the report quality. PRISMA is a 27-item tool, with each item answered by the following possible choice: “Complete report”, “Partial report” or “Unreported”.

Each domain in these three measurement tools were assessed by two reviewers (QQ-F and FY-Z) independently. In the event of any inconsistency and/or disagreement in the process of assessments (methodological-, evidence- and reporting-quality assessments), the two sides discussed and arrived at consensus with the third evaluator (Z-Z).

In order to quantify and compare the quality of evidence from six dimensions (year of publication, research type, AMSTAR-2 score, PRISMA score, homogeneity, and publication bias) among the included studies, we adopted the “Veritas Scoring System” and Veritas plots.24 For each dimension, the best study or the study with the highest score was given n points where n is the total number of the included studies. The second-best study received (n – 1) points, and so on. In the case of m studies performing equally well, m studies received the same score; and the study with the next highest score would receive (n – m) points.24 The quality score of each SR/MA is the average of all dimension scores. Consequently, the level of evidence quality can be compared among all selected SRs/MAs. Microsoft Excel (2019 Version) and Adobe Illustrator CC (2020 Version) were used to draw and optimize the Veritas plots.24

Results

Study Identification

A database search identified 86 publications, 19 of which were excluded after de-duplication. Screening title and/or abstract resulted in the exclusion of 52 articles, and 15 full-text review articles were subjected to eligibility criteria analysis. Following review of the complete manuscripts, nine articles eventually met the eligibility criteria, and six articles were excluded. Figure 1 depicts the flow diagram for the selection procedure. The excluded studies with detailed reasons for exclusion are displayed in .

Figure 1

Flow diagram of the literature search and study selection process.

Characteristics of Included SRs/MAs

The characteristics of the nine SRs/MAs included in this overview for qualitative analysis are summarized in Table 1. All included studies were published between 2012 and 2021, including eight SRs/MAs (88.9%) were published in Chinese25–32 and the remaining one (11.1%) was published in English.11 That study, published in English, was a collaboration between Australian and Chinese academics,11 while the rest were conducted in China alone. All included SRs/MAs only included RCTs but no quasi-RCTs.

Table 1

Characteristics of Included Nine SRs/MAs

Authors, Year	Country	Databases Searched	Study Design (n); Participants (n)	Treatment Group	Control Group	Quality Assessment	Outcomes	Major Results
He et al 202125	China	①②③	RCTs (n = 27); PMI women (n = 2349)	Acupuncture	(i) hypnotics (or + oryzanol)(ii) HRT	Cochrane Collaboration’s risk of bias tool	(i) PSQI(ii) total clinical effectiveness rate	(i) acupuncture is better than Western medication in improving PMI(ii) incidence of AEs in acupuncture group (3.5%) is lower than that in Western medication group (12.5%)
Li et al 202026	China	①②③④⑤⑦⑧⑫	RCTs (n = 10); PMI women (n = 708)	(i) acupuncture(ii) acupuncture + hypnotics(iii) acupuncture + CHM	(i) waitlist control(ii) placebo-acupuncture(iii) hypnotics(iv) CHM	Cochrane Collaboration’s risk of bias tool	(i) PSQI(ii) KI(iii) serum FSH and E2 levels(iv) ISI	(i) acupuncture is effective in improving PMI and other perimenopausal symptoms(ii) acupuncture combined with CHM or Western medication is more effective than CHM or Western medication alone in decreasing FSH level; no significant difference in regulating E2 level between these two groups(iii) no AEs is reported in acupuncture group, while worsening of insomnia is reported in placebo-acupuncture group (10.8%)
Ma et al 202027	China	①②③④⑤⑦⑧	RCTs (n = 6); PMI women (n = 453)	(i) acupuncture(ii) acupuncture + CHM	(i) waitlist-control(i) hypnotics(iii) CHM	Cochrane Collaboration’s risk of bias tool	(i) PSQI(ii) AIS(iii) SRSS(iv) KI(v) total clinical effectiveness rate	(i) acupuncture combined with CHM is better than CHM in improving PMI(ii) no significant difference in improving PMI between acupuncture and Western medication
Wang 202128	China	①②③	RCTs (n = 12); PMI women (n = 1054)	Acupuncture + CHM	(i) hypnotics(ii) oryzanol	Cochrane Collaboration’s risk of bias tool	Total clinical effectiveness rate	(i) acupuncture combined with CHM is better than Western medication in improving PMI(ii) mild dizziness is reported in acupuncture group, while more AEs such as dry mouth, dizziness, fatigue, rash, etc. is reported in hypnotics; AEs also affected the patients’ treatment compliance
Zhang et al 201629	China	①②③④⑤⑦⑧⑬⑭⑯⑰	RCTs (n = 8); PMI women (n = 539)	Acupuncture	(i) hypnotics (or + oryzanol)(ii) CHM (or + oryzanol)	(i) Cochrane Collaboration’s risk of bias tool(ii) GRADE	(i) PSQI(ii) total clinical effectiveness rate	(i) acupuncture is better than either CHM or Western medication in improving PMI(ii) incidence of AEs in acupuncture group (0%) is lower than that in Western medication group (4.2%)
Zhang et al 201230	China	①②③④⑤⑦⑧	RCTs (n = 11); PMI women (n = 858)	Acupuncture	(i) hypnotics (or + oryzanol)(ii) CHM (or + oryzanol)	Cochrane Collaboration’s risk of bias tool	(i) PSQI(ii) serum E2 level(iii) total clinical effectiveness rate	(i) acupuncture is better than Western medication in improving PMI(ii) no significant difference in improving PMI between acupuncture and CHM
Zhao et al 202111	Australia; China	①②③④⑤⑥⑦⑧⑨⑩⑪⑬⑮	RCTs (n = 15); PMI women (n= 1410)	(i) acupuncture(ii) acupuncture + hypnotics	(i) hypnotics(ii) HRT	(i) Cochrane Collaboration’s risk of bias tool(ii) STRICTA	(i) PSQI(ii) KI(iii) serum FSH, LH and E2 levels(iv) total clinical effectiveness rate	(i) acupuncture alone or acupuncture combined with Western medication is better than Western medication alone in improving PMI and other perimenopausal symptoms(ii) no significant differences were identified between acupuncture and Western medication in regulating either FSH or E2 levels(iii) incidence of AEs in acupuncture group is lower than that in Western medication group
Zhou et al 202031	China	①②③⑤⑦⑧	RCTs (n = 12); PMI women (n= 1192)	Acupuncture	(i) placebo-acupuncture(ii) hypnotics (or + oryzanol)	Cochrane Collaboration’s risk of bias tool	(i) PSQI(ii) total clinical effectiveness rate	(i) acupuncture is better than either Western medication or placebo-acupuncture in improving PMI
Jiang et al 202132	China	①②③④⑤⑦⑧⑫	RCTs (n = 10); PMI women (n= 879)	Acupuncture	(i) hypnotics (or + oryzanol)	(i) Cochrane Collaboration’s risk of bias tool(ii) GRADE	(i) PSQI(ii) AIS(iii) serum E2 level(iv) HAMD(v) HAMA(vi) total clinical effectiveness rate	(i) acupuncture is better than Western medication in improving PMI(ii) no significant differences were identified between acupuncture and Western medication in regulating E2 level(iii) acupuncture is better than Western medication in improving depression and anxiety due to PMI

Notes: Databases include [①China National Knowledge Infrastructure (CNKI), ②Chongqing VIP database (CQVIP), ③Wanfang database, ④China biomedical literature service system (SinoMed), ⑤Cochrane Central Register of Controlled Trials (CENTRAL), ⑥Sciverse ScienceDirect, ⑦PubMed, ⑧EMBASE, ⑨Springer, ⑩Allied and Complementary Medicine Database (AMED), ⑪PsycINFO (ProQuest), ⑫Web of Science, ⑬ClinicalTrials.gov, ⑭Chinese Clinical Trial Registry (ChiCTR), ⑮World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) search portal, ⑯ (CCT), ⑰Google scholar].

Abbreviations: RCTs, randomized controlled trials; CHM, Chinese herbal medicine; PMI, perimenopausal insomnia; HRT, hormone replacement therapy; AEs, adverse events; PSQI, Pittsburgh Sleep Quality Index; AIS, Athens Insomnia Scale; ISI, Insomnia Severity Index; SRSS, Self-Rating Scale of Sleep; KI, Kupperman Index; HAMA, Hamilton Anxiety Scale; HAMD, Hamilton depression scale; FSH, follicle-stimulating hormone; E2, Estradiol; LH, luteinizing hormone; STRICTA, revised Standards for Reporting Interventions in Clinical Trials of Acupuncture.

Each SR/MA comprised 6 to 27 original trials. The total participants included in each SR/MA ranged from 453 to 2349. Interventions in the treatment group were mainly acupuncture, acupuncture combined with hypnotics/CHM, and the control group involved waitlist control, placebo-/sham-acupuncture, hypnotics/CHM (or hypnotics/CHM + oryzanol), HRT. Despite common usage in various types of insomnia, CBT-I or psychotropic agents other than hypnotics (eg, Mirtazapine, Trazodone, etc.) were not identified in any included SRs/MAs (Table 1).

Three commonly used Chinese databases (CNKI, CQVIP and Wanfang) were the source of all SRs/MAs. Two out of nine SRs/MAs (22.2%) did not search any international/English databases.25,28 Only two SRs/MAs (22.2%) retrieved one or more clinical trial registration platform/center,11,29 which is officially suggested as a crucial component of databases and searching work in SRs/MAs20 (Table 1).

The outcome indicators in these SRs/MAs involved sleep scales/questionnaires [Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Athens Insomnia Scale (AIS), and Self-Rating Scale of Sleep (SRSS)], mood-related scales [Hamilton Depression Scale (HAMD) and Hamilton Anxiety Scale (HAMA)], total clinical effectiveness rate, scale for menopausal symptoms [Kupperman Index (KI)], and serum reproductive levels (Table 1).

Cochrane Collaboration’s risk of bias tool33 was used as the methodological quality evaluation system in all SRs/MAs. One SR/MA11 also employed the revised Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) checklist34 to illustrate the details of acupuncture procedure in each original RCT, and in turn, appraise their completeness and reporting quality. GRADE system21 were adopted in two SRs/MAs.29,32 Among them, one SR/MA29 used the results derived from GRADE system to replace the methodological evaluation for each included original RCT, which was regarded as an obvious misuse. The GRADE provides a system for assessing the evidence quality for SRs/MAs and clinical guidelines as well as grading recommendations strength in guidelines,35 but not for a single RCT (Table 1).

Effectiveness and Safety of Acupuncture for PMI

In spite of notable limitations in quality of original RCTs, all the included SRs/MAs supported the satisfactory therapeutic effect and safety of acupuncture in PMI management. These positive results can be briefly divided into two categories: acupuncture as an independent therapy or acupuncture as an adjunct to pharmacotherapy (CHM or hypnotics) (Table 1).

Acupuncture Used as an Independent Therapy

Two SRs/MAs26,31 reported that acupuncture was better than placebo-acupuncture in improving PMI. However, the conclusion was based on the same two RCTs.36,37 Similarly, two SRs/MAs26,27 each included one RCT (acupuncture vs waitlist control) and qualitatively described the clinical effectiveness of acupuncture.

Six SRs/MAs11,25,29–32 addressed the comparison between acupuncture and Western medication [HRT or hypnotics (or hypnotics + oryzanol)]. Via observing significant decline in PSQI and higher total clinical effectiveness rate, all these reviews favored acupuncture in mitigating PMI. Unfortunately, there was a lack of pooled estimate effects on objective sleep parameters, such as sleep efficiency, wake after sleep onset, or sleep stage/architecture outcomes measured with polysomnography, which is more reliable.

In addition to PMI, other climacteric symptoms are likely to be attenuated with acupuncture treatment. Acupuncture was found to be more effective in reducing KI scores in comparison with either placebo-acupuncture26 or hypnotics.11 Two SRs/MAs11,30 reported that acupuncture significantly upregulated serum estradiol (E2) level, and/or downregulated serum follicle-stimulating hormone (FSH) level.

Five SRs/MAs11,25,26,28,29 included reports of safety. The incidence of AEs in acupuncture (0 to 3.5%) was shown to be significantly lower than that in either placebo-acupuncture (10.8%) or Western medication (4.2% to 12.5%) in all studies. Of these, the major two AEs associated with acupuncture were hematoma and slight pain of acupoints, which usually disappeared quickly after removal of the needles. Worsening of insomnia was the only AE for placebo-acupuncture. More AEs were reported following the usage of hypnotics/HRT, such as daytime sleepiness and fatigue, dizziness, memory loss, and mild nausea and/or vomiting (Table 1).

Acupuncture Used as an Adjunct to Pharmacotherapy

Acupuncture is likely to further enhance the effectiveness of drugs in improving PMI. Two SRs/MAs reported that acupuncture combined with hypnotics11 or CHM27 was more effective than hypnotics alone11 or CHM alone27 in reducing PSQI/AIS scores. Another interesting finding was that this combination therapy (acupuncture + drug) was more effective than drug alone in decreasing serum FSH level; however, there was no significant difference in regulating serum E2 level between these two therapies (combination therapy vs drug).26

Quality of Methodology

In accordance with the AMSTAR-2 classification, one SR/MA was rated as “Low” in methodological quality, while the remaining were rated as “Critically low” in methodological quality because of essential weaknesses in at least two critical domains. Consistent methodological deficiency was identified in item 7 (no SR/MA provided a detailed list of excluded studies and justified the exclusions). Another factor reducing the methodological quality in most SRs/MAs (88.9%) was item 2 (the lack of a written protocol or registered guide prior to the commencement of the SR/MA), which is also a critical item. Only one review11 provided searchable registration information with explicit statement of research methods in PROSPERO platform. Among noncritical items, methodological quality of included SRs/MAs was widely affected by item 10 (the lack of reports on the funding sources of original RCTs included in the SRs/MAs). Only one review11 included the analysis of this criterion via the Cochrane Collaboration’s risk of bias tool (other bias = baseline balance and funding sources of RCTs) (Table 2).

Table 2

Critical Appraisal for Methodological Quality of SRs/MAs Based on AMSTAR-2 Checklist

Authors, Year	AMSTAR-2																Quality Assessment
	Q1	Q2	Q3	Q4	Q5	Q6	Q7	Q8	Q9	Q10	Q11	Q12	Q13	Q14	Q15	Q16
He et al 202125	Y	N	Y	PY	Y	Y	N	Y	Y	N	Y	Y	Y	Y	Y	Y	CL
Li et al 202026	Y	N	Y	PY	Y	Y	N	PY	Y	N	Y	N	Y	Y	N	Y	CL
Ma et al 202027	Y	N	Y	PY	Y	Y	N	PY	Y	N	Y	Y	Y	Y	N	Y	CL
Wang 202128	Y	N	Y	PY	Y	Y	N	PY	Y	N	Y	Y	Y	Y	Y	N	CL
Zhang et al 201629	Y	N	Y	Y	Y	Y	N	PY	Y	N	Y	N	Y	Y	Y	Y	CL
Zhang et al 201230	Y	N	Y	PY	Y	Y	N	PY	Y	N	Y	N	Y	Y	N	N	CL
Zhao et al 202111	Y	Y	Y	Y	Y	Y	N	Y	Y	Y	Y	Y	Y	Y	Y	Y	L
Zhou et al 202031	Y	N	Y	PY	Y	Y	N	PY	Y	N	Y	N	Y	Y	N	Y	CL
Jiang et al 202132	Y	N	Y	PY	Y	Y	N	PY	Y	N	Y	Y	Y	Y	Y	N	CL

Abbreviations: Y, yes; PY, partial yes; N, no; CL, critically low; L, low; M, moderate; H, high.

For both item 4 (comprehensive literature retrieval strategy)11,29 and item 8 (detailed description of RCTs’ characteristic),11,25 two SRs/MAs were judged as “Yes” respectively, while the other reviews were only judged as “Partial Yes” due to inadequate information provided. Half of the SRs/MAs without sensitivity analysis26,29–31 and/or without publication bias analysis26,27,30,31 were determined to be defective in item 12 and item 15, respectively. Three reviews28,30,32 missed the reports in competing interests/conflict of interests (item 16) (Table 2).

Quality of Evidence

Twenty-five outcomes associated with effectiveness and efficacy of acupuncture for PMI were included in nine SRs/MAs. In terms of the GRADE system, the quality of evidence ranged between moderate and very low (three were judged as “Moderate”; six were judged as “Low”; and 16 were judged as “Very low”). Furthermore, 36% (n = 9) of the outcomes were provided on the basis of no more than three original RCTs. The most common degradation factor was the risk of bias within the original RCTs, which involved all the outcomes; some outcomes were also downgraded because of inconsistency, imprecision, and the possibility of publication bias. The decreased quality of evidence in none of the outcomes was attributed to indirectness (Table 3).

Table 3

Critical Appraisal for Evidence Quality of SRs/MAs Based on GRADE System

Authors, Year	Interventions	Outcomes	Numbers of Studies (Numbers of Participants)	MD/SMD/RR, 95% CI	I2% (p)	Limitations	Inconsistency	Indirectness	Imprecision	Publication Bias	Quality of Evidence
He et al 202125	Acupuncture vs hypnotics (or + oryzanol)/HRT	PSQI	13 (1115)	−0.63 [−0.75, −0.51]	96 (<0.001)	−1①	−1②	0	0	−1⑤	Very low
		Effectiveness rate	20 (1737)	1.85 [1.57, 2.18]	17 (<0.001)	−1①	0	0	0	−1⑤	Low
Li et al 202026	Acupuncture vs placebo-acupuncture	PSQI	3 (185)	−3.98 [−6.35, −1.61]	92 (0.001)	−1①	−1②	0	−1③	0	Very low
		ISI	2 (149)	−6.05 [−9.44, −2.65]	84 (<0.001)	−1①	−1②	0	−1③	0	Very low
	Acupuncture + hypnotics/CHM vs hypnotics/CHM	Serum FSH level	4 (312)	−11.26 [−12.71, −9.82]	31 (<0.001)	−1①	0	0	−1③	0	Low
		Serum E2 level	4 (312)	8.48 [−7.58, 24.54]	97 (0.30)	−1①	−1②	0	−1③	0	Very low
	Acupuncture (or + hypnotics/CHM) vs waitlist control/hypnotics/CHM	PSQI	7 (509)	−3.66 [−4.50, −2.22]	89 (<0.001)	−1①	−1②	0	0	0	Low
Ma et al 202027	Acupuncture + CHM vs CHM	Effectiveness rate	4 (300)	1.19 [1.08, 1.31]	0 (<0.001)	−1①	0	0	0	0	Moderate
Wang 202128	Acupuncture + CHM vs hypnotics (or oryzanol)	Effectiveness rate	12 (939)	1.10 [1.05, 1.15]	38 (<0.001)	−1①	0	0	0	0	Moderate
Zhang et al 201629	Acupuncture vs hypnotics	PSQI	2 (162)	−4.43 [−5.20, −3.66]	0 (<0.001)	−1①	0	0	−1③	−1④	Very low
	Acupuncture vs hypnotics (or + oryzanol)	Effectiveness rate	4 (292)	4.09 [2.02, 8.28]	0 (<0.001)	−1①	0	0	−1③	−1④	Very low
Zhang et al 201230	Acupuncture vs hypnotics/CHM	PSQI	2 (120)	−0.62 [−2.75, 1.51]	81 (0.57)	−1①	−1②	0	−1③	−1⑤	Very low
		Serum E2 level	2 (180)	33.69 [29.87, 37.50]	0 (<0.001)	−1①	0	0	−1③	−1⑤	Very low
	Acupuncture vs hypnotics/CHM (or + oryzanol)	Effectiveness rate	11 (858)	5.99 [2.32, 15.49]	76 (<0.001)	−1①	−1②	0	0	−1⑤	Very low
Zhao et al 202111	Acupuncture vs hypnotics	PSQI	10 (1059)	−2.38 [−3.38, −1.37]	93 (<0.001)	−1①	−1②	0	0	0	Low
		KI	4 (274)	−5.95 [−10.68, −1.21]	96 (0.01)	−1①	−1②	0	−1③	0	Very low
		Serum FSH level	3 (366)	−0.53 [−1.45, 0.39]	94 (0.26)	−1①	−1②	0	−1③	0	Very low
		Serum E2 level	3 (366)	0.63 [−0.51, 1.77]	96 (0.28)	−1①	−1②	0	−1③	0	Very low
		Effectiveness rate	10 (1047)	1.10 [1.05, 1.16]	20 (<0.001)	−1①	0	0	0	0	Moderate
	Acupuncture + hypnotics vs hypnotics	PSQI	3 (208)	−3.13 [−5.43, −0.83]	84 (0.008)	−1①	−1②	0	−1③	0	Very low
Zhou et al 202031	Acupuncture vs hypnotics (or + oryzanol)	Effectiveness rate	11 (1102)	2.85 [2.02, 4.04]	37 (<0.001)	−1①	0	0	0	−1④	Low
		PSQI	8 (720)	−3.80 [−5.46,-2.14]	97 (<0.001)	−1①	−1②	0	0	−1④	Very low
Jiang et al 202032	Acupuncture vs hypnotics (or + oryzanol)	PSQI	8 (737)	−3.46 [−4.95, −1.96]	97 (<0.001)	−1①	−1②	0	0	−1④	Very low
		Effectiveness rate	10 (879)	4.14 [2.66, 6.46]	0 (<0.001)	−1①	0	0	0	−1④	Low
		Serum E2 level	3 (374)	1.70 [−7.52, 10.92]	93 (0.72)	−1①	−1②	0	−1③	−1④	Very low

Notes: GRADE [①The design of the trial has a large bias in randomization, allocation concealment, blinding or other factors; ②the confidence interval overlaps less, the heterogeneity test p is very small, the I2 is larger, and the heterogeneity could not be completely explained by conducting subgroup analysis, sensitivity analysis, or meta-regression; ③the confidence interval is not narrow enough;④funnel graph asymmetry;⑤fewer studies are included due to incomplete retrieval and there may be a greater risk of publication bias].

Abbreviations: Effectiveness rate, total clinical effectiveness rate; PSQI, Pittsburgh Sleep Quality Index; ISI, Insomnia Severity Index; KI, Kupperman Index; FSH, follicle-stimulating hormone; E2, Estradiol; CHM, Chinese herbal medicine.

Quality of Reporting

In terms of the PRISMA-2009 statement, 44.4% SRs/MAs25–27,31 only provided search terms without detailed search strategies. Consistent with item 2 of AMSTAR-2, protocol and/or registration information was required in PRISMA. However, it was identified in only one review.11 A flow diagram is usually recommended to describe the literature screening process. However, four SRs/MAs27,30–32 only reported the total number of articles retrieved and/or the number of articles eventually included, without presenting a study selection flow chart or describing each step of the study selection process in detail. 88.9% SRs/MAs (n = 8)25–32 were judged as “Partial report” in item 18 (study characteristics) due to incomplete description, particularly the specific intervention regimen in either treatment or control group (eg, frequency, duration, dosage, wave form of electroacupuncture, composition of CHM decoction, etc.). Two SRs/MAs26,30 did not include the investigation of publication bias; One SR/MA27 reported the investigation approach of publication bias (Funnel plot) in the methodology section, but ultimately did not perform the investigation as less than 10 original RCTs were included; one SR/MA31 reported the findings of publication bias in the result section, but did not clearly describe how publication bias was tested in the methodology section. Two SRs/MAs26,31 did not illustrate any approach to address potential heterogeneity. One study stated that subgroup- and sensitivity-analysis would be used to explore the possible heterogeneity, but elaborated neither the approach used for sensitivity analysis nor criteria used for classifying subgroups.29 Funding information was ignored in three SRs/MAs28,30,32 (Table 4).

Table 4

Critical Appraisal for Reporting Quality of SRs/MAs Based on PRISMA-2009 Statement

Checklist Item	He et al 202125	Li et al 202026	Ma et al 202027	Wang 202128	Zhang et al 201629	Zhang et al 201230	Zhao et al 202111	Zhou et al 202031	Jiang et al 202032
Title
Title	CR	CR	CR	CR	CR	CR	CR	CR	CR
Abstract
Structured summary	CR	CR	CR	CR	CR	CR	CR	CR	CR
Introduction
Rationale	CR	CR	CR	CR	CR	CR	CR	CR	CR
Objectives	CR	CR	CR	CR	CR	CR	CR	CR	CR
Methods
Protocol and registration	UR	UR	UR	UR	UR	UR	CR	UR	UR
Eligibility criteria	CR	CR	CR	CR	CR	CR	CR	CR	CR
Information sources	CR	CR	CR	CR	CR	CR	CR	CR	CR
Search	UR	UR	UR	CR	CR	CR	CR	UR	CR
Study selection	CR	CR	CR	CR	CR	CR	CR	CR	CR
Data collection process	CR	CR	CR	CR	CR	CR	CR	CR	CR
Data items	CR	CR	CR	CR	CR	CR	CR	CR	CR
Risk of bias in individual studies	CR	CR	CR	CR	CR	CR	CR	CR	CR
Summary measures	CR	CR	CR	CR	CR	CR	CR	CR	CR
Synthesis of results	CR	CR	CR	CR	CR	CR	CR	CR	CR
Risk of bias across studies	CR	UR	CR	CR	CR	UR	CR	UR	CR
Additional analyses	CR	UR	CR	CR	PR	CR	CR	UR	CR
Results
Study selection	CR	CR	PR	CR	CR	PR	CR	PR	PR
Study characteristics	PR	PR	PR	PR	PR	PR	CR	PR	PR
Risk of bias within studies	CR	CR	CR	CR	UR	CR	CR	CR	CR
Results of individual studies	CR	CR	CR	CR	CR	CR	CR	CR	CR
Synthesis of results	CR	CR	CR	CR	CR	CR	CR	CR	CR
Risk of bias across studies	CR	UR	PR	CR	CR	UR	CR	PR	CR
Additional analysis	CR	UR	CR	CR	UR	CR	CR	UR	CR
Discussion
Summary of evidence	CR	CR	CR	CR	CR	CR	CR	CR	CR
Limitations	CR	CR	CR	CR	CR	CR	CR	CR	CR
Conclusions	CR	CR	CR	CR	CR	CR	CR	CR	CR
Funding
Funding	CR	CR	CR	UR	CR	UR	CR	CR	UR

Abbreviations: CR, complete report; PR, partial report; UR, unreported.

Evaluation Based on Veritas Plot

The Veritas plots (Figure 2) derived from “Veritas Scoring System” () illustrate the quality ranking of all included SRs/MAs. Among nine studies, the SR/MA with the highest Veritas score was carried out by Wang28 with 8.3 points, while the study with the lowest Veritas score was carried out by Zhang et al,30 with 3.3 points. There were five SRs/MAs11,25,28,29,32 with the Veritas scores higher than 6.4 points, which was the average score. The mean dimension rank scores of publication year, type of study, AMSTAR-2 score, PRISMA score, homogeneity, and publication bias were 6.0, 9.0, 5.4, 5.2, 6.1, and 6.3, respectively.

Figure 2

Veritas plots for the included SRs/MAs. Dashed line shows Veritas score of each SR/MA; solid line shows rank score in each quality dimension of each SR/MA.

Discussion

Summary of Findings

In our overview, evidence on the effectiveness and safety of acupuncture for PMI was synthesized from nine SRs/MAs. Overall, the existing evidence strongly insinuates that acupuncture is more effective than a placebo or hypnotics in treating PMI and other accompanied climacteric symptoms; and that, acupuncture combined with hypnotic/CHM is more effective than hypnotic/CHM alone in relieving PMI. No serious AEs are associated with acupuncture. The number and scale of original RCTs included in SRs/MAs were too few and small; and the overall methodological quality and data reporting quality in those RCTs was generally poor. These factors prevented almost all included SRs/MAs from drawing firm and solid conclusions. Furthermore, in terms of the appraisal results of AMSTAR-2, GRADE and PRISMA-2009, methodological and evidence quality of most SRs/MAs was unsatisfactory; and the reporting quality requires improvement. These findings suggest that the results of all existing SRs/MAs may overestimate the true effects of acupuncture. Hence, further studies with the improved methodological design are needed to accurately determine the real effectiveness and safety of acupuncture in the management of PMI.

Strengths and Limitations

Prior to being officially recommended, the clinical efficacy and safety of any modality including acupuncture ought to be critically and rigorously appraised in the context of evidence-based medicine.38 To the best of our knowledge, this is the first overview appraising the SRs/MAs of acupuncture treatment of PMI.

An explicit and structured approach with comprehensive retrieval strategy and internationally recognized measurement tools were applied in our review. Reliable and valid AMSTAR-2, GRADE, and PRISMA-2009 were employed to comprehensively disclose methodological shortcomings of published SRs/MAs and original RCTs, which provides critical guidance for future high-quality RCTs and SRs/MAs. We also illustrated the quality assessment of the included SRs/MAs through Veritas plots, which allows readers to visually and intuitively understand the overall quality of the available SRs/MAs regarding acupuncture for PMI, thereby guiding clinicians to use the best available evidence.

However, our overview is also subject to limitations. The major restriction is that we were unable to compare different modalities of acupuncture and discuss each of them independently due to too few SRs/MAs being available. For instance, although manual acupuncture and laser acupuncture are commonly practiced, they differ greatly in their underlying mechanism of action.16 It is therefore necessary to assess their independent therapeutic effect on PMI.

Interpretation of Findings

Except for one study published in 2012,30 the other eight SRs/MAs were published in the last 5 years (2016–2021), implying an increasing interest in the potential of acupuncture in PMI management, as well as a growing awareness in acupuncture practitioners/researchers for using systematic summaries and assessments of the body of evidence to guide clinical decision-making.

SRs/MAs, as the highest level of evidence hierarchy, are becoming increasingly crucial for evidence-based clinical decision-making.14,39 High-quality SRs/MAs contribute to producing scientific reference for evidence users, particularly health-care providers and patients when acupuncture becomes an alternative/option of treatment.17 On a more disappointing note, the overall methodological quality of the included SRs/MAs is not encouraging (Table 2). A transparent and comprehensive screening process is necessary and effective in avoiding publication bias.12 Providing a complete list of potentially relevant studies with justification for the exclusion of each is an indispensable part in this process.12,14 However, all the included SRs/MAs were criticized for item 7 of the AMSTAR-2 due to the lack of this exclusion list. It is hence impossible for readers to determine whether there is a risk in unjustified exclusion which may directly bias the findings of SRs/MAs.14 Another linchpin in the elevation of the authenticity and transparency of SRs/MAs is to provide protocols and/or registration in advance, which ensures that all methodological decisions are carefully considered and justified, as well as reducing the bias of reviewers.14,40,41 Eight out of the nine SRs/MAs, however, did not include relevant information,25–32 which can result in an unintended duplication as well as a larger adjustment of the research process than expected and subsequently weaken the rigorousness of the SRs/MAs.42 Registration on an international platform (eg, Cochrane Library, PROSPERO, INPLASY, JBI, etc.) and/or publication of protocol in advance is strongly recommended for future acupuncture-related SRs/MAs. Furthermore, the lack of registration may also affect the publication since SRs/MAs registration information is currently required for many peer-reviewed journals. Seven studies25–28,30–32 were judged as having incomplete retrieval and rated as “Partial Yes” of the AMSTAR-2, mainly due to the lack of retrieval of trial registries, reference lists of included studies, and grey literatures. The omission of retrieval processes may directly affect the result of pooled estimate effects.12 The AMSTAR-2 also identified two methodological flaws related to interest or financial support: 1) failure in reporting funding sources of original RCTs (88.9% SRs/MAs); and 2) failure in reporting competing interests (33.3% SRs/MAs). SRs/MAs with financial ties to a corporation are usually associated with favorable conclusions.43,44 To reinforce transparency of studies and reduce bias, researchers of both SRs/MAs and included original trials hence are requisite to declare their potential conflict of interest and source of funding.14 All of these findings suggested that there is still considerable room for addressing the methodological quality in this field. Researchers ought to strictly control the methodological quality of future SRs/MAs by complying with the requirements of each item of the AMSTAR-2 checklist.

The evidence quality embodies the degree to which confidence in an estimate of the effect is sufficient to support a specific recommendation.45 The evaluation based on the GRADE system demonstrated that most of the quality of evidence ranged from low to very low, and only 12% outcomes (n = 3) were rated as moderate. All these outcomes with moderate quality were total clinical effectiveness rate, and presented two major results: 1) acupuncture was more effective than hypnotic in improving PMI; and 2) acupuncture combined with CHM was more effective than CHM alone or hypnotic alone in improving PMI. All of the 25 outcomes were downgraded due to the limitations caused by risk of bias within the original RCTs. We further analyzed both results and discussion sections of the included SRs/MAs, and summarized the limitations from original RCTs as follows: 1) most RCTs did not describe whether and how allocation concealment was conducted; 2) the lack of blinding in both patients and outcome evaluators; 3) some RCTs claimed to perform randomization, but failed to explicitly state the specific methods to achieve randomization; and (4) the lack of protocol impeded a clear judgment on selective outcome reporting. Conclusions supported by the results from such inadequately powered and/or reported RCTs might be less than authentic and reliable.14 Because of the high heterogeneity that could not be completely explained, evidence quality in 60% outcomes (n = 15) were demoted according to the “inconsistency” domain of the GRADE. Quality of evidence in 52% outcomes (n = 13) were degraded due to “imprecision”, which is another critical downgrade factor and defined as the confidence interval was not narrow enough (Table 3). These findings indicated that rigorous design is urgently needed to improve the methodological quality of original RCTs regarding acupuncture for PMI, which is also a prerequisite for providing reliable clinical evidence. After all, the lower the quality of evidence, the more likely further study would change our confidence in the estimates as well as the estimates themselves.46

Accurate, complete, and transparent reporting is an indispensable component of a valuable research,47 and the prerequisite for a SR/MA to draw unbiased conclusions.14,48 However, the reporting quality of the included SRs/MAs was far from satisfactory. Some items of the PRISMA-2009 overlap with those of the AMSTAR-2 or the GRADE system. Hence, many deficiencies of reporting quality identified by the PRISMA-2009 were discussed in detail in the earlier paragraphs and were not repeated here. The major problems of reporting quality in nine included SRs/MAs were briefly summarized as follows: 1) incomplete reporting in retrieval method due to the lack of specific search strategies; 2) no access to available registration information/protocols; 3) indistinct reporting in literature screening process in some SRs/MAs because of the lack of a flow chart with clear description; it thereby was hard for readers to know which or how many articles were excluded with what reasons in each screening step; 4) incomplete description of intervention details in both therapies used for comparison; and 5) inadequate investigation and explanation for the source of high heterogeneity among original RCTs. The first three problems reduced the transparency and trustworthiness of study process.14,40,41 The fourth defect resulted in a lack of completeness of included SRs/MAs.17 The final limitation can directly weaken the credibility of the results and conclusions of those SRs/MAs, and indirectly confuse physicians and patients in utilizing the evidence produced by those SRs/MAs. On the basis of these findings, we rated the overall reporting quality of included SRs/MAs from low to moderate. To better achieve clinical translation of PMI management with acupuncture, reporting quality in future SRs/MAs and original trials deserve adequate attention and should be improved. Poor-quality reporting of medical study carries with it an extra risk of inadequate and misguiding research results being utilized by clinicians and patients, and consequently cause harm to the patients.47 Meanwhile, scarce health-care resources may be devoted and wasted on ineffective health-care treatments due to such inadequate reporting.47

Conclusion

Acupuncture, either in isolation or as an adjunct to pharmacotherapy, appears to be an effective and safe treatment option for women sustaining PMI. However, the interpretation of these “promising” outcomes should be cautiously considered due to issues related to the methods, evidence, and reporting quality of both SRs/MAs and original trials included. For future researchers to overcome these issues, it is recommended that they at least adhere to the PRISMA-2020 statement when conducting SRs/MAs, and consider adopting the norms of the AMSTAR-2 and the GRADE.