Reporting quality of polycystic ovary syndrome practice guidelines based on the RIGHT checklist.

BACKGROUND: Polycystic ovary syndrome (PCOS) is a prevalent endocrine disorder involving hyperandrogenism, menstrual disorder, metabolic problems, infertility, obesity, and acne. The main aim of this study was to assess the reporting quality of clinical practice guidelines (CPGs) in the field of PCOS to provide a reporting specification for this study.
METHODS: We evaluated the reporting quality of clinical guidelines of PCOS using the Reporting Items for Practice Guidelines in HealThcare (RIGHT) checklist. Nine databases and 3 medical associations were searched. These included Medline, Embase, PubMed, National Institute for Health and Care Excellence (NICE), Guidelines International Network (GIN), National Guideline Clearinghouse (NGC), China National Knowledge Infrastructure, Wanfang and Chinese Science, and Technology Journal Database (VIP). Three medical associations included the European Society of Human Reproduction and Embryology, the American Society for Reproductive Medicine and the Agency for Healthcare Research and Quality. Two independent authors assessed the reporting quality of PCOS CPGs by the RIGHT checklist, and Spearman's correlation was used to assess inter-rater reliability.
RESULTS: Twelve PCOS CPGs were included. On average, 20.0 (57.1%) of the 35 items in the RIGHT checklist were reported. All items were fully reported by one of these CPGs. The number of reported items ranged from 10 (28.6%) to 35 (100%). Overall, 16.7%, 66.7%, and 16.7% of included guidelines were of high, medium, and low quality, respectively. The reporting proportions of the 7 domains (i.e., Basic information, Background, Evidence, Recommendations, Reviewand quality assurance, Funding and declaration and management of interests, and Other information) in the RIGHT checklist were 62.0%, 69.1%, 53.3%, 60.7%, 33.3%, 31.2%, and 69.4%, respectively.
CONCLUSIONS: The evaluation of these CPGs by the RIGHT checklist revealed that the reporting quality varied among guidelines. Low quality items were the processes of evidence decision and the declaration of funding in most included CPGs. Guideline developers should pay more attention to these items to disseminate and implement better guidelines in near future. TRIAL REGISTRATION NUMBER: registration at PROSPERO CRD42020163435.

Introduction

Polycystic ovary syndrome (PCOS) is a heterogeneous endocrine disorder that has reproductive, metabolic, and psychological implications for affected women.[ It is characterized by hyperandrogenism, ovulatory dysfunction, and polycystic ovarian morphologic (PCOM) features. PCOS is the most common endocrine abnormality of reproductive aged women affecting 6% to 15% of reproductive-aged women with up to 70% of affected women remaining undiagnosed.[ Compared with adults, adolescent have a lower prevalence but a higher difficulty to diagnosis.[ It is important to diagnose the condition as early as possible to evaluate and treat metabolic and cardiovascular (CV) risks, as well as the psychologic and dermatologic issues. However, PCOS phenotypes vary widely depending on a patients life stage, ethnicity, genotype, and environmental factors including lifestyle and bodyweight, diagnoses and treatments are diverse and controversial. This confusion leads to delayed diagnosis, and patients report poor diagnosis experiences and dissatisfaction with care internationally.[

Trustworthy CPGs based on an important systematic review of the literature, provide ratings of the quality of evidence and the strength of recommendations, optimize patient values, and establish norms of practice.[ With increasing international attention to PCOS, a large number of CPGs for PCOS have emerged. Nevertheless, these guidelines are not comprehensive and standardized, varying in terms of the respective country's definition of PCOS.[ For instance, there are 3 recognized PCOS diagnostic criteria which are published by the National Institutes of Health (NIH) criteria, Rotterdam criteria, and the Androgen Excess and PCOS (AE-PCOS) Society criteria, different CPGs use different diagnostic criteria leading to respective recommended levels of evidence. Many doubts remain for the clinician who has to decide the diagnosis and treatment of patients. Similarly, the transparency report of the CPGs has a significant impact on the development of recommendations for the CPGs. In some previous CPGs on PCOS, it can be found that the evidence synthesis in the guideline development process and the conflicts of interest of the guideline development group were not well reported. This will directly lead to a decrease in the credibility of recommendations. Delayed diagnosis and ineffective treatment because of insufficient understanding of the diverse features of PCOS, inadequate support for quality research, and a lack of standard guidelines.[ Quality of reporting has been recognized as a key measure for successful translation of evidence to practice, reducing overall waste in research and eliminating non-replicable studies,[ but the reporting quality of CPGs seems low,[ and the existing used tools do not accurately address quality assessment and reporting in a single statement. In 2010, Chen et al from the WHO established the RIGHT checklist (Reporting Items for Practice Guidelines in Healthcare) which focused on using presentation format.[ The RIGHT checklist was developed to assist guideline developers in reporting, journal editors and peer reviewers in decision making, and health care practitioners in understanding and implementing guidelines.[ Thus far, the RIGHT checklist has not been used to evaluate clinical guidelines for PCOS.

In this study, we aimed to assess and compare the quality of reporting in internationally produced CPGs of PCOS by using the RIGHT checklist. This will help to promote the use of RIGHT checklist and to improve the reporting quality of future guidelines, as well as to provide better guidance for clinical treatment of PCOS.

Materials and methods

Study design

We used the RIGHT checklist to assess a review of PCOS CPGs.

Review protocol

This study was performed in accordance with the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses.[

Inclusion and exclusion criteria

Types of guidelines

Guidelines are including on preventive and/or therapeutic intervention in PCOS, whereas those solely describing epidemiology, diagnosis, training, research methods, or legal issues regarding PCOS were excluded. Furthermore, summarized, recommended or translated organizational guidelines, statements, comparative analysis, or correspondence studies were excluded.

Types of participants and public involvement

There were no patients involved in this study. In this study, we focused on guidelines and not participants themselves, which needs no “Patient and Public Involvement.” In those included guidelines, the primary population was adolescents, reproductive age, postmenopausal and infertile women with PCOS, not limited by course or cause of disease.

Types of interventions

There were no limitations with regard to interventions. Drug therapies and non-pharmacotherapy recommended in the guidelines were included.

Searching methods

To identify matching eligibility guidelines, we searched 9 databases: Medline (http://guide.medlive.cn), Embase (https://embase.com/), PubMed (https://www.ncbi.nlm.nih.gov/pubmed/), National Institute for Health and Care Excellence (http://www.nice.org.uk), Guidelines International Network (GIN; https://g-i-n.net/), National Guideline Clearinghouse (NGC; http://www.guideline.gov), China National Knowledge Infrastructure, and Wanfang and Chinese Science, and Technology Journal Database (VIP). We also searched 3 medical associations: the European Society of Human Reproduction and Embryology (https://www.eshre.eu/), the American Society for Reproductive Medicine (http://www.medsci.cn/), and the Agency for Healthcare Research and Quality (https://www.ahrq.gov/). We selected articles published from January 2014 to December 2019. The search strategy used the terms “polycystic ovary syndrome,” “PCOS,” or “Stein Leventhal Syndrome”, and “guideline,” “guidance,” “clinical practice guideline,” “recommendation,” or “consensus”. Two authors (HL and YZ) independently screened the literature to identify guidelines for inclusion, and only studies that met the inclusion criteria were assessed in the final analysis. Disagreements among the authors regarding inclusion were discussed and resolved by consensus after consultation with an independent reviewer.

Data extraction

We extracted general information about each guideline including title, year of publication, regions of development, source of the guidance, association responsible for publication, number of authors, target population, funding, and whether it was an updated version. Other information relating to guideline table included basic information, background, evidence, recommendations, review and quality assurance, funding, declaration and management of interests, and other information. Serial number 13 to 24 in Tables 1 and 2 means the reference number of included guidelines in this study.

Table 1

Characteristics of guidelines on PCOS.

Table 2

RIGHT Checklist.

Reporting quality assessment

The RIGHT checklist is a checklist that can be used to assess the reporting quality of CPGs and to help health care practitioners understanding and implement a guideline. It can support journal editors and peer reviewers when considering guideline reports, and assist guideline developers in reporting guidelines.[ The RIGHT checklist contains 22 requirements organized into 7 sections with a total of 35 items: basic information (6 items), background (8 items), evidence (5 items), recommendations (7 items), review and quality assurance (2 items), funding and declaration and management of interest (4 items), and other information (3 items). Two authors (HZL and YZ) independently assessed the adherence of each PCOS clinical guideline with the RIGHT checklist, and “yes” indicated full reporting of necessary information, whereas “no” indicated partial or no reporting. We defined reporting to be of high quality if the “yes” responses were >70%, medium quality if they were 40% to 70% and low quality if they were <40%. Spearman's correlation was used to assess inter-rater reliability (P > .7 indicated good inter-rater reliability). If opinions differed, a third author (LL) made a final decision. The percentage of fully reported items was expressed to assess reporting quality of guidelines.

Data analysis

Data were analyzed by using SPSS V.19.0 and Office Excel 2019, which was used to summarize the reporting rates and percentages of the RIGHT items and domains for the guidelines. The Spearman's correlation was calculated for each domain of the RIGHT instrument by using intraclass correlation coefficient with a P value.

Results

Eight hundred thirty sixrecords in total were identified by searching the database. Eighty two additional records were identified by searching the websites of guideline development organizations. After screening, 12 guidelines were ultimately included (Fig. 1).

Figure 1

PRISMA flow diagram for this study. PRISMA = Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Guideline characteristics

The characteristics of each included guideline are presented in Table 1. Of the included guidelines, 1[ produced by NHMRC was an international evidence-based guideline, and another[ guideline was specific to PCOS in adolescents. Two[ were published by the AACE (the collaboration of the American Association of Clinical Endocrinologists) in the USA (1 focused on PCOM, hyperandrogenism, and ovulatory dysfunction, whereas the other focused on insulin resistance, type 2 diabetes mellitus, cardiovascular disorder, and reproductive and genetic issues in PCOS). One guideline[ published by ACOG (American College of Obstetricians and Gynecologists) was for the diagnosis and treatment of PCOS, 1[ published in Canada mainly discussed ovulation induction, 1[ produced by the UK outlined long-term consequences of PCOS, and 2[ guidelines referred to treatments involving a ketogenic diet and metformin. Two[ guidelines were published in China, 1 by the endocrinology and metabolism branch of the Chinese physicians association, and another by the endocrinology and guidance group of the obstetrics and gynecology branch of the Chinese medical association. Both guidelines in China referred to treatments involving traditional Chinese medicine (TCM), but were limited to a summary of this approach. The correlation between the 2 estimators is shown in Table 3. The Spearman's correlation coefficient was 0.98 and the P of each item was >.7, indicating that the reliability of our results was high.

Table 3

RIGHT Checklist.

Quality of reporting evaluation by RIGHT

Twelveparameters in total for the quality of guideline reporting were evaluated by the RIGHT checklist. Most of the clinical guidelines were of medium quality (Table 2). We strictly assessed each item in accordance with the standard and calculated the percentage of fully reported items. 20.0 (57.1%) of the 35 items in the RIGHT checklist were reported in average. The number of reported items ranged from 10 (28.6%) to 35 (100%) across the guidelines. The guideline, which produced by NHMRC, reported all items. Among them, 16.7% of the included guidelines were of high quality, 66.7% were of medium quality, and 16.7% were of low quality overall, (Fig. 2). The distribution of CPGs across the world is shown in Figure 3.

Figure 2

Quality of reporting evaluation by RIGHT. (A) Reporting quality for each guideline. The horizontal axis represents the number of CPGs corresponding to reference numbers. (B) Overall evaluation of reporting quality. (C) Reporting proportion of 7 domains (i.e., Basic information, Background, Evidence, Recommendations, Review and quality assurance, Funding and declaration and management of interests, and Other information) in the RIGHT checklist. The horizontal axis represents each domain.

Figure 3

The distribution of CPGs across the world, where the circular areas represent the numbers of guidelines in each country.

As for special items, the most frequently reported items among the 35 items in the RIGHT checklist were related to the key elements of the guidelines such as items1a, 7a, 10a, and 13a, however, the reporting rates of the decision process, the quality control description, details of the development process, and funding statements were low, as reported in items 151,718a and 18b.

The reporting proportion of the 7 domains (i.e., Basic information, Background, Evidence, Recommendations, Review and quality assurance, Funding and declaration and management of interests, and Other information) in the RIGHT checklist were 62.0%, 69.1%, 53.3%, 60.7%, 33.3%, 31.2%, and 69.4%, respectively (Fig. 2).

Basic information

Total 9 guidelines summarized in the recommendations, only 3 of them[ summarized in an executive summary. More than half of the guidelines reported the corresponding developers or authors, whereas 5 (41.7%) of the guidelines showed a list of abbreviations or acronyms. In this term, the reporting rate of foreign CPGs was higher than Chinese CPGs.

Background

The rate of reporting of the background was comparatively high among all 7 domains, but there were still many details which required improvement. Three guideline[ did not describe the epidemiology of PCOS, 1 guideline[ did not clearly describe the aim of the guideline and the intended primary users, half of the guidelines had no subgroups, 2 guidelines[ did not list all involved individuals who developing the guideline, and eight guidelines (83.3%) did not describe the specific roles and the authors responsibilities Furthermore, only 2 [ were intended to focus on the Australian or American healthcare settings.

Evidence

Reporting of evidence was insufficient. Although all CPGs reported on the key question on which the guideline was based, only 3[ guidelines completely reported on how the outcomes were selected and sorted. Most included guidelines indicated that they were based on systematic reviews, but the adherence of the use of systematic reviews and selection basis was >60%, insufficient description and unclear detail. Five[ guidelines assessed the quality of the body of evidence by using the

Grading of Recommendations Assessment, Development and Evaluation approach (GRADE) evidence profiles.

Recommendation

Clear, precise, and actionable recommendations were provided in every guideline, but 7 (58.3%) guidelines did not described the strength of the recommendation. Five[ did not present specified recommendations for PCOS in adolescents. Most of included CPGs considered the values and preferences of the target population, however there were partial reports on whether the CPG developers considered costs and resources implications in the formulation of recommendations. Only 1[ stated that guideline development groups had made decisions through repeated discussions, and included voting when making its final recommendations.

Review and quality assurance

Four [ guidelines gave the final draft to an independent reviewer and underwent quality assurance processes: 1 guideline reported the peer reviewers and the review process, while 3 guidelines mentioned that the guideline had undergone a review only.

Funding and declaration and management of interests

Four[ guidelines described the specific sources of funding but only one[ CPG made by NHMRC sufficiently reported the specific sources of funding for all stages and all feedback was reviewed by the project board. Five[ guidelines did not describe the conflicts of interest which were relevant to the guideline developers. Only 3[ precisely declared the management of conflicts of interest.

Other information

Most guidelines provided accession websites for the full guidelines and their related documents but 2[ did not. Four[ guidelines did not describe the gaps in the evidence and provide suggestions for further research. Five[ guidelines did not indicate the limitations of the presented evidence.

Discussion

In this study, we identified 12 CPGs on PCOS published in the world, including both western medicine and traditional medicine (TM). Evaluation of these CPGs by using the RIGHT checklist revealed that the reporting quality varied among guidelines. One[ CPG made by NHMRC showed very high quality and favorable recommendations, this guideline could be used by clinical and public healthcare providers as the basis for management of PCOS and could be regarded as a report specification for guideline developers. However, considering individual differences among patients and actual situation among regions, each recommendation should be used with caution. Other included CPGs showed low quality in some items, for example, guideline application setting, healthcare related questions, evidence to decision processes, quality assurance, funding source(s) and role(s) of the funder, therefore, guideline developers should pay more attention to these items to disseminate and implement better guidelines in future.

The main strength of this study lies in the fact, which the quality of CPGs in PCOS was evaluated by using the scientific international guidelines reporting the standard evaluation checklist—RIGHT. Two researches independently conducted a systematic search and evaluation. This study found that CPGs for PCOS are wide-ranging and have been published in many regions.

However, according to RIGHT checklists, the reporting quality varied among 12 guidelines. Only 2 of the included guidelines reported the specific healthcare environment to avoid inappropriate applicability of recommendations. Furthermore, the greatest limitation in some included guidelines was a lack of consideration on the different recommendations between adults and adolescents with PCOS. For example, the uncertainty regarding to appropriate ultrasonography criteria for PCOM in adolescents is too great to use PCOM as a diagnostic criterion in adolescents,[ but it is a common method to assess PCOM in reproductive women.[ An ideal clinical guideline should guide clinical doctors and avoid improper interventions by clearly indicating the special management of PCOS in adolescents from our opinion. Three guidelines mentioned TCM interventions for PCOS, but the reporting quality of these guidelines were really poor. As we all knew, glucocorticoid as a common treatment for PCOS has many adverse effects such as skin, gastro-intestinal, adrenal, cardio-metabolic, neuropsychiatric systems and so on.[ Thus, to reduce the side effects of glucocorticoid drugs, TM or other complementary alternative therapies should be recommended for use in clinical applications according to a systematic review. The findings of this study may serve as an alert for PCOS guidelines development and improve the strength of evidence related to TM.

Limitations

There were also some limitations to this review. Only 9 databases mentioned in the study were searched, which can not cover all channels for guideline publication. The included papers were only published in English and Chinese, resulting in omission of guidelines published in other geographical locations such as Turkey, Korea, and India. Individual differences are existing among patients, then the recommendations should be implemented for specific conditions and settings. Furthermore, there are 3 internationally recommended diagnostic criteria for PCOS by varying expert groups, the NIH Criteria, Rotterdam Consensus Criteria, and AE-PCOS, but we did not classify results by different diagnostic schema; thus, one should be cautious when extrapolating conclusions. Last but not the least, it is important to keep in mind that the RIGHT checklist does not use to assess the quality of the guideline methodology and the effectiveness of the recommendations in the guidelines, but it can help healthcare practitioners to understand and implement a guideline, to assist developers in reporting, and verifying the clarity of the guidelines presentation. In the future, it is necessary to appraise CPGs of PCOS by AGREE II tool to assess the methodology quality. Future research should focus on diagnosis patients with PCOS as early as possible and effective interventions other than those reviewed here in a manner supported by sufficient evidence to facilitate guideline development.

Conclusion

The evaluation of these CPGs by the RIGHT checklist revealed that the reporting quality varied among guidelines. Low quality items were the processes of evidence decision and the declaration of funding in most included CPGs. Guideline developers should pay more attention to these items to disseminate and implement better guidelines in the near future.

Acknowledgments

We are grateful to Dr LL for all scientific support during this study.

Author contributions

HL and YZ contributed equally to this work. This review was drafted by HL and YZ and revised by WY and LL. HL and YZ searched the database, selected guidelines, and completed the data synthesis independently. WY and LL arbitrated in cases of disagreement and all authors have read and approved the final manuscript.

Conceptualization: Hongzhu Li, Yu Zhang.

Data curation: Hongzhu Li, Yu Zhang.

Formal analysis: Hongzhu Li, Yu Zhang.

Investigation: Yu Zhang.

Methodology: Liming Lu.

Software: Liming Lu.

Supervision: Liming Lu, Wei Yi.

Validation: Wei Yi.

Writing – original draft: Hongzhu Li, Yu Zhang.

Writing – review & editing: Liming Lu, Wei Yi.