Quality assessment of conventional and traditional oriental medicine clinical practice guidelines for knee osteoarthritis using AGREE II instrument.

INTRODUCTION: Knee osteoarthritis is a degenerative disease and its prevalence tends to increase. Clinical practice guidelines (CPGs) are evidence-based recommendations for treatment that help policymakers, practitioners, and patients make more appropriate and efficient decisions during the course of management. This study aimed to evaluate the quality of knee osteoarthritis CPGs using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument.
METHOD: The retrieval engines and websites were utilized from January 2010 to December 2020. The search words were "Clinical practice guideline" OR "Critical practice guideline" OR "guideline∗" AND "Osteoarthritis." The quality of the CPGs was independently examined by four appraisers using the AGREE II instrument. Consequently, the selected CPGs were graded as Classes A, B, and C according to the level of recommendation. RESULT: In this study, 13 CPGs for knee osteoarthritis were selected and evaluated qualitatively using the AGREE II instrument. The overall quality percentage score was as follows: clarity of presentation, 72.6%, scope and purpose, 62.6%, rigor of development, 54.2%, stakeholder investment, 50.5%, editorial independence, 46.5%, applicability, 22.5%.
CONCLUSION: Auxiliary materials for the treatment process of knee OA should be supplemented in future revised versions for quality improvement of knee OA CPGs. Also, more evidence should be accumulated to support the recommendation of traditional oriental medical treatments in the clinical field. From the perspective of integrative medicine, along with conventional pharmacological treatment, exercise, weight loss, and acupuncture can be combined together in clinical situations.

Introduction

Degenerative arthritis is the most common form of arthritis, and it tends to occur in joints of the lower extremities such as the knee and hip, during the aging process. The prevalence of knee osteoarthritis (OA) has also increased in recent years due to an increase in the number of older adults and the prevalence of obesity in the entire population.[ The articular cartilage of a joint is a connective tissue that does not include blood vessels, lymph, and nerves, and provides a joint surface that minimizes friction during exercise. The mechanism of knee OA is summarized as type II collagen decomposition and degenerative changes brought about by collagen-degrading enzymes, which leads to a gradual decrease in the ability to recover.[ Knee OA is a disease that can induced naturally with increasing age, manifesting great socioeconomic costs and burden in each patient.[ Thus, it is necessary to discuss the most effective treatment methods for knee OA.

Pain and motor function limitations due to cartilage loss are common, and management is more important than treatment for knee OA. Conventional treatments for knee OA, such as non-steroidal anti-inflammatory drugs (NSAIDs) and injections, have limitations in pain relief and prevention of joint damage. Therefore, to achieve the goal of alleviating symptoms and improving quality of life, effective integrative approach in knee OA is required.[

Clinical practice guidelines (CPGs) are evidence-based recommendations for clinical treatment that help policy makers, practitioners, and patients make more appropriate and efficient decisions regarding the treatment plan.[ To provide highly appropriate and high-quality care, it is necessary to evaluate the quality of CPGs. It has become important to select CPGs with high validity. To fulfill this objective, sufficient financial support, support from professional organizations, and systematic programs must be provided.[ In addition, evaluating of the quality of existing developed guidelines is also required because the production process of clinical pathway (CP)s and CPGs is conducted by referring to existing CPGs.

The World Health Organization developed and approved an international assessment instrument called “Appraisal of Guidelines for Research and Evaluation II (AGREE II)” to evaluate the quality of CPGs.[ The objective of this study was to evaluate the quality of each CPG for knee OA treatment by extracting and analyzing the CPGs published from 2010 to 2020 using the AGREE II instrument and to select high-quality CPGs for an integrative treatment. The objective of this research was to select high-quality medical guidelines for traditional oriental and conventional medicine, and eventually conflate bilateral high-quality CPGs for integrative treatment of CP. Therefore, information on diagnosis, evaluation, treatment, management as exercise and weight loss, prognosis, and follow-up in high-quality CPGs will be evaluated. These elements will serve as the basis for creating integrative knee OA CP in the future.

Methods

Search strategy of clinical practice guidelines

Websites and electronic databases for knee OA CPGs were also searched. The search period was limited to a 10-year period, from 2010 to 2020. If there were revisions in this period, this study used the revised CPGs while excluding older versions.

The databases and websites used in this article were the Guidelines International Network (GIN; www.g-i-n.net), and the National Institute for Health and Clinical Excellence (NICE; www.nice.org.uk), Australian Clinical Practice Guidelines (https://www.clinicalguidelines.gov.au), Canadian Clinical Practice Guidelines (https://www.cma.ca/En/Pages/clinical-practice-guidelines.aspx), Scottish Intercollegiate Guidelines Network (SIGN; www.sign.co.uk), Korean Medical Guideline Information Center (KoMGI;guideline.or.kr), National Clearinghouse for Korean Medicine (NCKM; http://www.nckm.or.kr/main/index.do), MEDLINE, EMBASE, Cochrane Library, China National Knowledge Infrastructure (CNKI), WanFang database, Citation Information by NII (CiNii), and Research Information Sharing Service (RISS).

The search words for the CPGs were “clinical practice guideline,” “critical practice guideline,” and “osteoarthritis” for each of the aforementioned databases and websites. The specific search methods were as follows: “Clinical practice guideline” OR “Critical practice guideline” OR “Guideline∗” AND “Osteoarthritis.” The search strategy was adjusted for each database and website.

Selection of clinical practice guidelines

All CPGs for traditional oriental medicine including Korean, Chinese, and Japanese traditional medicine, and conventional medicine supported by an official global medical organization at the time of writing that were related to the diagnosis, evaluation, management, treatment, prevention, and follow-up were selected for inclusion in this study. The exclusion criteria for CPGs were as follows:

guidelines that only covered surgical treatment;

guidelines without any recommendations;

secondary publication from CPGs;

consensus conferences based on the opinion of panels;

systematic reviews, clinical trials, and editorials;

published in languages other than English, Korean, Japanese, and Chinese; and

guidelines that only covered OA in other areas (e.g., hip joint, hand joint, shoulder joint).

CPGs dealing with multiple sites of OA occurring simultaneously, including the knee, were included in this study. Titles, abstracts, and a summary of recommendations were manually screened to identify eligible CPGs according to the inclusion criteria. Two reviewers independently screened and assessed each CPG title, abstract, and summary. Full texts were then obtained, which the reviewers screened using language and evidence/recommendation criteria. If disagreements occurred at any stage during the selection procedure, they were resolved by discussion or involvement of a third reviewer. The specific procedure for CPG selection is described in the flowchart in Figure 1.

Figure 1

The flow chart of clinical practice guideline (CPG) selection. CiNii = Citation Information by NII, CNKI = China National Knowledge Infrastructure, GIN = Guidelines International Network, KoMGI = Korean Medical Guideline Information Center, NCKM = National Clearinghouse for Korean Medicine, NICE = National Institute for Health and Clinical Excellence, RISS = Research Information Sharing Service.

Data collection and quality assessment

Two reviewers (JY and YC) independently extracted eligible CPGs and classified the CPG characteristics (e.g., country, organization, year of publication, number of authors, number of references, target population, subject, and treatment). If disagreements occurred, they were resolved by conversation or by a third reviewer (YH).

According to the AGREE II instrument appraising manual, at least two evaluators were required, and four evaluators were suitable for increasing the reliability of the quality evaluation of CPGs. For this reason, we recruited four appraisers (JY, YC, BH, and JH), assessed all of the included CPGs, and scored each CPG according to the AGREE II instrument scoring criteria. The evaluators were Korean medicine doctors majoring in acupuncture and moxibustion and working in a university hospital as a specialist at a joint medical center and have experience in developing Korean medicine guideline for lumbar disc herniation[ and shoulder joint pain.[ Before evaluating each included CPG, the assessors received the Korean version of the AGREE II instrument guidebook, and the actual evaluation was conducted using the AGREE II Korean version developed by the Korean Academy of Medical Sciences in 2011. After being acquainted with the above guidebook, all evaluators independently assessed the included CPGs about knee OA using the AGREE II instrument. Each evaluator independently scored CPGs by each section of the AGREE II instrument, and each item was scored in a range of 1 to 7 points. The AGREE II tool consists of 23 items and is categorized into 6 sections.

Scope and purpose: the overall objective of the guideline, the health question covered by the guideline, and the targeted population.

Stakeholder involvement: the relevant professional group in guideline development, views and preferences of the target population, and target users of the guideline.

Rigor of development: systematic methods used for evidence, criteria for selecting evidence clearly described, strengths and limitations of the body of evidence, methods for formulating the recommendations, health benefits, side effects and risks, the explicit link between the recommendations, the guideline reviewed by experts prior to publication, and the procedure for updating the guidelines.

Clarity of presentation: specific and unambiguous recommendations, different options for management of the health issues, and easily identifiable key recommendations.

Applicability: the facilitators and barriers of guideline application, the advice and/or tools for practice, the resource implications of applying recommendations, and the monitoring and/or auditing criteria.

Editorial independence: the funding body's influence on the guideline, and the competing interests of the group members in guideline development.

After the four appraisers individually scored each included CPGs using the AGREE II instrument, the overall assessment of CPGs was deducted, and the overall level of recommendation for each CPG was described. After the evaluators independently evaluated each item, they all discussed the detailed evaluation criteria of the corresponding item when there was a difference of 3 or more out of 7 points by comparing the scores for each item. The objective of this procedure is to increase the reliability of the study by minimizing discrepancies between evaluators and conducting a consistent evaluation between each evaluator. Through this process, each CPG was evaluated on a more consistent basis than that in the original evaluation.

Domain scores were calculated using the AGREE II calculation system ([obtained score—minimum possible score] divided by [maximum possible score—minimum possible score]) and expressed on a scale of 0% to 100% by multiplying by 100. The final decision of “recommend,” “recommend with modification,” or “do not recommend” was individually made by each evaluator, and an assent was reached for each CPG. The decisions for the CPGs were classified into three levels according to the score of each section, and four appraisers finally determined the level of CPGs as Class A, B, and C.

Class A (high quality, recommend): At least 4 out of the 6 domain scores were calculated as >60%, with one of the domains being section 3 (rigor of development).

Class B (medial quality, recommended with modification): The number of domains with a score of ≥30% was >3, but ≤60% of the fields needed appropriate modification.

Class C (low quality, not recommended): The number of domains with a score of <30% was ≥3 and could not be recommended in clinical situations.

According to a previous appraisal of CPGs in other diseases, there was no apparent guidance in the AGREE II instrument for the general recommendation of CPGs.[ In other previous appraisals, the recommendation level of each domain was set up as “recommended” >60%, “recommended with modification” between 30% and 60%, and “not recommended” for scores <30%.[ The decision to use the recommended overall classification of CPGs as Class A, B, or C was established with reference to previous quality assessments of the CPG criteria.[

Summary of CPGs recommendations

The recommendations derived from each CPG were classified by the subjects and treatments and are summarized in tables for concise comparison. The general information of CPGs is organized in the following tables, where readers can easily obtain information about included traditional oriental medicine CPGs and conventional medicine CPGs.

Statistical analysis

To obtain relevant statistics, the authors used the total score provided by each appraiser and the score for each section. The data were obtained by calculating the score of each CPG after applying the AGREE II instrument using Excel version 2010 (Microsoft Corporation, Redmond, WA). The authors judged that institutional review board approval (IRB) was not necessary because this was a literature study and that there was no direct involvement with patients or body samples.

Results

Study selection

Of the 440 papers retrieved from each database and website, 53 were selected for full-text screening. Forty were excluded because they were secondary publications from CPGs, or just covered surgical treatment, guidelines without any recommendations, or old version guidelines. After this procedure, 13 articles were included in the critical appraisal. The flow diagram (Fig. 1) shows the details of the excluded articles and explains the reasons for exclusion. The selected CPGs are as follows: American Academy of Orthopedic Surgeons (AAOS) 2013,[ Royal Australian College of General Practitioners (RACGP) 2018,[ Malaysian Health Technology Assessment Section (MaHTAS) 2013,[ National Clinical Guideline Center (NCGC) 2014,[ Royal Dutch Society for Physical Therapy (KNGF) 2020,[ Ottawa Panel (OTP) 2017,[ Osteoarthritis Research Society International (OARSI) 2019,[ American College of Rheumatology (ACR) 2020,[ Korean Acupuncture and Moxibustion Medicine Society (KAMMS) 2017,[ Wangjing Hospital of China Academy of Traditional Chinese Medical Science (WHCA) 2020,[ Orthopedic Section of China Association of Chinese Medicine (OCACM) 2019,[ China Journal of Joint Surgery (CJJS) 2018,[ and the National Clinical Research Center for Geriatric Disorders (NRCGD) 2020.[

Clinical practice guideline components

General information on the included guidelines is provided in Table 1. Four CPGs were developed by China, followed by the United States (n = 3), and other countries (Republic of Korea, UK, Malaysia, Australia, the Netherlands, Canada). The areas included in the CPGs were categorized as “diagnosis,” “evaluation,” “management,” “treatment,” “prevention,” and “follow-up.” Treatment items were subdivided into “conservative treatment,” “pharmacological treatment,” “injection treatment,” “surgical treatment,” and “alternative medicine treatment.” One CPG did not report the exact number of authors, and another CPG did not report the exact number of references.

Table 1

General information on the included guidelines.

Study ID	Country	Organization	Year	Number (authors)	Number (reference)	Target population	Subject	Treatment
AAOS 2013	USA	American Academy of Orthopedic Surgeons	2013	42	137	Adults over age 19	1. Treatment	1. Conservative 2. Pharmacological 3. Injection 4. Surgical
RACGP 2018	Australia	Royal Australian College of General Practitioners	2018	19	127	Adults	1. Diagnosis 2. Management 3. Treatment 4. Evaluation	1. Conservative 2. Pharmacological 3. Injection 4. Surgical
MaHTAS 2013	Malaysia	Malaysia Health Technology Assessment Section	2013	33	112	Adults	1. Diagnosis 2. Management 3. Treatment 4. Evaluation	1. Conservative 2. Pharmacological 3. Injection 4. Alternative 5. Surgical
NCGC 2014	UK	National Clinical Guideline Centre	2014	25	514	Adults	1. Diagnosis 2. Treatment 3. Management 4. Follow-up	1. Conservative 2. Pharmacological 3. Injection 4. Surgery
KNGF 2020	Netherland	Royal Dutch Society of Physical Therapy	2020	4	102	Adults before arthroplasty	1. Diagnosis 2. Treatment	1. Conservative
OTP 2017	Canada	University of Ottawa	2017	28	225	OA patients	1. Treatment	1. Conservative
OARSI 2019	USA	Osteoarthritis Research Society International	2019	21	39	OA patients	1. Treatment	1. Conservative 2. Pharmacological 3. Injection
ACR 2020	USA	American College of Rheumatology	2020	32	41	OA patients	1. Treatment	1. Conservative 2. Pharmacological 3. Injection
KAMMS 2017	Republic of Korea	Korean Acupuncture and Moxibustion Medicine Society	2017	23	Not reported	Adults	1. Diagnosis 2. Assessment 3. Treatment 4. Prevention 5. Management	1. Acupuncture 2. Phrmacoacupuncture 3. Herbal medicine 4. Moxibustion 5. Acupotomy 6. Cupping 7.Conservative
WHCA 2020	China	Wangjing Hospital of China Academy of Traditional Chinese Medical Science	2020	8	54	Adults	1. Diagnosis 2. Treatment 3. Prevention	1. Conservative 2. Chuna 3. Acupuncture&Moxibustion 4. Herbal medicine 5. Western medicine 6. Injection 7. Surgical
OCACM 2019	China	Orthopedics Section of China Association of Chinese Medicine	2019	Not reported	47	Adults	1. Diagnosis 2. Treatment 3. Management	1. Conservative 2. Herbal medicine 3. Acupuncture&Moxibustion 4. Western medicine 5. Injection 6. Surgical
CJJS 2018	China	Chinese Journal of Joint Surgery	2018	Not reported	36	Adults	1. Treatment	1. Injection
NRCGD 2020	China	National Clinical Research Center for Geriatric Disorders	2020	32	82	Adults	Diagnosis Treatment	1. Conservative 2. Pharmacological 3. Injection 4. Surgical

AAOS = American Academy of Orthopedic Surgeons, ACR = American College of Rheumatology, CJJS = China Journal of Joint Surgery, KAMMS = Korean Acupuncture and Moxibustion Medicine Society, KNGF = Royal Dutch Society for Physical Therapy, MaHTAS = Malaysian Health Technology Assessment Section, NCGC = National Clinical Guideline Centre, NRCGD = National Clinical Research Center for Geriatric Disorders, OARSI = Osteoarthritis Research Society International, OCACM = Orthopedic Section of China Association of Chinese Medicine, OTP = Ottawa Panel, RACGP = Royal Australian College of General Practitioners, WHCA = Wangjing Hospital of China Academy of Traditional Chinese Medical Science.

Most of the target populations of each CPG are adults, and they are associated with the disease characteristics of OA, in which degenerative changes occur frequently in older adults. Diagnosis and treatment are covered by most CPGs, but few CPGs cover follow-up and prevention.

Treatment was described as conservative treatment, pharmacological treatment, injection treatment, surgical treatment, and alternative medicine treatment as aforementioned, but the subcategory of CPGs dealing with traditional oriental medicines was further subdivided into “acupuncture,” “moxibustion,” “chuna,” “herbal medicine,” “acupotomy,” and “cupping.”

AGREE II appraisal results

A total of 13 CPGs for knee OA were reviewed using AGREE II; four reviewers were assigned to each CPG. The results of the assessments for each CPG are described in Table 2.

Table 2

AGREE II Domain-Standardized Scores for Selected CPGs and Average Overall Quality Score, and Overall Assessment of the Reviewers.

Study ID	Scope and purpose (%)	Stakeholders involvement (%)	Rigor of development (%)	Clarity of presentation (%)	Applicability (%)	Editorial independence (%)	Overall assessment
AAOS 2013	54	64	79	72	7	63	A
RACGP 2018	93	67	65	96	43	81	A
MaHTAS 2013	75	54	44	82	27	63	B
NCGC 2014	99	86	88	96	72	71	A
KNGF 2020	79	40	49	69	7	38	B
OTP 2017	32	39	42	64	0	24	B
OARSI 2019	72	56	60	83	29	71	A
ACR 2020	75	53	68	82	18	42	B
KAMMS2017	92	78	78	83	34	54	A
WHCA 2020	26	24	15	57	3	15	C
OCACM 2019	43	31	17	60	2	15	C
CJJS 2018	35	31	46	43	26	35	B
NRCGD 2020	39	33	54	57	24	33	B

AAOS = American Academy of Orthopedic Surgeons, ACR = American College of Rheumatology, CJJS = China Journal of Joint Surgery, KAMMS = Korean Acupuncture and Moxibustion Medicine Society, KNGF = Royal Dutch Society for Physical Therapy, MaHTAS = Malaysian Health Technology Assessment Section, NCGC = National Clinical Guideline Centre, NRCGD = National Clinical Research Center for Geriatric Disorders, OARSI = Osteoarthritis Research Society International, OCACM = Orthopedic Section of China Association of Chinese Medicine, OTP = Ottawa Panel, RACGP = Royal Australian College of General Practitioners, WHCA = Wangjing Hospital of China Academy of Traditional Chinese Medical Science.

The general scores of each CPG had consistent characteristic. In general, the CPGs received low scores in the area of “applicability.” However, a high score was recorded in the area of “clarity of presentation.”

The overall evaluation of each included CPG was as follows: five under Class A, six under Class B, and two under Class C. Although evaluated as class A, AAOS 2013[ received a low score in the area of “applicability” that necessitated supplementation of the area through revision. Excluding the applicability part, most of the included CPGs were evaluated as generally recommended clinical guidelines. The assessment results are listed in Table 2.

Scope and purpose

The evaluation of this section demonstrates the overall aim of the CPGs, the specific health questions, and the target population. It was concerned with whether or not the objective and target population of and the health question in the guideline was described manifestly. The average percentage score for this section was 62.6%, and the range of scores was 35% to 99%. NGCG 2014[ recorded the highest percentage score at 99%, while CJJS 2018[ recorded the lowest percentage score at 35%. Seven CPGs were above 60%, and none of the CPGs scored below 30%. RACGP 2018,[ NCGC 2014,[ and KAMMS 2017[ all presented specific information on the target population, and the key questions on the guidelines were summarized in a “patient/population, intervention, comparison, and outcomes” format. CJJS 2018[ provided general recommendations for treating knee OA, although there were no clinical data in the form of obvious questions, and the description of the target population was ambiguous.

Stakeholder involvement

This section evaluated whether the CPGs covered a wide range of perspectives and interests of clinical experts and patients. It was concerned with whether CPGs included appropriate professional individuals in the processing of guidelines, the perspective of the target patient group was considered, and the target users of the guideline were clearly verified. The average percentage score for this section was 50.5%, and the range of scores was 24% to 86%. NCGC 2014[ recorded the highest percentage score at 86%, while WHCA 2020[ recorded the lowest percentage score at 24%. Four CPGs were above 60%, and one was below 30%. NCGC 2014[ and KAMMS 2017[ selected appropriate professionals, including health economists, statisticians, and methodologists, in the process of developing guidelines. NCGC 2014[ described patients’ wording specifically in the section “1.5 The impact on the individual.” WHCA 2020[ did not provide any information on the guideline processing group and did not mention the perspective of the target population and the main users of the guidelines.

Rigor of development

The evaluation in this section represents the methodological rigor in the development of CPGs. This domain was related to systematic searching strategy, inclusion/exclusion criteria of the selection, quality assessment tool, method of drawing recommendation, consideration of side effects and risk factors, clear correlation between recommendations and evidence, revision conducted by external experts, and appropriate update procedure of guidelines. The average percentage score for this section was 54.2%, and the range of scores was 15% to 88%. Six CPGs were above 60%, and two scored below 30%. Since one of the objectives of this study was to determine a more methodologically appropriate development process of traditional oriental medicine CPGs, it was necessary to consider the details of this section and the results of calculating the scores (Table 3).

Table 3

Rigor of development: standardized average scores in AGREE II domain for selected CPGs.

Title ID	Systematic methods used (%)	Selecting criteria presented (%)	The strengths and limitations of the body of evidence (%)	Formulating the recommendations described (%)	Benefits, side effects, and risks considered (%)	Explicit link between the recommendations and evidence (%)	Reviewed by external experts (%)	Updating procedure provided (%)
AAOS 2013	100	83	92	100	100	58	25	71
RACGP 2018	75	79	88	88	79	75	0	38
MaHTAS 2013	83	4	38	33	67	67	21	42
NCGC 2014	88	100	100	100	96	79	38	100
KNGF 2020	100	79	71	29	63	21	33	0
OTP 2017	38	67	46	63	58	63	0	4
OARSI 2019	100	71	92	83	70	54	8	0
ACR 2020	96	83	100	88	67	83	29	0
KAMMS 2017	96	92	83	88	75	50	58	79
WHCA 2020	0	0	0	21	58	42	0	0
OCACM 2019	0	0	0	38	58	42	0	0
CJJS 2018	63	29	50	42	67	58	25	33
NRCGD 2020	71	67	46	63	71	58	29	29

AAOS = American Academy of Orthopedic Surgeons, ACR = American College of Rheumatology, CJJS = China Journal of Joint Surgery, KAMMS = Korean Acupuncture and Moxibustion Medicine Society, KNGF = Royal Dutch Society for Physical Therapy, MaHTAS = Malaysian Health Technology Assessment Section, NCGC = National Clinical Guideline Centre, NRCGD = National Clinical Research Center for Geriatric Disorders, OARSI = Osteoarthritis Research Society International, OCACM = Orthopedic Section of China Association of Chinese Medicine, OTP = Ottawa Panel, RACGP = Royal Australian College of General Practitioners, WHCA = Wangjing Hospital of China Academy of Traditional Chinese Medical Science.

NCGC 2014[ recorded the highest percentage score at 88%, while WHCA 2020[ and OCACM 2019[ recorded the lowest percentage scores at 15% and 17%, respectively. NCGS 2014[ presented the basis of each recommendation, demonstrated as the study design and quality evaluation summarized in the evidence tables. WHCA 2020[ and OCACM 2019[ both lacked a detailed description of the type of search engine, search word, recommendation derivation technique, and detailed method of quality evaluation, and there was no description of the revision process by external experts. To acquire high scores in this section, the guideline developing group of future CPGs should remember that clear descriptions of quality assessment tools such as Grading of Recommendations, Assessment, Development and Evaluations, formal consensus techniques such as Delphi, revision schedule with detailed revision conditions, and an external expert review process should be included. In our included CPGs, the average percentage score of each digit was 70.0%, 60.9%, 62.0%, 64.3%, 71.5%, 57.7%, 20.5%, and 30.5% respectively, as calculated from Table 3.

Clarity of presentation

The evaluation of this section demonstrates the clarity of the included CPGs. This section assessed whether the recommendations were concise and specific. It also assessed whether various options for screening, prevention, diagnosis, and treatment were offered, and if the key recommendations were easily founded in each guideline. The average percentage score for this section was 72.6%, and the range of scores was 43% to 96%. RACGP 2018[ and NCGC 2014[ recorded the highest percentage score at 96%, and CJJS 2018[ recorded the lowest percentage score at 43%. Ten CPGs scored above 60%, and none of the CPGs scored below 30%. The major objective of this study was to assess traditional and conventional medicine CPGs and to evaluate whether these CPGs provide appropriate information about screening, prevention, diagnosis, and treatment. After the evaluation, the ultimate goal of this study was to combine CPGs to develop the most appropriate traditional-conventional cooperative CP of knee OA in the future. Each digit of this section was calculated in detail and is presented in Table 4. In our included CPGs, the average percentage score of each digit was 71.1%, 73.6%, and 73.3%, calculated based on Table 4.

Table 4

Clarity of presentation: standardized average scores in AGREE II domain for selected CPGs.

Title ID	Specific and unambiguous recommendations (%)	Various options for management (%)	Easily identifiable key recommendations (%)
AAOS 2013	67	58	92
RACGP 2018	88	100	100
MaHTAS 2013	79	83	83
NCGC 2014	88	100	100
KNGF 2020	71	63	75
OTP 2017	50	58	83
OARSI 2019	67	83	100
ACR 2020	67	83	96
KAMMS 2017	71	79	100
WHCA 2020	67	71	33
OCACM 2019	75	75	29
CJJS 2018	67	33	29
NRCGD 2020	67	71	33

AAOS = American Academy of Orthopedic Surgeons, ACR = American College of Rheumatology, CJJS = China Journal of Joint Surgery, KAMMS = Korean Acupuncture and Moxibustion Medicine Society, KNGF = Royal Dutch Society for Physical Therapy, MaHTAS = Malaysian Health Technology Assessment Section, NCGC = National Clinical Guideline Centre, NRCGD = National Clinical Research Center for Geriatric Disorders, OARSI = Osteoarthritis Research Society International, OCACM = Orthopedic Section of China Association of Chinese Medicine, OTP = Ottawa Panel, RACGP = Royal Australian College of General Practitioners, WHCA = Wangjing Hospital of China Academy of Traditional Chinese Medical Science.

In particular, KAMMS 2017,[ WHCA 2020,[ and OCACM 2019[ are traditional oriental medicine CPGs, which include herbal medicine, acupuncture, moxibustion, acupotomy, cupping, chuna, and pharmacopuncture. These CPGs cover various treatment tools and provide the diagnostic criteria of traditional oriental medicine's “pattern identification.” Knee OA was classified into five patterns:

liver kidney yin deficiency,

qi obstruction due to bloodstream malfunction,

wind and cold moisture,

moisture and heat, and

qi-hyul weakness.

Applicability

This section evaluated whether each CPG described appropriate socioeconomic factors, clinical assistant materials (e.g., checklist, algorithm, survey tool), and supervision and evaluation criteria. The average percentage score for this section was 22.5%, and the range of scores was 0%∼72%. NCGC 2014[ recorded the highest percentage score at 72%, and OTP 2017[ recorded the lowest percentage score at 0%. One CPG scored above 60% and 10 CPGs scored below 30%. NGCG 2014[ provided an analysis of cost-effectiveness and an algorithm for holistic assessment and targeted treatment. Although MaHTAS 2013[ achieved a low score of 27% in this section, this CPG presented various clinical treatment auxiliary tools as a checklist of management, appropriate measurement of walking stick with illustrations, and quadriceps exercise pamphlet with illustrations. Almost all of the included CPGs in this study had low scores in this section, generally because there was insufficient mention of socioeconomic costs or supervisory and evaluation criteria.

Editorial independence

This section evaluates the perspective of the funding body and competing interests of guideline development group members. The average percentage score was 46.5%, and the range of scores was 15% to 81%. RACGP 2018[ recorded the highest percentage score at 81%, while WHCA 2020[ and OCACM 2019[ recorded the lowest at 15%. Five CPGs were above 60%, and three CPGs were below 30%. Low scores were obtained when there was no mention of financial support organizations or when there was no information on conflicts of interest. RACGP 2018[ reported a conflict of interest and attached the homepage uniform resource locator (www.racgp.org.au/support/policies/organisational) to verify each author's declaration of conflict of interest.

Overall evaluation

The 13 CPGs were analyzed using the AGREE II instrument. Five CPGs were evaluated as Class A, six as Class B, and two as Class C. For the recommendation levels, “recommended” was considered grade A, “recommended with modification” was grade B, and “not recommended” was grade C. The average percentage scores were as follows: clarity of presentation (72.6%), followed by scope and purpose (62.6%), rigor of development (54.2%), stakeholder involvement (50.5%), editorial independence (46.5%), and applicability (22.5%), which were the lowest. Overall, recommendations are clearly presented, and recommendations of traditional and conventional medicine are largely divided into diagnosis, pharmacological treatment, and non-pharmacological treatment. This information is presented in Tables 5 and 6. In order to be evaluated as an appropriate CPG by AGREE II standards, the additional supplementation of “applicability” would be required in the process of future revision of knee OA guidelines.

Table 5

Recommendations for knee osteoarthritis in traditional oriental medicine CPGs.

Title ID	Diagnosis	Herbal pharmacological treatment	Non-herbal pharmacological treatment	Management
KAMMS 2017	1. Clinical evaluation 2. Physical examination 3. Imaging 4. Laboratory investigation	1. Prescribed herbal medicine according to pattern identification 1)Liver Kidney Yin deficiency 2)Qi obstruction due to bloodstream malfunction 2. Manufactured hermbal medicine 1) Daeganghwaltang 2) Cheongyulsaseuptang 3) Sopunghwalhyeoltang 4) Daebangpungtang 5) Yeongseonjaetongeum 6) Ojeoksan 7) Samgieum 8) Banhagoongchultang 3. fumigation	1. Acupuncture 1) General acupuncture 2)Ear acupuncture 3)Electroacupuncture 4)Fire needling 2. Pharmacoacupuncture 3. Moxibustion 4. Acuptomy 5. Cupping 6. Conservative 1)TENS 2)ICT	1. Exercise 2. Weight loss
WHCA 2020	1. Clinical evaluation 2. Imaging 3. Physical examination 4. TCM classification	1. Prescribed herbal medicine according to pattern identification 1) Qi obstruction due to bloodstream malfunction (Hyulbuchukoetang) 2) Wind&Cold moisture (Chucheonchokbitang) 3) Liver Kidney Yin deficiency(Jwaguihwan, Wooguihwan) 4) Moisture&Heatness(Samyosan) 2. fumigation	1. Conservative 1) Exercise 2) Weight loss 3) TENS 2. Chuna 3. Acupuncture&Moxibustion 4. Western medicine 1) NSAID 5. Injection 1) Hyaluronic acid 2) Glucocorticoid 6. Surgical	1. Exercise 2. Weight loss
OCACM 2019	1. Clinical evaluation 2. Imaging 3. Physical examination 4. TCM classification 5. Laboratory investigation	1. Prescribed herbal medicine according to pattern identification 1)Wind&Cold moisture (Chokbitang) 2) Moisture&Heatness (Samyosan) 3) Qi obstruction due to bloodstream malfunction (Dohongsamultang) 4) Liver Kidney Yin deficiency (Dokhwalgisaengtang) 5)QiHyul weakness (Paljintang) 2. fumigation	1. Conservative 1) Exercise 2) Weight loss 2. Acupuncture&Moxibustion 3. Western medicine 1) NSAID 2) Opioid 3) Acetaminophen 4. Injection 1) Hyaluronic acid 2) Glucocorticoid 5. Surgical	1. Exercise 2. Weight loss

KAMMS = Korean Acupuncture and Moxibustion Medicine Society, OCACM = Orthopedic Section of China Association of Chinese Medicine, WHCA = Wangjing Hospital of China Academy of Traditional Chinese Medical Science.

Table 6

Recommendations for knee osteoarthritis in conventional CPGs.

Title ID	Diagnosis	Pharmacological treatment	Non-pharmacological treatment	Management
AAOS 2013	Not reported	1. NSAID 2.Tramadol 3.Opioid 4.Pain patches	1. Conservative 1)Exercise 2)Weight loss 3)Acupuncture 4)Electrotherapy 5)Manual therapy 6)Lateral wedge insole 7) Glucosamine & Chondroitin 2. Injection 1)Intra-articular corticosteroid 2) Hyaluronic acid 3) Platelet rich plasma 4)Needle lavage 3.Surgical 1) Arthroscopy lavage 2) Arthroscopic meniscectomy 3) Osteotomy 4) Free floating device	None
RACGP 2018	1. Background risk 2.Person's risk factor 3.Patient's symptoms 4.Physical examination 5.Imaging	1. Paracetamol 2. NSAID 3. Opioid 4. Duloxetine 5. Doxycycline 6. Bisphosphonate 7. Calcitonin 8. Stronitum ranelate 9. IL-1 inhibitor 10. FGF 11. Colchicine 12. Anti-NGF 13. Methotrexate 14. Herbal medicine	1. Conservative 1)Education 2)Behavioural 3)Exercise 4)Manual 5)Brace.Orthotic.Taping.Cane 6)Electrotherapy 7)Ultrasound 8)Acupuncture 2. Injection 1)Corticosteroid 2)PRP 3)Viscosupplementation 4)Stem cell 5)Dexrose prolotherapy 3. Surgical 1)Arthroscopic lavage 2)Meniscectomy 3)Cartilage repair	1.Education&Behavioural change 2. Physical activity 3. Exercise
MaHTAS 2013	1.Clinical feature 2.Background risk 3.Imaging 4.Laboratory	1. Paracetamol 2. Tramadol 3. NSAID 4.Glucosamin&Chondroitin 5. Diacerein	1. Conservative 1)Education 2)Life style change 3) Physiotherapy 4)Occupational therapy 5)Orthoses 2. Injection 1)Corticosteroid 2)Viscosupplementation 3. Alternative 1)Acupuncture 2)Herb 4.Surgical 1)Arthroscopic surgery 2)High tibial osteotomy 3)Joint replacement 4)Arthrodesis	1. Weight loss 2. Avoiding trauma
NCGC 2014	1.Risk factor 2.Patient's symptom 3.Imaging	1. Paracetamol 2. NSAID 3. Opioid 4.Tricyclics 5.SSRI 6.SNRI 7.Capsaicin 8.Rubefaciants	1. Conservative 1)Exercise&Manual 2)Weight loss 3)Electro 4)Nutraceutical 5)Acupuncture 6)Aid&Device 2. Injection 1)Corticosteroid 2)Hyaluronic acid 3. Surgical 1)Invasive treatment 2)Total joint replacement	1. Pain management 2. Follow-up
KNGF 2020	1.History taking 2.Red flags 3.Physical examination 4.Measurement instruments	Not reported	1. Conservative 1)Education 2)Exercise 3)Electro 3)CPM 4)Thermo 5)Laser 6)Ultrasound 7)Taping	Not reported
OTP 2017	Not reported	1. Paracetamol	1. Conservative 1)Yoga 2)TaiChi Qigong 3)Exercise program 4)Aerobic program	Not reported
OARSI 2019	1.Patient's symptom	1. NSAID + PPI 2. Cox-2 inhibitor	1. Conservative 1)Aquatic exercise 2)Gait aids 3)Self-management programs 2. Injection 1)Corticosteroid 2)Hyaluronic acid	Not reported
ACR 2020	Not reported	1.NSAID 2.Capsaicin 3.Acetaminophen 4.Duloxetine 5.Tramadol 6.Non-tramadol opioid 7.Colchicine 8.Fish oil 9.Bisphosphonate 10.Glucosamine 11.Chondroitin sulfate 12.Hydroxychloroquine 13.Methotrexate	1.Conservative 1)Exercise 2)Balance training 3)Weight loss 4)Self-efficacy and self-management programs 5)Tai-chi 6)Yoga 7)Cognitive behavioral therapy 8)Cane 9)Braces 10)Shoes 11)Insoles 12)Acupuncture 13)Paraffine 14)Thermal 15)Radiofrequency ablation 16)Massage 17)Manual therapy 18)Kinesio taping 19)TENS 20)Pulsed vibration therapy 2.Injection 1)Glucocorticoid 2)Hyaluronic acid 3)Botulinum 4)Prolo therapy 5)Platelet-rich plasma 6)Stem-cell	Not reported
CJJS 2018	Not reported	Not reported	1. Injection 1)Prolo therapy 2)Hyaluronic acid 3)Glucocorticoid	Not reported
NRCGD 2020	1. Physical examination 2. Imaging	1. NSAID	1. Conservative 1)Exercise 2)Electro 2. Injection 1)Sodium phosphate 3. Surgical	Not reported

AAOS = American Academy of Orthopedic Surgeons, ACR = American College of Rheumatology, CJJS = China Journal of Joint Surgery, KNGF = Royal Dutch Society for Physical Therapy, MaHTAS = Malaysian Health Technology Assessment Section, NCGC = National Clinical Guideline Centre, NRCGD = National Clinical Research Center for Geriatric Disorders, OARSI = Osteoarthritis Research Society International, OTP = Ottawa Panel, RACGP = Royal Australian College of General Practitioners.

Discussion

Considering the prevalence of knee OA, the number of studies selected was less than expected. This is due to the exclusion of CPGs that only cover the surgical treatment. Referring to the general information of each CPG summarized in Table 1, all 13 documents for treatment were presented, and information on diagnosis, management, and evaluation was mentioned in eight, five, and two CPGs, respectively. Two CPGs described the prevention and only one CPG described follow-up. Most of the selected CPGs contained substantial information on the treatment and diagnosis of knee OA.

Conventional treatments for knee OA can be classified into four broad categories: non-drug treatment, drug treatment, alternative treatment, and surgical treatment. Non-drug treatment includes strengthening muscle strength through exercise program therapy or aqua exercise performed in water, including physical therapy and ultrasound therapy.[ To achieve analgesia, treatments such as oral administration of acetaminophen, opioids, nonsteroidal anti-inflammatory drugs, and capsaicin or injection of hyaluronic acid or steroids into the joint cavity are commonly used as drug treatment in clinical situations.[

Traditional oriental medicine can be an alternative to surgical treatment.[ Various traditional medical treatment methods for improving pain and quality of life include acupuncture,[ moxibustion,[ herbal medicine,[ bee venom pharmacopuncture,[ and Tai chi gong.[ Traditional oriental medicine treatment for knee OA is gradually developing clinical evidence by actively conducting studies with high evidence standards, such as randomized controlled trials, systematic reviews, and meta-analysis.[ In this study, three were traditional CPGs: KAMMS 2017,[ a grade A CPG published in Korea; and WHCA 2020[ and OCACM 2019[ were rated as C. The CPGs for traditional oriental medicine published in China received low scores due to the lack of presentation of the retrieval strategy or statistical methodology for the evidence data.

In terms of contents of the traditional CPGs, the guidelines published in Korea and China proposed a “pattern identification” diagnostic tool, and KAMMS 2017[ covered various treatment tools as herbal medicine, general acupuncture, ear acupuncture, electro-acupuncture, fire needling, pharmacopuncture, moxibustion, acupotomy, and cupping for knee OA. They recommended a vast majority of treatments as “recommend with modification” due to the lack of data accumulation in clinical studies, such as the randomized control study of the traditional oriental medicine intervention for knee OA. Therefore, more systematic and evidence-based clinical studies should be conducted to derive strong recommendations for traditional oriental medicine treatment in the future.

Considering the concrete aspects of the overall CPGs, the “applicability” area showed a relatively low percentage point score compared to the other sections of the AGREEE II instrument. The evaluation of obstacles and facilitating factors in the CPG encompasses the contents of socioeconomic performance conditions, and the supervisory and evaluation criteria cover the contents of additional treatment and management when certain conditions of disease are satisfied. Since knee OA is not a disease that can be clearly quantified and monitored, such as high blood pressure or diabetes mellitus, the CPG of OA is bound to obtain a low percentage point score in the “applicability.” In order to improve the percentage point score in the “applicability” section, clinical algorithms or practical questionnaires, and exercise teaching pamphlets should be provided in future revisions.

Meanwhile, from the perspective of integrative medicine, CPGs have proposed various types of information such as diagnosis, management, treatment, and evaluation. NCGC 2014[ covered a holistic approach to knee OA, and this provided the key idea of knee OA integrative CP. At the start of treatment, practitioners communicate closely with the patient and make a clinical plan based on patient preferences. Then, diagnosis and evaluation processes are conducted based on clinical signs, laboratory investigations, and imaging.[ After this procedure, treatment is proposed and management is provided by practitioners. In conventional treatment approaches, pharmacological treatment is the basic treatment option. Injection and surgical treatment may be considered depending on the patient's condition. RAGCP 2018[ and NCGC 2014[ covered information regarding exercise as management. Exercise programs can be divided into self-exercise and medical-supervised exercises. RACGP 2018,[ NCGC 2014,[ and OARSI 2019[ also emphasized weight loss as a management strategy. In the future knee OA integrated CP, the above clinical decisions are anticipated to be guided organically.

Considering the CPGs selected for this study from the perspective of integrative medicine, some of the guidelines for conventional medicine suggested acupuncture and herbal medicines as part of complementary and alternative medicine. Acupuncture is a representative traditional medical treatment used in the included guidelines. AAOS 2013,[ RACGP 2018,[ and NCGC 2014[ declared that they did not recommend the use of acupuncture for knee OA. This is because the randomized group assignment procedure is inadequate or the control group's technique information (e.g., sham acupuncture) for treatment is insufficient. In contrast, according to MaHTAS 2013[ and ACR 2020,[ acupuncture can be applied to knee OA according to the condition of each patient. MaHTAS 2013[ cited the Cochrane systematic review study, which concluded that acupuncture treatment was more effective than sham acupuncture in terms of pain management and functional recovery. ACR 2020[ described as many clinical studies and meta-analyses have reported that acupuncture is effective for pain management, and since the risk of side effects is low, acupuncture can be recommended according to the patient's condition.

To the best of our knowledge, this study is the first to conduct a qualitative evaluation of traditional and conventional CPGs for knee OA. In addition, the revision direction of future CPGs to receive an excellent quality evaluation was suggested by this research. Finally, this study is significant as a cornerstone for constructing the CP of integrative medicine.

There are a few limitations to our study. First, as the AGREE II tool does not present the apparent criteria for comprehensive evaluation, the general evaluation of CPGs as Class A, B, and C was assessed using the criteria of a previous appraisal article. Second, the AGREE II instrument is an evaluation tool in which the subjective influence of each evaluator may be prominent. Because of these characteristics, the evaluation conducted by the AGREE II instrument inevitably has limitations in that a completely objective evaluation is impossible. Also, there is a limitation that there is an inherent bias in the evaluation because the evaluators consist only of specialists in traditional oriental medicine. Third, this study only covered the CPGs written in English, Korean, Chinese, and Japanese, and it is possible that good-quality CPGs written in other languages were not reflected in this research. Fourth, there is a limitation that translation bias exists in the process of all appraisers evaluating the CPGs written in English, Chinese, and Japanese because it is not written in evaluator's native language.

Conclusions

Thirteen CPGs were included in this research, and quality assessment for each CPG was conducted by four appraisers. The 13 CPGs were analyzed using the AGREE II instrument. Five CPGs were evaluated as Class A, six as Class B, and two as Class C. Most of included CPG got low score in the section of “applicability” because knee OA cannot be clearly quantified and monitored such as blood pressure or blood sugar. In the future knee OA CPGs revision process, “applicability” area should be supplemented by introducing a lot of checklists and auxiliary materials for the treatment process. In addition, more evidence should be accumulated to support the recommendation of traditional oriental medical treatments in the clinical field.

In addition, from the perspective of integrative medicine, communication with patients is important in decision-making during the treatment process. Exercise and weight loss are recommended for all CPGs. Acupuncture is recommended according to the patient's condition in several high-quality CPGs. Along with conventional pharmacological treatment, exercise, weight loss, and acupuncture can be combined in clinical situations, and which has implications for the development of future CP.

Author contributions

Conceptualization: Yeon Cheol Park.

Data curation: Jun Yeon Kim, Jung Hyun Kim, Bonhyuk Goo, Yong Hyeon Baek.

Methodology: Jun Yeon Kim, Yeon Cheol Park, Yong Hyeon Baek, Byung Kwan Seo.

Supervision: Yong Hyeon Baek.

Writing – original draft: Jun Yeon Kim, Yeon Cheol Park.