Critical appraisal and comparison of recommendations of clinical practice guidelines for treatment of schizophrenia in children and adolescents: a methodological survey protocol.

INTRODUCTION: The number of clinical practice guidelines (CPGs) have increased substantially mainly in the paediatric area of mental health. However, little is known about the quality or how recommendations for the treatment of disorders such as schizophrenia in children and adolescents have changed over time. The aim of this study will be to assess the quality of the development of CPGs for the treatment and management of schizophrenia in children and adolescents over time using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool and to compare the recommendations and interventions described in these documents. METHODS AND ANALYSIS: CPGs will be identified using a prospective protocol through a systematic search of multiple databases (Medline, Embase, Health Systems Evidence, Epistemonikos, Lilacs, etc) and guideline websites from 2004 to December 2020. The quality of the guidelines will be assessed by three reviewers, independently using the AGREE II. CPGs will be considered of high-quality if they scored ≥60% in four or more domains of the AGREE II instrument. Non-parametric tests will be used to test for the change of quality over time. We will summarise the different evidence grading systems and compare the recommendations. ETHICS AND DISSEMINATION: Ethical approval is not required since it is a literature-based study. Future results of the research can be submitted for publication in scientific journals of high impact, peer reviewed and also published in national and international conferences. The results derived from this study will contribute to the improvement of health institutions and policies, informing about existing recommendation guidelines and about deficiencies and qualities found in those. This study may also identify key areas for future research. This study may guide the search and choice for high quality CPGs by health policy makers and health professionals and subsidise future adaptations. PROTOCOL REGISTRATION NUMBER: CRD42020164899. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study will add to current knowledge by highlighting clinical practice guidelines (CPGs) of great quality that we might be able to use in current clinical practice.

We expect to identify the main characteristics and flaws of CPGs for schizophrenia in children and adolescents, which can help guide the development of recommendations guidelines of high methodological rigour for this disorder.

The critical appraisal of the CPGs for the treatment of schizophrenia in children and adolescents was never performed.

This study will be limited to subjective analysis of the Appraisal of Guidelines for Research and Evaluation II instrument, which can be a limiting factor.

The wide inclusion criteria, which can provide an ample overview of the CPGs developed for the disorder, might also make synthesis of the evidence more challenging.

Introduction

Schizophrenia spectrum disorders are a group of disorders in which individuals experience perceptive distortions of reality and impairments on thinking, behaviour and affect.1 Throughout the protocol, we have decided to focus on schizophrenia (International Classification of Diseases-10: F20), since the criteria used in the diagnosis in children and adolescents is agreed to be the same descripted in the Diagnostic and Statistical Manual of Mental Disorders (fifth edition) for this particular disorder.1–3 Usually, schizophrenia diagnosis occurs in very early adulthood, being rarer in children and adolescents.4 In this population, onset frequently develops between 13 and 17 years of age, being prevalent in 1–2 individuals in every 1000; onsets before 13 years of age have a prevalence of 1 in every 10 000.5

In children diagnosed with schizophrenia, the presence of pre-morbid motor, language and social disorders is common, as well as previous learning difficulties and diagnosis of mood or anxiety disorders.2 Among diagnosed adolescents and preadolescents, many have comorbidities such as post-traumatic stress disorder, attention deficit/hyperactivity disorder, and history of disturbing behaviours and conduct disorders.6

Schizophrenia in such cases is described as a psychotic disorder in which life expectancy is reduced and impairments on the social, psychological, educational and occupational spheres are frequently severe and debilitating.7 8 The diagnosis process of schizophrenia in children and adolescents must involve a very detailed physical and psychological examination in order to exclude any possibility of organic causes for the psychosis or any kind of misdiagnosis.2

Because of its social impairments and stereotypical behaviours, a misdiagnosis of autism spectrum disorder is possible in children, being the presence of hallucinations and delusions what distinguish those two disorders.3 In teenagers, the overlapping of affective symptoms (mania and depression) and psychotic symptoms (delusions, hallucinations, incoherent or non-sense speech, inappropriate behaviour) can cause difficulties in the diagnosis of schizophrenia, generally misleading to an affective disorder diagnosis.9

Another obstacle in the diagnosis of schizophrenia in children and adolescents is that, although psychotic symptoms are found in children with no psychopathology in a relatively high prevalence,10 schizophrenia in this population is rare and have a lack of epidemiological data about diagnoses based on standardised clinical assessments.2

Psychological interventions are recommended as a first line of treatment of schizophrenia in children and adolescents, with better outcomes when applied to individuals on their first psychotic symptoms, before the onset of the disorder.8 Although antipsychotic medication is the main form of treatment of schizophrenia, evidence of their efficacy in the treatment of this specific population is still limited.7 8 11 Clozapine is indicated as being the most effective in comparison to other antipsychotics, even though second-generation antipsychotics have shown higher incidence of side-effects.2 8 11

To help in the interventions on schizophrenia young patients, guidelines have been created in the past years based on developments in the management of schizophrenia in children and adolescents.12 13 Clinical practice guidelines (CPGs) have a significant importance in the transposition of research evidence into clinical practice, formulating health questions that are fundamental to ensure recommendations are applicable.14 For this to be possible, the CPG must be developed according to the best available evidence.15

CPGs for schizophrenia in children and adolescents normally are adaptations of already existing guidelines for adult-onset schizophrenia, due to the lacking of specific evidence about this age range.12 16 17 Implementing a CPG may take time depending on how much change is needed on the health service, becoming easier to put them into practice when they are aligned with the local priorities.17

To assess the methodological rigour and transparency in a CPG, the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument was developed by an international group in 2003, and have been updated to the second version in 2009. This instrument has been widely used and offers a comprehensive, rapid and consistent assessment of CPGs.18

During a preliminary search, no systematic assessment that had carried out a critical appraisal on the development of CPGs for the treatment of schizophrenia in children and adolescents was found. In this study, the aim is to assess whether CPGs for the treatment and management of schizophrenia in children and adolescents have been developed with sufficient transparency and methodological quality for its implementation over time. It also aims to compare the recommendations and interventions for schizophrenia in children and adolescents described in those documents, in order to subsidise adaptations from future panellists.

Methods and analysis

Study design

The present systematic assessment of CPGs for schizophrenia in children and adolescents will be conducted to compare the recommendations of the interventions and the methodological quality in their development, available in these documents.

Protocol and registration

This study will be reported according to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols.19

Patient and public involvement

Patients did not participate on the study design. However, by the end of the study, we aim to contact health policy makers to inform about the results and to ask to collaborate with us in the dissemination plan.

Eligibility criteria

Inclusion criteria

Overall or specific guidelines for clinical practice including psychosocial, psychological and pharmacological interventions for the treatment of children and adolescents (age <18 years) with schizophrenia will be included. Documents published from 2004 (5 years before the latest version of the AGREE II instrument) to December 2020 will be considered, with no language restrictions.

Exclusion criteria

Guidelines for schizophrenia caused by misuse of substances and guidelines for schizophrenia associated with other mental disorders will be excluded. If there is another more up-to-date version of the guideline; the available version is incomplete or contains only a summary of the information; the document is the translation of a guideline published in another language; and if there is a consensus document, evidence summary or algorithm, it will be excluded, since they are not equivalent to guidelines.

Measured outcomes

The methodological quality of the CPGs for interventions for schizophrenia in children and adolescents will be evaluated; the scores of each domain of the AGREE II instrument,18 associated with the methodological quality of the guidelines will be identified; and the recommendations provided by the guidelines will be described and compared.

Selection of studies

Data sources

The following electronic databases from 2004 to December 2020 will be searched: EMBASE (Excerpta Medical Database, via Ovid); MEDLINE (via Ovid); PsycINFO (via Ovid); Trip Database; Epistemonikos; Lilacs; WHO; Health Systems Evidence. Specific databases for clinical guidelines will be also searched, for example: ECRI Institute (www.guidelines.ecri.org), National Institute for Health and Care Excellence (www.nice.org.uk), Canadian Agency for Drugs and Technologies in Health (www.cadth.ca), Canadian Medical Association (www.cma.ca), Canadian CPG Infobase: CPGs Database (www.cma.ca/En/ Pages/clinical-practice-guidelines.aspx), Scottish Intercollegiate Guidelines Network (www.sign.ac.uk), Australian CPGs (http://www.clinicalguidelines.gov.au/) and Guidelines International Network (http://www.g-i-n.net/).

Other data sources features

Reviewers will check the reference list of eligible studies, review studies and secondary studies in order to identify other possible guidelines. Authors will be contacted in case of guidelines published only in summary or where important information is missing.

Search strategies

The key words will be used according to the terms of the Medical Subject Headings to identify relevant studies. The search terms that will be used for Embase (via Ovid), Medline (via Ovid and PubMed) and PsycInfo (via Ovid) are provided as online supplemental material (see online supplemental files 1 and 2). The search strategy will be adapted for each database consulted.

Determination of eligibility

References will be managed in EndNote (version X8.2 New York City: Thomson Reuters, 2018), and duplicates will be removed. Titles and abstracts will be assessed by groups of three reviewers, independently, to check if they meet the eligibility criteria. A full read of the CPG will be conducted by the same reviewers, also independently, in order to confirm the eligibility of the guidelines. Discrepancies will be solved by consensus and a fourth reviewer will be able to assist in the final decision if necessary. The most up-to-date guideline will be used if there is a case of duplicate publications. All documents related to the guidelines (cited as supplemental documents, summaries of recommendations and others) will be searched manually by one or two reviewers.

Data extraction

The information will be organised in a Microsoft Excel worksheet; the same groups of three reviewers, independently, will extract the data. Discrepancies will be resolved through discussion and consensus. If this process is not effective, a fourth reviewer will be responsible for the tiebreaker. Previously, reviewers will be calibrated by extracting at least three documents of different quality levels and reaching consensus. Results will be discussed with a previously trained fourth reviewer. This procedure will be repeated until the reviewers can extract the data.

For this study, the following data will be considered: number of authors, year of publication, update time, organisations (government, medical society, university or other), type of guideline (formulated, adapted, updated or revised), country of development, type (diagnosis, prevention, pharmacological and non-pharmacological treatment, and/or other), treatments described, target population, design of studies included (systematic review, consensus, overview of systematic reviews and/or other), methods of recommendation formulation (consensus, not mentioned, others) and methods of classifying the quality of evidence (Grades of Recommendation, Assessment, Development, and Evaluation (GRADE), Oxford, not mentioned or other).

Quality assessment of CPGs

The AGREE II will be used to evaluate the quality of the guidelines. The tool has been translated and validated for the Portuguese language (Brazil), and this version will be used in this study. It includes six domains: (1) scope and purpose; (2) stakeholder involvement; (3) rigour of development; (4) clarity of presentation; (5) applicability; and (6) editorial independence, containing 23 items in total. Scores are in Likert scale of 1 (totally disagree) to 7 (totally agree) for each item.18 20

A group of three reviewers will conduct the quality assessment of the guidelines and differences between two or more scores for each item will be considered as discrepant. The final score will be decided by consensus. In case of no consensus, a fourth reviewer will help in the final decision. The quality of the CPG will be calculated for each domain as instructed by the AGREE II user manual. Since the six domains are independent, the scores should therefore be calculated as the sum of the individual items in each domain. The total obtained will be presented as a relation percentage to the maximum possible score for each domain. A descriptive statistical analysis will be conducted. Agreement between reviewers will be assessed using random single-unit bidirectional intra-class correlation coefficients (ICC).21 22 Cohen’s weighted kappa will be calculated to compare with the ICC using squared weights, since we have an ordinal scale.1–7 22 As performed by Hayawi, Graham, Tugwell and Abdelrazeq,23 based on Cicchetti,21 the degree of agreement between reviewers will be categorised as: ICC <0.40 poor; 0.40–0.59 moderate; 0.60–0.74 good; 0.75–1.00 excellent. CPGs will be considered of high quality if they score ≥60% in four or more domains including the domain for rigour of development. The evaluation will be conducted using the ‘My AGREE PLUS’ platform.18 Previously, a training will be done to use the AGREE II instrument.

To evaluate if there was any change and improvement in the quality of guidelines over time, after the latest version of the AGREE instrument, the Wilcoxon rank-sum test (Mann-Whitney test), will be used to test for statistical significant differences in domain scores between CPGs published before and in/after 2009 (year of the AGREE II update).

Description and comparison of the recommendations of the interventions

The assessment will describe and compare the psychological, psychosocial and pharmacological recommendations of intervention. We anticipate important influence of culture/country on the recommendation of psychosocial and psychological interventions. If appropriate we will analyse such difference.

In this study, we will compare the recommendations found in high quality CPGs, this is, CPGs that get ≥60% on domains associated with the reliability (3 and 6) and applicability (5) available in the AGREE II tool. Recommendations on treatment and classification of the level of evidence of the included CPG, will be extracted independently by two researchers. Disagreements between researchers will be resolved by consensus; in the absence of consensus, a third investigator will help in the decision. Whenever available, the GRADE approach will be used for the extraction and synthesis of recommendations of the selected CPG. If GRADE is unavailable, the CPG will be classified based on the highest score in domain 3.

The recommendations will be grouped into the following topics: pharmacological, psychosocial and psychological, according to their similarities through an interactive process between researchers. CPGs that share similar recommendations will be noted. We will evaluate if recommendations from different CPGs address the same topics and will compare them to identify differences. When two or more CPGs show conflicting recommendations, this will be defined as a disagreement. Those and the level of evidence supporting them will be highlighted.

Data synthesis

Descriptive tables will be made to show the results. For all AGREE II domains, descriptive statistics will be calculated as mean (SD) and median (IQR). When needed, graphs will be plotted. The level of significance will be 5%. Statistical analyses will be performed using Microsoft Excel and STATA software (V.14.2), except inter-rater reliability (ICC and weighted kappa), that will be performed using R statistical software.

Ethics and dissemination

Since it is a literature-based study, ethical approval is not required. The results will be shared through publication in scientific journals of high impact, peer reviewed and also published in national and international conferences.

Discussion

Successful implementation of recommendations should be related to the use of appropriate methodologies and rigorous strategies in the guideline development process. Thus, we will work towards the identification of high-quality CPGs that describe interventions for schizophrenia in children and adolescents or possible deficiencies observed in these documents. With this study, beyond the quality assessment of the CPGs, we hope to create a subsidy to the process of adaptation for future panellists, providing organised information to the development of high-quality CPGs.

The description of available recommendations on interventions and its supporting evidences can contribute to the choice of treatment for schizophrenia in children and adolescents. Aiming to contribute to the improvement of health institutions and policies, we expect to inform about existing recommendation guidelines, about deficiencies found in those, and make recommendations for future research.

Explicit eligibility criteria, broad and comprehensive database research, and structured evaluation for study selection comprise the method of this methodological survey. This study, however, will be limited to subjective analysis of the AGREE II instrument, which can be a limiting factor.