Accuracy of the 2008 Simplified Criteria for the Diagnosis of Autoimmune Hepatitis in Children.

PURPOSE: Classical criteria for diagnosis of autoimmune hepatitis (AIH) are intended as research tool and are difficult to apply at patient's bedside. We aimed to study the accuracy of simplified criteria and the concordance with the expert diagnosis based on the original criteria.
METHODS: A cohort of children under study for liver disorder was selected through consecutive sampling to obtain the prevalence of AIH within the group of differential diagnoses. AIH was defined, based on classical criteria, through committee review of medical reports. Validity indicators of the simplified criteria were obtained in an intention to diagnose approach. Optimal cut-off and the area under the receiver operating characteristic (ROC) curve were calculated.
RESULTS: Out of 212 cases reviewed, 47.2% were AIH. For the optimal cut-off (6 points), the simplified criteria showed a sensitivity of 72.0% and a specificity of 96.4%, with a 94.7% positive and a 79.4% negative predictive value. The area under the ROC curve was 94.3%. There was a good agreement in the pre-treatment concordance between the classical and the simplified criteria (kappa index, 0.775).
CONCLUSION: Simplified criteria provide a moderate sensitivity for the diagnosis of AIH, but may help in indicating treatment in cases under suspicion with 6 or more points.

INTRODUCTION

Autoimmune hepatitis (AIH) is an inflammatory liver disorder with a wide clinical spectrum, that ranges from isolated acute or chronic hypertransaminasemia to acute liver failure (ALF) [12]. It is characterized by several analytical features, especially the presence of autoantibodies, high levels of immunoglobulin G (IgG) and, histologically, by liver lymphocytic or lymphoplasmacytic chronic infiltration that is typically displayed as interface hepatitis [3]. AIH tends to naturally evolve towards liver cirrhosis but when treated, it responds properly in the majority of patients [4].

With reference to diagnosis, the International Autoimmune Hepatitis Group (IAIHG) proposed, in 1993, a classification criteria to facilitate comparison of diagnostic definitions in clinical studies [5]. Criteria were revised in 1999 and, as a result, their specificity was improved up to 90% [6]. IAIHG revised diagnostic criteria have been proven suitable even in an exclusively pediatric population [7]. However, their practical application remains challenging for clinical use due to its complexity, including 13 categories, some of them impractical in children. To overcome these difficulties, the IAIHG proposed a simplified scoring system in 2008 (Table 1) [89]. Some validation studies have been carried out in adults in relation to the simplified criteria, demonstrating its usefulness [10111213141516]. However, only two studies have been published assessing its validity in children, with discrepant conclusions regarding its sensitivity [1718]. Furthermore, their study designs did not allow for the calculation of accurate predictive values. The aims of this study are to determine the validity indexes of simplified IAIHG criteria, to estimate the frequency of AIH in children consecutively evaluated for liver disease at our centers and, according to it, to assess their diagnostic capacity in this population, which could set a basis for further clinical decision analysis.

Table 1

Simplified Scoring System for Autoimmune Hepatitis Diagnosis by the International Autoimmune Hepatitis Group

Adapted from Hennes et al. (Hepatology 2008;48:169-76) with permission [8].

ANA: anti-nuclear antibody, SMA: anti-smooth muscle antibody, LKM: anti-liver/kidney microsomal antibody, SLA: anti-soluble liver antibodies; IgG: immunoglobulin G, AIH: autoimmune hepatitis.

*Maximum number of points for all autoantibodies is 2. Total points are 8. †Compatible features are a picture of chronic hepatitis with lymphocytic infiltration without all the features considered typical. ‡Interface hepatitis, lymphocytic/lymphoplasmacytic infiltrates in portal tracts and extending into the lobule, emperipolesis (active penetration by one cell into and through a large cell), and hepatic rosette formation.

MATERIALS AND METHODS

Design

A cross-sectional diagnostic accuracy study was performed. Data were collected mainly in a retrospective way since January 2005 but a prospective wing was included from January 2016 to January 2017 in order to increase the sample size, verify previously collected information, correct missing data and reclassify patients' first diagnosis if needed.

Study population

Patients were all under 18 years old, from Sant Joan de Déu Hospital and Vall d'Hebron Hospital that share a common Pediatric Hepatology unit. They were intended to be representative of a population in clinical or analytical situation potentially attributable to AIH. Inclusion criteria were: 1) patients with liver biopsy performed because of signs of acute or chronic hepatocellular damage, regardless of the coexistence of cholestasis and the technique for sample collection (percutaneous, laparoscopic or transjugular); 2) ALF defined according to the consensus reached by the Pediatric Acute Liver Failure Study Group [19]. Patients with the following conditions were excluded: 1) children who underwent liver or hepatocyte transplantation; 2) previous diagnosis of any congenital or acquired liver disease; 3) previous diagnosis of any systemic or metabolic disorder with liver involvement; 4) hepatic space-occupying lesion; 5) liver biopsy indicated for sample collection in the setting of fever of undetermined origin investigation; 6) infants less than six months old (to rule out alloimmune hepatitis and other causes of young infant ALF). Participants formed a consecutive series and were identified through International Classification of Diseases, 9th revision (ICD-9) codes, cross referencing with Pediatric Hepatology database and radiology reports.

The only act performed in study participants was retrieving their medical information from hospital reports. Nonetheless, informed consent was obtained from all individual participants included in the study. The protocol was approved by the ethics committee of Sant Joan de Déu Hospital, Barcelona (number: PIC-99-16).

Scoring systems

We considered classical revised 1999 criteria as a not completely accurate gold standard and designed the study to minimize misestimation of validity indexes for these reason [20]. Post-treatment scores were taken into account if possible. Therefore, we established a threshold of ≥12 to be a positive case, regardless its classification in probable (12 to 17) or definite AIH (>17). In a pre-treatment basis, a 10 to 15 score meant probable AIH, and >15 meant definite AIH. Labelling between these two categories is based on variations in clinical manifestations and does not reflect differences in the reliability of the diagnosis [21]. Auto-antibodies titers under 1:40 were given 1 point [36] and, unless specified in another way in the medical reports, alcohol intake was judged not significant. To ensure diagnostic robustness and reduce false positives, two conditions were considered necessary: 1) Treatment response after three months of remission induction, defined as symptoms relief and liver transaminases decrease (complete or with posterior relapse) and 2) liver histology describing, at least, features compatibles with AIH (chronic hepatitis with lymphocytic infiltration) [22].

During the prospective wing, in order to reduce false negatives, those cases initially labelled as non AIH by classical criteria were reviewed. They were reclassified to AIH if there was an explicit AIH diagnosis in medical reports and the two necessary conditions described above were simultaneously fulfilled.

Suggested cut-offs for 2008 simplified criteria positivity are 6 points (probable diagnosis) and 7 points (definite diagnosis) [8]. All the possible values (0–8) were explored in the analysis. Data were recorded around the biopsy date.

Implementation of the reference standard for diagnosis was done before applying the simplified criteria so that the result of the index test was not available to the assessors.

Analysis

Missing data correction and diagnosis updating were carried out prospectively when possible. The planning consisted of excluding those patients from analyses if missing data recovery were not possible and had impact on ensure the reference standard result. On the other hand, the study strategy included using of a 3×2 table with an intention to diagnose approach to handle simplified criteria indeterminate results [23]. Sensitivity, specificity and predictive values were calculated with their 95% confidence interval (CI). Other validity indexes like likelihood ratios and the Good-Turing weight of evidence were also obtained [24]. A receiver operating characteristic (ROC) curve was plotted, and its area under the curve calculated. An optimal cut-off value was estimated using a costs ratio of 1 to penalize false negatives and false positives equally [25].

Binary results were reported as percentages with 95% CI by Wilson method and continuous variables as median and interquartile range. The Mann-Whitney U-test was used to evaluate differences in continuous variables between groups and χ2 test for dichotomous variables. Fisher's exact test was used when appropriate. A p-value of <0.05 was considered statistically significant. Statistical analyses were performed on IBM SPSS Statistics ver. 21.0 (IBM Co., Armonk, NY, USA).

A pilot study was conducted to initially estimate the prevalence and expected sensitivity and sensitivity for the cut off in 6 points. Exactly 49 out of 102 patients were classified as having AIH. Buderer formulae with 10% absolute accuracy and 0.05 α error [26] gave a recommended minimum sample size of 154.

Agreement between classifications with 1999 and 2008 criteria (definite AIH, probable AIH or no AIH) was studied by the kappa statistic with quadratic weighting.

RESULTS

An initial 425 patients with a variety of liver diseases were identified. Out of them, 207 met exclusion criteria. We finally intended to conduct the analysis in 218 patients, 6 of whom were also excluded because data recovery was not possible due to loss to follow-up. Seventeen patients were included in the prospective wing. The prevalence of AIH in the remaining 212 subjects was 47.2% (95% CI, 40.6% to 53.9%).

Clinical and demographic characteristics according to final diagnosis are depicted in Table 2. Proportion between type 1 and 2 AIH in our sample was nearly 5:1. None of the patients had an explicit diagnosis of overlap syndrome with primary sclerosing cholangitis (PSC). Nevertheless, not all patients enrolled at the first years of the inclusion period had a cholangiographic evaluation performed.

Table 2

Patients'Clinical, Analytical and Demographic Characteristics According to Final Diagnosis

Data between parentheses represent interquartile range for quantitative variables and 95% confidence interval for proportions.

AIH: autoimmune hepatitis, IgG: immunoglobulin G, AST: aspartate aminotransferase, ALT: alanine aminotransferase, AP: alkaline phosphatase, GGT: gamma glutamyl transferase, ANA: antinuclear antibodies. Anti-SM: anti-smooth muscle antibodies, Anti-LKM1: anti-liver/kidney microsomal antibodies.

*Or at first biopsy date if diagnosis initially unclear. †Interquartile range not informative due to few patients.

Revised 1999 criteria

Following the 1999 revised criteria, 85 children were correctly diagnosed with AIH after the first liver biopsy (sensitivity, 85.0%; 95% CI, 76.7% to 90.7%) and 15 more patients were subsequently reclassified as having the same condition. Ten out of these 15 patients reached classical criteria during the follow-up period, raising its sensitivity up to 95.0%, 95% CI: 88.8% to 97.8%. The remaining 5 patients were diagnosed according to histology features, clinical presentation and complete treatment response despite not meeting classical criteria. Interestingly, they all scored 6 points in the simplified criteria. No patients were misdiagnosed as AIH by 1999 criteria (specificity, 100%; 95% CI, 96.7% to 100%).

The revised 1999 classical criteria categorized 73% of the AIH as probable and 22% as definite.

Simplified criteria

Simplified criteria's sensitivity with a 6 points cut-off at first diagnostic attempt was 72.0% (95% CI, 62.5% to 79.9%). It went down to 45.0% (95% CI, 35.6% to 54.8%) when using a 7 points cut-off. In terms of specificity it showed a better performance, with 96.4% (95% CI, 91.2% to 98.6%) and 100% (95% CI, 96.7% to 100%) with 6 and 7 points cut-off respectively. Other validity indicators are listed in Table 3. Analysis in our sample also demonstrated that 6 points are the optimal threshold to consider that a case meets simplified criteria. According to sensitivity and specificity, the indication area came from 2% to 86%, which is the range between two prevalences wherein the gain in certainty is higher than the loss of certainty [27]. Specifically, validity of 2008 criteria reaches its better overall performance in populations with AIH prevalence around 29.3%, which makes both predictive values equal at around 89.3%. In the context of the prevalence obtained in our scenario, a positive result when applying the 2008 simplified criteria is especially valuable, given a 94.7% positive predictive value (95% CI, 87.2% to 97.9%). Additionally, simplified criteria behaved as a good discrimination tool, with an area under ROC curve of 0.943 (95% CI, 0.903 to 0.970) considering it as a quantitative test (Fig. 1). The area under ROC curve of the diagnostic model at the established cut-off was 0.842 (95% CI, 0.786 to 0.888).

Table 3

Autoimmune Hepatitis 2008 Simplified Criteria Validity Indicators in Children

Values in parentheses represent 95% confidence interval.

*For sample prevalence. †Calculated with the midpoint of the specificity confidence interval.

Fig. 1

Simplified criteria receiver operating characteristic curve. CI: confidence interval.

The simplified criteria defined 27% as probable AIH, 45% as definite AIH, and 28% were not identified as AIH. Furthermore, the 2008 criteria correctly classified 108 out of 112 non-AIH cases (96.4%) but 4 children (3.6%) were misdiagnosed as probable AIH. Concerning the misclassifications by the simplified criteria, it obtained 28 false negatives. Proportion of normal levels of IgG was significantly higher in these children in comparison to those with AIH correctly diagnosed by the 2008 criteria (78.6% vs. 15.3%, p<0.001). They also showed lower autoantibodies levels, with a proportion of children with>1/80 titers of 50.0%, vs. 84.7% in true positive patients (p<0.001). Two cases out of the 2008 criteria false negatives had normal autoantibody titers, and only one seronegative patient was found in the true positive group. Positive viral hepatitis markers were found in 28.6% misclassified patients (single infections by Epstein-Barr virus in 4 cases, cytomegalovirus in 3 and hepatitis A virus in 1), who subsequently evolved towards AIH, significantly more than in correctly diagnosed AIH patients (8.3%, p=0.009). No differences were found regarding proportion of female patients or liver histology features.

All 4 false positive patients were females, with slightly elevated IgG levels and >1/80 antinuclear antibodies titers that were classified as probable AIH by the simplified criteria. Two of them were finally diagnosed with PSC and the other 2 were classified as transient acute cryptogenic hepatitis. All of them were studied with magnetic resonance cholangiography and had AIH correctly excluded by 1999 classical revised criteria.

Agreement between scores

The kappa statistic showed a good agreement between diagnosis made by the 2008 simplified criteria and the 1999 revised classical criteria in a pre-treatment basis (0.775; 95% CI, 0.704 to 0.846) and including the treatment response item in the original diagnostic system (0.750; 95% CI, 0.676 to 0.824).

DISCUSSION

This is a diagnostic accuracy study conducted on the 2008 simplified criteria for pediatric AIH. It has been specifically designed to obtain its validity indicators in the clinical setting where it may be more useful: as a diagnostic test for everyday clinical practice without considering response to treatment. Thus, simplified criteria have been applied in a cross-sectional way in a population of children with a still undiagnosed liver disorder. To do so, we recruited patients according to the need of performing a liver biopsy, as described previously. The principle underlying this decision was the current recommendation, by the American Association for the Study of Liver Diseases, of conducting histological liver studies before applying any treatment provided that there are no contraindications [28]. In this respect, all AIH cases presenting with ALF could be biopsied after transfusion of fresh-frozen plasma or by transjugular access. We took this into consideration although it has been pointed out that, in some cases, biopsy samples might not need to be collected [29].

Several studies have been carried out including both adult and pediatric patients, evaluating simplified criteria compared to codified descriptive criteria developed by the IAIHG in 1993 [68101530]. Some other pediatric studies used 1999 original revised criteria as a reference standard [1718], arguing its established use in clinical and research scenarios and the demonstrated validity in children [7]. In spite of this, there are some concerns about classical criteria working as a truly gold standard. Assuming 100% accuracy of patients' diagnoses of AIH when using 1999 revised criteria may be erroneous, indeed [20]. Actually, five of our AIH patients were diagnosed according to histology features, clinical presentation and complete treatment response despite not meeting classical criteria. All were males without personal or familial autoimmune diseases history, without hyper-IgG levels and with low positive autoantibody titres. In that sense, childhood AIH diagnostic criteria proposed by Mieli-Vergani et al. [31] might have been helpful but have not been specifically validated and they will also hardly behave as a gold standard.

IgG and serum globulin can be used interchangeably in applying simplified criteria for AIH in children [18]. We chose to use IgG alone in our analysis because of the many missing globulin levels in our patients. The rest of the criteria items could be totally fulfilled without missing data.

We think that our study design has managed to build representative samples of AIH and non-AIH cases to assess the goodness of simplified criteria. Three diagnoses configured more than a half of the alternative cases: acute cryptogenic hepatitis, Wilson's disease and viral hepatitis. This differs from the non-AIH samples in the two previous studies about the utility of the simplified criteria in children. Hiejima et al. [17] build a non-cases group where chronic hepatitis C was the main diagnosis, and Mileti et al. [18] studied 2008 criteria mainly in children with metabolic liver disease, non-alcoholic steatohepatitis and PSC. This may have had an impact on simplified criteria performance, as children with PSC frequently have overlap with AIH, termed autoimmune sclerosing cholangitis (AISC), but the AIH component is not always recognized. Actually, it has been found that 50% of patients with AIH characteristics meeting classical and simplified criteria have bile duct disease at presentation [32]. Current guidelines include the need for a cholangiogram in children with AIH to rule out these two entities [283133]. However, as a limitation, AIH patients in our sample had cholangiogram done only in up to 48% cases, especially those diagnosed in the second half of the inclusion period. The five PSC patients from Hiejima et al. [17] and 3/8 from Mileti et al. [18] were classified as AIH. Two out of 5 from our study were so misdiagnosed by the 2008 simplified criteria. Moreover, even though it does not necessarily imply another therapeutic option (beyond adding ursodeoxycholic acid), lack of cholangiogram in some AIH cases could imply an under-diagnose rate of AISC. Studies evaluating the IAIHG scoring systems in adults with PSC and AISC show that 1999 classical criteria and 2008 simplified criteria have similar specificity [343536]. Consequently, our results add evidence that performing image studies of the bile tract is advisable in the diagnostic approach of AIH, regardless the result of the scoring systems.

Performance parameters of the scoring systems in acute onset AIH has been another subject under discussion. Classical and simplified criteria's utility in adult and pediatric patients have proven to be poor in ALF [1517]. However, the 4 cases in our cohort matching the definition of ALF were correctly classified by 1999 and 2008 scoring systems. All of them had mild signs of encephalopathy that recovered after treatment. It is possible that clinical severity of ALF could have an impact in AIH diagnostic criteria reliability and, as a result, previous results could not be strictly inferred to our sample.

Even in this non-selected population, 2008 criteria have shown an overall appropriate performance. The specificity of 96.4% is higher than that obtained by Hiejima et al. [17] (86%) and Mileti et al. [18] (95%), in part due to an optimal classification of some PSC patients. Notwithstanding the better accuracy than in Hiejima et al. [17] for this diagnostic group, validity of simplified criteria was neither appropriate. As interpreted by Mileti et al. [18], our results also pointed out that diagnosis and management should not only be guided by the scoring system, but by the biopsy result if PSC is suspected. In terms of sensitivity, we obtained an intermediate result (72.0%) between that reported by Hiejima et al. [17] (55.0%) and Mileti et al. [18] (91.9%). Apart from that, in the real clinical scenario that we defined, a score over 6 points using the 2008 criteria renders a probability of 94.7% of a true AIH, indicating the good reliability of the simplified scoring system. AIH rate may be high in countries with limited frequency of viral hepatitis or non-alcoholic fatty liver disease. In a lower prevalence setting, positive predictive value would be inferior, so we suggest calculating it for each population.

The reasons for misclassifications according to simplified criteria results were diverse, and some of them have already been considered for pediatric AIH diagnosis. In fact, the need for excluding PSC, AISC and Wilson's disease, and the fact that children often exhibit lower autoantibodies titers, have been included in the pediatric-specific criteria proposed by Mieli-Vergani et al. in 2009 [31]. They have not been thought as a scoring system and, to our knowledge, have not been validated neither, which may be a subject for future studies.

As a conclusion, simplified criteria have shown a moderate sensitivity and a high specificity, similar to previous reports both in adults and children. Around half the children that start with liver disease and who are studied with a liver biopsy, are finally diagnosed of AIH. In our setting, this has meant that simplified criteria are a good tool to predict AIH diagnosis and may play a role in indicating treatment even in ALF onset. Since there are only four variables to consider in the calculation, it is easier to apply compared to 1999 classical criteria, though its accuracy is lower. Nevertheless, cholangiographic studies should be performed to exclude PSC and AISC.