Efficacy of metabolic and psychological screening for mood disorders among children with type 1 diabetes.

OBJECTIVE: To compare the diagnostic accuracy and time expenditure of screening models based on glycated hemoglobin (HbA(1c)) level and psychometric measures for mood disorder (MD) among children with type 1 diabetes. RESEARCH DESIGN AND METHODS: With semistructured clinical interviews (Schedule for Affective Disorders and Schizophrenia for Children-Present and Lifetime version, 120 min/patient) as a reference for diagnosing MD, including major depressive disorder (MDD), we tested 163 subjects, aged 8 to 18 years, with type 1 diabetes. We evaluated four screening approaches: 1) Children's Depression Inventory (CDI) at 30 min/patient, 2) HbA(1c) level, 3) HbA(1c) level plus CDI, and 4) HbA(1c) level plus Children's Depression Rating Scale (CDRS) at 40 min/patient. These tests were conducted with all participants, and the total time expenditure for all four approaches was calculated as the total time needed to implement successfully the screening for MD or MDD in the center.
RESULTS: HbA(1c) performed on par with individual psychometric tests in diagnosing MD or MDD. The HbA(1c) plus CDRS model was the best screening procedure for both MD and MDD, with diagnostic thresholds for HbA(1c) established at 8.7% and 9.0%, respectively. Cutoff points for CDRS assessed after filtering by HbA(1c) were 26 (MD) and 30 (MDD) points. Center-wide application of this procedure would result in an 83% reduction of the examination time necessary for the psychiatrist for MD screening and a 91% reduction for MDD screening, as compared with standard screening with CDI.
CONCLUSIONS: Use of HbA(1c) level followed by CDRS is a time-efficient procedure to screen for MD in children with type 1 diabetes.

Diabetes is a risk factor for comorbid psychiatric disorders among children. During a 10-year observation period, nearly half of prospectively evaluated children with type 1 diabetes met diagnostic criteria for psychiatric comorbidity, with major depressive disorder (MDD) showing the highest prevalence (27%) (1). Psychiatric comorbidity leads to nonadherence, lower quality of life, poor metabolic control, and resultant diabetes complications (2–5). Mood disorders (MDs) are of particular importance because of the increased intensity of depressive symptoms. Their presence, as confirmed by a dedicated screening tool, was associated with a 2.5-fold increased risk of hospitalization for diabetic ketoacidosis in youth with type 1 diabetes (6). Identification of patients with MDs should therefore be considered an important aspect of pediatric diabetes care. American Diabetes Association guidelines for children and adolescents with type 1 diabetes include routine screening for psychiatric disorders among youth who do not have achievement of treatment goals or who experience recurrent diabetic ketoacidosis. Routine annual screening for depression should be performed for all patients >10 years old (7). The diabetic care team, which typically evaluates a patient several times per year and has access to medical records and family information, is well-positioned to observe symptoms of psychiatric disorders (8); however, recognition of depressive symptoms requires skill, careful attention, and sufficient time for individualized assessment.

Whereas the U.S. Preventive Services Task Force recommends that primary care clinicians screen adolescents (9), the International Society for Pediatric and Adolescent Diabetes states that this screening should be performed by mental health professionals (10). The screening itself may be time-consuming and costly and it may require the assistance of dedicated, specialized personnel. We hypothesized that routine measurements of glycated hemoglobin (HbA1c) levels could be used as first-line screening for MDs in children with diabetes. Such a combined psychological and metabolic screening procedure could capitalize on the uniqueness of pediatric diabetic care and reduce the projected time expenditure by implementing psychometric tools on a center-wide scale. The study was an attempt to estimate the efficiency and required time expenditure of initial screening composed of a combination of psychometric tools and HbA1c and to establish the most effective model which could be feasibly introduced to routine pediatric diabetologic practice.

RESEARCH DESIGN AND METHODS

Patients were enrolled into a long-term study on psychiatric comorbidity in type 1 diabetes. Preliminary data on behavioral problems were published previously elsewhere (11). The inclusion criteria were as follows: 1) age ≥8 years; 2) diabetes duration ≥1 year; 3) at least three HbA1c measurements per year, and 4) lack of significant coexisting diseases (asthma, celiac disease, juvenile rheumatoid arthritis, cystic fibrosis, previous diagnosis of mental retardation, severe congenital malformations or chromosomal disorders). All patients required insulin from the time of diagnosis. All examinations needed to be performed during hospital visits because of the time and supervision required for some of the tests; patients admitted for monitoring of diabetes complications or those referred to the in-hospital department for reeducation and insulin regimen modification were therefore assessed for eligibility. The visits were not associated with any acute events and were not the first visits of this type for any of the patients. Among 632 inpatients treated for type 1 diabetes between January 2010 and July 2011 in the Lodzkie administrative region in central Poland, 211 (33.3%) consecutively admitted children met the predefined inclusion criteria. Of that group, 176 agreed to participate in the study. The ethics committee of the Medical University of Lodz approved the study. Parents or legal guardians of all participants provided written, informed consent before initiation of any study procedures.

Children’s Depression Inventory

The Children’s Depression Inventory (CDI) is a self-report questionnaire consisting of 27 items widely used among youth with chronic health conditions, specifically diabetes (12). Results from CDI were normalized by transformation into T-scores, which ranged from 0 to 100 and the score of 50 corresponds to mean value in the population. With respect to its epidemiological definition, CDI T-scores >65 points (i.e., 1.5 SDs above the mean) are considered clinically significant. Parents completed the CDI:P, a parent report of the subject’s depressive symptoms developed for use in conjunction with the youth-reported CDI. For CDI:P, T-scores in the range of 59–61 correspond to clinical significance (13). In studies among children with type 1 diabetes, a cutoff point raw score of >13 on the CDI or >17 on the CDI:P may be used as a criterion for increased depressive symptoms without adjustment for sex or age (14). In the current study, both the child (CDI) and parent (CDI:P) reports of the tool were used for the purpose of time expenditure calculation, and we assumed that each child needed 15 min to fill in the form. The same amount of time was required the parent to answer the questions in CDI:P. The effort to review the questionnaire and to calculate and interpret the results according to appropriate reference ranges was assumed to equal 15 min of work for the clinical psychologist. In the study population, the internal consistency values (assessed with Cronbach α) of the CDI and CDI:P were 0.81 and 0.78, respectively. The CDI had moderate concurrent validity with the CDI:P (r = 0.30; P < 0.001).

Children's Depression Rating Scale–Revised

The Children's Depression Rating Scale–Revised (CDRS) is a clinician-rated scale of severity of depression. The revised version of the CDRS is widely administered among children and adolescents in clinical trials (15) to assess depressive symptoms in the course of both depressive and bipolar disorders (16,17). The tool is commonly applied to validate other study instruments because of its high diagnostic accuracy for depressive disorders (18,19). Because the original CDRS is free, it has great potential usefulness as a psychometric screening tool. Both children and their parents were interviewed by means of the CDRS. The total score was calculated as the average of the scores obtained from the child and parent interviews. We assumed that evaluation of a single child with the CDRS required 30 min of the patient’s time, 30 min for the parent’s interview, and 60 min of the clinical psychologist’s time. Because the scale was not originally designed for the adolescent population, psychometric properties were calculated. Cronbach α was 0.82, and the tool showed moderate concurrent validity with the CDI (r = 0.42; P < 0.001) and CDI:P (r = 0.35; P < 0.001).

Schedule for Affective Disorders and Schizophrenia for Children–Present and Lifetime version

The Schedule for Affective Disorders and Schizophrenia for Children–Present and Lifetime version (KSADS-PL) is a validated semistructured diagnostic interview that is based on the DSM-IV (20) and is widely used as the reference standard in evaluation of other psychiatric measures (21,22). The final diagnosis is based on the clinician’s synthesis of independently conducted child and parent interviews. Its main drawback is that the evaluation has to be performed by a trained psychiatrist and is time-consuming, with interview times ranging from 1.5 to 3 h. This interview tool was used with all enrolled children and served as a reference standard for psychiatric diagnoses. All analyzed screening models were tested for accuracy against the KSADS-PL results. To calculate the time expenditure, we assumed that examination of one patient with the KSADS-PL required 60 min for evaluation of the patient, 60 min for the parent interview, and 120 min of work for the consulting child psychiatrist.

HbA1c measures

For each patient, the mean HbA1c level from the preceding year was calculated. Prospectively collected HbA1c data from the diabetologic pediatric department were obtained from an existing long-term project addressing metabolic control in children with diabetes (23,24). Throughout the study period, laboratory methods for HbA1c assessment were consistent. HbA1c assays were performed by ion-exchange high-performance liquid chromatography with the Bio-Rad VARIANT Hemoglobin A1c Program (Bio-Rad Laboratories, Inc, Hercules, CA). The VARIANT Hemoglobin A1c Program has been certified by the National Glycohemoglobin Standardization Program as meeting the Diabetes Control and Complications Trial standard. The within-run coefficients of variation determined by the manufacturer were 1.05% and 0.94% for people without diabetes and for people with diabetes, respectively; the between-run coefficients of variation were 1.61% and 1.16% for people without diabetes and for people with diabetes, respectively. Blood samples were collected with the HbA1c Capillary Collection System (Bio-Rad Laboratories) and analyzed within 2–6 days (according to the manufacturer’s manual) and ≥24 h after blood collection, to allow complete Schiff base removal. Specimens prepared in this manner are stable for 2 weeks at room temperature or 4 weeks at 2–8°C.

For each patient, a mean HbA1c level and percentile were calculated from at least three measurements from the preceding year. A centile grid of HbA1c for the center was obtained with data from all 632 patients with type 1 diabetes who were treated for at ≥1 year preceding the starting date of the study.

Screening procedures

Parents and children completed CDI forms individually, in private, without assistance from medical personnel. The CDRS interview was conducted with children and their parents separately. To assess for comorbid MDs, patients and their parents were separately evaluated with the KSADS-PL in private. Both interviews were performed by a child and adolescent psychiatrist (A.B.) or a clinical psychologist (A.Z.) not involved in diabetes treatment of the patient. CDRS and KSADS-PL were assessed separately by different clinicians, who remained blinded to each other’s evaluation results throughout the testing procedures. Similarly, each patient’s HbA1c levels and final scores of self-report measures were unknown to both interviewers until the end of the evaluation procedure by the attending diabetologists.

To evaluate the necessary time incurred by different MD screening protocols, four hypothetical scenarios were compared. The first method was based on self-report and parent measures of CDI and CDI:P. The second variant included only a threshold value of HbA1c, followed by the KSADS-PL, which represents the scenario of a diabetologist obtaining mental health consultation for patients with the poorest metabolic control without engaging additional resources. The third approach included an initial HbA1c screening with a threshold value, followed up with the CDI and CDI:P. In the final scenario, initial HbA1c screening with a threshold value was followed up with examination with the CDRS (Fig. 1).

Figure 1

Efficacy and time expenditure data for four MD screening models showing scenarios used in clinical practice and potential combinations of metabolic and psychometric tools. The first two models consisted of a single initial step with the best thresholds in a psychometric scale or HbA1c level. All patients who met these criteria were considered candidates for psychiatric evaluation. Models 3 and 4 included an additional step in which patients who met the initial HbA1c criterion were evaluated with CDI and CDI:P or with CDRS. Only those who fulfilled both criteria were considered candidates for further psychiatric studies. FN, false negative; FP, false positive; NA, not applicable; TN, true negative; TP, true positive.

Results of the KSADS-PL were used as accuracy references in all four screening protocols. For all screening models, workload was defined as the time needed to diagnose a single patient with MD. For all models, the time needed to perform center-wide screening was calculated to compare with the costs of psychiatric or psychological consultation in other potential settings. Time needed to perform all diagnostic procedures was computed on the basis of the respective tools’ manuals. The manuals of all the tools, however, report only the time required to fill in the questionnaire by the patient or caregiver. The working time needed to interpret the self-report measures was therefore based on the authors’ professional experience. Because the study was intended to assess medical resource use, our main interest was in the time expenditure of the medical personnel rather than the time spent by the patients and their parents. HbA1c measurements did not require the physician to perform the procedure personally, did not involve any additional procedures, and were performed as part of routine management. The time needed to perform HbA1c tests was therefore set equal to 0 min.

Statistical analysis

Categorical variables were compared with the two-tailed Fisher exact test. The Mann-Whitney U test was used for comparisons of continuous variables, and the Spearman rank correlation was used for their assessment. The optimal threshold scores for predicting diagnosis of MD and MDD were identified through receiver operating characteristic curve analysis by selecting the cutoff values with the lowest overall error rate. Overall performance for each screening method is presented as positive likelihood ratio (LR), negative LR, positive predictive value (PPV), negative predictive value (NPV), and area under the curve (AUC). The C test for AUC comparison was used for pairwise comparisons of the analyzed screening tools. The level of statistical significance was set at P < 0.05. Statistical analysis was performed with Statistica 9.0 (StatSoft, Tulsa, OK).

RESULTS

Complete psychometric data (KSADS-PL, CSRS, CDI, and CDI:P) were collected from 163 patients and their parents (158 cases) or legal guardians (5 cases). There was no statistically significant difference in HbA1c between study group members and nonparticipants (7.94 ± 1.68% vs. 7.76 ± 1.17%; P = 0.12). Mean age of the subjects equaled 13.6 ± 2.6 years, and 43% of the subjects (n = 70) were female. The reported median monthly income per family member was U.S. $206 (interquartile range 147–441), which was marginally higher than the social minimum (U.S. $201) for households with three family members in Poland. As a social minimum, we have adapted a definition of the income needed to maintain a minimum acceptable standard of living. The level of maternal education was higher education for 30% of participants (n = 48), secondary education for 38% (n = 60), postprimary vocational education for 27% (n = 42), and primary education for 5% (n = 8). The level of paternal education was higher education for 23% of participants (n = 36), secondary education for 24% (n = 38), postprimary vocational education for 48% (n = 75), and primary education for 5% (N = 9). There were 44 children raised in single-parent families. Five patients were under the care of other family members (two were cared for by grandparents, two by siblings of one of the parents, and one by the child’s siblings). Characteristics of the study group are presented in the Table 1. Seven patients (4.3%) were diagnosed as having MD of any kind at the time of evaluation. Four (2.5%) met DSM-IV criteria for MDD. Two patients were diagnosed with dysthymia and one with cyclothymic disorder. Subjects with MD had higher scores for all psychometric measures and had significantly higher HbA1c levels (Table 1). HbA1c levels correlated positively with CDI (R = 0.26; P < 0.001), CDI:P (R = 0.22; P < 0.01), and CDRS (R = 0.22; P < 0.01) scores. Thus high HbA1c levels were observed among children with a high risk of MD. Diagnostic efficacy was calculated with regard to MD and MDD for all study parameters (Table 2). No differences were observed between AUCs for HbA1c levels and psychometric measures for MD (Supplementary Fig. 1) or MDD (all P > 0.4). Because of very low prevalence of MD and MDD in the study group, all tested screening tools were characterized by high NPVs (97.9–100.0) and low PPVs (12.1–36.4).

Table 1

Demographic, social, and clinical characteristics of participants

Table 2

Comparison of diagnostic accuracies of HbA1c level and psychometric measures for the diagnoses of any MD and MDD

Efficacy and time expenditure were compared among the four screening protocols for MD and MDD. Best diagnostic parameters were obtained with the HbA1c and CDRS sequence as the reference. Furthermore, this procedure was associated with the lowest time expenditure of the psychiatrist because it was the most effective at filtering out false-positive cases at both stages (Fig. 1). The HbA1c plus CDRS screening protocol used self-rated measures to reduce the number of patients who would be unnecessarily directed to psychiatric consultation from 42 patients to 2. Center-wide application of the protocol would result in a reduction of the psychiatry consultant time expenditure by 83% (302 h) for MD and by 91% (310 h) for MDD screening relative to use of the CDI and CDI:P (Fig. 2). Analysis of medical personnel’s and patients’ time use yielded similar results, also choosing the HbA1c and CDRS model of screening as the most efficient (Supplementary Table 1).

Figure 2

Efficacy and time expenditure data for four screening models for MDD showing scenarios used in clinical practice and potential combinations of metabolic and psychometric tools as in Fig. 1. FN, false negative; FP, false positive; NA, not applicable; TN, true negative; TP, true positive.

CONCLUSIONS

Depression is diagnosed by the same diagnostic criteria (DSM-IV or ICD-10) in the general population and in patients with comorbid physical illness. Screening tools designed for physically healthy individuals are also used in patients with somatic diseases (25). Unfortunately, the general depression questionnaires widely used in primary care show a low detection rate for diabetic patients. This necessitates the use of different cutoff thresholds for patients with chronic disease to ensure adequate screening performance (26,27).

This study demonstrated that pediatric diabetologists have an excellent diabetes-specific measure, HbA1c level, which could be used as a first step in MD screening. The findings demonstrate that HbA1c is a useful marker for detecting MD among individuals with type 1 diabetes, and the use of HbA1c as a screening test could reduce the workload on clinical psychologists. Screening with a threshold HbA1c level followed by subsequent CDRS evaluation was very efficient. The suggested approach may result in substantial savings of financial and human resources relative to the standard screening, which according to International Society for Pediatric and Adolescent Diabetes guidelines should be performed by mental health professionals (10). Combination of HbA1c and CDRS screening and examinations made on the basis of clinical decisions would provide a feasible way of implementing a formalized prevention program of MD and MDD among children with type 1 diabetes.

A potentially useful trait of HbA1c level is that it may be converted to a center-specific percentile value through the use of a centile grid of HbA1c for all patients from a given department or clinic. Such transformation of HbA1c levels makes this measure independent of the exact metabolic control rate of the patient and quality of care in a particular setting, thus providing an unbiased threshold.

The thresholds obtained in the current study (8.7% for MD and 9.0% for MDD) correspond to the 93rd and 95th percentiles of results in the center, respectively. This information allows generalization of results to target the 7% of patients with the poorest metabolic control of diabetes for further work-up for MD. Age of the studied patients may be considered as a confounding factor, because adolescents are expected to have higher HbA1c levels than younger children; however, the risks of MD and MDD (and their complications) are also greater in adolescents. Because of sample size limitations and relatively low prevalence of the studied disorders, we were unable to perform age-adjusted multivariate analysis of the diagnostic accuracy of either test. Because of the previously described association of age with both higher HbA1c and risk of MD, however, we came to the conclusion that the percentile threshold will be crossed by more adolescents than younger children, which is in line with the expected likelihood of MD.

There may have been other factors affecting the risk of MD associated with psychosocial factors. The scope of an analysis encompassing all these variables exceeded the available sample size. We therefore refrained from performing detailed analyses, although we plan to investigate further in follow-up studies.

Our study has several limitations resulting from the specific challenges of pediatric and adolescent psychiatry and diabetology. The proposed protocol aims to select patients with clinical depressive symptoms that worsen metabolic control of diabetes. The initial HbA1c-dependent step may miss patients with subthreshold depressive symptoms and patients with HbA1c values in the normal range. Children with mild mood and behavioral changes could be directed to preventive interventions according to clinical judgment. Thorough clinical observation still remains the best way to identify patients in need of the intervention who do not meet proposed thresholds for screening but have suspected mental illness.

The single-center setting may have also introduced bias by factors such as socioeconomic background and organization of health care in the region; however, such a setting provided standardized conditions of diabetologic care and diagnostic procedures delivered by mental health professionals. Furthermore, 23% of patients initially meeting inclusion criteria could not be included in the analysis because of lack of agreement to participate (n = 36) or incomplete psychometric data (n = 13). There is a possibility that comorbid affective disorders could be the underlying reason for nonparticipation, because such a phenomenon has been observed in previous clinical trials (28). Among the patients excluded from analysis, one was diagnosed by KSADS-PL as having dysthymic disorder but refused to complete any other psychometric measures, which resulted in his exclusion from further analyses. There were no significant symptoms of MD among the other 12 patients evaluated with KSADS-PL who did not want to complete psychometric tests. In addition, the study was conducted on inpatients hospitalized for poor metabolic control (26 cases) or routine monitoring of diabetes complications (137 cases). Consequently, 24% of the sample had HbA1c levels above the estimated 93rd percentile of the whole center. The presence of MD is expected to be a potential cause of metabolic deterioration, however, so screening for MD should primarily target these patients. Finally, the small sample size is a limitation, because it may have reduced the statistical power needed to detect some true differences between AUCs of the different screening models. Validation on a larger, independent cohort would validate the findings and help establish appropriate cutoff values of HbA1c and CDRS.

To sum up, our findings suggest that HbA1c levels may be an effective first screening tool for MD in children with type 1 diabetes. Application of a threshold HbA1c level screening followed by CDRS constitutes a highly accurate, time-efficient screening procedure for MD in children with type 1 diabetes.