The diagnostic accuracy and validity of the teen screen questionnaire-mental health for clinical and epidemiological studies in primary-care settings.

BACKGROUND: To validate a brief, self-reported, Teen Symptom Questionnaire-Mental Health (TSQ-M), for identifying adolescents with mental ill-health, designed for conducting epidemiological studies and clinical work in primary-care settings.
MATERIALS AND METHODS: In this prospective, cross-sectional study of 146 adolescents, re-cruited six rural and urban schools, the newly developed TSQ-M as the measure for validation and General Health Questionnaire-12 item (GHQ-12) as the gold standard measure were administered by independent trained raters. Tests for diagnostic accuracy and validity were conducted.
RESULTS: A TSQ-M score of ≥29 (Sn=75.68%, Sp=68.06, +LR=2.37, -LR=0.36, PPV=70.9, NPV=73.1) with the AUC of 0.79, is suggested for screening use in Indian populations. Besides the adequate face and content validity, TSQ-M has moderate internal consistency (Cronbach's α = .64) suggesting that the construct of mental ill-health as conceptualized by TSQ-M has multiple sub-constructs. The presence of sub-constructs was demonstrated by an 8- factor structure, which explained 60% of variance.
CONCLUSION: The TSQ-M is a psychometrically adequate, yet a brief measure, for clinical and research work in identifying mental ill-health among adolescents in primary-care settings in India.

INTRODUCTION

Among adolescents, across the globe, chronic mental health issues have become more prevalent than physical problems in the last 30 years.[1] The global mental health burden among youth currently stands at 45% of the total burden[2] and thus identification of psychopathology is a pressing need among this population. It is estimated that there are about 243 million adolescents in India.[3] To mitigate the years lost because of disability caused by mental illnesses, with far smaller resources available than for the high income countries, it is important to develop India-centric measures and management programs.[4] Secondly, it is important to identify the mental health problems and disorders of adolescents, as early as possible, since many of these problems have the propensity to continue into adulthood.[5] Finally, with the paucity of human resource for mental health in India, it is important to develop and validate measures to conduct epidemiological and clinical work for these maladies at the primary-care settings itself. Only 2% of these children are treated by psychiatrists and 75% of children are seen by primary care pediatricians even in high-income countries.[6] In low to middle-income countries, for many of the adolescents schools are the most coherent system available, where both their physical and mental health needs can be identified and managed. This study presents the diagnostic accuracy and validation of a self-administered, Teens Symptoms Questionnaire-Mental Health (TSQ-M) for identifying mental health problems in school setting by auxiliary health professionals.

MATERIALS AND METHODS

This study was conducted as part of the Adolescent Health District Plan (AHDP) Project with the support of National Rural Health Mission (NRHM) Kerala state. Data was collected from six schools at Thiruvananthapuram district, giving equal sample representation to rural (one school from each of the rural taluks of Chirayinkeezhu, Nedumangad and Neyyattinkara) and urban schools (three schools from Thiruvananthapuram city). Schools from both these settings were randomly selected and students, of both genders, from 9th to 12th standard were included in the study if they gave verbal assent to their participation. Prior to the data collection, written permission to conduct the study was obtained from the District Educational Office and the project was approved by the Institutional Review Board of the Child Development Centre, Thiruvanthapuram. The confidentiality of the data was protected with reversible anonymization and by limiting the availability of the data to only the primary investigator and who did the statistical analysis.

Measures

Teens Symptoms Questionnaire (TSQ) is a self-rated 37-item checklist. It has two sections namely: Reproductive Health Section (TSQ-R) and Mental Health Section (TSQ-M). The Reproductive Health section had 16 questions (e.g., Body Mass Index, What is the average number of days of your menstrual cycle each month?), the mental health section had 21 questions on Attention Deficit Hyperactivity Disorder (e.g., Do you have a problem in sitting still for long?), Conduct Disorders (e.g., Have you been involved in setting fires, stealing, lying or fist fights?), Elimination Disorder (e.g., Do you wet your bed?), Anxiety Disorders (e.g., Do you have any fears that are perceived by others as unreasonable?), Mood Disorders (e.g., Do you think that you are hopeless or guilty in any way?), Psychoses (e.g., Have you heard voices or seen people when there were none around?), Substance Use Disorder (e.g., Do you smoke?), Non-specific biological symptoms (e.g., Has there been any change in your sleep?). The endorsement pattern for the TSQ-M was in a 3-point likert scale of ‘Never’, ‘Sometimes’, and ‘Often’. While most of the reproductive health-related questions were more objectively measurable, the mental health questions were more subjective and needed further validation against a gold standard. Thus the TSQ-M was the index measure for validation in this study.

General Health Questionnaire-12 item (GHQ-12) is a 12-item measure of current mental health among adults since 1970 and among adolescents as well.[7] This gold standard, self-reported measure of psychological morbidity, detects psychiatric disorders in community settings and non-psychiatric settings.[8] GHQ-12 cut-off of 2/3 (sensitivity= 87.4% and specificity= 79.2%) was used to define ‘caseness’ in this study as suggested for the Indian population.[9]

Data collection was done by qualified Clinical Child Developmental therapist trained in using the TSQ and GHQ-12. The data was collected with face-face interviews with the adolescent by these trained auxiliary health professionals using TSQ and GHQ-12, independently on the same day to minimize the rater bias as well as maximize the stability of the rating with time.

Data analysis

Sensitivity, specificity, likelihood ration and predictive values for various TSQ-M cut-off scores were calculated in order to determine the optimal screening threshold with Receiver Operating Characteristic (ROC) analyses, against the GHQ-12 cut-off of ≥2/3, and also Area Under the Curve (AUC) was calculated. For internal consistency, Cronbach's α coefficient was determined. To establish the sampling adequacy for the Exploratory Factor Analysis, we conducted the Kaiser–Meyer–Olkin and Barlett's test of sphericity. The Factor structure of TSQ-M was demonstrated by principal components analysis with promax rotation. The items were excluded if they failed to load on any factor (loading < 0.40) or had unacceptably high secondary loadings/cross loading (>0.40). Data was analyzed using SPSS (version 19) and MedCalc 12.2.1.

RESULTS

Among the 146 participants, 41% and 58% were boys and girls, respectively. The mean (SD) age of the teenagers was 15.67(3.52) years. Most of the participants were from a nuclear family (67.21%) and others were from extended (17.20%) and joint families (15.69%).

A total TSQ score of ≥29 had a sensitivity of 76%, specificity of 68%, positive likelihood ratio of 2.37, negative likelihood ratio of 0.36, positive predictive value of 70.9%, and a negative predictive value of 73.1% making it appropriate for screening in the primary-care settings. The area under curve (AUC) in the ROC for the TSQ was 0.79 (z statistics = 0.743, P = 0.0001) [Figure 1 and Table 1].

Figure 1

The receiver operating curve characteristics for TSQ-M. AUC = 0.79; z statistics=0.743, P = 0.0001. Coordinate points and the 95% confidence Interval for the ROC cure are represented by the broken lines.

Table 1

The diagnostic accuracy of TSQ-M

The face validity was judged by the psychiatry, pediatric and clinical psychologist faculty of the two participating institutions. Content validity was appropriate as all the 21 questions in the TSQ-M were endorsed by more than 95% of the participating students. The internal consistency, as measured by Cronbach's α, was 0.64. For further investigation of the construct validity, we explored the factor structure of the items in the TSQ-M as the Kaiser — Meyer — Olkin measure of sampling adequacy was 0.67 and Barlett's test of sphericity was significant (P=0.001). We extracted those factors with an eigen value of 1, and thus a 8-factor structure was derived. All the TSQ-M item loaded on to one factor but one item, “Do you use tobacco/or Pan?” cross-loaded on to more than one factor. This 8-factor structure explained 60.33% of the variance. The factor structure details are summarized in Table 2.

Table 2

The factor structure of TSQ-M

DISCUSSION

Teen Symptoms Questionnaire-Mental Health (TSQ-M) is a population measure of mental ill health due to an underlying psychiatric disorder, validated for use among teenagers in schools. Thus developing and validating a measure for epidemiological studies in primary-care setting has been accomplished. Construct wise, this questionnaire does not measure the hedonic and eudemonic wellbeing of a teenager as does the Warwick-Edinburgh Mental Well-being Scale validated for teenagers.[10] Conversely, TSQ-M can be compared in it conceptualization like other generic measures for mental ill health namely Strength and Difficulties Questionnaire, Indian Adolescent Health Questionnaire, and Patient Health Questionnaire. However, the TSQ-M includes more number of Priority Mental Health Disorders of adolescence than the Strength and Difficulties Questionnaire as well as Patient Health questionnaire.[1112] Also, the psychometric properties of TSQ-M are better psychometric than the Strength and Difficulties Questionnaire in this age group.[11] The validation methodology of TSQ-M has used both urban as well as rural schools unlike as in the Indian Adolescent Health Questionnaire,[13] where only participants from urban schools had been used.

The diagnostic accuracy of TSQ-M at the cut-off score of ≥29 has adequate sensitivity to be used as a screening instrument in the primary-care and our presentation of the diagnostic accuracy details [Table 1] allows the researchers to choose the cut-off scores as required by them for specific contexts and needs. The internal consistency of 0.64 was suggestive of multiple sub-constructs within the construct of mental ill-health as measured by TSQ-M. This was theoretically expected because of the inclusion of symptoms of many psychiatric disorders in the questionnaire. This was further proved by the 8-factor structure in the construct of mental ill-health by TSQ-M. Direct comparison of the TSQ-M cut-off values or other psychometric properties with data in the literature is not possible as TSQ-M is a newly developed measure and psychometric maturation will accrue only over the coming years.

The strength of our study is that the validation methodology followed the guidelines as given COnsensus-based Standards for the selection of health Measurement Instruments

(COSMIN) protocol[14] for validation and STAndards for the Reporting of Diagnostic accuracy studies (STARD) guidelines[15] for diagnostic accuracy, respectively, as well as used school sample in the study. The main limitations of the study is that, although the sample size of 146 participants is adequate for an Exploratory Factor Analysis, larger sample size could have generated more stable factor structure models thereby improving the confidence in the validity of identified construct.

In conclusion, TSQ-M is an easy to use measure with adequate psychometric properties and can be used for clinical work and research in primary-care setting by teachers, auxiliary health-care professionals, policy makers in India.