A Validation Study of the Brief Alcohol Use Disorder Identification Test (AUDIT): A Brief Screening Tool Derived from the AUDIT.

BACKGROUND: The prevalence of alcohol use disorder (AUD) is very high in Korea. To identify AUD in the busy practice setting, brevity of screening tools is very important. We derived the brief Alcohol Use Disorders Identification Test (AUDIT) and evaluated its performance as a brief screening test.
METHODS: One hundred male drinkers from Kangbuk Samsung Hospital primary care outpatient clinic and psychiatric ward for alcoholism treatment completed questionnaires including the AUDIT, cut down, annoyed, guilty, eye-opener (CAGE), and National Alcoholism Screening Test (NAST) from April to July, 2007. AUD (alcohol abuse and dependence), defined by a physician in accordance with Diagnostic and Statistical Manual of Mental Disorders-IV, was used as a diagnostic criteria. To derive the brief AUDIT, factor analysis was performed using the principal component extraction method with a varimax rotated solution. Receiver operating characteristic (ROC) curve analysis was performed to investigate the discrimination ability of the brief AUDIT. Areas under the ROC curve were compared performance of screening questionnaires with 95% confidence intervals.
RESULTS: The derived brief AUDIT consists of 4 items: frequency of heavy drinking (item 3), impaired control over drinking (item 4), increased salience of drinking (item 5), and alcohol-related injury (item 9). Brief AUDIT exhibited an AUD screening accuracy better than CAGE, and equally to that of NAST. Areas under the ROC curves were 0.87 (0.80-0.94), 0.76 (0.66-0.85), and 0.81 (0.73-0.90) for the brief AUDIT, CAGE, and NAST for AUD, and 0.97 (0.95-0.99), 0.93 (0.88-0.98) and 0.93 (0.88-0.98) for alcohol dependence.
CONCLUSION: The new brief AUDIT seems to be effective in detecting male AUD in the primary care setting in Korea. Further evaluation for women and different age groups is needed.

INTRODUCTION

The proportion of Korean adults who drink alcohol is among the world's highest and is increasing-from 45.8% in 1989 to 59.2% in 2005.1,2) According to the Korean national health and nutrition examination survey, the overall age-adjusted prevalence of alcohol use disorder (AUD: alcohol abuse and alcohol dependence) in 2009 was 38.8%. Men were about 7 times as likely as women to meet the criteria for AUD.3) A large portion of Korean men drink alcohol excessively. In addition, the prevalence of AUD is much higher in Korea compared to other countries.4-6)

Alcohol is implicated in a wide variety of diseases, disorders, and injuries, as well as many social and legal problems. The majority of excessive drinkers are undiagnosed. Often they present with symptoms or problems that would not normally be linked to their drinking. Brief intervention is a low-cost, effective preventive measure for heavy drinkers in outpatient settings. Therefore, screening heavy drinkers who need brief intervention is very important, especially in primary care.7)

Cut down, annoyed, guilty, eye-opener (CAGE), the Michigan Alcoholism Screening Test (MAST), National Alcoholism Screening Test (NAST), and Alcohol Use Disorders Identification Test (AUDIT) are well known tools to screen for AUD. CAGE is the briefest effective screening test for lifetime alcohol abuse and/or dependence but it is insensitive for detecting heavy drinking and does not distinguish between active and past AUD.8) MAST was devised as a structured interview instrument, and consists of 25 questions.9) NAST (Alcoholism Screening Test of Seoul National Mental Hospital) was devised to screen Korean alcoholism. It was originally designed with 12 items to screen AUD in those who need psychiatric intervention in psychiatric inpatients.10)

The AUDIT is the first screening test designed specifically for use in primary care settings, and validated in Korea.11) The AUDIT was developed to screen for excessive drinking and in particular to help practitioners identify people who would benefit from reducing or ceasing drinking.12,13) Although its sensitivity, specificity, and other psychometric properties compare at least equally or exceed those of other screening instruments, the length of the AUDIT may preclude its use in settings where brevity is critical, such as in a primary care setting or emergency room. For such purposes, abbreviated versions of the AUDIT exist such as AUDIT-consumption (C), AUDIT-Primary Care, AUDIT-3, Rapid Alcohol Problems Screen (RAPS), RAPS4-quantity-frequency, and Alcohol Screening Test (FAST).14-18) Among them, AUDIT-C, which consists of the first 3 items of the AUDIT (the consumption factor items), has generated wide research interest.19) Recently, AUDIT-C appeared to perform better than AUDIT-3 in the Korean primary care setting, but validated only against the full AUDIT rather than against an independent, formal alcohol diagnosis or hazardous drinking criterion.20)

As alcohol drinking behavior depends heavily on culture and ethnicity, we tried to derive a Korean short version of the AUDIT (brief AUDIT). We evaluated the brief AUDIT as a quick screening test for AUD, and compared the performance of the brief AUDIT with that of other validated alcohol screening questionnaires.

METHODS

1. Study Population

This study was conducted at Kangbuk Samsung Hospital primary care outpatient clinic and psychiatric ward for alcoholism treatment. The study population was 85 male drinkers from the primary care setting, and 15 male alcohol dependency patients from a psychiatric hospital from April to July, 2007, who consented with the study. The questionnaire was filled out by patients, and then their physician (1 psychiatrist and 4 family physicians) defined AUD in accordance with the Diagnostic and Statistical Manual of Mental Disorders-IV without knowing the results of the questionnaire.

2. Measures

The questionnaire consisted of demographic factors, AUDIT, CAGE, and NAST. The AUDIT core questionnaire consists of 10 items: three questions on quantity and frequency of drinking, three items on alcohol dependence and four questions on problems caused by drinking. Weighted scoring with respect to the frequency or the time of occurrence results in total scores ranging from 0 to 40.12)

The CAGE is an acronym based on its four items: cut down on drinking, annoyed by criticism, guilty feelings, and eye opener. NAST consists of 12 items: easily feels self-pity, prefers drinking alone, drinks in the morning, feels a compelling desire to drink continuously, has difficulty resisting the impulse to drink, has recent amnesia about events during drunkenness, feels drinking is harmful to social life, has had damage to occupation, has a spouse about to leave, suffers from alcohol withdrawal symptom, suffers from serious alcohol withdrawal symptom, has had treatment for alcohol related problems.10)

3. Statistical Analysis

To derive brief AUDIT, factor analysis was performed using the principal component extraction method with a varimax rotated solution. One-way analysis of variance was used to compare brief AUDIT between groups (normal, alcohol abuse, and alcohol dependence). Pearson correlation analysis was used to assess the relationship between tests.

Screening performance was evaluated by means of sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and areas under receiver operating characteristics (ROCs). ROC curves plotted sensitivity versus 1-specificity, and areas under the curves and their 95% confidence intervals were determined. The area under an ROC curve (AUROC) is a measure of a screening test's overall screening performance taking into account all possible screening thresholds. Areas under ROC curves closer to 1.0 indicate a better performance and an area under a curve of 0.5 yields no useful information. Cronbach's alpha was calculated to assess internal consistency of brief AUDIT. All statistics were performed using the SPSS ver. 15.0 (SPSS Inc., Chicago, IL, USA).

RESULTS

1. Characteristics of Study Population

Among subjects in the study population, 49 (49%) were defined as the normal group, 27 (27%) as abusive of alcohol, and 24 (24%) as alcohol-dependent. Mean age was 45.4 ± 11.0 (standard deviation) years. Table 1 shows the demographic characteristics of the study population.

Table 1

Demographic characteristics of study subjects

2. Development of Brief Alcohol Use Disorders Identification Test

Factor analysis was conducted to select questions from 10 items. Principal component extraction method identified 2 factors, and they accounted for 65% of total variance. The two factors and their components are presented in Table 2. One was alcohol-dependency and related problems (impaired control over drinking, increased salience of drinking, morning drinking, guilt after drinking, blackouts, alcohol-related injuries, and others concerned about drinking), and the other was consumption (frequency, typical quantity, and frequency of heavy drinking). To derive items for brief AUDIT, two criteria were adjusted. First, items with a factor loading value over 0.8 were selected. Items number 2, 3, 4, 5, and 9 were selected. Items 4, 5, and 9 belonged to factor one, and 2 and 3 to factor two (Table 2). Second, items which do not attribute Cronbach's alpha value were excluded. The AUDIT showed Cronbach's alpha of 0.872. After exclusion of item 2, alpha was 0.875 (Table 3). Item 2 was near the typical quantity. Among consumption items, it seemed frequency of heavy drinking (item 3) perform better than the typical quantity (item 2). Item 3 is the one of AUDIT-3. So, item 3 was left for the brief AUDIT. The final items for the brief AUDIT were frequency of heavy drinking (item 3), impaired control over drinking (item 4), increased salience of drinking (item 5), and alcohol-related injury (item 9). The score system was applied the same as in AUDIT, total scores ranging from 0 to 16.

Table 2

Result of factor analysis for Alcohol Use Disorders Identification Test*

*Values are factor loading.

Table 3

Item analysis of Alcohol Use Disorders Identification Test

Cronbach's alpha was calculated to assess internal consistency of the brief AUDIT. The brief AUDIT showed an alpha value of 0.82. The Cronbach's alpha of NAST, CAGE, and original AUDIT were 0.86, 0.69, and 0.87 each. The internal consistency analysis yielded high Cronbach's alpha values for the full and brief AUDIT and NAST. CAGE was not sufficiently consistent compared to that of other questionnaires.

3. Performance of Brief Alcohol Use Disorders Identification Test

The brief AUDIT showed a high correlation with the 10-item original AUDIT, with a coefficient of 0.93, good correlation with NAST of 0.80, and CAGE of 0.76 (P < 0.05). The brief AUDIT discriminated AUD diagnosis as presented in Table 4. The mean of each group was 3.8, that of alcohol abuse 6.0, and alcohol dependence 11.9. The difference was statistically significant (F = 93.473, df = 2, P < 0.001) (Table 4). ROC curves are depicted in the Figure 1, and the AUC for each ROC curve was calculated. Sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio by cut-off points are presented in Table 5.

Table 4

Comparison of brief AUDIT scores between normal, alcohol abuse, and AD

AUDIT: Alcohol Use Disorders Identification Test, AD: alcohol dependence.

*Analysis of variance and post-hoc.

Figure 1

Receiver operating characteristic curves of the brief Alcohol Use Disorders Identification Test (AUDIT), National Alcoholism Screening Test (NAST), and cut down, annoyed, guilty, eye-opener (CAGE) for alcohol use disorder (A) and alcohol dependence (B).

Table 5

Performance of brief AUDIT questionnaire for detecting any AUDs or AD

AUDIT: Alcohol Use Disorders Identification Test, AUD: alcohol use disorders, AD: alcohol dependence, +LR: positive likelihood ratio, -LR: negative likelihood ratio.

Optimum cut-off value was identified with the ROC curve. The brief AUDIT threshold that simultaneously maximized sensitivity and specificity for detecting AUD was ≥6 (sensitivity 83.3%, specificity 96.1%), and for alcohol dependence was ≥10 (sensitivity 83.3%, specificity 96.1%) (Table 5). As we included hospitalized AUD patients in the study, it is possible that our values are higher than in the primary care setting. The purpose of the brief AUDIT is to detect AUD in a short time. As maximizing sensitivity is the best policy for screening disease like AUDs, we chose a cutoff point of ≥5 (sensitivity 80.4%, positive likelihood ratio [+LR] 2.62) for AUDs, and ≥9 (sensitivity 87.5%, +LR 9.51) for alcohol dependence (Table 5).

The brief AUDIT's AUROC curve for AUD was 0.87 ± 0.04 (standard error [SE]). The brief AUDIT had a greater AUROC curve than either CAGE 0.76 ± 0.05 (SE), or NAST 0.82 ± 0.04 (SE) for alcohol abuse.

The brief AUDIT's AUROC curve for alcohol dependence was 0.97 ± 0.01 (SE). It was greater than that of NAST 0.93 ± 0.02 (SE) or CAGE 0.93 ± 0.03 (SE) (Table 6).

Table 6

AUCs comparison of brief AUDIT with CAGE and NAST

AUC: area under the curve, AUDIT: Alcohol Use Disorders Identification Test, CAGE: cut down, annoyed, guilty, eye-opener, NAST: National Alcoholism Screening Test, CI: confidence interval, AUD: alcohol use disorders, AD: alcohol dependence.

DISCUSSION

To our knowledge, this is the first study to derive a short form of the AUDIT (brief AUDIT) and compare the performance of it with that of well known AUD screening questionnaires, CAGE, and NAST, in the primary care setting in Korea.

The new brief AUDIT consists of 4 items: frequency of heavy drinking (item 3), impaired control over drinking (item 4), increased salience of drinking (item 5), and alcohol-related injury (item 9), with a scoring range from 0 to 16. The brief AUDIT discriminated AUD very well. The cut-off point was 5 to detect any AUD (sensitivity 80.4%, specificity 69.4%), and 9 for alcohol dependence (sensitivity 87.5%, specificity 90.8%). The brief AUDIT's AUROC curve for alcohol-use disorder was 0.87 ± 0.04 (SE), and for alcohol dependence was 0.97 ± 0.01 (SE). The brief AUDIT had a greater AUROC curve than either CAGE 0.76 ± 0.05 (SE), or NAST 0.82 ± 0.04 (SE) for Any AUD, and greater than either NAST 0.93 ± 0.02 (SE) or CAGE 0.93 ± 0.03 (SE) for alcohol dependence. As we targeted primary outpatient setting, we put weight for higher sensitivity to detect any AUD and higher specificity for alcohol dependence.

AUDIT was originally designed for practitioners to screen problem drinkers, i.e., drinking problems less severe than diagnosable disorders of AUD as well as AUD. The full 10-item AUDIT with its multidimensional scoring for each item has been called "cumbersome" for use in some settings where rapidity of scoring, as well as accuracy, is important.21) Many abbreviated versions of AUDIT were generated and tested to meet this practical need. AUDIT-C, the most commonly tested derived AUDIT, consists of the first 3 items of the AUDIT (the consumption factor items). AUDIT-C was reported useful to screen problem drinkers, and AUD in western countries. FAST is a 4-item scale consisting of item 3 (modified for men by increasing the number of drinks from 1 occasion to 8) as well as items 5, 8, and 10 from the original AUDIT. Unfortunately, these researchers validated the abbreviated scale only against the full AUDIT rather than also against an independent, formal alcohol diagnosis or hazardous drinking criterion.22)

Drinking behavior of the western population may be quite different from that of Korea. Heavy drinking is very popular because alcohol is accepted generously and quite often, and is socially forced in work-related gatherings, especially for men. This consumption factor might not be enough to discriminate AUD, however it is the main reason we decided to derive a short version for Koreans rather than using the existing one. We compared the performance of the brief AUDIT with the AUDIT-C and FAST. The AUROC against AUD was 0.868 for the brief AUDIT, 0.739 for the AUDIT-C, and 0.898 for FAST. The AUROC for alcohol dependence was 0.972 for the brief AUDIT, 0.664 for the AUDIT-C, and 0.942 for FAST (data not presented). The performance of the brief AUDIT was about equal to FAST, and better than the AUDIT-C. The performance of the AUDIT-C to detect AUD against formal alcohol diagnosis need to be further evaluated for Koreans.

The AUDIT questionnaire is measured consistently over time and is particularly useful in detecting recent problem drinking. Investigators in a six-country World Health Organization project developed the AUDIT to screen patients in primary health care settings for hazardous or harmful drinking and serve as a basis for discussion with patents in brief therapy.12) Hazardous drinkers are individuals at high risk for alcohol-related damage to physical or psychological health; harmful drinkers already experience such problems.23) The original cut-off point to detect hazardous drinker is 8, yielding a sensitivity of 0.90 s, and specificity of 0.80 s. To find the hazardous drinking cut-off point, the gold standard criteria need to be multidimensional: heavy consumption (i.e., 14 or more drinks per week), laboratory results or clinical diagnosis like alcoholic hepatitis, as well as diagnosis of AUD. The performance of AUDIT against those criteria needs to be evaluated further to make the AUDIT more practical, and at the same time, accurate for Koreans. Also, further study to evaluate the performance of the abbreviated version of the AUDIT against other criteria than AUD might be useful to detect problem drinkers.

Limitations of this study were as follows. Firstly, the study subjects were not representative of the entire population. Women were excluded, and the age range of study subjects was 32 to 63. Cut-off points could be different for different populations, as alcohol affects women differently. Women were excluded because alcohol screening questionnaires function differently in men and women19) and we could not recruit an adequate number of women through which to base any conclusions regarding questionnaire performance. Also we collected 15% of study subjects from the psychiatric ward and not only from primary outpatient clinics for the same reason. Secondly, performance of the brief AUDIT on criteria of problem drinking was not measured as mentioned above. Thirdly, test-retest was not evaluated. Lastly, we used the same population to derive and validate the brief AUDIT.

The brief AUDIT was short and performed well to screen alcohol-use disorders in middle-aged men. It will facilitate screening for alcohol problems where significant constraints on time exist. Further research to evaluate performance of the brief AUDIT against criteria of problem drinking in large populations including other age groups and women are needed to verify its effectiveness.