Validation and proposal for cut-off values of an abbreviated version of the Alcohol Use Disorder Identification Test using the Korean National Health and Nutrition Examination Survey.

OBJECTIVE: Several abbreviated versions of the Alcohol Use Disorder Identification Test (AUDIT) have been developed and are widely used in clinical settings. In this study, we provide evidence supporting the use of abbreviated versions of AUDIT by comparing the utility of various abbreviated versions and determining cut-off values for the population of South Korea.
METHODS: Data were obtained from the 4th to 6th Korean National Health and Nutrition Examination Surveys. After calculating the whole AUDIT score, we applied the cut-off value of at-risk drinking proposed by the World Health Organization and divided the study sample into normal and at-risk drinking groups. Receiver operating characteristic curves were drawn for AUDIT-3rd question (Q3) alone, AUDIT-quantity and frequency (QF), AUDIT-consumption (C), AUDIT-4, and AUDIT-primary clinic (PC), and optimal cut-off values were obtained for each group.
RESULTS: A total of 46,450 subjects were analyzed. The at-risk drinking group comprised 29.2% of all subjects. The area under receiver operating characteristic curve (AUROC) of the abbreviated versions of AUDIT increased from 0.954 to 0.991 as the number of questions increased from one to four. The differences in AUROC between the abbreviated versions of AUDIT were statistically significant. The most appropriate cut-off values for AUDIT-Q3 alone, AUDIT-QF, AUDIT-C, AUDIT-4, and AUDIT-PC for adults over age 19 were 2, 4, 5, 6, and 4 points, respectively.
CONCLUSION: As the number of items analyzed increased from one to four items, the AUROC increased to a statistically significant level. Cut-off values for abbreviated versions of AUDIT are similar in South Korea to other countries.

INTRODUCTION

It is necessary to understand current patterns of alcohol consumption before setting up regulatory policies. Alcohol consumption patterns are analyzed through national health indicators in the United States and other countries [1]. The South Korean government conducts a nationwide survey, the Korean National Health and Nutrition Examination Survey (KNHANES), that includes the Alcohol Use Disorder Identification Test (AUDIT) developed by the World Health Organization [2]. The National Institute on Alcohol Abuse and Alcoholism has proposed several clinical situations as key opportunities to screen for at-risk drinking, including in emergency departments or urgent care centers, and when prescribing a medication that interacts with alcohol [3]. The emergency department is an important clinical setting for screening for at-risk drinking and conducting brief interventions, and many studies have shown that brief interventions in the emergency department focusing on alcohol consumption are effective [4-7]. However, it is often difficult to administer the full AUDIT questionnaire, which consists of 10 questions, in the emergency department, where urgent care is conducted simultaneously. A substantial amount of research on the abbreviated versions of AUDIT has been performed, and various tests and cut-off values have been presented [8-15]. While these tests are internationally available, they must take into account each country’s culture and other characteristics. Specifically, cut-off values must be determined based on research findings and expert opinion. Research conducted to date on the AUDIT in South Korea is sparse and has been limited to males [15]. In the present study, we provide evidence supporting the use of abbreviated versions of AUDIT in South Korea by comparing the utility of various abbreviated versions in a national sample and determining appropriate cut-off values.

METHODS

Study design

We obtained the study data from the KNHANES website and analyzed alcohol-related items. This study, peformed using national public health data, was exempt from informed consent. The study was approved by the hospital’s institutional review board (EMCIRB 18-13).

Study population

Analyses were conducted on 46,450 individuals at least 19 years of age using data from the 4th to 6th KNHANES, conducted between 2007 and 2015.

Study protocol

Data were obtained by requesting raw data through the KNHANES homepage of the Korea Centers for Disease Control and Prevention [16]. The extraction framework of the KNHANES uses the most recent population and housing census data available at the time of sample design. The sample design of the KNHANES IV (2007 to 2009) used 2005 population and housing census data. In the case of the KNHANES V (2010 to 2012), the population and housing census data were aged at the time of the design of the survey, and the resident population and apartment complex survey data were used as the extraction framework. The KNHANES VI (2013 to 2015) stratified the extraction framework based on provinces, cities, towns and villages, as well as housing types. The KNHANES included a common variable survey, a health questionnaire, a screening survey, and a nutrition survey. Socioeconomic location indicators were also collected, and weights for the selected subjects in the study area were calculated. The health questionnaire included 12 alcohol-related items: 10 questions based on the AUDIT and 2 questions about drinking experience and expert consultation. The alcohol-related questions did not vary between survey waves, with the exception of the second year of KNHANES IV (2014). For this reason, data from the KNHANES IV-2 (2014) were excluded from analysis (Fig. 1).

Fig. 1.

Profile of study population. KNHANES, Korean National Health and Nutrition Examination Survey.

We summarized the abbreviated versions of AUDIT to be analyzed based on a literature review [8-15]. After obtaining the whole AUDIT score, we applied the cut-off values (8 for males, 7 for females and 7 for elderly over 65 years old) of at-risk drinking proposed by the World Health Organization and divided survey respondents into normal and at-risk drinking groups [17]. At-risk drinking is defined as drinking that increases the risk of alcohol-related problems. The at-risk drinking group was stratified into three groups (male, female, and elderly).

Statistical analysis

Receiver operating characteristic curves were drawn for AUDIT-3rd question (Q3) alone, AUDIT-quantity and frequency (QF), AUDIT-consumption (C), AUDIT-4, and AUDIT-primary clinic (PC), and optimal cut-off values were obtained for each group. The KNHANES was designed to extract representative samples for the population over 1 year of age living in South Korea. The survey was conducted using two-stage stratified cluster sampling, a complex sampling method requiring that strata, cluster, and sample weight be taken into account in data analysis. However, in the present study we did not intend to measure national prevalence of at-risk drinking, and our analyses were conducted using simple random sampling methods. Area under receiver operating characteristic curves (AUROC) of previous abbreviated versions of AUDIT were obtained, and the difference between the areas was analyzed using the method of DeLong et al. [18] Analyses were conducted using MedCalc Statistical ver. 17.2 (MedCalc Software, Ostend, Belgium; https://www.medcalc.org, 2017).

RESULTS

Baseline characteristics

A total of 46,450 subjects were analyzed. 19,703 (42.4%) were male and 26,747 (56.7%) were female. The average age was 50.0 years. The at-risk drinking group comprised 29.2% of all subjects, and 51.7%, 17.6%, and 15.2% of the male, female, and elderly groups, respectively.

Comparison of outcomes

Six abbreviated versions of AUDIT were analyzed, as shown in Table 1 [9,14,19-31]. The number of questions and the development process varied between versions. The AUROC of the abbreviated versions of AUDIT increased from 0.954 to 0.991 as the number of questions analyzed increased. The differences in AUROC between the abbreviated versions were statistically significant. For the Fast Alcohol Screening Test, the AUROC was not applied as a dichotomous test (Tables 2, 3, and Fig. 2).

Table 1.

Abbreviated versions of AUDIT

Test		Selection methods	Reference standard	Year
AUDIT	Q1–Q10	a. Grouping of 150 items (domains)	Hazardous or harmful alcohol use (any one of the following): hazardous daily level of consumption, recurrent intoxication, abnormal drinking behavior, at least one alcohol-related problem in the last year, an alcoholrelated disease, or a perceived drinking problem	1989 [19]
		→ select 4 domains by intrascale reliability, correlation analysis and factor analysis with alcohol intake
		→ select items by mean item-to-total correlation coefficient
		b. Judgement of experts
AUDIT-Q3 alone	Q3	Previous study [20-22]		1998 [14]
AUDIT-QF	Q1, Q2	Previous study [23]		2006 [24]
AUDIT-C	Q1, Q2, Q3	Previous study [25-29]		1998 [14]
AUDIT-4	Q1, Q2, Q3, Q10	Logistic regression analysis	Weekly alcohol consumption 280 g for males, 168 g for females, or criteria for hazardous or harmful drinking (WHO)	2002 [9]
AUDIT-PC	Q1, Q2, Q4, Q5, Q10	Logistic regression analysis	Alcohol dependence, harmful alcohol use (ICD-10), daily pure ethanol 40 g for males, 20 g for females	1997 [30]
FAST	Q3 → Q5, Q8, Q10	Principle component analysis → combination with best sensitivity, specificity	Whole AUDIT score	2002 [31]

AUDIT, Alcohol Use Disorder Identification Test; Q, question; QF, quantity and frequency; C, consumption; WHO, World Health Organization; PC, primary clinic; ICD-10, International classification of disease, 10th revision; FAST, Fast Alcohol Screening Test.

Table 2.

AUROCs of abbreviated versions of the AUDIT

	AUROC	95% CI
AUDIT-Q3 alone	0.954	0.952–0.956
AUDIT-QF	0.972	0.971–0.974
AUDIT-C	0.981	0.980–0.982
AUDIT-4	0.990	0.989–0.991
AUDIT-PC	0.991	0.990–0.991
FAST	-	-

AUROC, area under the receiver operating characteristic curve; AUDIT, Alcohol Use Disorder Identification Test; CI, confidence interval; Q3, 3rd question; QF, quantity and frequency; C, consumption; PC, primary clinic; FAST, Fast Alcohol Screening Test.

Table 3.

Pairwise comparisons of area under receiver operating curve of abbreviated versions of AUDIT

Abbreviated test		DBA	95% CI	P-value
AUDIT-Q3 alone	AUDIT-QF	0.019	0.017–0.021	< 0.001
	AUDIT-C	0.027	0.026–0.029	< 0.001
	AUDIT-4	0.036	0.034–0.038	< 0.001
	AUDIT-PC	0.037	0.035–0.039	< 0.001
AUDIT-QF	AUDIT-C	0.009	0.008–0.009	< 0.001
	AUDIT-4	0.018	0.017–0.018	< 0.001
	AUDIT-PC	0.018	0.017–0.019	< 0.001
AUDIT-C	AUDIT-4	0.009	0.008–0.009	< 0.001
	AUDIT-PC	0.010	0.008–0.010	< 0.001
AUDIT-4	AUDIT-PC	0.001	0.000–0.001	0.001

AUDIT, Alcohol Use Disorder Identification Test; DBA, difference between areas; CI, confidence interval; Q3, 3rd question; QF, quantity and frequency; C, consumption; PC, primary clinic.

Fig. 2.

Receiver operating characteristic curves of abbreviated versions of the Alcohol Use Disorder Identification Test (AUDIT). Q3, 3rd question; QF, quantity and frequency; C, consumption; PC, primary clinic.

The most appropriate cut-off values for the AUDIT-Q3 alone, AUDIT-QF, AUDIT-C, AUDIT-4 and AUDIT-PC for all adults over 19 years were 2, 4, 5, 6, and 4 points, respectively. In subgroup analyses, the most appropriate cut-off values were 2, 5, 6, 7, and 5 points for males; 2, 3, 5, 5, and 4 points for females; and 1, 3, 3, 5, and 4 points for elderly. Most cut-off values were higher in males than in females and lower in elderly individuals than in the non-elderly (Table 4).

Table 4.

Cut-off values, sensitivities, and specificities of abbreviated versions of the AUDIT

Subgroup	Cut-off value	Sensitivity, 95% CI		Specificity, 95% CI
AUDIT-Q3 alone
All subjects	2	87.4	86.8–87.9	92.5	92.2–92.7
Male	2	91.6	91.0–92.2	84.5	83.6–85.3
Female	2	85.5	84.3–86.6	92.7	92.3–93.1
Elderly	1	86.9	85.2–88.4	92.7	92.1–93.2
AUDIT-QF
All subjects	4	89.7	89.2–90.2	92.9	92.6–93.2
Male	5	82.8	82.0–83.6	93.6	93.0–94.1
Female	3	94.7	93.9–95.4	85.8	85.3–86.3
Elderly	3	94.5	93.3–95.5	90.4	89.8–91.0
AUDIT-C
All subjects	5	93.7	93.3–94.1	91.2	90.9–91.5
Male	6	93.0	92.4–93.5	87.5	86.7–88.3
Female	5	90.3	89.2–91.2	93.6	93.2–93.9
Elderly	3	98.2	97.4–98.7	88.2	87.5–88.8
AUDIT-4
All subjects	6	93.0	92.6–93.5	95.9	95.6–96.1
Male	7	91.7	91.1–92.3	94.9	94.4–95.4
Female	5	93.7	92.8–94.4	93.3	92.9–93.7
Elderly	5	95.9	94.8–96.7	96.5	96.1–96.9
AUDIT-PC
All subjects	4	98.1	97.9–98.4	90.9	90.6–91.2
Male	5	97.4	97.1–97.8	90.7	90.0–91.4
Female	4	94.5	93.7–95.3	95.4	95.1–95.7
Elderly	4	97.9	97.1–98.5	94.2	93.7–94.7
FAST
All subjects	-	89.4	88.9–89.9	96.7	96.5–96.9
Male	-	93.6	93.1–94.1	92.9	92.2–93.5
Female	-	84.0	82.7–85.2	97.1	96.8–97.4
Elderly	-	80.2	78.3–82.1	98.8	98.6–99.0

AUDIT, Alcohol Use Disorder Identification Test; CI, confidence interval; Q3, 3rd question; QF, quantity and frequency; C, consumption; PC, primary clinic; FAST, Fast Alcohol Screening Test.

DISCUSSION

To validate the abbreviated versions of AUDIT and to propose appropriate cut-off values for South Korea, we used data from the KNHANES IV to VI excluding KNHANES VI-2 (2014). Diagnosis of at-risk drinking was based on the whole AUDIT score. AUROC, cut-off value, sensitivity, and specificity were obtained for each abbreviated version of AUDIT. In previous studies on AUDIT-C, the most widely used abbreviated version, the gold standards for at-risk drinking group were set as (1) whole AUDIT score [15,31,32], (2) 4 standard drinks in a day or 14 standards drinks in a week for men, 3 standard drinks in a day or 7 standards drinks in a week for women and elderly [11-13], (3) pure alcohol consumption per week of 280 g for men or 168 g for women [9], and (4) daily alcohol consumption of 40 grams for men or 20 g for women [10]. In our study, the whole AUDIT score was used as a gold standard for at-risk drinking. The AUDIT is the most widely used alcohol screening test in the world, and its reliability has been demonstrated in various studies. The cut-off values of AUDIT for at-risk drinking proposed by the World Health Organization are 8 points for men, 7 points for women and 7 points for elderly people. To examine the previous studies of AUDIT cut-off values for at-risk drinking or problem drinking in South Korea, Kim et al. [33] conducted a study in which individuals were classified as ‘normal,’ ‘physical problem drinking,’ ‘alcohol abuse,’ or ‘alcohol dependence.’ The gold standard was a Diagnostic and Statistical Manual of Mental Disorders, 4th edition questionnaire combined with a blood test. In this study, the term ‘physical problem drinking’ refers to ‘no mental-social problems,’ i.e., abnormal blood tests, but not alcohol abuse or alcohol dependence. As a result, a cut-off value of 12 points on the AUDIT was proposed for physical problem drinking in South Korea. In another study from South Korea [34], the AUDIT cut-off values for at-risk drinkers were 10 in males, 6 in females, and 8 in all subjects combined, using a simple 4 questions self test for alcohol screening, CAGE (cut down, annoyed, guilty, eye-opener) 1 point as the gold standard. It should be noted that accepting US standards without modification may be problematic. However, in previous studies of AUDIT cut-off values in Korea [33,34], different definitions were applied for problem drinking or at-risk drinking. As it would be unreasonable to apply the results obtained by that method as is, we instead chose to apply internationally accepted standards [17].

In our study, the AUROC increased to a statistically significant level as the number of items was increased from one to four. There was no difference in the AUROC between the four-item AUDIT-4 and the AUDIT-PC. In terms of the AUROC, as the number of items increases, the test becomes more accurate. However, in order to maximize simplicity and brevity of testing, using fewer items would be advantageous. From this perspective, AUDIT-QF or AUDIT-C would be preferred as abbreviated versions of AUDIT.

In this study, optimal cut-off values were obtained for all adults over 19 years of age and for three subgroups. The cut-off values were 2, 2, 2, and 1 for AUDIT-Q3 alone; 4, 5, 3, and 3 for AUDIT-QF; 5, 6, 5, and 3 for AUDIT-C; 6, 7, 5, and 5 for AUDIT-4; and 4, 5, 4, and 4 for AUDIT-PC in all adults, males, females and elderly individuals, respectively. Most cut-off values were higher in males than in females and lower in elderly compared to non-elderly individuals. Appropriate cut-off values for screening for at-risk drinking have been presented in several studies conducted in various countries. In a Korean validation study of AUDIT-C for screening for problem drinking [15], the AUDIT-C cut-off value for risk drinking in adult males was 8, based on whole AUDIT scores ranging up to 12. In contrast, in a study conducted in Spain in 2002 [9], the proposed AUDIT-C cut-off values were 5 for males and 4 for females based on alcohol consumption per week of 280 g for men and 168 g for women. In a study conducted in Taiwan in 2008 [10], the determined cut-off values for AUDIT-C were 4 for females and 5 for males, based on 40 g per serving for men and 20 g per serving for women. In several studies conducted in the United States, the AUDIT-C cut-off value for problem drinking was in the range of 4 to 5 for men and 2 to 3 for women, with results varying slightly between studies [8,11,12]. Cut-off values of abbreviated versions of AUDIT found in other countries are 1 to 2 for AUDIT-Q3 alone, 3 to 4 for AUDIT-QF, 4 to 6 for AUDIT-C for males and 2 to 5 for females, 5 to 7 for AUDIT-4, and 4 to 5 for AUDIT-PC [8-12,14,15,17,24,30,31,35,36], which are similar to our results. In our study, we propose appropriate cut-off values by analyzing not only gender but also the elderly population over 65 years of age. In addition, using KNHANES as a source of representative national health data, our results are generalizable to the entire country. The primary limitation of our study was that the exact amount or frequency of alcohol consumption was unknown, because alcohol-related items in KNHANES mainly consist of AUDIT questions. Using the National Institute on Alcohol Abuse and Alcoholism criteria for at-risk drinking (4 standard drinks in a day or 14 standards drinks in a week for men, 3 standard drinks in a day or 7 standards drinks in a week for women and elderly) as a gold standard, the results of our study will be more reliable and easily compared to other studies.

In summary, using the KNHANES, the sensitivity, specificity, AUROC and appropriate cut-off value of abbreviated versions of AUDIT for Korean adults are proposed in this study. The AUROC increased to a statistically significant level as the number of items analyzed increased from one to four. The cut-off values found for the abbreviated versions of AUDIT were similar to those found in other countries. By understanding current patterns of alcohol consumption and establishing cut-off values for Korea, we expect the abbreviated versions of AUDIT to play an important role as a valid screening test.