Primary care validation of a single-question alcohol screening test.

BACKGROUND: Unhealthy alcohol use is prevalent but under-diagnosed in primary care settings.
OBJECTIVE: To validate, in primary care, a single-item screening test for unhealthy alcohol use recommended by the National Institute on Alcohol Abuse and Alcoholism (NIAAA).
DESIGN: Cross-sectional study. PARTICIPANTS: Adult English-speaking patients recruited from primary care waiting rooms. MEASUREMENTS: Participants were asked the single screening question, “How many times in the past year have you had X or more drinks in a day?”, where X is 5 for men and 4 for women, and a response of 1 or greater [corrected] is considered positive. Unhealthy alcohol use was defined as the presence of an alcohol use disorder, as determined by a standardized diagnostic interview, or risky consumption, as determined using a validated 30-day calendar method. MAIN
RESULTS: Of 394 eligible primary care patients, 286 (73%) completed the interview. The single-question screen was 81.8% sensitive (95% confidence interval (CI) 72.5% to 88.5%) and 79.3% specific (95% CI 73.1% to 84.4%) for the detection of unhealthy alcohol use. It was slightly more sensitive (87.9%, 95% CI 72.7% to 95.2%) but was less specific (66.8%, 95% CI 60.8% to 72.3%) for the detection of a current alcohol use disorder. Test characteristics were similar to that of a commonly used three-item screen, and were affected very little by subject demographic characteristics.
CONCLUSIONS: The single screening question recommended by the NIAAA accurately identified unhealthy alcohol use in this sample of primary care patients. These findings support the use of this brief screen in primary care.

INTRODUCTION

Unhealthy alcohol use, the spectrum from risky consumption to the alcohol use disorders, alcohol abuse and dependence, is prevalent in the primary care setting and is under-diagnosed1. Screening and brief intervention by primary care physicians for those with unhealthy alcohol use reduces risky consumption2. Because of this, practice guidelines recommend universal screening3. Time is limited, however, and commonly-used alcohol screening instruments are comprised of multiple questions, often do not cover the full spectrum of unhealthy use, can be time consuming to administer and may require scoring4,5. Consequently, many patients are not screened6,7. Single-question screening tests for unhealthy alcohol use may help to increase the frequency of screening in primary care. The National Institute on Alcohol Abuse and Alcoholism (NIAAA) recommends, in its clinician’s guide, one such single-question screen for unhealthy alcohol use8. The recommended question asks “How many times in the past year have you had X or more drinks in a day?” (where X is 5 for men and 4 for women, and a response of ≥ 1 is considered positive). While similar single-question screens (which used different phrasing, alcohol quantity and time cutoffs) have been validated in various settings, the NIAAA recommended screening test has not been validated in the primary care setting9–12. Because of the wide dissemination of this guide and practice recommendation, we attempted to validate this version of the screening question in a sample of primary care patients.

SUBJECTS AND METHODS

Subjects

Subjects were selected by a research associate who systematically approached patients in the waiting room of a primary care clinic in an urban safety net hospital. Prior to being approached for eligibility screening patients saw no advertisement or indication by the research associate as to what the study was about. Patients who were under the age of 18 were excluded, as were those who, in the judgment of the research associate, would be unable to complete the questionnaire because of limited English, cognitive impairment or acute illness. People in the waiting room accompanying patients who reported not themselves being patients of the clinic were also excluded. The Institutional Review Board of Boston University Medical Center reviewed and approved all study procedures.

Data Collection

Interviews were conducted by trained research staff in a private setting and data were recorded anonymously, unaccompanied by any unique identifiers.

Screening Question

In accordance with the strategy recommended in the NIAAA Clinician’s Guide, subjects were first asked a pre-screening question, “Do you sometimes drink alcoholic beverages?”, and then the single screening question, “How many times in the past year have you had X or more drinks in a day?” (where X is 5 for men and 4 for women, and a response of ≥1 is considered positive). Subjects responding negatively to the pre-screening question were still asked the single screening question. If asked to clarify, the research associate provided definitions of a standard drink (12 ounces of beer, 5 ounces of wine, or 1.5 ounces of 80 proof spirits). For comparison purposes, the three-item Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) was administered following the single-question screen and before the other assessments13. After this, alcohol consumption and the presence or absence of an alcohol use disorder were assessed using reference standards.

Risky Alcohol Consumption Amounts

A validated calendar-based method (the timeline followback) was used for the measurement of alcohol consumption14. Using this method, subjects estimated the amount of alcohol consumed on each of the 30 days preceding the interview (summarized in analyses using the same standard drink definitions that appear above). Subjects were considered to have consumed risky amounts of alcohol if their average weekly alcohol intake over the preceding 30 days exceeded recommended limits (>14 drinks per week for men and >7 drinks per week for women) or if they reported exceeding recommended daily limits (>4 drinks per occasion for men and >3 drinks per occasion for women) on any of the 30 days15. Average weekly alcohol intake was calculated by multiplying by 7 the average number of standard drinks consumed per day during the 30 days.

Alcohol Related Problems

Subjects were then asked if they had ever experienced any of a list of problems related to alcohol use, from the Short Inventory of Problems (SIP)16. Subjects were considered to have alcohol related problems if they consumed risky amounts of alcohol and responded positively to any of the 15 SIP questions.

Alcohol Use Disorders

The computerized version of the Composite International Diagnostic Interview (CIDI) Substance Abuse Module was used for the assessment of current (12-month) alcohol use disorders (abuse and dependence)17. This structured questionnaire was administered by the research assistant, and subject responses were recorded electronically. The responses were then analyzed, using an algorithm, to yield a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnosis of alcohol abuse or dependence. Subjects with alcohol abuse or dependence as determined by the CIDI and who reported experiencing symptoms within the past 12 months were considered to have a current alcohol use disorder.

Unhealthy Alcohol Use

Subjects with unhealthy alcohol use either consumed risky amounts of alcohol (with or without associated alcohol problems), or had a current alcohol use disorder based on (as defined above) the timeline followback and the CIDI, respectively.

Statitical Analysis

We calculated the sensitivity (proportion of subjects with the condition of interest who tested positive), specificity (proportion of subjects without the condition of interest who tested negative) and likelihood ratios for the single-question screen for the detection of risky alcohol consumption amounts, risky consumption associated with problems, a current alcohol use disorder, or for unhealthy alcohol use in general (either the consumption of risky amounts or a disorder, the usual target for universal screening). A positive likelihood ratio is determined by dividing the sensitivity by (1 - specificity), while a negative likelihood ratio is (1 - sensitivity) divided by the specificity. The NIAAA clinician’s guide recommends a two-step screening process: subjects are asked if they sometimes drink alcoholic beverages, and only those who answer affirmatively are asked the screening question. In the main analysis we determined the test characteristics of this approach. In a sensitivity analysis we also determined the test characteristics of the single-question alone, without regard to the response to the pre-screening question. For comparison with the single-question screen, we calculated the sensitivity, specificity and likelihood ratios of the AUDIT-C for the detection of the same conditions. The AUDIT-C, which consists of three items, each with four possible responses, yields a score between 0 and 12. A total of more than three points is considered a positive test18. We calculated 95% confidence intervals using published formulas19. Statistical analyses were performed using Version 9.1 of the SAS System (copyright SAS Institute Inc.).

RESULTS

Subject recruitment Of the 1,781 people approached, 903 (51%) agreed to be screened for study eligibility (Fig. 1). Of these, 509 (56%) were ineligible for the study: 302 (33%) did not speak English, and 207 (23%) were not clinic patients. Of the 394 patients who were eligible, 4 (1%) refused to participate and 87 (22%) did not show up for the planned interview after the visit with their physician. Of the 303 subjects who arrived and gave consent to participate, 3 (1%) were unable to complete the interview. The data of 14 subjects (5%) were lost due to an electronic error, leaving 286 subjects whose data were analyzed (73% of those eligible).

Figure 1

Recruitment of subjects.

Subject Characteristics Of the 286 subjects, 54% were women, and the median age was 49 (range 21–86) (Table 1). The majority of subjects (63%) identified themselves as black or African-American, with whites (17%) and Hispanics (16%) comprising most of the remainder. Most (78%) had completed high school, but only 14% had completed college. Unhealthy alcohol use was reported by 31% of subjects: 6% consumed risky amounts but did not have alcohol-related problems or a disorder; 13% consumed risky amounts and had problems but no current disorder; and 12% had a current alcohol use disorder (2% of subjects reported a past year alcohol use disorder, but not past month risky consumption). The lifetime prevalence of alcohol use disorders (44%) and drug use disorders (47%) was high.

Table 1

Subject Characteristics

Characteristic	(n = 286) (%, n)
Female	54.2 (155)
Age
Mean ± SD	49.0 ± 12.3
Median (Range)	49.0 (21–86)
Education
Some high school	28.3 (81)
High school graduate	37.4 (107)
Some college	20.6 (59)
College graduate	9.8 (28)
Post-graduate education	3.9 (11)
Race
American Indian/Alaskan Native	2.8 (8)
Asian	2.5 (7)
Black or African American	62.6 (179)
Native Hawaiian/PI	1.0 (3)
White	17.1 (49)
Unknown	14.0 (40)
Hispanic or Latino ethnicity	16.1 (46)
English is first language	78.0 (223)
Drug use
Past year drug use*	34.6 (99)
Past year drug use disorder†	12.2 (35)
Lifetime drug use disorder†	46.5 (133)
Alcohol use
Unhealthy alcohol use║	30.8 (88)
Risky consumption amounts‡	28.7 (82)
Without alcohol related problems or current disorder†§	6.3 (18)
With alcohol related problem, but no disorder†§	12.9 (37)
Problem use or a disorder†§	24.5 (70)
Current (12 month) alcohol use disorder†	11.5 (33)
Current alcohol abuse†	2.8 (8)
Current alcohol dependence †	8.7 (25)
Any lifetime alcohol use disorder (abuse or dependence)†	44.1 (126)
Any lifetime alcohol problems	50.0 (143)

*As part of the CIDI interview subjects are asked about their use, during the past 12 months, of illicit drugs or of prescription drugs for non-medical reasons

†Lifetime and current alcohol and drug use disorders as determined by responses to the CIDI

‡For men, an average > 14 drinks per week over the past 30 days, or > 4 drinks on any one day during the past 30 days (for women, >7 drinks per week, or >3 drinks per occasion)

§Subjects were considered to have alcohol related problems if they consumed risky amounts of alcohol and responded positively to any of the 15 Short Inventory of Problems (SIP) questions

║Hazardous consumption amounts, problem use, or current disorder. Some subjects reported a current (past year) disorder but not (past month) hazardous consumption amounts

¶A positive response to any of the questions from the SIP questionnaire

Test Characteristics The single-question screen was 81.8% sensitive (95% CI 72.5% to 88.5%) and 79.3% specific (95% CI 73.1% to 84.4%) for the detection of unhealthy alcohol use (Table 2). It was slightly more sensitive (87.9%, 95% CI 72.7% to 95.2%) and was less specific (66.8%, 95% CI 60.8% to72.3%) for the detection of a current alcohol use disorder. The longer AUDIT-C screen was slightly less sensitive (73.9%, 95% CI 63.8% to 81.9%) for the detection of unhealthy alcohol use and slightly more specific (82.8%, 95% CI 77.0% to 87.4%) for the detection of an alcohol use disorder, but overall its test characteristics were similar to those of the single-question screen. Subject gender, ethnicity, education and primary language affected point estimates of the sensitivity and specificity of the single item screen very little, though some groups had small sample sizes and so larger differences could not be excluded (Table 3). In a sensitivity analysis, disregarding the pre-screening question response resulted in slightly increased sensitivity (87.5%, 95% CI 79.0% to 92.9%), and decreased specificity (73.7%, 95% CI 67.2% to 79.4%) for the detection of unhealthy alcohol use, when compared to analysis of responses to the single item screening question only when the patient reported sometimes using alcohol.

Table 2

Sensitivity, Specificity and Likelihood Ratios for the Detection of Unhealthy Alcohol Use: Single Screening Question and AUDIT-C (n = 286)

For detection of:	Sensitivity (95% CI)		Specificity (95% CI)
	Single Question	AUDIT-C	Single Question	AUDIT-C
Risky consumption amounts	84% (75%, 91%)	74% (64%, 83%)	78% (72%, 84%)	81% (76%, 86%)
Alcohol related problems or disorder	84% (74%, 91%)	80% (69%, 88%)	75% (69%, 80%)	80% (74%, 85%)
Current alcohol use disorder	88% (73%, 95%)	88% (73%, 95%)	67% (61%, 72%)	72% (67%, 78%)
Unhealthy alcohol use (risky amounts or disorder)	82% (73%, 89%)	73.9% (64%, 82%)	79% (73%, 84%)	83% (77%, 87%)
For detection of:	Positive LR (95% CI)		Negative LR (95% CI)
	Single Question	AUDIT-C	Single Question	AUDIT-C
Risky consumption amounts	3.9 (3.0, 5.2)	4.0 (2.9, 5.5)	0.2 (0.1, 0.3)	0.3 (0.2, 0.4)
Alcohol related problems or disorder	3.4 (2.6, 4.3)	4.0 (3.0, 5.4)	0.2 (0.1, 0.4)	0.3 (0.2, 0.4)
Current alcohol use disorder	2.6 (2.1, 3.3)	3.2 (2.5, 4.0)	0.2 (0.1, 0.4)	0.2 (0.1, 0.4)
Unhealthy alcohol use (risky amounts or disorder)	4.0 (3.0, 5.3)	4.3 (3.1, 6.0)	0.2 (0.1, 0.4)	0.3 (0.2, 0.4)

Table 3

Single question Screen for the Detection of Unhealthy Alcohol Use, in Selected Subgroups

	n	Sensitivity (95% CI)	Specificity (95% CI)
Male	131	82.5% (70.6%, 90.2%)	71.6% (60.5%, 80.6%)
Female	155	80.6% (63.7%, 90.8%)	83.9% (76.4%, 89.3%)
Non-Hispanic white	45	78.6% (52.4%, 92.4%)	87.1% (71.2%, 94.9%)
Non-Hispanic black	176	79.0% (66.7%, 87.5%)	79.0% (70.8%, 85.4%)
Hispanic	46	93.3% (70.2%, 98.8%)	71.0% (53.4%, 83.9%)
English primary language	223	80.0% (69.6%, 87.5%)	77.7% (70.3%, 83.7%)
English not primary language	63	92.3% (66.7%, 98.6%)	84.0% (71.5%, 91.7%)
High school graduate	205	78.7% (66.9%, 87.1%)	79.9% (72.6%, 85.6%)
Not high school graduate	81	88.9% (71.9%, 96.2%)	77.8% (65.1%, 86.8%)

CONCLUSION

A single-question screen was sensitive and specific for the detection of unhealthy alcohol use in a sample of primary care patients. Its test characteristics were similar to those of a longer screening tool in this sample, as well as in numerous studies reported in the literature13,20–23.

Unhealthy alcohol use is prevalent in primary care, and brief intervention in this setting effectively reduces consumption among those without dependence, and improves patient outcomes1,2. Lack of detection of unhealthy alcohol use, however, stands as a barrier to such treatment6. Time constraints in the primary care setting have been cited as a reason for non-adherence to screening and prevention guidelines in general, and for the under-diagnosis of unhealthy alcohol use specifically (according to one estimate, providing all recommended preventive services to an average primary care panel would require 7.4 hours out of each work day)24. Among the best validated options for alcohol screening in primary care settings are the CAGE questionnaire, the AUDIT, and the MAST25–27. More recently, and therefore with fewer validation studies in general care settings, researchers have tested instruments as short as single items and as long as 80 items requiring scoring algorithms and keys for interpretation10,28. One widely known brief screening tool, the CAGE questionnaire, while accurately identifying more severe unhealthy alcohol use (i.e. dependence), was not developed to detect risky consumption amounts or alcohol problems that are more amenable to brief interventions in primary care29. The MAST similarly identifies alcohol dependence and is less well validated for detecting risky use and at 25 items (or 10 items for a briefer version) does not present advantages in length25. The 10-item AUDIT, although well-validated for detecting risky drinking, is less well known or used by primary care physicians, likely in part because it requires scoring and it is not easily memorized for incorporation into the medical interview. The AUDIT and ASSIST may have promise as electronic record systems with decision support become more widespread (and as evidence for the validity of the ASSIST accumulates). The ASSIST has one other major limitation — it does not directly identify risky consumption amounts. The single-question screen proposed by Williams et al. is not identical to that recommended by the NIAAA but it too has proven to be accurate for identifying unhealthy alcohol use among emergency department patients, in primary care, and among respondents to a household survey9,10,12. In summary, in terms of brevity, ease of scoring, and validity for detecting the conditions of interest in primary care, and therefore, likely greater ease for widespread implementation as recommended by practice guidelines, the single item recommended by NIAAA appears to have favorable characteristics.

The results we report are similar to those from studies using different populations and different formulations of the single-question alcohol screen. This study adds to existing literature by validating the version recommended by the NIAAA in a sample of primary care patients — one of the main populations in which it was intended to be used. This version of the single-question screen was derived from a national household survey on alcohol use, the results of which were reported by Dawson, et al.15. While they did not report test characteristics, and although the subjects were not primary care patients, analysis of their published results yields a sensitivity of 89.8% and a specificity of 68.3% for the detection of a current alcohol use disorder, results which were very close to those reported in the current study. In addition to being recommended for widespread use by a health authority, the question phrasing normalizes drinking of large amounts likely increasing honesty in replies, and it directly queries amounts that are defined as risky by national guidelines. The similar single-question screen proposed by Williams et al. that used different cut-off values (‘When was the last time you had more than X drinks in 1 day,’ with X = 4 for women and 5 for men, and a response of less than 3 months ago considered a positive screen) yielded sensitivities of between 80% and 85% and specificities of between 70% and 77% for the detection of unhealthy alcohol use, and was validated in a sample of primary care patients by Seale, et al.10,12. A third formulation of the single-question screen, using the third question of the AUDIT and its multiple response options (‘How often in the last year have you had 6 or more drinks on one occasion’ with a response other than ‘never’ considered a positive screen), had a sensitivity of 77% and a specificity of 83% for the detection of unhealthy alcohol use in a sample of male veteran primary care patients, though the sensitivity was lower in a separate study of female veterans (both findings confirmed in subsequent studies of non-veterans)11,13,20,21. These comparisons suggest that using slightly different cut-offs or changing the phrasing of the question affects the test characteristics to only a small degree.

In order for a screening test for unhealthy alcohol use to be useful, it must be applicable to the broad range of people seen in primary care. The diversity of our subject sample allowed us to examine the effect of gender, ethnicity, primary language and education on the accuracy of the single-question screen. While variations were seen in the sensitivity and specificity of the test across these groups, the differences were small. The single-question screen performed well in an urban, predominately minority population, a population different from those in which single-question screens had been tested previously. This, taken together with the results of the other studies, conducted in a number of different settings, of the other single-question screens that similarly ask about heavy drinking, lends strong support to their use.

Our study has several limitations. Almost half of the patients approached in the primary care waiting room refused to be screened for eligibility in the study, and approximately one fourth of eligible subjects did not complete the study. A lack of information about those who did not participate raises the possibility that those studied were not representative of primary care patients, potentially limiting the generalizability of our results. A higher than expected proportion of subjects reported substance use disorders, likely reflecting the fact that they were recruited from an urban safety-net hospital located in a community where the prevalence of such problems is high, but potentially also reflecting selection bias. The evaluation of a test in an atypical population can result in spectrum bias if, for instance, the unusual severity of the condition renders it more or less easily detectable. While the very close approximation of our results to those of this question and similar questions in other settings suggests that such bias, if present, is small, further study of the question’s test characteristics in a more affluent, lower-risk population may be justified. A limitation of the NIAAA recommended question, and, as far as we know, of the other single-question screens, is that they have not yet been validated languages other than English. This represents another potential future area of study. Subjects were also assured anonymity, a condition which improves the accuracy of the reference standard interview but which may also serve to over-estimate the accuracy of the screening test itself. This is consistent, however, with the methodology of most other alcohol screening test studies.

The single-question screen accurately identified subjects with unhealthy alcohol use. Some patients who screen positive will have severe alcohol use disorders requiring referral to substance abuse treatment, while those who consume excessive amounts of alcohol but have not experienced severe health or interpersonal problems would benefit from brief intervention by the primary care provider. The lack of an efficient way to distinguish these two groups (the NIAAA Clinician’s Guide recommends following up a positive screening test with 13 questions about drinking amounts and alcohol problems), is a challenge that must be addressed when implementing screening for unhealthy alcohol use. The AUDIT and ASSIST, in providing scores, provide a measure of severity. Even though they may be too long for universal screening in many settings, they might be done as brief assessments after a single-item screening question is answered in the affirmative. But this approach has not been tested or validated. Vinson et al. found that two follow-up questions (about drinking in hazardous situations and drinking more or for longer than intended) could identify alcohol use disorders among those with a positive response to a single-question screen30. This approach, if validated, might represent a more efficient solution than applying a longer test to all patients.

The single-question screen recommended by the NIAAA accurately identified unhealthy alcohol use in this sample of primary care patients. The sensitivity and specificity of this single question was comparable to that reported for longer instruments in other studies. These findings of validity support the use of this brief screen in primary care as recommended by NIAAA, which should, in turn, help with the implementation of universal screening for unhealthy alcohol use as recommended by national practice guidelines.