Convergence between the Penn Alcohol Craving Scale and diagnostic interview for the assessment of alcohol craving.

INTRODUCTION: The Penn Alcohol Craving Scale (PACS) is one of the most widely used instruments to measure craving for alcohol. Recent research has suggested that scores on the PACS can be used as a "stand in" for the diagnostic criterion of alcohol craving with a proposed cutoff of >20 on the PACS indicating a "positive" alcohol craving symptom. The present study examined the convergence between the PACS and face-to-face diagnostic interview for the assessment of alcohol craving.
METHOD: A sample of non-treatment seeking heavy drinkers (N = 338) enrolled in experimental studies of AUD completed the PACS as well as a face-to-face diagnostic interview for AUD, which included the craving item from the Semi-Structured Assessment for the Genetics of Alcoholism (SSAGA).
RESULTS: Using the PACS cut-off score of >20, 12.9% (N = 43) of the sample met criteria for alcohol craving compared to 21% (N = 74) of the sample meeting criteria based on the diagnostic interview. Using the PACS cutoff of >20, sensitivity (i.e., true positive rate) was 41% and specificity (i.e., true negative rate) was 95%. Exploratory analyses suggested that a cut-off score of ≥15 achieved the optimal balance of sensitivity (67%) and specificity (81%) in our sample.
CONCLUSIONS: Advancing the assessment of alcohol craving and the conversion from DSM-IV to DSM-5 diagnostic criteria represents an important research direction. The present study recommends that a PACS score cut off of ≥15 should be used as an indicator of clinically significant alcohol craving in community samples of non-treatment seekers.

Introduction

The symptom of craving has been re-introduced to DSM-5 as one of the 11 criteria for alcohol use disorder (AUD). This revision to the diagnostic system is supported by a host of studies implicating craving in the phenomenology of AUD, including documenting the experience of craving in individuals seeking treatment for AUD (Oslin et al., 2009), association of craving with severity of AUD (MacKillop et al., 2010), craving as a predictor of relapse to drinking (Higley et al., 2011), and human laboratory investigations of craving during experimental provocation (Ray, 2011; Sinha, 2009). In light of the adoption of craving as a symptom of AUD, questions about the optimal assessment of craving in clinical populations have been raised, particularly given the lack of consensus on the development and maintenance of craving (Rosenberg, 2009).

Craving is inherently a subjective experience, marked by the strong desire to consume a given substance. As reviewed in detail elsewhere (Ray et al., 2013), there are a host of instruments psychometrically validated to measure craving for alcohol. Importantly, the assessment of craving is often divided into tonic and phasic states, with tonic describing a more stable and unproved experience of craving while phasic is marked by provocation and short duration of assessment. One of the most widely used assessments of tonic alcohol craving is the Penn Alcohol Craving Scale (PACS) which has demonstrated reliability and validity (Flannery et al., 2003; Flannery, Volpicelli, & Pettinati, 1999). As noted by Murphy et al. (2014), using a psychometrically validated measure of tonic craving such as the PACS can advance the diagnosis of alcohol craving. To that end, Murphy et al. (2014) proposed a cutoff of >20 on the PACS to ascribe a positive “symptom” of alcohol craving. In a sample of treatment-seekers with AUD, this cutoff resulted in 47% of participants meeting criteria for the craving symptom (Murphy et al., 2014) whereas in our recent study of non-treatment seekers with alcohol problems only 16% met for craving based on the same cutoff (Hartwell & Ray, 2017). Thus, as a new symptom introduced to DSM-5, the optimal assessment of craving represents a diagnostic challenge to which the PACS may provide a useful heuristic framework (Murphy et al., 2014). In other words, the PACS may be used to capture alcohol craving for diagnostic purposes and to allow a translation from DSM-IV to DSM-5 diagnostic systems. Though a wide variety of measures, such as the Obsessive Compulsive Drinking Scale (Anton, Moak, & Latham, 1995), are also designed to capture the experience of tonic alcohol craving, the PACS benefits from assessing craving as a multi-dimensional experience in a brief, comprehendible, easily scored way that produces a composite score of the characteristics of craving, rather than being rooted in aversive or incentive experiences (Flannery et al., 1999).

In order to advance the application of the PACS as an instrument to capture the diagnostic symptom of craving, the present study provides a direct comparison between craving symptom derived by the proposed PACS cutoff of >20 (Hartwell & Ray, 2017; Murphy et al., 2014) and the craving symptom obtained from a face-to-face diagnostic interview in large sample of community-based non-treatment seeking problem drinkers that completed the PACS as well as a structured interview for DSM-5 AUD. The diagnostic assessment of alcohol craving consisted of the following item: “In situations where you couldn't drink, did you ever have such a strong desire for it that you could not think of anything else?” This single item was obtained from the Semi-Structured Assessment for the Genetics of Alcoholism (SSAGA), which included craving as an ICD-10 symptom (Bucholz et al., 1994). Specifically, this study addresses the sensitivity and specificity of the PACS cutoff of >20 in relation to the assessment of alcohol craving using standard diagnostic interview methods among non-treatment seekers.

Method

Participants

Non-treatment seeking, heavy alcohol users (N = 338) were recruited from the greater Los Angeles community for participation in three human laboratory studies of AUD conducted at the University of California Los Angeles (UCLA). Two studies tested novel pharmacological interventions for AUD, specifically ibudilast (Ray et al., 2017a) and ivermectin (Roche et al., 2016) and a third study tested the relationship between subjective responses to alcohol and progressive ratio alcohol self-administration (Bujarski et al., 2018). All studies were approved by the UCLA Institutional Review Board.

Participants were recruited via online and print advertising. After completion of an initial telephone interview to determine eligibility, individuals were invited to an in-person interview during which all measures used in this study were completed. All studies recruited a community sample of non-treatment seeking heavy drinkers (defined as self-reported consumption of >48 drinks/month for psychopharmacology studies, defined as outside the NIAAA guidelines (U.S. Department of Health and Human Services, 2009) for the third study). All participants were required to be 21 years of age or older. Exclusion criteria were: (1) in or seeking alcohol treatment, (2) current, self-reported use of psychoactive substances other than marijuana, (3) diagnosis of lifetime psychotic or bipolar disorders, (4) pregnancy or refusal of birth control for women, (5) lack of English fluency.

Measures

All participants completed a demographic questionnaire querying age, race, and sex. A master's level clinician, supervised by a licensed clinical psychologist, administered the Structured Clinical Interview for DSM (SCID) to assess for current alcohol use disorder according to DSM-5 criteria; the SCID for DSM-5 was not published at the time of data collection, thus, the SSAGA item substituted for the craving criterion question. The Timeline Follow-Back (TLFB; Sobell et al., 1988) assessed quantity and frequency of alcohol consumption during the past 30 days. Participants completed the Penn Alcohol Craving Scale (PACS; Flannery et al., 1999), a 5-item scale scored on a Likert Scale form 0–6. The PACS benefits from specifically querying frequency, duration, and difficulty resisting craving over the previous week. To further assess and characterize alcohol use, participants also completed the Alcohol Use Disorder Identification Test (AUDIT; Saunders et al., 1993) and the Alcohol Dependence Scale (ADS; Skinner & Horn, 1984).

Data analysis

Consistent with the emergent literature, PACS scores were tabulated and participants placed into correct craving diagnosis based on the previously determined cut-offs, such that PACS total score >20 were considered to meet the threshold of craving on the diagnostic interview, PACS 15–20 were considered subthreshold, and <15 were considered non-cravers (Hartwell & Ray, 2017; Murphy et al., 2014). Sensitivity, the assessment of the test's ability to correctly identify true positives, and specificity, the ability to capture true negatives, were calculated using the previously established PACS cut-offs (Hartwell & Ray, 2017; Murphy et al., 2014) and the diagnostic interview craving question. Subsequently, in exploratory analyses we tested which cutoff might be optimal through computing sensitivity and specificity for each possible PACS score. This was performed by plotting a Receiver Operating Characteristic (ROC) curve and determining the optimal cutoff as defined by the greatest average sensitivity and specificity within the ROC curve.

Results

Participants were predominantly male (N = 239, 68.3%) and had a mean age of 32.9 (SD = 9.8). They reported drinking an average of 18.5 (SD = 6.8) days of the past month and having 6.5 (SD = 4.0) drinks per drinking day. Mean AUDIT score was 17.0 (SD = 7.8), indicating that participants are experiencing significant alcohol related problems and are likely to meet criteria for alcohol use disorder (i.e., AUDIT > 15 has been associated with significant likelihood of DSM-IV alcohol dependence; (Johnson et al., 2013). The ADS scores were on average 13.3 (SD = 7.7), which is indicative of significant alcohol related symptomatology. Of the possible ten DSM-5 symptoms (other than alcohol craving), participants met for an average of 3.1 (SD = 2.5) symptoms and 65.6% of participants met DSM-5 criteria for an AUD.

Seventy-four (21%) participants met criteria for alcohol craving based on the diagnostic interview. Means PACS score was 11.1 (SD = 7.2; full distribution of PACS scores shown in Fig. 1). Using the recommended PACS cut-off score of >20, 43 participants (12.9%) would meet current craving criteria, with an additional 55 participants meeting for sub-threshold craving (i.e. scores of 15–20). Using the PACS cutoff of 20, sensitivity was 41% and specificity was 95%. Based on this relatively poor sensitivity, we conducted exploratory analyses to determine the optimal cutoff as defined by the greatest average sensitivity and specificity by examining the Receiver Operating Characteristic (ROC) curve. Using this approach, we found that a cut-off score of ≥15 achieved the optimal balance of sensitivity (67%) and specificity (81%; Fig. 2). The corresponding positive and negative predictive power values are 49% and 90%, respectively.

Fig. 1

Histogram of PACS scores in this sample.

Fig. 2

Receiver Operating Characteristic (ROC) curve for PACS scores in relation to diagnostic interview ratings of alcohol craving.

Discussion

The goal of the present study was to examine the use of the PACS as an assessment of alcohol craving in relation to diagnostic interview data. Implementing the recommended PACS cut-off score of >20, 12.9% of participants in this study met criteria for alcohol craving. However, based on clinical interview data which utilized the craving assessment question from the SSAGA, 21% of the participants endorsed experiencing alcohol craving in the past 3 months. The discrepancy in prevalence of craving between the PACS-based assessment and the diagnostic interview format may be due to how each was assessed, with the first being a self-report and the latter being a face-to-face interview. Differences may also be due to the differential in time frame assessed. The PACS queries past week experiences of craving whereas the diagnostic interview sought to capture craving over the previous three months. Further, the cut-off score of 20 was initially proposed in a treatment seeking sample (Murphy et al., 2014), which is likely to exhibit higher levels of subjective craving for alcohol compared to their non-treatment seekers, insofar as treatment seeking samples are in fact more severe in their clinical presentation than non-treatment seeking samples (Hartwell & Ray, 2017; Ray et al., 2017b; Rohn et al., 2017). Thus, it is possible, if not likely, that a threshold established in a treatment seeking context would be too stringent when generalized to a non-treatment seeking setting.

To test whether the previously proposed cutoff was appropriate in a non-treatment seeking context, we then investigated the sensitivity and specificity of the PACS in relation to the diagnostic interview criterion, which was used as a gold-standard for diagnostic assessment. Using the cut-off proposed by Murphy et al. (2014), sensitivity and specificity were 41% and 95%, respectively. While the calculated specificity was very high, sensitivity at this cut off level was quite poor which was consistent with the notion that a cutoff derived in a treatment context may be too stringent when generalized. Thus, we investigated alternative cut offs that may provide an optimal balance between sensitivity and specificity in our sample. In our data, the cut-off score of ≥15, optimally balanced sensitivity and specificity with a sensitivity of 67% and specificity of 81%. Sensitivity and specificity values >80% are ideal, however, those above 50% are considered acceptable (Parikh et al., 2008). While the positive predictive value was moderate, the negative predictive power was high. Therefore, the PACS cut off of 15 or higher is hereby recommended for capturing the symptom of alcohol craving in non-treatment seeking samples and achieving a balance between sensitivity and specificity. Future work should examine this cut-off in larger populations that includes treatment seekers.

In sum, the present study sought to advance the assessment of craving for alcohol as a diagnostic criterion for AUD by examining the performance of a PACS-based assessment in relation to a face-to-face diagnostic interview. Strengths of this study include the diverse sample of heavy alcohol users recruited from the community. This type of sample is representative of those typically recruited for participation in experimental AUD research. Further, this study utilized reliable and valid measures to characterize alcohol use and alcohol craving. Limitations include the moderate sample size for this type of analysis. Furthermore, these data were collected prior to the publication of the SCID for DSM-5, therefore necessitating the inclusion of the SSAGA item to assess for craving. However, the wording of the craving item on the newly released SCID is rather comparable (First et al., 2015), “Did you have a strong desire or urge to drink in between those times when you were using? (Was there a time when you had such strong urges to use that you had trouble thinking about anything else?)” As craving represents a diagnostic symptom of AUD with translational properties (i.e., can be studied under a range of methodologies and samples), better understanding of the assessment of craving is necessary. As such, the present study recommends that a PACS score cut off of ≥15 be used as an indicator of clinically significant alcohol craving in community samples of non-treatment seekers in order to achieve optimal sensitivity and specificity in relation to the gold-standard face-to-face diagnostic assessment.

Contributors

Dr. Ray wrote the protocol. Drs. Hartwell & Ray designed the study. Dr. Hartwell conducted literature searches and wrote the first draft. Drs. Hartwell & Bujarski performed the statistical analyses. Ms. Green assisted with manuscript preparation. All authors have approved the final version of this manuscript.

Declaration of Competing Interest

Dr. Ray is a consultant for GSK. We have no other conflicts of interest to declare.