User login
STUDY DESIGN: Cross-sectional survey.
POPULATION: Patients waiting for care at 12 primary care sites in western Pennsylvania from October 1995 to December 1997.
OUTCOMES MEASURED: Sensitivity, specificity, likelihood ratios, and predictive values for the AUDIT, AUDIT-C, and AUDIT-3.
RESULTS: A total of 13,438 patients were surveyed. Compared with a quantity-frequency definition of hazardous drinking (Ž16 drinks/week for men and Ž12 drinks/week for women), the AUDIT, AUDIT-C, and AUDIT-3 had areas under the receiver-operating characteristic curves (AUROC) of 0.940, 0.949, and 0.871, respectively. The AUROCs of the AUDIT and AUDIT-C were significantly different (P=.004). The AUROCs of the AUDIT-C (P <.001) and AUDIT (P <.001) were significantly larger than the AUDIT-3. When compared with a positive AUDIT score of 8 or higher, the AUDIT-C (score Ž3) and the AUDIT-3 (score Ž1) were 94.9% and 99.6% sensitive and 68.8% and 51.1% specific in detecting individuals as hazardous drinkers.
CONCLUSIONS: In a large primary care sample, a 3-question version of the AUDIT identified hazardous drinkers as well as the full AUDIT when such drinkers were defined by quantity-frequency criterion. This version of the AUDIT may be useful as an initial screen for assessing hazardous drinking behavior.
Hazardous drinkers consume enough alcohol to be at risk for adverse consequences but do not meet criteria for alcohol abuse or dependence. They are, however, are at risk for more harmful alcohol abuse.1-5 Such drinking behavior has been defined by quantity and frequency criteria.6 It is estimated that up to 20% of primary care patients are at least hazardous drinkers.7-9 Effective interventions to reduce alcohol consumption exist in primary care settings, so it is important for care providers to reliably and efficiently identify patients who are hazardous drinkers.1,10,11 Traditionally,12-14 care providers are poor at identifying such drinkers, and as many as 72% escape their detection.15-17 This ineffectiveness may be because of a lack of brief and simple questions that aid in patient identification.18-20
Formal screening instruments have been promoted to aid in identification of patients with alcohol problems. The Alcohol Use Disorders Identification Test (AUDIT) was developed by the World Health Organization (WHO) and consists of 2 distinct instruments: a 10-item AUDIT core questionnaire and a clinical screening procedure.1,9,21 The AUDIT core questions can detect hazardous drinkers and have been used alone as a screening instrument.22 The AUDIT questions address intake, dependence, and adverse consequences of drinking,23 emphasize drinking in the past year,5,24 and are indifferent to sex or ethnicity.4,25 It is most useful at detecting drinkers who do not meet criteria for alcohol abuse or dependence.26 Because of its ability to detect less severe alcohol drinkers, the AUDIT seems to have practical value in primary care settings.4,15,21,26
Because of trends toward shorter patient visits, the 10-question AUDIT may be too lengthy to be clinically useful in primary care settings.5,27,28 The shorter CAGE questionnaire, therefore, is often recommended for use in limited time situations.18,20 However, although the CAGE is a valuable tool for identifying alcohol abuse and dependence, it is not as useful for identifying less serious behaviors, such as hazardous drinking.5,28-32
A shorter version of the AUDIT may prove beneficial for use by the busy physician for identifying hazardous drinking behavior. The AUDIT-C (consisting of the first 3 questions of the AUDIT) was shown to be as effective as the full AUDIT in detecting hazardous drinking in a population of veterans.33 Also, the AUDIT-3 (the third question of the AUDIT) may be effective for identifying hazardous drinkers.5,34
We investigated the performance of the AUDIT, AUDIT-C, and AUDIT-3 in detecting such drinkers in a large primary care sample. We also compared the AUDIT-C and the AUDIT-3 to the full AUDIT. We hypothesized that the abbreviated instruments would be comparable with the AUDIT for detecting hazardous drinkers as defined by a quantity-frequency standard.
Methods
Design
We based our study on screening data obtained as part of a large randomized clinical trial of brief interventions for hazardous drinkers (the Early Lifestyle Modification [ELM] Study). Screening forms were administered at 12 primary care sites in the western Pennsylvania area from October 1995 to December 1997. The institutional review board at the University of Pittsburgh and equivalent review groups from each primary care setting approved the ELM study and screening protocol.
Setting
The 12 primary care sites included a Veterans Affairs Medical Center internal medicine clinic, a university-based internal medicine clinic, 2 university-affiliated community care clinics, 3 health maintenance organization clinics, 3 university-affiliated family medicine clinics, and 2 private practice family medicine clinics. All clinics were staffed by physicians. The Veterans Administration and university-based clinics had internal medicine residents participating in patient care. Also, physician assistants and nurse practitioners were involved with primary care at some sites.
Patients
Patients were eligible for the study if they were in the waiting rooms of one of the clinic sites during the screening period, were approached by a research assistant in the waiting room, and agreed to answer the screening questionnaire. Screening of eligible patients occurred from October 1995 to December 1997.
Screening Instruments
In the primary care clinics, patients self-administered an 8-page survey consisting of questions about lifestyle habits. Research assistants approached as many patients in the waiting rooms of the primary care sites as possible. The survey included questions about stress management (8 questions), smoking habits (8), the AUDIT (10), and quantity-frequency questions (3). In initial surveys, just before the alcohol-related questions patients were instructed not to answer alcohol questions if they responded “never” to the following question: “How often do you have a drink containing alcohol (for example: beer, wine, wine coolers, sherry, gin, vodka, or other hard liquor)?” This question was removed in later surveys, because it limited the number of people responding to the alcohol instruments under comparison. We also asked for categorical responses to questions about age, sex, education background, race, marital status, and occupation. We did not compensate patients for completing this questionnaire, and their responses were anonymous.
Alcohol Screening Instruments
The AUDIT consists of 10 questions Figure 1. The AUDIT-C includes the following first 3 questions of the AUDIT: How often do you have a drink containing alcohol? How many drinks containing alcohol do you have on a typical day when you are drinking? How often do you have 6 or more drinks on one occasion? The AUDIT-3 is the third question alone. Each individual question is scored from 0 to 4 points on a Likert scale, with higher numbers indicating more severe drinking behavior. For AUDIT questions with only 3 possible answers (questions 9 and 10), the scores were 0, 2, and 4. Thus, the range of possible scores on the AUDIT are from 0 to 40, the AUDIT-C from 0 to 12, and the AUDIT-3 from 0 to 4.
Our quantity-frequency questionnaire consisted of the questions: “If you drink, how many days per week do you have a drink?” (answers: 1 through 7); “If you drink less than once a week, how many days per month do you drink?” (answers: 1 or less, 2, 3, or 4 or more); and “How many drinks containing alcohol do you have on a typical day when you are drinking?” (answers: 1 through 10 or more). The number of average drinks per week was computed for analysis, and if the answer was “or more” we used the maximum number indicated in the question (4 or 10).
Outcome Measures and Analysis
The main outcome measures concerned the accuracy of the full AUDIT, AUDITC, and AUDIT-3. For comparative purposes, patients with missing responses were not used in subsequent statistical analyses.
The AUDIT, AUDIT-C, and AUDIT-3 were compared with a hazardous drinking criterion defined by the quantity-frequency questions. This criterion was 16 or more drinks per week for men and 12 or more drinks per week for women.6 Clinically, quantity-frequency assessment is often used to determine hazardous drinking behavior.16 The area under the receiver-operating characteristic curve (AUROC) was used to compare each instrument’s diagnostic ability.35 AUROC is an indication of the ability of a test to discriminate between false positives and false negatives. A score approaching 1 will be more sensitive and specific over a range of cutoff points than a score of 0.5, which is a nondiscriminating test. To calculate and compare the AUROCs we used published standards developed by Hanley and McNeil36,37 for curves derived by same cases. We performed chi-square analysis on comparisons of categorical data.
We then compared the AUDIT-C and AUDIT-3 with a score of the full AUDIT as a criterion for hazardous drinking. An AUDIT result of 8 or higher was accepted as a criterion for hazardous drinking. We measured the sensitivity and specificity of the AUDIT-C, using a cutoff score of 3 or higher and AUDIT-3 cutoff score of 1 or higher compared with the criterion of hazardous drinking defined by the full AUDIT.38
Results
At least 1 question on the screening survey was answered by 13,438 patients. Overall, 13,198 (98%) either answered that they currently drink or never drank alcohol. Not all questions were answered by each individual. Of patients who indicated that they currently drink alcohol, the full AUDIT was completed by 7035 (52%). The AUDIT-C was completed by 7190 (54%) patients, and the AUDIT-3 was answered by 7303 (54%) patients. Both the entire AUDIT and quantity-frequency questions were answered by 6954 (52%) patients. Of the 13,438 patients, 36% indicated no alcohol consumption. The majority of the surveyed sample were men, white, married, employed, high school graduates, and younger than 60 years Table 1.
AUDITs Compared with Quantity-Frequency Criterion
Table 2 compares the likelihood of hazardous alcohol use, defined by a quantity-frequency criterion, associated with each score of the AUDIT, AUDIT-C, and AUDIT-3. It is important to note that at each score the true positives of screening are only persons identified at that score. For example, at an AUDIT score of 7, 53 of 754 hazardous drinkers were identified with the resulting likelihood ratio (3.0) and predictive value (26.9%).
For comparisons of the AUDIT, AUDIT-C, and AUDIT-3 at identifying hazardous drinkers who scored at or greater than a minimal cutoff, the sensitivity and specificity compared with a quantity-frequency criterion is shown in Table 3. For cut-point values of an AUDIT score of 8 or higher, the sensitivity of the AUDIT was 76%. Similarly, the sensitivities of the AUDIT-C (with score Ž3) and AUDIT-3 (with score Ž1) were 99.6% and 89.1%, as sensitive as the quantity-frequency questions in detecting these patients. Specificity of the AUDIT, AUDIT-C, and AUDIT-3 at these cutoff values was 92%, 48%, and 65%, respectively.
AUROCs were constructed from all cut-point values Figure 2. Computation of the AUROC indicates the effectiveness of the instrument to discriminate hazardous drinkers over a range of AUDIT scores. The AUROCs for the AUDIT, AUDIT-C, and AUDIT-3 were significantly more discriminating than the line of identity (AUROC=0.5). The AUROC of the AUDIT was significantly different from the AUDIT-C (z=2.69; P=.004). The AUDIT-3 AUROC was significantly different than the AUDIT (z=10.03; P <.001) and AUDIT-C (z=12.69; P <.001).
Abbreviated AUDITs Compared with Full AUDIT Criterion
The full AUDIT is often used as a standard to assess hazardous drinking. We compared the abbreviated instruments to the full AUDIT, with a positive score of 8 or higher as a criterion for such drinking. The AUDIT-C (score (3) and AUDIT-3 (score Ž1) were 94.9% and 99.9% as sensitive and 68.8% and 51.1% as specific as the full AUDIT in obtaining a positive score. We also determined the performance of the AUDIT-3 when compared with a reference standard of a positive AUDIT-C. The AUDIT-3 (score Ž1) was 69% sensitive and 95% specific as the AUDIT-C (score Ž3) at identifying hazardous drinkers (data not shown).
Discussion
We evaluated the performance of the AUDIT and abbreviated AUDIT instruments to detect hazardous drinking in a large multisite primary care sample. The abbreviated forms of the AUDIT were as effective as the AUDIT at identifying hazardous drinkers. Compared with quantity-frequency questions, the AUDIT and AUDIT-C were superior at identifying hazardous drinkers than the AUDIT-3. The abbreviated forms of the AUDIT were as sensitive as the full AUDIT at detecting hazardous drinkers when using standard cutoff values for hazardous drinking.
As with the 4-item CAGE questionnaire for alcohol dependence, a 1- or 3-item AUDIT instrument may increase care providers’ recognition of hazardous drinkers. Providers do not routinely ask standard alcohol questions, are particularly poor at identifying hazardous drinkers, and do not enter patients into alcohol treatment.16 Therefore, providing clinicians with a few easily remembered questions to determine hazardous drinking behavior would be beneficial.39 A short questionnaire would be simple to administer and applicable in a wide variety of practice settings. A positive response would increase suspicion regarding hazardous or abusive drinking behavior and prompt additional questions about patients’ alcohol use.18,40 For example, care providers could use the AUDIT-3 or AUDIT-C (to detect at least hazardous drinking), then administer the questionnaire (to detect abuse and dependence), if the patient’s response was positive.
It is important to realize that the AUDIT and its abbreviated forms are only sensitive to detect hazardous drinking, not to specifically assign patients’ drinking habits as hazardous only. The AUDIT was originally designed to distinguish a person with hazardous drinking from one with nonhazardous drinking.26 As such, this instrument may not be specific enough to distinguish hazardous drinkers from others with severe alcohol behaviors; people who score positive may qualify for alcohol abuse and dependence. For less risky alcohol behaviors, it is more important for a health care provider to identify all hazardous drinkers (true positives), at the risk of falsely identifying a person who may not have this behavior (false positives).18 Therefore, when screening to establish a threshold level of treatment intervention, screening instruments should maximize sensitivity, even at the expense of low specificity.
In our sample, the AUDIT (score Ž8), AUDIT-C (score Ž3), and AUDIT-3 (score Ž1) were as sensitive as quantity-frequency questions in detecting hazardous drinking. This increase in sensitivity of the AUDIT-C and AUDIT-3 is likely related to the consumption focus of these questions. The AUDIT-C consists of quantity and frequency type questions, and the third question is specific for quantity of drinking at one session. However, the performance of AUDIT instruments in our study is comparable and confirms results found in a study by Bush and colleagues,34 even though the studies used different criteria for assessment of abbreviated instruments.
If the AUDIT is used as a standard to detect hazardous drinkers, would the AUDIT-C or AUDIT-3 identify the same patients as the full AUDIT? Using cutoff points for the AUDIT-C of 3 or higher and an AUDIT-3 score of 1 or higher, these instruments were 99.7% and 98.3% as sensitive as the full AUDIT. As expected, specificity is much less for both abbreviated instruments. However, the high sensitivity suggests a clinical utility for these abbreviated instruments. It is unlikely that by asking the 3 AUDIT-C questions or the single AUDIT-3 question, a primary care provider will miss identification of a person who is at least a hazardous drinker.
Limitations
There is no gold standard to identify hazardous drinkers.41 The definition of what level of drinking constitutes that label is controversial, and providers do not routinely ask standard questions about drinking behavior.16 Our criterion, the quantity-frequency questions, may be considered a poor standard to compare survey instruments.7,42 In research, quantity-frequency consumption questions are helpful in specific identification of hazardous drinkers when a cutoff value is defined.5 However, patients may prefer not to answer questions about quantity or frequency of alcohol use and may not respond consistently to heterogeneous provider questions. Therefore, quantity-frequency questions may be useful as a standard to compare similar instruments such as the AUDIT and its abbreviations, although they may not be particularly effective in clinical practice.
Not all surveyed individuals completed the full AUDIT instrument. This was primarily because patients who answered “never” to our initial question regarding any alcohol use did not proceed to the AUDIT. Before completion of the study, we eliminated this question from the survey. However, it is not known whether patients who answered “never” to the initial question, were, in fact, drinkers. Consistency response bias may also have occurred as patients may have wished to answer similar items similarly. In addition, the similarity of the AUDIT-C and AUDIT-3 to quantity-frequency questions suggests that our sensitivity analysis is perhaps only the upper bounds of the briefer instruments.
We derived the abbreviated tests directly from the AUDIT, thus no assessment of the instruments out of the context of the full AUDIT was performed. Independent testing of abbreviated AUDIT instruments is needed. Recruitment was conducted by research assistants who solicited and provided forms to patients in the waiting rooms of primary care clinics. This convenience sample may have led to a selection bias in obtaining survey data. Patient recall bias may also have affected survey answers as patients may have had difficulty answering quantity and frequency questions accurately (an advantage of the AUDIT over quantity-frequency type questions). Also, our study investigates identification of individuals who are at least hazardous drinkers, but may also be abusive or dependent. We did not study the instruments’ ability to distinguish between hazardous drinking and abuse or dependence.
Conclusions
Our results confirm that the AUDIT-C and AUDIT-3 are useful screening tests for hazardous drinking. Because treatment of such drinkers can be effective, identifying people with less severe alcohol problems is crucial and an important public health initiative.21 Abbreviated instruments identify hazardous drinkers quickly, efficiently, and effectively, and may encourage early treatment to prevent the occurrence of alcohol-related consequences, abuse, or dependence. We recommend using the AUDIT-C and CAGE as brief screening instruments for hazardous drinking and alcohol abuse and dependence. This approach warrants further investigation.
Acknowledgments
Our work was supported by a grant to Dr Maisto from the National Institute of Alcohol Abuse and Alcoholism (AA10291). Dr Gordon is supported by a faculty development grant in general internal medicine from the VA Pittsburgh Healthcare System and the VISN 4 Mental Illness Research, Education, and Clinical Center. Dr Kraemer is supported by a Mentored Clinical Scientist Development Award from the National Institute of Alcohol Abuse and Alcoholism (AA00235). Dr J. Conigliaro is supported by a Career Development Award from the HSR & D Service, Department of Veterans Affairs (CD-97324-A) and is a generalist physician faculty scholar of the Robert Wood Johnson Foundation (#031500). We thank the ELM research study staff, Monica O’Connor (the ELM project coordinator), and all the patients who participated in the ELM study.
1. A cross-national trial of brief interventions with heavy drinkers: WHO Brief Intervention Study Group. Am J Public Health 1996;86:948-55.
2. Babor TF, de la Fuente JR, Saunders J, Grant M. The Alcohol Use Disorders Identification Test: guidelines for use in primary health care. Geneva, Switzerland: World Health Organization; 1989.
3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th ed. Washington, DC: American Psychiatric Association; 1995.
4. Allen JP, Litten RZ, Fertig JB, Babor T. A review of research on the Alcohol Use Disorders Identification Test (AUDIT). Alcoholism 1997;21:613-19.
5. Bohn MJ, Babor TF, Kranzler HR. The Alcohol Use Disorders Identification Test (AUDIT): validation of a screening instrument for use in medical settings. J Studies Alcohol 1995;56:423-32.
6. Sanchez-Craig M, Wilkinson DA, Davila R. Empirically based guidelines for moderate drinking: 1-year results from three studies with problem drinkers. Am J Public Health 1995;85:823-28.
7. Bradley KA. Screening and diagnosis of alcoholism in the primary care setting. West J Med 1992;156:166-71.
8. Schorling JB, Klas PT, Willems JP, Everett AS. Addressing alcohol use among primary care patients: differences between family medicine and internal medicine residents. J Gen Intern Med 1994;9:248-54.
9. Saunders JB, Aasland OG, Amundsen A, Grant M. Alcohol consumption and related problems among primary health care patients: WHO collaborative project on early detection of persons with harmful alcohol consumption—I. Addiction 1993;88:349-62.
10. Wallace P, Cutler S, Haines A. Randomised controlled trial of general practitioner intervention in patients with excessive alcohol consumption. BMJ 1988;297:663-68.
11. Fleming MF, Barry KL, Manwell LB, Johnson K, London R. Brief physician advice for problem alcohol drinkers: a randomized controlled trial in community-based primary care practices. JAMA 1997;277:1039-45.
12. Kristenson H, Ohlin H, Hulten-Nosslin MB, Trell E, Hood B. Identification and intervention of heavy drinking in middle-aged men: results and follow-up of 24-60 months of long-term study with randomized controls. Alcoholism 1983;7:203-09.
13. Bien TH, Miller WR, Tonigan JS. Brief interventions for alcohol problems: a review. Addiction 1993;88:315-35.
14. Wilk AI, Jensen NM, Havighurst TC. Meta-analysis of randomized control trials addressing brief interventions in heavy alcohol drinkers. J Gen Intern Med 1997;12:274-83.
15. Conigrave KM, Saunders JB, Reznik RB. Predictive capacity of the AUDIT questionnaire for alcohol-related harm. Addiction 1995;90:1479-85.
16. Friedmann PD, McCullough D, Chin MH, Saitz R. Screening and intervention for alcohol problems: a national survey of primary care physicians and psychiatrists. J Gen Intern Med 2000;15:84-91.
17. Bowen OR, Sammons JH. The alcohol-abusing patient: a challenge to the profession. JAMA 1988;260:2267-70.
18. Allen JP, Maisto SA, Connors GJ. Self-report screening tests for alcohol problems in primary care. Arch Intern Med 1995;155:1726-30.
19. Saunders JB, Conigrave KM. Early identification of alcohol problems. CMAJ 1990;143:1060-69.
20. Mayfield D, McLeod G, Hall P. The CAGE questionnaire: validation of a new alcoholism screening instrument. Am J Psychiatry 1974;131:1121-23.
21. Barry KL, Fleming MF. The Alcohol Use Disorders Identification Test (AUDIT) and the SMAST-13: predictive validity in a rural primary care sample. Alcohol Alcoholism 1993;28:33-42.
22. Schmidt A, Barry KL, Fleming MF. Detection of problem drinkers: the Alcohol Use Disorders Identification Test (AUDIT). South Med J 1995;88:52-59.
23. Volk RJ, Steinbauer JR, Cantor SB, Holzer CE, III. The Alcohol Use Disorders Identification Test (AUDIT) as a screen for at-risk drinking in primary care patients of different racial/ethnic backgrounds. Addiction 1997;92:197-206.
24. Kettl PA. Detecting problem drinkers in your practice. Patient Care 1997;30:27-41.
25. Steinbauer JR, Cantor SB, Holzer CE, III, Volk RJ. Ethnic and sex bias in primary care screening tests for alcohol use disorders. Ann Intern Med 1998;129:353-62.
26. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption—II. Addiction 1993;88:791-804.
27. Foster AI, Blondell RD, Looney SW. The practicality of using the SMAST and AUDIT to screen for alcoholism among adolescents in an urban private family practice. J Kentucky Med Assoc 1997;95:105-07.
28. Bradley KA, Bush KR, McDonell MB, Malone T, Fihn SD. Screening for problem drinking: comparison of CAGE and AUDIT: Ambulatory Care Quality Improvement Project (ACQUIP). J Gen Intern Med 1998;13:379-88.
29. Morton JL, Jones TV, Manganaro MA. Performance of alcoholism screening questionnaires in elderly veterans. Am J Med 1996;101:153-59.
30. MacKenzie D, Langa A, Brown TM. Identifying hazardous or harmful alcohol use in medical admissions: a comparison of audit, cage and brief mast. Alcohol Alcoholism 1996;31:591-99.
31. Buchsbaum DG, Buchanan RG, Centor RM, Schnoll SH, Lawton MJ. Screening for alcohol abuse using CAGE scores and likelihood ratios. Ann Intern Med 1991;115:774-77.
32. Seppa K, Makela R, Sillanaukee P. Effectiveness of the Alcohol Use Disorders Identification Test in occupational health screenings. Alcoholism 1995;19:999-1003.
33. Piccinelli M, Tessari E, Bortolomasi M, et al. Efficacy of the alcohol use disorders identification test as a screening tool for hazardous alcohol intake and related disorders in primary care: a validity study. BMJ 1997;314:420-24.
34. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Arch Intern Med 1998;158:1789-95.
35. van Kammen DP, Kelley ME, Gurklis JA, et al. Behavioral vs biochemical prediction of clinical stability following haloperidol withdrawal in schizophrenia. Arch Gen Psychiatry 1995;52:673-78.
36. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver-operating characteristic (ROC) curve. Radiology 1982;143:29-36.
37. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839-43.
38. Conigrave KM, Hall WD, Saunders JB. The AUDIT questionnaire: choosing a cut-off score. Alcohol Use Disorder Identification Test. Addiction 1995;90:1349-56.
39. Cherpitel CJ, Clark WB. Ethnic differences in performance of screening instruments for identifying harmful drinking and alcohol dependence in the emergency room. Alcoholism 1995;19:628-34.
40. Reid MC, Fiellin DA, O’Connor PG. Hazardous and harmful alcohol consumption in primary care. Arch Intern Med 1999;159:1681-89.
41. Fink A, Hays RD, Moore AA, Beck JC. Alcohol-related problems in older persons. Determinants, consequences, and screening. Arch Intern Med 1996;156:1150-56.
42. Bradley KA, Boyd-Wickizer J, Powell SH, Burman ML. Alcohol screening questionnaires in women: a critical review. JAMA 1998;280:166-71.
STUDY DESIGN: Cross-sectional survey.
POPULATION: Patients waiting for care at 12 primary care sites in western Pennsylvania from October 1995 to December 1997.
OUTCOMES MEASURED: Sensitivity, specificity, likelihood ratios, and predictive values for the AUDIT, AUDIT-C, and AUDIT-3.
RESULTS: A total of 13,438 patients were surveyed. Compared with a quantity-frequency definition of hazardous drinking (Ž16 drinks/week for men and Ž12 drinks/week for women), the AUDIT, AUDIT-C, and AUDIT-3 had areas under the receiver-operating characteristic curves (AUROC) of 0.940, 0.949, and 0.871, respectively. The AUROCs of the AUDIT and AUDIT-C were significantly different (P=.004). The AUROCs of the AUDIT-C (P <.001) and AUDIT (P <.001) were significantly larger than the AUDIT-3. When compared with a positive AUDIT score of 8 or higher, the AUDIT-C (score Ž3) and the AUDIT-3 (score Ž1) were 94.9% and 99.6% sensitive and 68.8% and 51.1% specific in detecting individuals as hazardous drinkers.
CONCLUSIONS: In a large primary care sample, a 3-question version of the AUDIT identified hazardous drinkers as well as the full AUDIT when such drinkers were defined by quantity-frequency criterion. This version of the AUDIT may be useful as an initial screen for assessing hazardous drinking behavior.
Hazardous drinkers consume enough alcohol to be at risk for adverse consequences but do not meet criteria for alcohol abuse or dependence. They are, however, are at risk for more harmful alcohol abuse.1-5 Such drinking behavior has been defined by quantity and frequency criteria.6 It is estimated that up to 20% of primary care patients are at least hazardous drinkers.7-9 Effective interventions to reduce alcohol consumption exist in primary care settings, so it is important for care providers to reliably and efficiently identify patients who are hazardous drinkers.1,10,11 Traditionally,12-14 care providers are poor at identifying such drinkers, and as many as 72% escape their detection.15-17 This ineffectiveness may be because of a lack of brief and simple questions that aid in patient identification.18-20
Formal screening instruments have been promoted to aid in identification of patients with alcohol problems. The Alcohol Use Disorders Identification Test (AUDIT) was developed by the World Health Organization (WHO) and consists of 2 distinct instruments: a 10-item AUDIT core questionnaire and a clinical screening procedure.1,9,21 The AUDIT core questions can detect hazardous drinkers and have been used alone as a screening instrument.22 The AUDIT questions address intake, dependence, and adverse consequences of drinking,23 emphasize drinking in the past year,5,24 and are indifferent to sex or ethnicity.4,25 It is most useful at detecting drinkers who do not meet criteria for alcohol abuse or dependence.26 Because of its ability to detect less severe alcohol drinkers, the AUDIT seems to have practical value in primary care settings.4,15,21,26
Because of trends toward shorter patient visits, the 10-question AUDIT may be too lengthy to be clinically useful in primary care settings.5,27,28 The shorter CAGE questionnaire, therefore, is often recommended for use in limited time situations.18,20 However, although the CAGE is a valuable tool for identifying alcohol abuse and dependence, it is not as useful for identifying less serious behaviors, such as hazardous drinking.5,28-32
A shorter version of the AUDIT may prove beneficial for use by the busy physician for identifying hazardous drinking behavior. The AUDIT-C (consisting of the first 3 questions of the AUDIT) was shown to be as effective as the full AUDIT in detecting hazardous drinking in a population of veterans.33 Also, the AUDIT-3 (the third question of the AUDIT) may be effective for identifying hazardous drinkers.5,34
We investigated the performance of the AUDIT, AUDIT-C, and AUDIT-3 in detecting such drinkers in a large primary care sample. We also compared the AUDIT-C and the AUDIT-3 to the full AUDIT. We hypothesized that the abbreviated instruments would be comparable with the AUDIT for detecting hazardous drinkers as defined by a quantity-frequency standard.
Methods
Design
We based our study on screening data obtained as part of a large randomized clinical trial of brief interventions for hazardous drinkers (the Early Lifestyle Modification [ELM] Study). Screening forms were administered at 12 primary care sites in the western Pennsylvania area from October 1995 to December 1997. The institutional review board at the University of Pittsburgh and equivalent review groups from each primary care setting approved the ELM study and screening protocol.
Setting
The 12 primary care sites included a Veterans Affairs Medical Center internal medicine clinic, a university-based internal medicine clinic, 2 university-affiliated community care clinics, 3 health maintenance organization clinics, 3 university-affiliated family medicine clinics, and 2 private practice family medicine clinics. All clinics were staffed by physicians. The Veterans Administration and university-based clinics had internal medicine residents participating in patient care. Also, physician assistants and nurse practitioners were involved with primary care at some sites.
Patients
Patients were eligible for the study if they were in the waiting rooms of one of the clinic sites during the screening period, were approached by a research assistant in the waiting room, and agreed to answer the screening questionnaire. Screening of eligible patients occurred from October 1995 to December 1997.
Screening Instruments
In the primary care clinics, patients self-administered an 8-page survey consisting of questions about lifestyle habits. Research assistants approached as many patients in the waiting rooms of the primary care sites as possible. The survey included questions about stress management (8 questions), smoking habits (8), the AUDIT (10), and quantity-frequency questions (3). In initial surveys, just before the alcohol-related questions patients were instructed not to answer alcohol questions if they responded “never” to the following question: “How often do you have a drink containing alcohol (for example: beer, wine, wine coolers, sherry, gin, vodka, or other hard liquor)?” This question was removed in later surveys, because it limited the number of people responding to the alcohol instruments under comparison. We also asked for categorical responses to questions about age, sex, education background, race, marital status, and occupation. We did not compensate patients for completing this questionnaire, and their responses were anonymous.
Alcohol Screening Instruments
The AUDIT consists of 10 questions Figure 1. The AUDIT-C includes the following first 3 questions of the AUDIT: How often do you have a drink containing alcohol? How many drinks containing alcohol do you have on a typical day when you are drinking? How often do you have 6 or more drinks on one occasion? The AUDIT-3 is the third question alone. Each individual question is scored from 0 to 4 points on a Likert scale, with higher numbers indicating more severe drinking behavior. For AUDIT questions with only 3 possible answers (questions 9 and 10), the scores were 0, 2, and 4. Thus, the range of possible scores on the AUDIT are from 0 to 40, the AUDIT-C from 0 to 12, and the AUDIT-3 from 0 to 4.
Our quantity-frequency questionnaire consisted of the questions: “If you drink, how many days per week do you have a drink?” (answers: 1 through 7); “If you drink less than once a week, how many days per month do you drink?” (answers: 1 or less, 2, 3, or 4 or more); and “How many drinks containing alcohol do you have on a typical day when you are drinking?” (answers: 1 through 10 or more). The number of average drinks per week was computed for analysis, and if the answer was “or more” we used the maximum number indicated in the question (4 or 10).
Outcome Measures and Analysis
The main outcome measures concerned the accuracy of the full AUDIT, AUDITC, and AUDIT-3. For comparative purposes, patients with missing responses were not used in subsequent statistical analyses.
The AUDIT, AUDIT-C, and AUDIT-3 were compared with a hazardous drinking criterion defined by the quantity-frequency questions. This criterion was 16 or more drinks per week for men and 12 or more drinks per week for women.6 Clinically, quantity-frequency assessment is often used to determine hazardous drinking behavior.16 The area under the receiver-operating characteristic curve (AUROC) was used to compare each instrument’s diagnostic ability.35 AUROC is an indication of the ability of a test to discriminate between false positives and false negatives. A score approaching 1 will be more sensitive and specific over a range of cutoff points than a score of 0.5, which is a nondiscriminating test. To calculate and compare the AUROCs we used published standards developed by Hanley and McNeil36,37 for curves derived by same cases. We performed chi-square analysis on comparisons of categorical data.
We then compared the AUDIT-C and AUDIT-3 with a score of the full AUDIT as a criterion for hazardous drinking. An AUDIT result of 8 or higher was accepted as a criterion for hazardous drinking. We measured the sensitivity and specificity of the AUDIT-C, using a cutoff score of 3 or higher and AUDIT-3 cutoff score of 1 or higher compared with the criterion of hazardous drinking defined by the full AUDIT.38
Results
At least 1 question on the screening survey was answered by 13,438 patients. Overall, 13,198 (98%) either answered that they currently drink or never drank alcohol. Not all questions were answered by each individual. Of patients who indicated that they currently drink alcohol, the full AUDIT was completed by 7035 (52%). The AUDIT-C was completed by 7190 (54%) patients, and the AUDIT-3 was answered by 7303 (54%) patients. Both the entire AUDIT and quantity-frequency questions were answered by 6954 (52%) patients. Of the 13,438 patients, 36% indicated no alcohol consumption. The majority of the surveyed sample were men, white, married, employed, high school graduates, and younger than 60 years Table 1.
AUDITs Compared with Quantity-Frequency Criterion
Table 2 compares the likelihood of hazardous alcohol use, defined by a quantity-frequency criterion, associated with each score of the AUDIT, AUDIT-C, and AUDIT-3. It is important to note that at each score the true positives of screening are only persons identified at that score. For example, at an AUDIT score of 7, 53 of 754 hazardous drinkers were identified with the resulting likelihood ratio (3.0) and predictive value (26.9%).
For comparisons of the AUDIT, AUDIT-C, and AUDIT-3 at identifying hazardous drinkers who scored at or greater than a minimal cutoff, the sensitivity and specificity compared with a quantity-frequency criterion is shown in Table 3. For cut-point values of an AUDIT score of 8 or higher, the sensitivity of the AUDIT was 76%. Similarly, the sensitivities of the AUDIT-C (with score Ž3) and AUDIT-3 (with score Ž1) were 99.6% and 89.1%, as sensitive as the quantity-frequency questions in detecting these patients. Specificity of the AUDIT, AUDIT-C, and AUDIT-3 at these cutoff values was 92%, 48%, and 65%, respectively.
AUROCs were constructed from all cut-point values Figure 2. Computation of the AUROC indicates the effectiveness of the instrument to discriminate hazardous drinkers over a range of AUDIT scores. The AUROCs for the AUDIT, AUDIT-C, and AUDIT-3 were significantly more discriminating than the line of identity (AUROC=0.5). The AUROC of the AUDIT was significantly different from the AUDIT-C (z=2.69; P=.004). The AUDIT-3 AUROC was significantly different than the AUDIT (z=10.03; P <.001) and AUDIT-C (z=12.69; P <.001).
Abbreviated AUDITs Compared with Full AUDIT Criterion
The full AUDIT is often used as a standard to assess hazardous drinking. We compared the abbreviated instruments to the full AUDIT, with a positive score of 8 or higher as a criterion for such drinking. The AUDIT-C (score (3) and AUDIT-3 (score Ž1) were 94.9% and 99.9% as sensitive and 68.8% and 51.1% as specific as the full AUDIT in obtaining a positive score. We also determined the performance of the AUDIT-3 when compared with a reference standard of a positive AUDIT-C. The AUDIT-3 (score Ž1) was 69% sensitive and 95% specific as the AUDIT-C (score Ž3) at identifying hazardous drinkers (data not shown).
Discussion
We evaluated the performance of the AUDIT and abbreviated AUDIT instruments to detect hazardous drinking in a large multisite primary care sample. The abbreviated forms of the AUDIT were as effective as the AUDIT at identifying hazardous drinkers. Compared with quantity-frequency questions, the AUDIT and AUDIT-C were superior at identifying hazardous drinkers than the AUDIT-3. The abbreviated forms of the AUDIT were as sensitive as the full AUDIT at detecting hazardous drinkers when using standard cutoff values for hazardous drinking.
As with the 4-item CAGE questionnaire for alcohol dependence, a 1- or 3-item AUDIT instrument may increase care providers’ recognition of hazardous drinkers. Providers do not routinely ask standard alcohol questions, are particularly poor at identifying hazardous drinkers, and do not enter patients into alcohol treatment.16 Therefore, providing clinicians with a few easily remembered questions to determine hazardous drinking behavior would be beneficial.39 A short questionnaire would be simple to administer and applicable in a wide variety of practice settings. A positive response would increase suspicion regarding hazardous or abusive drinking behavior and prompt additional questions about patients’ alcohol use.18,40 For example, care providers could use the AUDIT-3 or AUDIT-C (to detect at least hazardous drinking), then administer the questionnaire (to detect abuse and dependence), if the patient’s response was positive.
It is important to realize that the AUDIT and its abbreviated forms are only sensitive to detect hazardous drinking, not to specifically assign patients’ drinking habits as hazardous only. The AUDIT was originally designed to distinguish a person with hazardous drinking from one with nonhazardous drinking.26 As such, this instrument may not be specific enough to distinguish hazardous drinkers from others with severe alcohol behaviors; people who score positive may qualify for alcohol abuse and dependence. For less risky alcohol behaviors, it is more important for a health care provider to identify all hazardous drinkers (true positives), at the risk of falsely identifying a person who may not have this behavior (false positives).18 Therefore, when screening to establish a threshold level of treatment intervention, screening instruments should maximize sensitivity, even at the expense of low specificity.
In our sample, the AUDIT (score Ž8), AUDIT-C (score Ž3), and AUDIT-3 (score Ž1) were as sensitive as quantity-frequency questions in detecting hazardous drinking. This increase in sensitivity of the AUDIT-C and AUDIT-3 is likely related to the consumption focus of these questions. The AUDIT-C consists of quantity and frequency type questions, and the third question is specific for quantity of drinking at one session. However, the performance of AUDIT instruments in our study is comparable and confirms results found in a study by Bush and colleagues,34 even though the studies used different criteria for assessment of abbreviated instruments.
If the AUDIT is used as a standard to detect hazardous drinkers, would the AUDIT-C or AUDIT-3 identify the same patients as the full AUDIT? Using cutoff points for the AUDIT-C of 3 or higher and an AUDIT-3 score of 1 or higher, these instruments were 99.7% and 98.3% as sensitive as the full AUDIT. As expected, specificity is much less for both abbreviated instruments. However, the high sensitivity suggests a clinical utility for these abbreviated instruments. It is unlikely that by asking the 3 AUDIT-C questions or the single AUDIT-3 question, a primary care provider will miss identification of a person who is at least a hazardous drinker.
Limitations
There is no gold standard to identify hazardous drinkers.41 The definition of what level of drinking constitutes that label is controversial, and providers do not routinely ask standard questions about drinking behavior.16 Our criterion, the quantity-frequency questions, may be considered a poor standard to compare survey instruments.7,42 In research, quantity-frequency consumption questions are helpful in specific identification of hazardous drinkers when a cutoff value is defined.5 However, patients may prefer not to answer questions about quantity or frequency of alcohol use and may not respond consistently to heterogeneous provider questions. Therefore, quantity-frequency questions may be useful as a standard to compare similar instruments such as the AUDIT and its abbreviations, although they may not be particularly effective in clinical practice.
Not all surveyed individuals completed the full AUDIT instrument. This was primarily because patients who answered “never” to our initial question regarding any alcohol use did not proceed to the AUDIT. Before completion of the study, we eliminated this question from the survey. However, it is not known whether patients who answered “never” to the initial question, were, in fact, drinkers. Consistency response bias may also have occurred as patients may have wished to answer similar items similarly. In addition, the similarity of the AUDIT-C and AUDIT-3 to quantity-frequency questions suggests that our sensitivity analysis is perhaps only the upper bounds of the briefer instruments.
We derived the abbreviated tests directly from the AUDIT, thus no assessment of the instruments out of the context of the full AUDIT was performed. Independent testing of abbreviated AUDIT instruments is needed. Recruitment was conducted by research assistants who solicited and provided forms to patients in the waiting rooms of primary care clinics. This convenience sample may have led to a selection bias in obtaining survey data. Patient recall bias may also have affected survey answers as patients may have had difficulty answering quantity and frequency questions accurately (an advantage of the AUDIT over quantity-frequency type questions). Also, our study investigates identification of individuals who are at least hazardous drinkers, but may also be abusive or dependent. We did not study the instruments’ ability to distinguish between hazardous drinking and abuse or dependence.
Conclusions
Our results confirm that the AUDIT-C and AUDIT-3 are useful screening tests for hazardous drinking. Because treatment of such drinkers can be effective, identifying people with less severe alcohol problems is crucial and an important public health initiative.21 Abbreviated instruments identify hazardous drinkers quickly, efficiently, and effectively, and may encourage early treatment to prevent the occurrence of alcohol-related consequences, abuse, or dependence. We recommend using the AUDIT-C and CAGE as brief screening instruments for hazardous drinking and alcohol abuse and dependence. This approach warrants further investigation.
Acknowledgments
Our work was supported by a grant to Dr Maisto from the National Institute of Alcohol Abuse and Alcoholism (AA10291). Dr Gordon is supported by a faculty development grant in general internal medicine from the VA Pittsburgh Healthcare System and the VISN 4 Mental Illness Research, Education, and Clinical Center. Dr Kraemer is supported by a Mentored Clinical Scientist Development Award from the National Institute of Alcohol Abuse and Alcoholism (AA00235). Dr J. Conigliaro is supported by a Career Development Award from the HSR & D Service, Department of Veterans Affairs (CD-97324-A) and is a generalist physician faculty scholar of the Robert Wood Johnson Foundation (#031500). We thank the ELM research study staff, Monica O’Connor (the ELM project coordinator), and all the patients who participated in the ELM study.
STUDY DESIGN: Cross-sectional survey.
POPULATION: Patients waiting for care at 12 primary care sites in western Pennsylvania from October 1995 to December 1997.
OUTCOMES MEASURED: Sensitivity, specificity, likelihood ratios, and predictive values for the AUDIT, AUDIT-C, and AUDIT-3.
RESULTS: A total of 13,438 patients were surveyed. Compared with a quantity-frequency definition of hazardous drinking (Ž16 drinks/week for men and Ž12 drinks/week for women), the AUDIT, AUDIT-C, and AUDIT-3 had areas under the receiver-operating characteristic curves (AUROC) of 0.940, 0.949, and 0.871, respectively. The AUROCs of the AUDIT and AUDIT-C were significantly different (P=.004). The AUROCs of the AUDIT-C (P <.001) and AUDIT (P <.001) were significantly larger than the AUDIT-3. When compared with a positive AUDIT score of 8 or higher, the AUDIT-C (score Ž3) and the AUDIT-3 (score Ž1) were 94.9% and 99.6% sensitive and 68.8% and 51.1% specific in detecting individuals as hazardous drinkers.
CONCLUSIONS: In a large primary care sample, a 3-question version of the AUDIT identified hazardous drinkers as well as the full AUDIT when such drinkers were defined by quantity-frequency criterion. This version of the AUDIT may be useful as an initial screen for assessing hazardous drinking behavior.
Hazardous drinkers consume enough alcohol to be at risk for adverse consequences but do not meet criteria for alcohol abuse or dependence. They are, however, are at risk for more harmful alcohol abuse.1-5 Such drinking behavior has been defined by quantity and frequency criteria.6 It is estimated that up to 20% of primary care patients are at least hazardous drinkers.7-9 Effective interventions to reduce alcohol consumption exist in primary care settings, so it is important for care providers to reliably and efficiently identify patients who are hazardous drinkers.1,10,11 Traditionally,12-14 care providers are poor at identifying such drinkers, and as many as 72% escape their detection.15-17 This ineffectiveness may be because of a lack of brief and simple questions that aid in patient identification.18-20
Formal screening instruments have been promoted to aid in identification of patients with alcohol problems. The Alcohol Use Disorders Identification Test (AUDIT) was developed by the World Health Organization (WHO) and consists of 2 distinct instruments: a 10-item AUDIT core questionnaire and a clinical screening procedure.1,9,21 The AUDIT core questions can detect hazardous drinkers and have been used alone as a screening instrument.22 The AUDIT questions address intake, dependence, and adverse consequences of drinking,23 emphasize drinking in the past year,5,24 and are indifferent to sex or ethnicity.4,25 It is most useful at detecting drinkers who do not meet criteria for alcohol abuse or dependence.26 Because of its ability to detect less severe alcohol drinkers, the AUDIT seems to have practical value in primary care settings.4,15,21,26
Because of trends toward shorter patient visits, the 10-question AUDIT may be too lengthy to be clinically useful in primary care settings.5,27,28 The shorter CAGE questionnaire, therefore, is often recommended for use in limited time situations.18,20 However, although the CAGE is a valuable tool for identifying alcohol abuse and dependence, it is not as useful for identifying less serious behaviors, such as hazardous drinking.5,28-32
A shorter version of the AUDIT may prove beneficial for use by the busy physician for identifying hazardous drinking behavior. The AUDIT-C (consisting of the first 3 questions of the AUDIT) was shown to be as effective as the full AUDIT in detecting hazardous drinking in a population of veterans.33 Also, the AUDIT-3 (the third question of the AUDIT) may be effective for identifying hazardous drinkers.5,34
We investigated the performance of the AUDIT, AUDIT-C, and AUDIT-3 in detecting such drinkers in a large primary care sample. We also compared the AUDIT-C and the AUDIT-3 to the full AUDIT. We hypothesized that the abbreviated instruments would be comparable with the AUDIT for detecting hazardous drinkers as defined by a quantity-frequency standard.
Methods
Design
We based our study on screening data obtained as part of a large randomized clinical trial of brief interventions for hazardous drinkers (the Early Lifestyle Modification [ELM] Study). Screening forms were administered at 12 primary care sites in the western Pennsylvania area from October 1995 to December 1997. The institutional review board at the University of Pittsburgh and equivalent review groups from each primary care setting approved the ELM study and screening protocol.
Setting
The 12 primary care sites included a Veterans Affairs Medical Center internal medicine clinic, a university-based internal medicine clinic, 2 university-affiliated community care clinics, 3 health maintenance organization clinics, 3 university-affiliated family medicine clinics, and 2 private practice family medicine clinics. All clinics were staffed by physicians. The Veterans Administration and university-based clinics had internal medicine residents participating in patient care. Also, physician assistants and nurse practitioners were involved with primary care at some sites.
Patients
Patients were eligible for the study if they were in the waiting rooms of one of the clinic sites during the screening period, were approached by a research assistant in the waiting room, and agreed to answer the screening questionnaire. Screening of eligible patients occurred from October 1995 to December 1997.
Screening Instruments
In the primary care clinics, patients self-administered an 8-page survey consisting of questions about lifestyle habits. Research assistants approached as many patients in the waiting rooms of the primary care sites as possible. The survey included questions about stress management (8 questions), smoking habits (8), the AUDIT (10), and quantity-frequency questions (3). In initial surveys, just before the alcohol-related questions patients were instructed not to answer alcohol questions if they responded “never” to the following question: “How often do you have a drink containing alcohol (for example: beer, wine, wine coolers, sherry, gin, vodka, or other hard liquor)?” This question was removed in later surveys, because it limited the number of people responding to the alcohol instruments under comparison. We also asked for categorical responses to questions about age, sex, education background, race, marital status, and occupation. We did not compensate patients for completing this questionnaire, and their responses were anonymous.
Alcohol Screening Instruments
The AUDIT consists of 10 questions Figure 1. The AUDIT-C includes the following first 3 questions of the AUDIT: How often do you have a drink containing alcohol? How many drinks containing alcohol do you have on a typical day when you are drinking? How often do you have 6 or more drinks on one occasion? The AUDIT-3 is the third question alone. Each individual question is scored from 0 to 4 points on a Likert scale, with higher numbers indicating more severe drinking behavior. For AUDIT questions with only 3 possible answers (questions 9 and 10), the scores were 0, 2, and 4. Thus, the range of possible scores on the AUDIT are from 0 to 40, the AUDIT-C from 0 to 12, and the AUDIT-3 from 0 to 4.
Our quantity-frequency questionnaire consisted of the questions: “If you drink, how many days per week do you have a drink?” (answers: 1 through 7); “If you drink less than once a week, how many days per month do you drink?” (answers: 1 or less, 2, 3, or 4 or more); and “How many drinks containing alcohol do you have on a typical day when you are drinking?” (answers: 1 through 10 or more). The number of average drinks per week was computed for analysis, and if the answer was “or more” we used the maximum number indicated in the question (4 or 10).
Outcome Measures and Analysis
The main outcome measures concerned the accuracy of the full AUDIT, AUDITC, and AUDIT-3. For comparative purposes, patients with missing responses were not used in subsequent statistical analyses.
The AUDIT, AUDIT-C, and AUDIT-3 were compared with a hazardous drinking criterion defined by the quantity-frequency questions. This criterion was 16 or more drinks per week for men and 12 or more drinks per week for women.6 Clinically, quantity-frequency assessment is often used to determine hazardous drinking behavior.16 The area under the receiver-operating characteristic curve (AUROC) was used to compare each instrument’s diagnostic ability.35 AUROC is an indication of the ability of a test to discriminate between false positives and false negatives. A score approaching 1 will be more sensitive and specific over a range of cutoff points than a score of 0.5, which is a nondiscriminating test. To calculate and compare the AUROCs we used published standards developed by Hanley and McNeil36,37 for curves derived by same cases. We performed chi-square analysis on comparisons of categorical data.
We then compared the AUDIT-C and AUDIT-3 with a score of the full AUDIT as a criterion for hazardous drinking. An AUDIT result of 8 or higher was accepted as a criterion for hazardous drinking. We measured the sensitivity and specificity of the AUDIT-C, using a cutoff score of 3 or higher and AUDIT-3 cutoff score of 1 or higher compared with the criterion of hazardous drinking defined by the full AUDIT.38
Results
At least 1 question on the screening survey was answered by 13,438 patients. Overall, 13,198 (98%) either answered that they currently drink or never drank alcohol. Not all questions were answered by each individual. Of patients who indicated that they currently drink alcohol, the full AUDIT was completed by 7035 (52%). The AUDIT-C was completed by 7190 (54%) patients, and the AUDIT-3 was answered by 7303 (54%) patients. Both the entire AUDIT and quantity-frequency questions were answered by 6954 (52%) patients. Of the 13,438 patients, 36% indicated no alcohol consumption. The majority of the surveyed sample were men, white, married, employed, high school graduates, and younger than 60 years Table 1.
AUDITs Compared with Quantity-Frequency Criterion
Table 2 compares the likelihood of hazardous alcohol use, defined by a quantity-frequency criterion, associated with each score of the AUDIT, AUDIT-C, and AUDIT-3. It is important to note that at each score the true positives of screening are only persons identified at that score. For example, at an AUDIT score of 7, 53 of 754 hazardous drinkers were identified with the resulting likelihood ratio (3.0) and predictive value (26.9%).
For comparisons of the AUDIT, AUDIT-C, and AUDIT-3 at identifying hazardous drinkers who scored at or greater than a minimal cutoff, the sensitivity and specificity compared with a quantity-frequency criterion is shown in Table 3. For cut-point values of an AUDIT score of 8 or higher, the sensitivity of the AUDIT was 76%. Similarly, the sensitivities of the AUDIT-C (with score Ž3) and AUDIT-3 (with score Ž1) were 99.6% and 89.1%, as sensitive as the quantity-frequency questions in detecting these patients. Specificity of the AUDIT, AUDIT-C, and AUDIT-3 at these cutoff values was 92%, 48%, and 65%, respectively.
AUROCs were constructed from all cut-point values Figure 2. Computation of the AUROC indicates the effectiveness of the instrument to discriminate hazardous drinkers over a range of AUDIT scores. The AUROCs for the AUDIT, AUDIT-C, and AUDIT-3 were significantly more discriminating than the line of identity (AUROC=0.5). The AUROC of the AUDIT was significantly different from the AUDIT-C (z=2.69; P=.004). The AUDIT-3 AUROC was significantly different than the AUDIT (z=10.03; P <.001) and AUDIT-C (z=12.69; P <.001).
Abbreviated AUDITs Compared with Full AUDIT Criterion
The full AUDIT is often used as a standard to assess hazardous drinking. We compared the abbreviated instruments to the full AUDIT, with a positive score of 8 or higher as a criterion for such drinking. The AUDIT-C (score (3) and AUDIT-3 (score Ž1) were 94.9% and 99.9% as sensitive and 68.8% and 51.1% as specific as the full AUDIT in obtaining a positive score. We also determined the performance of the AUDIT-3 when compared with a reference standard of a positive AUDIT-C. The AUDIT-3 (score Ž1) was 69% sensitive and 95% specific as the AUDIT-C (score Ž3) at identifying hazardous drinkers (data not shown).
Discussion
We evaluated the performance of the AUDIT and abbreviated AUDIT instruments to detect hazardous drinking in a large multisite primary care sample. The abbreviated forms of the AUDIT were as effective as the AUDIT at identifying hazardous drinkers. Compared with quantity-frequency questions, the AUDIT and AUDIT-C were superior at identifying hazardous drinkers than the AUDIT-3. The abbreviated forms of the AUDIT were as sensitive as the full AUDIT at detecting hazardous drinkers when using standard cutoff values for hazardous drinking.
As with the 4-item CAGE questionnaire for alcohol dependence, a 1- or 3-item AUDIT instrument may increase care providers’ recognition of hazardous drinkers. Providers do not routinely ask standard alcohol questions, are particularly poor at identifying hazardous drinkers, and do not enter patients into alcohol treatment.16 Therefore, providing clinicians with a few easily remembered questions to determine hazardous drinking behavior would be beneficial.39 A short questionnaire would be simple to administer and applicable in a wide variety of practice settings. A positive response would increase suspicion regarding hazardous or abusive drinking behavior and prompt additional questions about patients’ alcohol use.18,40 For example, care providers could use the AUDIT-3 or AUDIT-C (to detect at least hazardous drinking), then administer the questionnaire (to detect abuse and dependence), if the patient’s response was positive.
It is important to realize that the AUDIT and its abbreviated forms are only sensitive to detect hazardous drinking, not to specifically assign patients’ drinking habits as hazardous only. The AUDIT was originally designed to distinguish a person with hazardous drinking from one with nonhazardous drinking.26 As such, this instrument may not be specific enough to distinguish hazardous drinkers from others with severe alcohol behaviors; people who score positive may qualify for alcohol abuse and dependence. For less risky alcohol behaviors, it is more important for a health care provider to identify all hazardous drinkers (true positives), at the risk of falsely identifying a person who may not have this behavior (false positives).18 Therefore, when screening to establish a threshold level of treatment intervention, screening instruments should maximize sensitivity, even at the expense of low specificity.
In our sample, the AUDIT (score Ž8), AUDIT-C (score Ž3), and AUDIT-3 (score Ž1) were as sensitive as quantity-frequency questions in detecting hazardous drinking. This increase in sensitivity of the AUDIT-C and AUDIT-3 is likely related to the consumption focus of these questions. The AUDIT-C consists of quantity and frequency type questions, and the third question is specific for quantity of drinking at one session. However, the performance of AUDIT instruments in our study is comparable and confirms results found in a study by Bush and colleagues,34 even though the studies used different criteria for assessment of abbreviated instruments.
If the AUDIT is used as a standard to detect hazardous drinkers, would the AUDIT-C or AUDIT-3 identify the same patients as the full AUDIT? Using cutoff points for the AUDIT-C of 3 or higher and an AUDIT-3 score of 1 or higher, these instruments were 99.7% and 98.3% as sensitive as the full AUDIT. As expected, specificity is much less for both abbreviated instruments. However, the high sensitivity suggests a clinical utility for these abbreviated instruments. It is unlikely that by asking the 3 AUDIT-C questions or the single AUDIT-3 question, a primary care provider will miss identification of a person who is at least a hazardous drinker.
Limitations
There is no gold standard to identify hazardous drinkers.41 The definition of what level of drinking constitutes that label is controversial, and providers do not routinely ask standard questions about drinking behavior.16 Our criterion, the quantity-frequency questions, may be considered a poor standard to compare survey instruments.7,42 In research, quantity-frequency consumption questions are helpful in specific identification of hazardous drinkers when a cutoff value is defined.5 However, patients may prefer not to answer questions about quantity or frequency of alcohol use and may not respond consistently to heterogeneous provider questions. Therefore, quantity-frequency questions may be useful as a standard to compare similar instruments such as the AUDIT and its abbreviations, although they may not be particularly effective in clinical practice.
Not all surveyed individuals completed the full AUDIT instrument. This was primarily because patients who answered “never” to our initial question regarding any alcohol use did not proceed to the AUDIT. Before completion of the study, we eliminated this question from the survey. However, it is not known whether patients who answered “never” to the initial question, were, in fact, drinkers. Consistency response bias may also have occurred as patients may have wished to answer similar items similarly. In addition, the similarity of the AUDIT-C and AUDIT-3 to quantity-frequency questions suggests that our sensitivity analysis is perhaps only the upper bounds of the briefer instruments.
We derived the abbreviated tests directly from the AUDIT, thus no assessment of the instruments out of the context of the full AUDIT was performed. Independent testing of abbreviated AUDIT instruments is needed. Recruitment was conducted by research assistants who solicited and provided forms to patients in the waiting rooms of primary care clinics. This convenience sample may have led to a selection bias in obtaining survey data. Patient recall bias may also have affected survey answers as patients may have had difficulty answering quantity and frequency questions accurately (an advantage of the AUDIT over quantity-frequency type questions). Also, our study investigates identification of individuals who are at least hazardous drinkers, but may also be abusive or dependent. We did not study the instruments’ ability to distinguish between hazardous drinking and abuse or dependence.
Conclusions
Our results confirm that the AUDIT-C and AUDIT-3 are useful screening tests for hazardous drinking. Because treatment of such drinkers can be effective, identifying people with less severe alcohol problems is crucial and an important public health initiative.21 Abbreviated instruments identify hazardous drinkers quickly, efficiently, and effectively, and may encourage early treatment to prevent the occurrence of alcohol-related consequences, abuse, or dependence. We recommend using the AUDIT-C and CAGE as brief screening instruments for hazardous drinking and alcohol abuse and dependence. This approach warrants further investigation.
Acknowledgments
Our work was supported by a grant to Dr Maisto from the National Institute of Alcohol Abuse and Alcoholism (AA10291). Dr Gordon is supported by a faculty development grant in general internal medicine from the VA Pittsburgh Healthcare System and the VISN 4 Mental Illness Research, Education, and Clinical Center. Dr Kraemer is supported by a Mentored Clinical Scientist Development Award from the National Institute of Alcohol Abuse and Alcoholism (AA00235). Dr J. Conigliaro is supported by a Career Development Award from the HSR & D Service, Department of Veterans Affairs (CD-97324-A) and is a generalist physician faculty scholar of the Robert Wood Johnson Foundation (#031500). We thank the ELM research study staff, Monica O’Connor (the ELM project coordinator), and all the patients who participated in the ELM study.
1. A cross-national trial of brief interventions with heavy drinkers: WHO Brief Intervention Study Group. Am J Public Health 1996;86:948-55.
2. Babor TF, de la Fuente JR, Saunders J, Grant M. The Alcohol Use Disorders Identification Test: guidelines for use in primary health care. Geneva, Switzerland: World Health Organization; 1989.
3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th ed. Washington, DC: American Psychiatric Association; 1995.
4. Allen JP, Litten RZ, Fertig JB, Babor T. A review of research on the Alcohol Use Disorders Identification Test (AUDIT). Alcoholism 1997;21:613-19.
5. Bohn MJ, Babor TF, Kranzler HR. The Alcohol Use Disorders Identification Test (AUDIT): validation of a screening instrument for use in medical settings. J Studies Alcohol 1995;56:423-32.
6. Sanchez-Craig M, Wilkinson DA, Davila R. Empirically based guidelines for moderate drinking: 1-year results from three studies with problem drinkers. Am J Public Health 1995;85:823-28.
7. Bradley KA. Screening and diagnosis of alcoholism in the primary care setting. West J Med 1992;156:166-71.
8. Schorling JB, Klas PT, Willems JP, Everett AS. Addressing alcohol use among primary care patients: differences between family medicine and internal medicine residents. J Gen Intern Med 1994;9:248-54.
9. Saunders JB, Aasland OG, Amundsen A, Grant M. Alcohol consumption and related problems among primary health care patients: WHO collaborative project on early detection of persons with harmful alcohol consumption—I. Addiction 1993;88:349-62.
10. Wallace P, Cutler S, Haines A. Randomised controlled trial of general practitioner intervention in patients with excessive alcohol consumption. BMJ 1988;297:663-68.
11. Fleming MF, Barry KL, Manwell LB, Johnson K, London R. Brief physician advice for problem alcohol drinkers: a randomized controlled trial in community-based primary care practices. JAMA 1997;277:1039-45.
12. Kristenson H, Ohlin H, Hulten-Nosslin MB, Trell E, Hood B. Identification and intervention of heavy drinking in middle-aged men: results and follow-up of 24-60 months of long-term study with randomized controls. Alcoholism 1983;7:203-09.
13. Bien TH, Miller WR, Tonigan JS. Brief interventions for alcohol problems: a review. Addiction 1993;88:315-35.
14. Wilk AI, Jensen NM, Havighurst TC. Meta-analysis of randomized control trials addressing brief interventions in heavy alcohol drinkers. J Gen Intern Med 1997;12:274-83.
15. Conigrave KM, Saunders JB, Reznik RB. Predictive capacity of the AUDIT questionnaire for alcohol-related harm. Addiction 1995;90:1479-85.
16. Friedmann PD, McCullough D, Chin MH, Saitz R. Screening and intervention for alcohol problems: a national survey of primary care physicians and psychiatrists. J Gen Intern Med 2000;15:84-91.
17. Bowen OR, Sammons JH. The alcohol-abusing patient: a challenge to the profession. JAMA 1988;260:2267-70.
18. Allen JP, Maisto SA, Connors GJ. Self-report screening tests for alcohol problems in primary care. Arch Intern Med 1995;155:1726-30.
19. Saunders JB, Conigrave KM. Early identification of alcohol problems. CMAJ 1990;143:1060-69.
20. Mayfield D, McLeod G, Hall P. The CAGE questionnaire: validation of a new alcoholism screening instrument. Am J Psychiatry 1974;131:1121-23.
21. Barry KL, Fleming MF. The Alcohol Use Disorders Identification Test (AUDIT) and the SMAST-13: predictive validity in a rural primary care sample. Alcohol Alcoholism 1993;28:33-42.
22. Schmidt A, Barry KL, Fleming MF. Detection of problem drinkers: the Alcohol Use Disorders Identification Test (AUDIT). South Med J 1995;88:52-59.
23. Volk RJ, Steinbauer JR, Cantor SB, Holzer CE, III. The Alcohol Use Disorders Identification Test (AUDIT) as a screen for at-risk drinking in primary care patients of different racial/ethnic backgrounds. Addiction 1997;92:197-206.
24. Kettl PA. Detecting problem drinkers in your practice. Patient Care 1997;30:27-41.
25. Steinbauer JR, Cantor SB, Holzer CE, III, Volk RJ. Ethnic and sex bias in primary care screening tests for alcohol use disorders. Ann Intern Med 1998;129:353-62.
26. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption—II. Addiction 1993;88:791-804.
27. Foster AI, Blondell RD, Looney SW. The practicality of using the SMAST and AUDIT to screen for alcoholism among adolescents in an urban private family practice. J Kentucky Med Assoc 1997;95:105-07.
28. Bradley KA, Bush KR, McDonell MB, Malone T, Fihn SD. Screening for problem drinking: comparison of CAGE and AUDIT: Ambulatory Care Quality Improvement Project (ACQUIP). J Gen Intern Med 1998;13:379-88.
29. Morton JL, Jones TV, Manganaro MA. Performance of alcoholism screening questionnaires in elderly veterans. Am J Med 1996;101:153-59.
30. MacKenzie D, Langa A, Brown TM. Identifying hazardous or harmful alcohol use in medical admissions: a comparison of audit, cage and brief mast. Alcohol Alcoholism 1996;31:591-99.
31. Buchsbaum DG, Buchanan RG, Centor RM, Schnoll SH, Lawton MJ. Screening for alcohol abuse using CAGE scores and likelihood ratios. Ann Intern Med 1991;115:774-77.
32. Seppa K, Makela R, Sillanaukee P. Effectiveness of the Alcohol Use Disorders Identification Test in occupational health screenings. Alcoholism 1995;19:999-1003.
33. Piccinelli M, Tessari E, Bortolomasi M, et al. Efficacy of the alcohol use disorders identification test as a screening tool for hazardous alcohol intake and related disorders in primary care: a validity study. BMJ 1997;314:420-24.
34. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Arch Intern Med 1998;158:1789-95.
35. van Kammen DP, Kelley ME, Gurklis JA, et al. Behavioral vs biochemical prediction of clinical stability following haloperidol withdrawal in schizophrenia. Arch Gen Psychiatry 1995;52:673-78.
36. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver-operating characteristic (ROC) curve. Radiology 1982;143:29-36.
37. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839-43.
38. Conigrave KM, Hall WD, Saunders JB. The AUDIT questionnaire: choosing a cut-off score. Alcohol Use Disorder Identification Test. Addiction 1995;90:1349-56.
39. Cherpitel CJ, Clark WB. Ethnic differences in performance of screening instruments for identifying harmful drinking and alcohol dependence in the emergency room. Alcoholism 1995;19:628-34.
40. Reid MC, Fiellin DA, O’Connor PG. Hazardous and harmful alcohol consumption in primary care. Arch Intern Med 1999;159:1681-89.
41. Fink A, Hays RD, Moore AA, Beck JC. Alcohol-related problems in older persons. Determinants, consequences, and screening. Arch Intern Med 1996;156:1150-56.
42. Bradley KA, Boyd-Wickizer J, Powell SH, Burman ML. Alcohol screening questionnaires in women: a critical review. JAMA 1998;280:166-71.
1. A cross-national trial of brief interventions with heavy drinkers: WHO Brief Intervention Study Group. Am J Public Health 1996;86:948-55.
2. Babor TF, de la Fuente JR, Saunders J, Grant M. The Alcohol Use Disorders Identification Test: guidelines for use in primary health care. Geneva, Switzerland: World Health Organization; 1989.
3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th ed. Washington, DC: American Psychiatric Association; 1995.
4. Allen JP, Litten RZ, Fertig JB, Babor T. A review of research on the Alcohol Use Disorders Identification Test (AUDIT). Alcoholism 1997;21:613-19.
5. Bohn MJ, Babor TF, Kranzler HR. The Alcohol Use Disorders Identification Test (AUDIT): validation of a screening instrument for use in medical settings. J Studies Alcohol 1995;56:423-32.
6. Sanchez-Craig M, Wilkinson DA, Davila R. Empirically based guidelines for moderate drinking: 1-year results from three studies with problem drinkers. Am J Public Health 1995;85:823-28.
7. Bradley KA. Screening and diagnosis of alcoholism in the primary care setting. West J Med 1992;156:166-71.
8. Schorling JB, Klas PT, Willems JP, Everett AS. Addressing alcohol use among primary care patients: differences between family medicine and internal medicine residents. J Gen Intern Med 1994;9:248-54.
9. Saunders JB, Aasland OG, Amundsen A, Grant M. Alcohol consumption and related problems among primary health care patients: WHO collaborative project on early detection of persons with harmful alcohol consumption—I. Addiction 1993;88:349-62.
10. Wallace P, Cutler S, Haines A. Randomised controlled trial of general practitioner intervention in patients with excessive alcohol consumption. BMJ 1988;297:663-68.
11. Fleming MF, Barry KL, Manwell LB, Johnson K, London R. Brief physician advice for problem alcohol drinkers: a randomized controlled trial in community-based primary care practices. JAMA 1997;277:1039-45.
12. Kristenson H, Ohlin H, Hulten-Nosslin MB, Trell E, Hood B. Identification and intervention of heavy drinking in middle-aged men: results and follow-up of 24-60 months of long-term study with randomized controls. Alcoholism 1983;7:203-09.
13. Bien TH, Miller WR, Tonigan JS. Brief interventions for alcohol problems: a review. Addiction 1993;88:315-35.
14. Wilk AI, Jensen NM, Havighurst TC. Meta-analysis of randomized control trials addressing brief interventions in heavy alcohol drinkers. J Gen Intern Med 1997;12:274-83.
15. Conigrave KM, Saunders JB, Reznik RB. Predictive capacity of the AUDIT questionnaire for alcohol-related harm. Addiction 1995;90:1479-85.
16. Friedmann PD, McCullough D, Chin MH, Saitz R. Screening and intervention for alcohol problems: a national survey of primary care physicians and psychiatrists. J Gen Intern Med 2000;15:84-91.
17. Bowen OR, Sammons JH. The alcohol-abusing patient: a challenge to the profession. JAMA 1988;260:2267-70.
18. Allen JP, Maisto SA, Connors GJ. Self-report screening tests for alcohol problems in primary care. Arch Intern Med 1995;155:1726-30.
19. Saunders JB, Conigrave KM. Early identification of alcohol problems. CMAJ 1990;143:1060-69.
20. Mayfield D, McLeod G, Hall P. The CAGE questionnaire: validation of a new alcoholism screening instrument. Am J Psychiatry 1974;131:1121-23.
21. Barry KL, Fleming MF. The Alcohol Use Disorders Identification Test (AUDIT) and the SMAST-13: predictive validity in a rural primary care sample. Alcohol Alcoholism 1993;28:33-42.
22. Schmidt A, Barry KL, Fleming MF. Detection of problem drinkers: the Alcohol Use Disorders Identification Test (AUDIT). South Med J 1995;88:52-59.
23. Volk RJ, Steinbauer JR, Cantor SB, Holzer CE, III. The Alcohol Use Disorders Identification Test (AUDIT) as a screen for at-risk drinking in primary care patients of different racial/ethnic backgrounds. Addiction 1997;92:197-206.
24. Kettl PA. Detecting problem drinkers in your practice. Patient Care 1997;30:27-41.
25. Steinbauer JR, Cantor SB, Holzer CE, III, Volk RJ. Ethnic and sex bias in primary care screening tests for alcohol use disorders. Ann Intern Med 1998;129:353-62.
26. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption—II. Addiction 1993;88:791-804.
27. Foster AI, Blondell RD, Looney SW. The practicality of using the SMAST and AUDIT to screen for alcoholism among adolescents in an urban private family practice. J Kentucky Med Assoc 1997;95:105-07.
28. Bradley KA, Bush KR, McDonell MB, Malone T, Fihn SD. Screening for problem drinking: comparison of CAGE and AUDIT: Ambulatory Care Quality Improvement Project (ACQUIP). J Gen Intern Med 1998;13:379-88.
29. Morton JL, Jones TV, Manganaro MA. Performance of alcoholism screening questionnaires in elderly veterans. Am J Med 1996;101:153-59.
30. MacKenzie D, Langa A, Brown TM. Identifying hazardous or harmful alcohol use in medical admissions: a comparison of audit, cage and brief mast. Alcohol Alcoholism 1996;31:591-99.
31. Buchsbaum DG, Buchanan RG, Centor RM, Schnoll SH, Lawton MJ. Screening for alcohol abuse using CAGE scores and likelihood ratios. Ann Intern Med 1991;115:774-77.
32. Seppa K, Makela R, Sillanaukee P. Effectiveness of the Alcohol Use Disorders Identification Test in occupational health screenings. Alcoholism 1995;19:999-1003.
33. Piccinelli M, Tessari E, Bortolomasi M, et al. Efficacy of the alcohol use disorders identification test as a screening tool for hazardous alcohol intake and related disorders in primary care: a validity study. BMJ 1997;314:420-24.
34. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Arch Intern Med 1998;158:1789-95.
35. van Kammen DP, Kelley ME, Gurklis JA, et al. Behavioral vs biochemical prediction of clinical stability following haloperidol withdrawal in schizophrenia. Arch Gen Psychiatry 1995;52:673-78.
36. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver-operating characteristic (ROC) curve. Radiology 1982;143:29-36.
37. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839-43.
38. Conigrave KM, Hall WD, Saunders JB. The AUDIT questionnaire: choosing a cut-off score. Alcohol Use Disorder Identification Test. Addiction 1995;90:1349-56.
39. Cherpitel CJ, Clark WB. Ethnic differences in performance of screening instruments for identifying harmful drinking and alcohol dependence in the emergency room. Alcoholism 1995;19:628-34.
40. Reid MC, Fiellin DA, O’Connor PG. Hazardous and harmful alcohol consumption in primary care. Arch Intern Med 1999;159:1681-89.
41. Fink A, Hays RD, Moore AA, Beck JC. Alcohol-related problems in older persons. Determinants, consequences, and screening. Arch Intern Med 1996;156:1150-56.
42. Bradley KA, Boyd-Wickizer J, Powell SH, Burman ML. Alcohol screening questionnaires in women: a critical review. JAMA 1998;280:166-71.