Long H. Ngo, PhD

Study Sample and Design

We analyzed data from the 3D‐CAM validation study,[16] which prospectively enrolled participants from a large urban teaching hospital in Boston, Massachusetts, using a consecutive enrollment sampling strategy. Inclusion criteria were: (1) 75 years old, (2) admitted to general or geriatric medicine services, (3) able to communicate in English, (4) without terminal conditions, (5) expected hospital stay of 2 days, (6) not a previous study participant. Experienced clinicians screened patients for eligibility. If the patient lacked capacity to provide consent, the designated surrogate decision maker was contacted. The study was approved by the institutional review board.

Reference Standard Delirium Diagnosis

The reference standard delirium diagnosis was based on an extensive (45 minutes) face‐to‐face patient interview by experienced clinician assessors (neuropsychologists or advanced practice nurses), medical record review, and input from the nurse and family members. This comprehensive assessment included: (1) reason for hospital admission, hospital course, and presence of cognitive concerns, (2) family, social, and functional history, (3) Montreal Cognitive Assessment,[18] (4) Geriatric Depression Scale,[19] (5) medical record review including scoring of comorbidities using the Charlson index,[20] determination of functional status using the basic and Instrumental Activities of Daily Living,[21, 22] psychoactive medications administered, and (6) a family member interview to assess the patient's baseline cognitive status that included the Eight‐Item Interview to Differentiate Aging and Dementia,[23] to assess the presence of dementia. Using all of these data, an expert panel, including the clinical assessor, the study principal investigator (E.R.M.), a geriatrician, and an experienced neuropsychologist, adjudicated the final delirium diagnoses using Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM‐IV) criteria. The panel also adjudicated for the presence or absence of dementia and mild cognitive impairment based on National Institute on Aging‐Alzheimer's Association (NIA‐AA) criteria.[24] This approach has been used in other delirium studies.[25]

3D‐CAM Assessments

After the reference standard assessment, the 3D‐CAM was administered by trained research assistants (RAs) who were blinded to the results of the reference standard. To reduce the likelihood of fluctuations or temporal changes, all assessments were completed between 11:00 _am and 2:00 _pm and for each participant, within a 2‐hour time period (for example, 11:23 _am to 1:23 _pm).

Statistical Analyses to Determine the Best Single‐ and Two‐Item Screeners

To determine the best single 3D‐CAM item to identify delirium, the responses of the 20 individual items in the 3D‐CAM (see Supporting Table 1 in the online version of this article) were compared to the reference standard to determine their sensitivity and specificity. Similarly, an algorithm was used to generate all unique 2‐item combinations of the 20 items (190 unique pairs), which were compared to the reference. An error, no response, or an answer of I do not know by the patient was considered a positive screen for delirium. The 2‐item screeners were considered positive if 1 or both of the items were positive. Sensitivity and specificity were calculated along with 95% confidence intervals (CIs).

Subset analyses were performed to determine sensitivity and specificity of individual items and pairs of items stratified by the patient's baseline cognitive status. Two strata were createdpatients with dementia (N=56), and patients with normal baseline cognitive status or mild cognitive impairment (MCI) (N=145). We chose to group MCI with normal for 2 reasons: (1) dementia is a well‐established and strong risk factor for delirium, whereas the evidence for MCI being a risk factor for delirium is less established and (2) to achieve adequate allocation of delirious cases in both strata. Last, we report the sensitivity of altered level of consciousness (LOC), which included lethargy, stupor, coma, and hypervigilance as a single screening item for delirium in the overall sample and by cognitive status. Analyses were conducted using commercially available software (SAS version 9.3; SAS Institute, Inc., Cary, NC).

Single Item Screens

Table 2 reports the results of single‐item screens for delirium with sensitivity, the ability to correctly identify delirium when it is present by the reference standard, and specificity, the ability to correctly identify patients without delirium when it is not present by reference standard and 95% CIs. Items are listed in descending order of sensitivity; in the case of ties, the item with the higher specificity is listed first. The screening items with the highest sensitivity for delirium are Months of the year backwards, and Four digits backwards, both with a sensitivity of 83% (95% CI: 69%‐93%). Of these 2 items, Months of the year backwards had a much better specificity of 69% (95% CI: 61%‐76%), whereas Four digits backwards had a specificity of 52% (95% CI: 44%‐60%). The item What is the day of the week? had lower sensitivity at 71% (95% CI: 55%‐84%), but excellent specificity at 92% (95% CI: 87%‐96%).

We then examined performance of single‐item screeners in patients with and without dementia (Table 3). In persons with dementia, the best single item was also Months of the year backwards, with a sensitivity of 89% (95% CI: 72%‐98%) and a specificity of 61% (95% CI: 41%‐78%). In persons with normal baseline cognition or MCI, the best performing single item was Four digits backwards, with sensitivity of 79% (95% CI: 49%‐95%) and specificity of 51% (95% CI: 42%‐60%). Months of the year backwards also performed well, with sensitivity of 71% (95% CI: 42%‐92%) and specificity of 71% (95% CI: 62%‐79%).

Two‐Item Screens

Table 4 reports the results of 2‐item screens for delirium with sensitivity, specificity, and 95% CIs. Item pairs are listed in descending order of sensitivity following the same convention as in Table 2. The 2‐item screen with the highest sensitivity for delirium is the combination of What is the day of the week? and Months of the year backwards, with a sensitivity of 93% (95% CI: 81%‐99%) and specificity of 64% (95% CI: 56%‐70%). This screen had a positive and negative likelihood ratio (LR) of 2.59 and 0.11, respectively. The combination of What is the day of the week? and Four digits backwards had the same sensitivity 93% (95% CI: 81%‐99%), but lower specificity of 48% (95% CI: 40%‐56%). The combination of What type of place is this? (hospital) and Four digits backwards had a sensitivity of 90% (95% CI: 77%‐97%) and specificity of 51% (95% CI: 43%‐50%).

When subjects were stratified by baseline cognition, the best 2‐item screens for normal and MCI patients was What is the day of the week? and Four digits backwards, with 93% sensitivity (95% CI: 66%‐100%) and 50% specificity (95% CI: 42%‐59%). The best pair of items for patients with dementia (Table 5) was the same as the overall sample, What is the day of the week? and Months of the year backwards, but its performance differed with a higher sensitivity of 96% (95% CI: 82%‐100%) and lower specificity of 43% (95% CI: 24%‐63%). This same pair of items had 86% sensitivity (95% CI: 57%‐98%) and 69% (95% CI: 60%‐77%) specificity for persons with either normal cognition or MCI.

Altered Level of Consciousness as a Screener for Delirium

Altered level of consciousness (ALOC) was uncommon in our sample, with an overall prevalence of 10/201 (4.9%). When examined as a screening item for delirium, ALOC had very poor sensitivity of 19% (95% CI: 9%‐34%) but had excellent specificity 99% (95% CI: 96%‐100%). Altered LOC also demonstrated poor screening performance when stratified by cognitive status, with a sensitivity of 14% in the normal and MCI group (95% CI: 2%‐43%) and sensitivity of 21% (95% CI: 8%‐41%) in persons with dementia.

Positive and Negative Predictive Values

Although we focused on sensitivity and specificity in evaluating 1‐ and 2‐item screeners, we also examined positive and negative predictive values. These values will vary depending on the overall prevalence of delirium, which was 21% in this dataset. The best 1‐item screener, Months of the year backwards, had a positive predictive value of 31% and negative predictive value of 94%. The best 2‐item screener, Months of the year backwards with What is the day of the week?, had a positive predictive value of 41% and negative predictive value of 97% (see Supporting Tables 2 and 3 in the online version of this article) LRs for the items are in Tables 2 through 5.

Study Sample and Design

We analyzed data from the 3D‐CAM validation study,[16] which prospectively enrolled participants from a large urban teaching hospital in Boston, Massachusetts, using a consecutive enrollment sampling strategy. Inclusion criteria were: (1) 75 years old, (2) admitted to general or geriatric medicine services, (3) able to communicate in English, (4) without terminal conditions, (5) expected hospital stay of 2 days, (6) not a previous study participant. Experienced clinicians screened patients for eligibility. If the patient lacked capacity to provide consent, the designated surrogate decision maker was contacted. The study was approved by the institutional review board.

Reference Standard Delirium Diagnosis

The reference standard delirium diagnosis was based on an extensive (45 minutes) face‐to‐face patient interview by experienced clinician assessors (neuropsychologists or advanced practice nurses), medical record review, and input from the nurse and family members. This comprehensive assessment included: (1) reason for hospital admission, hospital course, and presence of cognitive concerns, (2) family, social, and functional history, (3) Montreal Cognitive Assessment,[18] (4) Geriatric Depression Scale,[19] (5) medical record review including scoring of comorbidities using the Charlson index,[20] determination of functional status using the basic and Instrumental Activities of Daily Living,[21, 22] psychoactive medications administered, and (6) a family member interview to assess the patient's baseline cognitive status that included the Eight‐Item Interview to Differentiate Aging and Dementia,[23] to assess the presence of dementia. Using all of these data, an expert panel, including the clinical assessor, the study principal investigator (E.R.M.), a geriatrician, and an experienced neuropsychologist, adjudicated the final delirium diagnoses using Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM‐IV) criteria. The panel also adjudicated for the presence or absence of dementia and mild cognitive impairment based on National Institute on Aging‐Alzheimer's Association (NIA‐AA) criteria.[24] This approach has been used in other delirium studies.[25]

3D‐CAM Assessments

After the reference standard assessment, the 3D‐CAM was administered by trained research assistants (RAs) who were blinded to the results of the reference standard. To reduce the likelihood of fluctuations or temporal changes, all assessments were completed between 11:00 _am and 2:00 _pm and for each participant, within a 2‐hour time period (for example, 11:23 _am to 1:23 _pm).

Statistical Analyses to Determine the Best Single‐ and Two‐Item Screeners

To determine the best single 3D‐CAM item to identify delirium, the responses of the 20 individual items in the 3D‐CAM (see Supporting Table 1 in the online version of this article) were compared to the reference standard to determine their sensitivity and specificity. Similarly, an algorithm was used to generate all unique 2‐item combinations of the 20 items (190 unique pairs), which were compared to the reference. An error, no response, or an answer of I do not know by the patient was considered a positive screen for delirium. The 2‐item screeners were considered positive if 1 or both of the items were positive. Sensitivity and specificity were calculated along with 95% confidence intervals (CIs).

Subset analyses were performed to determine sensitivity and specificity of individual items and pairs of items stratified by the patient's baseline cognitive status. Two strata were createdpatients with dementia (N=56), and patients with normal baseline cognitive status or mild cognitive impairment (MCI) (N=145). We chose to group MCI with normal for 2 reasons: (1) dementia is a well‐established and strong risk factor for delirium, whereas the evidence for MCI being a risk factor for delirium is less established and (2) to achieve adequate allocation of delirious cases in both strata. Last, we report the sensitivity of altered level of consciousness (LOC), which included lethargy, stupor, coma, and hypervigilance as a single screening item for delirium in the overall sample and by cognitive status. Analyses were conducted using commercially available software (SAS version 9.3; SAS Institute, Inc., Cary, NC).

Single Item Screens

Table 2 reports the results of single‐item screens for delirium with sensitivity, the ability to correctly identify delirium when it is present by the reference standard, and specificity, the ability to correctly identify patients without delirium when it is not present by reference standard and 95% CIs. Items are listed in descending order of sensitivity; in the case of ties, the item with the higher specificity is listed first. The screening items with the highest sensitivity for delirium are Months of the year backwards, and Four digits backwards, both with a sensitivity of 83% (95% CI: 69%‐93%). Of these 2 items, Months of the year backwards had a much better specificity of 69% (95% CI: 61%‐76%), whereas Four digits backwards had a specificity of 52% (95% CI: 44%‐60%). The item What is the day of the week? had lower sensitivity at 71% (95% CI: 55%‐84%), but excellent specificity at 92% (95% CI: 87%‐96%).

We then examined performance of single‐item screeners in patients with and without dementia (Table 3). In persons with dementia, the best single item was also Months of the year backwards, with a sensitivity of 89% (95% CI: 72%‐98%) and a specificity of 61% (95% CI: 41%‐78%). In persons with normal baseline cognition or MCI, the best performing single item was Four digits backwards, with sensitivity of 79% (95% CI: 49%‐95%) and specificity of 51% (95% CI: 42%‐60%). Months of the year backwards also performed well, with sensitivity of 71% (95% CI: 42%‐92%) and specificity of 71% (95% CI: 62%‐79%).

Two‐Item Screens

Table 4 reports the results of 2‐item screens for delirium with sensitivity, specificity, and 95% CIs. Item pairs are listed in descending order of sensitivity following the same convention as in Table 2. The 2‐item screen with the highest sensitivity for delirium is the combination of What is the day of the week? and Months of the year backwards, with a sensitivity of 93% (95% CI: 81%‐99%) and specificity of 64% (95% CI: 56%‐70%). This screen had a positive and negative likelihood ratio (LR) of 2.59 and 0.11, respectively. The combination of What is the day of the week? and Four digits backwards had the same sensitivity 93% (95% CI: 81%‐99%), but lower specificity of 48% (95% CI: 40%‐56%). The combination of What type of place is this? (hospital) and Four digits backwards had a sensitivity of 90% (95% CI: 77%‐97%) and specificity of 51% (95% CI: 43%‐50%).

When subjects were stratified by baseline cognition, the best 2‐item screens for normal and MCI patients was What is the day of the week? and Four digits backwards, with 93% sensitivity (95% CI: 66%‐100%) and 50% specificity (95% CI: 42%‐59%). The best pair of items for patients with dementia (Table 5) was the same as the overall sample, What is the day of the week? and Months of the year backwards, but its performance differed with a higher sensitivity of 96% (95% CI: 82%‐100%) and lower specificity of 43% (95% CI: 24%‐63%). This same pair of items had 86% sensitivity (95% CI: 57%‐98%) and 69% (95% CI: 60%‐77%) specificity for persons with either normal cognition or MCI.

Altered Level of Consciousness as a Screener for Delirium

Altered level of consciousness (ALOC) was uncommon in our sample, with an overall prevalence of 10/201 (4.9%). When examined as a screening item for delirium, ALOC had very poor sensitivity of 19% (95% CI: 9%‐34%) but had excellent specificity 99% (95% CI: 96%‐100%). Altered LOC also demonstrated poor screening performance when stratified by cognitive status, with a sensitivity of 14% in the normal and MCI group (95% CI: 2%‐43%) and sensitivity of 21% (95% CI: 8%‐41%) in persons with dementia.

Positive and Negative Predictive Values

Although we focused on sensitivity and specificity in evaluating 1‐ and 2‐item screeners, we also examined positive and negative predictive values. These values will vary depending on the overall prevalence of delirium, which was 21% in this dataset. The best 1‐item screener, Months of the year backwards, had a positive predictive value of 31% and negative predictive value of 94%. The best 2‐item screener, Months of the year backwards with What is the day of the week?, had a positive predictive value of 41% and negative predictive value of 97% (see Supporting Tables 2 and 3 in the online version of this article) LRs for the items are in Tables 2 through 5.

Study Sample and Design

We analyzed data from the 3D‐CAM validation study,[16] which prospectively enrolled participants from a large urban teaching hospital in Boston, Massachusetts, using a consecutive enrollment sampling strategy. Inclusion criteria were: (1) 75 years old, (2) admitted to general or geriatric medicine services, (3) able to communicate in English, (4) without terminal conditions, (5) expected hospital stay of 2 days, (6) not a previous study participant. Experienced clinicians screened patients for eligibility. If the patient lacked capacity to provide consent, the designated surrogate decision maker was contacted. The study was approved by the institutional review board.

Reference Standard Delirium Diagnosis

The reference standard delirium diagnosis was based on an extensive (45 minutes) face‐to‐face patient interview by experienced clinician assessors (neuropsychologists or advanced practice nurses), medical record review, and input from the nurse and family members. This comprehensive assessment included: (1) reason for hospital admission, hospital course, and presence of cognitive concerns, (2) family, social, and functional history, (3) Montreal Cognitive Assessment,[18] (4) Geriatric Depression Scale,[19] (5) medical record review including scoring of comorbidities using the Charlson index,[20] determination of functional status using the basic and Instrumental Activities of Daily Living,[21, 22] psychoactive medications administered, and (6) a family member interview to assess the patient's baseline cognitive status that included the Eight‐Item Interview to Differentiate Aging and Dementia,[23] to assess the presence of dementia. Using all of these data, an expert panel, including the clinical assessor, the study principal investigator (E.R.M.), a geriatrician, and an experienced neuropsychologist, adjudicated the final delirium diagnoses using Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM‐IV) criteria. The panel also adjudicated for the presence or absence of dementia and mild cognitive impairment based on National Institute on Aging‐Alzheimer's Association (NIA‐AA) criteria.[24] This approach has been used in other delirium studies.[25]

3D‐CAM Assessments

After the reference standard assessment, the 3D‐CAM was administered by trained research assistants (RAs) who were blinded to the results of the reference standard. To reduce the likelihood of fluctuations or temporal changes, all assessments were completed between 11:00 _am and 2:00 _pm and for each participant, within a 2‐hour time period (for example, 11:23 _am to 1:23 _pm).

Statistical Analyses to Determine the Best Single‐ and Two‐Item Screeners

To determine the best single 3D‐CAM item to identify delirium, the responses of the 20 individual items in the 3D‐CAM (see Supporting Table 1 in the online version of this article) were compared to the reference standard to determine their sensitivity and specificity. Similarly, an algorithm was used to generate all unique 2‐item combinations of the 20 items (190 unique pairs), which were compared to the reference. An error, no response, or an answer of I do not know by the patient was considered a positive screen for delirium. The 2‐item screeners were considered positive if 1 or both of the items were positive. Sensitivity and specificity were calculated along with 95% confidence intervals (CIs).

Subset analyses were performed to determine sensitivity and specificity of individual items and pairs of items stratified by the patient's baseline cognitive status. Two strata were createdpatients with dementia (N=56), and patients with normal baseline cognitive status or mild cognitive impairment (MCI) (N=145). We chose to group MCI with normal for 2 reasons: (1) dementia is a well‐established and strong risk factor for delirium, whereas the evidence for MCI being a risk factor for delirium is less established and (2) to achieve adequate allocation of delirious cases in both strata. Last, we report the sensitivity of altered level of consciousness (LOC), which included lethargy, stupor, coma, and hypervigilance as a single screening item for delirium in the overall sample and by cognitive status. Analyses were conducted using commercially available software (SAS version 9.3; SAS Institute, Inc., Cary, NC).

Single Item Screens

Table 2 reports the results of single‐item screens for delirium with sensitivity, the ability to correctly identify delirium when it is present by the reference standard, and specificity, the ability to correctly identify patients without delirium when it is not present by reference standard and 95% CIs. Items are listed in descending order of sensitivity; in the case of ties, the item with the higher specificity is listed first. The screening items with the highest sensitivity for delirium are Months of the year backwards, and Four digits backwards, both with a sensitivity of 83% (95% CI: 69%‐93%). Of these 2 items, Months of the year backwards had a much better specificity of 69% (95% CI: 61%‐76%), whereas Four digits backwards had a specificity of 52% (95% CI: 44%‐60%). The item What is the day of the week? had lower sensitivity at 71% (95% CI: 55%‐84%), but excellent specificity at 92% (95% CI: 87%‐96%).

We then examined performance of single‐item screeners in patients with and without dementia (Table 3). In persons with dementia, the best single item was also Months of the year backwards, with a sensitivity of 89% (95% CI: 72%‐98%) and a specificity of 61% (95% CI: 41%‐78%). In persons with normal baseline cognition or MCI, the best performing single item was Four digits backwards, with sensitivity of 79% (95% CI: 49%‐95%) and specificity of 51% (95% CI: 42%‐60%). Months of the year backwards also performed well, with sensitivity of 71% (95% CI: 42%‐92%) and specificity of 71% (95% CI: 62%‐79%).

Two‐Item Screens

Table 4 reports the results of 2‐item screens for delirium with sensitivity, specificity, and 95% CIs. Item pairs are listed in descending order of sensitivity following the same convention as in Table 2. The 2‐item screen with the highest sensitivity for delirium is the combination of What is the day of the week? and Months of the year backwards, with a sensitivity of 93% (95% CI: 81%‐99%) and specificity of 64% (95% CI: 56%‐70%). This screen had a positive and negative likelihood ratio (LR) of 2.59 and 0.11, respectively. The combination of What is the day of the week? and Four digits backwards had the same sensitivity 93% (95% CI: 81%‐99%), but lower specificity of 48% (95% CI: 40%‐56%). The combination of What type of place is this? (hospital) and Four digits backwards had a sensitivity of 90% (95% CI: 77%‐97%) and specificity of 51% (95% CI: 43%‐50%).

When subjects were stratified by baseline cognition, the best 2‐item screens for normal and MCI patients was What is the day of the week? and Four digits backwards, with 93% sensitivity (95% CI: 66%‐100%) and 50% specificity (95% CI: 42%‐59%). The best pair of items for patients with dementia (Table 5) was the same as the overall sample, What is the day of the week? and Months of the year backwards, but its performance differed with a higher sensitivity of 96% (95% CI: 82%‐100%) and lower specificity of 43% (95% CI: 24%‐63%). This same pair of items had 86% sensitivity (95% CI: 57%‐98%) and 69% (95% CI: 60%‐77%) specificity for persons with either normal cognition or MCI.

Altered Level of Consciousness as a Screener for Delirium

Altered level of consciousness (ALOC) was uncommon in our sample, with an overall prevalence of 10/201 (4.9%). When examined as a screening item for delirium, ALOC had very poor sensitivity of 19% (95% CI: 9%‐34%) but had excellent specificity 99% (95% CI: 96%‐100%). Altered LOC also demonstrated poor screening performance when stratified by cognitive status, with a sensitivity of 14% in the normal and MCI group (95% CI: 2%‐43%) and sensitivity of 21% (95% CI: 8%‐41%) in persons with dementia.

Positive and Negative Predictive Values

Although we focused on sensitivity and specificity in evaluating 1‐ and 2‐item screeners, we also examined positive and negative predictive values. These values will vary depending on the overall prevalence of delirium, which was 21% in this dataset. The best 1‐item screener, Months of the year backwards, had a positive predictive value of 31% and negative predictive value of 94%. The best 2‐item screener, Months of the year backwards with What is the day of the week?, had a positive predictive value of 41% and negative predictive value of 97% (see Supporting Tables 2 and 3 in the online version of this article) LRs for the items are in Tables 2 through 5.

Setting and Patients

We conducted a retrospective cohort study using data from 286 US nonfederal acute‐care facilities contributing to the database maintained by Premier (Premier Healthcare Solutions, Inc., Charlotte, NC). This database, created to measure healthcare utilization and quality of care, is drawn from voluntarily participating hospitals and contains data on approximately 1 in every 4 discharges nationwide.[11] Participating hospitals are similar in geographic distribution and metropolitan (urban/rural) status to hospitals nationwide, although large, nonteaching hospitals are slightly overrepresented in Premier. The database contains patient demographics, International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes, hospital demographics, and a date‐stamped log of all charges during the course of each hospitalization, including diagnostic tests, therapeutic treatments, and medications with dose and route of administration. The study was approved by the institutional review board at Beth Israel Deaconess Medical Center and granted a waiver of informed consent.

We studied a cohort of all adult nonsurgical admissions to participating hospitals from July 1, 2009, through June 30, 2010. We chose to study nonsurgical admissions, as patients undergoing surgical procedures have a clear indication for, and almost always receive, opioid pain medications. We defined a nonsurgical admission as an admission in which there were no charges for operating‐room procedures (including labor and delivery) and the attending of record was not a surgeon. We excluded admissions with unknown gender, since this is a key demographic variable, and admissions with a length of stay greater than 365 days, as these admissions are not representative of the typical admission to an acute‐care hospital. At the hospital level, we excluded hospitals contributing <100 admissions owing to resultant lack of precision in corresponding hospital prescribing rates, and hospitals that did not prescribe the full range of opioid medications (these hospitals had charges for codeine only), as these facilities seemed likely to have unusual limitations on prescribing or incomplete data capture.

Opioid Exposure

We defined opioid exposure as presence of 1 charge for an opioid medication during the admission. Opioid medications included morphine, hydrocodone, hydromorphone, oxycodone, fentanyl, meperidine, methadone, codeine, tramadol, buprenorphine, levorphanol, oxymorphone, pentazocine, propoxyphene, tapentadol, butorphanol, dezocine, and nalbuphine. We grouped the last 9 into an other category owing to infrequent use and/or differing characteristics from the main opioid drug types, such as synthetic, semisynthetic, and partial agonist qualities.

Severe Opioid‐Related Adverse Events

We defined severe opioid‐related adverse events as either naloxone exposure or an opioid‐related adverse drug event diagnosis code. Naloxone use in an adult patient exposed to opioids is one of the Institute for Healthcare Improvement's Trigger Tools for identifying adverse drug events[12] and previously has been demonstrated to have high positive predictive value for a confirmed adverse drug event.[13] We defined naloxone exposure as presence of 1 charge for naloxone. We excluded charges on hospital day 1 to focus on nosocomial events. We defined opioid‐related adverse drug events using ICD‐9‐CM diagnosis codes for poisoning by opioids (overdose, wrong substance given, or taken in error; ICD‐9‐CM 965.02, 965.09, E850.1, E850.2) and drugs causing adverse effects in therapeutic use (ICD‐9‐CM E935.1, E935.2), as specified in prior analyses by the Agency for Healthcare Research and Quality (AHRQ).[14, 15] To avoid capturing adverse events associated with outpatient use, we required the ICD‐9‐CM code to be qualified as not present on admission using the present on admission indicator required by the Centers for Medicare and Medicaid Services for all discharge diagnosis codes since 2008.[16]

Covariates of Interest

We were interested in the relationship between both patient and hospital characteristics and opioid exposure. Patient characteristics of interest included (1) demographic variables, such as age, sex, race (self‐reported by patients at the time of admission), marital status, and payer; (2) whether or not the patient spent any time in the intensive care unit (ICU); (3) comorbidities, identified via ICD‐9‐CM secondary diagnosis codes and diagnosis‐related groups using Healthcare Cost and Utilization Project Comorbidity Software, version 3.7, based on the work of Elixhauser et al[17, 18]; (4) primary ICD‐9‐CM discharge diagnosis groupings, selected based on hypothesized associations with receipt of opioids, and based on the Clinical Classifications Software (CCS)a diagnosis and procedure categorization scheme maintained by the AHRQ, and defined in the Appendix[19]; (5) and nonoperating‐room‐based procedures potentially necessitating opioids during the admission, selected from the 50 most common ICD‐9‐CM procedure codes in our cohort and grouped as cardiovascular procedures (catheterization and insertion of vascular stents), gastrointestinal procedures (upper and lower endoscopy), and mechanical ventilation, further defined in the Appendix. Hospital characteristics of interest included number of beds, population served (urban vs rural), teaching status, and US census region (Northeast, Midwest, South, West).

Statistical Analysis

We calculated the percent of admissions with exposure to any opioid and the percent exposed to each opioid, along with the total number of different opioid medications used during each admission. We also calculated the percent of admissions with parenteral administration and the percent of admissions with oral administration, among those exposed to the individual categories, and in aggregate. Because medications after discharge were unavailable in Premier's dataset, we report the percent of patients with a charge for opioids on the day of discharge.

We determined the daily dose of an opioid by taking the sum of the doses for that opioid charged on a given day. The average daily dose of an opioid was determined by taking the sum of the daily doses and dividing by the number of days on which 1 dose was charged. To facilitate comparison, all opioids, with the exception of those for which standard equivalences are unavailable (tramadol, other opioid category, oral fentanyl, epidural route for all), were converted to oral morphine equivalents using a standard equivalence conversion table.[20, 21] We excluded from our dosage calculations those charges for which standard morphine equivalence was unavailable, or for which dosage was missing. We also excluded from our dosage calculations any dose that was >3 standard deviations (SD) above the mean dose for that opioid, as such extreme values seemed physiologically implausible and more likely to be a data entry error that could lead to significant overestimation of the mean for that opioid.

All multivariable models used a generalized estimating equation (GEE) via the genmod procedure in SAS, with a Poisson distribution error term and a log link, controlling for repeated patient admissions with an autoregressive correlation structure.

To identify independent predictors of opioid receipt, we used a GEE model of opioid receipt where all patient and hospital characteristics listed in Table 1 were included as independent variables.

To assess hospital variation in opioid prescribing after adjusting for patient characteristics, we used a GEE model of opioid receipt, controlling for all patient characteristics listed in Table 1. We then took the mean of the predicted probabilities of opioid receipt for the patients within each hospital in our cohort to derive the hospital prescribing rate adjusted for patient characteristics. We report the mean, SD, and range of the prescribing rates for the hospitals in our cohort before and after adjustment for patient characteristics.

To assess whether patients admitted to hospitals with higher rates of opioid prescribing have higher relative risk of severe opioid‐related adverse events, we stratified hospitals into opioid‐prescribing rate quartiles and compared the rates of opioid‐related adverse eventsboth overall and among opioid exposedbetween quartiles. To adjust for patient characteristics, we used a GEE model in which severe opioid‐related adverse event (yes/no) was the dependent variable and hospital‐prescribing rate quartile and all patient characteristics in Table 1 were independent variables. We also performed a sensitivity analysis in which we assessed the association between hospital prescribing‐rate quartile and the individual components of our composite outcome. Our results were qualitatively unchanged using this approach, and only the results of our main analysis are presented.

All analyses were carried out using SAS software, version 9.2 (SAS Institute Inc., Cary, NC).

Patient Admission Characteristics

There were 3,190,934 adult admissions to 300 acute‐care hospitals during our study period. After excluding admissions with a length of stay >365 days (n=25), missing patient sex (n=17), and charges for operating‐room procedures or a surgical attending of record (n=2,018,553), 1,172,339 admissions were available for analysis. There were 12 hospitals with incomplete opioid‐prescribing data (n=32,794) and 2 hospitals that contributed <100 admissions each (n=126), leaving 1,139,419 admissions in 286 hospitals in our analytic cohort. The median age of the cohort was 64 years (interquartile range, 4979 years), and 527,062 (46%) were men. Table 1 shows the characteristics of the admissions in the cohort.

Rate, Route, and Dose of Opioid Exposures

Overall, there were 576,373 (51%) admissions with charges for opioid medications. Among those exposed, 244,760 (43%) had charges for multiple opioids during the admission; 172,090 (30%) had charges for 2 different opioids; and 72,670 (13%) had charges for 3 different opioids. Table 2 shows the percent exposed to each opioid, the percent of exposed with parenteral and oral routes of administration, and the mean daily dose received in oral morphine equivalents.

Among the medications/routes for which conversion to morphine equivalents was possible, dosage was missing in 39,728 out of 2,294,673 opioid charges (2%). The average daily dose received in oral morphine equivalents was 68 mg. A total dose of 50 mg per day was received in 39% of exposed, and a total dose of 100 mg per day was received in 23% of exposed. Among those exposed, 52% (26% of overall admissions) had charges for opioids on the day of discharge.

Rates of Opioid Use by Patient and Hospital Characteristics

Table 3 reports the association between admission characteristics and opioid use. Use was highest in patients between the ages of 25 and 54 years. Although use declined with age, 44% of admissions age 65 years had charges for opioid medication. After adjustment for patient demographics, comorbidities, and hospital characteristics, opioid use was more common in females than males, those age 2554 years compared with those older and younger, those of Caucasian race compared with non‐Caucasian race, and those with Medicare or Medicaid primary insurance. Among the primary discharge diagnoses, patients with musculoskeletal injuries, various specific and nonspecific pain‐related diagnoses, and cancer were significantly more likely to receive opioids than patients without these diagnoses, whereas patients with alcohol‐related disorders and psychiatric disorders were significantly less likely to receive opioids than patients without these diagnoses. Patients admitted to hospitals in the Midwest, South, and West were significantly more likely to receive opioid medications than patients in the Northeast.

Variation in Opioid Prescribing

Figure 1 shows the histograms of hospital opioid prescribing rate for the 286 hospitals in our cohort (A) before and (B) after adjustment for patient characteristics. The observed rates ranged from 5% in the lowest‐prescribing hospital to 72% in the highest‐prescribing hospital, with a mean (SD) of 51% (10%). After adjusting for patient characteristics, the adjusted opioid‐prescribing rates ranged from 33% to 64%, with a mean (SD) of 50% (4%).

Figure 1

Severe Opioid‐Related Adverse Events

Among admissions with opioid exposure (n=576,373), naloxone use occurred in 2345 (0.41%) and opioid‐related adverse drug events in 1174 (0.20%), for a total of 3441 (0.60%) severe opioid‐related adverse events (some patients experienced both). Table 4 reports the opioid exposure and severe opioid‐related adverse‐event rates within hospital opioid‐prescribing rate quartiles, along with the adjusted association between the hospital opioid‐prescribing rate quartile and severe opioid‐related adverse events. After adjusting for patient characteristics, the relative risk of a severe opioid‐related adverse event was significantly greater in hospitals with higher opioid‐prescribing rates, both overall and among opioid exposed.

Setting and Patients

We conducted a retrospective cohort study using data from 286 US nonfederal acute‐care facilities contributing to the database maintained by Premier (Premier Healthcare Solutions, Inc., Charlotte, NC). This database, created to measure healthcare utilization and quality of care, is drawn from voluntarily participating hospitals and contains data on approximately 1 in every 4 discharges nationwide.[11] Participating hospitals are similar in geographic distribution and metropolitan (urban/rural) status to hospitals nationwide, although large, nonteaching hospitals are slightly overrepresented in Premier. The database contains patient demographics, International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes, hospital demographics, and a date‐stamped log of all charges during the course of each hospitalization, including diagnostic tests, therapeutic treatments, and medications with dose and route of administration. The study was approved by the institutional review board at Beth Israel Deaconess Medical Center and granted a waiver of informed consent.

We studied a cohort of all adult nonsurgical admissions to participating hospitals from July 1, 2009, through June 30, 2010. We chose to study nonsurgical admissions, as patients undergoing surgical procedures have a clear indication for, and almost always receive, opioid pain medications. We defined a nonsurgical admission as an admission in which there were no charges for operating‐room procedures (including labor and delivery) and the attending of record was not a surgeon. We excluded admissions with unknown gender, since this is a key demographic variable, and admissions with a length of stay greater than 365 days, as these admissions are not representative of the typical admission to an acute‐care hospital. At the hospital level, we excluded hospitals contributing <100 admissions owing to resultant lack of precision in corresponding hospital prescribing rates, and hospitals that did not prescribe the full range of opioid medications (these hospitals had charges for codeine only), as these facilities seemed likely to have unusual limitations on prescribing or incomplete data capture.

Opioid Exposure

We defined opioid exposure as presence of 1 charge for an opioid medication during the admission. Opioid medications included morphine, hydrocodone, hydromorphone, oxycodone, fentanyl, meperidine, methadone, codeine, tramadol, buprenorphine, levorphanol, oxymorphone, pentazocine, propoxyphene, tapentadol, butorphanol, dezocine, and nalbuphine. We grouped the last 9 into an other category owing to infrequent use and/or differing characteristics from the main opioid drug types, such as synthetic, semisynthetic, and partial agonist qualities.

Severe Opioid‐Related Adverse Events

We defined severe opioid‐related adverse events as either naloxone exposure or an opioid‐related adverse drug event diagnosis code. Naloxone use in an adult patient exposed to opioids is one of the Institute for Healthcare Improvement's Trigger Tools for identifying adverse drug events[12] and previously has been demonstrated to have high positive predictive value for a confirmed adverse drug event.[13] We defined naloxone exposure as presence of 1 charge for naloxone. We excluded charges on hospital day 1 to focus on nosocomial events. We defined opioid‐related adverse drug events using ICD‐9‐CM diagnosis codes for poisoning by opioids (overdose, wrong substance given, or taken in error; ICD‐9‐CM 965.02, 965.09, E850.1, E850.2) and drugs causing adverse effects in therapeutic use (ICD‐9‐CM E935.1, E935.2), as specified in prior analyses by the Agency for Healthcare Research and Quality (AHRQ).[14, 15] To avoid capturing adverse events associated with outpatient use, we required the ICD‐9‐CM code to be qualified as not present on admission using the present on admission indicator required by the Centers for Medicare and Medicaid Services for all discharge diagnosis codes since 2008.[16]

Covariates of Interest

We were interested in the relationship between both patient and hospital characteristics and opioid exposure. Patient characteristics of interest included (1) demographic variables, such as age, sex, race (self‐reported by patients at the time of admission), marital status, and payer; (2) whether or not the patient spent any time in the intensive care unit (ICU); (3) comorbidities, identified via ICD‐9‐CM secondary diagnosis codes and diagnosis‐related groups using Healthcare Cost and Utilization Project Comorbidity Software, version 3.7, based on the work of Elixhauser et al[17, 18]; (4) primary ICD‐9‐CM discharge diagnosis groupings, selected based on hypothesized associations with receipt of opioids, and based on the Clinical Classifications Software (CCS)a diagnosis and procedure categorization scheme maintained by the AHRQ, and defined in the Appendix[19]; (5) and nonoperating‐room‐based procedures potentially necessitating opioids during the admission, selected from the 50 most common ICD‐9‐CM procedure codes in our cohort and grouped as cardiovascular procedures (catheterization and insertion of vascular stents), gastrointestinal procedures (upper and lower endoscopy), and mechanical ventilation, further defined in the Appendix. Hospital characteristics of interest included number of beds, population served (urban vs rural), teaching status, and US census region (Northeast, Midwest, South, West).

Statistical Analysis

We calculated the percent of admissions with exposure to any opioid and the percent exposed to each opioid, along with the total number of different opioid medications used during each admission. We also calculated the percent of admissions with parenteral administration and the percent of admissions with oral administration, among those exposed to the individual categories, and in aggregate. Because medications after discharge were unavailable in Premier's dataset, we report the percent of patients with a charge for opioids on the day of discharge.

We determined the daily dose of an opioid by taking the sum of the doses for that opioid charged on a given day. The average daily dose of an opioid was determined by taking the sum of the daily doses and dividing by the number of days on which 1 dose was charged. To facilitate comparison, all opioids, with the exception of those for which standard equivalences are unavailable (tramadol, other opioid category, oral fentanyl, epidural route for all), were converted to oral morphine equivalents using a standard equivalence conversion table.[20, 21] We excluded from our dosage calculations those charges for which standard morphine equivalence was unavailable, or for which dosage was missing. We also excluded from our dosage calculations any dose that was >3 standard deviations (SD) above the mean dose for that opioid, as such extreme values seemed physiologically implausible and more likely to be a data entry error that could lead to significant overestimation of the mean for that opioid.

All multivariable models used a generalized estimating equation (GEE) via the genmod procedure in SAS, with a Poisson distribution error term and a log link, controlling for repeated patient admissions with an autoregressive correlation structure.

To identify independent predictors of opioid receipt, we used a GEE model of opioid receipt where all patient and hospital characteristics listed in Table 1 were included as independent variables.

To assess hospital variation in opioid prescribing after adjusting for patient characteristics, we used a GEE model of opioid receipt, controlling for all patient characteristics listed in Table 1. We then took the mean of the predicted probabilities of opioid receipt for the patients within each hospital in our cohort to derive the hospital prescribing rate adjusted for patient characteristics. We report the mean, SD, and range of the prescribing rates for the hospitals in our cohort before and after adjustment for patient characteristics.

To assess whether patients admitted to hospitals with higher rates of opioid prescribing have higher relative risk of severe opioid‐related adverse events, we stratified hospitals into opioid‐prescribing rate quartiles and compared the rates of opioid‐related adverse eventsboth overall and among opioid exposedbetween quartiles. To adjust for patient characteristics, we used a GEE model in which severe opioid‐related adverse event (yes/no) was the dependent variable and hospital‐prescribing rate quartile and all patient characteristics in Table 1 were independent variables. We also performed a sensitivity analysis in which we assessed the association between hospital prescribing‐rate quartile and the individual components of our composite outcome. Our results were qualitatively unchanged using this approach, and only the results of our main analysis are presented.

All analyses were carried out using SAS software, version 9.2 (SAS Institute Inc., Cary, NC).

Patient Admission Characteristics

There were 3,190,934 adult admissions to 300 acute‐care hospitals during our study period. After excluding admissions with a length of stay >365 days (n=25), missing patient sex (n=17), and charges for operating‐room procedures or a surgical attending of record (n=2,018,553), 1,172,339 admissions were available for analysis. There were 12 hospitals with incomplete opioid‐prescribing data (n=32,794) and 2 hospitals that contributed <100 admissions each (n=126), leaving 1,139,419 admissions in 286 hospitals in our analytic cohort. The median age of the cohort was 64 years (interquartile range, 4979 years), and 527,062 (46%) were men. Table 1 shows the characteristics of the admissions in the cohort.

Rate, Route, and Dose of Opioid Exposures

Overall, there were 576,373 (51%) admissions with charges for opioid medications. Among those exposed, 244,760 (43%) had charges for multiple opioids during the admission; 172,090 (30%) had charges for 2 different opioids; and 72,670 (13%) had charges for 3 different opioids. Table 2 shows the percent exposed to each opioid, the percent of exposed with parenteral and oral routes of administration, and the mean daily dose received in oral morphine equivalents.

Among the medications/routes for which conversion to morphine equivalents was possible, dosage was missing in 39,728 out of 2,294,673 opioid charges (2%). The average daily dose received in oral morphine equivalents was 68 mg. A total dose of 50 mg per day was received in 39% of exposed, and a total dose of 100 mg per day was received in 23% of exposed. Among those exposed, 52% (26% of overall admissions) had charges for opioids on the day of discharge.

Rates of Opioid Use by Patient and Hospital Characteristics

Table 3 reports the association between admission characteristics and opioid use. Use was highest in patients between the ages of 25 and 54 years. Although use declined with age, 44% of admissions age 65 years had charges for opioid medication. After adjustment for patient demographics, comorbidities, and hospital characteristics, opioid use was more common in females than males, those age 2554 years compared with those older and younger, those of Caucasian race compared with non‐Caucasian race, and those with Medicare or Medicaid primary insurance. Among the primary discharge diagnoses, patients with musculoskeletal injuries, various specific and nonspecific pain‐related diagnoses, and cancer were significantly more likely to receive opioids than patients without these diagnoses, whereas patients with alcohol‐related disorders and psychiatric disorders were significantly less likely to receive opioids than patients without these diagnoses. Patients admitted to hospitals in the Midwest, South, and West were significantly more likely to receive opioid medications than patients in the Northeast.

Variation in Opioid Prescribing

Figure 1 shows the histograms of hospital opioid prescribing rate for the 286 hospitals in our cohort (A) before and (B) after adjustment for patient characteristics. The observed rates ranged from 5% in the lowest‐prescribing hospital to 72% in the highest‐prescribing hospital, with a mean (SD) of 51% (10%). After adjusting for patient characteristics, the adjusted opioid‐prescribing rates ranged from 33% to 64%, with a mean (SD) of 50% (4%).

Figure 1

Severe Opioid‐Related Adverse Events

Among admissions with opioid exposure (n=576,373), naloxone use occurred in 2345 (0.41%) and opioid‐related adverse drug events in 1174 (0.20%), for a total of 3441 (0.60%) severe opioid‐related adverse events (some patients experienced both). Table 4 reports the opioid exposure and severe opioid‐related adverse‐event rates within hospital opioid‐prescribing rate quartiles, along with the adjusted association between the hospital opioid‐prescribing rate quartile and severe opioid‐related adverse events. After adjusting for patient characteristics, the relative risk of a severe opioid‐related adverse event was significantly greater in hospitals with higher opioid‐prescribing rates, both overall and among opioid exposed.

Setting and Patients

We conducted a retrospective cohort study using data from 286 US nonfederal acute‐care facilities contributing to the database maintained by Premier (Premier Healthcare Solutions, Inc., Charlotte, NC). This database, created to measure healthcare utilization and quality of care, is drawn from voluntarily participating hospitals and contains data on approximately 1 in every 4 discharges nationwide.[11] Participating hospitals are similar in geographic distribution and metropolitan (urban/rural) status to hospitals nationwide, although large, nonteaching hospitals are slightly overrepresented in Premier. The database contains patient demographics, International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes, hospital demographics, and a date‐stamped log of all charges during the course of each hospitalization, including diagnostic tests, therapeutic treatments, and medications with dose and route of administration. The study was approved by the institutional review board at Beth Israel Deaconess Medical Center and granted a waiver of informed consent.

We studied a cohort of all adult nonsurgical admissions to participating hospitals from July 1, 2009, through June 30, 2010. We chose to study nonsurgical admissions, as patients undergoing surgical procedures have a clear indication for, and almost always receive, opioid pain medications. We defined a nonsurgical admission as an admission in which there were no charges for operating‐room procedures (including labor and delivery) and the attending of record was not a surgeon. We excluded admissions with unknown gender, since this is a key demographic variable, and admissions with a length of stay greater than 365 days, as these admissions are not representative of the typical admission to an acute‐care hospital. At the hospital level, we excluded hospitals contributing <100 admissions owing to resultant lack of precision in corresponding hospital prescribing rates, and hospitals that did not prescribe the full range of opioid medications (these hospitals had charges for codeine only), as these facilities seemed likely to have unusual limitations on prescribing or incomplete data capture.

Opioid Exposure

We defined opioid exposure as presence of 1 charge for an opioid medication during the admission. Opioid medications included morphine, hydrocodone, hydromorphone, oxycodone, fentanyl, meperidine, methadone, codeine, tramadol, buprenorphine, levorphanol, oxymorphone, pentazocine, propoxyphene, tapentadol, butorphanol, dezocine, and nalbuphine. We grouped the last 9 into an other category owing to infrequent use and/or differing characteristics from the main opioid drug types, such as synthetic, semisynthetic, and partial agonist qualities.

Severe Opioid‐Related Adverse Events

We defined severe opioid‐related adverse events as either naloxone exposure or an opioid‐related adverse drug event diagnosis code. Naloxone use in an adult patient exposed to opioids is one of the Institute for Healthcare Improvement's Trigger Tools for identifying adverse drug events[12] and previously has been demonstrated to have high positive predictive value for a confirmed adverse drug event.[13] We defined naloxone exposure as presence of 1 charge for naloxone. We excluded charges on hospital day 1 to focus on nosocomial events. We defined opioid‐related adverse drug events using ICD‐9‐CM diagnosis codes for poisoning by opioids (overdose, wrong substance given, or taken in error; ICD‐9‐CM 965.02, 965.09, E850.1, E850.2) and drugs causing adverse effects in therapeutic use (ICD‐9‐CM E935.1, E935.2), as specified in prior analyses by the Agency for Healthcare Research and Quality (AHRQ).[14, 15] To avoid capturing adverse events associated with outpatient use, we required the ICD‐9‐CM code to be qualified as not present on admission using the present on admission indicator required by the Centers for Medicare and Medicaid Services for all discharge diagnosis codes since 2008.[16]

Covariates of Interest

We were interested in the relationship between both patient and hospital characteristics and opioid exposure. Patient characteristics of interest included (1) demographic variables, such as age, sex, race (self‐reported by patients at the time of admission), marital status, and payer; (2) whether or not the patient spent any time in the intensive care unit (ICU); (3) comorbidities, identified via ICD‐9‐CM secondary diagnosis codes and diagnosis‐related groups using Healthcare Cost and Utilization Project Comorbidity Software, version 3.7, based on the work of Elixhauser et al[17, 18]; (4) primary ICD‐9‐CM discharge diagnosis groupings, selected based on hypothesized associations with receipt of opioids, and based on the Clinical Classifications Software (CCS)a diagnosis and procedure categorization scheme maintained by the AHRQ, and defined in the Appendix[19]; (5) and nonoperating‐room‐based procedures potentially necessitating opioids during the admission, selected from the 50 most common ICD‐9‐CM procedure codes in our cohort and grouped as cardiovascular procedures (catheterization and insertion of vascular stents), gastrointestinal procedures (upper and lower endoscopy), and mechanical ventilation, further defined in the Appendix. Hospital characteristics of interest included number of beds, population served (urban vs rural), teaching status, and US census region (Northeast, Midwest, South, West).

Statistical Analysis

We calculated the percent of admissions with exposure to any opioid and the percent exposed to each opioid, along with the total number of different opioid medications used during each admission. We also calculated the percent of admissions with parenteral administration and the percent of admissions with oral administration, among those exposed to the individual categories, and in aggregate. Because medications after discharge were unavailable in Premier's dataset, we report the percent of patients with a charge for opioids on the day of discharge.

We determined the daily dose of an opioid by taking the sum of the doses for that opioid charged on a given day. The average daily dose of an opioid was determined by taking the sum of the daily doses and dividing by the number of days on which 1 dose was charged. To facilitate comparison, all opioids, with the exception of those for which standard equivalences are unavailable (tramadol, other opioid category, oral fentanyl, epidural route for all), were converted to oral morphine equivalents using a standard equivalence conversion table.[20, 21] We excluded from our dosage calculations those charges for which standard morphine equivalence was unavailable, or for which dosage was missing. We also excluded from our dosage calculations any dose that was >3 standard deviations (SD) above the mean dose for that opioid, as such extreme values seemed physiologically implausible and more likely to be a data entry error that could lead to significant overestimation of the mean for that opioid.

All multivariable models used a generalized estimating equation (GEE) via the genmod procedure in SAS, with a Poisson distribution error term and a log link, controlling for repeated patient admissions with an autoregressive correlation structure.

To identify independent predictors of opioid receipt, we used a GEE model of opioid receipt where all patient and hospital characteristics listed in Table 1 were included as independent variables.

To assess hospital variation in opioid prescribing after adjusting for patient characteristics, we used a GEE model of opioid receipt, controlling for all patient characteristics listed in Table 1. We then took the mean of the predicted probabilities of opioid receipt for the patients within each hospital in our cohort to derive the hospital prescribing rate adjusted for patient characteristics. We report the mean, SD, and range of the prescribing rates for the hospitals in our cohort before and after adjustment for patient characteristics.

To assess whether patients admitted to hospitals with higher rates of opioid prescribing have higher relative risk of severe opioid‐related adverse events, we stratified hospitals into opioid‐prescribing rate quartiles and compared the rates of opioid‐related adverse eventsboth overall and among opioid exposedbetween quartiles. To adjust for patient characteristics, we used a GEE model in which severe opioid‐related adverse event (yes/no) was the dependent variable and hospital‐prescribing rate quartile and all patient characteristics in Table 1 were independent variables. We also performed a sensitivity analysis in which we assessed the association between hospital prescribing‐rate quartile and the individual components of our composite outcome. Our results were qualitatively unchanged using this approach, and only the results of our main analysis are presented.

All analyses were carried out using SAS software, version 9.2 (SAS Institute Inc., Cary, NC).

Patient Admission Characteristics

There were 3,190,934 adult admissions to 300 acute‐care hospitals during our study period. After excluding admissions with a length of stay >365 days (n=25), missing patient sex (n=17), and charges for operating‐room procedures or a surgical attending of record (n=2,018,553), 1,172,339 admissions were available for analysis. There were 12 hospitals with incomplete opioid‐prescribing data (n=32,794) and 2 hospitals that contributed <100 admissions each (n=126), leaving 1,139,419 admissions in 286 hospitals in our analytic cohort. The median age of the cohort was 64 years (interquartile range, 4979 years), and 527,062 (46%) were men. Table 1 shows the characteristics of the admissions in the cohort.

Rate, Route, and Dose of Opioid Exposures

Overall, there were 576,373 (51%) admissions with charges for opioid medications. Among those exposed, 244,760 (43%) had charges for multiple opioids during the admission; 172,090 (30%) had charges for 2 different opioids; and 72,670 (13%) had charges for 3 different opioids. Table 2 shows the percent exposed to each opioid, the percent of exposed with parenteral and oral routes of administration, and the mean daily dose received in oral morphine equivalents.

Among the medications/routes for which conversion to morphine equivalents was possible, dosage was missing in 39,728 out of 2,294,673 opioid charges (2%). The average daily dose received in oral morphine equivalents was 68 mg. A total dose of 50 mg per day was received in 39% of exposed, and a total dose of 100 mg per day was received in 23% of exposed. Among those exposed, 52% (26% of overall admissions) had charges for opioids on the day of discharge.

Rates of Opioid Use by Patient and Hospital Characteristics

Table 3 reports the association between admission characteristics and opioid use. Use was highest in patients between the ages of 25 and 54 years. Although use declined with age, 44% of admissions age 65 years had charges for opioid medication. After adjustment for patient demographics, comorbidities, and hospital characteristics, opioid use was more common in females than males, those age 2554 years compared with those older and younger, those of Caucasian race compared with non‐Caucasian race, and those with Medicare or Medicaid primary insurance. Among the primary discharge diagnoses, patients with musculoskeletal injuries, various specific and nonspecific pain‐related diagnoses, and cancer were significantly more likely to receive opioids than patients without these diagnoses, whereas patients with alcohol‐related disorders and psychiatric disorders were significantly less likely to receive opioids than patients without these diagnoses. Patients admitted to hospitals in the Midwest, South, and West were significantly more likely to receive opioid medications than patients in the Northeast.

Variation in Opioid Prescribing

Figure 1 shows the histograms of hospital opioid prescribing rate for the 286 hospitals in our cohort (A) before and (B) after adjustment for patient characteristics. The observed rates ranged from 5% in the lowest‐prescribing hospital to 72% in the highest‐prescribing hospital, with a mean (SD) of 51% (10%). After adjusting for patient characteristics, the adjusted opioid‐prescribing rates ranged from 33% to 64%, with a mean (SD) of 50% (4%).

Figure 1

Severe Opioid‐Related Adverse Events

Among admissions with opioid exposure (n=576,373), naloxone use occurred in 2345 (0.41%) and opioid‐related adverse drug events in 1174 (0.20%), for a total of 3441 (0.60%) severe opioid‐related adverse events (some patients experienced both). Table 4 reports the opioid exposure and severe opioid‐related adverse‐event rates within hospital opioid‐prescribing rate quartiles, along with the adjusted association between the hospital opioid‐prescribing rate quartile and severe opioid‐related adverse events. After adjusting for patient characteristics, the relative risk of a severe opioid‐related adverse event was significantly greater in hospitals with higher opioid‐prescribing rates, both overall and among opioid exposed.