Peter K. Lindenauer, MD, MSc

Hospital readmissions are emblematic of the numerous challenges facing the US health care system. Despite high levels of spending, nearly 20% of Medicare beneficiaries are readmitted within 30 days of hospital discharge, many readmissions are considered preventable, and rates vary widely by hospital and region.1 Further, while readmissions have been estimated to cost taxpayers as much as $17 billion annually, the current fee‐for‐service method of paying for the acute care needs of seniors rewards hospitals financially for readmission, not their prevention.2

Pneumonia is the second most common reason for hospitalization among Medicare beneficiaries, accounting for approximately 650,000 admissions annually,3 and has been a focus of national quality‐improvement efforts for more than a decade.4, 5 Despite improvements in key processes of care, rates of readmission within 30 days of discharge following a hospitalization for pneumonia have been reported to vary from 10% to 24%.68 Among several factors, readmissions are believed to be influenced by the quality of both inpatient and outpatient care, and by care‐coordination activities occurring in the transition from inpatient to outpatient status.912

Public reporting of hospital performance is considered a key strategy for improving quality, reducing costs, and increasing the value of hospital care, both in the US and worldwide.13 In 2009, the Centers for Medicare & Medicaid Services (CMS) expanded its reporting initiatives by adding risk‐adjusted hospital readmission rates for acute myocardial infarction, heart failure, and pneumonia to the Hospital Compare website.14, 15 Readmission rates are an attractive focus for public reporting for several reasons. First, in contrast to most process‐based measures of quality (eg, whether a patient with pneumonia received a particular antibiotic), a readmission is an adverse outcome that matters to patients and families.16 Second, unlike process measures whose assessment requires detailed review of medical records, readmissions can be easily determined from standard hospital claims. Finally, readmissions are costly, and their prevention could yield substantial savings to society.

A necessary prerequisite for public reporting of readmission is a validated, risk‐adjusted measure that can be used to track performance over time and can facilitate comparisons across institutions. Toward this end, we describe the development, validation, and results of a National Quality Forum‐approved and CMS‐adopted model to estimate hospital‐specific, risk‐standardized, 30‐day readmission rates for Medicare patients hospitalized with pneumonia.17

METHODS

RESULTS

Model Derivation and Performance

After exclusions were applied, the 2006 sample included 453,251 pneumonia hospitalizations (Figure 1). The development sample consisted of 226,545 hospitalizations at 4675 hospitals, with an overall unadjusted 30‐day readmission rate of 17.4%. In 11,694 index cases (5.2%), the patient died within 30 days without being readmitted. Median readmission rate was 16.3%, 25th and 75th percentile rates were 11.1% and 21.3%, and at the 10th and 90th percentile, hospital readmission rates ranged from 4.6% to 26.7% (Figure 2).

Figure 1

Figure 2

The claims model included 39 variables (age, sex, and 37 clinical variables) (Table 1). The mean age of the cohort was 80.0 years, with 55.5% women and 11.1% nonwhite patients. Mean observed readmission rate in the development sample ranged from 9% in the lowest decile of predicted pneumonia readmission rates to 32% in the highest predicted decile, a range of 23%. The AUC was 0.63. For comparison, a model with only age and sex had an AUC of 0.51, and a model with all candidate variables had an AUC equal to 0.63 (Table 2).

Table 2.Readmission Model Performance of Administrative Claims Models

		Calibration (0, 1)*	Discrimination		Residuals Lack of Fit (Pearson Residual Fall %)				Model 2 (No. of Covariates)
			Predictive Ability (Lowest Decile, Highest Decile)	AUC	(<2)	(2, 0)	(0, 2)	(2+)
NOTE: Over‐fitting indices (0, 1) provide evidence of over‐fitting and require several steps to calculate. Let b denote the estimated vector of regression coefficients. Predicted Probabilities (p) = 1/(1+exp{Xb}), and Z = Xb (eg, the linear predictor that is a scalar value for everyone). A new logistic regression model that includes only an intercept and a slope by regressing the logits on Z is fitted in the validation sample; eg, Logit(P(Y = 1|Z)) = 0 + 1Z. Estimated values of 0 far from 0 and estimated values of 1 far from 1 provide evidence of over‐fitting. Abbreviations: AUC, area under the receiver operating curve. * Max‐rescaled R‐square. Observed rates. Wald chi‐square.
Development sample
2006	(1st half) N = 226,545	(0, 1)	(0.09, 0.32)	0.63	0	82.62	7.39	9.99	6,843 (40)
Validation sample
2006	(2nd half) N = 226,706	(0.002, 0.997)	(0.09, 0.31)	0.63	0	82.55	7.45	9.99	6,870 (40)
2005	N = 536,015	(0.035, 1.008)	(0.08, 0.31)	0.63	0	82.67	7.31	10.03	16,241 (40)

Hospital Risk‐Standardized Readmission Rates

Risk‐standardized readmission rates varied across hospitals (Figure 3). Median risk‐standardized readmission rate was 17.3%, and the 25th and 75th percentiles were 16.9% and 17.9%, respectively. The 5th percentile was 16.0% and the 95th percentile was 19.1%. Odds of readmission for a hospital one standard deviation above average was 1.4 times that of a hospital one standard deviation below average.

Figure 3

Medical Record Validation

After exclusions, the medical record sample taken from the National Pneumonia Project included 47,429 cases, with an unadjusted 30‐day readmission rate of 17.0%. The final medical record risk‐adjustment model included a total of 35 variables, whose prevalence and association with readmission risk varied modestly (Table 3). Performance of the medical record and administrative models was similar (areas under the ROC curve 0.59 and 0.63, respectively) (Table 4). Additionally, in the administrative model, predicted readmission rates ranged from 8% in the lowest predicted decile to 30% in the highest predicted decile, while in the medical record model, the corresponding rates varied from 10% to 26%.

Table 3.Regression Model Results from Medical Record Sample

Variable	Percent	Estimate	Standard Error	Odds Ratio	95% CI
NOTE: Between‐state variance = 0.024; standard error = 0.00. Abbreviations: BP, blood pressure; BUN, blood urea nitrogen; CI, confidence interval; SD, standard deviation; WBC, white blood cell count.
Age 65, mean (SD)	15.24 (7.87)	0.003	0.002	0.997	0.993	1.000
Male	46.18	0.122	0.025	1.130	1.075	1.188
Nursing home resident	17.71	0.035	0.037	1.036	0.963	1.114
Neoplastic disease	6.80	0.130	0.049	1.139	1.034	1.254
Liver disease	1.04	0.089	0.123	0.915	0.719	1.164
History of heart failure	28.98	0.234	0.029	1.264	1.194	1.339
History of renal disease	8.51	0.188	0.047	1.206	1.100	1.323
Altered mental status	17.95	0.009	0.034	1.009	0.944	1.080
Pleural effusion	21.20	0.165	0.030	1.179	1.111	1.251
BUN 30 mg/dl	23.28	0.160	0.033	1.174	1.100	1.252
BUN missing	14.56	0.101	0.185	0.904	0.630	1.298
Systolic BP <90 mmHg	2.95	0.068	0.070	1.070	0.932	1.228
Systolic BP missing	11.21	0.149	0.425	1.160	0.504	2.669
Pulse 125/min	7.73	0.036	0.047	1.036	0.945	1.137
Pulse missing	11.22	0.210	0.405	1.234	0.558	2.729
Respiratory rate 30/min	16.38	0.079	0.034	1.082	1.012	1.157
Respiratory rate missing	11.39	0.204	0.240	1.226	0.765	1.964
Sodium <130 mmol/L	4.82	0.136	0.057	1.145	1.025	1.280
Sodium missing	14.39	0.049	0.143	1.050	0.793	1.391
Glucose 250 mg/dl	5.19	0.005	0.057	0.995	0.889	1.114
Glucose missing	15.44	0.156	0.105	0.855	0.696	1.051
Hematocrit <30%	7.77	0.270	0.044	1.310	1.202	1.428
Hematocrit missing	13.62	0.071	0.135	0.932	0.715	1.215
Creatinine 2.5 mg/dL	4.68	0.109	0.062	1.115	0.989	1.258
Creatinine missing	14.63	0.200	0.167	1.221	0.880	1.695
WBC 6‐12 b/L	38.04	0.021	0.049	0.979	0.889	1.079
WBC >12 b/L	41.45	0.068	0.049	0.934	0.848	1.029
WBC missing	12.85	0.167	0.162	1.181	0.860	1.623
Immunosuppressive therapy	15.01	0.347	0.035	1.415	1.321	1.516
Chronic lung disease	42.16	0.137	0.028	1.147	1.086	1.211
Coronary artery disease	39.57	0.150	0.028	1.162	1.100	1.227
Diabetes mellitus	20.90	0.137	0.033	1.147	1.076	1.223
Alcohol/drug abuse	3.40	0.099	0.071	0.906	0.788	1.041
Dementia/Alzheimer's disease	16.38	0.125	0.038	1.133	1.052	1.222
Splenectomy	0.44	0.016	0.186	1.016	0.706	1.463

Table 4.Model Performance of Medical Record Model

Model	Calibration (0, 1)*	Discrimination		Residuals Lack of Fit (Pearson Residual Fall %)				Model 2 (No. of Covariates)
		Predictive Ability (Lowest Decile, Highest Decile)	AUC	(<2)	(2, 0)	(0, 2)	(2+)
Abbreviations: AUC, area under the receiver operating curve. * Max‐rescaled R‐square. Observed rates. Wald chi‐square.
Medical Record Model Development Sample (NP)
N = 47,429 No. of 30‐day readmissions = 8,042	(1, 0)	(0.10, 0.26)	0.59	0	83.04	5.28	11.68	710 (35)
Linked Administrative Model Validation Sample
N = 47,429 No. of 30‐day readmissions = 8,042	(1, 0)	(0.08, 0.30)	0.63	0	83.04	6.94	10.01	1,414 (40)

The correlation coefficient of the estimated state‐specific standardized readmission rates from the administrative and medical record models was 0.96, and the proportion of the variance explained by the model was 0.92 (Figure 4).

Figure 4

DISCUSSION

We have described the development, validation, and results of a hospital, 30‐day, risk‐standardized readmission model for pneumonia that was created to support current federal transparency initiatives. The model uses administrative claims data from Medicare fee‐for‐service patients and produces results that are comparable to a model based on information obtained through manual abstraction of medical records. We observed an overall 30‐day readmission rate of 17%, and our analyses revealed substantial variation across US hospitals, suggesting that improvement by lower performing institutions is an achievable goal.

Because more than one in six pneumonia patients are rehospitalized shortly after discharge, and because pneumonia hospitalizations represent an enormous expense to the Medicare program, prevention of readmissions is now widely recognized to offer a substantial opportunity to improve patient outcomes while simultaneously lowering health care costs. Accordingly, promotion of strategies to reduce readmission rates has become a key priority for payers and quality‐improvement organizations. These range from policy‐level attempts to stimulate change, such as publicly reporting hospital readmission rates on government websites, to establishing accreditation standardssuch as the Joint Commission's requirement to accurately reconcile medications, to the creation of quality improvement collaboratives focused on sharing best practices across institutions. Regardless of the approach taken, a valid, risk‐adjusted measure of performance is required to evaluate and track performance over time. The measure we have described meets the National Quality Forum's measure evaluation criteria in that it addresses an important clinical topic for which there appears to be significant opportunities for improvement, the measure is precisely defined and has been subjected to validity and reliability testing, it is risk‐adjusted based on patient clinical factors present at the start of care, is feasible to produce, and is understandable by a broad range of potential users.21 Because hospitalists are the physicians primarily responsible for the care of patients with pneumonia at US hospitals, and because they frequently serve as the physician champions for quality improvement activities related to pneumonia, it is especially important that they maintain a thorough understanding of the measures and methodologies underlying current efforts to measure hospital performance.

Several features of our approach warrant additional comment. First, we deliberately chose to measure all readmission events rather than attempt to discriminate between potentially preventable and nonpreventable readmissions. From the patient perspective, readmission for any reason is a concern, and limiting the measure to pneumonia‐related readmissions could make it susceptible to gaming by hospitals. Moreover, determining whether a readmission is related to a potential quality problem is not straightforward. For example, a patient with pneumonia whose discharge medications were prescribed incorrectly may be readmitted with a hip fracture following an episode of syncope. It would be inappropriate to treat this readmission as unrelated to the care the patient received for pneumonia. Additionally, while our approach does not presume that every readmission is preventable, the goal is to reduce the risk of readmissions generally (not just in narrowly defined subpopulations), and successful interventions to reduce rehospitalization have typically demonstrated reductions in all‐cause readmission.9, 22 Second, deaths that occurred within 30 days of discharge, yet that were not accompanied by a hospital readmission, were not counted as a readmission outcome. While it may seem inappropriate to treat a postdischarge death as a nonevent (rather than censoring or excluding such cases), alternative analytic approaches, such as using a hierarchical survival model, are not currently computationally feasible with large national data sets. Fortunately, only a relatively small proportion of discharges fell into this category (5.2% of index cases in the 2006 development sample died within 30 days of discharge without being readmitted). An alternative approach to handling the competing outcome of death would have been to use a composite outcome of readmission or death. However, we believe that it is important to report the outcomes separately because factors that predict readmission and mortality may differ, and when making comparisons across hospitals it would not be possible to determine whether differences in rate were due to readmission or mortality. Third, while the patient‐level readmission model showed only modest discrimination, we intentionally excluded covariates such as race and socioeconomic status, as well as in‐hospital events and potential complications of care, and whether patients were discharged home or to a skilled nursing facility. While these variables could have improved predictive ability, they may be directly or indirectly related to quality or supply factors that should not be included in a model that seeks to control for patient clinical characteristics. For example, if hospitals with a large share of poor patients have higher readmission rates, then including income in the model will obscure differences that are important to identify. While we believe that the decision to exclude such factors in the model is in the best interest of patients, and supports efforts to reduce health inequality in society more generally, we also recognize that hospitals that care for a disproportionate share of poor patients are likely to require additional resources to overcome these social factors. Fourth, we limited the analysis to patients with a principal diagnosis of pneumonia, and chose not to also include those with a principal diagnosis of sepsis or respiratory failure coupled with a secondary diagnosis of pneumonia. While the broader definition is used by CMS in the National Pneumonia Project, that initiative relied on chart abstraction to differentiate pneumonia present at the time of admission from cases developing as a complication of hospitalization. Additionally, we did not attempt to differentiate between community‐acquired and healthcare‐associated pneumonia, however our approach is consistent with the National Pneumonia Project and Pneumonia Patient Outcomes Research Team.18 Fifth, while our model estimates readmission rates at the hospital level, we recognize that readmissions are influenced by a complex and extensive range of factors. In this context, greater cooperation between hospitals and other care providers will almost certainly be required in order to achieve dramatic improvement in readmission rates, which in turn will depend upon changes to the way serious illness is paid for. Some options that have recently been described include imposing financial penalties for early readmission, extending the boundaries of case‐based payment beyond hospital discharge, and bundling payments between hospitals and physicians.2325

Our measure has several limitations. First, our models were developed and validated using Medicare data, and the results may not apply to pneumonia patients less than 65 years of age. However, most patients hospitalized with pneumonia in the US are 65 or older. In addition, we were unable to test the model with a Medicare managed care population, because data are not currently available on such patients. Finally, the medical record‐based validation was conducted by state‐level analysis because the sample size was insufficient to carry this out at the hospital level.

In conclusion, more than 17% of Medicare beneficiaries are readmitted within 30 days following discharge after a hospitalization for pneumonia, and rates vary substantially across institutions. The development of a valid measure of hospital performance and public reporting are important first steps towards focusing attention on this problem. Actual improvement will now depend on whether hospitals and partner organizations are successful at identifying and implementing effective methods to prevent readmission.

Hospital readmissions are emblematic of the numerous challenges facing the US health care system. Despite high levels of spending, nearly 20% of Medicare beneficiaries are readmitted within 30 days of hospital discharge, many readmissions are considered preventable, and rates vary widely by hospital and region.1 Further, while readmissions have been estimated to cost taxpayers as much as $17 billion annually, the current fee‐for‐service method of paying for the acute care needs of seniors rewards hospitals financially for readmission, not their prevention.2

Pneumonia is the second most common reason for hospitalization among Medicare beneficiaries, accounting for approximately 650,000 admissions annually,3 and has been a focus of national quality‐improvement efforts for more than a decade.4, 5 Despite improvements in key processes of care, rates of readmission within 30 days of discharge following a hospitalization for pneumonia have been reported to vary from 10% to 24%.68 Among several factors, readmissions are believed to be influenced by the quality of both inpatient and outpatient care, and by care‐coordination activities occurring in the transition from inpatient to outpatient status.912

Public reporting of hospital performance is considered a key strategy for improving quality, reducing costs, and increasing the value of hospital care, both in the US and worldwide.13 In 2009, the Centers for Medicare & Medicaid Services (CMS) expanded its reporting initiatives by adding risk‐adjusted hospital readmission rates for acute myocardial infarction, heart failure, and pneumonia to the Hospital Compare website.14, 15 Readmission rates are an attractive focus for public reporting for several reasons. First, in contrast to most process‐based measures of quality (eg, whether a patient with pneumonia received a particular antibiotic), a readmission is an adverse outcome that matters to patients and families.16 Second, unlike process measures whose assessment requires detailed review of medical records, readmissions can be easily determined from standard hospital claims. Finally, readmissions are costly, and their prevention could yield substantial savings to society.

A necessary prerequisite for public reporting of readmission is a validated, risk‐adjusted measure that can be used to track performance over time and can facilitate comparisons across institutions. Toward this end, we describe the development, validation, and results of a National Quality Forum‐approved and CMS‐adopted model to estimate hospital‐specific, risk‐standardized, 30‐day readmission rates for Medicare patients hospitalized with pneumonia.17

METHODS

RESULTS

Model Derivation and Performance

After exclusions were applied, the 2006 sample included 453,251 pneumonia hospitalizations (Figure 1). The development sample consisted of 226,545 hospitalizations at 4675 hospitals, with an overall unadjusted 30‐day readmission rate of 17.4%. In 11,694 index cases (5.2%), the patient died within 30 days without being readmitted. Median readmission rate was 16.3%, 25th and 75th percentile rates were 11.1% and 21.3%, and at the 10th and 90th percentile, hospital readmission rates ranged from 4.6% to 26.7% (Figure 2).

Figure 1

Figure 2

The claims model included 39 variables (age, sex, and 37 clinical variables) (Table 1). The mean age of the cohort was 80.0 years, with 55.5% women and 11.1% nonwhite patients. Mean observed readmission rate in the development sample ranged from 9% in the lowest decile of predicted pneumonia readmission rates to 32% in the highest predicted decile, a range of 23%. The AUC was 0.63. For comparison, a model with only age and sex had an AUC of 0.51, and a model with all candidate variables had an AUC equal to 0.63 (Table 2).

Table 2.Readmission Model Performance of Administrative Claims Models

		Calibration (0, 1)*	Discrimination		Residuals Lack of Fit (Pearson Residual Fall %)				Model 2 (No. of Covariates)
			Predictive Ability (Lowest Decile, Highest Decile)	AUC	(<2)	(2, 0)	(0, 2)	(2+)
NOTE: Over‐fitting indices (0, 1) provide evidence of over‐fitting and require several steps to calculate. Let b denote the estimated vector of regression coefficients. Predicted Probabilities (p) = 1/(1+exp{Xb}), and Z = Xb (eg, the linear predictor that is a scalar value for everyone). A new logistic regression model that includes only an intercept and a slope by regressing the logits on Z is fitted in the validation sample; eg, Logit(P(Y = 1|Z)) = 0 + 1Z. Estimated values of 0 far from 0 and estimated values of 1 far from 1 provide evidence of over‐fitting. Abbreviations: AUC, area under the receiver operating curve. * Max‐rescaled R‐square. Observed rates. Wald chi‐square.
Development sample
2006	(1st half) N = 226,545	(0, 1)	(0.09, 0.32)	0.63	0	82.62	7.39	9.99	6,843 (40)
Validation sample
2006	(2nd half) N = 226,706	(0.002, 0.997)	(0.09, 0.31)	0.63	0	82.55	7.45	9.99	6,870 (40)
2005	N = 536,015	(0.035, 1.008)	(0.08, 0.31)	0.63	0	82.67	7.31	10.03	16,241 (40)

Hospital Risk‐Standardized Readmission Rates

Risk‐standardized readmission rates varied across hospitals (Figure 3). Median risk‐standardized readmission rate was 17.3%, and the 25th and 75th percentiles were 16.9% and 17.9%, respectively. The 5th percentile was 16.0% and the 95th percentile was 19.1%. Odds of readmission for a hospital one standard deviation above average was 1.4 times that of a hospital one standard deviation below average.

Figure 3

Medical Record Validation

After exclusions, the medical record sample taken from the National Pneumonia Project included 47,429 cases, with an unadjusted 30‐day readmission rate of 17.0%. The final medical record risk‐adjustment model included a total of 35 variables, whose prevalence and association with readmission risk varied modestly (Table 3). Performance of the medical record and administrative models was similar (areas under the ROC curve 0.59 and 0.63, respectively) (Table 4). Additionally, in the administrative model, predicted readmission rates ranged from 8% in the lowest predicted decile to 30% in the highest predicted decile, while in the medical record model, the corresponding rates varied from 10% to 26%.

Table 3.Regression Model Results from Medical Record Sample

Variable	Percent	Estimate	Standard Error	Odds Ratio	95% CI
NOTE: Between‐state variance = 0.024; standard error = 0.00. Abbreviations: BP, blood pressure; BUN, blood urea nitrogen; CI, confidence interval; SD, standard deviation; WBC, white blood cell count.
Age 65, mean (SD)	15.24 (7.87)	0.003	0.002	0.997	0.993	1.000
Male	46.18	0.122	0.025	1.130	1.075	1.188
Nursing home resident	17.71	0.035	0.037	1.036	0.963	1.114
Neoplastic disease	6.80	0.130	0.049	1.139	1.034	1.254
Liver disease	1.04	0.089	0.123	0.915	0.719	1.164
History of heart failure	28.98	0.234	0.029	1.264	1.194	1.339
History of renal disease	8.51	0.188	0.047	1.206	1.100	1.323
Altered mental status	17.95	0.009	0.034	1.009	0.944	1.080
Pleural effusion	21.20	0.165	0.030	1.179	1.111	1.251
BUN 30 mg/dl	23.28	0.160	0.033	1.174	1.100	1.252
BUN missing	14.56	0.101	0.185	0.904	0.630	1.298
Systolic BP <90 mmHg	2.95	0.068	0.070	1.070	0.932	1.228
Systolic BP missing	11.21	0.149	0.425	1.160	0.504	2.669
Pulse 125/min	7.73	0.036	0.047	1.036	0.945	1.137
Pulse missing	11.22	0.210	0.405	1.234	0.558	2.729
Respiratory rate 30/min	16.38	0.079	0.034	1.082	1.012	1.157
Respiratory rate missing	11.39	0.204	0.240	1.226	0.765	1.964
Sodium <130 mmol/L	4.82	0.136	0.057	1.145	1.025	1.280
Sodium missing	14.39	0.049	0.143	1.050	0.793	1.391
Glucose 250 mg/dl	5.19	0.005	0.057	0.995	0.889	1.114
Glucose missing	15.44	0.156	0.105	0.855	0.696	1.051
Hematocrit <30%	7.77	0.270	0.044	1.310	1.202	1.428
Hematocrit missing	13.62	0.071	0.135	0.932	0.715	1.215
Creatinine 2.5 mg/dL	4.68	0.109	0.062	1.115	0.989	1.258
Creatinine missing	14.63	0.200	0.167	1.221	0.880	1.695
WBC 6‐12 b/L	38.04	0.021	0.049	0.979	0.889	1.079
WBC >12 b/L	41.45	0.068	0.049	0.934	0.848	1.029
WBC missing	12.85	0.167	0.162	1.181	0.860	1.623
Immunosuppressive therapy	15.01	0.347	0.035	1.415	1.321	1.516
Chronic lung disease	42.16	0.137	0.028	1.147	1.086	1.211
Coronary artery disease	39.57	0.150	0.028	1.162	1.100	1.227
Diabetes mellitus	20.90	0.137	0.033	1.147	1.076	1.223
Alcohol/drug abuse	3.40	0.099	0.071	0.906	0.788	1.041
Dementia/Alzheimer's disease	16.38	0.125	0.038	1.133	1.052	1.222
Splenectomy	0.44	0.016	0.186	1.016	0.706	1.463

Table 4.Model Performance of Medical Record Model

Model	Calibration (0, 1)*	Discrimination		Residuals Lack of Fit (Pearson Residual Fall %)				Model 2 (No. of Covariates)
		Predictive Ability (Lowest Decile, Highest Decile)	AUC	(<2)	(2, 0)	(0, 2)	(2+)
Abbreviations: AUC, area under the receiver operating curve. * Max‐rescaled R‐square. Observed rates. Wald chi‐square.
Medical Record Model Development Sample (NP)
N = 47,429 No. of 30‐day readmissions = 8,042	(1, 0)	(0.10, 0.26)	0.59	0	83.04	5.28	11.68	710 (35)
Linked Administrative Model Validation Sample
N = 47,429 No. of 30‐day readmissions = 8,042	(1, 0)	(0.08, 0.30)	0.63	0	83.04	6.94	10.01	1,414 (40)

The correlation coefficient of the estimated state‐specific standardized readmission rates from the administrative and medical record models was 0.96, and the proportion of the variance explained by the model was 0.92 (Figure 4).

Figure 4

DISCUSSION

We have described the development, validation, and results of a hospital, 30‐day, risk‐standardized readmission model for pneumonia that was created to support current federal transparency initiatives. The model uses administrative claims data from Medicare fee‐for‐service patients and produces results that are comparable to a model based on information obtained through manual abstraction of medical records. We observed an overall 30‐day readmission rate of 17%, and our analyses revealed substantial variation across US hospitals, suggesting that improvement by lower performing institutions is an achievable goal.

Because more than one in six pneumonia patients are rehospitalized shortly after discharge, and because pneumonia hospitalizations represent an enormous expense to the Medicare program, prevention of readmissions is now widely recognized to offer a substantial opportunity to improve patient outcomes while simultaneously lowering health care costs. Accordingly, promotion of strategies to reduce readmission rates has become a key priority for payers and quality‐improvement organizations. These range from policy‐level attempts to stimulate change, such as publicly reporting hospital readmission rates on government websites, to establishing accreditation standardssuch as the Joint Commission's requirement to accurately reconcile medications, to the creation of quality improvement collaboratives focused on sharing best practices across institutions. Regardless of the approach taken, a valid, risk‐adjusted measure of performance is required to evaluate and track performance over time. The measure we have described meets the National Quality Forum's measure evaluation criteria in that it addresses an important clinical topic for which there appears to be significant opportunities for improvement, the measure is precisely defined and has been subjected to validity and reliability testing, it is risk‐adjusted based on patient clinical factors present at the start of care, is feasible to produce, and is understandable by a broad range of potential users.21 Because hospitalists are the physicians primarily responsible for the care of patients with pneumonia at US hospitals, and because they frequently serve as the physician champions for quality improvement activities related to pneumonia, it is especially important that they maintain a thorough understanding of the measures and methodologies underlying current efforts to measure hospital performance.

Several features of our approach warrant additional comment. First, we deliberately chose to measure all readmission events rather than attempt to discriminate between potentially preventable and nonpreventable readmissions. From the patient perspective, readmission for any reason is a concern, and limiting the measure to pneumonia‐related readmissions could make it susceptible to gaming by hospitals. Moreover, determining whether a readmission is related to a potential quality problem is not straightforward. For example, a patient with pneumonia whose discharge medications were prescribed incorrectly may be readmitted with a hip fracture following an episode of syncope. It would be inappropriate to treat this readmission as unrelated to the care the patient received for pneumonia. Additionally, while our approach does not presume that every readmission is preventable, the goal is to reduce the risk of readmissions generally (not just in narrowly defined subpopulations), and successful interventions to reduce rehospitalization have typically demonstrated reductions in all‐cause readmission.9, 22 Second, deaths that occurred within 30 days of discharge, yet that were not accompanied by a hospital readmission, were not counted as a readmission outcome. While it may seem inappropriate to treat a postdischarge death as a nonevent (rather than censoring or excluding such cases), alternative analytic approaches, such as using a hierarchical survival model, are not currently computationally feasible with large national data sets. Fortunately, only a relatively small proportion of discharges fell into this category (5.2% of index cases in the 2006 development sample died within 30 days of discharge without being readmitted). An alternative approach to handling the competing outcome of death would have been to use a composite outcome of readmission or death. However, we believe that it is important to report the outcomes separately because factors that predict readmission and mortality may differ, and when making comparisons across hospitals it would not be possible to determine whether differences in rate were due to readmission or mortality. Third, while the patient‐level readmission model showed only modest discrimination, we intentionally excluded covariates such as race and socioeconomic status, as well as in‐hospital events and potential complications of care, and whether patients were discharged home or to a skilled nursing facility. While these variables could have improved predictive ability, they may be directly or indirectly related to quality or supply factors that should not be included in a model that seeks to control for patient clinical characteristics. For example, if hospitals with a large share of poor patients have higher readmission rates, then including income in the model will obscure differences that are important to identify. While we believe that the decision to exclude such factors in the model is in the best interest of patients, and supports efforts to reduce health inequality in society more generally, we also recognize that hospitals that care for a disproportionate share of poor patients are likely to require additional resources to overcome these social factors. Fourth, we limited the analysis to patients with a principal diagnosis of pneumonia, and chose not to also include those with a principal diagnosis of sepsis or respiratory failure coupled with a secondary diagnosis of pneumonia. While the broader definition is used by CMS in the National Pneumonia Project, that initiative relied on chart abstraction to differentiate pneumonia present at the time of admission from cases developing as a complication of hospitalization. Additionally, we did not attempt to differentiate between community‐acquired and healthcare‐associated pneumonia, however our approach is consistent with the National Pneumonia Project and Pneumonia Patient Outcomes Research Team.18 Fifth, while our model estimates readmission rates at the hospital level, we recognize that readmissions are influenced by a complex and extensive range of factors. In this context, greater cooperation between hospitals and other care providers will almost certainly be required in order to achieve dramatic improvement in readmission rates, which in turn will depend upon changes to the way serious illness is paid for. Some options that have recently been described include imposing financial penalties for early readmission, extending the boundaries of case‐based payment beyond hospital discharge, and bundling payments between hospitals and physicians.2325

Our measure has several limitations. First, our models were developed and validated using Medicare data, and the results may not apply to pneumonia patients less than 65 years of age. However, most patients hospitalized with pneumonia in the US are 65 or older. In addition, we were unable to test the model with a Medicare managed care population, because data are not currently available on such patients. Finally, the medical record‐based validation was conducted by state‐level analysis because the sample size was insufficient to carry this out at the hospital level.

In conclusion, more than 17% of Medicare beneficiaries are readmitted within 30 days following discharge after a hospitalization for pneumonia, and rates vary substantially across institutions. The development of a valid measure of hospital performance and public reporting are important first steps towards focusing attention on this problem. Actual improvement will now depend on whether hospitals and partner organizations are successful at identifying and implementing effective methods to prevent readmission.

Hospital readmissions are emblematic of the numerous challenges facing the US health care system. Despite high levels of spending, nearly 20% of Medicare beneficiaries are readmitted within 30 days of hospital discharge, many readmissions are considered preventable, and rates vary widely by hospital and region.1 Further, while readmissions have been estimated to cost taxpayers as much as $17 billion annually, the current fee‐for‐service method of paying for the acute care needs of seniors rewards hospitals financially for readmission, not their prevention.2

Pneumonia is the second most common reason for hospitalization among Medicare beneficiaries, accounting for approximately 650,000 admissions annually,3 and has been a focus of national quality‐improvement efforts for more than a decade.4, 5 Despite improvements in key processes of care, rates of readmission within 30 days of discharge following a hospitalization for pneumonia have been reported to vary from 10% to 24%.68 Among several factors, readmissions are believed to be influenced by the quality of both inpatient and outpatient care, and by care‐coordination activities occurring in the transition from inpatient to outpatient status.912

Public reporting of hospital performance is considered a key strategy for improving quality, reducing costs, and increasing the value of hospital care, both in the US and worldwide.13 In 2009, the Centers for Medicare & Medicaid Services (CMS) expanded its reporting initiatives by adding risk‐adjusted hospital readmission rates for acute myocardial infarction, heart failure, and pneumonia to the Hospital Compare website.14, 15 Readmission rates are an attractive focus for public reporting for several reasons. First, in contrast to most process‐based measures of quality (eg, whether a patient with pneumonia received a particular antibiotic), a readmission is an adverse outcome that matters to patients and families.16 Second, unlike process measures whose assessment requires detailed review of medical records, readmissions can be easily determined from standard hospital claims. Finally, readmissions are costly, and their prevention could yield substantial savings to society.

A necessary prerequisite for public reporting of readmission is a validated, risk‐adjusted measure that can be used to track performance over time and can facilitate comparisons across institutions. Toward this end, we describe the development, validation, and results of a National Quality Forum‐approved and CMS‐adopted model to estimate hospital‐specific, risk‐standardized, 30‐day readmission rates for Medicare patients hospitalized with pneumonia.17

METHODS

RESULTS

Model Derivation and Performance

After exclusions were applied, the 2006 sample included 453,251 pneumonia hospitalizations (Figure 1). The development sample consisted of 226,545 hospitalizations at 4675 hospitals, with an overall unadjusted 30‐day readmission rate of 17.4%. In 11,694 index cases (5.2%), the patient died within 30 days without being readmitted. Median readmission rate was 16.3%, 25th and 75th percentile rates were 11.1% and 21.3%, and at the 10th and 90th percentile, hospital readmission rates ranged from 4.6% to 26.7% (Figure 2).

Figure 1

Figure 2

The claims model included 39 variables (age, sex, and 37 clinical variables) (Table 1). The mean age of the cohort was 80.0 years, with 55.5% women and 11.1% nonwhite patients. Mean observed readmission rate in the development sample ranged from 9% in the lowest decile of predicted pneumonia readmission rates to 32% in the highest predicted decile, a range of 23%. The AUC was 0.63. For comparison, a model with only age and sex had an AUC of 0.51, and a model with all candidate variables had an AUC equal to 0.63 (Table 2).

Table 2.Readmission Model Performance of Administrative Claims Models

		Calibration (0, 1)*	Discrimination		Residuals Lack of Fit (Pearson Residual Fall %)				Model 2 (No. of Covariates)
			Predictive Ability (Lowest Decile, Highest Decile)	AUC	(<2)	(2, 0)	(0, 2)	(2+)
NOTE: Over‐fitting indices (0, 1) provide evidence of over‐fitting and require several steps to calculate. Let b denote the estimated vector of regression coefficients. Predicted Probabilities (p) = 1/(1+exp{Xb}), and Z = Xb (eg, the linear predictor that is a scalar value for everyone). A new logistic regression model that includes only an intercept and a slope by regressing the logits on Z is fitted in the validation sample; eg, Logit(P(Y = 1|Z)) = 0 + 1Z. Estimated values of 0 far from 0 and estimated values of 1 far from 1 provide evidence of over‐fitting. Abbreviations: AUC, area under the receiver operating curve. * Max‐rescaled R‐square. Observed rates. Wald chi‐square.
Development sample
2006	(1st half) N = 226,545	(0, 1)	(0.09, 0.32)	0.63	0	82.62	7.39	9.99	6,843 (40)
Validation sample
2006	(2nd half) N = 226,706	(0.002, 0.997)	(0.09, 0.31)	0.63	0	82.55	7.45	9.99	6,870 (40)
2005	N = 536,015	(0.035, 1.008)	(0.08, 0.31)	0.63	0	82.67	7.31	10.03	16,241 (40)

Hospital Risk‐Standardized Readmission Rates

Risk‐standardized readmission rates varied across hospitals (Figure 3). Median risk‐standardized readmission rate was 17.3%, and the 25th and 75th percentiles were 16.9% and 17.9%, respectively. The 5th percentile was 16.0% and the 95th percentile was 19.1%. Odds of readmission for a hospital one standard deviation above average was 1.4 times that of a hospital one standard deviation below average.

Figure 3

Medical Record Validation

After exclusions, the medical record sample taken from the National Pneumonia Project included 47,429 cases, with an unadjusted 30‐day readmission rate of 17.0%. The final medical record risk‐adjustment model included a total of 35 variables, whose prevalence and association with readmission risk varied modestly (Table 3). Performance of the medical record and administrative models was similar (areas under the ROC curve 0.59 and 0.63, respectively) (Table 4). Additionally, in the administrative model, predicted readmission rates ranged from 8% in the lowest predicted decile to 30% in the highest predicted decile, while in the medical record model, the corresponding rates varied from 10% to 26%.

Table 3.Regression Model Results from Medical Record Sample

Variable	Percent	Estimate	Standard Error	Odds Ratio	95% CI
NOTE: Between‐state variance = 0.024; standard error = 0.00. Abbreviations: BP, blood pressure; BUN, blood urea nitrogen; CI, confidence interval; SD, standard deviation; WBC, white blood cell count.
Age 65, mean (SD)	15.24 (7.87)	0.003	0.002	0.997	0.993	1.000
Male	46.18	0.122	0.025	1.130	1.075	1.188
Nursing home resident	17.71	0.035	0.037	1.036	0.963	1.114
Neoplastic disease	6.80	0.130	0.049	1.139	1.034	1.254
Liver disease	1.04	0.089	0.123	0.915	0.719	1.164
History of heart failure	28.98	0.234	0.029	1.264	1.194	1.339
History of renal disease	8.51	0.188	0.047	1.206	1.100	1.323
Altered mental status	17.95	0.009	0.034	1.009	0.944	1.080
Pleural effusion	21.20	0.165	0.030	1.179	1.111	1.251
BUN 30 mg/dl	23.28	0.160	0.033	1.174	1.100	1.252
BUN missing	14.56	0.101	0.185	0.904	0.630	1.298
Systolic BP <90 mmHg	2.95	0.068	0.070	1.070	0.932	1.228
Systolic BP missing	11.21	0.149	0.425	1.160	0.504	2.669
Pulse 125/min	7.73	0.036	0.047	1.036	0.945	1.137
Pulse missing	11.22	0.210	0.405	1.234	0.558	2.729
Respiratory rate 30/min	16.38	0.079	0.034	1.082	1.012	1.157
Respiratory rate missing	11.39	0.204	0.240	1.226	0.765	1.964
Sodium <130 mmol/L	4.82	0.136	0.057	1.145	1.025	1.280
Sodium missing	14.39	0.049	0.143	1.050	0.793	1.391
Glucose 250 mg/dl	5.19	0.005	0.057	0.995	0.889	1.114
Glucose missing	15.44	0.156	0.105	0.855	0.696	1.051
Hematocrit <30%	7.77	0.270	0.044	1.310	1.202	1.428
Hematocrit missing	13.62	0.071	0.135	0.932	0.715	1.215
Creatinine 2.5 mg/dL	4.68	0.109	0.062	1.115	0.989	1.258
Creatinine missing	14.63	0.200	0.167	1.221	0.880	1.695
WBC 6‐12 b/L	38.04	0.021	0.049	0.979	0.889	1.079
WBC >12 b/L	41.45	0.068	0.049	0.934	0.848	1.029
WBC missing	12.85	0.167	0.162	1.181	0.860	1.623
Immunosuppressive therapy	15.01	0.347	0.035	1.415	1.321	1.516
Chronic lung disease	42.16	0.137	0.028	1.147	1.086	1.211
Coronary artery disease	39.57	0.150	0.028	1.162	1.100	1.227
Diabetes mellitus	20.90	0.137	0.033	1.147	1.076	1.223
Alcohol/drug abuse	3.40	0.099	0.071	0.906	0.788	1.041
Dementia/Alzheimer's disease	16.38	0.125	0.038	1.133	1.052	1.222
Splenectomy	0.44	0.016	0.186	1.016	0.706	1.463

Table 4.Model Performance of Medical Record Model

Model	Calibration (0, 1)*	Discrimination		Residuals Lack of Fit (Pearson Residual Fall %)				Model 2 (No. of Covariates)
		Predictive Ability (Lowest Decile, Highest Decile)	AUC	(<2)	(2, 0)	(0, 2)	(2+)
Abbreviations: AUC, area under the receiver operating curve. * Max‐rescaled R‐square. Observed rates. Wald chi‐square.
Medical Record Model Development Sample (NP)
N = 47,429 No. of 30‐day readmissions = 8,042	(1, 0)	(0.10, 0.26)	0.59	0	83.04	5.28	11.68	710 (35)
Linked Administrative Model Validation Sample
N = 47,429 No. of 30‐day readmissions = 8,042	(1, 0)	(0.08, 0.30)	0.63	0	83.04	6.94	10.01	1,414 (40)

The correlation coefficient of the estimated state‐specific standardized readmission rates from the administrative and medical record models was 0.96, and the proportion of the variance explained by the model was 0.92 (Figure 4).

Figure 4

DISCUSSION

We have described the development, validation, and results of a hospital, 30‐day, risk‐standardized readmission model for pneumonia that was created to support current federal transparency initiatives. The model uses administrative claims data from Medicare fee‐for‐service patients and produces results that are comparable to a model based on information obtained through manual abstraction of medical records. We observed an overall 30‐day readmission rate of 17%, and our analyses revealed substantial variation across US hospitals, suggesting that improvement by lower performing institutions is an achievable goal.

Because more than one in six pneumonia patients are rehospitalized shortly after discharge, and because pneumonia hospitalizations represent an enormous expense to the Medicare program, prevention of readmissions is now widely recognized to offer a substantial opportunity to improve patient outcomes while simultaneously lowering health care costs. Accordingly, promotion of strategies to reduce readmission rates has become a key priority for payers and quality‐improvement organizations. These range from policy‐level attempts to stimulate change, such as publicly reporting hospital readmission rates on government websites, to establishing accreditation standardssuch as the Joint Commission's requirement to accurately reconcile medications, to the creation of quality improvement collaboratives focused on sharing best practices across institutions. Regardless of the approach taken, a valid, risk‐adjusted measure of performance is required to evaluate and track performance over time. The measure we have described meets the National Quality Forum's measure evaluation criteria in that it addresses an important clinical topic for which there appears to be significant opportunities for improvement, the measure is precisely defined and has been subjected to validity and reliability testing, it is risk‐adjusted based on patient clinical factors present at the start of care, is feasible to produce, and is understandable by a broad range of potential users.21 Because hospitalists are the physicians primarily responsible for the care of patients with pneumonia at US hospitals, and because they frequently serve as the physician champions for quality improvement activities related to pneumonia, it is especially important that they maintain a thorough understanding of the measures and methodologies underlying current efforts to measure hospital performance.

Several features of our approach warrant additional comment. First, we deliberately chose to measure all readmission events rather than attempt to discriminate between potentially preventable and nonpreventable readmissions. From the patient perspective, readmission for any reason is a concern, and limiting the measure to pneumonia‐related readmissions could make it susceptible to gaming by hospitals. Moreover, determining whether a readmission is related to a potential quality problem is not straightforward. For example, a patient with pneumonia whose discharge medications were prescribed incorrectly may be readmitted with a hip fracture following an episode of syncope. It would be inappropriate to treat this readmission as unrelated to the care the patient received for pneumonia. Additionally, while our approach does not presume that every readmission is preventable, the goal is to reduce the risk of readmissions generally (not just in narrowly defined subpopulations), and successful interventions to reduce rehospitalization have typically demonstrated reductions in all‐cause readmission.9, 22 Second, deaths that occurred within 30 days of discharge, yet that were not accompanied by a hospital readmission, were not counted as a readmission outcome. While it may seem inappropriate to treat a postdischarge death as a nonevent (rather than censoring or excluding such cases), alternative analytic approaches, such as using a hierarchical survival model, are not currently computationally feasible with large national data sets. Fortunately, only a relatively small proportion of discharges fell into this category (5.2% of index cases in the 2006 development sample died within 30 days of discharge without being readmitted). An alternative approach to handling the competing outcome of death would have been to use a composite outcome of readmission or death. However, we believe that it is important to report the outcomes separately because factors that predict readmission and mortality may differ, and when making comparisons across hospitals it would not be possible to determine whether differences in rate were due to readmission or mortality. Third, while the patient‐level readmission model showed only modest discrimination, we intentionally excluded covariates such as race and socioeconomic status, as well as in‐hospital events and potential complications of care, and whether patients were discharged home or to a skilled nursing facility. While these variables could have improved predictive ability, they may be directly or indirectly related to quality or supply factors that should not be included in a model that seeks to control for patient clinical characteristics. For example, if hospitals with a large share of poor patients have higher readmission rates, then including income in the model will obscure differences that are important to identify. While we believe that the decision to exclude such factors in the model is in the best interest of patients, and supports efforts to reduce health inequality in society more generally, we also recognize that hospitals that care for a disproportionate share of poor patients are likely to require additional resources to overcome these social factors. Fourth, we limited the analysis to patients with a principal diagnosis of pneumonia, and chose not to also include those with a principal diagnosis of sepsis or respiratory failure coupled with a secondary diagnosis of pneumonia. While the broader definition is used by CMS in the National Pneumonia Project, that initiative relied on chart abstraction to differentiate pneumonia present at the time of admission from cases developing as a complication of hospitalization. Additionally, we did not attempt to differentiate between community‐acquired and healthcare‐associated pneumonia, however our approach is consistent with the National Pneumonia Project and Pneumonia Patient Outcomes Research Team.18 Fifth, while our model estimates readmission rates at the hospital level, we recognize that readmissions are influenced by a complex and extensive range of factors. In this context, greater cooperation between hospitals and other care providers will almost certainly be required in order to achieve dramatic improvement in readmission rates, which in turn will depend upon changes to the way serious illness is paid for. Some options that have recently been described include imposing financial penalties for early readmission, extending the boundaries of case‐based payment beyond hospital discharge, and bundling payments between hospitals and physicians.2325

Our measure has several limitations. First, our models were developed and validated using Medicare data, and the results may not apply to pneumonia patients less than 65 years of age. However, most patients hospitalized with pneumonia in the US are 65 or older. In addition, we were unable to test the model with a Medicare managed care population, because data are not currently available on such patients. Finally, the medical record‐based validation was conducted by state‐level analysis because the sample size was insufficient to carry this out at the hospital level.

In conclusion, more than 17% of Medicare beneficiaries are readmitted within 30 days following discharge after a hospitalization for pneumonia, and rates vary substantially across institutions. The development of a valid measure of hospital performance and public reporting are important first steps towards focusing attention on this problem. Actual improvement will now depend on whether hospitals and partner organizations are successful at identifying and implementing effective methods to prevent readmission.

Acute exacerbations of chronic obstructive pulmonary disease (AECOPD) are responsible for more than 600,000 hospitalizations annually, resulting in direct costs of over $20 billion.1 Bacterial infections appear responsible for 50% of such exacerbations,25 and current COPD guidelines recommend treatment with antibiotics for patients with severe exacerbations or a change in sputum.1^,69 These recommendations are based on a number of small randomized trials, most of which were conducted more than 20 years ago using narrow spectrum antibiotics that are no longer commonly used.10 Only 4 studies, totaling 321 subjects, included hospitalized patients, and most studies excluded patients who required steroids. Because no clinical trials have compared different antibiotic regimens for AECOPD, existing guidelines offer a range of treatment options, including amoxicillin‐clavulonate, macrolides, quinolones, cephalosporins, aminopenicillins, and tetracyclines.

Among hospitalized patients, macrolides and quinolones appear to be the most frequently prescribed antibiotics.11 Both are available in oral formulations, have excellent bioavailability, and are administered once daily. In addition to their antimicrobial activity, macrolides are believed to have antiinflammatory effects, which could be especially advantageous in AECOPD.1214 In trials of chronic bronchitis, however, fluoroquinolones have been shown to reduce the risk of recurrent exacerbation when compared to macrolides.15 The wide variation that has been observed in antibiotic selection for patients hospitalized for AECOPD suggests a high degree of uncertainty among clinicians about the benefits of different treatment options.11 Given the limited evidence from randomized trials, we sought to evaluate the comparative effectiveness of macrolides and quinolones among a large, representative sample of patients hospitalized with AECOPD.

Subjects and Methods

Results

Of 26,248 AECOPD patients treated with antibiotics, 19,608 patients met the inclusion criteria; of these, 6139 (31%) were treated initially with a macrolide; the median age was 70 years; 60% were female; and 78% were white. A total of 86% of patients had a primary diagnosis of obstructive chronic bronchitis with acute exacerbation, and 6% had respiratory failure. The most common comorbidities were hypertension, diabetes, and congestive heart failure. Twenty‐two percent had been admitted at least once in the preceding 12 months. Treatment failure occurred in 7.7% of patients, and 1.3% died in the hospital. Mean length of stay was 4.8 days. Hospital prescribing rates for macrolides varied from 0% to 100%, with a mean of 33% and an interquartile range of 14% to 46% (Supporting Appendix Figure 1).

Compared to patients receiving macrolides, those receiving quinolones were older, more likely to have respiratory failure, to be cared for by a pulmonologist, and to have an admission in the previous year (Table 1). They were also more likely to be treated with bronchodilators, methylxanthines, steroids, diuretics, and noninvasive positive pressure ventilation, and to have an arterial blood gas, but less likely to receive concomitant treatment with a cephalosporin (11% vs. 57%). With the exception of cephalosporin treatment, these differences were small, but due to the large sample were statistically significant. Comorbidities were similar in both groups. Patients in the quinolone group were also more likely to experience treatment failure (8.1% vs. 6.8%), death (1.5% vs. 1.0%), and antibiotic‐associated diarrhea (1.1% vs. 0.5%).

In the unadjusted analysis, compared to patients receiving quinolones, those treated with macrolides were less likely to experience treatment failure (OR, 0.83; 95% CI, 0.740.94) (Table 2). Adjusting for all patient, hospital, and physician covariates, including the propensity for treatment with macrolides, increased the OR to 0.89 and the results were no longer significant (95% CI, 0.781.01). Propensity matching successfully balanced all measured covariates except for the use of short‐acting bronchodilators and additional antibiotics (Table 1). In the propensity‐matched sample (Figure 1), quinolone‐treated patients were more likely to experience antibiotic‐associated diarrhea (1.2% vs. 0.6%; P = 0.0003) and late mechanical ventilation (1.3% vs. 0.8%; P = 0.02). There were no differences in adjusted cost or length of stay between the 2 groups. The results of the grouped treatment analysis, substituting the hospital's specific rate of macrolide use in place of the actual treatment that each patient received suggested that the 2 antibiotics were associated with similar rates of treatment failure. The OR for a 100% hospital rate of macrolide treatment vs. a 0% rate was 1.01 (95% CI, 0.751.35).

Figure 1

Discussion

In this large observational study conducted at 375 hospitals, we took advantage of a natural experiment in which antibiotic prescribing patterns varied widely across hospitals to compare the effectiveness of 2 common antibiotic regimens for AECOPD. Treatment with macrolides and quinolones were associated with a similar risk of treatment failure, costs, and length of stay; however, patients treated with macrolides were less likely to experience late mechanical ventilation or treatment for antibiotic‐associated diarrhea.

Despite broad consensus in COPD guidelines that patients with severe acute exacerbations should receive antibiotics, there is little agreement about the preferred empiric agent. Controversy exists regarding antibiotics' comparative effectiveness, and even over which pathogens cause COPD exacerbations. Given the frequency of hospitalization for AECOPD, understanding the comparative effectiveness of treatments in this setting could have important implications for health outcomes and costs. Unfortunately, most antibiotic studies in AECOPD were conducted >20 years ago, using antibiotics that rarely appeared in our sample.26 Consequently, clinical practice guidelines offer conflicting recommendations. For example, the National Institute for Clinical Excellence recommends empirical treatment with an aminopenicillin, a macrolide, or a tetracycline,8 while the American Thoracic Society recommends amoxicillin‐clavulonate or a fluoroquinolone.9

As might be expected in light of so much uncertainty, we found wide variation in prescribing patterns across hospitals. Overall, approximately one‐third of patients received a macrolide and two‐thirds a quinolone. Both regimens provide adequate coverage of H. influenza, S. pneumoniae, and M. catarrhalis, and conform to at least 1 COPD guideline. Nevertheless, patients receiving macrolides often received a cephalosporin as well; this pattern of treatment suggests that antibiotic selection is likely to have been influenced more by guidelines for the treatment of community‐acquired pneumonia than COPD.

Previous studies comparing antibiotic effectiveness suffer from shortcomings that limit their application to patients hospitalized with AECOPD. First, they enrolled patients with chronic bronchitis, and included patients without obstructive lung disease, and most studies included patients as young as 18 years old. Second, many either did not include treatment with steroids or excluded patients receiving more than 10 mg of prednisone daily. Third, almost all enrolled only ambulatory patients.

While there are no studies comparing quinolones and macrolides in patients hospitalized for AECOPD, a meta‐analysis comparing quinolones, macrolides, and amoxicillin‐clavulonate identified 19 trials of ambulatory patients with chronic bronchitis. That study found that all 3 drugs had similar efficacy initially, but that quinolones resulted in the fewest relapses over a 26‐week period.27 Macrolides and quinolones had similar rates of adverse effects. In contrast, we did not find a difference in treatment failure, cost, or length of stay, but did find a higher rate of diarrhea associated with quinolones. Others have also documented an association between fluoroquinolones and C. difficile diarrhea.2830 This trend, first noted in 2001, is of particular concern because the fluoroquinolone‐resistant strains appear to be hypervirulent and have been associated with nosocomial epidemics.3134

Our study has several limitations. First, its observational design leaves open the possibility of selection bias. For this reason we analyzed our data in several ways, including using a grouped treatment approach, an adaptation of the instrumental variable technique, and accepted only those differences which were consistent across all models. Second, our study used claims data, and therefore we could not directly adjust for physiological measures of severity. However, the highly detailed nature of the data allowed us to adjust for numerous tests and treatments that reflected the clinician's assessment of the patient's severity, as well as the number of prior COPD admissions. Third, we cannot exclude the possibility that some patients may have had concurrent pneumonia without an ICD‐9 code. We think that the number would be small because reimbursement for pneumonia is generally higher than for COPD, so hospitals have an incentive to code pneumonia as the principal diagnosis when present. Finally, we compared initial antibiotics only. More than one‐quarter of our patients received an additional antibiotic before discharge. In particular, patients receiving macrolides were often prescribed a concomitant cephalosporin. We do not know to what extent these additional antibiotics may have affected the outcomes.

Despite the large number of patients hospitalized annually for AECOPD, there are no randomized trials comparing different antibiotics in this population. Studies comparing antibiotics in chronic bronchitis can offer little guidance, since they have primarily focused on proving equivalence between existing antibiotics and newer, more expensive formulations.35 Because many of the patients enrolled in such trials do not benefit from antibiotics at all, either because they do not have COPD or because their exacerbation is not caused by bacteria, it is relatively easy to prove equivalence. Given that AECOPD is one of the leading causes of hospitalization in the United States, large, randomized trials comparing the effectiveness of different antibiotics should be a high priority. In the meantime, macrolides (often given together with cephalosporins) and quinolones appear to be equally effective initial antibiotic choices; considering antibiotic‐associated diarrhea, macrolides appear to be the safer of the 2.

Acute exacerbations of chronic obstructive pulmonary disease (AECOPD) are responsible for more than 600,000 hospitalizations annually, resulting in direct costs of over $20 billion.1 Bacterial infections appear responsible for 50% of such exacerbations,25 and current COPD guidelines recommend treatment with antibiotics for patients with severe exacerbations or a change in sputum.1^,69 These recommendations are based on a number of small randomized trials, most of which were conducted more than 20 years ago using narrow spectrum antibiotics that are no longer commonly used.10 Only 4 studies, totaling 321 subjects, included hospitalized patients, and most studies excluded patients who required steroids. Because no clinical trials have compared different antibiotic regimens for AECOPD, existing guidelines offer a range of treatment options, including amoxicillin‐clavulonate, macrolides, quinolones, cephalosporins, aminopenicillins, and tetracyclines.

Among hospitalized patients, macrolides and quinolones appear to be the most frequently prescribed antibiotics.11 Both are available in oral formulations, have excellent bioavailability, and are administered once daily. In addition to their antimicrobial activity, macrolides are believed to have antiinflammatory effects, which could be especially advantageous in AECOPD.1214 In trials of chronic bronchitis, however, fluoroquinolones have been shown to reduce the risk of recurrent exacerbation when compared to macrolides.15 The wide variation that has been observed in antibiotic selection for patients hospitalized for AECOPD suggests a high degree of uncertainty among clinicians about the benefits of different treatment options.11 Given the limited evidence from randomized trials, we sought to evaluate the comparative effectiveness of macrolides and quinolones among a large, representative sample of patients hospitalized with AECOPD.

Subjects and Methods

Results

Of 26,248 AECOPD patients treated with antibiotics, 19,608 patients met the inclusion criteria; of these, 6139 (31%) were treated initially with a macrolide; the median age was 70 years; 60% were female; and 78% were white. A total of 86% of patients had a primary diagnosis of obstructive chronic bronchitis with acute exacerbation, and 6% had respiratory failure. The most common comorbidities were hypertension, diabetes, and congestive heart failure. Twenty‐two percent had been admitted at least once in the preceding 12 months. Treatment failure occurred in 7.7% of patients, and 1.3% died in the hospital. Mean length of stay was 4.8 days. Hospital prescribing rates for macrolides varied from 0% to 100%, with a mean of 33% and an interquartile range of 14% to 46% (Supporting Appendix Figure 1).

Compared to patients receiving macrolides, those receiving quinolones were older, more likely to have respiratory failure, to be cared for by a pulmonologist, and to have an admission in the previous year (Table 1). They were also more likely to be treated with bronchodilators, methylxanthines, steroids, diuretics, and noninvasive positive pressure ventilation, and to have an arterial blood gas, but less likely to receive concomitant treatment with a cephalosporin (11% vs. 57%). With the exception of cephalosporin treatment, these differences were small, but due to the large sample were statistically significant. Comorbidities were similar in both groups. Patients in the quinolone group were also more likely to experience treatment failure (8.1% vs. 6.8%), death (1.5% vs. 1.0%), and antibiotic‐associated diarrhea (1.1% vs. 0.5%).

In the unadjusted analysis, compared to patients receiving quinolones, those treated with macrolides were less likely to experience treatment failure (OR, 0.83; 95% CI, 0.740.94) (Table 2). Adjusting for all patient, hospital, and physician covariates, including the propensity for treatment with macrolides, increased the OR to 0.89 and the results were no longer significant (95% CI, 0.781.01). Propensity matching successfully balanced all measured covariates except for the use of short‐acting bronchodilators and additional antibiotics (Table 1). In the propensity‐matched sample (Figure 1), quinolone‐treated patients were more likely to experience antibiotic‐associated diarrhea (1.2% vs. 0.6%; P = 0.0003) and late mechanical ventilation (1.3% vs. 0.8%; P = 0.02). There were no differences in adjusted cost or length of stay between the 2 groups. The results of the grouped treatment analysis, substituting the hospital's specific rate of macrolide use in place of the actual treatment that each patient received suggested that the 2 antibiotics were associated with similar rates of treatment failure. The OR for a 100% hospital rate of macrolide treatment vs. a 0% rate was 1.01 (95% CI, 0.751.35).

Figure 1

Discussion

In this large observational study conducted at 375 hospitals, we took advantage of a natural experiment in which antibiotic prescribing patterns varied widely across hospitals to compare the effectiveness of 2 common antibiotic regimens for AECOPD. Treatment with macrolides and quinolones were associated with a similar risk of treatment failure, costs, and length of stay; however, patients treated with macrolides were less likely to experience late mechanical ventilation or treatment for antibiotic‐associated diarrhea.

Despite broad consensus in COPD guidelines that patients with severe acute exacerbations should receive antibiotics, there is little agreement about the preferred empiric agent. Controversy exists regarding antibiotics' comparative effectiveness, and even over which pathogens cause COPD exacerbations. Given the frequency of hospitalization for AECOPD, understanding the comparative effectiveness of treatments in this setting could have important implications for health outcomes and costs. Unfortunately, most antibiotic studies in AECOPD were conducted >20 years ago, using antibiotics that rarely appeared in our sample.26 Consequently, clinical practice guidelines offer conflicting recommendations. For example, the National Institute for Clinical Excellence recommends empirical treatment with an aminopenicillin, a macrolide, or a tetracycline,8 while the American Thoracic Society recommends amoxicillin‐clavulonate or a fluoroquinolone.9

As might be expected in light of so much uncertainty, we found wide variation in prescribing patterns across hospitals. Overall, approximately one‐third of patients received a macrolide and two‐thirds a quinolone. Both regimens provide adequate coverage of H. influenza, S. pneumoniae, and M. catarrhalis, and conform to at least 1 COPD guideline. Nevertheless, patients receiving macrolides often received a cephalosporin as well; this pattern of treatment suggests that antibiotic selection is likely to have been influenced more by guidelines for the treatment of community‐acquired pneumonia than COPD.

Previous studies comparing antibiotic effectiveness suffer from shortcomings that limit their application to patients hospitalized with AECOPD. First, they enrolled patients with chronic bronchitis, and included patients without obstructive lung disease, and most studies included patients as young as 18 years old. Second, many either did not include treatment with steroids or excluded patients receiving more than 10 mg of prednisone daily. Third, almost all enrolled only ambulatory patients.

While there are no studies comparing quinolones and macrolides in patients hospitalized for AECOPD, a meta‐analysis comparing quinolones, macrolides, and amoxicillin‐clavulonate identified 19 trials of ambulatory patients with chronic bronchitis. That study found that all 3 drugs had similar efficacy initially, but that quinolones resulted in the fewest relapses over a 26‐week period.27 Macrolides and quinolones had similar rates of adverse effects. In contrast, we did not find a difference in treatment failure, cost, or length of stay, but did find a higher rate of diarrhea associated with quinolones. Others have also documented an association between fluoroquinolones and C. difficile diarrhea.2830 This trend, first noted in 2001, is of particular concern because the fluoroquinolone‐resistant strains appear to be hypervirulent and have been associated with nosocomial epidemics.3134

Our study has several limitations. First, its observational design leaves open the possibility of selection bias. For this reason we analyzed our data in several ways, including using a grouped treatment approach, an adaptation of the instrumental variable technique, and accepted only those differences which were consistent across all models. Second, our study used claims data, and therefore we could not directly adjust for physiological measures of severity. However, the highly detailed nature of the data allowed us to adjust for numerous tests and treatments that reflected the clinician's assessment of the patient's severity, as well as the number of prior COPD admissions. Third, we cannot exclude the possibility that some patients may have had concurrent pneumonia without an ICD‐9 code. We think that the number would be small because reimbursement for pneumonia is generally higher than for COPD, so hospitals have an incentive to code pneumonia as the principal diagnosis when present. Finally, we compared initial antibiotics only. More than one‐quarter of our patients received an additional antibiotic before discharge. In particular, patients receiving macrolides were often prescribed a concomitant cephalosporin. We do not know to what extent these additional antibiotics may have affected the outcomes.

Despite the large number of patients hospitalized annually for AECOPD, there are no randomized trials comparing different antibiotics in this population. Studies comparing antibiotics in chronic bronchitis can offer little guidance, since they have primarily focused on proving equivalence between existing antibiotics and newer, more expensive formulations.35 Because many of the patients enrolled in such trials do not benefit from antibiotics at all, either because they do not have COPD or because their exacerbation is not caused by bacteria, it is relatively easy to prove equivalence. Given that AECOPD is one of the leading causes of hospitalization in the United States, large, randomized trials comparing the effectiveness of different antibiotics should be a high priority. In the meantime, macrolides (often given together with cephalosporins) and quinolones appear to be equally effective initial antibiotic choices; considering antibiotic‐associated diarrhea, macrolides appear to be the safer of the 2.

Acute exacerbations of chronic obstructive pulmonary disease (AECOPD) are responsible for more than 600,000 hospitalizations annually, resulting in direct costs of over $20 billion.1 Bacterial infections appear responsible for 50% of such exacerbations,25 and current COPD guidelines recommend treatment with antibiotics for patients with severe exacerbations or a change in sputum.1^,69 These recommendations are based on a number of small randomized trials, most of which were conducted more than 20 years ago using narrow spectrum antibiotics that are no longer commonly used.10 Only 4 studies, totaling 321 subjects, included hospitalized patients, and most studies excluded patients who required steroids. Because no clinical trials have compared different antibiotic regimens for AECOPD, existing guidelines offer a range of treatment options, including amoxicillin‐clavulonate, macrolides, quinolones, cephalosporins, aminopenicillins, and tetracyclines.

Among hospitalized patients, macrolides and quinolones appear to be the most frequently prescribed antibiotics.11 Both are available in oral formulations, have excellent bioavailability, and are administered once daily. In addition to their antimicrobial activity, macrolides are believed to have antiinflammatory effects, which could be especially advantageous in AECOPD.1214 In trials of chronic bronchitis, however, fluoroquinolones have been shown to reduce the risk of recurrent exacerbation when compared to macrolides.15 The wide variation that has been observed in antibiotic selection for patients hospitalized for AECOPD suggests a high degree of uncertainty among clinicians about the benefits of different treatment options.11 Given the limited evidence from randomized trials, we sought to evaluate the comparative effectiveness of macrolides and quinolones among a large, representative sample of patients hospitalized with AECOPD.

Subjects and Methods

Results

Of 26,248 AECOPD patients treated with antibiotics, 19,608 patients met the inclusion criteria; of these, 6139 (31%) were treated initially with a macrolide; the median age was 70 years; 60% were female; and 78% were white. A total of 86% of patients had a primary diagnosis of obstructive chronic bronchitis with acute exacerbation, and 6% had respiratory failure. The most common comorbidities were hypertension, diabetes, and congestive heart failure. Twenty‐two percent had been admitted at least once in the preceding 12 months. Treatment failure occurred in 7.7% of patients, and 1.3% died in the hospital. Mean length of stay was 4.8 days. Hospital prescribing rates for macrolides varied from 0% to 100%, with a mean of 33% and an interquartile range of 14% to 46% (Supporting Appendix Figure 1).

Compared to patients receiving macrolides, those receiving quinolones were older, more likely to have respiratory failure, to be cared for by a pulmonologist, and to have an admission in the previous year (Table 1). They were also more likely to be treated with bronchodilators, methylxanthines, steroids, diuretics, and noninvasive positive pressure ventilation, and to have an arterial blood gas, but less likely to receive concomitant treatment with a cephalosporin (11% vs. 57%). With the exception of cephalosporin treatment, these differences were small, but due to the large sample were statistically significant. Comorbidities were similar in both groups. Patients in the quinolone group were also more likely to experience treatment failure (8.1% vs. 6.8%), death (1.5% vs. 1.0%), and antibiotic‐associated diarrhea (1.1% vs. 0.5%).

In the unadjusted analysis, compared to patients receiving quinolones, those treated with macrolides were less likely to experience treatment failure (OR, 0.83; 95% CI, 0.740.94) (Table 2). Adjusting for all patient, hospital, and physician covariates, including the propensity for treatment with macrolides, increased the OR to 0.89 and the results were no longer significant (95% CI, 0.781.01). Propensity matching successfully balanced all measured covariates except for the use of short‐acting bronchodilators and additional antibiotics (Table 1). In the propensity‐matched sample (Figure 1), quinolone‐treated patients were more likely to experience antibiotic‐associated diarrhea (1.2% vs. 0.6%; P = 0.0003) and late mechanical ventilation (1.3% vs. 0.8%; P = 0.02). There were no differences in adjusted cost or length of stay between the 2 groups. The results of the grouped treatment analysis, substituting the hospital's specific rate of macrolide use in place of the actual treatment that each patient received suggested that the 2 antibiotics were associated with similar rates of treatment failure. The OR for a 100% hospital rate of macrolide treatment vs. a 0% rate was 1.01 (95% CI, 0.751.35).

Figure 1

Discussion

In this large observational study conducted at 375 hospitals, we took advantage of a natural experiment in which antibiotic prescribing patterns varied widely across hospitals to compare the effectiveness of 2 common antibiotic regimens for AECOPD. Treatment with macrolides and quinolones were associated with a similar risk of treatment failure, costs, and length of stay; however, patients treated with macrolides were less likely to experience late mechanical ventilation or treatment for antibiotic‐associated diarrhea.

Despite broad consensus in COPD guidelines that patients with severe acute exacerbations should receive antibiotics, there is little agreement about the preferred empiric agent. Controversy exists regarding antibiotics' comparative effectiveness, and even over which pathogens cause COPD exacerbations. Given the frequency of hospitalization for AECOPD, understanding the comparative effectiveness of treatments in this setting could have important implications for health outcomes and costs. Unfortunately, most antibiotic studies in AECOPD were conducted >20 years ago, using antibiotics that rarely appeared in our sample.26 Consequently, clinical practice guidelines offer conflicting recommendations. For example, the National Institute for Clinical Excellence recommends empirical treatment with an aminopenicillin, a macrolide, or a tetracycline,8 while the American Thoracic Society recommends amoxicillin‐clavulonate or a fluoroquinolone.9

As might be expected in light of so much uncertainty, we found wide variation in prescribing patterns across hospitals. Overall, approximately one‐third of patients received a macrolide and two‐thirds a quinolone. Both regimens provide adequate coverage of H. influenza, S. pneumoniae, and M. catarrhalis, and conform to at least 1 COPD guideline. Nevertheless, patients receiving macrolides often received a cephalosporin as well; this pattern of treatment suggests that antibiotic selection is likely to have been influenced more by guidelines for the treatment of community‐acquired pneumonia than COPD.

Previous studies comparing antibiotic effectiveness suffer from shortcomings that limit their application to patients hospitalized with AECOPD. First, they enrolled patients with chronic bronchitis, and included patients without obstructive lung disease, and most studies included patients as young as 18 years old. Second, many either did not include treatment with steroids or excluded patients receiving more than 10 mg of prednisone daily. Third, almost all enrolled only ambulatory patients.

While there are no studies comparing quinolones and macrolides in patients hospitalized for AECOPD, a meta‐analysis comparing quinolones, macrolides, and amoxicillin‐clavulonate identified 19 trials of ambulatory patients with chronic bronchitis. That study found that all 3 drugs had similar efficacy initially, but that quinolones resulted in the fewest relapses over a 26‐week period.27 Macrolides and quinolones had similar rates of adverse effects. In contrast, we did not find a difference in treatment failure, cost, or length of stay, but did find a higher rate of diarrhea associated with quinolones. Others have also documented an association between fluoroquinolones and C. difficile diarrhea.2830 This trend, first noted in 2001, is of particular concern because the fluoroquinolone‐resistant strains appear to be hypervirulent and have been associated with nosocomial epidemics.3134

Our study has several limitations. First, its observational design leaves open the possibility of selection bias. For this reason we analyzed our data in several ways, including using a grouped treatment approach, an adaptation of the instrumental variable technique, and accepted only those differences which were consistent across all models. Second, our study used claims data, and therefore we could not directly adjust for physiological measures of severity. However, the highly detailed nature of the data allowed us to adjust for numerous tests and treatments that reflected the clinician's assessment of the patient's severity, as well as the number of prior COPD admissions. Third, we cannot exclude the possibility that some patients may have had concurrent pneumonia without an ICD‐9 code. We think that the number would be small because reimbursement for pneumonia is generally higher than for COPD, so hospitals have an incentive to code pneumonia as the principal diagnosis when present. Finally, we compared initial antibiotics only. More than one‐quarter of our patients received an additional antibiotic before discharge. In particular, patients receiving macrolides were often prescribed a concomitant cephalosporin. We do not know to what extent these additional antibiotics may have affected the outcomes.

Despite the large number of patients hospitalized annually for AECOPD, there are no randomized trials comparing different antibiotics in this population. Studies comparing antibiotics in chronic bronchitis can offer little guidance, since they have primarily focused on proving equivalence between existing antibiotics and newer, more expensive formulations.35 Because many of the patients enrolled in such trials do not benefit from antibiotics at all, either because they do not have COPD or because their exacerbation is not caused by bacteria, it is relatively easy to prove equivalence. Given that AECOPD is one of the leading causes of hospitalization in the United States, large, randomized trials comparing the effectiveness of different antibiotics should be a high priority. In the meantime, macrolides (often given together with cephalosporins) and quinolones appear to be equally effective initial antibiotic choices; considering antibiotic‐associated diarrhea, macrolides appear to be the safer of the 2.

Acknowledging striking deficiencies in the quality and safety of healthcare, the Institute of Medicine, policy makers, and payors have called for transformation of the US healthcare system.1 Public reporting of hospital performance is one key strategy for accelerating improvement2 and may improve quality in several ways. First, feedback about performance relative to peers may stimulate quality improvement activities by appealing to professionalism. Second, the desire to preserve one's reputation by not appearing on a list of poor performers may be a powerful incentive. Finally, patients and referring providers could use reports to select high‐quality hospitals, thereby shifting care from low‐quality to high‐quality hospitals and stimulating quality improvement efforts to maintain or enhance market share.

Almost 20 years after New York and Pennsylvania began reporting cardiac surgery outcomes,3 the evidence that public reporting improves healthcare quality is equivocal.4 Moreover, stakeholders have embraced public reporting to differing degrees. Public reporting does lead to greater engagement in quality improvement activities,58 and additional financial incentives provide modest incremental benefits.9 Purchasers, too, are starting to pay attention.10 In New York State, payors appear to contract more with high‐quality surgeons and avoid poorly performing outliers.11 Some payors are creating tiered systems, assigning higher patient copayments for hospitals with poor quality metrics. These new systems have not been rigorously studied and should raise concern among hospitals.12

In contrast to hospitals and payors, patients have been slow to embrace public reporting. In a survey of coronary artery bypass graft (CABG) patients in Pennsylvania, only 2% said that public reporting of mortality rates affected their decision making.13 Eight years later, only 11% of patients sought information about hospitals before deciding on elective major surgery,14 although a majority of patients in both studies expressed interest in the information. It is not clear whether recent proliferation of information on the internet will change patient behavior, but to date public reporting appears not to effect market share.5, 15, 16

Barriers to patients' use of public reporting include difficulty accessing the information, lack of trust, information that is not salient, and data that are difficult to interpret.17 In the absence of consensus on what or how to report, a growing number of organizations, including state and federal government, accrediting bodies, private foundations, and for‐profit companies report a variety of measures relating to structure, processes, and outcomes. Although these sites purport to target consumers, they sometimes offer conflicting information18 and are not easily interpreted by lay readers.19

To realize the benefits of public reporting, and minimize the unintended consequences, rating systems must report salient information in a way that is comprehensible to patients and trusted by the doctors who advise them. At the same time, they should be fair to hospitals and offer useful data for quality improvement. We offer 10 recommendations for improving the public reporting of healthcare quality information: 5 describing what to report and 5 detailing how it should be reported (Figure 1). We also examine 3 leading performance reporting programs to see how well they implement these recommendations.

Figure 1

Recommendations to Make Data Salient for Patients

Recommendations to Ensure That Data Reflects Hospital Quality

Discussion

The growing commitment to making hospital performance data public could transform the quality and safety of care in the US, introducing competition on quality and price and fostering informed consumer choice. To date, the promise of public reporting remains only partially fulfilled. Few hospitals have done more than comply with regulatory mandates and payer incentives, and consumers have failed to respond. To capture the full benefits of public reporting, we have made 10 recommendations to benefit patients and better engage hospitals. We suggest that reporting be patient‐centered, with an emphasis on making the data useful, meaningful, important, interpretable, and relevant. At the same time, hospitals, which are being judged on their performance, should have a level playing field, with measures that are timely, consistent, severity‐adjusted, evidence‐based, and which foster good clinical care. Of the 3 services we examined, Hospital Compare came closest to meeting these recommendations.

Although this blueprint for public reporting is easy to draft, it is challenging to implement. In particular, some of our suggestions, such as the one regarding risk adjustment, may not currently be feasible, because the complexity and cost of collecting clinical data, even in the era of electronic medical records, may be prohibitive. Until such data are readily available, it may be preferable to report nothing at all, rather than report data that are misleading. In the rush to make hospitals accountable, enthusiasm has often outstripped science,31 and several measures have had to be revised for unintended consequences.32

Any initiative to improve public reporting should have the buy‐in of all stakeholders, but particularly hospitals, which stand to benefit in several ways. By receiving regular feedback, they can focus on improving care, becoming better organizations. These improvements may be rewarded through direct compensation (pay‐for‐performance), decreased costs from complications, or increased market share. Hospitals will be more engaged if the data reflect actual quality, are adequately adjusted for severity, and acknowledge the role of chance. Otherwise, they will merely comply, or worse, look for opportunities to game the system. To succeed, public reporting needs to involve hospitals in establishing standards for reporting and validation, as well as auditing procedures to prevent fraud.33 The Hospital Quality Alliance (HQA, Washington, DC), a first step in this direction, at present has few measures. NSQIP (American College of Surgeons, Chicago, IL) is perhaps a better example of hospitals cooperating to set measurement standards to promote best‐practices. Public release of NSQIP data might accelerate progress. Alternatively, the National Quality Forum (NQF, Washington, DC) could expand its role from endorsing quality measures to include standardizing the way these measures are used in public reporting.

Still, if you build it, will they come? To date, public reporting has not been embraced by the public, despite its stated interest in the information. Several explanations could be offered. First, we may be presenting the wrong data. Process measures and mortality rates are important but represent abstract concepts for most patients. Surveys tell us that patients value most the experiences of other patients.14, 21 They want to know whether their pain will be controlled, whether the doctor will listen to them, whether the nurse will come when they call. The recent advent of the HCAHPS survey (AHRQ, Washington, DC) is another positive step. Stratifying the results by diagnosis and adding a few diagnosis‐specific questions would make HCAHPS even more valuable. Second, the data may not be readily available. Although most public reporting is done on the web, older patients who are deciding about hospitals may not have Internet access. Some reports are still proprietary, and cost could present another obstacle. Finally, even if freely‐available and patient‐centered, the results may not be interpretable by physicians, let alone patients.34

If public reporting is to succeed, it will require measures that better reflect patients' concerns. In order to collect the massive amounts of data required and present them in a timely fashion, better electronic record systems will be necessary. But these are no panacea; others have noted that the Department of Veterans Affairs, a leader in electronic records, still invests considerable time and money to review charts for NSQIP.35 Given the value that Americans place on transparency in other facets of their lives, it is clear that public reporting is here to stay. While much progress has been made over the past 5 years, additional research is needed to better measure quality from the patient's perspective, and to determine how this information can be used to help guide decision‐making, and to reward hospitals for offering the highest‐quality care.