Natasha Nichols, MD

The American Medical Association publicly acknowledged in November 2020 that race is a social construct without biological basis, with many other leading medical organizations following suit.¹ Historically, biased science based on observed human physical differences has incorrectly asserted a racial biological hierarchy.^2,3 Today, leading health care organizations recognize that the effects of racist policies in housing, education, employment, and the criminal justice system contribute to health disparities and have a disproportionately negative impact on Black, Indigenous, and People of Color.^3,4 

Racial classification systems are fraught with bias. Trying to classify a complex and nuanced identity such as race into discrete categories does not capture the extensive heterogeneity at the individual level or within the increasingly diverse, multiracial population.⁵ Racial and ethnic categories used in collecting census data and research, as defined by the US Office of Management and Budget, have evolved over time.⁶ These changes in classification are a reflection of changes in the political environment, not changes in scientific knowledge of race and ethnicity.⁶

The Use of Race in Research and Practice

In the United States, racial minorities bear a disproportionate burden of morbidity and mortality across all major disease categories.³ These disparities cannot be explained by genetics.⁴ The Human Genome Project in 2003 confirmed that racial categories have no biologic or genetic basis and that there is more intraracial than interracial genetic variation.³ Nevertheless, significant misapplication of race in medical research and clinical practice remains. Instead of attributing observed differences in health outcomes between racial groups to innate physiological differences between the groups, clinicians and researchers must carefully consider the impact of racism.⁷ This includes considering the complex interactions between socioeconomic, political, and environmental factors, and how they affect health.³

While race is not biologic, the effects of racism can have biologic effects, and advocates appropriately cite the need to collect race as an important category in epidemiological analysis. When race and ethnicity are used as a study variable, bioethicists Kaplan and Bennett recommend that researchers: (1) account for limitations due to imprecision of racial categories; (2) avoid attributing causality when there is an association between race/ethnicity and a health outcome; and (3) refrain from exacerbating racial disparities.⁶

At the bedside, race has become embedded in clinical, seemingly objective, decision-making tools used across medical specialties.⁸ These algorithms often use observational outcomes data and draw conclusions by explicitly or implicitly assuming biological differences among races. By crudely adjusting for race without identifying the root cause for observed racial differences, these tools can further magnify health inequities.⁸ With the increased recognition that race cannot be used as a proxy for genetic ancestry, and that racial and ethnic categories are complex sociopolitical constructs that have changed over time, the practice of race-based medicine is increasingly being criticized.⁸

This article presents a case for the removal of the race coefficient from estimated glomerular filtration rate (eGFR) calculations that exacerbate disparities in kidney health by overestimating kidney function in Black patients.⁸ The main justification for using the race coefficient stems from the disproven assumption that Black people have more muscle mass compared with non-Black people.⁹  The questioning of this racist assertion has led to a national movement to reevaluate the use of race in eGFR calculations.

Racial Disparities in Kidney Disease

According to epidemiological data published by the National Kidney Foundation (NKF) and American Society of Nephrology (ASN), 37 million people in the United States have chronic kidney disease (CKD).¹⁰ Black Americans make up 13% of the US population yet they account for more than 30% of patients with end-stage kidney disease (ESKD) and 35% of those on dialysis.^10,11 There is a 3 times greater risk for progression from early-stage CKD to ESKD in Black Americans when compared to the risk for White Americans.¹¹ Black patients are younger at the time of CKD diagnosis and, once diagnosed, experience a faster progression to ESKD.¹² These disparities are partially attributable to delays in diagnosis, preventative measures, and referrals to nephrology care.¹²  

In a VA medical center study, although Black patients were referred to nephrology care at higher rates than White patients, Black patients had faster progression to CKD stage 5.¹³ An earlier study showed that, at any given eGFR, Black patients have higher levels of albuminuria compared to White patients.¹⁴ While the reasons behind this observation are likely complex and multifactorial, one hypothesis is that Black patients were already at a more advanced stage of kidney disease at the time of referral as a result of the overestimation of eGFR calculations related to the use of a race coefficient.

Additionally, numerous analyses have revealed that Black patients are less likely to be identified as transplant candidates, less likely to be referred for transplant evaluation and, once on the waiting list, wait longer than do White patients.^11,15

Estimated Glomerular Filtration Rate

It is imperative that clinicians have the most accurate measure of GFR to ensure timely diagnosis and appropriate management in patients with CKD. The gold standard for determining renal function requires measuring GFR using an ideal, exogenous, filtration marker such as iothalamate. However, this process is complex and time-consuming, rendering it infeasible in routine care. As a result, we usually estimate GFR using endogenous serum markers such as creatinine and cystatin C. Due to availability and cost, serum creatinine (SCr) is the most widely used marker for estimating kidney function. However, many pitfalls are inherent in its use, including the effects of tubular secretion, extrarenal clearance, and day-to-day variability in creatinine generation related to muscle mass, diet, and activity.¹⁶ The 2 most widely used estimation equations are the Modification of Diet in Renal Disease (MDRD) study equation and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) creatinine equation; both equations incorporate correction factors for age, sex, and race. 

The VA uses MDRD, which was derived and validated in a cohort of 1628 patients that included only 197 Black patients (12%), resulting in an eGFR for Black patients that is 21% higher than is the eGFR for non-Black patients with the same SCr value.⁹ In the VA electronic health record, the race coefficient is incorporated directly into eGFR laboratory calculations based on the race that the veteran self-identified during intake. Because the laboratory reports only a race-adjusted eGFR, there is a lack of transparency as many health care providers and patients are unaware that a race coefficient is used in eGFR calculations at the VA.  

Case for Removing Race Coefficient

When applied to cohorts outside the original study, both the MDRD and CKD-EPI equations have proved to be highly biased, imprecise, and inaccurate when compared to measured GFR (mGFR).^15,17 For any given eGFR, the possible mGFR may span 3 stages of CKD, underscoring the limitations of using such a crude estimate in clinical decision making.¹⁷ 

Current Kidney Estimation Pitfalls

A recent cohort study by Zelnick and colleagues that included 1658 self-identified Black adults showed less bias between mGFR and eGFR without the use of a race coefficient, and a shorter median time to transplant eligibility by 1.9 years.¹⁵ This study provides further evidence that these equations were derived from a biased observational data set that overestimates eGFR in Black patients living with CKD. This overestimation is particularly egregious for frail or malnourished patients with CKD and multiple comorbidities, with many potential harmful clinical consequences.

In addition, multiple international studies in African countries have demonstrated worse performance of eGFR calculations when using the race coefficient than without it. In the Democratic Republic of the Congo, eGFR was calculated for adults using MDRD with and without the race coefficient, as well as CKD-EPI with and without the race coefficient, and then compared to mGFR. Both the MDRD and the CKD-EPI equations overestimated GFR when using the race coefficient, and notably the equations without the race coefficient had better correlation to mGFR.¹⁸ Similar data were also found in studies from South Africa, the Ivory Coast, Brazil, and Europe.^19-22

Clinical Consequences of Race Coefficient Use

The use of a race coefficient in these estimation equations causes adverse clinical outcomes. In early stages of CKD, overestimation of eGFR using the race coefficient can cause an under-recognition of CKD, and can lead to delays in diagnosis and failure to implement measures to slow its progression, such as minimizing drug-related nephrotoxic injury and iatrogenic acute kidney injury. Consequently, a patient with an overestimated eGFR may suffer an accelerated progression to ESKD and premature mortality from cardiovascular disease.²³ 

In advanced CKD stages, eGFR overestimation may result in delayed referral to a nephrologist (recommended at eGFR < 30mL/min/1.73 m²), nutrition counseling, renal replacement therapy education, timely referral for renal replacement therapy access placement, and transplant evaluation (can be listed when eGFR < 20 mL/min/1.73 m²).^16,24,25 

In the Clinical Vignette, it is clear from the information presented that Mr. C’s concerns are well-founded. Table 1 presents the impact on eGFR caused by the race coefficient using the MDRD and CKD-EPI equations. In many VA systems, this overestimation would prevent him from being referred for a kidney transplant at this visit, thereby perpetuating racial health disparities in kidney transplantation. 

Concerns About Removal of Race From eGFR Calculations

Opponents of removing the race coefficient assert that a lower eGFR will preclude some patients from qualifying for medications such as metformin and certain anticoagulants, or that it may result in subtherapeutic dosing of drugs such as antibiotics and chemotherapeutic agents.²⁶ These recommendations are in place for patient safety, so conversely maintaining the race coefficient and overestimating eGFR will expose some patients to medication toxicity. Another fear is that lower eGFRs will have the unintended consequence of limiting the kidney donor pool. However, this can be prevented by following current guidelines to use mGFR in settings where accurate GFR is imperative.¹⁶ Additionally, some nephrologists have expressed concern that diagnosing more patients with advanced stages of CKD will result in inappropriately early initiation of dialysis. Again, this risk can be mitigated by ensuring that nephrologists consider multiple clinical factors and data points, not simply eGFR when deciding to initiate dialysis. Also, an increase in referrals to nephrology may occur when the race coefficient is removed and increased wait times at some VA medical centers could be a concern. An increase in appropriate referrals would show that removing the race coefficient was having its intended effect—more veterans with advanced CKD being seen by nephrologists.

When considering the lack of biological plausibility, inaccuracy, and the clinical harms associated with the use of the race coefficient in eGFR calculations, the benefits of removing the race coefficient from eGFR calculations within the VA far outweigh any potential risks.  

A Call for Equity

The National Conversation on Race and eGFR

To advance health equity, members of the medical community have advocated for the removal of the race coefficient from eGFR calculations for years. Beth Israel Deaconess Medical Center was the first establishment to institute this change in 2017. Since then, many health systems across the country that are affiliated with Veterans Health Administration (VHA) medical centers have removed the race coefficient from eGFR equations (Table 2). Many other hospital systems are contemplating this change. 

In July 2020, the NKF and the ASN established a joint task force dedicated to reassessing the inclusion of race in eGFR calculations. This task force acknowledges that race is a social, not biological, construct.¹² The NKF/ASN task force is now in the second of its 3-phase process. In March 2021, prior to publication of their phase 1 findings, they announced “(1) race modifiers should not be included in equations to estimate kidney function; and (2) current race-based equations should be replaced by a suitable approach that is accurate, inclusive, and standardized in every laboratory in the United States. Any such approach must not differentially introduce bias, inaccuracy, or inequalities.”²⁷

Health Equity in the VHA

In January 2021, President Biden issued an executive order to advance racial equity and support underserved communities through the federal government and its agencies. The VHA is the largest integrated health care system in the United States serving 9 million veterans and is one of the largest federal agencies. As VA clinicians, it is our responsibility to examine the evidence, consider national guidance, and ensure health equity for veterans by practicing unbiased medicine. The evidence and the interim guidance from the NKF-ASN task force clearly indicate that the race coefficient should no longer be used.²⁷ It is imperative that we make these changes immediately knowing that the use of race in kidney function calculators is harming Black veterans. Similar to finding evidence of harm in a treatment group in a clinical trial, it is unethical to wait. Removal of the race coefficient in eGFR calculations will allow VHA clinicians to provide timely and high-quality care to our patients as well as establish the VHA as a national leader in health equity.

VISN 12 Leads the Way

On May 11, 2021, the VA Great Lakes Health Care System, Veterans Integrated Service Network (VISN) 12, leaders responded to this author group’s call to advance health equity and voted to remove the race coefficient from eGFR calculations. Other VISNs should follow, and the VHA should continue to work with national leaders and experts to establish and implement superior tools to ensure the highest quality of kidney health care for all veterans.  

Acknowledgments
The authors would like to thank the medical students across the nation who have been leading the charge on this important issue. The authors are also thankful for the collaboration and support of all members of the Jesse Brown for Black Lives (JB4BL) Task Force. 

The American Medical Association publicly acknowledged in November 2020 that race is a social construct without biological basis, with many other leading medical organizations following suit.¹ Historically, biased science based on observed human physical differences has incorrectly asserted a racial biological hierarchy.^2,3 Today, leading health care organizations recognize that the effects of racist policies in housing, education, employment, and the criminal justice system contribute to health disparities and have a disproportionately negative impact on Black, Indigenous, and People of Color.^3,4 

Racial classification systems are fraught with bias. Trying to classify a complex and nuanced identity such as race into discrete categories does not capture the extensive heterogeneity at the individual level or within the increasingly diverse, multiracial population.⁵ Racial and ethnic categories used in collecting census data and research, as defined by the US Office of Management and Budget, have evolved over time.⁶ These changes in classification are a reflection of changes in the political environment, not changes in scientific knowledge of race and ethnicity.⁶

The Use of Race in Research and Practice

In the United States, racial minorities bear a disproportionate burden of morbidity and mortality across all major disease categories.³ These disparities cannot be explained by genetics.⁴ The Human Genome Project in 2003 confirmed that racial categories have no biologic or genetic basis and that there is more intraracial than interracial genetic variation.³ Nevertheless, significant misapplication of race in medical research and clinical practice remains. Instead of attributing observed differences in health outcomes between racial groups to innate physiological differences between the groups, clinicians and researchers must carefully consider the impact of racism.⁷ This includes considering the complex interactions between socioeconomic, political, and environmental factors, and how they affect health.³

While race is not biologic, the effects of racism can have biologic effects, and advocates appropriately cite the need to collect race as an important category in epidemiological analysis. When race and ethnicity are used as a study variable, bioethicists Kaplan and Bennett recommend that researchers: (1) account for limitations due to imprecision of racial categories; (2) avoid attributing causality when there is an association between race/ethnicity and a health outcome; and (3) refrain from exacerbating racial disparities.⁶

At the bedside, race has become embedded in clinical, seemingly objective, decision-making tools used across medical specialties.⁸ These algorithms often use observational outcomes data and draw conclusions by explicitly or implicitly assuming biological differences among races. By crudely adjusting for race without identifying the root cause for observed racial differences, these tools can further magnify health inequities.⁸ With the increased recognition that race cannot be used as a proxy for genetic ancestry, and that racial and ethnic categories are complex sociopolitical constructs that have changed over time, the practice of race-based medicine is increasingly being criticized.⁸

This article presents a case for the removal of the race coefficient from estimated glomerular filtration rate (eGFR) calculations that exacerbate disparities in kidney health by overestimating kidney function in Black patients.⁸ The main justification for using the race coefficient stems from the disproven assumption that Black people have more muscle mass compared with non-Black people.⁹  The questioning of this racist assertion has led to a national movement to reevaluate the use of race in eGFR calculations.

Racial Disparities in Kidney Disease

According to epidemiological data published by the National Kidney Foundation (NKF) and American Society of Nephrology (ASN), 37 million people in the United States have chronic kidney disease (CKD).¹⁰ Black Americans make up 13% of the US population yet they account for more than 30% of patients with end-stage kidney disease (ESKD) and 35% of those on dialysis.^10,11 There is a 3 times greater risk for progression from early-stage CKD to ESKD in Black Americans when compared to the risk for White Americans.¹¹ Black patients are younger at the time of CKD diagnosis and, once diagnosed, experience a faster progression to ESKD.¹² These disparities are partially attributable to delays in diagnosis, preventative measures, and referrals to nephrology care.¹²  

In a VA medical center study, although Black patients were referred to nephrology care at higher rates than White patients, Black patients had faster progression to CKD stage 5.¹³ An earlier study showed that, at any given eGFR, Black patients have higher levels of albuminuria compared to White patients.¹⁴ While the reasons behind this observation are likely complex and multifactorial, one hypothesis is that Black patients were already at a more advanced stage of kidney disease at the time of referral as a result of the overestimation of eGFR calculations related to the use of a race coefficient.

Additionally, numerous analyses have revealed that Black patients are less likely to be identified as transplant candidates, less likely to be referred for transplant evaluation and, once on the waiting list, wait longer than do White patients.^11,15

Estimated Glomerular Filtration Rate

It is imperative that clinicians have the most accurate measure of GFR to ensure timely diagnosis and appropriate management in patients with CKD. The gold standard for determining renal function requires measuring GFR using an ideal, exogenous, filtration marker such as iothalamate. However, this process is complex and time-consuming, rendering it infeasible in routine care. As a result, we usually estimate GFR using endogenous serum markers such as creatinine and cystatin C. Due to availability and cost, serum creatinine (SCr) is the most widely used marker for estimating kidney function. However, many pitfalls are inherent in its use, including the effects of tubular secretion, extrarenal clearance, and day-to-day variability in creatinine generation related to muscle mass, diet, and activity.¹⁶ The 2 most widely used estimation equations are the Modification of Diet in Renal Disease (MDRD) study equation and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) creatinine equation; both equations incorporate correction factors for age, sex, and race. 

The VA uses MDRD, which was derived and validated in a cohort of 1628 patients that included only 197 Black patients (12%), resulting in an eGFR for Black patients that is 21% higher than is the eGFR for non-Black patients with the same SCr value.⁹ In the VA electronic health record, the race coefficient is incorporated directly into eGFR laboratory calculations based on the race that the veteran self-identified during intake. Because the laboratory reports only a race-adjusted eGFR, there is a lack of transparency as many health care providers and patients are unaware that a race coefficient is used in eGFR calculations at the VA.  

Case for Removing Race Coefficient

When applied to cohorts outside the original study, both the MDRD and CKD-EPI equations have proved to be highly biased, imprecise, and inaccurate when compared to measured GFR (mGFR).^15,17 For any given eGFR, the possible mGFR may span 3 stages of CKD, underscoring the limitations of using such a crude estimate in clinical decision making.¹⁷ 

Current Kidney Estimation Pitfalls

A recent cohort study by Zelnick and colleagues that included 1658 self-identified Black adults showed less bias between mGFR and eGFR without the use of a race coefficient, and a shorter median time to transplant eligibility by 1.9 years.¹⁵ This study provides further evidence that these equations were derived from a biased observational data set that overestimates eGFR in Black patients living with CKD. This overestimation is particularly egregious for frail or malnourished patients with CKD and multiple comorbidities, with many potential harmful clinical consequences.

In addition, multiple international studies in African countries have demonstrated worse performance of eGFR calculations when using the race coefficient than without it. In the Democratic Republic of the Congo, eGFR was calculated for adults using MDRD with and without the race coefficient, as well as CKD-EPI with and without the race coefficient, and then compared to mGFR. Both the MDRD and the CKD-EPI equations overestimated GFR when using the race coefficient, and notably the equations without the race coefficient had better correlation to mGFR.¹⁸ Similar data were also found in studies from South Africa, the Ivory Coast, Brazil, and Europe.^19-22

Clinical Consequences of Race Coefficient Use

The use of a race coefficient in these estimation equations causes adverse clinical outcomes. In early stages of CKD, overestimation of eGFR using the race coefficient can cause an under-recognition of CKD, and can lead to delays in diagnosis and failure to implement measures to slow its progression, such as minimizing drug-related nephrotoxic injury and iatrogenic acute kidney injury. Consequently, a patient with an overestimated eGFR may suffer an accelerated progression to ESKD and premature mortality from cardiovascular disease.²³ 

In advanced CKD stages, eGFR overestimation may result in delayed referral to a nephrologist (recommended at eGFR < 30mL/min/1.73 m²), nutrition counseling, renal replacement therapy education, timely referral for renal replacement therapy access placement, and transplant evaluation (can be listed when eGFR < 20 mL/min/1.73 m²).^16,24,25 

In the Clinical Vignette, it is clear from the information presented that Mr. C’s concerns are well-founded. Table 1 presents the impact on eGFR caused by the race coefficient using the MDRD and CKD-EPI equations. In many VA systems, this overestimation would prevent him from being referred for a kidney transplant at this visit, thereby perpetuating racial health disparities in kidney transplantation. 

Concerns About Removal of Race From eGFR Calculations

Opponents of removing the race coefficient assert that a lower eGFR will preclude some patients from qualifying for medications such as metformin and certain anticoagulants, or that it may result in subtherapeutic dosing of drugs such as antibiotics and chemotherapeutic agents.²⁶ These recommendations are in place for patient safety, so conversely maintaining the race coefficient and overestimating eGFR will expose some patients to medication toxicity. Another fear is that lower eGFRs will have the unintended consequence of limiting the kidney donor pool. However, this can be prevented by following current guidelines to use mGFR in settings where accurate GFR is imperative.¹⁶ Additionally, some nephrologists have expressed concern that diagnosing more patients with advanced stages of CKD will result in inappropriately early initiation of dialysis. Again, this risk can be mitigated by ensuring that nephrologists consider multiple clinical factors and data points, not simply eGFR when deciding to initiate dialysis. Also, an increase in referrals to nephrology may occur when the race coefficient is removed and increased wait times at some VA medical centers could be a concern. An increase in appropriate referrals would show that removing the race coefficient was having its intended effect—more veterans with advanced CKD being seen by nephrologists.

When considering the lack of biological plausibility, inaccuracy, and the clinical harms associated with the use of the race coefficient in eGFR calculations, the benefits of removing the race coefficient from eGFR calculations within the VA far outweigh any potential risks.  

A Call for Equity

The National Conversation on Race and eGFR

To advance health equity, members of the medical community have advocated for the removal of the race coefficient from eGFR calculations for years. Beth Israel Deaconess Medical Center was the first establishment to institute this change in 2017. Since then, many health systems across the country that are affiliated with Veterans Health Administration (VHA) medical centers have removed the race coefficient from eGFR equations (Table 2). Many other hospital systems are contemplating this change. 

In July 2020, the NKF and the ASN established a joint task force dedicated to reassessing the inclusion of race in eGFR calculations. This task force acknowledges that race is a social, not biological, construct.¹² The NKF/ASN task force is now in the second of its 3-phase process. In March 2021, prior to publication of their phase 1 findings, they announced “(1) race modifiers should not be included in equations to estimate kidney function; and (2) current race-based equations should be replaced by a suitable approach that is accurate, inclusive, and standardized in every laboratory in the United States. Any such approach must not differentially introduce bias, inaccuracy, or inequalities.”²⁷

Health Equity in the VHA

In January 2021, President Biden issued an executive order to advance racial equity and support underserved communities through the federal government and its agencies. The VHA is the largest integrated health care system in the United States serving 9 million veterans and is one of the largest federal agencies. As VA clinicians, it is our responsibility to examine the evidence, consider national guidance, and ensure health equity for veterans by practicing unbiased medicine. The evidence and the interim guidance from the NKF-ASN task force clearly indicate that the race coefficient should no longer be used.²⁷ It is imperative that we make these changes immediately knowing that the use of race in kidney function calculators is harming Black veterans. Similar to finding evidence of harm in a treatment group in a clinical trial, it is unethical to wait. Removal of the race coefficient in eGFR calculations will allow VHA clinicians to provide timely and high-quality care to our patients as well as establish the VHA as a national leader in health equity.

VISN 12 Leads the Way

On May 11, 2021, the VA Great Lakes Health Care System, Veterans Integrated Service Network (VISN) 12, leaders responded to this author group’s call to advance health equity and voted to remove the race coefficient from eGFR calculations. Other VISNs should follow, and the VHA should continue to work with national leaders and experts to establish and implement superior tools to ensure the highest quality of kidney health care for all veterans.  

Acknowledgments
The authors would like to thank the medical students across the nation who have been leading the charge on this important issue. The authors are also thankful for the collaboration and support of all members of the Jesse Brown for Black Lives (JB4BL) Task Force. 

The American Medical Association publicly acknowledged in November 2020 that race is a social construct without biological basis, with many other leading medical organizations following suit.¹ Historically, biased science based on observed human physical differences has incorrectly asserted a racial biological hierarchy.^2,3 Today, leading health care organizations recognize that the effects of racist policies in housing, education, employment, and the criminal justice system contribute to health disparities and have a disproportionately negative impact on Black, Indigenous, and People of Color.^3,4 

Racial classification systems are fraught with bias. Trying to classify a complex and nuanced identity such as race into discrete categories does not capture the extensive heterogeneity at the individual level or within the increasingly diverse, multiracial population.⁵ Racial and ethnic categories used in collecting census data and research, as defined by the US Office of Management and Budget, have evolved over time.⁶ These changes in classification are a reflection of changes in the political environment, not changes in scientific knowledge of race and ethnicity.⁶

The Use of Race in Research and Practice

In the United States, racial minorities bear a disproportionate burden of morbidity and mortality across all major disease categories.³ These disparities cannot be explained by genetics.⁴ The Human Genome Project in 2003 confirmed that racial categories have no biologic or genetic basis and that there is more intraracial than interracial genetic variation.³ Nevertheless, significant misapplication of race in medical research and clinical practice remains. Instead of attributing observed differences in health outcomes between racial groups to innate physiological differences between the groups, clinicians and researchers must carefully consider the impact of racism.⁷ This includes considering the complex interactions between socioeconomic, political, and environmental factors, and how they affect health.³

While race is not biologic, the effects of racism can have biologic effects, and advocates appropriately cite the need to collect race as an important category in epidemiological analysis. When race and ethnicity are used as a study variable, bioethicists Kaplan and Bennett recommend that researchers: (1) account for limitations due to imprecision of racial categories; (2) avoid attributing causality when there is an association between race/ethnicity and a health outcome; and (3) refrain from exacerbating racial disparities.⁶

At the bedside, race has become embedded in clinical, seemingly objective, decision-making tools used across medical specialties.⁸ These algorithms often use observational outcomes data and draw conclusions by explicitly or implicitly assuming biological differences among races. By crudely adjusting for race without identifying the root cause for observed racial differences, these tools can further magnify health inequities.⁸ With the increased recognition that race cannot be used as a proxy for genetic ancestry, and that racial and ethnic categories are complex sociopolitical constructs that have changed over time, the practice of race-based medicine is increasingly being criticized.⁸

This article presents a case for the removal of the race coefficient from estimated glomerular filtration rate (eGFR) calculations that exacerbate disparities in kidney health by overestimating kidney function in Black patients.⁸ The main justification for using the race coefficient stems from the disproven assumption that Black people have more muscle mass compared with non-Black people.⁹  The questioning of this racist assertion has led to a national movement to reevaluate the use of race in eGFR calculations.

Racial Disparities in Kidney Disease

According to epidemiological data published by the National Kidney Foundation (NKF) and American Society of Nephrology (ASN), 37 million people in the United States have chronic kidney disease (CKD).¹⁰ Black Americans make up 13% of the US population yet they account for more than 30% of patients with end-stage kidney disease (ESKD) and 35% of those on dialysis.^10,11 There is a 3 times greater risk for progression from early-stage CKD to ESKD in Black Americans when compared to the risk for White Americans.¹¹ Black patients are younger at the time of CKD diagnosis and, once diagnosed, experience a faster progression to ESKD.¹² These disparities are partially attributable to delays in diagnosis, preventative measures, and referrals to nephrology care.¹²  

In a VA medical center study, although Black patients were referred to nephrology care at higher rates than White patients, Black patients had faster progression to CKD stage 5.¹³ An earlier study showed that, at any given eGFR, Black patients have higher levels of albuminuria compared to White patients.¹⁴ While the reasons behind this observation are likely complex and multifactorial, one hypothesis is that Black patients were already at a more advanced stage of kidney disease at the time of referral as a result of the overestimation of eGFR calculations related to the use of a race coefficient.

Additionally, numerous analyses have revealed that Black patients are less likely to be identified as transplant candidates, less likely to be referred for transplant evaluation and, once on the waiting list, wait longer than do White patients.^11,15

Estimated Glomerular Filtration Rate

It is imperative that clinicians have the most accurate measure of GFR to ensure timely diagnosis and appropriate management in patients with CKD. The gold standard for determining renal function requires measuring GFR using an ideal, exogenous, filtration marker such as iothalamate. However, this process is complex and time-consuming, rendering it infeasible in routine care. As a result, we usually estimate GFR using endogenous serum markers such as creatinine and cystatin C. Due to availability and cost, serum creatinine (SCr) is the most widely used marker for estimating kidney function. However, many pitfalls are inherent in its use, including the effects of tubular secretion, extrarenal clearance, and day-to-day variability in creatinine generation related to muscle mass, diet, and activity.¹⁶ The 2 most widely used estimation equations are the Modification of Diet in Renal Disease (MDRD) study equation and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) creatinine equation; both equations incorporate correction factors for age, sex, and race. 

The VA uses MDRD, which was derived and validated in a cohort of 1628 patients that included only 197 Black patients (12%), resulting in an eGFR for Black patients that is 21% higher than is the eGFR for non-Black patients with the same SCr value.⁹ In the VA electronic health record, the race coefficient is incorporated directly into eGFR laboratory calculations based on the race that the veteran self-identified during intake. Because the laboratory reports only a race-adjusted eGFR, there is a lack of transparency as many health care providers and patients are unaware that a race coefficient is used in eGFR calculations at the VA.  

Case for Removing Race Coefficient

When applied to cohorts outside the original study, both the MDRD and CKD-EPI equations have proved to be highly biased, imprecise, and inaccurate when compared to measured GFR (mGFR).^15,17 For any given eGFR, the possible mGFR may span 3 stages of CKD, underscoring the limitations of using such a crude estimate in clinical decision making.¹⁷ 

Current Kidney Estimation Pitfalls

A recent cohort study by Zelnick and colleagues that included 1658 self-identified Black adults showed less bias between mGFR and eGFR without the use of a race coefficient, and a shorter median time to transplant eligibility by 1.9 years.¹⁵ This study provides further evidence that these equations were derived from a biased observational data set that overestimates eGFR in Black patients living with CKD. This overestimation is particularly egregious for frail or malnourished patients with CKD and multiple comorbidities, with many potential harmful clinical consequences.

In addition, multiple international studies in African countries have demonstrated worse performance of eGFR calculations when using the race coefficient than without it. In the Democratic Republic of the Congo, eGFR was calculated for adults using MDRD with and without the race coefficient, as well as CKD-EPI with and without the race coefficient, and then compared to mGFR. Both the MDRD and the CKD-EPI equations overestimated GFR when using the race coefficient, and notably the equations without the race coefficient had better correlation to mGFR.¹⁸ Similar data were also found in studies from South Africa, the Ivory Coast, Brazil, and Europe.^19-22

Clinical Consequences of Race Coefficient Use

The use of a race coefficient in these estimation equations causes adverse clinical outcomes. In early stages of CKD, overestimation of eGFR using the race coefficient can cause an under-recognition of CKD, and can lead to delays in diagnosis and failure to implement measures to slow its progression, such as minimizing drug-related nephrotoxic injury and iatrogenic acute kidney injury. Consequently, a patient with an overestimated eGFR may suffer an accelerated progression to ESKD and premature mortality from cardiovascular disease.²³ 

In advanced CKD stages, eGFR overestimation may result in delayed referral to a nephrologist (recommended at eGFR < 30mL/min/1.73 m²), nutrition counseling, renal replacement therapy education, timely referral for renal replacement therapy access placement, and transplant evaluation (can be listed when eGFR < 20 mL/min/1.73 m²).^16,24,25 

In the Clinical Vignette, it is clear from the information presented that Mr. C’s concerns are well-founded. Table 1 presents the impact on eGFR caused by the race coefficient using the MDRD and CKD-EPI equations. In many VA systems, this overestimation would prevent him from being referred for a kidney transplant at this visit, thereby perpetuating racial health disparities in kidney transplantation. 

Concerns About Removal of Race From eGFR Calculations

Opponents of removing the race coefficient assert that a lower eGFR will preclude some patients from qualifying for medications such as metformin and certain anticoagulants, or that it may result in subtherapeutic dosing of drugs such as antibiotics and chemotherapeutic agents.²⁶ These recommendations are in place for patient safety, so conversely maintaining the race coefficient and overestimating eGFR will expose some patients to medication toxicity. Another fear is that lower eGFRs will have the unintended consequence of limiting the kidney donor pool. However, this can be prevented by following current guidelines to use mGFR in settings where accurate GFR is imperative.¹⁶ Additionally, some nephrologists have expressed concern that diagnosing more patients with advanced stages of CKD will result in inappropriately early initiation of dialysis. Again, this risk can be mitigated by ensuring that nephrologists consider multiple clinical factors and data points, not simply eGFR when deciding to initiate dialysis. Also, an increase in referrals to nephrology may occur when the race coefficient is removed and increased wait times at some VA medical centers could be a concern. An increase in appropriate referrals would show that removing the race coefficient was having its intended effect—more veterans with advanced CKD being seen by nephrologists.

When considering the lack of biological plausibility, inaccuracy, and the clinical harms associated with the use of the race coefficient in eGFR calculations, the benefits of removing the race coefficient from eGFR calculations within the VA far outweigh any potential risks.  

A Call for Equity

The National Conversation on Race and eGFR

To advance health equity, members of the medical community have advocated for the removal of the race coefficient from eGFR calculations for years. Beth Israel Deaconess Medical Center was the first establishment to institute this change in 2017. Since then, many health systems across the country that are affiliated with Veterans Health Administration (VHA) medical centers have removed the race coefficient from eGFR equations (Table 2). Many other hospital systems are contemplating this change. 

In July 2020, the NKF and the ASN established a joint task force dedicated to reassessing the inclusion of race in eGFR calculations. This task force acknowledges that race is a social, not biological, construct.¹² The NKF/ASN task force is now in the second of its 3-phase process. In March 2021, prior to publication of their phase 1 findings, they announced “(1) race modifiers should not be included in equations to estimate kidney function; and (2) current race-based equations should be replaced by a suitable approach that is accurate, inclusive, and standardized in every laboratory in the United States. Any such approach must not differentially introduce bias, inaccuracy, or inequalities.”²⁷

Health Equity in the VHA

In January 2021, President Biden issued an executive order to advance racial equity and support underserved communities through the federal government and its agencies. The VHA is the largest integrated health care system in the United States serving 9 million veterans and is one of the largest federal agencies. As VA clinicians, it is our responsibility to examine the evidence, consider national guidance, and ensure health equity for veterans by practicing unbiased medicine. The evidence and the interim guidance from the NKF-ASN task force clearly indicate that the race coefficient should no longer be used.²⁷ It is imperative that we make these changes immediately knowing that the use of race in kidney function calculators is harming Black veterans. Similar to finding evidence of harm in a treatment group in a clinical trial, it is unethical to wait. Removal of the race coefficient in eGFR calculations will allow VHA clinicians to provide timely and high-quality care to our patients as well as establish the VHA as a national leader in health equity.

VISN 12 Leads the Way

On May 11, 2021, the VA Great Lakes Health Care System, Veterans Integrated Service Network (VISN) 12, leaders responded to this author group’s call to advance health equity and voted to remove the race coefficient from eGFR calculations. Other VISNs should follow, and the VHA should continue to work with national leaders and experts to establish and implement superior tools to ensure the highest quality of kidney health care for all veterans.  

Acknowledgments
The authors would like to thank the medical students across the nation who have been leading the charge on this important issue. The authors are also thankful for the collaboration and support of all members of the Jesse Brown for Black Lives (JB4BL) Task Force. 

In recent years, hospitals and health systems have engaged in considerable efforts to reduce readmissions, in part due to financial incentives from the Medicare Hospital Readmission Reduction Program.^1,2 Though efforts to improve transitions of care for all patients are laudable, risk for readmission is not distributed equally; a small subset of patients accounts for a disproportionate number of hospital readmissions.³ This phenomenon of frequently hospitalized patients is similar to that seen in other populations in which a small proportion of patients account for a majority of healthcare utilization.^3,4

Recognizing that the current system of healthcare delivery does not meet the needs of this population, healthcare organizations have begun to implement interventions that supplement or redesign the system of care for frequently hospitalized patients.^5-7 Descriptive reviews of ambulatory "high-need, high-cost" patients emphasize complex case management and interdisciplinary, team-based care.^8,9 Prior systematic reviews of studies aimed at patients with high use of emergency care demonstrate improvements in social outcomes such as homelessness but mixed results in reducing emergency department (ED) use.¹⁰ However, we were unable to identify any prior reviews that evaluated interventions intended specifically for patients with frequent hospital admissions. Our objective in this systematic review was to characterize interventions for frequently admitted patients and determine whether these interventions decrease use of healthcare resources, improve health outcomes, and/or reduce costs.

Literature Search

We registered our study protocol in the PROSPERO database. A librarian (L.O.) collaboratively developed the search strategies with other review authors (A.G., B.H., N.N.) and in January 2018 ran searches on "super users," "high utilizers," and similar terms in the following databases: PubMed MEDLINE, Embase (embase.com), and Cochrane Central Register of Controlled Trials (CENTRAL) on the Wiley platform. The complete search strategies used are available in Appendix A.  

We attempted to discover additional studies by searching the reference lists of key publications and contacted authors of relevant abstracts to determine whether studies had been published or were planned for peer-reviewed publication. We also contacted authors of included studies to locate additional studies meeting inclusion criteria.

Data Collection Process

Studies were eligible for inclusion in our review if they were (1) published in a peer-reviewed source, (2) defined a study population of patients frequently admitted to inpatient medical services, (3) evaluated an intervention targeting frequently hospitalized patients, and (4) included patients who were >18 years old and (5) admitted as inpatients on medical services. Of note, studies with patients admitted to psychiatric, obstetric, or surgical wards were not included, as the authors did not define these as "general medicine" units. Studies focused solely on an ambulatory population were similarly excluded. Given the heterogeneity of how studies defined frequently hospitalized patients, we did not establish a prespecified number of admissions for inclusion to ensure that we did not exclude interventions not meeting a strict set of criteria. The goal was not to examine interventions to reduce all readmissions, but rather, to look at patients who were recurrently hospitalized. Thus, patients had to be repeatedly admitted, but we let the studies define that usage explicitly.  

Two members of a four-physician team (A.G., B.H., K.O., and N.N.) screened all initial results for eligibility through title and abstract review; potentially relevant articles were retained for full-text review to assess each study's eligibility. If a study's abstract did not clearly indicate whether inclusion criteria were met, we retained the article for full-text review. Two team members (A.G. and B.H.) independently reviewed the full text of each selected article to determine final inclusion in the study. The previously described inclusion criteria were again applied, and a final set of articles was identified for data extraction. Disagreements regarding inclusion in the final review (such as whether a study measured medical or psychiatric hospitalizations) were resolved through discussion among the entire four-physician review team to achieve consensus or, when required, by contacting authors of individual studies.  

Data Abstraction and Risk of Bias Assessment

After selecting the final set of articles, we abstracted data using a tool developed by the Cochrane Effective Practice and Organization of Care Group.¹¹ We then compiled study-level data into a single database for reporting. Extracted elements included study design, setting, patient characteristics, inclusion and exclusion criteria, control group identification, outcome measures, results, and length of follow-up. We also extracted individual characteristics of each intervention, including common intervention elements such as intervention setting, use of health information technology resources, and whether programs developed interdisciplinary care plans. We assessed the risk of bias of each study and the quality of studies using the Downs and Black Scale.^12,13 Two team members (A.G. and B.H.) independently assessed the risk of bias for all nine studies, and differences were resolved by consensus. Due to the variation in the outcomes used, we were unable to conduct a meta-analysis.

Search Results

We found a total of 4,762 references in the three databases. After de-duplication using the EndNote software, there were 3,314 references to screen. We identified 116 studies for full-text review. Of those, we selected nine studies that met the criteria for this study (Figure). The most common reason for exclusion of an article for full-text review was that the patients studied were not defined as high utilizers of inpatient resources and were instead high-utilizers of ambulatory or emergency care (32 studies). We identified five of the included studies through the primary search and four through review of the references of the included papers.

Study Designs and  Included Studies

Of the nine included studies, three were randomized controlled trials, three were controlled retrospective cohort studies, and three were uncontrolled pre-post studies. The key characteristics of each study are described in Table 1.^14-22 The included studies had different definitions for patients who were high utilizers of hospital care. Eight used a "threshold" model that predicted future admissions using past patterns; these studies included patients with at least two admissions over a period of 6 to 12 months, although many had higher thresholds. Zulman et al. used a prediction algorithm to identify patients at risk of future admission. Four studies also included some measure of medical complexity, such as a certain number of chronic medical conditions;^14,17,18,22  in contrast, Sledge et al. excluded the most complex and high-cost patients.²⁰

All studies measured hospital admissions as a primary or a secondary outcome (Table 1). Although all studies demonstrated a reduction in hospital admissions following implementation, those with the greatest reductions did not have a control group.^14,15,17 All three randomized controlled trials showed equal reductions in admission rates between the intervention and control groups.^18,20,22 Among those specifically examining readmissions to the hospital, similar trends emerged, although one study (Plant et al.) found a nonsignificant decrease in hospital readmissions (17% reduction in 24 months, P = .07).¹⁸

In the secondary outcome analysis, six of the nine studies found nonsignificant reductions in ED admissions (Table 1). Four studies measured costs to the hospital or the local hospital system, though none examined costs to patients or payors. Studies estimated cost differently, including the use of estimated hospital costs,^17,20 "facility patient costs" at the VA,²² and a combination of inpatient and ED costs.¹⁹ The latter study (Shah et al., which implemented complex case management services) was the only one to find a statistically significant decrease in mean cost per year pre- and postintervention ($20,298 versus $7,053, P < .001).¹⁹

Only one study measured the quality of life, finding no significant change in summary scores after the intervention compared with controls (93.4 versus 92, P = .32).21 Another study conducted at a VA clinic network found no difference in a patient activation scale following the intervention but found significantly increased satisfaction with overall VA care (3.16 versus 2.90, P = .04).²²

Intervention Characteristics

Intervention characteristics are summarized in Table 2. Although there was heterogeneity in study interventions, we identified common themes. Five of the nine interventions^14-17,22 consisted of interdisciplinary teams that included community health workers, nurses, social workers, and physicians. Physicians were not included on every team; three interventions used them in direct care roles while two others contained physicians as advisors or in indirect roles. Intervention teams also had a variable level of involvement in a patient's care. Mercer et al. developed care plans for patients without physical interaction,¹⁷ whereas Zulman et al. recruited patients to a separate, intensive outpatient clinic outside the usual VA care team structure.²²

The majority of interventions added direct services or support - most commonly, a social worker - to usual care processes. Patient panel sizes were relatively small, with most of the teams recruiting fewer than 150 patients per interdisciplinary team (range, 25-251). There was variation in the length of intervention, from 35 days of case management following hospital discharge to one year of intensive social work support to others of an indefinite length.^15,17,22
Additional common themes included caring for patients across settings and incorporating information technology (IT) into workflows. Four interventions reported either interacting with patients in multiple settings, such as the hospital, clinic, and day hospital, ED, at home, or in the community.^14,19,21,22 Two others^16,20 interacted with patients only in the clinic but expanded the scope of a "traditional" primary care practice to include open scheduling, flexible appointment times, interdisciplinary visits, or outreach. In addition, IT resources assisted seven of the nine interventions, most commonly by identifying eligible patients via an electronic data tracking system or by automated alerts when their patients arrived at affiliated care locations. 

Risk of Study Bias

We systematically assessed the risk of bias of the nine included studies (Appendix B). Using the scale published by Downs and Black, a point-based scale in which a score of 18 denotes a high-quality study, the studies in this review scored 15.55 on average (range 6-22, standard deviation [SD] 5.0). Four of the nine studies met the benchmark for high quality.^12,13,18-22 The risk of bias was highest for measures of internal validity and confounding (range 0-5, mean 2.83, SD 1.94). The risk of bias was lowest for reporting measures (range 0-13, mean 7.40, SD 3.43).

Overall, studies reported mixed results related to readmissions and hospital utilization. While low-quality studies found reductions in hospital use over time, higher quality studies found similar reductions in utilization between the intervention and control groups. Johnson et al. showed that frequent hospitalization rates in a cohort of high-utilizer patients declined naturally over the course of 1-2 years; only 10% of individuals in the initial cohort remained "chronically hospitalized."⁶ Thus, expanding on these findings, the decline in hospitalizations over time as observed in some of the studies included in this review may be due to study patients being identified during a "spike" in utilization, which naturally decreases as the underlying medical or social factors driving rehospitalization resolve. Alternatively, reduction in hospitalizations may represent patients choosing to pursue care at other neighboring hospitals.²³ No study included in our review evaluated healthcare use at institutions other than their study hospital or health system.

A striking theme of this review was the heterogeneity in each study's patient population. Thresholds for "high utilizers" varied from two hospital admissions in six months to two to three admissions in 30 days, to a combination of ED and hospital admissions, and to the use of predictive algorithms. A standard "case definition" for this population could guide future research, enabling comparison of outcomes across settings. Thus, we propose that future studies use three or more hospital admissions within six months when evaluating interventions targeting "high utilizer" patients. Although patients with one prior hospitalization in the past year are at elevated risk of rehospitalization,² we feel that a higher "threshold" for this population will identify those at the highest strata of risk. Although predictive models may be better than "threshold" models, more work in validating these tools needs to be done before these can be put to use across settings.

In contrast to interventions designed to reduce readmissions for heart failure, pneumonia, or other diagnoses, frequently admitted patients do not encompass one disease or pathology pattern. Rinehart et al., in a study characterizing frequently admitted patients across a health system, identified five "subgroups" of patients, including those with (1) unstable housing, (2) comorbid medical and psychiatric illness, (3) severe complex medical illness, (4) dual-diagnosis psychiatric illness and substance abuse, and (5) a combination of medical and psychosocial barriers.25 In light of this population's heterogeneity, interventions may need to be flexible and tailored to the needs of individual patients, while simultaneously accounting for the capabilities and priorities of the health system. More specific and standardized interventions, targeting more homogenous groups, may be appropriate for populations defined according to pathology (such as heart failure or sickle cell disease).²⁷

The components of interventions used for frequently hospitalized patients were diverse. Although most of the studies used interdisciplinary teams, they focused their efforts in a variety of settings, often crossing modern "boundaries of care" by providing direct or indirect input on care across healthcare settings. Care fragmentation probably plays an important role in the risk for readmissions in this population;9 as such, interventions that address factors across the continuum of care may be more likely to succeed.²¹ Notably, six of nine studies were conducted at academic medical centers and an additional one at a VA facility affiliated with an academic center. Only two were located at community-based clinical networks, indicating a theoretical potential for publication bias as academic centers may be more likely to study and publish their work. There may be successful interventions that have not been formally studied or published in the peer-reviewed literature.

The breadth of the outcome measures in the included studies raises questions about what metrics should define success. Although all the studies looked at hospital utilization and readmission, measure definitions varied. Importantly, a minority of studies investigated quality of life and patient satisfaction, outcomes that may ultimately provide a more fertile ground for inquiry and intervention. Two studies looked at quality of life as an outcome,^19,22 but only one found that patients reported increased satisfaction despite showing nonsignificant reductions in hospital use.²² As shown in multiple prior studies, patient engagement is associated with increased satisfaction and can be associated with lower healthcare costs.^26,27 Hibbard et al. have demonstrated that patient activation is a specific component of patient engagement and inversely impacts healthcare cost, with lower levels of patient activation showing increased costs in comparison to those patients more engaged in their own care.²⁷ By focusing on changing patients' perceptions about their own health and involvement in their own care team as a partner, programs may be able to make a greater impact.  

Our systematic review has several limitations. Although we used a search strategy designed to identify all relevant studies, reviewed the references of included studies, and contacted the authors, we identified only nine studies meeting our inclusion criteria. Four of the nine studies were identified from a manual review of references of the included studies, suggesting the possibility of a suboptimal search strategy. Although the inclusion of articles that appear in a check of reference lists is a valid step in the systematic review article acquisition process, we conducted a post hoc investigation of alternate search strategies. We checked the titles, abstracts, and subject headings of the four articles found by reference review to determine whether the original search could have been improved. An analysis of the articles revealed that the terminology used was not consistent with the super user/utilizer terminology we were operating under, and that the four articles used terms such as "high risk" and "complex patients," which are more generic than our targeted terms. Only on a careful read of the abstracts and full-text did we find that these articles were useful to the study. Adjusting the original search to include these general terms would have resulted in an unwieldy set of results; hence, we felt it best to adhere to our original search strategy.

Additional limitations include that only four of the nine included studies were at low risk of bias. In addition to limitations based on study design and small sample sizes, the interventions were often limited to a short period. In light of the multiple factors that contribute to frequent hospitalizations, some of which cannot be addressed quickly, studies to evaluate interventions for longer durations are warranted.  

We found mixed results for the effect of interventions on outcomes for frequently hospitalized patients. While low-quality studies found reductions in hospital use over time, higher quality studies generally found similar reductions in utilization between the intervention and control groups. The range of definitions, interventions, and outcomes used for frequently hospitalized patients is partly explained by the heterogeneity of the population. More rigorous studies using multifaceted interventions that adapt to patients' unique needs should be conducted to assess the effect on outcomes relevant to both providers and patients.

Acknowledgments

The authors would like the thank Dr. Luke Hansen, Dr. Margaret Chapman, and McKay Barra for their support and contributions to this paper and to Northwestern Memorial Hospital's CHAMP (Complex High Admission Management Program).

Disclosures

The authors have nothing to disclose.

Funding

The authors received no funding from external or internal sources for the completion of this project.

In recent years, hospitals and health systems have engaged in considerable efforts to reduce readmissions, in part due to financial incentives from the Medicare Hospital Readmission Reduction Program.^1,2 Though efforts to improve transitions of care for all patients are laudable, risk for readmission is not distributed equally; a small subset of patients accounts for a disproportionate number of hospital readmissions.³ This phenomenon of frequently hospitalized patients is similar to that seen in other populations in which a small proportion of patients account for a majority of healthcare utilization.^3,4

Recognizing that the current system of healthcare delivery does not meet the needs of this population, healthcare organizations have begun to implement interventions that supplement or redesign the system of care for frequently hospitalized patients.^5-7 Descriptive reviews of ambulatory "high-need, high-cost" patients emphasize complex case management and interdisciplinary, team-based care.^8,9 Prior systematic reviews of studies aimed at patients with high use of emergency care demonstrate improvements in social outcomes such as homelessness but mixed results in reducing emergency department (ED) use.¹⁰ However, we were unable to identify any prior reviews that evaluated interventions intended specifically for patients with frequent hospital admissions. Our objective in this systematic review was to characterize interventions for frequently admitted patients and determine whether these interventions decrease use of healthcare resources, improve health outcomes, and/or reduce costs.

Literature Search

We registered our study protocol in the PROSPERO database. A librarian (L.O.) collaboratively developed the search strategies with other review authors (A.G., B.H., N.N.) and in January 2018 ran searches on "super users," "high utilizers," and similar terms in the following databases: PubMed MEDLINE, Embase (embase.com), and Cochrane Central Register of Controlled Trials (CENTRAL) on the Wiley platform. The complete search strategies used are available in Appendix A.  

We attempted to discover additional studies by searching the reference lists of key publications and contacted authors of relevant abstracts to determine whether studies had been published or were planned for peer-reviewed publication. We also contacted authors of included studies to locate additional studies meeting inclusion criteria.

Data Collection Process

Studies were eligible for inclusion in our review if they were (1) published in a peer-reviewed source, (2) defined a study population of patients frequently admitted to inpatient medical services, (3) evaluated an intervention targeting frequently hospitalized patients, and (4) included patients who were >18 years old and (5) admitted as inpatients on medical services. Of note, studies with patients admitted to psychiatric, obstetric, or surgical wards were not included, as the authors did not define these as "general medicine" units. Studies focused solely on an ambulatory population were similarly excluded. Given the heterogeneity of how studies defined frequently hospitalized patients, we did not establish a prespecified number of admissions for inclusion to ensure that we did not exclude interventions not meeting a strict set of criteria. The goal was not to examine interventions to reduce all readmissions, but rather, to look at patients who were recurrently hospitalized. Thus, patients had to be repeatedly admitted, but we let the studies define that usage explicitly.  

Two members of a four-physician team (A.G., B.H., K.O., and N.N.) screened all initial results for eligibility through title and abstract review; potentially relevant articles were retained for full-text review to assess each study's eligibility. If a study's abstract did not clearly indicate whether inclusion criteria were met, we retained the article for full-text review. Two team members (A.G. and B.H.) independently reviewed the full text of each selected article to determine final inclusion in the study. The previously described inclusion criteria were again applied, and a final set of articles was identified for data extraction. Disagreements regarding inclusion in the final review (such as whether a study measured medical or psychiatric hospitalizations) were resolved through discussion among the entire four-physician review team to achieve consensus or, when required, by contacting authors of individual studies.  

Data Abstraction and Risk of Bias Assessment

After selecting the final set of articles, we abstracted data using a tool developed by the Cochrane Effective Practice and Organization of Care Group.¹¹ We then compiled study-level data into a single database for reporting. Extracted elements included study design, setting, patient characteristics, inclusion and exclusion criteria, control group identification, outcome measures, results, and length of follow-up. We also extracted individual characteristics of each intervention, including common intervention elements such as intervention setting, use of health information technology resources, and whether programs developed interdisciplinary care plans. We assessed the risk of bias of each study and the quality of studies using the Downs and Black Scale.^12,13 Two team members (A.G. and B.H.) independently assessed the risk of bias for all nine studies, and differences were resolved by consensus. Due to the variation in the outcomes used, we were unable to conduct a meta-analysis.

Search Results

We found a total of 4,762 references in the three databases. After de-duplication using the EndNote software, there were 3,314 references to screen. We identified 116 studies for full-text review. Of those, we selected nine studies that met the criteria for this study (Figure). The most common reason for exclusion of an article for full-text review was that the patients studied were not defined as high utilizers of inpatient resources and were instead high-utilizers of ambulatory or emergency care (32 studies). We identified five of the included studies through the primary search and four through review of the references of the included papers.

Study Designs and  Included Studies

Of the nine included studies, three were randomized controlled trials, three were controlled retrospective cohort studies, and three were uncontrolled pre-post studies. The key characteristics of each study are described in Table 1.^14-22 The included studies had different definitions for patients who were high utilizers of hospital care. Eight used a "threshold" model that predicted future admissions using past patterns; these studies included patients with at least two admissions over a period of 6 to 12 months, although many had higher thresholds. Zulman et al. used a prediction algorithm to identify patients at risk of future admission. Four studies also included some measure of medical complexity, such as a certain number of chronic medical conditions;^14,17,18,22  in contrast, Sledge et al. excluded the most complex and high-cost patients.²⁰

All studies measured hospital admissions as a primary or a secondary outcome (Table 1). Although all studies demonstrated a reduction in hospital admissions following implementation, those with the greatest reductions did not have a control group.^14,15,17 All three randomized controlled trials showed equal reductions in admission rates between the intervention and control groups.^18,20,22 Among those specifically examining readmissions to the hospital, similar trends emerged, although one study (Plant et al.) found a nonsignificant decrease in hospital readmissions (17% reduction in 24 months, P = .07).¹⁸

In the secondary outcome analysis, six of the nine studies found nonsignificant reductions in ED admissions (Table 1). Four studies measured costs to the hospital or the local hospital system, though none examined costs to patients or payors. Studies estimated cost differently, including the use of estimated hospital costs,^17,20 "facility patient costs" at the VA,²² and a combination of inpatient and ED costs.¹⁹ The latter study (Shah et al., which implemented complex case management services) was the only one to find a statistically significant decrease in mean cost per year pre- and postintervention ($20,298 versus $7,053, P < .001).¹⁹

Only one study measured the quality of life, finding no significant change in summary scores after the intervention compared with controls (93.4 versus 92, P = .32).21 Another study conducted at a VA clinic network found no difference in a patient activation scale following the intervention but found significantly increased satisfaction with overall VA care (3.16 versus 2.90, P = .04).²²

Intervention Characteristics

Intervention characteristics are summarized in Table 2. Although there was heterogeneity in study interventions, we identified common themes. Five of the nine interventions^14-17,22 consisted of interdisciplinary teams that included community health workers, nurses, social workers, and physicians. Physicians were not included on every team; three interventions used them in direct care roles while two others contained physicians as advisors or in indirect roles. Intervention teams also had a variable level of involvement in a patient's care. Mercer et al. developed care plans for patients without physical interaction,¹⁷ whereas Zulman et al. recruited patients to a separate, intensive outpatient clinic outside the usual VA care team structure.²²

The majority of interventions added direct services or support - most commonly, a social worker - to usual care processes. Patient panel sizes were relatively small, with most of the teams recruiting fewer than 150 patients per interdisciplinary team (range, 25-251). There was variation in the length of intervention, from 35 days of case management following hospital discharge to one year of intensive social work support to others of an indefinite length.^15,17,22
Additional common themes included caring for patients across settings and incorporating information technology (IT) into workflows. Four interventions reported either interacting with patients in multiple settings, such as the hospital, clinic, and day hospital, ED, at home, or in the community.^14,19,21,22 Two others^16,20 interacted with patients only in the clinic but expanded the scope of a "traditional" primary care practice to include open scheduling, flexible appointment times, interdisciplinary visits, or outreach. In addition, IT resources assisted seven of the nine interventions, most commonly by identifying eligible patients via an electronic data tracking system or by automated alerts when their patients arrived at affiliated care locations. 

Risk of Study Bias

We systematically assessed the risk of bias of the nine included studies (Appendix B). Using the scale published by Downs and Black, a point-based scale in which a score of 18 denotes a high-quality study, the studies in this review scored 15.55 on average (range 6-22, standard deviation [SD] 5.0). Four of the nine studies met the benchmark for high quality.^12,13,18-22 The risk of bias was highest for measures of internal validity and confounding (range 0-5, mean 2.83, SD 1.94). The risk of bias was lowest for reporting measures (range 0-13, mean 7.40, SD 3.43).

Overall, studies reported mixed results related to readmissions and hospital utilization. While low-quality studies found reductions in hospital use over time, higher quality studies found similar reductions in utilization between the intervention and control groups. Johnson et al. showed that frequent hospitalization rates in a cohort of high-utilizer patients declined naturally over the course of 1-2 years; only 10% of individuals in the initial cohort remained "chronically hospitalized."⁶ Thus, expanding on these findings, the decline in hospitalizations over time as observed in some of the studies included in this review may be due to study patients being identified during a "spike" in utilization, which naturally decreases as the underlying medical or social factors driving rehospitalization resolve. Alternatively, reduction in hospitalizations may represent patients choosing to pursue care at other neighboring hospitals.²³ No study included in our review evaluated healthcare use at institutions other than their study hospital or health system.

A striking theme of this review was the heterogeneity in each study's patient population. Thresholds for "high utilizers" varied from two hospital admissions in six months to two to three admissions in 30 days, to a combination of ED and hospital admissions, and to the use of predictive algorithms. A standard "case definition" for this population could guide future research, enabling comparison of outcomes across settings. Thus, we propose that future studies use three or more hospital admissions within six months when evaluating interventions targeting "high utilizer" patients. Although patients with one prior hospitalization in the past year are at elevated risk of rehospitalization,² we feel that a higher "threshold" for this population will identify those at the highest strata of risk. Although predictive models may be better than "threshold" models, more work in validating these tools needs to be done before these can be put to use across settings.

In contrast to interventions designed to reduce readmissions for heart failure, pneumonia, or other diagnoses, frequently admitted patients do not encompass one disease or pathology pattern. Rinehart et al., in a study characterizing frequently admitted patients across a health system, identified five "subgroups" of patients, including those with (1) unstable housing, (2) comorbid medical and psychiatric illness, (3) severe complex medical illness, (4) dual-diagnosis psychiatric illness and substance abuse, and (5) a combination of medical and psychosocial barriers.25 In light of this population's heterogeneity, interventions may need to be flexible and tailored to the needs of individual patients, while simultaneously accounting for the capabilities and priorities of the health system. More specific and standardized interventions, targeting more homogenous groups, may be appropriate for populations defined according to pathology (such as heart failure or sickle cell disease).²⁷

The components of interventions used for frequently hospitalized patients were diverse. Although most of the studies used interdisciplinary teams, they focused their efforts in a variety of settings, often crossing modern "boundaries of care" by providing direct or indirect input on care across healthcare settings. Care fragmentation probably plays an important role in the risk for readmissions in this population;9 as such, interventions that address factors across the continuum of care may be more likely to succeed.²¹ Notably, six of nine studies were conducted at academic medical centers and an additional one at a VA facility affiliated with an academic center. Only two were located at community-based clinical networks, indicating a theoretical potential for publication bias as academic centers may be more likely to study and publish their work. There may be successful interventions that have not been formally studied or published in the peer-reviewed literature.

The breadth of the outcome measures in the included studies raises questions about what metrics should define success. Although all the studies looked at hospital utilization and readmission, measure definitions varied. Importantly, a minority of studies investigated quality of life and patient satisfaction, outcomes that may ultimately provide a more fertile ground for inquiry and intervention. Two studies looked at quality of life as an outcome,^19,22 but only one found that patients reported increased satisfaction despite showing nonsignificant reductions in hospital use.²² As shown in multiple prior studies, patient engagement is associated with increased satisfaction and can be associated with lower healthcare costs.^26,27 Hibbard et al. have demonstrated that patient activation is a specific component of patient engagement and inversely impacts healthcare cost, with lower levels of patient activation showing increased costs in comparison to those patients more engaged in their own care.²⁷ By focusing on changing patients' perceptions about their own health and involvement in their own care team as a partner, programs may be able to make a greater impact.  

Our systematic review has several limitations. Although we used a search strategy designed to identify all relevant studies, reviewed the references of included studies, and contacted the authors, we identified only nine studies meeting our inclusion criteria. Four of the nine studies were identified from a manual review of references of the included studies, suggesting the possibility of a suboptimal search strategy. Although the inclusion of articles that appear in a check of reference lists is a valid step in the systematic review article acquisition process, we conducted a post hoc investigation of alternate search strategies. We checked the titles, abstracts, and subject headings of the four articles found by reference review to determine whether the original search could have been improved. An analysis of the articles revealed that the terminology used was not consistent with the super user/utilizer terminology we were operating under, and that the four articles used terms such as "high risk" and "complex patients," which are more generic than our targeted terms. Only on a careful read of the abstracts and full-text did we find that these articles were useful to the study. Adjusting the original search to include these general terms would have resulted in an unwieldy set of results; hence, we felt it best to adhere to our original search strategy.

Additional limitations include that only four of the nine included studies were at low risk of bias. In addition to limitations based on study design and small sample sizes, the interventions were often limited to a short period. In light of the multiple factors that contribute to frequent hospitalizations, some of which cannot be addressed quickly, studies to evaluate interventions for longer durations are warranted.  

We found mixed results for the effect of interventions on outcomes for frequently hospitalized patients. While low-quality studies found reductions in hospital use over time, higher quality studies generally found similar reductions in utilization between the intervention and control groups. The range of definitions, interventions, and outcomes used for frequently hospitalized patients is partly explained by the heterogeneity of the population. More rigorous studies using multifaceted interventions that adapt to patients' unique needs should be conducted to assess the effect on outcomes relevant to both providers and patients.

Acknowledgments

The authors would like the thank Dr. Luke Hansen, Dr. Margaret Chapman, and McKay Barra for their support and contributions to this paper and to Northwestern Memorial Hospital's CHAMP (Complex High Admission Management Program).

Disclosures

The authors have nothing to disclose.

Funding

The authors received no funding from external or internal sources for the completion of this project.

In recent years, hospitals and health systems have engaged in considerable efforts to reduce readmissions, in part due to financial incentives from the Medicare Hospital Readmission Reduction Program.^1,2 Though efforts to improve transitions of care for all patients are laudable, risk for readmission is not distributed equally; a small subset of patients accounts for a disproportionate number of hospital readmissions.³ This phenomenon of frequently hospitalized patients is similar to that seen in other populations in which a small proportion of patients account for a majority of healthcare utilization.^3,4

Recognizing that the current system of healthcare delivery does not meet the needs of this population, healthcare organizations have begun to implement interventions that supplement or redesign the system of care for frequently hospitalized patients.^5-7 Descriptive reviews of ambulatory "high-need, high-cost" patients emphasize complex case management and interdisciplinary, team-based care.^8,9 Prior systematic reviews of studies aimed at patients with high use of emergency care demonstrate improvements in social outcomes such as homelessness but mixed results in reducing emergency department (ED) use.¹⁰ However, we were unable to identify any prior reviews that evaluated interventions intended specifically for patients with frequent hospital admissions. Our objective in this systematic review was to characterize interventions for frequently admitted patients and determine whether these interventions decrease use of healthcare resources, improve health outcomes, and/or reduce costs.

Literature Search

We registered our study protocol in the PROSPERO database. A librarian (L.O.) collaboratively developed the search strategies with other review authors (A.G., B.H., N.N.) and in January 2018 ran searches on "super users," "high utilizers," and similar terms in the following databases: PubMed MEDLINE, Embase (embase.com), and Cochrane Central Register of Controlled Trials (CENTRAL) on the Wiley platform. The complete search strategies used are available in Appendix A.  

We attempted to discover additional studies by searching the reference lists of key publications and contacted authors of relevant abstracts to determine whether studies had been published or were planned for peer-reviewed publication. We also contacted authors of included studies to locate additional studies meeting inclusion criteria.

Data Collection Process

Studies were eligible for inclusion in our review if they were (1) published in a peer-reviewed source, (2) defined a study population of patients frequently admitted to inpatient medical services, (3) evaluated an intervention targeting frequently hospitalized patients, and (4) included patients who were >18 years old and (5) admitted as inpatients on medical services. Of note, studies with patients admitted to psychiatric, obstetric, or surgical wards were not included, as the authors did not define these as "general medicine" units. Studies focused solely on an ambulatory population were similarly excluded. Given the heterogeneity of how studies defined frequently hospitalized patients, we did not establish a prespecified number of admissions for inclusion to ensure that we did not exclude interventions not meeting a strict set of criteria. The goal was not to examine interventions to reduce all readmissions, but rather, to look at patients who were recurrently hospitalized. Thus, patients had to be repeatedly admitted, but we let the studies define that usage explicitly.  

Two members of a four-physician team (A.G., B.H., K.O., and N.N.) screened all initial results for eligibility through title and abstract review; potentially relevant articles were retained for full-text review to assess each study's eligibility. If a study's abstract did not clearly indicate whether inclusion criteria were met, we retained the article for full-text review. Two team members (A.G. and B.H.) independently reviewed the full text of each selected article to determine final inclusion in the study. The previously described inclusion criteria were again applied, and a final set of articles was identified for data extraction. Disagreements regarding inclusion in the final review (such as whether a study measured medical or psychiatric hospitalizations) were resolved through discussion among the entire four-physician review team to achieve consensus or, when required, by contacting authors of individual studies.  

Data Abstraction and Risk of Bias Assessment

After selecting the final set of articles, we abstracted data using a tool developed by the Cochrane Effective Practice and Organization of Care Group.¹¹ We then compiled study-level data into a single database for reporting. Extracted elements included study design, setting, patient characteristics, inclusion and exclusion criteria, control group identification, outcome measures, results, and length of follow-up. We also extracted individual characteristics of each intervention, including common intervention elements such as intervention setting, use of health information technology resources, and whether programs developed interdisciplinary care plans. We assessed the risk of bias of each study and the quality of studies using the Downs and Black Scale.^12,13 Two team members (A.G. and B.H.) independently assessed the risk of bias for all nine studies, and differences were resolved by consensus. Due to the variation in the outcomes used, we were unable to conduct a meta-analysis.

Search Results

We found a total of 4,762 references in the three databases. After de-duplication using the EndNote software, there were 3,314 references to screen. We identified 116 studies for full-text review. Of those, we selected nine studies that met the criteria for this study (Figure). The most common reason for exclusion of an article for full-text review was that the patients studied were not defined as high utilizers of inpatient resources and were instead high-utilizers of ambulatory or emergency care (32 studies). We identified five of the included studies through the primary search and four through review of the references of the included papers.

Study Designs and  Included Studies

Of the nine included studies, three were randomized controlled trials, three were controlled retrospective cohort studies, and three were uncontrolled pre-post studies. The key characteristics of each study are described in Table 1.^14-22 The included studies had different definitions for patients who were high utilizers of hospital care. Eight used a "threshold" model that predicted future admissions using past patterns; these studies included patients with at least two admissions over a period of 6 to 12 months, although many had higher thresholds. Zulman et al. used a prediction algorithm to identify patients at risk of future admission. Four studies also included some measure of medical complexity, such as a certain number of chronic medical conditions;^14,17,18,22  in contrast, Sledge et al. excluded the most complex and high-cost patients.²⁰

All studies measured hospital admissions as a primary or a secondary outcome (Table 1). Although all studies demonstrated a reduction in hospital admissions following implementation, those with the greatest reductions did not have a control group.^14,15,17 All three randomized controlled trials showed equal reductions in admission rates between the intervention and control groups.^18,20,22 Among those specifically examining readmissions to the hospital, similar trends emerged, although one study (Plant et al.) found a nonsignificant decrease in hospital readmissions (17% reduction in 24 months, P = .07).¹⁸

In the secondary outcome analysis, six of the nine studies found nonsignificant reductions in ED admissions (Table 1). Four studies measured costs to the hospital or the local hospital system, though none examined costs to patients or payors. Studies estimated cost differently, including the use of estimated hospital costs,^17,20 "facility patient costs" at the VA,²² and a combination of inpatient and ED costs.¹⁹ The latter study (Shah et al., which implemented complex case management services) was the only one to find a statistically significant decrease in mean cost per year pre- and postintervention ($20,298 versus $7,053, P < .001).¹⁹

Only one study measured the quality of life, finding no significant change in summary scores after the intervention compared with controls (93.4 versus 92, P = .32).21 Another study conducted at a VA clinic network found no difference in a patient activation scale following the intervention but found significantly increased satisfaction with overall VA care (3.16 versus 2.90, P = .04).²²

Intervention Characteristics

Intervention characteristics are summarized in Table 2. Although there was heterogeneity in study interventions, we identified common themes. Five of the nine interventions^14-17,22 consisted of interdisciplinary teams that included community health workers, nurses, social workers, and physicians. Physicians were not included on every team; three interventions used them in direct care roles while two others contained physicians as advisors or in indirect roles. Intervention teams also had a variable level of involvement in a patient's care. Mercer et al. developed care plans for patients without physical interaction,¹⁷ whereas Zulman et al. recruited patients to a separate, intensive outpatient clinic outside the usual VA care team structure.²²

The majority of interventions added direct services or support - most commonly, a social worker - to usual care processes. Patient panel sizes were relatively small, with most of the teams recruiting fewer than 150 patients per interdisciplinary team (range, 25-251). There was variation in the length of intervention, from 35 days of case management following hospital discharge to one year of intensive social work support to others of an indefinite length.^15,17,22
Additional common themes included caring for patients across settings and incorporating information technology (IT) into workflows. Four interventions reported either interacting with patients in multiple settings, such as the hospital, clinic, and day hospital, ED, at home, or in the community.^14,19,21,22 Two others^16,20 interacted with patients only in the clinic but expanded the scope of a "traditional" primary care practice to include open scheduling, flexible appointment times, interdisciplinary visits, or outreach. In addition, IT resources assisted seven of the nine interventions, most commonly by identifying eligible patients via an electronic data tracking system or by automated alerts when their patients arrived at affiliated care locations. 

Risk of Study Bias

We systematically assessed the risk of bias of the nine included studies (Appendix B). Using the scale published by Downs and Black, a point-based scale in which a score of 18 denotes a high-quality study, the studies in this review scored 15.55 on average (range 6-22, standard deviation [SD] 5.0). Four of the nine studies met the benchmark for high quality.^12,13,18-22 The risk of bias was highest for measures of internal validity and confounding (range 0-5, mean 2.83, SD 1.94). The risk of bias was lowest for reporting measures (range 0-13, mean 7.40, SD 3.43).

Overall, studies reported mixed results related to readmissions and hospital utilization. While low-quality studies found reductions in hospital use over time, higher quality studies found similar reductions in utilization between the intervention and control groups. Johnson et al. showed that frequent hospitalization rates in a cohort of high-utilizer patients declined naturally over the course of 1-2 years; only 10% of individuals in the initial cohort remained "chronically hospitalized."⁶ Thus, expanding on these findings, the decline in hospitalizations over time as observed in some of the studies included in this review may be due to study patients being identified during a "spike" in utilization, which naturally decreases as the underlying medical or social factors driving rehospitalization resolve. Alternatively, reduction in hospitalizations may represent patients choosing to pursue care at other neighboring hospitals.²³ No study included in our review evaluated healthcare use at institutions other than their study hospital or health system.

A striking theme of this review was the heterogeneity in each study's patient population. Thresholds for "high utilizers" varied from two hospital admissions in six months to two to three admissions in 30 days, to a combination of ED and hospital admissions, and to the use of predictive algorithms. A standard "case definition" for this population could guide future research, enabling comparison of outcomes across settings. Thus, we propose that future studies use three or more hospital admissions within six months when evaluating interventions targeting "high utilizer" patients. Although patients with one prior hospitalization in the past year are at elevated risk of rehospitalization,² we feel that a higher "threshold" for this population will identify those at the highest strata of risk. Although predictive models may be better than "threshold" models, more work in validating these tools needs to be done before these can be put to use across settings.

In contrast to interventions designed to reduce readmissions for heart failure, pneumonia, or other diagnoses, frequently admitted patients do not encompass one disease or pathology pattern. Rinehart et al., in a study characterizing frequently admitted patients across a health system, identified five "subgroups" of patients, including those with (1) unstable housing, (2) comorbid medical and psychiatric illness, (3) severe complex medical illness, (4) dual-diagnosis psychiatric illness and substance abuse, and (5) a combination of medical and psychosocial barriers.25 In light of this population's heterogeneity, interventions may need to be flexible and tailored to the needs of individual patients, while simultaneously accounting for the capabilities and priorities of the health system. More specific and standardized interventions, targeting more homogenous groups, may be appropriate for populations defined according to pathology (such as heart failure or sickle cell disease).²⁷

The components of interventions used for frequently hospitalized patients were diverse. Although most of the studies used interdisciplinary teams, they focused their efforts in a variety of settings, often crossing modern "boundaries of care" by providing direct or indirect input on care across healthcare settings. Care fragmentation probably plays an important role in the risk for readmissions in this population;9 as such, interventions that address factors across the continuum of care may be more likely to succeed.²¹ Notably, six of nine studies were conducted at academic medical centers and an additional one at a VA facility affiliated with an academic center. Only two were located at community-based clinical networks, indicating a theoretical potential for publication bias as academic centers may be more likely to study and publish their work. There may be successful interventions that have not been formally studied or published in the peer-reviewed literature.

The breadth of the outcome measures in the included studies raises questions about what metrics should define success. Although all the studies looked at hospital utilization and readmission, measure definitions varied. Importantly, a minority of studies investigated quality of life and patient satisfaction, outcomes that may ultimately provide a more fertile ground for inquiry and intervention. Two studies looked at quality of life as an outcome,^19,22 but only one found that patients reported increased satisfaction despite showing nonsignificant reductions in hospital use.²² As shown in multiple prior studies, patient engagement is associated with increased satisfaction and can be associated with lower healthcare costs.^26,27 Hibbard et al. have demonstrated that patient activation is a specific component of patient engagement and inversely impacts healthcare cost, with lower levels of patient activation showing increased costs in comparison to those patients more engaged in their own care.²⁷ By focusing on changing patients' perceptions about their own health and involvement in their own care team as a partner, programs may be able to make a greater impact.  

Our systematic review has several limitations. Although we used a search strategy designed to identify all relevant studies, reviewed the references of included studies, and contacted the authors, we identified only nine studies meeting our inclusion criteria. Four of the nine studies were identified from a manual review of references of the included studies, suggesting the possibility of a suboptimal search strategy. Although the inclusion of articles that appear in a check of reference lists is a valid step in the systematic review article acquisition process, we conducted a post hoc investigation of alternate search strategies. We checked the titles, abstracts, and subject headings of the four articles found by reference review to determine whether the original search could have been improved. An analysis of the articles revealed that the terminology used was not consistent with the super user/utilizer terminology we were operating under, and that the four articles used terms such as "high risk" and "complex patients," which are more generic than our targeted terms. Only on a careful read of the abstracts and full-text did we find that these articles were useful to the study. Adjusting the original search to include these general terms would have resulted in an unwieldy set of results; hence, we felt it best to adhere to our original search strategy.

Additional limitations include that only four of the nine included studies were at low risk of bias. In addition to limitations based on study design and small sample sizes, the interventions were often limited to a short period. In light of the multiple factors that contribute to frequent hospitalizations, some of which cannot be addressed quickly, studies to evaluate interventions for longer durations are warranted.  

We found mixed results for the effect of interventions on outcomes for frequently hospitalized patients. While low-quality studies found reductions in hospital use over time, higher quality studies generally found similar reductions in utilization between the intervention and control groups. The range of definitions, interventions, and outcomes used for frequently hospitalized patients is partly explained by the heterogeneity of the population. More rigorous studies using multifaceted interventions that adapt to patients' unique needs should be conducted to assess the effect on outcomes relevant to both providers and patients.

Acknowledgments

The authors would like the thank Dr. Luke Hansen, Dr. Margaret Chapman, and McKay Barra for their support and contributions to this paper and to Northwestern Memorial Hospital's CHAMP (Complex High Admission Management Program).

Disclosures

The authors have nothing to disclose.

Funding

The authors received no funding from external or internal sources for the completion of this project.