Affiliations
Division of Infectious Diseases, Department of Pediatrics, Children's Mercy Hospital, University of Missouri–Kansas City School of Medicine, Kansas City, Missouri
Given name(s)
Fran
Family name
Balamuth
Degrees
MD, PhD, MSCE

Association of Herpes Simplex Virus Testing with Hospital Length of Stay for Infants ≤60 Days of Age Undergoing Evaluation for Meningitis

Article Type
Changed
Sun, 08/04/2019 - 22:59

Neonatal herpes simplex virus (HSV) is associated with significant morbidity and mortality,1 particularly when the diagnosis or treatment is delayed.2 Therefore, many infants aged ≤60 days being evaluated for meningitis undergo cerebrospinal fluid (CSF) HSV polymerase chain reaction (PCR) testing even though the risk of HSV infection is low [estimated at 0.4% of those undergoing evaluation for central nervous system (CNS) infection].3 A single-center study demonstrated that CSF HSV PCR testing increases the hospital length of stay (LOS) for infants aged ≤56 days,4 although these single-center findings may not be generalizable. To this end, we measured the association between CSF HSV PCR testing and LOS in a multicenter cohort of hospitalized young infants.

METHODS

Study Design

We conducted a planned secondary analysis of a retrospective cohort of infants aged ≤60 days who presented to the emergency department (ED) between January 1, 2005 and December 31, 2013, enrolled in the Pediatric Emergency Medicine Collaborative Research Committee (PEM CRC) HSV study.3 Our study was limited to the 20 hospitals that contributed hospital LOS data. The study protocol was approved by each site’s institutional review board with permission for data sharing.

Study Population

Eligible infants were identified at each site using a site-specific electronic search strategy. Infants were eligible for inclusion if a CSF culture was obtained in the ED or within 24 hours of ED arrival. We excluded infants who were discharged from the ED and those with missing hospital LOS data.

 

 

Data Collection

Site investigators extracted the following data elements either electronically or from medical records: patient demographics; ED arrival date and time; hospital discharge date and time; urinalysis results; peripheral and CSF cell counts; blood, urine, and CSF bacterial culture results; as well as the results of HSV PCR and viral cultures. Infants with growth of a pathogen in blood or CSF, or a catheterized urine culture with ≥50,000 colony-forming units (CFUs)/mL of a single pathogenic bacteria, or 10,000-50,000 CFUs/mL of a single pathogenic bacteria with an abnormal urinalysis (ie, positive nitrite or leukocyte esterase on urine dipstick or >5 white blood cells [WBCs] per high power field on urine microscopy) were classified as having a serious bacterial infection (SBI).5,6 Infants with a positive HSV PCR or viral culture from any site were classified as having HSV infection.3 Hospitalized infants who did not have an HSV PCR test performed were assumed not to have HSV disease if not diagnosed during the hospital stay or repeat ED encounter.3

Outcome Measures

The primary outcome was hospital LOS, defined at all hospitals as the time from ED arrival to provider signature of the hospital discharge order, calculated in minutes and then converted into days.

Statistical Analysis

We described LOS using medians with interquartile ranges (IQR) and compared between infants with and without a CSF HSV PCR test performed using the Mann–Whitney U test. To evaluate the association between performance of CSF HSV PCR testing and hospital LOS, we used negative binomial regression given the count variable outcome (LOS) with an overdispersed distribution. For this analysis, we clustered by hospital after adjusting for the following factors determined a priori: age, gender, study year, and presence of serious bacterial or HSV infection. Using the relative marginal modeled estimates of LOS (tested vs not tested), we determined the percentage increase in LOS. We then repeated the analyses after stratifying by the location of testing (ie, in-house vs send-out), age (≤28 days vs 29-60 days), and presence or absence of CSF pleocytosis (defined as a CSF WBC of ≥16 cells/mm3for infants aged ≤28 days and ≥10 cells/mm3for infants aged 29-60 days),7 because infants aged 29-60 days and those without CSF pleocytosis are reported to be at very low risk for CNS HSV infection.3,8 We utilized Stata Data Analysis and Statistical Software, version 15.0 (StataCorp, Inc.; College Station, Texas) for statistical analyses.

RESULTS

Of 24,103 infants with CSF cultures obtained at the 20 participating sites, we excluded 2,673 (11.1%) discharged from the ED or with missing disposition and 934 (3.9%) with missing LOS, leaving a study cohort of 20,496 infants (Figure). Overall, 1,780 infants (8.7%) had an SBI and 99 (0.5%) had an HSV infection, of which 46 (46.5%) had a CNS HSV infection.

Among the 20,496 study infants, 7,399 (36.1%) had a CSF HSV PCR test performed; 5,935 infants (80.2% of those tested) had in-house and 1,464 (19.8%) had send-out testing. Among infants with available CSF cell counts, a CSF HSV PCR test was more commonly performed in infants with CSF pleocytosis than in those without (1,865/4,439 [42.0%] with CSF pleocytosis vs 3,705/12,002 [30.9%] without CSF pleocytosis; odds ratio [OR] 1.6, 95% CI 1.5-1.7). Of the 7,399 infants who had a CSF HSV PCR test performed, 46 (0.6%) had a positive test. Of the tested infants, 5,570 (75.3%) had an available CSF WBC count; a positive CSF HSV PCR test was more common in infants with CSF pleocytosis than in those without (25 positive tests/1,865 infants with CSF pleocytosis [1.3%] vs 9/3,705 [0.2%] without CSF pleocytosis; OR 5.6, 95% CI 2.6-12.0). Among the 5,308 infants aged 29-60 days without CSF pleocytosis, 1,110 (20.9%) had a CSF HSV PCR test performed and only one infant (0.09% of those tested) had a positive test.

Without adjustment, infants with a CSF HSV PCR test had a longer median LOS than infants who were not tested (2.5 vs 2.3 days; P < .001). After adjustment, infants with a CSF HSV PCR test performed had a 23% longer duration of hospitalization. The association between testing and LOS was similar for older vs younger infants, infants with and without CSF pleocytosis, and in-house vs send-out testing (Table).

 

 

DISCUSSION

In a large, multicenter cohort of more than 20,000 hospitalized infants aged ≤60 days undergoing evaluation for meningitis, we examined the association of CSF HSV PCR testing with hospital LOS. Approximately one-third of study infants had a CSF HSV PCR test obtained. After adjustment for patient- and hospital-level factors, the treating clinician’s decision to obtain a CSF HSV PCR test was associated with a 23% longer hospital LOS (nearly one-half day).

Our findings are consistent with those of previous studies. First, our observed association of the decision to obtain a CSF HSV PCR test and LOS was similar in magnitude to that of a previous single-center investigation.4 Second, we also found that older infants and those without CSF pleocytosis were at very low risk of HSV infection.3,8 For the otherwise low-risk infants, the longer LOS may be due to delays in obtaining CSF HSV PCR test results, which should be explored in future research. Our study has greater generalizability than previous single-center studies by substantially increasing the population size as well as the variety of clinical settings. Ensuring clinicians’ access to rapid HSV PCR testing platforms will further mitigate the impact of HSV testing on LOS.

When deciding to perform a CSF HSV PCR test for infants aged ≤60 days, clinicians must balance the low incidence of neonatal HSV3 with the risk of delayed diagnosis and treatment of HSV infection, which include neurologic sequelae or even death.1,2 As infants with CNS HSV infection commonly present nonspecifically and only a minority of infected infants have skin vesicles,1 controversy exists as to which infants should be evaluated for HSV infection, resulting in considerable variability in HSV testing.3 Some clinicians advocate for more conservative testing strategies that include the performance of CSF HSV PCR testing in all febrile infants aged ≤21 days.9 Others suggest limiting testing to infants who meet high-risk criteria (eg, seizures, ill-appearance, or CSF pleocytosis).10,11 Further investigation will need to elucidate the clinical and laboratory predictors of HSV infection to identify those infants who would benefit most from HSV testing as well as the outcomes of infants not tested.

Our study has several limitations. First, we could not determine the reason why clinicians elected to obtain a CSF HSV PCR test, and we do not know the test turnaround time or the time when results became available to the clinical team. Second, we did not abstract clinical data such as ill-appearance or seizures. Although we adjusted for the presence of serious bacterial or HSV infection as proxy measures for illness severity, it is possible that other clinical factors were associated with HSV testing and LOS. Third, although we adjusted for patient- and hospital-level factors in our regression model, the potential for residual confounding persists. Fourth, we did not explore acyclovir administration as a factor associated with LOS as some sites did not provide data on acyclovir. Fifth, we did not evaluate the impact of HSV testing of other sample types (eg, blood or skin) on LOS. Sixth, our study was conducted primarily at children’s hospitals, and our findings may not be generalizable to general hospitals with hospitalized neonates.

 

 

CONCLUSIONS

For infants aged ≤60 days undergoing evaluation for meningitis, CSF HSV PCR testing was associated with a slightly longer hospital LOS. Improved methods to identify and target testing to higher risk infants may mitigate the impact on LOS for low-risk infants.

Acknowledgments

The authors acknowledge the following collaborators in the Pediatric Emergency Medicine Clinical Research Network (PEM CRC) Herpes Simplex Virus (HSV) Study Group who collected data for this study and/or the parent study: Joseph L Arms, MD (Minneapolis, Minnesota), Stuart A Bradin, DO (Ann Arbor, Michigan), Sarah J Curtis, MD, MSc (Edmonton, Alberta, Canada), Paul T Ishimine, MD (San Diego, California), Dina Kulik, MD (Toronto, Ontario, Canada), Prashant Mahajan, MD, MPH, MBA (Ann Arbor, Michigan), Aaron S Miller, MD, MSPH (St. Louis, Missouri), Pamela J Okada, MD (Dallas, Texas), Christopher M Pruitt, MD (Birmingham, Alabama), Suzanne M Schmidt, MD (Chicago, Illinois), David Schnadower, Amy D Thompson, MD (Wilmington, Delaware), Joanna E Thomson, MD, MPH (Cincinnati, Ohio), MD, MPH (St. Louis, Missouri), and Neil G. Uspal, MD (Seattle, Washington).

Disclosures

Dr. Aronson reports grants from the Agency for Healthcare Research and Quality during the conduct of the study. Dr. Shah reports grants from Patient-Centered Outcomes Research Institute, grants from the National Institute of Allergy and Infectious Diseases, and grants from the National Heart Lung Blood Institute, outside the submitted work. Dr. Shah is the Editor-in-Chief of the Journal of Hospital Medicine. All other authors have no conflicts of interest or financial relationships relevant to this article to disclose.

Funding

This project was supported by the Section of Emergency Medicine of the American Academy of Pediatrics (AAP) and Baylor College of Medicine and by the grant number K08HS026006 (Aronson) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Stephen Freedman is supported by the Alberta Children’s Hospital Foundation Professorship in Child Health and Wellness.

 

References

1. Kimberlin DW, Lin CY, Jacobs RF, et al. Natural history of neonatal herpes simplex virus infections in the acyclovir era. Pediatrics. 2001;108(2):223-229. PubMed
2. Shah SS, Aronson PL, Mohamad Z, Lorch SA. Delayed acyclovir therapy and death among neonates with herpes simplex virus infection. Pediatrics. 2011;128(6):1153-1160. https://doi.org/10.1136/eb-2012-100674.
3. Cruz AT, Freedman SB, Kulik DM, et al. Herpes simplex virus infection in infants undergoing meningitis evaluation. Pediatrics. 2018;141(2):e20171688. https://doi.org/10.1542/peds.2017-1688.
4. Shah SS, Volk J, Mohamad Z, Hodinka RL, Zorc JJ. Herpes simplex virus testing and hospital length of stay in neonates and young infants. J Pediatr. 2010;156(5):738-743. https://doi.org/10.1016/j.jpeds.2009.11.079.
5. Mahajan P, Kuppermann N, Mejias A, et al. Association of RNA biosignatures with bacterial infections in febrile infants aged 60 days or younger. JAMA. 2016;316(8):846-857. https://doi.org/10.1001/jama.2016.9207.
6. Schnadower D, Kuppermann N, Macias CG, et al. Febrile infants with urinary tract infections at very low risk for adverse events and bacteremia. Pediatrics. 2010;126(6):1074-1083. https://doi.org/10.1542/peds.2010-0479.
7. Thomson J, Sucharew H, Cruz AT, et al. Cerebrospinal fluid reference values for young infants undergoing lumbar puncture. Pediatrics. 2018;141(3):e20173405. https://doi.org/10.1542/peds.2017-3405.
8. Caviness AC, Demmler GJ, Almendarez Y, Selwyn BJ. The prevalence of neonatal herpes simplex virus infection compared with serious bacterial illness in hospitalized neonates. J Pediatr. 2008;153(2):164-169. https://doi.org/10.1016/j.jpeds.2008.02.031.
9. Long SS. In defense of empiric acyclovir therapy in certain neonates. J Pediatr. 2008;153(2):157-158. https://doi.org/10.1016/j.jpeds.2008.04.071.
10. Brower L, Schondelmeyer A, Wilson P, Shah SS. Testing and empiric treatment for neonatal herpes simplex virus: challenges and opportunities for improving the value of care. Hosp Pediatr. 2016;6(2):108-111. https://doi.org/10.1542/hpeds.2015-0166.
11. Kimberlin DW. When should you initiate acyclovir therapy in a neonate? J Pediatr. 2008;153(2):155-156. https://doi.org/10.1016/j.jpeds.2008.04.027.

Article PDF
Issue
Journal of Hospital Medicine 14(8)
Publications
Topics
Page Number
492-495
Sections
Article PDF
Article PDF

Neonatal herpes simplex virus (HSV) is associated with significant morbidity and mortality,1 particularly when the diagnosis or treatment is delayed.2 Therefore, many infants aged ≤60 days being evaluated for meningitis undergo cerebrospinal fluid (CSF) HSV polymerase chain reaction (PCR) testing even though the risk of HSV infection is low [estimated at 0.4% of those undergoing evaluation for central nervous system (CNS) infection].3 A single-center study demonstrated that CSF HSV PCR testing increases the hospital length of stay (LOS) for infants aged ≤56 days,4 although these single-center findings may not be generalizable. To this end, we measured the association between CSF HSV PCR testing and LOS in a multicenter cohort of hospitalized young infants.

METHODS

Study Design

We conducted a planned secondary analysis of a retrospective cohort of infants aged ≤60 days who presented to the emergency department (ED) between January 1, 2005 and December 31, 2013, enrolled in the Pediatric Emergency Medicine Collaborative Research Committee (PEM CRC) HSV study.3 Our study was limited to the 20 hospitals that contributed hospital LOS data. The study protocol was approved by each site’s institutional review board with permission for data sharing.

Study Population

Eligible infants were identified at each site using a site-specific electronic search strategy. Infants were eligible for inclusion if a CSF culture was obtained in the ED or within 24 hours of ED arrival. We excluded infants who were discharged from the ED and those with missing hospital LOS data.

 

 

Data Collection

Site investigators extracted the following data elements either electronically or from medical records: patient demographics; ED arrival date and time; hospital discharge date and time; urinalysis results; peripheral and CSF cell counts; blood, urine, and CSF bacterial culture results; as well as the results of HSV PCR and viral cultures. Infants with growth of a pathogen in blood or CSF, or a catheterized urine culture with ≥50,000 colony-forming units (CFUs)/mL of a single pathogenic bacteria, or 10,000-50,000 CFUs/mL of a single pathogenic bacteria with an abnormal urinalysis (ie, positive nitrite or leukocyte esterase on urine dipstick or >5 white blood cells [WBCs] per high power field on urine microscopy) were classified as having a serious bacterial infection (SBI).5,6 Infants with a positive HSV PCR or viral culture from any site were classified as having HSV infection.3 Hospitalized infants who did not have an HSV PCR test performed were assumed not to have HSV disease if not diagnosed during the hospital stay or repeat ED encounter.3

Outcome Measures

The primary outcome was hospital LOS, defined at all hospitals as the time from ED arrival to provider signature of the hospital discharge order, calculated in minutes and then converted into days.

Statistical Analysis

We described LOS using medians with interquartile ranges (IQR) and compared between infants with and without a CSF HSV PCR test performed using the Mann–Whitney U test. To evaluate the association between performance of CSF HSV PCR testing and hospital LOS, we used negative binomial regression given the count variable outcome (LOS) with an overdispersed distribution. For this analysis, we clustered by hospital after adjusting for the following factors determined a priori: age, gender, study year, and presence of serious bacterial or HSV infection. Using the relative marginal modeled estimates of LOS (tested vs not tested), we determined the percentage increase in LOS. We then repeated the analyses after stratifying by the location of testing (ie, in-house vs send-out), age (≤28 days vs 29-60 days), and presence or absence of CSF pleocytosis (defined as a CSF WBC of ≥16 cells/mm3for infants aged ≤28 days and ≥10 cells/mm3for infants aged 29-60 days),7 because infants aged 29-60 days and those without CSF pleocytosis are reported to be at very low risk for CNS HSV infection.3,8 We utilized Stata Data Analysis and Statistical Software, version 15.0 (StataCorp, Inc.; College Station, Texas) for statistical analyses.

RESULTS

Of 24,103 infants with CSF cultures obtained at the 20 participating sites, we excluded 2,673 (11.1%) discharged from the ED or with missing disposition and 934 (3.9%) with missing LOS, leaving a study cohort of 20,496 infants (Figure). Overall, 1,780 infants (8.7%) had an SBI and 99 (0.5%) had an HSV infection, of which 46 (46.5%) had a CNS HSV infection.

Among the 20,496 study infants, 7,399 (36.1%) had a CSF HSV PCR test performed; 5,935 infants (80.2% of those tested) had in-house and 1,464 (19.8%) had send-out testing. Among infants with available CSF cell counts, a CSF HSV PCR test was more commonly performed in infants with CSF pleocytosis than in those without (1,865/4,439 [42.0%] with CSF pleocytosis vs 3,705/12,002 [30.9%] without CSF pleocytosis; odds ratio [OR] 1.6, 95% CI 1.5-1.7). Of the 7,399 infants who had a CSF HSV PCR test performed, 46 (0.6%) had a positive test. Of the tested infants, 5,570 (75.3%) had an available CSF WBC count; a positive CSF HSV PCR test was more common in infants with CSF pleocytosis than in those without (25 positive tests/1,865 infants with CSF pleocytosis [1.3%] vs 9/3,705 [0.2%] without CSF pleocytosis; OR 5.6, 95% CI 2.6-12.0). Among the 5,308 infants aged 29-60 days without CSF pleocytosis, 1,110 (20.9%) had a CSF HSV PCR test performed and only one infant (0.09% of those tested) had a positive test.

Without adjustment, infants with a CSF HSV PCR test had a longer median LOS than infants who were not tested (2.5 vs 2.3 days; P < .001). After adjustment, infants with a CSF HSV PCR test performed had a 23% longer duration of hospitalization. The association between testing and LOS was similar for older vs younger infants, infants with and without CSF pleocytosis, and in-house vs send-out testing (Table).

 

 

DISCUSSION

In a large, multicenter cohort of more than 20,000 hospitalized infants aged ≤60 days undergoing evaluation for meningitis, we examined the association of CSF HSV PCR testing with hospital LOS. Approximately one-third of study infants had a CSF HSV PCR test obtained. After adjustment for patient- and hospital-level factors, the treating clinician’s decision to obtain a CSF HSV PCR test was associated with a 23% longer hospital LOS (nearly one-half day).

Our findings are consistent with those of previous studies. First, our observed association of the decision to obtain a CSF HSV PCR test and LOS was similar in magnitude to that of a previous single-center investigation.4 Second, we also found that older infants and those without CSF pleocytosis were at very low risk of HSV infection.3,8 For the otherwise low-risk infants, the longer LOS may be due to delays in obtaining CSF HSV PCR test results, which should be explored in future research. Our study has greater generalizability than previous single-center studies by substantially increasing the population size as well as the variety of clinical settings. Ensuring clinicians’ access to rapid HSV PCR testing platforms will further mitigate the impact of HSV testing on LOS.

When deciding to perform a CSF HSV PCR test for infants aged ≤60 days, clinicians must balance the low incidence of neonatal HSV3 with the risk of delayed diagnosis and treatment of HSV infection, which include neurologic sequelae or even death.1,2 As infants with CNS HSV infection commonly present nonspecifically and only a minority of infected infants have skin vesicles,1 controversy exists as to which infants should be evaluated for HSV infection, resulting in considerable variability in HSV testing.3 Some clinicians advocate for more conservative testing strategies that include the performance of CSF HSV PCR testing in all febrile infants aged ≤21 days.9 Others suggest limiting testing to infants who meet high-risk criteria (eg, seizures, ill-appearance, or CSF pleocytosis).10,11 Further investigation will need to elucidate the clinical and laboratory predictors of HSV infection to identify those infants who would benefit most from HSV testing as well as the outcomes of infants not tested.

Our study has several limitations. First, we could not determine the reason why clinicians elected to obtain a CSF HSV PCR test, and we do not know the test turnaround time or the time when results became available to the clinical team. Second, we did not abstract clinical data such as ill-appearance or seizures. Although we adjusted for the presence of serious bacterial or HSV infection as proxy measures for illness severity, it is possible that other clinical factors were associated with HSV testing and LOS. Third, although we adjusted for patient- and hospital-level factors in our regression model, the potential for residual confounding persists. Fourth, we did not explore acyclovir administration as a factor associated with LOS as some sites did not provide data on acyclovir. Fifth, we did not evaluate the impact of HSV testing of other sample types (eg, blood or skin) on LOS. Sixth, our study was conducted primarily at children’s hospitals, and our findings may not be generalizable to general hospitals with hospitalized neonates.

 

 

CONCLUSIONS

For infants aged ≤60 days undergoing evaluation for meningitis, CSF HSV PCR testing was associated with a slightly longer hospital LOS. Improved methods to identify and target testing to higher risk infants may mitigate the impact on LOS for low-risk infants.

Acknowledgments

The authors acknowledge the following collaborators in the Pediatric Emergency Medicine Clinical Research Network (PEM CRC) Herpes Simplex Virus (HSV) Study Group who collected data for this study and/or the parent study: Joseph L Arms, MD (Minneapolis, Minnesota), Stuart A Bradin, DO (Ann Arbor, Michigan), Sarah J Curtis, MD, MSc (Edmonton, Alberta, Canada), Paul T Ishimine, MD (San Diego, California), Dina Kulik, MD (Toronto, Ontario, Canada), Prashant Mahajan, MD, MPH, MBA (Ann Arbor, Michigan), Aaron S Miller, MD, MSPH (St. Louis, Missouri), Pamela J Okada, MD (Dallas, Texas), Christopher M Pruitt, MD (Birmingham, Alabama), Suzanne M Schmidt, MD (Chicago, Illinois), David Schnadower, Amy D Thompson, MD (Wilmington, Delaware), Joanna E Thomson, MD, MPH (Cincinnati, Ohio), MD, MPH (St. Louis, Missouri), and Neil G. Uspal, MD (Seattle, Washington).

Disclosures

Dr. Aronson reports grants from the Agency for Healthcare Research and Quality during the conduct of the study. Dr. Shah reports grants from Patient-Centered Outcomes Research Institute, grants from the National Institute of Allergy and Infectious Diseases, and grants from the National Heart Lung Blood Institute, outside the submitted work. Dr. Shah is the Editor-in-Chief of the Journal of Hospital Medicine. All other authors have no conflicts of interest or financial relationships relevant to this article to disclose.

Funding

This project was supported by the Section of Emergency Medicine of the American Academy of Pediatrics (AAP) and Baylor College of Medicine and by the grant number K08HS026006 (Aronson) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Stephen Freedman is supported by the Alberta Children’s Hospital Foundation Professorship in Child Health and Wellness.

 

Neonatal herpes simplex virus (HSV) is associated with significant morbidity and mortality,1 particularly when the diagnosis or treatment is delayed.2 Therefore, many infants aged ≤60 days being evaluated for meningitis undergo cerebrospinal fluid (CSF) HSV polymerase chain reaction (PCR) testing even though the risk of HSV infection is low [estimated at 0.4% of those undergoing evaluation for central nervous system (CNS) infection].3 A single-center study demonstrated that CSF HSV PCR testing increases the hospital length of stay (LOS) for infants aged ≤56 days,4 although these single-center findings may not be generalizable. To this end, we measured the association between CSF HSV PCR testing and LOS in a multicenter cohort of hospitalized young infants.

METHODS

Study Design

We conducted a planned secondary analysis of a retrospective cohort of infants aged ≤60 days who presented to the emergency department (ED) between January 1, 2005 and December 31, 2013, enrolled in the Pediatric Emergency Medicine Collaborative Research Committee (PEM CRC) HSV study.3 Our study was limited to the 20 hospitals that contributed hospital LOS data. The study protocol was approved by each site’s institutional review board with permission for data sharing.

Study Population

Eligible infants were identified at each site using a site-specific electronic search strategy. Infants were eligible for inclusion if a CSF culture was obtained in the ED or within 24 hours of ED arrival. We excluded infants who were discharged from the ED and those with missing hospital LOS data.

 

 

Data Collection

Site investigators extracted the following data elements either electronically or from medical records: patient demographics; ED arrival date and time; hospital discharge date and time; urinalysis results; peripheral and CSF cell counts; blood, urine, and CSF bacterial culture results; as well as the results of HSV PCR and viral cultures. Infants with growth of a pathogen in blood or CSF, or a catheterized urine culture with ≥50,000 colony-forming units (CFUs)/mL of a single pathogenic bacteria, or 10,000-50,000 CFUs/mL of a single pathogenic bacteria with an abnormal urinalysis (ie, positive nitrite or leukocyte esterase on urine dipstick or >5 white blood cells [WBCs] per high power field on urine microscopy) were classified as having a serious bacterial infection (SBI).5,6 Infants with a positive HSV PCR or viral culture from any site were classified as having HSV infection.3 Hospitalized infants who did not have an HSV PCR test performed were assumed not to have HSV disease if not diagnosed during the hospital stay or repeat ED encounter.3

Outcome Measures

The primary outcome was hospital LOS, defined at all hospitals as the time from ED arrival to provider signature of the hospital discharge order, calculated in minutes and then converted into days.

Statistical Analysis

We described LOS using medians with interquartile ranges (IQR) and compared between infants with and without a CSF HSV PCR test performed using the Mann–Whitney U test. To evaluate the association between performance of CSF HSV PCR testing and hospital LOS, we used negative binomial regression given the count variable outcome (LOS) with an overdispersed distribution. For this analysis, we clustered by hospital after adjusting for the following factors determined a priori: age, gender, study year, and presence of serious bacterial or HSV infection. Using the relative marginal modeled estimates of LOS (tested vs not tested), we determined the percentage increase in LOS. We then repeated the analyses after stratifying by the location of testing (ie, in-house vs send-out), age (≤28 days vs 29-60 days), and presence or absence of CSF pleocytosis (defined as a CSF WBC of ≥16 cells/mm3for infants aged ≤28 days and ≥10 cells/mm3for infants aged 29-60 days),7 because infants aged 29-60 days and those without CSF pleocytosis are reported to be at very low risk for CNS HSV infection.3,8 We utilized Stata Data Analysis and Statistical Software, version 15.0 (StataCorp, Inc.; College Station, Texas) for statistical analyses.

RESULTS

Of 24,103 infants with CSF cultures obtained at the 20 participating sites, we excluded 2,673 (11.1%) discharged from the ED or with missing disposition and 934 (3.9%) with missing LOS, leaving a study cohort of 20,496 infants (Figure). Overall, 1,780 infants (8.7%) had an SBI and 99 (0.5%) had an HSV infection, of which 46 (46.5%) had a CNS HSV infection.

Among the 20,496 study infants, 7,399 (36.1%) had a CSF HSV PCR test performed; 5,935 infants (80.2% of those tested) had in-house and 1,464 (19.8%) had send-out testing. Among infants with available CSF cell counts, a CSF HSV PCR test was more commonly performed in infants with CSF pleocytosis than in those without (1,865/4,439 [42.0%] with CSF pleocytosis vs 3,705/12,002 [30.9%] without CSF pleocytosis; odds ratio [OR] 1.6, 95% CI 1.5-1.7). Of the 7,399 infants who had a CSF HSV PCR test performed, 46 (0.6%) had a positive test. Of the tested infants, 5,570 (75.3%) had an available CSF WBC count; a positive CSF HSV PCR test was more common in infants with CSF pleocytosis than in those without (25 positive tests/1,865 infants with CSF pleocytosis [1.3%] vs 9/3,705 [0.2%] without CSF pleocytosis; OR 5.6, 95% CI 2.6-12.0). Among the 5,308 infants aged 29-60 days without CSF pleocytosis, 1,110 (20.9%) had a CSF HSV PCR test performed and only one infant (0.09% of those tested) had a positive test.

Without adjustment, infants with a CSF HSV PCR test had a longer median LOS than infants who were not tested (2.5 vs 2.3 days; P < .001). After adjustment, infants with a CSF HSV PCR test performed had a 23% longer duration of hospitalization. The association between testing and LOS was similar for older vs younger infants, infants with and without CSF pleocytosis, and in-house vs send-out testing (Table).

 

 

DISCUSSION

In a large, multicenter cohort of more than 20,000 hospitalized infants aged ≤60 days undergoing evaluation for meningitis, we examined the association of CSF HSV PCR testing with hospital LOS. Approximately one-third of study infants had a CSF HSV PCR test obtained. After adjustment for patient- and hospital-level factors, the treating clinician’s decision to obtain a CSF HSV PCR test was associated with a 23% longer hospital LOS (nearly one-half day).

Our findings are consistent with those of previous studies. First, our observed association of the decision to obtain a CSF HSV PCR test and LOS was similar in magnitude to that of a previous single-center investigation.4 Second, we also found that older infants and those without CSF pleocytosis were at very low risk of HSV infection.3,8 For the otherwise low-risk infants, the longer LOS may be due to delays in obtaining CSF HSV PCR test results, which should be explored in future research. Our study has greater generalizability than previous single-center studies by substantially increasing the population size as well as the variety of clinical settings. Ensuring clinicians’ access to rapid HSV PCR testing platforms will further mitigate the impact of HSV testing on LOS.

When deciding to perform a CSF HSV PCR test for infants aged ≤60 days, clinicians must balance the low incidence of neonatal HSV3 with the risk of delayed diagnosis and treatment of HSV infection, which include neurologic sequelae or even death.1,2 As infants with CNS HSV infection commonly present nonspecifically and only a minority of infected infants have skin vesicles,1 controversy exists as to which infants should be evaluated for HSV infection, resulting in considerable variability in HSV testing.3 Some clinicians advocate for more conservative testing strategies that include the performance of CSF HSV PCR testing in all febrile infants aged ≤21 days.9 Others suggest limiting testing to infants who meet high-risk criteria (eg, seizures, ill-appearance, or CSF pleocytosis).10,11 Further investigation will need to elucidate the clinical and laboratory predictors of HSV infection to identify those infants who would benefit most from HSV testing as well as the outcomes of infants not tested.

Our study has several limitations. First, we could not determine the reason why clinicians elected to obtain a CSF HSV PCR test, and we do not know the test turnaround time or the time when results became available to the clinical team. Second, we did not abstract clinical data such as ill-appearance or seizures. Although we adjusted for the presence of serious bacterial or HSV infection as proxy measures for illness severity, it is possible that other clinical factors were associated with HSV testing and LOS. Third, although we adjusted for patient- and hospital-level factors in our regression model, the potential for residual confounding persists. Fourth, we did not explore acyclovir administration as a factor associated with LOS as some sites did not provide data on acyclovir. Fifth, we did not evaluate the impact of HSV testing of other sample types (eg, blood or skin) on LOS. Sixth, our study was conducted primarily at children’s hospitals, and our findings may not be generalizable to general hospitals with hospitalized neonates.

 

 

CONCLUSIONS

For infants aged ≤60 days undergoing evaluation for meningitis, CSF HSV PCR testing was associated with a slightly longer hospital LOS. Improved methods to identify and target testing to higher risk infants may mitigate the impact on LOS for low-risk infants.

Acknowledgments

The authors acknowledge the following collaborators in the Pediatric Emergency Medicine Clinical Research Network (PEM CRC) Herpes Simplex Virus (HSV) Study Group who collected data for this study and/or the parent study: Joseph L Arms, MD (Minneapolis, Minnesota), Stuart A Bradin, DO (Ann Arbor, Michigan), Sarah J Curtis, MD, MSc (Edmonton, Alberta, Canada), Paul T Ishimine, MD (San Diego, California), Dina Kulik, MD (Toronto, Ontario, Canada), Prashant Mahajan, MD, MPH, MBA (Ann Arbor, Michigan), Aaron S Miller, MD, MSPH (St. Louis, Missouri), Pamela J Okada, MD (Dallas, Texas), Christopher M Pruitt, MD (Birmingham, Alabama), Suzanne M Schmidt, MD (Chicago, Illinois), David Schnadower, Amy D Thompson, MD (Wilmington, Delaware), Joanna E Thomson, MD, MPH (Cincinnati, Ohio), MD, MPH (St. Louis, Missouri), and Neil G. Uspal, MD (Seattle, Washington).

Disclosures

Dr. Aronson reports grants from the Agency for Healthcare Research and Quality during the conduct of the study. Dr. Shah reports grants from Patient-Centered Outcomes Research Institute, grants from the National Institute of Allergy and Infectious Diseases, and grants from the National Heart Lung Blood Institute, outside the submitted work. Dr. Shah is the Editor-in-Chief of the Journal of Hospital Medicine. All other authors have no conflicts of interest or financial relationships relevant to this article to disclose.

Funding

This project was supported by the Section of Emergency Medicine of the American Academy of Pediatrics (AAP) and Baylor College of Medicine and by the grant number K08HS026006 (Aronson) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Stephen Freedman is supported by the Alberta Children’s Hospital Foundation Professorship in Child Health and Wellness.

 

References

1. Kimberlin DW, Lin CY, Jacobs RF, et al. Natural history of neonatal herpes simplex virus infections in the acyclovir era. Pediatrics. 2001;108(2):223-229. PubMed
2. Shah SS, Aronson PL, Mohamad Z, Lorch SA. Delayed acyclovir therapy and death among neonates with herpes simplex virus infection. Pediatrics. 2011;128(6):1153-1160. https://doi.org/10.1136/eb-2012-100674.
3. Cruz AT, Freedman SB, Kulik DM, et al. Herpes simplex virus infection in infants undergoing meningitis evaluation. Pediatrics. 2018;141(2):e20171688. https://doi.org/10.1542/peds.2017-1688.
4. Shah SS, Volk J, Mohamad Z, Hodinka RL, Zorc JJ. Herpes simplex virus testing and hospital length of stay in neonates and young infants. J Pediatr. 2010;156(5):738-743. https://doi.org/10.1016/j.jpeds.2009.11.079.
5. Mahajan P, Kuppermann N, Mejias A, et al. Association of RNA biosignatures with bacterial infections in febrile infants aged 60 days or younger. JAMA. 2016;316(8):846-857. https://doi.org/10.1001/jama.2016.9207.
6. Schnadower D, Kuppermann N, Macias CG, et al. Febrile infants with urinary tract infections at very low risk for adverse events and bacteremia. Pediatrics. 2010;126(6):1074-1083. https://doi.org/10.1542/peds.2010-0479.
7. Thomson J, Sucharew H, Cruz AT, et al. Cerebrospinal fluid reference values for young infants undergoing lumbar puncture. Pediatrics. 2018;141(3):e20173405. https://doi.org/10.1542/peds.2017-3405.
8. Caviness AC, Demmler GJ, Almendarez Y, Selwyn BJ. The prevalence of neonatal herpes simplex virus infection compared with serious bacterial illness in hospitalized neonates. J Pediatr. 2008;153(2):164-169. https://doi.org/10.1016/j.jpeds.2008.02.031.
9. Long SS. In defense of empiric acyclovir therapy in certain neonates. J Pediatr. 2008;153(2):157-158. https://doi.org/10.1016/j.jpeds.2008.04.071.
10. Brower L, Schondelmeyer A, Wilson P, Shah SS. Testing and empiric treatment for neonatal herpes simplex virus: challenges and opportunities for improving the value of care. Hosp Pediatr. 2016;6(2):108-111. https://doi.org/10.1542/hpeds.2015-0166.
11. Kimberlin DW. When should you initiate acyclovir therapy in a neonate? J Pediatr. 2008;153(2):155-156. https://doi.org/10.1016/j.jpeds.2008.04.027.

References

1. Kimberlin DW, Lin CY, Jacobs RF, et al. Natural history of neonatal herpes simplex virus infections in the acyclovir era. Pediatrics. 2001;108(2):223-229. PubMed
2. Shah SS, Aronson PL, Mohamad Z, Lorch SA. Delayed acyclovir therapy and death among neonates with herpes simplex virus infection. Pediatrics. 2011;128(6):1153-1160. https://doi.org/10.1136/eb-2012-100674.
3. Cruz AT, Freedman SB, Kulik DM, et al. Herpes simplex virus infection in infants undergoing meningitis evaluation. Pediatrics. 2018;141(2):e20171688. https://doi.org/10.1542/peds.2017-1688.
4. Shah SS, Volk J, Mohamad Z, Hodinka RL, Zorc JJ. Herpes simplex virus testing and hospital length of stay in neonates and young infants. J Pediatr. 2010;156(5):738-743. https://doi.org/10.1016/j.jpeds.2009.11.079.
5. Mahajan P, Kuppermann N, Mejias A, et al. Association of RNA biosignatures with bacterial infections in febrile infants aged 60 days or younger. JAMA. 2016;316(8):846-857. https://doi.org/10.1001/jama.2016.9207.
6. Schnadower D, Kuppermann N, Macias CG, et al. Febrile infants with urinary tract infections at very low risk for adverse events and bacteremia. Pediatrics. 2010;126(6):1074-1083. https://doi.org/10.1542/peds.2010-0479.
7. Thomson J, Sucharew H, Cruz AT, et al. Cerebrospinal fluid reference values for young infants undergoing lumbar puncture. Pediatrics. 2018;141(3):e20173405. https://doi.org/10.1542/peds.2017-3405.
8. Caviness AC, Demmler GJ, Almendarez Y, Selwyn BJ. The prevalence of neonatal herpes simplex virus infection compared with serious bacterial illness in hospitalized neonates. J Pediatr. 2008;153(2):164-169. https://doi.org/10.1016/j.jpeds.2008.02.031.
9. Long SS. In defense of empiric acyclovir therapy in certain neonates. J Pediatr. 2008;153(2):157-158. https://doi.org/10.1016/j.jpeds.2008.04.071.
10. Brower L, Schondelmeyer A, Wilson P, Shah SS. Testing and empiric treatment for neonatal herpes simplex virus: challenges and opportunities for improving the value of care. Hosp Pediatr. 2016;6(2):108-111. https://doi.org/10.1542/hpeds.2015-0166.
11. Kimberlin DW. When should you initiate acyclovir therapy in a neonate? J Pediatr. 2008;153(2):155-156. https://doi.org/10.1016/j.jpeds.2008.04.027.

Issue
Journal of Hospital Medicine 14(8)
Issue
Journal of Hospital Medicine 14(8)
Page Number
492-495
Page Number
492-495
Publications
Publications
Topics
Article Type
Sections
Article Source

© 2019 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Paul L Aronson, MD, MHS; E-mail: paul.aronson@yale.edu; Telephone: 203-785-3849.
Content Gating
Gated (full article locked unless allowed per User)
Alternative CME
Disqus Comments
Default
Un-Gate On Date
Sun, 05/10/2020 - 00:00
Use ProPublica
Hide sidebar & use full width
render the right sidebar.
Gating Strategy
First Peek Free
Article PDF Media

Febrile Infant Diagnosis Code Accuracy

Article Type
Changed
Tue, 05/16/2017 - 22:46
Display Headline
Accuracy of diagnosis codes to identify febrile young infants using administrative data

Fever is one of the most common reasons for emergency department (ED) evaluation of infants under 90 days of age.[1] Up to 10% to 20% of febrile young infants will have a serious bacterial infection (SBI),[2, 3, 4] but infants with SBI are difficult to distinguish from those without SBI based upon symptoms and physical examination findings alone.[5] Previously developed clinical prediction algorithms can help to identify febrile infants at low risk for SBI, but differ in age range as well as recommendations for testing and empiric treatment.[6, 7, 8] Consequently, there is widespread variation in management of febrile young infants at US children's hospitals,[9, 10, 11] and defining optimal management strategies remains an important issue in pediatric healthcare.[12] Administrative datasets are convenient and inexpensive, and can be used to evaluate practice variation, trends, and outcomes of a large, diverse group of patients within and across institutions.[9, 10] Accurately identifying febrile infants evaluated for suspected SBI in administrative databases would facilitate comparative effectiveness research, quality improvement initiatives, and institutional benchmarking.

Prior studies have validated the accuracy of administrative billing codes for identification of other common childhood illnesses, including urinary tract infection (UTI)[13] and pneumonia.[14] The accuracy of International Classification of Diseases, Ninth Revision (ICD‐9) diagnosis codes in identifying febrile young infants evaluated for SBI is not known. Reliance on administrative ICD‐9 diagnosis codes for patient identification can lead to misclassification of patients due to variable database quality, the validity of the diagnosis codes being utilized, and hospital coding practices.[15] Additionally, fever is a symptom and not a specific diagnosis. If a particular bacterial or viral diagnosis is established (eg, enterovirus meningitis), a discharge diagnosis of fever may not be attributed to the patient encounter. Thus, evaluating the performance characteristics and capture of clinical outcomes of different combinations of ICD‐9 diagnosis codes for identifying febrile infants is necessary for both the conduct and interpretation of studies that utilize administrative databases. The primary objective of this investigation was to identify the most accurate ICD‐9 coding strategies for the identification of febrile infants aged <90 days using administrative data. We also sought to evaluate capture of clinically important outcomes across identification strategies.

METHODS

Study Design and Setting

For this multicenter retrospective study, we used the Pediatric Health Information System (PHIS) database to identify infants <90 days of age[16] who presented between July 1, 2012 and June 30, 2013 to 1 of 8 EDs. We assessed performance characteristics of ICD‐9 diagnosis code case‐identification algorithms by comparing ICD‐9 code combinations to a fever reference standard determined by medical record review. The institutional review board at each participating site approved the study protocol.

Data Source

Data were obtained from 2 sources: the PHIS database and medical record review. We used the PHIS database to identify eligible patients by ICD‐9 diagnosis codes; patient encounters were randomly selected using a random number generator. The PHIS database contains demographic, diagnosis, and billing data from 44 hospitals affiliated with the Children's Hospital Association (Overland Park, Kansas) and represents 85% of freestanding children's hospitals in the United States.[17] Data are deidentified; encrypted unique patient identifiers permit tracking of patients across visits within a site.[18] The Children's Hospital Association and participating hospitals jointly assure the quality and integrity of the data.[19]

For each patient encounter identified in the PHIS database, detailed medical record review was performed by trained investigators at each of the 8 study sites (see Supporting Information, Appendix, in the online version of this article). A standardized data collection instrument was pilot tested by all investigators prior to use. Data were collected and managed using the Research Electronic Data Capture (REDCap) tool hosted at Boston Children's Hospital.[20]

Exclusions

Using PHIS data, prior to medical record review we excluded infants with a complex chronic condition as defined previously[21] and those transferred from another institution, as these infants may warrant a nonstandard evaluation and/or may have incomplete data.

ICD‐9 Diagnosis Code Groups

In the PHIS database, all patients discharged from the hospital (including hospitalized patients as well as patients discharged from the ED) receive 1 or more ICD‐9 discharge diagnosis codes. These diagnosis codes are ascribed after discharge from the hospital, or for ED patients, after ED discharge. Additionally, patients may receive an admission diagnosis, which reflects the diagnosis ascribed at the time of ED discharge or transfer to the inpatient unit.

We reviewed medical records of infants selected from the following ICD‐9 diagnosis code groups (Figure 1): (1) discharge diagnosis code of fever (780.6 [fever and other physiologic disturbances of temperature regulation], 778.4 [other disturbances of temperature regulation of newborn], 780.60 [fever, unspecified], or 780.61 [fever presenting with conditions classified elsewhere])[9, 10] regardless of the presence of admission diagnosis of fever or diagnosis of serious infection, (2) admission diagnosis code of fever without associated discharge diagnosis code of fever,[10] (3) discharge diagnosis code of serious infection determined a priori (see Supporting Information, Appendix, in the online version of this article) without discharge or admission diagnosis code of fever, and (4) infants without any diagnosis code of fever or serious infection.

Figure 1
Study population. 1Two of 584 medical records were unavailable for review. 2Five of 904 medical records were unavailable for review. Abbreviations: CCC, complex chronic condition; ED, emergency department.

Medical records reviewed in each of the 4 ICD‐9 diagnosis code groups were randomly selected from the overall set of ED encounters in the population of infants <90 days of age evaluated during the study period. Twenty‐five percent population sampling was used for 3 of the ICD‐9 diagnosis code groups, whereas 5% sampling was used for the no fever/no serious infection code group. The number of medical records reviewed in each ICD‐9 diagnosis code group was proportional to the distribution of ICD‐9 codes across the entire population of infants <90 days of age. These records were distributed equally across sites (228 records per site), except for 1 site that does not assign admission diagnoses (201 records).

Investigators were blinded to ICD‐9 diagnosis code groups during medical record review. Infants with multiple visits during the study period were eligible to be included more than once if the visits occurred more than 3 days apart. For infants with more than 1 ED visit on a particular calendar day, investigators were instructed to review the initial visit.

For each encounter, we also abstracted demographic characteristics (gender, race/ethnicity), insurance status, hospital region (using US Census categories[22]), and season from the PHIS database.

Reference Standard

The presence of fever was determined by medical record review. We defined fever as any documented temperature 100.4F (38.0C) at home or in the ED.[16]

ICD‐9 Code Case‐Identification Algorithms

Using the aforementioned ICD‐9 diagnosis code groups individually and in combination, the following 4 case‐identification algorithms, determined from prior study or group consensus, were compared to the reference standard: (1) ICD‐9 discharge diagnosis code of fever,[9] (2) ICD‐9 admission or discharge diagnosis code of fever,[10, 11] (3) ICD‐9 discharge diagnosis code of fever or serious infection, and (4) ICD‐9 discharge or admission diagnosis code of fever or serious infection. Algorithms were compared overall, separately for discharged and hospitalized infants, and across 3 distinct age groups (28 days, 2956 days, and 5789 days).

Patient‐Level Outcomes

To compare differences in outcomes by case‐identification algorithm, from the PHIS database we abstracted hospitalization rates, rates of UTI/pyelonephritis,[13] bacteremia/sepsis, and bacterial meningitis.[19] Severe outcomes were defined as intensive care unit admission, mechanical ventilation, central line placement, receipt of extracorporeal membrane oxygenation, or death. We assessed hospital length of stay for admitted infants and 3‐day revisits,[23, 24] and revisits resulting in hospitalization for infants discharged from the ED at the index visit. Patients billed for observation care were classified as being hospitalized.[25, 26]

Data Analysis

Accuracy of the 4 case‐identification algorithms (compared with the reference standard) was calculated using sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV), along with 95% confidence interval (CI). Prior to analysis, a 5‐fold weighting factor was applied to the no fever/no serious infection group to account for the differential sampling used for this group (5% vs 25% for the other 3 ICD‐9 diagnosis code groups). This weighting was done to approximate the true prevalence of each ICD‐9 code group within the larger population, so that an accurate rate of false negatives (infants with fever who had neither a diagnosis of fever nor serious infection) could be calculated.

We described continuous variables using median and interquartile range or range values and categorical variables using frequencies with 95% CIs. We compared categorical variables using a 2 test. We determined statistical significance as a 2‐tailed P value <0.05. Statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC).

RESULTS

Study Patients

During the 1‐year study period, 23,753 ED encounters for infants <90 days of age were identified in the PHIS database at the 8 participating sites. Of these infant encounters, 2166 (9.2%) were excluded (1658 infants who had a complex chronic condition and 508 transferred into the ED), leaving 21,587 infants available for selection. After applying our sampling strategy, we identified 1797 encounters for medical record review. Seven encounters from 3 hospitals with missing medical records were excluded, resulting in a final cohort of 1790 encounters (Figure 1). Among included infants, 552 (30.8%) were 28 days, 743 (41.5%) were 29 to 56 days, and 495 (27.8%) were 57 to 89 days of age; 737 (41.2%) infants were hospitalized. Patients differed in age, race, payer, and season across ICD‐9 diagnosis code groups (see Supporting Information, Table 1, in the online version of this article).

Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms According to Reference Standard (Overall, Hospitalized, and Discharged).*
ICD‐9 Diagnosis Code AlgorithmOverall
Sensitivity, % (95% CI)Specificity, % (95% CI)Negative Predictive Value, % (95% CI)Positive Predictive Value, % (95% CI)
  • NOTE: Abbreviations: CI, confidence interval; ED, emergency department; ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4 F (38.0 C) on review of electronic medical record.

Discharge diagnosis of fever53.2 (50.056.4)98.2 (97.898.6)90.8 (90.091.6)86.1 (83.388.9)
Hospitalized47.3 (43.151.5)97.7 (96.998.5)80.6 (78.682.6)90.2 (86.893.6)
Discharged from ED61.4 (56.666.2)98.4 (98.098.8)95.4 (94.796.1)82.1 (77.786.5)
Discharge or admission diagnosis of Fever71.1 (68.274.0)97.7 (97.398.1)94.1 (93.494.8)86.9 (84.589.3)
Hospitalized72.5 (68.876.2)97.1 (96.298.0)88.8 (87.190.5)91.7 (89.194.3)
Discharged from ED69.2 (64.773.7)98.0 (97.598.5)96.3 (95.796.9)80.8 (76.685.0)
Discharge diagnosis of fever or serious infection63.7 (60.666.8)96.5 (96.097.0)92.6 (91.893.4)79.6 (76.782.5)
Hospitalized63.9 (59.967.9)92.5 (91.094.0)85.1 (83.287.0)79.1 (75.382.9)
Discharged from ED63.4 (58.768.1)98.1 (97.698.6)95.6 (94.996.3)80.2 (75.884.6)
Discharge or admission diagnosis of fever or serious infection76.6 (73.979.3)96.2 (95.696.8)95.1 (94.595.7)81.0 (78.483.6)
Hospitalized80.8 (77.584.1)92.1 (90.693.6)91.5 (89.993.1)82.1 (78.985.3)
Discharged from ED71.0 (66.575.5)97.7 (97.298.2)96.5 (95.997.1)79.4 (75.283.6)

Among the 1790 patient encounters reviewed, a total of 766 infants (42.8%) met the reference standard definition for fever in the cohort. An additional 47 infants had abnormal temperature reported (documentation of tactile fever, history of fever without a specific temperature described, or hypothermia) but were classified as having no fever by the reference standard.

ICD‐9 Code Case‐Identification Algorithm Performance

Compared with the reference standard, the 4 case‐identification algorithms demonstrated specificity of 96.2% to 98.2% but lower sensitivity overall (Figure 2). Discharge diagnosis of fever alone demonstrated the lowest sensitivity. The algorithm of discharge or admission diagnosis of fever resulted in increased sensitivity and the highest PPV of all 4 algorithms (86.9%, 95% CI: 84.5‐89.3). Addition of serious infection codes to this algorithm resulted in a marginal increase in sensitivity and a similar decrease in PPV (Table 1). When limited to hospitalized infants, specificity was highest for the case‐identification algorithm of discharge diagnosis of fever and similarly high for discharge or admission diagnosis of fever; sensitivity was highest for the algorithm of discharge or admission diagnosis of fever or diagnosis of serious infection. For infants discharged from the ED, algorithm specificity was 97.7% to 98.4%, with lower sensitivity for all 4 algorithms (Table 1). Inclusion of the 47 infants with abnormal temperature as fever did not materially change algorithm performance (data not shown).

Figure 2
Algorithm sensitivity and false positive rate (1‐specificity) for identification of febrile infants aged ≤28 days, 29 to 56 days, 57 to 89 days, and overall. Horizontal and vertical bars represent 95% confidence intervals. Reference standard of fever was defined by documented temperature ≥100.4°F (38.0°C) on review of electronic medical record.

Across all 3 age groups (28 days, 2956 days, and 5789 days), the 4 case‐identification algorithms demonstrated specificity >96%, whereas algorithm sensitivity was highest in the 29‐ to 56‐days‐old age group and lowest among infants 57 to 89 days old across all 4 algorithms (Figure 2). Similar to the overall cohort, an algorithm of discharge or admission diagnosis of fever demonstrated specificity of nearly 98% in all age groups; addition of serious infection codes to this algorithm increased sensitivity, highest in the 29‐ to 56‐days‐old age group (Figure 2; see also Supporting Information, Table 2, in the online version of this article).

Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms Across the Eight Sites According to Reference Standard.*
ICD‐9 Diagnosis Code AlgorithmSensitivity, Median % (Range)Specificity, Median % (Range)Negative Predictive Value, Median % (Range)Positive Predictive Value, Median % (Range)
  • NOTE: Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4F (38.0 C) on review of electronic medical record.

Discharge diagnosis of fever56.2 (34.681.0)98.3 (96.499.1)92.1 (83.297.4)87.7 (74.093.2)
Discharge or Admission diagnosis of Fever76.7 (51.385.0)97.8 (96.298.7)95.6 (86.997.4)87.4 (80.092.9)
Discharge diagnosis of fever or serious infection68.3 (44.287.3)96.5 (95.498.0)93.6 (85.298.2)78.3 (74.289.0)
Discharge or admission diagnosis of fever or serious infection83.1 (58.390.7)95.8 (95.498.0)96.5 (88.598.2)79.1 (77.490.4)

Across the 8 study sites, median specificity was 95.8% to 98.3% for the 4 algorithms, with little interhospital variability; however, algorithm sensitivity varied widely by site. Median PPV was highest for discharge diagnosis of fever alone at 87.7% but ranged from 74.0% to 93.2% across sites. Median PPV for an algorithm of discharge or admission diagnosis of fever was similar (87.4%) but with less variation by site (range 80.0%92.9%) (Table 2).

Outcomes by ICD‐9 Diagnosis Code Group and Case‐Identification Algorithm

When compared with discharge diagnosis of fever, adding admission diagnosis of fever captured a higher proportion of hospitalized infants with SBIs (UTI/pyelonephritis, bacteremia/sepsis, or bacterial meningitis). However, median hospital length of stay, severe outcomes, and 3‐day revisits and revisits with hospitalization did not materially differ when including infants with admission diagnosis of fever in addition to discharge diagnosis of fever. Addition of infants with a diagnosis code for serious infection substantially increased the number of infants with SBIs and severe outcomes but did not capture additional 3‐day revisits (Table 3). There were no additional cases of SBI in the no fever/no serious illness diagnosis code group.

Outcomes by ICD‐9 Diagnosis Code Case‐Identification Algorithm
ICD‐9 Diagnosis Code AlgorithmOutcome3‐Day Revisit, % (95% CI)3‐Day Revisit With Hospitalization, % (95% CI)
Hospitalized, % (95% CI)UTI/Pyelonephritis, Bacteremia/Sepsis, or Bacterial Meningitis, % (95% CI)Severe Outcome, % (95% CI)*Length of Stay in Days, Median (IQR)
  • NOTE: Abbreviations: CI, confidence interval; ICD‐9, International Classification of Diseases, Ninth Revision; IQR, interquartile range; UTI, urinary tract infection. *Severe outcome was defined as intensive care unit admission, mechanical ventilation, central line placement, extracorporeal membrane oxygenation, or death. Length of stay for hospitalized infants. Percent of those discharged from the emergency department at the index visit.

Discharge diagnosis of fever44.3 (40.348.4)3.3 (1.84.7)1.4 (0.42.3)3 (23)11.7 (8.215.2)5.9 (3.38.4)
Discharge or admission diagnosis of fever52.4 (48.955.9)6.1 (4.47.8)1.9 (1.02.9)3 (23)10.9 (7.714.1)5.4 (3.17.8)
Discharge diagnosis of fever or serious infection54.0 (50.457.5)15.3 (12.717.8)3.8 (2.55.2)3 (24)11.0 (7.714.2)5.5 (3.17.9)
Discharge or admission diagnosis of fever or serious infection56.5 (53.259.7)12.9 (10.715.1)3.6 (2.44.8)3 (24)10.3 (7.313.3)5.2 (3.07.4)

Among infants who met the reference standard for fever but did not have a discharge or admission diagnosis of fever (false negatives), 11.8% had a diagnosis of SBI. Overall, 43.2% of febrile infants (and 84.4% of hospitalized infants) with SBI did not have an ICD‐9 discharge or admission diagnosis of fever. Addition of ICD‐9 diagnosis codes of serious infection to the algorithm of discharge or admission diagnosis of fever captured all additional SBIs, and no false negativeinfants missed with this algorithm had an SBI.

DISCUSSION

We described the performance of 4 ICD‐9 diagnosis code case‐identification algorithms for the identification of febrile young infants <90 days of age at US children's hospitals. Although the specificity was high across algorithms and institutions, the sensitivity was relatively low, particularly for discharge diagnosis of fever, and varied by institution. Given the high specificity, ICD‐9 diagnosis code case‐identification algorithms for fever reliably identify febrile infants using administrative data with low rates of inclusion of infants without fever. However, underidentification of patients, particularly those more prone to SBIs and severe outcomes depending on the algorithm utilized, can impact interpretation of comparative effectiveness studies or the quality of care delivered by an institution.

ICD‐9 discharge diagnosis codes are frequently used to identify pediatric patients across a variety of administrative databases, diseases, and symptoms.[19, 27, 28, 29, 30, 31] Although discharge diagnosis of fever is highly specific, sensitivity is substantially lower than other case‐identification algorithms we studied, particularly for hospitalized infants. This may be due to a fever code sometimes being omitted in favor of a more specific diagnosis (eg, bacteremia) prior to hospital discharge. Therefore, case identification relying only on ICD‐9 discharge diagnosis codes for fever may under‐report clinically important SBI or severe outcomes as demonstrated in our study. This is in contrast to ICD‐9 diagnosis code identification strategies for childhood UTI and pneumonia, which largely have higher sensitivity but lower specificity than fever codes.[13, 14]

Admission diagnosis of fever is important for febrile infants as they may not have an explicit diagnosis at the time of disposition from the ED. Addition of admission diagnosis of fever to an algorithm relying on discharge diagnosis code alone increased sensitivity without a demonstrable reduction in specificity and PPV, likely due to capture of infants with a fever diagnosis at presentation before a specific infection was identified. Although using an algorithm of discharge or admission diagnosis of fever captured a higher percentage of hospitalized febrile infants with SBIs, sensitivity was only 71% overall with this algorithm, and 43% of febrile infants with SBI would still have been missed. Importantly, though, addition of various ICD‐9 codes for serious infection to this algorithm resulted in capture of all febrile infants with SBI and should be used as a sensitivity analysis.

The test characteristics of diagnosis codes were highest in the 29‐ to 56‐days‐old age group. Given the differing low‐risk criteria[6, 7, 8] and lack of best practice guidelines[16] in this age group, the use of administrative data may allow for the comparison of testing and treatment strategies across a large cohort of febrile infants aged 29 to 56 days. However, individual hospital coding practices may affect algorithm performance, in particular sensitivity, which varied substantially by hospital. This variation in algorithm sensitivity may impact comparisons of outcomes across institutions. Therefore, when conducting studies of febrile infants using administrative data, sensitivity analyses or use of chart review should be considered to augment the use of ICD‐9 code‐based identification strategies, particularly for comparative benchmarking and outcomes studies. These additional analyses are particularly important for studies of febrile infants >56 days of age, in whom the sensitivity of diagnosis codes is particularly low. We speculate that the lower sensitivity in older febrile infants may relate to a lack of consensus on the clinical significance of fever in this age group and the varying management strategies employed.[10]

Strengths of this study include the assessment of ICD‐9 code algorithms across multiple institutions for identification of fever in young infants, and the patterns of our findings remained robust when comparing median performance characteristics of the algorithms across hospitals to our overall findings. We were also able to accurately estimate PPV and NPV using a case‐identification strategy weighted to the actual population sizes. Although sensitivity and specificity are the primary measures of test performance, predictive values are highly informative for investigators using administrative data. Additionally, our findings may inform public health efforts including disease surveillance, assessment of seasonal variation, and identification and monitoring of healthcare‐associated infections among febrile infants.

Our study has limitations. We did not review all identified records, which raises the possibility that our evaluated cohort may not be representative of the entire febrile infant population. We attempted to mitigate this possibility by using a random sampling strategy for our population selection that was weighted to the actual population sizes. Second, we identified serious infections using ICD‐9 diagnosis codes determined by group consensus, which may not capture all serious infection codes that identify febrile infants whose fever code was omitted. Third, 47 infants had abnormal temperature that did not meet our reference standard criteria for fever and were included in the no fever group. Although there may be disagreement regarding what constitutes a fever, we used a widely accepted reference standard to define fever.[16] Further, inclusion of these 47 infants as fever did not materially change algorithm performance. Last, our study was conducted at 8 large tertiary‐care children's hospitals, and our results may not be generalizable to other children's hospitals and community‐based hospitals.

CONCLUSIONS

Studies of febrile young infants that rely on ICD‐9 discharge diagnosis code of fever for case ascertainment have high specificity but low sensitivity for the identification of febrile infants, particularly among hospitalized patients. A case‐identification strategy that includes discharge or admission diagnosis of fever demonstrated higher sensitivity, and should be considered for studies of febrile infants using administrative data. However, additional strategies such as incorporation of ICD‐9 codes for serious infection should be used when comparing outcomes across institutions.

Acknowledgements

The Febrile Young Infant Research Collaborative includes the following additional collaborators who are acknowledged for their work on this study: Erica DiLeo, MA, Department of Medical Education and Research, Danbury Hospital, Danbury, Connecticut; Janet Flores, BS, Division of Emergency Medicine, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, Illinois.

Disclosures: This project funded in part by The Gerber Foundation Novice Researcher Award, (Ref No. 1827‐3835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in the design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript. The authors have no conflicts of interest relevant to this article to disclose.

Files
References
  1. Baskin MN. The prevalence of serious bacterial infections by age in febrile infants during the first 3 months of life. Pediatr Ann. 1993;22:462466.
  2. Huppler AR, Eickhoff JC, Wald ER. Performance of low‐risk criteria in the evaluation of young infants with fever: review of the literature. Pediatrics. 2010;125:228233.
  3. Schwartz S, Raveh D, Toker O, Segal G, Godovitch N, Schlesinger Y. A week‐by‐week analysis of the low‐risk criteria for serious bacterial infection in febrile neonates. Arch Dis Child. 2009;94:287292.
  4. Garcia S, Mintegi S, Gomez B, et al. Is 15 days an appropriate cut‐off age for considering serious bacterial infection in the management of febrile infants? Pediatr Infect Dis J. 2012;31:455458.
  5. Baker MD, Avner JR, Bell LM. Failure of infant observation scales in detecting serious illness in febrile, 4‐ to 8‐week‐old infants. Pediatrics. 1990;85:10401043.
  6. Baker MD, Bell LM, Avner JR. Outpatient management without antibiotics of fever in selected infants. N Engl J Med. 1993;329:14371441.
  7. Baskin MN, Fleisher GR, O'Rourke EJ. Identifying febrile infants at risk for a serious bacterial infection. J Pediatr. 1993;123:489490.
  8. Jaskiewicz JA, McCarthy CA, Richardson AC, et al. Febrile infants at low risk for serious bacterial infection—an appraisal of the Rochester criteria and implications for management. Febrile Infant Collaborative Study Group. Pediatrics. 1994;94:390396.
  9. Jain S, Cheng J, Alpern ER, et al. Management of febrile neonates in US pediatric emergency departments. Pediatrics. 2014;133:187195.
  10. Aronson PL, Thurm C, Alpern ER, et al. Variation in care of the febrile young infant <90 days in US pediatric emergency departments. Pediatrics. 2014;134:667677.
  11. Aronson PL, Thurm C, Williams DJ, et al. Association of clinical practice guidelines with emergency department management of febrile infants ≤56 days of age. J Hosp Med. 2015;10:358365.
  12. Hui C, Neto G, Tsertsvadze A, et al. Diagnosis and management of febrile infants (0‐3 months). Evid Rep Technol Assess (Full Rep). 2012;(205):1297.
  13. Tieder JS, Hall M, Auger KA, et al. Accuracy of administrative billing codes to detect urinary tract infection hospitalizations. Pediatrics. 2011;128:323330.
  14. Williams DJ, Shah SS, Myers A, et al. Identifying pediatric community‐acquired pneumonia hospitalizations: accuracy of administrative billing codes. JAMA Pediatr. 2013;167:851858.
  15. Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64:821829.
  16. American College of Emergency Physicians Clinical Policies Committee; American College of Emergency Physicians Clinical Policies Subcommittee on Pediatric Fever. Clinical policy for children younger than three years presenting to the emergency department with fever. Ann Emerg Med. 2003;42:530545.
  17. Wood JN, Feudtner C, Medina SP, Luan X, Localio R, Rubin DM. Variation in occult injury screening for children with suspected abuse in selected US children's hospitals. Pediatrics. 2012;130:853860.
  18. Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J AHIMA. 2004;75:2226.
  19. Mongelluzzo J, Mohamad Z, Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299:20482055.
  20. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
  21. Feudtner C, Hays RM, Haynes G, Geyer JR, Neff JM, Koepsell TD. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics. 2001;107:E99.
  22. US Census Bureau. Geographic terms and concepts—census divisions and census regions. Available at: https://www.census.gov/geo/reference/gtc/gtc_census_divreg.html. Accessed October 20, 2014.
  23. Gordon JA, An LC, Hayward RA, Williams BC. Initial emergency department diagnosis and return visits: risk versus perception. Ann Emerg Med. 1998;32:569573.
  24. Cho CS, Shapiro DJ, Cabana MD, Maselli JH, Hersh AL. A national depiction of children with return visits to the emergency department within 72 hours, 2001–2007. Pediatr Emerg Care. 2012;28:606610.
  25. Macy ML, Hall M, Shah SS, et al. Pediatric observation status: are we overlooking a growing population in children's hospitals? J Hosp Med. 2012;7:530536.
  26. Macy ML, Hall M, Shah SS, et al. Differences in designations of observation care in US freestanding children's hospitals: are they virtual or real? J Hosp Med. 2012;7:287293.
  27. Nigrovic LE, Fine AM, Monuteaux MC, Shah SS, Neuman MI. Trends in the management of viral meningitis at United States children's hospitals. Pediatrics. 2013;131:670676.
  28. Freedman SB, Hall M, Shah SS, et al. Impact of increasing ondansetron use on clinical outcomes in children with gastroenteritis. JAMA Pediatr. 2014;168:321329.
  29. Fleming‐Dutra KE, Shapiro DJ, Hicks LA, Gerber JS, Hersh AL. Race, otitis media, and antibiotic selection. Pediatrics. 2014;134:10591066.
  30. Parikh K, Hall M, Mittal V, et al. Establishing benchmarks for the hospitalized care of children with asthma, bronchiolitis, and pneumonia. Pediatrics. 2014;134:555562.
  31. Sheridan DC, Meckler GD, Spiro DM, Koch TK, Hansen ML. Diagnostic testing and treatment of pediatric headache in the emergency department. J Pediatr. 2013;163:16341637.
Article PDF
Issue
Journal of Hospital Medicine - 10(12)
Publications
Page Number
787-793
Sections
Files
Files
Article PDF
Article PDF

Fever is one of the most common reasons for emergency department (ED) evaluation of infants under 90 days of age.[1] Up to 10% to 20% of febrile young infants will have a serious bacterial infection (SBI),[2, 3, 4] but infants with SBI are difficult to distinguish from those without SBI based upon symptoms and physical examination findings alone.[5] Previously developed clinical prediction algorithms can help to identify febrile infants at low risk for SBI, but differ in age range as well as recommendations for testing and empiric treatment.[6, 7, 8] Consequently, there is widespread variation in management of febrile young infants at US children's hospitals,[9, 10, 11] and defining optimal management strategies remains an important issue in pediatric healthcare.[12] Administrative datasets are convenient and inexpensive, and can be used to evaluate practice variation, trends, and outcomes of a large, diverse group of patients within and across institutions.[9, 10] Accurately identifying febrile infants evaluated for suspected SBI in administrative databases would facilitate comparative effectiveness research, quality improvement initiatives, and institutional benchmarking.

Prior studies have validated the accuracy of administrative billing codes for identification of other common childhood illnesses, including urinary tract infection (UTI)[13] and pneumonia.[14] The accuracy of International Classification of Diseases, Ninth Revision (ICD‐9) diagnosis codes in identifying febrile young infants evaluated for SBI is not known. Reliance on administrative ICD‐9 diagnosis codes for patient identification can lead to misclassification of patients due to variable database quality, the validity of the diagnosis codes being utilized, and hospital coding practices.[15] Additionally, fever is a symptom and not a specific diagnosis. If a particular bacterial or viral diagnosis is established (eg, enterovirus meningitis), a discharge diagnosis of fever may not be attributed to the patient encounter. Thus, evaluating the performance characteristics and capture of clinical outcomes of different combinations of ICD‐9 diagnosis codes for identifying febrile infants is necessary for both the conduct and interpretation of studies that utilize administrative databases. The primary objective of this investigation was to identify the most accurate ICD‐9 coding strategies for the identification of febrile infants aged <90 days using administrative data. We also sought to evaluate capture of clinically important outcomes across identification strategies.

METHODS

Study Design and Setting

For this multicenter retrospective study, we used the Pediatric Health Information System (PHIS) database to identify infants <90 days of age[16] who presented between July 1, 2012 and June 30, 2013 to 1 of 8 EDs. We assessed performance characteristics of ICD‐9 diagnosis code case‐identification algorithms by comparing ICD‐9 code combinations to a fever reference standard determined by medical record review. The institutional review board at each participating site approved the study protocol.

Data Source

Data were obtained from 2 sources: the PHIS database and medical record review. We used the PHIS database to identify eligible patients by ICD‐9 diagnosis codes; patient encounters were randomly selected using a random number generator. The PHIS database contains demographic, diagnosis, and billing data from 44 hospitals affiliated with the Children's Hospital Association (Overland Park, Kansas) and represents 85% of freestanding children's hospitals in the United States.[17] Data are deidentified; encrypted unique patient identifiers permit tracking of patients across visits within a site.[18] The Children's Hospital Association and participating hospitals jointly assure the quality and integrity of the data.[19]

For each patient encounter identified in the PHIS database, detailed medical record review was performed by trained investigators at each of the 8 study sites (see Supporting Information, Appendix, in the online version of this article). A standardized data collection instrument was pilot tested by all investigators prior to use. Data were collected and managed using the Research Electronic Data Capture (REDCap) tool hosted at Boston Children's Hospital.[20]

Exclusions

Using PHIS data, prior to medical record review we excluded infants with a complex chronic condition as defined previously[21] and those transferred from another institution, as these infants may warrant a nonstandard evaluation and/or may have incomplete data.

ICD‐9 Diagnosis Code Groups

In the PHIS database, all patients discharged from the hospital (including hospitalized patients as well as patients discharged from the ED) receive 1 or more ICD‐9 discharge diagnosis codes. These diagnosis codes are ascribed after discharge from the hospital, or for ED patients, after ED discharge. Additionally, patients may receive an admission diagnosis, which reflects the diagnosis ascribed at the time of ED discharge or transfer to the inpatient unit.

We reviewed medical records of infants selected from the following ICD‐9 diagnosis code groups (Figure 1): (1) discharge diagnosis code of fever (780.6 [fever and other physiologic disturbances of temperature regulation], 778.4 [other disturbances of temperature regulation of newborn], 780.60 [fever, unspecified], or 780.61 [fever presenting with conditions classified elsewhere])[9, 10] regardless of the presence of admission diagnosis of fever or diagnosis of serious infection, (2) admission diagnosis code of fever without associated discharge diagnosis code of fever,[10] (3) discharge diagnosis code of serious infection determined a priori (see Supporting Information, Appendix, in the online version of this article) without discharge or admission diagnosis code of fever, and (4) infants without any diagnosis code of fever or serious infection.

Figure 1
Study population. 1Two of 584 medical records were unavailable for review. 2Five of 904 medical records were unavailable for review. Abbreviations: CCC, complex chronic condition; ED, emergency department.

Medical records reviewed in each of the 4 ICD‐9 diagnosis code groups were randomly selected from the overall set of ED encounters in the population of infants <90 days of age evaluated during the study period. Twenty‐five percent population sampling was used for 3 of the ICD‐9 diagnosis code groups, whereas 5% sampling was used for the no fever/no serious infection code group. The number of medical records reviewed in each ICD‐9 diagnosis code group was proportional to the distribution of ICD‐9 codes across the entire population of infants <90 days of age. These records were distributed equally across sites (228 records per site), except for 1 site that does not assign admission diagnoses (201 records).

Investigators were blinded to ICD‐9 diagnosis code groups during medical record review. Infants with multiple visits during the study period were eligible to be included more than once if the visits occurred more than 3 days apart. For infants with more than 1 ED visit on a particular calendar day, investigators were instructed to review the initial visit.

For each encounter, we also abstracted demographic characteristics (gender, race/ethnicity), insurance status, hospital region (using US Census categories[22]), and season from the PHIS database.

Reference Standard

The presence of fever was determined by medical record review. We defined fever as any documented temperature 100.4F (38.0C) at home or in the ED.[16]

ICD‐9 Code Case‐Identification Algorithms

Using the aforementioned ICD‐9 diagnosis code groups individually and in combination, the following 4 case‐identification algorithms, determined from prior study or group consensus, were compared to the reference standard: (1) ICD‐9 discharge diagnosis code of fever,[9] (2) ICD‐9 admission or discharge diagnosis code of fever,[10, 11] (3) ICD‐9 discharge diagnosis code of fever or serious infection, and (4) ICD‐9 discharge or admission diagnosis code of fever or serious infection. Algorithms were compared overall, separately for discharged and hospitalized infants, and across 3 distinct age groups (28 days, 2956 days, and 5789 days).

Patient‐Level Outcomes

To compare differences in outcomes by case‐identification algorithm, from the PHIS database we abstracted hospitalization rates, rates of UTI/pyelonephritis,[13] bacteremia/sepsis, and bacterial meningitis.[19] Severe outcomes were defined as intensive care unit admission, mechanical ventilation, central line placement, receipt of extracorporeal membrane oxygenation, or death. We assessed hospital length of stay for admitted infants and 3‐day revisits,[23, 24] and revisits resulting in hospitalization for infants discharged from the ED at the index visit. Patients billed for observation care were classified as being hospitalized.[25, 26]

Data Analysis

Accuracy of the 4 case‐identification algorithms (compared with the reference standard) was calculated using sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV), along with 95% confidence interval (CI). Prior to analysis, a 5‐fold weighting factor was applied to the no fever/no serious infection group to account for the differential sampling used for this group (5% vs 25% for the other 3 ICD‐9 diagnosis code groups). This weighting was done to approximate the true prevalence of each ICD‐9 code group within the larger population, so that an accurate rate of false negatives (infants with fever who had neither a diagnosis of fever nor serious infection) could be calculated.

We described continuous variables using median and interquartile range or range values and categorical variables using frequencies with 95% CIs. We compared categorical variables using a 2 test. We determined statistical significance as a 2‐tailed P value <0.05. Statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC).

RESULTS

Study Patients

During the 1‐year study period, 23,753 ED encounters for infants <90 days of age were identified in the PHIS database at the 8 participating sites. Of these infant encounters, 2166 (9.2%) were excluded (1658 infants who had a complex chronic condition and 508 transferred into the ED), leaving 21,587 infants available for selection. After applying our sampling strategy, we identified 1797 encounters for medical record review. Seven encounters from 3 hospitals with missing medical records were excluded, resulting in a final cohort of 1790 encounters (Figure 1). Among included infants, 552 (30.8%) were 28 days, 743 (41.5%) were 29 to 56 days, and 495 (27.8%) were 57 to 89 days of age; 737 (41.2%) infants were hospitalized. Patients differed in age, race, payer, and season across ICD‐9 diagnosis code groups (see Supporting Information, Table 1, in the online version of this article).

Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms According to Reference Standard (Overall, Hospitalized, and Discharged).*
ICD‐9 Diagnosis Code AlgorithmOverall
Sensitivity, % (95% CI)Specificity, % (95% CI)Negative Predictive Value, % (95% CI)Positive Predictive Value, % (95% CI)
  • NOTE: Abbreviations: CI, confidence interval; ED, emergency department; ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4 F (38.0 C) on review of electronic medical record.

Discharge diagnosis of fever53.2 (50.056.4)98.2 (97.898.6)90.8 (90.091.6)86.1 (83.388.9)
Hospitalized47.3 (43.151.5)97.7 (96.998.5)80.6 (78.682.6)90.2 (86.893.6)
Discharged from ED61.4 (56.666.2)98.4 (98.098.8)95.4 (94.796.1)82.1 (77.786.5)
Discharge or admission diagnosis of Fever71.1 (68.274.0)97.7 (97.398.1)94.1 (93.494.8)86.9 (84.589.3)
Hospitalized72.5 (68.876.2)97.1 (96.298.0)88.8 (87.190.5)91.7 (89.194.3)
Discharged from ED69.2 (64.773.7)98.0 (97.598.5)96.3 (95.796.9)80.8 (76.685.0)
Discharge diagnosis of fever or serious infection63.7 (60.666.8)96.5 (96.097.0)92.6 (91.893.4)79.6 (76.782.5)
Hospitalized63.9 (59.967.9)92.5 (91.094.0)85.1 (83.287.0)79.1 (75.382.9)
Discharged from ED63.4 (58.768.1)98.1 (97.698.6)95.6 (94.996.3)80.2 (75.884.6)
Discharge or admission diagnosis of fever or serious infection76.6 (73.979.3)96.2 (95.696.8)95.1 (94.595.7)81.0 (78.483.6)
Hospitalized80.8 (77.584.1)92.1 (90.693.6)91.5 (89.993.1)82.1 (78.985.3)
Discharged from ED71.0 (66.575.5)97.7 (97.298.2)96.5 (95.997.1)79.4 (75.283.6)

Among the 1790 patient encounters reviewed, a total of 766 infants (42.8%) met the reference standard definition for fever in the cohort. An additional 47 infants had abnormal temperature reported (documentation of tactile fever, history of fever without a specific temperature described, or hypothermia) but were classified as having no fever by the reference standard.

ICD‐9 Code Case‐Identification Algorithm Performance

Compared with the reference standard, the 4 case‐identification algorithms demonstrated specificity of 96.2% to 98.2% but lower sensitivity overall (Figure 2). Discharge diagnosis of fever alone demonstrated the lowest sensitivity. The algorithm of discharge or admission diagnosis of fever resulted in increased sensitivity and the highest PPV of all 4 algorithms (86.9%, 95% CI: 84.5‐89.3). Addition of serious infection codes to this algorithm resulted in a marginal increase in sensitivity and a similar decrease in PPV (Table 1). When limited to hospitalized infants, specificity was highest for the case‐identification algorithm of discharge diagnosis of fever and similarly high for discharge or admission diagnosis of fever; sensitivity was highest for the algorithm of discharge or admission diagnosis of fever or diagnosis of serious infection. For infants discharged from the ED, algorithm specificity was 97.7% to 98.4%, with lower sensitivity for all 4 algorithms (Table 1). Inclusion of the 47 infants with abnormal temperature as fever did not materially change algorithm performance (data not shown).

Figure 2
Algorithm sensitivity and false positive rate (1‐specificity) for identification of febrile infants aged ≤28 days, 29 to 56 days, 57 to 89 days, and overall. Horizontal and vertical bars represent 95% confidence intervals. Reference standard of fever was defined by documented temperature ≥100.4°F (38.0°C) on review of electronic medical record.

Across all 3 age groups (28 days, 2956 days, and 5789 days), the 4 case‐identification algorithms demonstrated specificity >96%, whereas algorithm sensitivity was highest in the 29‐ to 56‐days‐old age group and lowest among infants 57 to 89 days old across all 4 algorithms (Figure 2). Similar to the overall cohort, an algorithm of discharge or admission diagnosis of fever demonstrated specificity of nearly 98% in all age groups; addition of serious infection codes to this algorithm increased sensitivity, highest in the 29‐ to 56‐days‐old age group (Figure 2; see also Supporting Information, Table 2, in the online version of this article).

Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms Across the Eight Sites According to Reference Standard.*
ICD‐9 Diagnosis Code AlgorithmSensitivity, Median % (Range)Specificity, Median % (Range)Negative Predictive Value, Median % (Range)Positive Predictive Value, Median % (Range)
  • NOTE: Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4F (38.0 C) on review of electronic medical record.

Discharge diagnosis of fever56.2 (34.681.0)98.3 (96.499.1)92.1 (83.297.4)87.7 (74.093.2)
Discharge or Admission diagnosis of Fever76.7 (51.385.0)97.8 (96.298.7)95.6 (86.997.4)87.4 (80.092.9)
Discharge diagnosis of fever or serious infection68.3 (44.287.3)96.5 (95.498.0)93.6 (85.298.2)78.3 (74.289.0)
Discharge or admission diagnosis of fever or serious infection83.1 (58.390.7)95.8 (95.498.0)96.5 (88.598.2)79.1 (77.490.4)

Across the 8 study sites, median specificity was 95.8% to 98.3% for the 4 algorithms, with little interhospital variability; however, algorithm sensitivity varied widely by site. Median PPV was highest for discharge diagnosis of fever alone at 87.7% but ranged from 74.0% to 93.2% across sites. Median PPV for an algorithm of discharge or admission diagnosis of fever was similar (87.4%) but with less variation by site (range 80.0%92.9%) (Table 2).

Outcomes by ICD‐9 Diagnosis Code Group and Case‐Identification Algorithm

When compared with discharge diagnosis of fever, adding admission diagnosis of fever captured a higher proportion of hospitalized infants with SBIs (UTI/pyelonephritis, bacteremia/sepsis, or bacterial meningitis). However, median hospital length of stay, severe outcomes, and 3‐day revisits and revisits with hospitalization did not materially differ when including infants with admission diagnosis of fever in addition to discharge diagnosis of fever. Addition of infants with a diagnosis code for serious infection substantially increased the number of infants with SBIs and severe outcomes but did not capture additional 3‐day revisits (Table 3). There were no additional cases of SBI in the no fever/no serious illness diagnosis code group.

Outcomes by ICD‐9 Diagnosis Code Case‐Identification Algorithm
ICD‐9 Diagnosis Code AlgorithmOutcome3‐Day Revisit, % (95% CI)3‐Day Revisit With Hospitalization, % (95% CI)
Hospitalized, % (95% CI)UTI/Pyelonephritis, Bacteremia/Sepsis, or Bacterial Meningitis, % (95% CI)Severe Outcome, % (95% CI)*Length of Stay in Days, Median (IQR)
  • NOTE: Abbreviations: CI, confidence interval; ICD‐9, International Classification of Diseases, Ninth Revision; IQR, interquartile range; UTI, urinary tract infection. *Severe outcome was defined as intensive care unit admission, mechanical ventilation, central line placement, extracorporeal membrane oxygenation, or death. Length of stay for hospitalized infants. Percent of those discharged from the emergency department at the index visit.

Discharge diagnosis of fever44.3 (40.348.4)3.3 (1.84.7)1.4 (0.42.3)3 (23)11.7 (8.215.2)5.9 (3.38.4)
Discharge or admission diagnosis of fever52.4 (48.955.9)6.1 (4.47.8)1.9 (1.02.9)3 (23)10.9 (7.714.1)5.4 (3.17.8)
Discharge diagnosis of fever or serious infection54.0 (50.457.5)15.3 (12.717.8)3.8 (2.55.2)3 (24)11.0 (7.714.2)5.5 (3.17.9)
Discharge or admission diagnosis of fever or serious infection56.5 (53.259.7)12.9 (10.715.1)3.6 (2.44.8)3 (24)10.3 (7.313.3)5.2 (3.07.4)

Among infants who met the reference standard for fever but did not have a discharge or admission diagnosis of fever (false negatives), 11.8% had a diagnosis of SBI. Overall, 43.2% of febrile infants (and 84.4% of hospitalized infants) with SBI did not have an ICD‐9 discharge or admission diagnosis of fever. Addition of ICD‐9 diagnosis codes of serious infection to the algorithm of discharge or admission diagnosis of fever captured all additional SBIs, and no false negativeinfants missed with this algorithm had an SBI.

DISCUSSION

We described the performance of 4 ICD‐9 diagnosis code case‐identification algorithms for the identification of febrile young infants <90 days of age at US children's hospitals. Although the specificity was high across algorithms and institutions, the sensitivity was relatively low, particularly for discharge diagnosis of fever, and varied by institution. Given the high specificity, ICD‐9 diagnosis code case‐identification algorithms for fever reliably identify febrile infants using administrative data with low rates of inclusion of infants without fever. However, underidentification of patients, particularly those more prone to SBIs and severe outcomes depending on the algorithm utilized, can impact interpretation of comparative effectiveness studies or the quality of care delivered by an institution.

ICD‐9 discharge diagnosis codes are frequently used to identify pediatric patients across a variety of administrative databases, diseases, and symptoms.[19, 27, 28, 29, 30, 31] Although discharge diagnosis of fever is highly specific, sensitivity is substantially lower than other case‐identification algorithms we studied, particularly for hospitalized infants. This may be due to a fever code sometimes being omitted in favor of a more specific diagnosis (eg, bacteremia) prior to hospital discharge. Therefore, case identification relying only on ICD‐9 discharge diagnosis codes for fever may under‐report clinically important SBI or severe outcomes as demonstrated in our study. This is in contrast to ICD‐9 diagnosis code identification strategies for childhood UTI and pneumonia, which largely have higher sensitivity but lower specificity than fever codes.[13, 14]

Admission diagnosis of fever is important for febrile infants as they may not have an explicit diagnosis at the time of disposition from the ED. Addition of admission diagnosis of fever to an algorithm relying on discharge diagnosis code alone increased sensitivity without a demonstrable reduction in specificity and PPV, likely due to capture of infants with a fever diagnosis at presentation before a specific infection was identified. Although using an algorithm of discharge or admission diagnosis of fever captured a higher percentage of hospitalized febrile infants with SBIs, sensitivity was only 71% overall with this algorithm, and 43% of febrile infants with SBI would still have been missed. Importantly, though, addition of various ICD‐9 codes for serious infection to this algorithm resulted in capture of all febrile infants with SBI and should be used as a sensitivity analysis.

The test characteristics of diagnosis codes were highest in the 29‐ to 56‐days‐old age group. Given the differing low‐risk criteria[6, 7, 8] and lack of best practice guidelines[16] in this age group, the use of administrative data may allow for the comparison of testing and treatment strategies across a large cohort of febrile infants aged 29 to 56 days. However, individual hospital coding practices may affect algorithm performance, in particular sensitivity, which varied substantially by hospital. This variation in algorithm sensitivity may impact comparisons of outcomes across institutions. Therefore, when conducting studies of febrile infants using administrative data, sensitivity analyses or use of chart review should be considered to augment the use of ICD‐9 code‐based identification strategies, particularly for comparative benchmarking and outcomes studies. These additional analyses are particularly important for studies of febrile infants >56 days of age, in whom the sensitivity of diagnosis codes is particularly low. We speculate that the lower sensitivity in older febrile infants may relate to a lack of consensus on the clinical significance of fever in this age group and the varying management strategies employed.[10]

Strengths of this study include the assessment of ICD‐9 code algorithms across multiple institutions for identification of fever in young infants, and the patterns of our findings remained robust when comparing median performance characteristics of the algorithms across hospitals to our overall findings. We were also able to accurately estimate PPV and NPV using a case‐identification strategy weighted to the actual population sizes. Although sensitivity and specificity are the primary measures of test performance, predictive values are highly informative for investigators using administrative data. Additionally, our findings may inform public health efforts including disease surveillance, assessment of seasonal variation, and identification and monitoring of healthcare‐associated infections among febrile infants.

Our study has limitations. We did not review all identified records, which raises the possibility that our evaluated cohort may not be representative of the entire febrile infant population. We attempted to mitigate this possibility by using a random sampling strategy for our population selection that was weighted to the actual population sizes. Second, we identified serious infections using ICD‐9 diagnosis codes determined by group consensus, which may not capture all serious infection codes that identify febrile infants whose fever code was omitted. Third, 47 infants had abnormal temperature that did not meet our reference standard criteria for fever and were included in the no fever group. Although there may be disagreement regarding what constitutes a fever, we used a widely accepted reference standard to define fever.[16] Further, inclusion of these 47 infants as fever did not materially change algorithm performance. Last, our study was conducted at 8 large tertiary‐care children's hospitals, and our results may not be generalizable to other children's hospitals and community‐based hospitals.

CONCLUSIONS

Studies of febrile young infants that rely on ICD‐9 discharge diagnosis code of fever for case ascertainment have high specificity but low sensitivity for the identification of febrile infants, particularly among hospitalized patients. A case‐identification strategy that includes discharge or admission diagnosis of fever demonstrated higher sensitivity, and should be considered for studies of febrile infants using administrative data. However, additional strategies such as incorporation of ICD‐9 codes for serious infection should be used when comparing outcomes across institutions.

Acknowledgements

The Febrile Young Infant Research Collaborative includes the following additional collaborators who are acknowledged for their work on this study: Erica DiLeo, MA, Department of Medical Education and Research, Danbury Hospital, Danbury, Connecticut; Janet Flores, BS, Division of Emergency Medicine, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, Illinois.

Disclosures: This project funded in part by The Gerber Foundation Novice Researcher Award, (Ref No. 1827‐3835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in the design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript. The authors have no conflicts of interest relevant to this article to disclose.

Fever is one of the most common reasons for emergency department (ED) evaluation of infants under 90 days of age.[1] Up to 10% to 20% of febrile young infants will have a serious bacterial infection (SBI),[2, 3, 4] but infants with SBI are difficult to distinguish from those without SBI based upon symptoms and physical examination findings alone.[5] Previously developed clinical prediction algorithms can help to identify febrile infants at low risk for SBI, but differ in age range as well as recommendations for testing and empiric treatment.[6, 7, 8] Consequently, there is widespread variation in management of febrile young infants at US children's hospitals,[9, 10, 11] and defining optimal management strategies remains an important issue in pediatric healthcare.[12] Administrative datasets are convenient and inexpensive, and can be used to evaluate practice variation, trends, and outcomes of a large, diverse group of patients within and across institutions.[9, 10] Accurately identifying febrile infants evaluated for suspected SBI in administrative databases would facilitate comparative effectiveness research, quality improvement initiatives, and institutional benchmarking.

Prior studies have validated the accuracy of administrative billing codes for identification of other common childhood illnesses, including urinary tract infection (UTI)[13] and pneumonia.[14] The accuracy of International Classification of Diseases, Ninth Revision (ICD‐9) diagnosis codes in identifying febrile young infants evaluated for SBI is not known. Reliance on administrative ICD‐9 diagnosis codes for patient identification can lead to misclassification of patients due to variable database quality, the validity of the diagnosis codes being utilized, and hospital coding practices.[15] Additionally, fever is a symptom and not a specific diagnosis. If a particular bacterial or viral diagnosis is established (eg, enterovirus meningitis), a discharge diagnosis of fever may not be attributed to the patient encounter. Thus, evaluating the performance characteristics and capture of clinical outcomes of different combinations of ICD‐9 diagnosis codes for identifying febrile infants is necessary for both the conduct and interpretation of studies that utilize administrative databases. The primary objective of this investigation was to identify the most accurate ICD‐9 coding strategies for the identification of febrile infants aged <90 days using administrative data. We also sought to evaluate capture of clinically important outcomes across identification strategies.

METHODS

Study Design and Setting

For this multicenter retrospective study, we used the Pediatric Health Information System (PHIS) database to identify infants <90 days of age[16] who presented between July 1, 2012 and June 30, 2013 to 1 of 8 EDs. We assessed performance characteristics of ICD‐9 diagnosis code case‐identification algorithms by comparing ICD‐9 code combinations to a fever reference standard determined by medical record review. The institutional review board at each participating site approved the study protocol.

Data Source

Data were obtained from 2 sources: the PHIS database and medical record review. We used the PHIS database to identify eligible patients by ICD‐9 diagnosis codes; patient encounters were randomly selected using a random number generator. The PHIS database contains demographic, diagnosis, and billing data from 44 hospitals affiliated with the Children's Hospital Association (Overland Park, Kansas) and represents 85% of freestanding children's hospitals in the United States.[17] Data are deidentified; encrypted unique patient identifiers permit tracking of patients across visits within a site.[18] The Children's Hospital Association and participating hospitals jointly assure the quality and integrity of the data.[19]

For each patient encounter identified in the PHIS database, detailed medical record review was performed by trained investigators at each of the 8 study sites (see Supporting Information, Appendix, in the online version of this article). A standardized data collection instrument was pilot tested by all investigators prior to use. Data were collected and managed using the Research Electronic Data Capture (REDCap) tool hosted at Boston Children's Hospital.[20]

Exclusions

Using PHIS data, prior to medical record review we excluded infants with a complex chronic condition as defined previously[21] and those transferred from another institution, as these infants may warrant a nonstandard evaluation and/or may have incomplete data.

ICD‐9 Diagnosis Code Groups

In the PHIS database, all patients discharged from the hospital (including hospitalized patients as well as patients discharged from the ED) receive 1 or more ICD‐9 discharge diagnosis codes. These diagnosis codes are ascribed after discharge from the hospital, or for ED patients, after ED discharge. Additionally, patients may receive an admission diagnosis, which reflects the diagnosis ascribed at the time of ED discharge or transfer to the inpatient unit.

We reviewed medical records of infants selected from the following ICD‐9 diagnosis code groups (Figure 1): (1) discharge diagnosis code of fever (780.6 [fever and other physiologic disturbances of temperature regulation], 778.4 [other disturbances of temperature regulation of newborn], 780.60 [fever, unspecified], or 780.61 [fever presenting with conditions classified elsewhere])[9, 10] regardless of the presence of admission diagnosis of fever or diagnosis of serious infection, (2) admission diagnosis code of fever without associated discharge diagnosis code of fever,[10] (3) discharge diagnosis code of serious infection determined a priori (see Supporting Information, Appendix, in the online version of this article) without discharge or admission diagnosis code of fever, and (4) infants without any diagnosis code of fever or serious infection.

Figure 1
Study population. 1Two of 584 medical records were unavailable for review. 2Five of 904 medical records were unavailable for review. Abbreviations: CCC, complex chronic condition; ED, emergency department.

Medical records reviewed in each of the 4 ICD‐9 diagnosis code groups were randomly selected from the overall set of ED encounters in the population of infants <90 days of age evaluated during the study period. Twenty‐five percent population sampling was used for 3 of the ICD‐9 diagnosis code groups, whereas 5% sampling was used for the no fever/no serious infection code group. The number of medical records reviewed in each ICD‐9 diagnosis code group was proportional to the distribution of ICD‐9 codes across the entire population of infants <90 days of age. These records were distributed equally across sites (228 records per site), except for 1 site that does not assign admission diagnoses (201 records).

Investigators were blinded to ICD‐9 diagnosis code groups during medical record review. Infants with multiple visits during the study period were eligible to be included more than once if the visits occurred more than 3 days apart. For infants with more than 1 ED visit on a particular calendar day, investigators were instructed to review the initial visit.

For each encounter, we also abstracted demographic characteristics (gender, race/ethnicity), insurance status, hospital region (using US Census categories[22]), and season from the PHIS database.

Reference Standard

The presence of fever was determined by medical record review. We defined fever as any documented temperature 100.4F (38.0C) at home or in the ED.[16]

ICD‐9 Code Case‐Identification Algorithms

Using the aforementioned ICD‐9 diagnosis code groups individually and in combination, the following 4 case‐identification algorithms, determined from prior study or group consensus, were compared to the reference standard: (1) ICD‐9 discharge diagnosis code of fever,[9] (2) ICD‐9 admission or discharge diagnosis code of fever,[10, 11] (3) ICD‐9 discharge diagnosis code of fever or serious infection, and (4) ICD‐9 discharge or admission diagnosis code of fever or serious infection. Algorithms were compared overall, separately for discharged and hospitalized infants, and across 3 distinct age groups (28 days, 2956 days, and 5789 days).

Patient‐Level Outcomes

To compare differences in outcomes by case‐identification algorithm, from the PHIS database we abstracted hospitalization rates, rates of UTI/pyelonephritis,[13] bacteremia/sepsis, and bacterial meningitis.[19] Severe outcomes were defined as intensive care unit admission, mechanical ventilation, central line placement, receipt of extracorporeal membrane oxygenation, or death. We assessed hospital length of stay for admitted infants and 3‐day revisits,[23, 24] and revisits resulting in hospitalization for infants discharged from the ED at the index visit. Patients billed for observation care were classified as being hospitalized.[25, 26]

Data Analysis

Accuracy of the 4 case‐identification algorithms (compared with the reference standard) was calculated using sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV), along with 95% confidence interval (CI). Prior to analysis, a 5‐fold weighting factor was applied to the no fever/no serious infection group to account for the differential sampling used for this group (5% vs 25% for the other 3 ICD‐9 diagnosis code groups). This weighting was done to approximate the true prevalence of each ICD‐9 code group within the larger population, so that an accurate rate of false negatives (infants with fever who had neither a diagnosis of fever nor serious infection) could be calculated.

We described continuous variables using median and interquartile range or range values and categorical variables using frequencies with 95% CIs. We compared categorical variables using a 2 test. We determined statistical significance as a 2‐tailed P value <0.05. Statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC).

RESULTS

Study Patients

During the 1‐year study period, 23,753 ED encounters for infants <90 days of age were identified in the PHIS database at the 8 participating sites. Of these infant encounters, 2166 (9.2%) were excluded (1658 infants who had a complex chronic condition and 508 transferred into the ED), leaving 21,587 infants available for selection. After applying our sampling strategy, we identified 1797 encounters for medical record review. Seven encounters from 3 hospitals with missing medical records were excluded, resulting in a final cohort of 1790 encounters (Figure 1). Among included infants, 552 (30.8%) were 28 days, 743 (41.5%) were 29 to 56 days, and 495 (27.8%) were 57 to 89 days of age; 737 (41.2%) infants were hospitalized. Patients differed in age, race, payer, and season across ICD‐9 diagnosis code groups (see Supporting Information, Table 1, in the online version of this article).

Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms According to Reference Standard (Overall, Hospitalized, and Discharged).*
ICD‐9 Diagnosis Code AlgorithmOverall
Sensitivity, % (95% CI)Specificity, % (95% CI)Negative Predictive Value, % (95% CI)Positive Predictive Value, % (95% CI)
  • NOTE: Abbreviations: CI, confidence interval; ED, emergency department; ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4 F (38.0 C) on review of electronic medical record.

Discharge diagnosis of fever53.2 (50.056.4)98.2 (97.898.6)90.8 (90.091.6)86.1 (83.388.9)
Hospitalized47.3 (43.151.5)97.7 (96.998.5)80.6 (78.682.6)90.2 (86.893.6)
Discharged from ED61.4 (56.666.2)98.4 (98.098.8)95.4 (94.796.1)82.1 (77.786.5)
Discharge or admission diagnosis of Fever71.1 (68.274.0)97.7 (97.398.1)94.1 (93.494.8)86.9 (84.589.3)
Hospitalized72.5 (68.876.2)97.1 (96.298.0)88.8 (87.190.5)91.7 (89.194.3)
Discharged from ED69.2 (64.773.7)98.0 (97.598.5)96.3 (95.796.9)80.8 (76.685.0)
Discharge diagnosis of fever or serious infection63.7 (60.666.8)96.5 (96.097.0)92.6 (91.893.4)79.6 (76.782.5)
Hospitalized63.9 (59.967.9)92.5 (91.094.0)85.1 (83.287.0)79.1 (75.382.9)
Discharged from ED63.4 (58.768.1)98.1 (97.698.6)95.6 (94.996.3)80.2 (75.884.6)
Discharge or admission diagnosis of fever or serious infection76.6 (73.979.3)96.2 (95.696.8)95.1 (94.595.7)81.0 (78.483.6)
Hospitalized80.8 (77.584.1)92.1 (90.693.6)91.5 (89.993.1)82.1 (78.985.3)
Discharged from ED71.0 (66.575.5)97.7 (97.298.2)96.5 (95.997.1)79.4 (75.283.6)

Among the 1790 patient encounters reviewed, a total of 766 infants (42.8%) met the reference standard definition for fever in the cohort. An additional 47 infants had abnormal temperature reported (documentation of tactile fever, history of fever without a specific temperature described, or hypothermia) but were classified as having no fever by the reference standard.

ICD‐9 Code Case‐Identification Algorithm Performance

Compared with the reference standard, the 4 case‐identification algorithms demonstrated specificity of 96.2% to 98.2% but lower sensitivity overall (Figure 2). Discharge diagnosis of fever alone demonstrated the lowest sensitivity. The algorithm of discharge or admission diagnosis of fever resulted in increased sensitivity and the highest PPV of all 4 algorithms (86.9%, 95% CI: 84.5‐89.3). Addition of serious infection codes to this algorithm resulted in a marginal increase in sensitivity and a similar decrease in PPV (Table 1). When limited to hospitalized infants, specificity was highest for the case‐identification algorithm of discharge diagnosis of fever and similarly high for discharge or admission diagnosis of fever; sensitivity was highest for the algorithm of discharge or admission diagnosis of fever or diagnosis of serious infection. For infants discharged from the ED, algorithm specificity was 97.7% to 98.4%, with lower sensitivity for all 4 algorithms (Table 1). Inclusion of the 47 infants with abnormal temperature as fever did not materially change algorithm performance (data not shown).

Figure 2
Algorithm sensitivity and false positive rate (1‐specificity) for identification of febrile infants aged ≤28 days, 29 to 56 days, 57 to 89 days, and overall. Horizontal and vertical bars represent 95% confidence intervals. Reference standard of fever was defined by documented temperature ≥100.4°F (38.0°C) on review of electronic medical record.

Across all 3 age groups (28 days, 2956 days, and 5789 days), the 4 case‐identification algorithms demonstrated specificity >96%, whereas algorithm sensitivity was highest in the 29‐ to 56‐days‐old age group and lowest among infants 57 to 89 days old across all 4 algorithms (Figure 2). Similar to the overall cohort, an algorithm of discharge or admission diagnosis of fever demonstrated specificity of nearly 98% in all age groups; addition of serious infection codes to this algorithm increased sensitivity, highest in the 29‐ to 56‐days‐old age group (Figure 2; see also Supporting Information, Table 2, in the online version of this article).

Performance Characteristics of ICD‐9 Diagnosis Code Case‐Identification Algorithms Across the Eight Sites According to Reference Standard.*
ICD‐9 Diagnosis Code AlgorithmSensitivity, Median % (Range)Specificity, Median % (Range)Negative Predictive Value, Median % (Range)Positive Predictive Value, Median % (Range)
  • NOTE: Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision. *Reference standard of fever was defined by documented temperature 100.4F (38.0 C) on review of electronic medical record.

Discharge diagnosis of fever56.2 (34.681.0)98.3 (96.499.1)92.1 (83.297.4)87.7 (74.093.2)
Discharge or Admission diagnosis of Fever76.7 (51.385.0)97.8 (96.298.7)95.6 (86.997.4)87.4 (80.092.9)
Discharge diagnosis of fever or serious infection68.3 (44.287.3)96.5 (95.498.0)93.6 (85.298.2)78.3 (74.289.0)
Discharge or admission diagnosis of fever or serious infection83.1 (58.390.7)95.8 (95.498.0)96.5 (88.598.2)79.1 (77.490.4)

Across the 8 study sites, median specificity was 95.8% to 98.3% for the 4 algorithms, with little interhospital variability; however, algorithm sensitivity varied widely by site. Median PPV was highest for discharge diagnosis of fever alone at 87.7% but ranged from 74.0% to 93.2% across sites. Median PPV for an algorithm of discharge or admission diagnosis of fever was similar (87.4%) but with less variation by site (range 80.0%92.9%) (Table 2).

Outcomes by ICD‐9 Diagnosis Code Group and Case‐Identification Algorithm

When compared with discharge diagnosis of fever, adding admission diagnosis of fever captured a higher proportion of hospitalized infants with SBIs (UTI/pyelonephritis, bacteremia/sepsis, or bacterial meningitis). However, median hospital length of stay, severe outcomes, and 3‐day revisits and revisits with hospitalization did not materially differ when including infants with admission diagnosis of fever in addition to discharge diagnosis of fever. Addition of infants with a diagnosis code for serious infection substantially increased the number of infants with SBIs and severe outcomes but did not capture additional 3‐day revisits (Table 3). There were no additional cases of SBI in the no fever/no serious illness diagnosis code group.

Outcomes by ICD‐9 Diagnosis Code Case‐Identification Algorithm
ICD‐9 Diagnosis Code AlgorithmOutcome3‐Day Revisit, % (95% CI)3‐Day Revisit With Hospitalization, % (95% CI)
Hospitalized, % (95% CI)UTI/Pyelonephritis, Bacteremia/Sepsis, or Bacterial Meningitis, % (95% CI)Severe Outcome, % (95% CI)*Length of Stay in Days, Median (IQR)
  • NOTE: Abbreviations: CI, confidence interval; ICD‐9, International Classification of Diseases, Ninth Revision; IQR, interquartile range; UTI, urinary tract infection. *Severe outcome was defined as intensive care unit admission, mechanical ventilation, central line placement, extracorporeal membrane oxygenation, or death. Length of stay for hospitalized infants. Percent of those discharged from the emergency department at the index visit.

Discharge diagnosis of fever44.3 (40.348.4)3.3 (1.84.7)1.4 (0.42.3)3 (23)11.7 (8.215.2)5.9 (3.38.4)
Discharge or admission diagnosis of fever52.4 (48.955.9)6.1 (4.47.8)1.9 (1.02.9)3 (23)10.9 (7.714.1)5.4 (3.17.8)
Discharge diagnosis of fever or serious infection54.0 (50.457.5)15.3 (12.717.8)3.8 (2.55.2)3 (24)11.0 (7.714.2)5.5 (3.17.9)
Discharge or admission diagnosis of fever or serious infection56.5 (53.259.7)12.9 (10.715.1)3.6 (2.44.8)3 (24)10.3 (7.313.3)5.2 (3.07.4)

Among infants who met the reference standard for fever but did not have a discharge or admission diagnosis of fever (false negatives), 11.8% had a diagnosis of SBI. Overall, 43.2% of febrile infants (and 84.4% of hospitalized infants) with SBI did not have an ICD‐9 discharge or admission diagnosis of fever. Addition of ICD‐9 diagnosis codes of serious infection to the algorithm of discharge or admission diagnosis of fever captured all additional SBIs, and no false negativeinfants missed with this algorithm had an SBI.

DISCUSSION

We described the performance of 4 ICD‐9 diagnosis code case‐identification algorithms for the identification of febrile young infants <90 days of age at US children's hospitals. Although the specificity was high across algorithms and institutions, the sensitivity was relatively low, particularly for discharge diagnosis of fever, and varied by institution. Given the high specificity, ICD‐9 diagnosis code case‐identification algorithms for fever reliably identify febrile infants using administrative data with low rates of inclusion of infants without fever. However, underidentification of patients, particularly those more prone to SBIs and severe outcomes depending on the algorithm utilized, can impact interpretation of comparative effectiveness studies or the quality of care delivered by an institution.

ICD‐9 discharge diagnosis codes are frequently used to identify pediatric patients across a variety of administrative databases, diseases, and symptoms.[19, 27, 28, 29, 30, 31] Although discharge diagnosis of fever is highly specific, sensitivity is substantially lower than other case‐identification algorithms we studied, particularly for hospitalized infants. This may be due to a fever code sometimes being omitted in favor of a more specific diagnosis (eg, bacteremia) prior to hospital discharge. Therefore, case identification relying only on ICD‐9 discharge diagnosis codes for fever may under‐report clinically important SBI or severe outcomes as demonstrated in our study. This is in contrast to ICD‐9 diagnosis code identification strategies for childhood UTI and pneumonia, which largely have higher sensitivity but lower specificity than fever codes.[13, 14]

Admission diagnosis of fever is important for febrile infants as they may not have an explicit diagnosis at the time of disposition from the ED. Addition of admission diagnosis of fever to an algorithm relying on discharge diagnosis code alone increased sensitivity without a demonstrable reduction in specificity and PPV, likely due to capture of infants with a fever diagnosis at presentation before a specific infection was identified. Although using an algorithm of discharge or admission diagnosis of fever captured a higher percentage of hospitalized febrile infants with SBIs, sensitivity was only 71% overall with this algorithm, and 43% of febrile infants with SBI would still have been missed. Importantly, though, addition of various ICD‐9 codes for serious infection to this algorithm resulted in capture of all febrile infants with SBI and should be used as a sensitivity analysis.

The test characteristics of diagnosis codes were highest in the 29‐ to 56‐days‐old age group. Given the differing low‐risk criteria[6, 7, 8] and lack of best practice guidelines[16] in this age group, the use of administrative data may allow for the comparison of testing and treatment strategies across a large cohort of febrile infants aged 29 to 56 days. However, individual hospital coding practices may affect algorithm performance, in particular sensitivity, which varied substantially by hospital. This variation in algorithm sensitivity may impact comparisons of outcomes across institutions. Therefore, when conducting studies of febrile infants using administrative data, sensitivity analyses or use of chart review should be considered to augment the use of ICD‐9 code‐based identification strategies, particularly for comparative benchmarking and outcomes studies. These additional analyses are particularly important for studies of febrile infants >56 days of age, in whom the sensitivity of diagnosis codes is particularly low. We speculate that the lower sensitivity in older febrile infants may relate to a lack of consensus on the clinical significance of fever in this age group and the varying management strategies employed.[10]

Strengths of this study include the assessment of ICD‐9 code algorithms across multiple institutions for identification of fever in young infants, and the patterns of our findings remained robust when comparing median performance characteristics of the algorithms across hospitals to our overall findings. We were also able to accurately estimate PPV and NPV using a case‐identification strategy weighted to the actual population sizes. Although sensitivity and specificity are the primary measures of test performance, predictive values are highly informative for investigators using administrative data. Additionally, our findings may inform public health efforts including disease surveillance, assessment of seasonal variation, and identification and monitoring of healthcare‐associated infections among febrile infants.

Our study has limitations. We did not review all identified records, which raises the possibility that our evaluated cohort may not be representative of the entire febrile infant population. We attempted to mitigate this possibility by using a random sampling strategy for our population selection that was weighted to the actual population sizes. Second, we identified serious infections using ICD‐9 diagnosis codes determined by group consensus, which may not capture all serious infection codes that identify febrile infants whose fever code was omitted. Third, 47 infants had abnormal temperature that did not meet our reference standard criteria for fever and were included in the no fever group. Although there may be disagreement regarding what constitutes a fever, we used a widely accepted reference standard to define fever.[16] Further, inclusion of these 47 infants as fever did not materially change algorithm performance. Last, our study was conducted at 8 large tertiary‐care children's hospitals, and our results may not be generalizable to other children's hospitals and community‐based hospitals.

CONCLUSIONS

Studies of febrile young infants that rely on ICD‐9 discharge diagnosis code of fever for case ascertainment have high specificity but low sensitivity for the identification of febrile infants, particularly among hospitalized patients. A case‐identification strategy that includes discharge or admission diagnosis of fever demonstrated higher sensitivity, and should be considered for studies of febrile infants using administrative data. However, additional strategies such as incorporation of ICD‐9 codes for serious infection should be used when comparing outcomes across institutions.

Acknowledgements

The Febrile Young Infant Research Collaborative includes the following additional collaborators who are acknowledged for their work on this study: Erica DiLeo, MA, Department of Medical Education and Research, Danbury Hospital, Danbury, Connecticut; Janet Flores, BS, Division of Emergency Medicine, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, Illinois.

Disclosures: This project funded in part by The Gerber Foundation Novice Researcher Award, (Ref No. 1827‐3835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in the design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript. The authors have no conflicts of interest relevant to this article to disclose.

References
  1. Baskin MN. The prevalence of serious bacterial infections by age in febrile infants during the first 3 months of life. Pediatr Ann. 1993;22:462466.
  2. Huppler AR, Eickhoff JC, Wald ER. Performance of low‐risk criteria in the evaluation of young infants with fever: review of the literature. Pediatrics. 2010;125:228233.
  3. Schwartz S, Raveh D, Toker O, Segal G, Godovitch N, Schlesinger Y. A week‐by‐week analysis of the low‐risk criteria for serious bacterial infection in febrile neonates. Arch Dis Child. 2009;94:287292.
  4. Garcia S, Mintegi S, Gomez B, et al. Is 15 days an appropriate cut‐off age for considering serious bacterial infection in the management of febrile infants? Pediatr Infect Dis J. 2012;31:455458.
  5. Baker MD, Avner JR, Bell LM. Failure of infant observation scales in detecting serious illness in febrile, 4‐ to 8‐week‐old infants. Pediatrics. 1990;85:10401043.
  6. Baker MD, Bell LM, Avner JR. Outpatient management without antibiotics of fever in selected infants. N Engl J Med. 1993;329:14371441.
  7. Baskin MN, Fleisher GR, O'Rourke EJ. Identifying febrile infants at risk for a serious bacterial infection. J Pediatr. 1993;123:489490.
  8. Jaskiewicz JA, McCarthy CA, Richardson AC, et al. Febrile infants at low risk for serious bacterial infection—an appraisal of the Rochester criteria and implications for management. Febrile Infant Collaborative Study Group. Pediatrics. 1994;94:390396.
  9. Jain S, Cheng J, Alpern ER, et al. Management of febrile neonates in US pediatric emergency departments. Pediatrics. 2014;133:187195.
  10. Aronson PL, Thurm C, Alpern ER, et al. Variation in care of the febrile young infant <90 days in US pediatric emergency departments. Pediatrics. 2014;134:667677.
  11. Aronson PL, Thurm C, Williams DJ, et al. Association of clinical practice guidelines with emergency department management of febrile infants ≤56 days of age. J Hosp Med. 2015;10:358365.
  12. Hui C, Neto G, Tsertsvadze A, et al. Diagnosis and management of febrile infants (0‐3 months). Evid Rep Technol Assess (Full Rep). 2012;(205):1297.
  13. Tieder JS, Hall M, Auger KA, et al. Accuracy of administrative billing codes to detect urinary tract infection hospitalizations. Pediatrics. 2011;128:323330.
  14. Williams DJ, Shah SS, Myers A, et al. Identifying pediatric community‐acquired pneumonia hospitalizations: accuracy of administrative billing codes. JAMA Pediatr. 2013;167:851858.
  15. Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64:821829.
  16. American College of Emergency Physicians Clinical Policies Committee; American College of Emergency Physicians Clinical Policies Subcommittee on Pediatric Fever. Clinical policy for children younger than three years presenting to the emergency department with fever. Ann Emerg Med. 2003;42:530545.
  17. Wood JN, Feudtner C, Medina SP, Luan X, Localio R, Rubin DM. Variation in occult injury screening for children with suspected abuse in selected US children's hospitals. Pediatrics. 2012;130:853860.
  18. Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J AHIMA. 2004;75:2226.
  19. Mongelluzzo J, Mohamad Z, Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299:20482055.
  20. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
  21. Feudtner C, Hays RM, Haynes G, Geyer JR, Neff JM, Koepsell TD. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics. 2001;107:E99.
  22. US Census Bureau. Geographic terms and concepts—census divisions and census regions. Available at: https://www.census.gov/geo/reference/gtc/gtc_census_divreg.html. Accessed October 20, 2014.
  23. Gordon JA, An LC, Hayward RA, Williams BC. Initial emergency department diagnosis and return visits: risk versus perception. Ann Emerg Med. 1998;32:569573.
  24. Cho CS, Shapiro DJ, Cabana MD, Maselli JH, Hersh AL. A national depiction of children with return visits to the emergency department within 72 hours, 2001–2007. Pediatr Emerg Care. 2012;28:606610.
  25. Macy ML, Hall M, Shah SS, et al. Pediatric observation status: are we overlooking a growing population in children's hospitals? J Hosp Med. 2012;7:530536.
  26. Macy ML, Hall M, Shah SS, et al. Differences in designations of observation care in US freestanding children's hospitals: are they virtual or real? J Hosp Med. 2012;7:287293.
  27. Nigrovic LE, Fine AM, Monuteaux MC, Shah SS, Neuman MI. Trends in the management of viral meningitis at United States children's hospitals. Pediatrics. 2013;131:670676.
  28. Freedman SB, Hall M, Shah SS, et al. Impact of increasing ondansetron use on clinical outcomes in children with gastroenteritis. JAMA Pediatr. 2014;168:321329.
  29. Fleming‐Dutra KE, Shapiro DJ, Hicks LA, Gerber JS, Hersh AL. Race, otitis media, and antibiotic selection. Pediatrics. 2014;134:10591066.
  30. Parikh K, Hall M, Mittal V, et al. Establishing benchmarks for the hospitalized care of children with asthma, bronchiolitis, and pneumonia. Pediatrics. 2014;134:555562.
  31. Sheridan DC, Meckler GD, Spiro DM, Koch TK, Hansen ML. Diagnostic testing and treatment of pediatric headache in the emergency department. J Pediatr. 2013;163:16341637.
References
  1. Baskin MN. The prevalence of serious bacterial infections by age in febrile infants during the first 3 months of life. Pediatr Ann. 1993;22:462466.
  2. Huppler AR, Eickhoff JC, Wald ER. Performance of low‐risk criteria in the evaluation of young infants with fever: review of the literature. Pediatrics. 2010;125:228233.
  3. Schwartz S, Raveh D, Toker O, Segal G, Godovitch N, Schlesinger Y. A week‐by‐week analysis of the low‐risk criteria for serious bacterial infection in febrile neonates. Arch Dis Child. 2009;94:287292.
  4. Garcia S, Mintegi S, Gomez B, et al. Is 15 days an appropriate cut‐off age for considering serious bacterial infection in the management of febrile infants? Pediatr Infect Dis J. 2012;31:455458.
  5. Baker MD, Avner JR, Bell LM. Failure of infant observation scales in detecting serious illness in febrile, 4‐ to 8‐week‐old infants. Pediatrics. 1990;85:10401043.
  6. Baker MD, Bell LM, Avner JR. Outpatient management without antibiotics of fever in selected infants. N Engl J Med. 1993;329:14371441.
  7. Baskin MN, Fleisher GR, O'Rourke EJ. Identifying febrile infants at risk for a serious bacterial infection. J Pediatr. 1993;123:489490.
  8. Jaskiewicz JA, McCarthy CA, Richardson AC, et al. Febrile infants at low risk for serious bacterial infection—an appraisal of the Rochester criteria and implications for management. Febrile Infant Collaborative Study Group. Pediatrics. 1994;94:390396.
  9. Jain S, Cheng J, Alpern ER, et al. Management of febrile neonates in US pediatric emergency departments. Pediatrics. 2014;133:187195.
  10. Aronson PL, Thurm C, Alpern ER, et al. Variation in care of the febrile young infant <90 days in US pediatric emergency departments. Pediatrics. 2014;134:667677.
  11. Aronson PL, Thurm C, Williams DJ, et al. Association of clinical practice guidelines with emergency department management of febrile infants ≤56 days of age. J Hosp Med. 2015;10:358365.
  12. Hui C, Neto G, Tsertsvadze A, et al. Diagnosis and management of febrile infants (0‐3 months). Evid Rep Technol Assess (Full Rep). 2012;(205):1297.
  13. Tieder JS, Hall M, Auger KA, et al. Accuracy of administrative billing codes to detect urinary tract infection hospitalizations. Pediatrics. 2011;128:323330.
  14. Williams DJ, Shah SS, Myers A, et al. Identifying pediatric community‐acquired pneumonia hospitalizations: accuracy of administrative billing codes. JAMA Pediatr. 2013;167:851858.
  15. Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64:821829.
  16. American College of Emergency Physicians Clinical Policies Committee; American College of Emergency Physicians Clinical Policies Subcommittee on Pediatric Fever. Clinical policy for children younger than three years presenting to the emergency department with fever. Ann Emerg Med. 2003;42:530545.
  17. Wood JN, Feudtner C, Medina SP, Luan X, Localio R, Rubin DM. Variation in occult injury screening for children with suspected abuse in selected US children's hospitals. Pediatrics. 2012;130:853860.
  18. Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J AHIMA. 2004;75:2226.
  19. Mongelluzzo J, Mohamad Z, Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299:20482055.
  20. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
  21. Feudtner C, Hays RM, Haynes G, Geyer JR, Neff JM, Koepsell TD. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics. 2001;107:E99.
  22. US Census Bureau. Geographic terms and concepts—census divisions and census regions. Available at: https://www.census.gov/geo/reference/gtc/gtc_census_divreg.html. Accessed October 20, 2014.
  23. Gordon JA, An LC, Hayward RA, Williams BC. Initial emergency department diagnosis and return visits: risk versus perception. Ann Emerg Med. 1998;32:569573.
  24. Cho CS, Shapiro DJ, Cabana MD, Maselli JH, Hersh AL. A national depiction of children with return visits to the emergency department within 72 hours, 2001–2007. Pediatr Emerg Care. 2012;28:606610.
  25. Macy ML, Hall M, Shah SS, et al. Pediatric observation status: are we overlooking a growing population in children's hospitals? J Hosp Med. 2012;7:530536.
  26. Macy ML, Hall M, Shah SS, et al. Differences in designations of observation care in US freestanding children's hospitals: are they virtual or real? J Hosp Med. 2012;7:287293.
  27. Nigrovic LE, Fine AM, Monuteaux MC, Shah SS, Neuman MI. Trends in the management of viral meningitis at United States children's hospitals. Pediatrics. 2013;131:670676.
  28. Freedman SB, Hall M, Shah SS, et al. Impact of increasing ondansetron use on clinical outcomes in children with gastroenteritis. JAMA Pediatr. 2014;168:321329.
  29. Fleming‐Dutra KE, Shapiro DJ, Hicks LA, Gerber JS, Hersh AL. Race, otitis media, and antibiotic selection. Pediatrics. 2014;134:10591066.
  30. Parikh K, Hall M, Mittal V, et al. Establishing benchmarks for the hospitalized care of children with asthma, bronchiolitis, and pneumonia. Pediatrics. 2014;134:555562.
  31. Sheridan DC, Meckler GD, Spiro DM, Koch TK, Hansen ML. Diagnostic testing and treatment of pediatric headache in the emergency department. J Pediatr. 2013;163:16341637.
Issue
Journal of Hospital Medicine - 10(12)
Issue
Journal of Hospital Medicine - 10(12)
Page Number
787-793
Page Number
787-793
Publications
Publications
Article Type
Display Headline
Accuracy of diagnosis codes to identify febrile young infants using administrative data
Display Headline
Accuracy of diagnosis codes to identify febrile young infants using administrative data
Sections
Article Source

© 2015 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Paul L. Aronson, MD, Section of Pediatric Emergency Medicine, Yale School of Medicine, 100 York Street, Suite 1F, New Haven, CT, 06511; Telephone: 203‐737‐7443; Fax: 203‐737‐7447; E‐mail: paul.aronson@yale.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files

Febrile Infant CPGs

Article Type
Changed
Sun, 05/21/2017 - 13:08
Display Headline
Association of clinical practice guidelines with emergency department management of febrile infants ≤56 days of age

Febrile young infants are at high risk for serious bacterial infection (SBI) with reported rates of 8.5% to 12%, even higher in neonates 28 days of age.[1, 2, 3] As a result, febrile infants often undergo extensive diagnostic evaluation consisting of a combination of urine, blood, and cerebrospinal fluid (CSF) testing.[4, 5, 6] Several clinical prediction algorithms use this diagnostic testing to identify febrile infants at low risk for SBI, but they differ with respect to age range, recommended testing, antibiotic administration, and threshold for hospitalization.[4, 5, 6] Additionally, the optimal management strategy for this population has not been defined.[7] Consequently, laboratory testing, antibiotic use, and hospitalization for febrile young infants vary widely among hospitals.[8, 9, 10]

Clinical practice guidelines (CPGs) are designed to implement evidence‐based care and reduce practice variability, with the goal of improving quality of care and optimizing costs.[11] Implementation of a CPG for management of febrile young infants in the Intermountain Healthcare System was associated with greater adherence to evidence‐based care and lower costs.[12] However, when strong evidence is lacking, different interpretations of febrile infant risk classification incorporated into local CPGs may be a major driver of the across‐hospital practice variation observed in prior studies.[8, 9] Understanding sources of variability as well as determining the association of CPGs with clinicians' practice patterns can help identify quality improvement opportunities, either through national benchmarking or local efforts.

Our primary objectives were to compare (1) recommendations of pediatric emergency departmentbased institutional CPGs for febrile young infants and (2) rates of urine, blood, CSF testing, hospitalization, and ceftriaxone use at emergency department (ED) discharge based upon CPG presence and the specific CPG recommendations. Our secondary objectives were to describe the association of CPGs with healthcare costs and return visits for SBI.

METHODS

Study Design

We used the Pediatric Health Information System (PHIS) to identify febrile infants 56 days of age who presented to the ED between January 1, 2013 and December 31, 2013. We also surveyed ED providers at participating PHIS hospitals. Informed consent was obtained from survey respondents. The institutional review board at Boston Children's Hospital approved the study protocol.

Clinical Practice Guideline Survey

We sent an electronic survey to medical directors or division directors at 37 pediatric EDs to determine whether their ED utilized a CPG for the management of the febrile young infant in 2013. If no response was received after the second attempt, we queried ED fellowship directors or other ED attending physicians at nonresponding hospitals. Survey items included the presence of a febrile young infant CPG, and if present, the year of implementation, ages targeted, and CPG content. As applicable, respondents were asked to share their CPG and/or provide the specific CPG recommendations.

We collected and managed survey data using the Research Electronic Data Capture (REDCap) electronic data capture tools hosted at Boston Children's Hospital. REDCap is a secure, Web‐based application designed to support data capture for research studies.[13]

Data Source

The PHIS database contains administrative data from 44 US children's hospitals. These hospitals, affiliated with the Children's Hospital Association, represent 85% of freestanding US children's hospitals.[14] Encrypted patient identifiers permit tracking of patients across encounters.[15] Data quality and integrity are assured jointly by the Children's Hospital Association and participating hospitals.[16] For this study, 7 hospitals were excluded due to incomplete ED data or known data‐quality issues.[17]

Patients

We identified study infants using the following International Classification of Diseases, 9th Revision (ICD‐9) admission or discharge diagnosis codes for fever as defined previously[8, 9]: 780.6, 778.4, 780.60, or 780.61. We excluded infants with a complex chronic condition[18] and those transferred from another institution, as these infants may warrant a nonstandard evaluation and/or may have incomplete data. For infants with >1 ED visit for fever during the study period, repeat visits within 3 days of an index visit were considered a revisit for the same episode of illness; visits >3 days following an index visit were considered as a new index visit.

Study Definitions

From the PHIS database, we abstracted demographic characteristics (gender, race/ethnicity), insurance status, and region where the hospital was located (using US Census categories[19]). Billing codes were used to assess whether urine, blood, and CSF testing (as defined previously[9]) were performed during the ED evaluation. To account for ED visits that spanned the midnight hour, for hospitalized patients we considered any testing or treatment occurring on the initial or second hospital day to be performed in the ED; billing code data in PHIS are based upon calendar day and do not distinguish testing performed in the ED versus inpatient setting.[8, 9] Patients billed for observation care were classified as being hospitalized.[20, 21]

We identified the presence of an SBI using ICD‐9 diagnosis codes for the following infections as described previously[9]: urinary tract infection or pyelonephritis,[22] bacteremia or sepsis, bacterial meningitis,[16] pneumonia,[23] or bacterial enteritis. To assess return visits for SBI that required inpatient management, we defined an ED revisit for an SBI as a return visit within 3 days of ED discharge[24, 25] that resulted in hospitalization with an associated ICD‐9 discharge diagnosis code for an SBI.

Hospitals charges in PHIS database were adjusted for hospital location by using the Centers for Medicare and Medicaid Services price/wage index. Costs were estimated by applying hospital‐level cost‐to‐charge ratios to charge data.[26]

Measured Exposures

The primary exposure was the presence of an ED‐based CPG for management of the febrile young infant aged 28 days and 29 to 56 days; 56 days was used as the upper age limit as all of the CPGs included infants up to this age or beyond. Six institutions utilized CPGs with different thresholds to define the age categories (eg, dichotomized at 27 or 30 days); these CPGs were classified into the aforementioned age groups to permit comparisons across standardized age groups. We classified institutions based on the presence of a CPG. To assess differences in the application of low‐risk criteria, the CPGs were further classified a priori based upon specific recommendations around laboratory testing and hospitalization, as well as ceftriaxone use for infants aged 29 to 56 days discharged from the ED. CPGs were categorized based upon whether testing, hospitalization, and ceftriaxone use were: (1) recommended for all patients, (2) recommended only if patients were classified as high risk (absence of low‐risk criteria), (3) recommended against, or (4) recommended to consider at clinician discretion.

Outcome Measures

Measured outcomes were performance of urine, blood, CSF testing, and hospitalization rate, as well as rate of ceftriaxone use for discharged infants aged 29 to 56 days, 3‐day revisits for SBI, and costs per visit, which included hospitalization costs for admitted patients.

Data Analysis

We described continuous variables using median and interquartile range or range values and categorical variables using frequencies. We compared medians using Wilcoxon rank sum and categorical variables using a [2] test. We compared rates of testing, hospitalization, ceftriaxone use, and 3‐day revisits for SBI based on the presence of a CPG, and when present, the specific CPG recommendations. Costs per visit were compared between institutions with and without CPGs and assessed separately for admitted and discharged patients. To adjust for potential confounders and clustering of patients within hospitals, we used generalized estimating equations with logistic regression to generate adjusted odd ratios (aORs) and 95% confidence intervals (CIs). Models were adjusted for geographic region, payer, race, and gender. Statistical analyses were performed by using SAS version 9.3 (SAS Institute, Cary, NC). We determined statistical significance as a 2‐tailed P value <0.05.

Febrile infants with bronchiolitis or a history of prematurity may be managed differently from full‐term febrile young infants without bronchiolitis.[6, 27] Therefore, we performed a subgroup analysis after exclusion of infants with an ICD‐9 discharge diagnosis code for bronchiolitis (466.11 and 466.19)[28] or prematurity (765).

Because our study included ED encounters in 2013, we repeated our analyses after exclusion of hospitals with CPGs implemented during the 2013 calendar year.

RESULTS

CPG by Institution

Thirty‐three (89.2%) of the 37 EDs surveyed completed the questionnaire. Overall, 21 (63.6%) of the 33 EDs had a CPG; 15 (45.5%) had a CPG for all infants 56 days of age, 5 (15.2%) had a CPG for infants 28 days only, and 1 (3.0%) had a CPG for infants 29 to 56 days but not 28 days of age (Figure 1). Seventeen EDs had an established CPG prior to 2013, and 4 hospitals implemented a CPG during the 2013 calendar year, 2 with CPGs for neonates 28 days and 2 with CPGs for both 28 days and 29 to 56 days of age. Hospitals with CPGs were more likely to be located in the Northeast and West regions of the United States and provide care to a higher proportion of non‐Hispanic white patients, as well as those with commercial insurance (Table 1).

Figure 1
Specific clinical practice guideline (CPG) recommendations for diagnostic testing, hospitalization, and ceftriaxone use at ED discharge by institution among the 21 institutions with a CPG. Urine testing is defined as urine dipstick, urinalysis, or urine culture; blood testing as complete blood count or blood culture, and cerebrospinal fluid (CSF) testing as cell count, culture, or procedure code for lumbar puncture. Abbreviations: ED, emergency department.
Characteristics of Patients in Hospitals With and Without CPGs for the Febrile Young Infant 56 Days of Age
Characteristic28 Days2956 Days
No CPG, n=996, N (%)CPG, n=2,149, N (%)P ValueNo CPG, n=2,460, N (%)CPG, n=3,772, N (%)P Value
  • NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range; UTI, urinary tract infection. *Includes UTI/pyelonephritis, bacteremia/sepsis, bacterial meningitis, pneumonia, and bacterial enteritis. Some infants had more than 1 site of infection.

Race      
Non‐Hispanic white325 (32.6)996 (46.3) 867 (35.2)1,728 (45.8) 
Non‐Hispanic black248 (24.9)381 (17.7) 593 (24.1)670 (17.8) 
Hispanic243 (24.4)531 (24.7) 655 (26.6)986 (26.1) 
Asian28 (2.8)78 (3.6) 40 (1.6)122 (3.2) 
Other Race152 (15.3)163 (7.6)<0.001305 (12.4)266 (7.1)<0.001
Gender      
Female435 (43.7)926 (43.1)0.761,067 (43.4)1,714 (45.4)0.22
Payer      
Commercial243 (24.4)738 (34.3) 554 (22.5)1,202 (31.9) 
Government664 (66.7)1,269 (59.1) 1,798 (73.1)2,342 (62.1) 
Other payer89 (8.9)142 (6.6)<0.001108 (4.4)228 (6.0)<0.001
Region      
Northeast39 (3.9)245 (11.4) 77 (3.1)572 (15.2) 
South648 (65.1)915 (42.6) 1,662 (67.6)1,462 (38.8) 
Midwest271 (27.2)462 (21.5) 506 (20.6)851 (22.6) 
West38 (3.8)527 (24.5)<0.001215 (8.7)887 (23.5)<0.001
Serious bacterial infection      
Overall*131 (13.2)242 (11.3)0.14191 (7.8)237 (6.3)0.03
UTI/pyelonephritis73 (7.3)153 (7.1) 103 (4.2)154 (4.1) 
Bacteremia/sepsis56 (5.6)91 (4.2) 78 (3.2)61 (1.6) 
Bacterial meningitis15 (1.5)15 (0.7) 4 (0.2)14 (0.4) 
Age, d, median (IQR)18 (11, 24)18 (11, 23)0.6746 (37, 53)45 (37, 53)0.11

All 20 CPGs for the febrile young infant 28 days of age recommended urine, blood, CSF testing, and hospitalization for all infants (Figure 1). Of the 16 hospitals with CPGs for febrile infants aged 29 to 56 days, all recommended urine and blood testing for all patients, except for 1 CPG, which recommended consideration of blood testing but not to obtain routinely. Hospitals varied in recommendations for CSF testing among infants aged 29 to 56 days: 8 (50%) recommended CSF testing in all patients and 8 (50%) recommended CSF testing only if the patient was high risk per defined criteria (based on history, physical examination, urine, and blood testing). In all 16 CPGs, hospitalization was recommended only for high‐risk infants. For low‐risk infants aged 2956 days being discharged from the ED, 3 hospitals recommended ceftriaxone for all, 9 recommended consideration of ceftriaxone, and 4 recommended against antibiotics (Figure 1).

Study Patients

During the study period, there were 10,415 infants 56 days old with a diagnosis of fever at the 33 participating hospitals. After exclusion of 635 (6.1%) infants with a complex chronic condition and 445 (4.3%) transferred from another institution (including 42 with a complex chronic condition), 9377 infants remained in our study cohort. Approximately one‐third of the cohort was 28 days of age and two‐thirds aged 29 to 56 days. The overall SBI rate was 8.5% but varied by age (11.9% in infants 28 days and 6.9% in infants 29 to 56 days of age) (Table 1).

CPGs and Use of Diagnostic Testing, Hospitalization Rates, Ceftriaxone Use, and Revisits for SBI

For infants 28 days of age, the presence of a CPG was not associated with urine, blood, CSF testing, or hospitalization after multivariable adjustment (Table 2). Among infants aged 29 to 56 days, urine testing did not differ based on the presence of a CPG, whereas blood testing was performed less often at the 1 hospital whose CPG recommended to consider, but not routinely obtain, testing (aOR: 0.4, 95% CI: 0.3‐0.7, P=0.001). Compared to hospitals without a CPG, CSF testing was performed less often at hospitals with CPG recommendations to only obtain CSF if high risk (aOR: 0.5, 95% CI: 0.3‐0.8, P=0.002). However, the odds of hospitalization did not differ at institutions with and without a febrile infant CPG (aOR: 0.7, 95% CI: 0.5‐1.1, P=0.10). For infants aged 29 to 56 days discharged from the ED, ceftriaxone was administered more often at hospitals with CPGs that recommended ceftriaxone for all discharged patients (aOR: 4.6, 95% CI: 2.39.3, P<0.001) and less often at hospitals whose CPGs recommended against antibiotics (aOR: 0.3, 95% CI: 0.1‐0.9, P=0.03) (Table 3). Our findings were similar in the subgroup of infants without bronchiolitis or prematurity (see Supporting Tables 1 and 2 in the online version of this article). After exclusion of hospitals with a CPG implemented during the 2013 calendar year (4 hospitals excluded in the 28 days age group and 2 hospitals excluded in the 29 to 56 days age group), infants aged 29 to 56 days cared for at a hospital with a CPG experienced a lower odds of hospitalization (aOR: 0.7, 95% CI: 0.4‐0.98, P=0.04). Otherwise, our findings in both age groups did not materially differ from the main analyses.

Variation in Testing and Hospitalization Based on CPG‐Specific Recommendations Among Infants 28 Days of Age With Diagnosis of Fever
Testing/HospitalizationNo. of HospitalsNo. of Patients% Received*aOR (95% CI)P Value
  • NOTE: Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; CPG, clinical practice guideline; CSF, cerebrospinal fluid. *Percent of infants who received test or were hospitalized. Adjusted for hospital clustering, geographic region, payer, race, and gender. Urine testing defined as urine dipstick, urinalysis, or urine culture; Blood testing defined as complete blood count or blood culture. ‖CSF testing defined as cell count, culture, or procedure code for lumbar puncture

Laboratory testing     
Urine testing     
No CPG1399675.6Ref 
CPG: recommend for all202,14980.71.2 (0.9‐1.7)0.22
Blood testing     
No CPG1399676.9Ref 
CPG: recommend for all202,14981.81.2 (0.9‐1.7)0.25
CSF testing     
No CPG1399671.0Ref 
CPG: recommend for all202,14977.51.3 (1.01.7)0.08
Disposition     
Hospitalization     
No CPG1399675.4Ref 
CPG: recommend for all202,14981.61.2 (0.9‐1.8)0.26
Variation in Testing, Hospitalization, and Ceftriaxone Use Based on CPG‐Specific Recommendations Among Infants 29 to 56 Days of Age With Diagnosis of Fever
Testing/HospitalizationNo. of HospitalsNo. of Patients% Received*aOR (95% CI)P Value
  • NOTE: Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; CPG, clinical practice guideline; CSF, cerebrospinal fluid. *Percent of infants who received test, were hospitalized, or received ceftriaxone. Adjusted for hospital clustering, geographic region, payer, race, and gender. Urine testing defined as urine dipstick, urinalysis, or urine culture. Blood testing defined as complete blood count or blood culture. CSF testing defined as cell count, culture, or procedure code for lumbar puncture. For low‐risk infants discharged from the emergency department.

Laboratory resting     
Urine testing     
No CPG172,46081.1Ref 
CPG: recommend for all163,77282.10.9 (0.7‐1.4)0.76
Blood testing     
No CPG172,46079.4Ref 
CPG: recommend for all153,62882.61.1 (0.7‐1.6)0.70
CPG: recommend consider114462.50.4 (0.3‐0.7)0.001
CSF testing     
No CPG172,46046.3Ref 
CPG: recommend for all81,51770.31.3 (0.9‐1.9)0.11
CPG: recommend if high‐risk82,25539.90.5 (0.3‐0.8)0.002
Disposition     
Hospitalization     
No CPG172,46047.0Ref 
CPG: recommend if high‐risk163,77242.00.7 (0.5‐1.1)0.10
Ceftriaxone if discharged     
No CPG171,30411.7Ref 
CPG: recommend against431310.90.3 (0.1‐0.9)0.03
CPG: recommend consider91,56714.41.5 (0.9‐2.4)0.09
CPG: recommend for all330664.14.6 (2.39.3)< 0.001

Three‐day revisits for SBI were similarly low at hospitals with and without CPGs among infants 28 days (1.5% vs 0.8%, P=0.44) and 29 to 56 days of age (1.4% vs 1.1%, P=0.44) and did not differ after exclusion of hospitals with a CPG implemented in 2013.

CPGs and Costs

Among infants 28 days of age, costs per visit did not differ for admitted and discharged patients based on CPG presence. The presence of an ED febrile infant CPG was associated with higher costs for both admitted and discharged infants 29 to 56 days of age (Table 4). The cost analysis did not significantly differ after exclusion of hospitals with CPGs implemented in 2013.

Costs per Visit for Febrile Young Infants 56 Days of Age at Institutions With and Without CPGs
 28 Days, Cost, Median (IQR)29 to 56 Days, Cost, Median (IQR)
No CPGCPGP ValueNo CPGCPGP Value
  • NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range.

Admitted$4,979 ($3,408$6,607) [n=751]$4,715 ($3,472$6,526) [n=1,753]0.79$3,756 ($2,725$5,041) [n=1,156]$3,923 ($3,077$5,243) [n=1,586]<0.001
Discharged$298 ($166$510) [n=245]$231 ($160$464) [n=396]0.10$681($398$982) [n=1,304)]$764 ($412$1,100) [n=2,186]<0.001

DISCUSSION

We described the content and association of CPGs with management of the febrile infant 56 days of age across a large sample of children's hospitals. Nearly two‐thirds of included pediatric EDs have a CPG for the management of young febrile infants. Management of febrile infants 28 days was uniform, with a majority hospitalized after urine, blood, and CSF testing regardless of the presence of a CPG. In contrast, CPGs for infants 29 to 56 days of age varied in their recommendations for CSF testing as well as ceftriaxone use for infants discharged from the ED. Consequently, we observed considerable hospital variability in CSF testing and ceftriaxone use for discharged infants, which correlates with variation in the presence and content of CPGs. Institutional CPGs may be a source of the across‐hospital variation in care of febrile young infants observed in prior study.[9]

Febrile infants 28 days of age are at particularly high risk for SBI, with a prevalence of nearly 20% or higher.[2, 3, 29] The high prevalence of SBI, combined with the inherent difficulty in distinguishing neonates with and without SBI,[2, 30] has resulted in uniform CPG recommendations to perform the full‐sepsis workup in this young age group. Similar to prior studies,[8, 9] we observed that most febrile infants 28 days undergo the full sepsis evaluation, including CSF testing, and are hospitalized regardless of the presence of a CPG.

However, given the conflicting recommendations for febrile infants 29 to 56 days of age,[4, 5, 6] the optimal management strategy is less certain.[7] The Rochester, Philadelphia, and Boston criteria, 3 published models to identify infants at low risk for SBI, primarily differ in their recommendations for CSF testing and ceftriaxone use in this age group.[4, 5, 6] Half of the CPGs recommended CSF testing for all febrile infants, and half recommended CSF testing only if the infant was high risk. Institutional guidelines that recommended selective CSF testing for febrile infants aged 29 to 56 days were associated with lower rates of CSF testing. Furthermore, ceftriaxone use varied based on CPG recommendations for low‐risk infants discharged from the ED. Therefore, the influence of febrile infant CPGs mainly relates to the limiting of CSF testing and targeted ceftriaxone use in low‐risk infants. As the rate of return visits for SBI is low across hospitals, future study should assess outcomes at hospitals with CPGs recommending selective CSF testing. Of note, infants 29 to 56 days of age were less likely to be hospitalized when cared for at a hospital with an established CPG prior to 2013 without increase in 3‐day revisits for SBI. This finding may indicate that longer duration of CPG implementation is associated with lower rates of hospitalization for low‐risk infants; this finding merits further study.

The presence of a CPG was not associated with lower costs for febrile infants in either age group. Although individual healthcare systems have achieved lower costs with CPG implementation,[12] the mere presence of a CPG is not associated with lower costs when assessed across institutions. Higher costs for admitted and discharged infants 29 to 56 days of age in the presence of a CPG likely reflects the higher rate of CSF testing at hospitals whose CPGs recommend testing for all febrile infants, as well as inpatient management strategies for hospitalized infants not captured in our study. Future investigation should include an assessment of the cost‐effectiveness of the various testing and treatment strategies employed for the febrile young infant.

Our study has several limitations. First, the validity of ICD‐9 diagnosis codes for identifying young infants with fever is not well established, and thus our study is subject to misclassification bias. To minimize missed patients, we included infants with either an ICD‐9 admission or discharge diagnosis of fever; however, utilization of diagnosis codes for patient identification may have resulted in undercapture of infants with a measured temperature of 38.0C. It is also possible that some patients who did not undergo testing were misclassified as having a fever or had temperatures below standard thresholds to prompt diagnostic testing. This is a potential reason that testing was not performed in 100% of infants, even at hospitals with CPGs that recommended testing for all patients. Additionally, some febrile infants diagnosed with SBI may not have an associated ICD‐9 diagnosis code for fever. Although the overall SBI rate observed in our study was similar to prior studies,[4, 31] the rate in neonates 28 days of age was lower than reported in recent investigations,[2, 3] which may indicate inclusion of a higher proportion of low‐risk febrile infants. With the exception of bronchiolitis, we also did not assess diagnostic testing in the presence of other identified sources of infection such as herpes simplex virus.

Second, we were unable to assess the presence or absence of a CPG at the 4 excluded EDs that did not respond to the survey or the institutions excluded for data‐quality issues. However, included and excluded hospitals did not differ in region or annual ED volume (data not shown).

Third, although we classified hospitals based upon the presence and content of CPGs, we were unable to fully evaluate adherence to the CPG at each site.

Last, though PHIS hospitals represent 85% of freestanding children's hospitals, many febrile infants are hospitalized at non‐PHIS institutions; our results may not be generalizable to care provided at nonchildren's hospitals.

CONCLUSIONS

Management of febrile neonates 28 days of age does not vary based on CPG presence. However, CPGs for the febrile infant aged 29 to 56 days vary in recommendations for CSF testing as well as ceftriaxone use for low‐risk patients, which significantly contributes to practice variation and healthcare costs across institutions.

Acknowledgements

The Febrile Young Infant Research Collaborative includes the following additional investigators who are acknowledged for their work on this study: Kao‐Ping Chua, MD, Harvard PhD Program in Health Policy, Harvard University, Cambridge, Massachusetts, and Division of Emergency Medicine, Department of Pediatrics, Boston Children's Hospital, Boston, Massachusetts; Elana A. Feldman, BA, University of Washington School of Medicine, Seattle, Washington; and Katie L. Hayes, BS, Division of Emergency Medicine, Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.

Disclosures

This project was funded in part by The Gerber Foundation Novice Researcher Award (Ref #18273835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript. The authors have no financial relationships relevant to this article to disclose. No payment was received for the production of this article. The authors have no conflicts of interest relevant to this article to disclose.

Files
References
  1. Huppler AR, Eickhoff JC, Wald ER. Performance of low‐risk criteria in the evaluation of young infants with fever: review of the literature. Pediatrics. 2010;125:228233.
  2. Schwartz S, Raveh D, Toker O, Segal G, Godovitch N, Schlesinger Y. A week‐by‐week analysis of the low‐risk criteria for serious bacterial infection in febrile neonates. Arch Dis Child. 2009;94:287292.
  3. Garcia S, Mintegi S, Gomez B, et al. Is 15 days an appropriate cut‐off age for considering serious bacterial infection in the management of febrile infants? Pediatr Infect Dis J. 2012;31:455458.
  4. Baker MD, Bell LM, Avner JR. Outpatient management without antibiotics of fever in selected infants. N Engl J Med. 1993;329:14371441.
  5. Baskin MN, Fleisher GR, O'Rourke EJ. Identifying febrile infants at risk for a serious bacterial infection. J Pediatr. 1993;123:489490.
  6. Jaskiewicz JA, McCarthy CA, Richardson AC, et al. Febrile infants at low risk for serious bacterial infection—an appraisal of the Rochester criteria and implications for management. Febrile Infant Collaborative Study Group. Pediatrics. 1994;94:390396.
  7. American College of Emergency Physicians Clinical Policies Committee; American College of Emergency Physicians Clinical Policies Subcommittee on Pediatric Fever. Clinical policy for children younger than three years presenting to the emergency department with fever. Ann Emerg Med. 2003;42:530545.
  8. Jain S, Cheng J, Alpern ER, et al. Management of febrile neonates in US pediatric emergency departments. Pediatrics. 2014;133:187195.
  9. Aronson PL, Thurm C, Alpern ER, et al. Variation in care of the febrile young infant <90 days in US pediatric emergency departments. Pediatrics. 2014;134:667677.
  10. Yarden‐Bilavsky H, Ashkenazi S, Amir J, Schlesinger Y, Bilavsky E. Fever survey highlights significant variations in how infants aged ≤60 days are evaluated and underline the need for guidelines. Acta Paediatr. 2014;103:379385.
  11. Bergman DA. Evidence‐based guidelines and critical pathways for quality improvement. Pediatrics. 1999;103:225232.
  12. Byington CL, Reynolds CC, Korgenski K, et al. Costs and infant outcomes after implementation of a care process model for febrile infants. Pediatrics. 2012;130:e16e24.
  13. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
  14. Wood JN, Feudtner C, Medina SP, Luan X, Localio R, Rubin DM. Variation in occult injury screening for children with suspected abuse in selected US children's hospitals. Pediatrics. 2012;130:853860.
  15. Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J AHIMA. 2004;75:2226.
  16. Mongelluzzo J, Mohamad Z, Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299:20482055.
  17. Kharbanda AB, Hall M, Shah SS, et al. Variation in resource utilization across a national sample of pediatric emergency departments. J Pediatr. 2013;163:230236.
  18. Feudtner C, Hays RM, Haynes G, Geyer JR, Neff JM, Koepsell TD. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics. 2001;107:E99.
  19. US Census Bureau. Geographic terms and concepts—census divisions and census regions. Available at: https://www.census.gov/geo/reference/gtc/gtc_census_divreg.html. Accessed September 10, 2014.
  20. Macy ML, Hall M, Shah SS, et al. Pediatric observation status: are we overlooking a growing population in children's hospitals? J Hosp Med. 2012;7:530536.
  21. Macy ML, Hall M, Shah SS, et al. Differences in designations of observation care in US freestanding children's hospitals: are they virtual or real? J Hosp Med. 2012;7:287293.
  22. Tieder JS, Hall M, Auger KA, et al. Accuracy of administrative billing codes to detect urinary tract infection hospitalizations. Pediatrics. 2011;128:323330.
  23. Williams DJ, Shah SS, Myers A, et al. Identifying pediatric community‐acquired pneumonia hospitalizations: accuracy of administrative billing codes. JAMA Pediatr. 2013;167:851858.
  24. Gordon JA, An LC, Hayward RA, Williams BC. Initial emergency department diagnosis and return visits: risk versus perception. Ann Emerg Med. 1998;32:569573.
  25. Cho CS, Shapiro DJ, Cabana MD, Maselli JH, Hersh AL. A national depiction of children with return visits to the emergency department within 72 hours, 2001–2007. Pediatr Emerg Care. 2012;28:606610.
  26. Healthcare Cost and Utilization Project. Cost‐to‐charge ratio files. Available at: http://www.hcup‐us.ahrq.gov/db/state/costtocharge.jsp. Accessed September 11, 2014.
  27. Levine DA, Platt SL, Dayan PS, et al. Risk of serious bacterial infection in young febrile infants with respiratory syncytial virus infections. Pediatrics. 2004;113:17281734.
  28. Parikh K, Hall M, Mittal V, et al. Establishing benchmarks for the hospitalized care of children with asthma, bronchiolitis, and pneumonia. Pediatrics. 2014;134:555562.
  29. Mintegi S, Benito J, Astobiza E, Capape S, Gomez B, Eguireun A. Well appearing young infants with fever without known source in the emergency department: are lumbar punctures always necessary? Eur J Emerg Med. 2010;17:167169.
  30. Baker MD, Bell LM. Unpredictability of serious bacterial illness in febrile infants from birth to 1 month of age. Arch Pediatr Adolesc Med. 1999;153:508511.
  31. Pantell RH, Newman TB, Bernzweig J, et al. Management and outcomes of care of fever in early infancy. JAMA. 2004;291:12031212.
Article PDF
Issue
Journal of Hospital Medicine - 10(6)
Publications
Page Number
358-365
Sections
Files
Files
Article PDF
Article PDF

Febrile young infants are at high risk for serious bacterial infection (SBI) with reported rates of 8.5% to 12%, even higher in neonates 28 days of age.[1, 2, 3] As a result, febrile infants often undergo extensive diagnostic evaluation consisting of a combination of urine, blood, and cerebrospinal fluid (CSF) testing.[4, 5, 6] Several clinical prediction algorithms use this diagnostic testing to identify febrile infants at low risk for SBI, but they differ with respect to age range, recommended testing, antibiotic administration, and threshold for hospitalization.[4, 5, 6] Additionally, the optimal management strategy for this population has not been defined.[7] Consequently, laboratory testing, antibiotic use, and hospitalization for febrile young infants vary widely among hospitals.[8, 9, 10]

Clinical practice guidelines (CPGs) are designed to implement evidence‐based care and reduce practice variability, with the goal of improving quality of care and optimizing costs.[11] Implementation of a CPG for management of febrile young infants in the Intermountain Healthcare System was associated with greater adherence to evidence‐based care and lower costs.[12] However, when strong evidence is lacking, different interpretations of febrile infant risk classification incorporated into local CPGs may be a major driver of the across‐hospital practice variation observed in prior studies.[8, 9] Understanding sources of variability as well as determining the association of CPGs with clinicians' practice patterns can help identify quality improvement opportunities, either through national benchmarking or local efforts.

Our primary objectives were to compare (1) recommendations of pediatric emergency departmentbased institutional CPGs for febrile young infants and (2) rates of urine, blood, CSF testing, hospitalization, and ceftriaxone use at emergency department (ED) discharge based upon CPG presence and the specific CPG recommendations. Our secondary objectives were to describe the association of CPGs with healthcare costs and return visits for SBI.

METHODS

Study Design

We used the Pediatric Health Information System (PHIS) to identify febrile infants 56 days of age who presented to the ED between January 1, 2013 and December 31, 2013. We also surveyed ED providers at participating PHIS hospitals. Informed consent was obtained from survey respondents. The institutional review board at Boston Children's Hospital approved the study protocol.

Clinical Practice Guideline Survey

We sent an electronic survey to medical directors or division directors at 37 pediatric EDs to determine whether their ED utilized a CPG for the management of the febrile young infant in 2013. If no response was received after the second attempt, we queried ED fellowship directors or other ED attending physicians at nonresponding hospitals. Survey items included the presence of a febrile young infant CPG, and if present, the year of implementation, ages targeted, and CPG content. As applicable, respondents were asked to share their CPG and/or provide the specific CPG recommendations.

We collected and managed survey data using the Research Electronic Data Capture (REDCap) electronic data capture tools hosted at Boston Children's Hospital. REDCap is a secure, Web‐based application designed to support data capture for research studies.[13]

Data Source

The PHIS database contains administrative data from 44 US children's hospitals. These hospitals, affiliated with the Children's Hospital Association, represent 85% of freestanding US children's hospitals.[14] Encrypted patient identifiers permit tracking of patients across encounters.[15] Data quality and integrity are assured jointly by the Children's Hospital Association and participating hospitals.[16] For this study, 7 hospitals were excluded due to incomplete ED data or known data‐quality issues.[17]

Patients

We identified study infants using the following International Classification of Diseases, 9th Revision (ICD‐9) admission or discharge diagnosis codes for fever as defined previously[8, 9]: 780.6, 778.4, 780.60, or 780.61. We excluded infants with a complex chronic condition[18] and those transferred from another institution, as these infants may warrant a nonstandard evaluation and/or may have incomplete data. For infants with >1 ED visit for fever during the study period, repeat visits within 3 days of an index visit were considered a revisit for the same episode of illness; visits >3 days following an index visit were considered as a new index visit.

Study Definitions

From the PHIS database, we abstracted demographic characteristics (gender, race/ethnicity), insurance status, and region where the hospital was located (using US Census categories[19]). Billing codes were used to assess whether urine, blood, and CSF testing (as defined previously[9]) were performed during the ED evaluation. To account for ED visits that spanned the midnight hour, for hospitalized patients we considered any testing or treatment occurring on the initial or second hospital day to be performed in the ED; billing code data in PHIS are based upon calendar day and do not distinguish testing performed in the ED versus inpatient setting.[8, 9] Patients billed for observation care were classified as being hospitalized.[20, 21]

We identified the presence of an SBI using ICD‐9 diagnosis codes for the following infections as described previously[9]: urinary tract infection or pyelonephritis,[22] bacteremia or sepsis, bacterial meningitis,[16] pneumonia,[23] or bacterial enteritis. To assess return visits for SBI that required inpatient management, we defined an ED revisit for an SBI as a return visit within 3 days of ED discharge[24, 25] that resulted in hospitalization with an associated ICD‐9 discharge diagnosis code for an SBI.

Hospitals charges in PHIS database were adjusted for hospital location by using the Centers for Medicare and Medicaid Services price/wage index. Costs were estimated by applying hospital‐level cost‐to‐charge ratios to charge data.[26]

Measured Exposures

The primary exposure was the presence of an ED‐based CPG for management of the febrile young infant aged 28 days and 29 to 56 days; 56 days was used as the upper age limit as all of the CPGs included infants up to this age or beyond. Six institutions utilized CPGs with different thresholds to define the age categories (eg, dichotomized at 27 or 30 days); these CPGs were classified into the aforementioned age groups to permit comparisons across standardized age groups. We classified institutions based on the presence of a CPG. To assess differences in the application of low‐risk criteria, the CPGs were further classified a priori based upon specific recommendations around laboratory testing and hospitalization, as well as ceftriaxone use for infants aged 29 to 56 days discharged from the ED. CPGs were categorized based upon whether testing, hospitalization, and ceftriaxone use were: (1) recommended for all patients, (2) recommended only if patients were classified as high risk (absence of low‐risk criteria), (3) recommended against, or (4) recommended to consider at clinician discretion.

Outcome Measures

Measured outcomes were performance of urine, blood, CSF testing, and hospitalization rate, as well as rate of ceftriaxone use for discharged infants aged 29 to 56 days, 3‐day revisits for SBI, and costs per visit, which included hospitalization costs for admitted patients.

Data Analysis

We described continuous variables using median and interquartile range or range values and categorical variables using frequencies. We compared medians using Wilcoxon rank sum and categorical variables using a [2] test. We compared rates of testing, hospitalization, ceftriaxone use, and 3‐day revisits for SBI based on the presence of a CPG, and when present, the specific CPG recommendations. Costs per visit were compared between institutions with and without CPGs and assessed separately for admitted and discharged patients. To adjust for potential confounders and clustering of patients within hospitals, we used generalized estimating equations with logistic regression to generate adjusted odd ratios (aORs) and 95% confidence intervals (CIs). Models were adjusted for geographic region, payer, race, and gender. Statistical analyses were performed by using SAS version 9.3 (SAS Institute, Cary, NC). We determined statistical significance as a 2‐tailed P value <0.05.

Febrile infants with bronchiolitis or a history of prematurity may be managed differently from full‐term febrile young infants without bronchiolitis.[6, 27] Therefore, we performed a subgroup analysis after exclusion of infants with an ICD‐9 discharge diagnosis code for bronchiolitis (466.11 and 466.19)[28] or prematurity (765).

Because our study included ED encounters in 2013, we repeated our analyses after exclusion of hospitals with CPGs implemented during the 2013 calendar year.

RESULTS

CPG by Institution

Thirty‐three (89.2%) of the 37 EDs surveyed completed the questionnaire. Overall, 21 (63.6%) of the 33 EDs had a CPG; 15 (45.5%) had a CPG for all infants 56 days of age, 5 (15.2%) had a CPG for infants 28 days only, and 1 (3.0%) had a CPG for infants 29 to 56 days but not 28 days of age (Figure 1). Seventeen EDs had an established CPG prior to 2013, and 4 hospitals implemented a CPG during the 2013 calendar year, 2 with CPGs for neonates 28 days and 2 with CPGs for both 28 days and 29 to 56 days of age. Hospitals with CPGs were more likely to be located in the Northeast and West regions of the United States and provide care to a higher proportion of non‐Hispanic white patients, as well as those with commercial insurance (Table 1).

Figure 1
Specific clinical practice guideline (CPG) recommendations for diagnostic testing, hospitalization, and ceftriaxone use at ED discharge by institution among the 21 institutions with a CPG. Urine testing is defined as urine dipstick, urinalysis, or urine culture; blood testing as complete blood count or blood culture, and cerebrospinal fluid (CSF) testing as cell count, culture, or procedure code for lumbar puncture. Abbreviations: ED, emergency department.
Characteristics of Patients in Hospitals With and Without CPGs for the Febrile Young Infant 56 Days of Age
Characteristic28 Days2956 Days
No CPG, n=996, N (%)CPG, n=2,149, N (%)P ValueNo CPG, n=2,460, N (%)CPG, n=3,772, N (%)P Value
  • NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range; UTI, urinary tract infection. *Includes UTI/pyelonephritis, bacteremia/sepsis, bacterial meningitis, pneumonia, and bacterial enteritis. Some infants had more than 1 site of infection.

Race      
Non‐Hispanic white325 (32.6)996 (46.3) 867 (35.2)1,728 (45.8) 
Non‐Hispanic black248 (24.9)381 (17.7) 593 (24.1)670 (17.8) 
Hispanic243 (24.4)531 (24.7) 655 (26.6)986 (26.1) 
Asian28 (2.8)78 (3.6) 40 (1.6)122 (3.2) 
Other Race152 (15.3)163 (7.6)<0.001305 (12.4)266 (7.1)<0.001
Gender      
Female435 (43.7)926 (43.1)0.761,067 (43.4)1,714 (45.4)0.22
Payer      
Commercial243 (24.4)738 (34.3) 554 (22.5)1,202 (31.9) 
Government664 (66.7)1,269 (59.1) 1,798 (73.1)2,342 (62.1) 
Other payer89 (8.9)142 (6.6)<0.001108 (4.4)228 (6.0)<0.001
Region      
Northeast39 (3.9)245 (11.4) 77 (3.1)572 (15.2) 
South648 (65.1)915 (42.6) 1,662 (67.6)1,462 (38.8) 
Midwest271 (27.2)462 (21.5) 506 (20.6)851 (22.6) 
West38 (3.8)527 (24.5)<0.001215 (8.7)887 (23.5)<0.001
Serious bacterial infection      
Overall*131 (13.2)242 (11.3)0.14191 (7.8)237 (6.3)0.03
UTI/pyelonephritis73 (7.3)153 (7.1) 103 (4.2)154 (4.1) 
Bacteremia/sepsis56 (5.6)91 (4.2) 78 (3.2)61 (1.6) 
Bacterial meningitis15 (1.5)15 (0.7) 4 (0.2)14 (0.4) 
Age, d, median (IQR)18 (11, 24)18 (11, 23)0.6746 (37, 53)45 (37, 53)0.11

All 20 CPGs for the febrile young infant 28 days of age recommended urine, blood, CSF testing, and hospitalization for all infants (Figure 1). Of the 16 hospitals with CPGs for febrile infants aged 29 to 56 days, all recommended urine and blood testing for all patients, except for 1 CPG, which recommended consideration of blood testing but not to obtain routinely. Hospitals varied in recommendations for CSF testing among infants aged 29 to 56 days: 8 (50%) recommended CSF testing in all patients and 8 (50%) recommended CSF testing only if the patient was high risk per defined criteria (based on history, physical examination, urine, and blood testing). In all 16 CPGs, hospitalization was recommended only for high‐risk infants. For low‐risk infants aged 2956 days being discharged from the ED, 3 hospitals recommended ceftriaxone for all, 9 recommended consideration of ceftriaxone, and 4 recommended against antibiotics (Figure 1).

Study Patients

During the study period, there were 10,415 infants 56 days old with a diagnosis of fever at the 33 participating hospitals. After exclusion of 635 (6.1%) infants with a complex chronic condition and 445 (4.3%) transferred from another institution (including 42 with a complex chronic condition), 9377 infants remained in our study cohort. Approximately one‐third of the cohort was 28 days of age and two‐thirds aged 29 to 56 days. The overall SBI rate was 8.5% but varied by age (11.9% in infants 28 days and 6.9% in infants 29 to 56 days of age) (Table 1).

CPGs and Use of Diagnostic Testing, Hospitalization Rates, Ceftriaxone Use, and Revisits for SBI

For infants 28 days of age, the presence of a CPG was not associated with urine, blood, CSF testing, or hospitalization after multivariable adjustment (Table 2). Among infants aged 29 to 56 days, urine testing did not differ based on the presence of a CPG, whereas blood testing was performed less often at the 1 hospital whose CPG recommended to consider, but not routinely obtain, testing (aOR: 0.4, 95% CI: 0.3‐0.7, P=0.001). Compared to hospitals without a CPG, CSF testing was performed less often at hospitals with CPG recommendations to only obtain CSF if high risk (aOR: 0.5, 95% CI: 0.3‐0.8, P=0.002). However, the odds of hospitalization did not differ at institutions with and without a febrile infant CPG (aOR: 0.7, 95% CI: 0.5‐1.1, P=0.10). For infants aged 29 to 56 days discharged from the ED, ceftriaxone was administered more often at hospitals with CPGs that recommended ceftriaxone for all discharged patients (aOR: 4.6, 95% CI: 2.39.3, P<0.001) and less often at hospitals whose CPGs recommended against antibiotics (aOR: 0.3, 95% CI: 0.1‐0.9, P=0.03) (Table 3). Our findings were similar in the subgroup of infants without bronchiolitis or prematurity (see Supporting Tables 1 and 2 in the online version of this article). After exclusion of hospitals with a CPG implemented during the 2013 calendar year (4 hospitals excluded in the 28 days age group and 2 hospitals excluded in the 29 to 56 days age group), infants aged 29 to 56 days cared for at a hospital with a CPG experienced a lower odds of hospitalization (aOR: 0.7, 95% CI: 0.4‐0.98, P=0.04). Otherwise, our findings in both age groups did not materially differ from the main analyses.

Variation in Testing and Hospitalization Based on CPG‐Specific Recommendations Among Infants 28 Days of Age With Diagnosis of Fever
Testing/HospitalizationNo. of HospitalsNo. of Patients% Received*aOR (95% CI)P Value
  • NOTE: Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; CPG, clinical practice guideline; CSF, cerebrospinal fluid. *Percent of infants who received test or were hospitalized. Adjusted for hospital clustering, geographic region, payer, race, and gender. Urine testing defined as urine dipstick, urinalysis, or urine culture; Blood testing defined as complete blood count or blood culture. ‖CSF testing defined as cell count, culture, or procedure code for lumbar puncture

Laboratory testing     
Urine testing     
No CPG1399675.6Ref 
CPG: recommend for all202,14980.71.2 (0.9‐1.7)0.22
Blood testing     
No CPG1399676.9Ref 
CPG: recommend for all202,14981.81.2 (0.9‐1.7)0.25
CSF testing     
No CPG1399671.0Ref 
CPG: recommend for all202,14977.51.3 (1.01.7)0.08
Disposition     
Hospitalization     
No CPG1399675.4Ref 
CPG: recommend for all202,14981.61.2 (0.9‐1.8)0.26
Variation in Testing, Hospitalization, and Ceftriaxone Use Based on CPG‐Specific Recommendations Among Infants 29 to 56 Days of Age With Diagnosis of Fever
Testing/HospitalizationNo. of HospitalsNo. of Patients% Received*aOR (95% CI)P Value
  • NOTE: Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; CPG, clinical practice guideline; CSF, cerebrospinal fluid. *Percent of infants who received test, were hospitalized, or received ceftriaxone. Adjusted for hospital clustering, geographic region, payer, race, and gender. Urine testing defined as urine dipstick, urinalysis, or urine culture. Blood testing defined as complete blood count or blood culture. CSF testing defined as cell count, culture, or procedure code for lumbar puncture. For low‐risk infants discharged from the emergency department.

Laboratory resting     
Urine testing     
No CPG172,46081.1Ref 
CPG: recommend for all163,77282.10.9 (0.7‐1.4)0.76
Blood testing     
No CPG172,46079.4Ref 
CPG: recommend for all153,62882.61.1 (0.7‐1.6)0.70
CPG: recommend consider114462.50.4 (0.3‐0.7)0.001
CSF testing     
No CPG172,46046.3Ref 
CPG: recommend for all81,51770.31.3 (0.9‐1.9)0.11
CPG: recommend if high‐risk82,25539.90.5 (0.3‐0.8)0.002
Disposition     
Hospitalization     
No CPG172,46047.0Ref 
CPG: recommend if high‐risk163,77242.00.7 (0.5‐1.1)0.10
Ceftriaxone if discharged     
No CPG171,30411.7Ref 
CPG: recommend against431310.90.3 (0.1‐0.9)0.03
CPG: recommend consider91,56714.41.5 (0.9‐2.4)0.09
CPG: recommend for all330664.14.6 (2.39.3)< 0.001

Three‐day revisits for SBI were similarly low at hospitals with and without CPGs among infants 28 days (1.5% vs 0.8%, P=0.44) and 29 to 56 days of age (1.4% vs 1.1%, P=0.44) and did not differ after exclusion of hospitals with a CPG implemented in 2013.

CPGs and Costs

Among infants 28 days of age, costs per visit did not differ for admitted and discharged patients based on CPG presence. The presence of an ED febrile infant CPG was associated with higher costs for both admitted and discharged infants 29 to 56 days of age (Table 4). The cost analysis did not significantly differ after exclusion of hospitals with CPGs implemented in 2013.

Costs per Visit for Febrile Young Infants 56 Days of Age at Institutions With and Without CPGs
 28 Days, Cost, Median (IQR)29 to 56 Days, Cost, Median (IQR)
No CPGCPGP ValueNo CPGCPGP Value
  • NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range.

Admitted$4,979 ($3,408$6,607) [n=751]$4,715 ($3,472$6,526) [n=1,753]0.79$3,756 ($2,725$5,041) [n=1,156]$3,923 ($3,077$5,243) [n=1,586]<0.001
Discharged$298 ($166$510) [n=245]$231 ($160$464) [n=396]0.10$681($398$982) [n=1,304)]$764 ($412$1,100) [n=2,186]<0.001

DISCUSSION

We described the content and association of CPGs with management of the febrile infant 56 days of age across a large sample of children's hospitals. Nearly two‐thirds of included pediatric EDs have a CPG for the management of young febrile infants. Management of febrile infants 28 days was uniform, with a majority hospitalized after urine, blood, and CSF testing regardless of the presence of a CPG. In contrast, CPGs for infants 29 to 56 days of age varied in their recommendations for CSF testing as well as ceftriaxone use for infants discharged from the ED. Consequently, we observed considerable hospital variability in CSF testing and ceftriaxone use for discharged infants, which correlates with variation in the presence and content of CPGs. Institutional CPGs may be a source of the across‐hospital variation in care of febrile young infants observed in prior study.[9]

Febrile infants 28 days of age are at particularly high risk for SBI, with a prevalence of nearly 20% or higher.[2, 3, 29] The high prevalence of SBI, combined with the inherent difficulty in distinguishing neonates with and without SBI,[2, 30] has resulted in uniform CPG recommendations to perform the full‐sepsis workup in this young age group. Similar to prior studies,[8, 9] we observed that most febrile infants 28 days undergo the full sepsis evaluation, including CSF testing, and are hospitalized regardless of the presence of a CPG.

However, given the conflicting recommendations for febrile infants 29 to 56 days of age,[4, 5, 6] the optimal management strategy is less certain.[7] The Rochester, Philadelphia, and Boston criteria, 3 published models to identify infants at low risk for SBI, primarily differ in their recommendations for CSF testing and ceftriaxone use in this age group.[4, 5, 6] Half of the CPGs recommended CSF testing for all febrile infants, and half recommended CSF testing only if the infant was high risk. Institutional guidelines that recommended selective CSF testing for febrile infants aged 29 to 56 days were associated with lower rates of CSF testing. Furthermore, ceftriaxone use varied based on CPG recommendations for low‐risk infants discharged from the ED. Therefore, the influence of febrile infant CPGs mainly relates to the limiting of CSF testing and targeted ceftriaxone use in low‐risk infants. As the rate of return visits for SBI is low across hospitals, future study should assess outcomes at hospitals with CPGs recommending selective CSF testing. Of note, infants 29 to 56 days of age were less likely to be hospitalized when cared for at a hospital with an established CPG prior to 2013 without increase in 3‐day revisits for SBI. This finding may indicate that longer duration of CPG implementation is associated with lower rates of hospitalization for low‐risk infants; this finding merits further study.

The presence of a CPG was not associated with lower costs for febrile infants in either age group. Although individual healthcare systems have achieved lower costs with CPG implementation,[12] the mere presence of a CPG is not associated with lower costs when assessed across institutions. Higher costs for admitted and discharged infants 29 to 56 days of age in the presence of a CPG likely reflects the higher rate of CSF testing at hospitals whose CPGs recommend testing for all febrile infants, as well as inpatient management strategies for hospitalized infants not captured in our study. Future investigation should include an assessment of the cost‐effectiveness of the various testing and treatment strategies employed for the febrile young infant.

Our study has several limitations. First, the validity of ICD‐9 diagnosis codes for identifying young infants with fever is not well established, and thus our study is subject to misclassification bias. To minimize missed patients, we included infants with either an ICD‐9 admission or discharge diagnosis of fever; however, utilization of diagnosis codes for patient identification may have resulted in undercapture of infants with a measured temperature of 38.0C. It is also possible that some patients who did not undergo testing were misclassified as having a fever or had temperatures below standard thresholds to prompt diagnostic testing. This is a potential reason that testing was not performed in 100% of infants, even at hospitals with CPGs that recommended testing for all patients. Additionally, some febrile infants diagnosed with SBI may not have an associated ICD‐9 diagnosis code for fever. Although the overall SBI rate observed in our study was similar to prior studies,[4, 31] the rate in neonates 28 days of age was lower than reported in recent investigations,[2, 3] which may indicate inclusion of a higher proportion of low‐risk febrile infants. With the exception of bronchiolitis, we also did not assess diagnostic testing in the presence of other identified sources of infection such as herpes simplex virus.

Second, we were unable to assess the presence or absence of a CPG at the 4 excluded EDs that did not respond to the survey or the institutions excluded for data‐quality issues. However, included and excluded hospitals did not differ in region or annual ED volume (data not shown).

Third, although we classified hospitals based upon the presence and content of CPGs, we were unable to fully evaluate adherence to the CPG at each site.

Last, though PHIS hospitals represent 85% of freestanding children's hospitals, many febrile infants are hospitalized at non‐PHIS institutions; our results may not be generalizable to care provided at nonchildren's hospitals.

CONCLUSIONS

Management of febrile neonates 28 days of age does not vary based on CPG presence. However, CPGs for the febrile infant aged 29 to 56 days vary in recommendations for CSF testing as well as ceftriaxone use for low‐risk patients, which significantly contributes to practice variation and healthcare costs across institutions.

Acknowledgements

The Febrile Young Infant Research Collaborative includes the following additional investigators who are acknowledged for their work on this study: Kao‐Ping Chua, MD, Harvard PhD Program in Health Policy, Harvard University, Cambridge, Massachusetts, and Division of Emergency Medicine, Department of Pediatrics, Boston Children's Hospital, Boston, Massachusetts; Elana A. Feldman, BA, University of Washington School of Medicine, Seattle, Washington; and Katie L. Hayes, BS, Division of Emergency Medicine, Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.

Disclosures

This project was funded in part by The Gerber Foundation Novice Researcher Award (Ref #18273835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript. The authors have no financial relationships relevant to this article to disclose. No payment was received for the production of this article. The authors have no conflicts of interest relevant to this article to disclose.

Febrile young infants are at high risk for serious bacterial infection (SBI) with reported rates of 8.5% to 12%, even higher in neonates 28 days of age.[1, 2, 3] As a result, febrile infants often undergo extensive diagnostic evaluation consisting of a combination of urine, blood, and cerebrospinal fluid (CSF) testing.[4, 5, 6] Several clinical prediction algorithms use this diagnostic testing to identify febrile infants at low risk for SBI, but they differ with respect to age range, recommended testing, antibiotic administration, and threshold for hospitalization.[4, 5, 6] Additionally, the optimal management strategy for this population has not been defined.[7] Consequently, laboratory testing, antibiotic use, and hospitalization for febrile young infants vary widely among hospitals.[8, 9, 10]

Clinical practice guidelines (CPGs) are designed to implement evidence‐based care and reduce practice variability, with the goal of improving quality of care and optimizing costs.[11] Implementation of a CPG for management of febrile young infants in the Intermountain Healthcare System was associated with greater adherence to evidence‐based care and lower costs.[12] However, when strong evidence is lacking, different interpretations of febrile infant risk classification incorporated into local CPGs may be a major driver of the across‐hospital practice variation observed in prior studies.[8, 9] Understanding sources of variability as well as determining the association of CPGs with clinicians' practice patterns can help identify quality improvement opportunities, either through national benchmarking or local efforts.

Our primary objectives were to compare (1) recommendations of pediatric emergency departmentbased institutional CPGs for febrile young infants and (2) rates of urine, blood, CSF testing, hospitalization, and ceftriaxone use at emergency department (ED) discharge based upon CPG presence and the specific CPG recommendations. Our secondary objectives were to describe the association of CPGs with healthcare costs and return visits for SBI.

METHODS

Study Design

We used the Pediatric Health Information System (PHIS) to identify febrile infants 56 days of age who presented to the ED between January 1, 2013 and December 31, 2013. We also surveyed ED providers at participating PHIS hospitals. Informed consent was obtained from survey respondents. The institutional review board at Boston Children's Hospital approved the study protocol.

Clinical Practice Guideline Survey

We sent an electronic survey to medical directors or division directors at 37 pediatric EDs to determine whether their ED utilized a CPG for the management of the febrile young infant in 2013. If no response was received after the second attempt, we queried ED fellowship directors or other ED attending physicians at nonresponding hospitals. Survey items included the presence of a febrile young infant CPG, and if present, the year of implementation, ages targeted, and CPG content. As applicable, respondents were asked to share their CPG and/or provide the specific CPG recommendations.

We collected and managed survey data using the Research Electronic Data Capture (REDCap) electronic data capture tools hosted at Boston Children's Hospital. REDCap is a secure, Web‐based application designed to support data capture for research studies.[13]

Data Source

The PHIS database contains administrative data from 44 US children's hospitals. These hospitals, affiliated with the Children's Hospital Association, represent 85% of freestanding US children's hospitals.[14] Encrypted patient identifiers permit tracking of patients across encounters.[15] Data quality and integrity are assured jointly by the Children's Hospital Association and participating hospitals.[16] For this study, 7 hospitals were excluded due to incomplete ED data or known data‐quality issues.[17]

Patients

We identified study infants using the following International Classification of Diseases, 9th Revision (ICD‐9) admission or discharge diagnosis codes for fever as defined previously[8, 9]: 780.6, 778.4, 780.60, or 780.61. We excluded infants with a complex chronic condition[18] and those transferred from another institution, as these infants may warrant a nonstandard evaluation and/or may have incomplete data. For infants with >1 ED visit for fever during the study period, repeat visits within 3 days of an index visit were considered a revisit for the same episode of illness; visits >3 days following an index visit were considered as a new index visit.

Study Definitions

From the PHIS database, we abstracted demographic characteristics (gender, race/ethnicity), insurance status, and region where the hospital was located (using US Census categories[19]). Billing codes were used to assess whether urine, blood, and CSF testing (as defined previously[9]) were performed during the ED evaluation. To account for ED visits that spanned the midnight hour, for hospitalized patients we considered any testing or treatment occurring on the initial or second hospital day to be performed in the ED; billing code data in PHIS are based upon calendar day and do not distinguish testing performed in the ED versus inpatient setting.[8, 9] Patients billed for observation care were classified as being hospitalized.[20, 21]

We identified the presence of an SBI using ICD‐9 diagnosis codes for the following infections as described previously[9]: urinary tract infection or pyelonephritis,[22] bacteremia or sepsis, bacterial meningitis,[16] pneumonia,[23] or bacterial enteritis. To assess return visits for SBI that required inpatient management, we defined an ED revisit for an SBI as a return visit within 3 days of ED discharge[24, 25] that resulted in hospitalization with an associated ICD‐9 discharge diagnosis code for an SBI.

Hospitals charges in PHIS database were adjusted for hospital location by using the Centers for Medicare and Medicaid Services price/wage index. Costs were estimated by applying hospital‐level cost‐to‐charge ratios to charge data.[26]

Measured Exposures

The primary exposure was the presence of an ED‐based CPG for management of the febrile young infant aged 28 days and 29 to 56 days; 56 days was used as the upper age limit as all of the CPGs included infants up to this age or beyond. Six institutions utilized CPGs with different thresholds to define the age categories (eg, dichotomized at 27 or 30 days); these CPGs were classified into the aforementioned age groups to permit comparisons across standardized age groups. We classified institutions based on the presence of a CPG. To assess differences in the application of low‐risk criteria, the CPGs were further classified a priori based upon specific recommendations around laboratory testing and hospitalization, as well as ceftriaxone use for infants aged 29 to 56 days discharged from the ED. CPGs were categorized based upon whether testing, hospitalization, and ceftriaxone use were: (1) recommended for all patients, (2) recommended only if patients were classified as high risk (absence of low‐risk criteria), (3) recommended against, or (4) recommended to consider at clinician discretion.

Outcome Measures

Measured outcomes were performance of urine, blood, CSF testing, and hospitalization rate, as well as rate of ceftriaxone use for discharged infants aged 29 to 56 days, 3‐day revisits for SBI, and costs per visit, which included hospitalization costs for admitted patients.

Data Analysis

We described continuous variables using median and interquartile range or range values and categorical variables using frequencies. We compared medians using Wilcoxon rank sum and categorical variables using a [2] test. We compared rates of testing, hospitalization, ceftriaxone use, and 3‐day revisits for SBI based on the presence of a CPG, and when present, the specific CPG recommendations. Costs per visit were compared between institutions with and without CPGs and assessed separately for admitted and discharged patients. To adjust for potential confounders and clustering of patients within hospitals, we used generalized estimating equations with logistic regression to generate adjusted odd ratios (aORs) and 95% confidence intervals (CIs). Models were adjusted for geographic region, payer, race, and gender. Statistical analyses were performed by using SAS version 9.3 (SAS Institute, Cary, NC). We determined statistical significance as a 2‐tailed P value <0.05.

Febrile infants with bronchiolitis or a history of prematurity may be managed differently from full‐term febrile young infants without bronchiolitis.[6, 27] Therefore, we performed a subgroup analysis after exclusion of infants with an ICD‐9 discharge diagnosis code for bronchiolitis (466.11 and 466.19)[28] or prematurity (765).

Because our study included ED encounters in 2013, we repeated our analyses after exclusion of hospitals with CPGs implemented during the 2013 calendar year.

RESULTS

CPG by Institution

Thirty‐three (89.2%) of the 37 EDs surveyed completed the questionnaire. Overall, 21 (63.6%) of the 33 EDs had a CPG; 15 (45.5%) had a CPG for all infants 56 days of age, 5 (15.2%) had a CPG for infants 28 days only, and 1 (3.0%) had a CPG for infants 29 to 56 days but not 28 days of age (Figure 1). Seventeen EDs had an established CPG prior to 2013, and 4 hospitals implemented a CPG during the 2013 calendar year, 2 with CPGs for neonates 28 days and 2 with CPGs for both 28 days and 29 to 56 days of age. Hospitals with CPGs were more likely to be located in the Northeast and West regions of the United States and provide care to a higher proportion of non‐Hispanic white patients, as well as those with commercial insurance (Table 1).

Figure 1
Specific clinical practice guideline (CPG) recommendations for diagnostic testing, hospitalization, and ceftriaxone use at ED discharge by institution among the 21 institutions with a CPG. Urine testing is defined as urine dipstick, urinalysis, or urine culture; blood testing as complete blood count or blood culture, and cerebrospinal fluid (CSF) testing as cell count, culture, or procedure code for lumbar puncture. Abbreviations: ED, emergency department.
Characteristics of Patients in Hospitals With and Without CPGs for the Febrile Young Infant 56 Days of Age
Characteristic28 Days2956 Days
No CPG, n=996, N (%)CPG, n=2,149, N (%)P ValueNo CPG, n=2,460, N (%)CPG, n=3,772, N (%)P Value
  • NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range; UTI, urinary tract infection. *Includes UTI/pyelonephritis, bacteremia/sepsis, bacterial meningitis, pneumonia, and bacterial enteritis. Some infants had more than 1 site of infection.

Race      
Non‐Hispanic white325 (32.6)996 (46.3) 867 (35.2)1,728 (45.8) 
Non‐Hispanic black248 (24.9)381 (17.7) 593 (24.1)670 (17.8) 
Hispanic243 (24.4)531 (24.7) 655 (26.6)986 (26.1) 
Asian28 (2.8)78 (3.6) 40 (1.6)122 (3.2) 
Other Race152 (15.3)163 (7.6)<0.001305 (12.4)266 (7.1)<0.001
Gender      
Female435 (43.7)926 (43.1)0.761,067 (43.4)1,714 (45.4)0.22
Payer      
Commercial243 (24.4)738 (34.3) 554 (22.5)1,202 (31.9) 
Government664 (66.7)1,269 (59.1) 1,798 (73.1)2,342 (62.1) 
Other payer89 (8.9)142 (6.6)<0.001108 (4.4)228 (6.0)<0.001
Region      
Northeast39 (3.9)245 (11.4) 77 (3.1)572 (15.2) 
South648 (65.1)915 (42.6) 1,662 (67.6)1,462 (38.8) 
Midwest271 (27.2)462 (21.5) 506 (20.6)851 (22.6) 
West38 (3.8)527 (24.5)<0.001215 (8.7)887 (23.5)<0.001
Serious bacterial infection      
Overall*131 (13.2)242 (11.3)0.14191 (7.8)237 (6.3)0.03
UTI/pyelonephritis73 (7.3)153 (7.1) 103 (4.2)154 (4.1) 
Bacteremia/sepsis56 (5.6)91 (4.2) 78 (3.2)61 (1.6) 
Bacterial meningitis15 (1.5)15 (0.7) 4 (0.2)14 (0.4) 
Age, d, median (IQR)18 (11, 24)18 (11, 23)0.6746 (37, 53)45 (37, 53)0.11

All 20 CPGs for the febrile young infant 28 days of age recommended urine, blood, CSF testing, and hospitalization for all infants (Figure 1). Of the 16 hospitals with CPGs for febrile infants aged 29 to 56 days, all recommended urine and blood testing for all patients, except for 1 CPG, which recommended consideration of blood testing but not to obtain routinely. Hospitals varied in recommendations for CSF testing among infants aged 29 to 56 days: 8 (50%) recommended CSF testing in all patients and 8 (50%) recommended CSF testing only if the patient was high risk per defined criteria (based on history, physical examination, urine, and blood testing). In all 16 CPGs, hospitalization was recommended only for high‐risk infants. For low‐risk infants aged 2956 days being discharged from the ED, 3 hospitals recommended ceftriaxone for all, 9 recommended consideration of ceftriaxone, and 4 recommended against antibiotics (Figure 1).

Study Patients

During the study period, there were 10,415 infants 56 days old with a diagnosis of fever at the 33 participating hospitals. After exclusion of 635 (6.1%) infants with a complex chronic condition and 445 (4.3%) transferred from another institution (including 42 with a complex chronic condition), 9377 infants remained in our study cohort. Approximately one‐third of the cohort was 28 days of age and two‐thirds aged 29 to 56 days. The overall SBI rate was 8.5% but varied by age (11.9% in infants 28 days and 6.9% in infants 29 to 56 days of age) (Table 1).

CPGs and Use of Diagnostic Testing, Hospitalization Rates, Ceftriaxone Use, and Revisits for SBI

For infants 28 days of age, the presence of a CPG was not associated with urine, blood, CSF testing, or hospitalization after multivariable adjustment (Table 2). Among infants aged 29 to 56 days, urine testing did not differ based on the presence of a CPG, whereas blood testing was performed less often at the 1 hospital whose CPG recommended to consider, but not routinely obtain, testing (aOR: 0.4, 95% CI: 0.3‐0.7, P=0.001). Compared to hospitals without a CPG, CSF testing was performed less often at hospitals with CPG recommendations to only obtain CSF if high risk (aOR: 0.5, 95% CI: 0.3‐0.8, P=0.002). However, the odds of hospitalization did not differ at institutions with and without a febrile infant CPG (aOR: 0.7, 95% CI: 0.5‐1.1, P=0.10). For infants aged 29 to 56 days discharged from the ED, ceftriaxone was administered more often at hospitals with CPGs that recommended ceftriaxone for all discharged patients (aOR: 4.6, 95% CI: 2.39.3, P<0.001) and less often at hospitals whose CPGs recommended against antibiotics (aOR: 0.3, 95% CI: 0.1‐0.9, P=0.03) (Table 3). Our findings were similar in the subgroup of infants without bronchiolitis or prematurity (see Supporting Tables 1 and 2 in the online version of this article). After exclusion of hospitals with a CPG implemented during the 2013 calendar year (4 hospitals excluded in the 28 days age group and 2 hospitals excluded in the 29 to 56 days age group), infants aged 29 to 56 days cared for at a hospital with a CPG experienced a lower odds of hospitalization (aOR: 0.7, 95% CI: 0.4‐0.98, P=0.04). Otherwise, our findings in both age groups did not materially differ from the main analyses.

Variation in Testing and Hospitalization Based on CPG‐Specific Recommendations Among Infants 28 Days of Age With Diagnosis of Fever
Testing/HospitalizationNo. of HospitalsNo. of Patients% Received*aOR (95% CI)P Value
  • NOTE: Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; CPG, clinical practice guideline; CSF, cerebrospinal fluid. *Percent of infants who received test or were hospitalized. Adjusted for hospital clustering, geographic region, payer, race, and gender. Urine testing defined as urine dipstick, urinalysis, or urine culture; Blood testing defined as complete blood count or blood culture. ‖CSF testing defined as cell count, culture, or procedure code for lumbar puncture

Laboratory testing     
Urine testing     
No CPG1399675.6Ref 
CPG: recommend for all202,14980.71.2 (0.9‐1.7)0.22
Blood testing     
No CPG1399676.9Ref 
CPG: recommend for all202,14981.81.2 (0.9‐1.7)0.25
CSF testing     
No CPG1399671.0Ref 
CPG: recommend for all202,14977.51.3 (1.01.7)0.08
Disposition     
Hospitalization     
No CPG1399675.4Ref 
CPG: recommend for all202,14981.61.2 (0.9‐1.8)0.26
Variation in Testing, Hospitalization, and Ceftriaxone Use Based on CPG‐Specific Recommendations Among Infants 29 to 56 Days of Age With Diagnosis of Fever
Testing/HospitalizationNo. of HospitalsNo. of Patients% Received*aOR (95% CI)P Value
  • NOTE: Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; CPG, clinical practice guideline; CSF, cerebrospinal fluid. *Percent of infants who received test, were hospitalized, or received ceftriaxone. Adjusted for hospital clustering, geographic region, payer, race, and gender. Urine testing defined as urine dipstick, urinalysis, or urine culture. Blood testing defined as complete blood count or blood culture. CSF testing defined as cell count, culture, or procedure code for lumbar puncture. For low‐risk infants discharged from the emergency department.

Laboratory resting     
Urine testing     
No CPG172,46081.1Ref 
CPG: recommend for all163,77282.10.9 (0.7‐1.4)0.76
Blood testing     
No CPG172,46079.4Ref 
CPG: recommend for all153,62882.61.1 (0.7‐1.6)0.70
CPG: recommend consider114462.50.4 (0.3‐0.7)0.001
CSF testing     
No CPG172,46046.3Ref 
CPG: recommend for all81,51770.31.3 (0.9‐1.9)0.11
CPG: recommend if high‐risk82,25539.90.5 (0.3‐0.8)0.002
Disposition     
Hospitalization     
No CPG172,46047.0Ref 
CPG: recommend if high‐risk163,77242.00.7 (0.5‐1.1)0.10
Ceftriaxone if discharged     
No CPG171,30411.7Ref 
CPG: recommend against431310.90.3 (0.1‐0.9)0.03
CPG: recommend consider91,56714.41.5 (0.9‐2.4)0.09
CPG: recommend for all330664.14.6 (2.39.3)< 0.001

Three‐day revisits for SBI were similarly low at hospitals with and without CPGs among infants 28 days (1.5% vs 0.8%, P=0.44) and 29 to 56 days of age (1.4% vs 1.1%, P=0.44) and did not differ after exclusion of hospitals with a CPG implemented in 2013.

CPGs and Costs

Among infants 28 days of age, costs per visit did not differ for admitted and discharged patients based on CPG presence. The presence of an ED febrile infant CPG was associated with higher costs for both admitted and discharged infants 29 to 56 days of age (Table 4). The cost analysis did not significantly differ after exclusion of hospitals with CPGs implemented in 2013.

Costs per Visit for Febrile Young Infants 56 Days of Age at Institutions With and Without CPGs
 28 Days, Cost, Median (IQR)29 to 56 Days, Cost, Median (IQR)
No CPGCPGP ValueNo CPGCPGP Value
  • NOTE: Abbreviations: CPG, clinical practice guideline; IQR, interquartile range.

Admitted$4,979 ($3,408$6,607) [n=751]$4,715 ($3,472$6,526) [n=1,753]0.79$3,756 ($2,725$5,041) [n=1,156]$3,923 ($3,077$5,243) [n=1,586]<0.001
Discharged$298 ($166$510) [n=245]$231 ($160$464) [n=396]0.10$681($398$982) [n=1,304)]$764 ($412$1,100) [n=2,186]<0.001

DISCUSSION

We described the content and association of CPGs with management of the febrile infant 56 days of age across a large sample of children's hospitals. Nearly two‐thirds of included pediatric EDs have a CPG for the management of young febrile infants. Management of febrile infants 28 days was uniform, with a majority hospitalized after urine, blood, and CSF testing regardless of the presence of a CPG. In contrast, CPGs for infants 29 to 56 days of age varied in their recommendations for CSF testing as well as ceftriaxone use for infants discharged from the ED. Consequently, we observed considerable hospital variability in CSF testing and ceftriaxone use for discharged infants, which correlates with variation in the presence and content of CPGs. Institutional CPGs may be a source of the across‐hospital variation in care of febrile young infants observed in prior study.[9]

Febrile infants 28 days of age are at particularly high risk for SBI, with a prevalence of nearly 20% or higher.[2, 3, 29] The high prevalence of SBI, combined with the inherent difficulty in distinguishing neonates with and without SBI,[2, 30] has resulted in uniform CPG recommendations to perform the full‐sepsis workup in this young age group. Similar to prior studies,[8, 9] we observed that most febrile infants 28 days undergo the full sepsis evaluation, including CSF testing, and are hospitalized regardless of the presence of a CPG.

However, given the conflicting recommendations for febrile infants 29 to 56 days of age,[4, 5, 6] the optimal management strategy is less certain.[7] The Rochester, Philadelphia, and Boston criteria, 3 published models to identify infants at low risk for SBI, primarily differ in their recommendations for CSF testing and ceftriaxone use in this age group.[4, 5, 6] Half of the CPGs recommended CSF testing for all febrile infants, and half recommended CSF testing only if the infant was high risk. Institutional guidelines that recommended selective CSF testing for febrile infants aged 29 to 56 days were associated with lower rates of CSF testing. Furthermore, ceftriaxone use varied based on CPG recommendations for low‐risk infants discharged from the ED. Therefore, the influence of febrile infant CPGs mainly relates to the limiting of CSF testing and targeted ceftriaxone use in low‐risk infants. As the rate of return visits for SBI is low across hospitals, future study should assess outcomes at hospitals with CPGs recommending selective CSF testing. Of note, infants 29 to 56 days of age were less likely to be hospitalized when cared for at a hospital with an established CPG prior to 2013 without increase in 3‐day revisits for SBI. This finding may indicate that longer duration of CPG implementation is associated with lower rates of hospitalization for low‐risk infants; this finding merits further study.

The presence of a CPG was not associated with lower costs for febrile infants in either age group. Although individual healthcare systems have achieved lower costs with CPG implementation,[12] the mere presence of a CPG is not associated with lower costs when assessed across institutions. Higher costs for admitted and discharged infants 29 to 56 days of age in the presence of a CPG likely reflects the higher rate of CSF testing at hospitals whose CPGs recommend testing for all febrile infants, as well as inpatient management strategies for hospitalized infants not captured in our study. Future investigation should include an assessment of the cost‐effectiveness of the various testing and treatment strategies employed for the febrile young infant.

Our study has several limitations. First, the validity of ICD‐9 diagnosis codes for identifying young infants with fever is not well established, and thus our study is subject to misclassification bias. To minimize missed patients, we included infants with either an ICD‐9 admission or discharge diagnosis of fever; however, utilization of diagnosis codes for patient identification may have resulted in undercapture of infants with a measured temperature of 38.0C. It is also possible that some patients who did not undergo testing were misclassified as having a fever or had temperatures below standard thresholds to prompt diagnostic testing. This is a potential reason that testing was not performed in 100% of infants, even at hospitals with CPGs that recommended testing for all patients. Additionally, some febrile infants diagnosed with SBI may not have an associated ICD‐9 diagnosis code for fever. Although the overall SBI rate observed in our study was similar to prior studies,[4, 31] the rate in neonates 28 days of age was lower than reported in recent investigations,[2, 3] which may indicate inclusion of a higher proportion of low‐risk febrile infants. With the exception of bronchiolitis, we also did not assess diagnostic testing in the presence of other identified sources of infection such as herpes simplex virus.

Second, we were unable to assess the presence or absence of a CPG at the 4 excluded EDs that did not respond to the survey or the institutions excluded for data‐quality issues. However, included and excluded hospitals did not differ in region or annual ED volume (data not shown).

Third, although we classified hospitals based upon the presence and content of CPGs, we were unable to fully evaluate adherence to the CPG at each site.

Last, though PHIS hospitals represent 85% of freestanding children's hospitals, many febrile infants are hospitalized at non‐PHIS institutions; our results may not be generalizable to care provided at nonchildren's hospitals.

CONCLUSIONS

Management of febrile neonates 28 days of age does not vary based on CPG presence. However, CPGs for the febrile infant aged 29 to 56 days vary in recommendations for CSF testing as well as ceftriaxone use for low‐risk patients, which significantly contributes to practice variation and healthcare costs across institutions.

Acknowledgements

The Febrile Young Infant Research Collaborative includes the following additional investigators who are acknowledged for their work on this study: Kao‐Ping Chua, MD, Harvard PhD Program in Health Policy, Harvard University, Cambridge, Massachusetts, and Division of Emergency Medicine, Department of Pediatrics, Boston Children's Hospital, Boston, Massachusetts; Elana A. Feldman, BA, University of Washington School of Medicine, Seattle, Washington; and Katie L. Hayes, BS, Division of Emergency Medicine, Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.

Disclosures

This project was funded in part by The Gerber Foundation Novice Researcher Award (Ref #18273835). Dr. Fran Balamuth received career development support from the National Institutes of Health (NHLBI K12‐HL109009). Funders were not involved in design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript. The authors have no financial relationships relevant to this article to disclose. No payment was received for the production of this article. The authors have no conflicts of interest relevant to this article to disclose.

References
  1. Huppler AR, Eickhoff JC, Wald ER. Performance of low‐risk criteria in the evaluation of young infants with fever: review of the literature. Pediatrics. 2010;125:228233.
  2. Schwartz S, Raveh D, Toker O, Segal G, Godovitch N, Schlesinger Y. A week‐by‐week analysis of the low‐risk criteria for serious bacterial infection in febrile neonates. Arch Dis Child. 2009;94:287292.
  3. Garcia S, Mintegi S, Gomez B, et al. Is 15 days an appropriate cut‐off age for considering serious bacterial infection in the management of febrile infants? Pediatr Infect Dis J. 2012;31:455458.
  4. Baker MD, Bell LM, Avner JR. Outpatient management without antibiotics of fever in selected infants. N Engl J Med. 1993;329:14371441.
  5. Baskin MN, Fleisher GR, O'Rourke EJ. Identifying febrile infants at risk for a serious bacterial infection. J Pediatr. 1993;123:489490.
  6. Jaskiewicz JA, McCarthy CA, Richardson AC, et al. Febrile infants at low risk for serious bacterial infection—an appraisal of the Rochester criteria and implications for management. Febrile Infant Collaborative Study Group. Pediatrics. 1994;94:390396.
  7. American College of Emergency Physicians Clinical Policies Committee; American College of Emergency Physicians Clinical Policies Subcommittee on Pediatric Fever. Clinical policy for children younger than three years presenting to the emergency department with fever. Ann Emerg Med. 2003;42:530545.
  8. Jain S, Cheng J, Alpern ER, et al. Management of febrile neonates in US pediatric emergency departments. Pediatrics. 2014;133:187195.
  9. Aronson PL, Thurm C, Alpern ER, et al. Variation in care of the febrile young infant <90 days in US pediatric emergency departments. Pediatrics. 2014;134:667677.
  10. Yarden‐Bilavsky H, Ashkenazi S, Amir J, Schlesinger Y, Bilavsky E. Fever survey highlights significant variations in how infants aged ≤60 days are evaluated and underline the need for guidelines. Acta Paediatr. 2014;103:379385.
  11. Bergman DA. Evidence‐based guidelines and critical pathways for quality improvement. Pediatrics. 1999;103:225232.
  12. Byington CL, Reynolds CC, Korgenski K, et al. Costs and infant outcomes after implementation of a care process model for febrile infants. Pediatrics. 2012;130:e16e24.
  13. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
  14. Wood JN, Feudtner C, Medina SP, Luan X, Localio R, Rubin DM. Variation in occult injury screening for children with suspected abuse in selected US children's hospitals. Pediatrics. 2012;130:853860.
  15. Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J AHIMA. 2004;75:2226.
  16. Mongelluzzo J, Mohamad Z, Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299:20482055.
  17. Kharbanda AB, Hall M, Shah SS, et al. Variation in resource utilization across a national sample of pediatric emergency departments. J Pediatr. 2013;163:230236.
  18. Feudtner C, Hays RM, Haynes G, Geyer JR, Neff JM, Koepsell TD. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics. 2001;107:E99.
  19. US Census Bureau. Geographic terms and concepts—census divisions and census regions. Available at: https://www.census.gov/geo/reference/gtc/gtc_census_divreg.html. Accessed September 10, 2014.
  20. Macy ML, Hall M, Shah SS, et al. Pediatric observation status: are we overlooking a growing population in children's hospitals? J Hosp Med. 2012;7:530536.
  21. Macy ML, Hall M, Shah SS, et al. Differences in designations of observation care in US freestanding children's hospitals: are they virtual or real? J Hosp Med. 2012;7:287293.
  22. Tieder JS, Hall M, Auger KA, et al. Accuracy of administrative billing codes to detect urinary tract infection hospitalizations. Pediatrics. 2011;128:323330.
  23. Williams DJ, Shah SS, Myers A, et al. Identifying pediatric community‐acquired pneumonia hospitalizations: accuracy of administrative billing codes. JAMA Pediatr. 2013;167:851858.
  24. Gordon JA, An LC, Hayward RA, Williams BC. Initial emergency department diagnosis and return visits: risk versus perception. Ann Emerg Med. 1998;32:569573.
  25. Cho CS, Shapiro DJ, Cabana MD, Maselli JH, Hersh AL. A national depiction of children with return visits to the emergency department within 72 hours, 2001–2007. Pediatr Emerg Care. 2012;28:606610.
  26. Healthcare Cost and Utilization Project. Cost‐to‐charge ratio files. Available at: http://www.hcup‐us.ahrq.gov/db/state/costtocharge.jsp. Accessed September 11, 2014.
  27. Levine DA, Platt SL, Dayan PS, et al. Risk of serious bacterial infection in young febrile infants with respiratory syncytial virus infections. Pediatrics. 2004;113:17281734.
  28. Parikh K, Hall M, Mittal V, et al. Establishing benchmarks for the hospitalized care of children with asthma, bronchiolitis, and pneumonia. Pediatrics. 2014;134:555562.
  29. Mintegi S, Benito J, Astobiza E, Capape S, Gomez B, Eguireun A. Well appearing young infants with fever without known source in the emergency department: are lumbar punctures always necessary? Eur J Emerg Med. 2010;17:167169.
  30. Baker MD, Bell LM. Unpredictability of serious bacterial illness in febrile infants from birth to 1 month of age. Arch Pediatr Adolesc Med. 1999;153:508511.
  31. Pantell RH, Newman TB, Bernzweig J, et al. Management and outcomes of care of fever in early infancy. JAMA. 2004;291:12031212.
References
  1. Huppler AR, Eickhoff JC, Wald ER. Performance of low‐risk criteria in the evaluation of young infants with fever: review of the literature. Pediatrics. 2010;125:228233.
  2. Schwartz S, Raveh D, Toker O, Segal G, Godovitch N, Schlesinger Y. A week‐by‐week analysis of the low‐risk criteria for serious bacterial infection in febrile neonates. Arch Dis Child. 2009;94:287292.
  3. Garcia S, Mintegi S, Gomez B, et al. Is 15 days an appropriate cut‐off age for considering serious bacterial infection in the management of febrile infants? Pediatr Infect Dis J. 2012;31:455458.
  4. Baker MD, Bell LM, Avner JR. Outpatient management without antibiotics of fever in selected infants. N Engl J Med. 1993;329:14371441.
  5. Baskin MN, Fleisher GR, O'Rourke EJ. Identifying febrile infants at risk for a serious bacterial infection. J Pediatr. 1993;123:489490.
  6. Jaskiewicz JA, McCarthy CA, Richardson AC, et al. Febrile infants at low risk for serious bacterial infection—an appraisal of the Rochester criteria and implications for management. Febrile Infant Collaborative Study Group. Pediatrics. 1994;94:390396.
  7. American College of Emergency Physicians Clinical Policies Committee; American College of Emergency Physicians Clinical Policies Subcommittee on Pediatric Fever. Clinical policy for children younger than three years presenting to the emergency department with fever. Ann Emerg Med. 2003;42:530545.
  8. Jain S, Cheng J, Alpern ER, et al. Management of febrile neonates in US pediatric emergency departments. Pediatrics. 2014;133:187195.
  9. Aronson PL, Thurm C, Alpern ER, et al. Variation in care of the febrile young infant <90 days in US pediatric emergency departments. Pediatrics. 2014;134:667677.
  10. Yarden‐Bilavsky H, Ashkenazi S, Amir J, Schlesinger Y, Bilavsky E. Fever survey highlights significant variations in how infants aged ≤60 days are evaluated and underline the need for guidelines. Acta Paediatr. 2014;103:379385.
  11. Bergman DA. Evidence‐based guidelines and critical pathways for quality improvement. Pediatrics. 1999;103:225232.
  12. Byington CL, Reynolds CC, Korgenski K, et al. Costs and infant outcomes after implementation of a care process model for febrile infants. Pediatrics. 2012;130:e16e24.
  13. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377381.
  14. Wood JN, Feudtner C, Medina SP, Luan X, Localio R, Rubin DM. Variation in occult injury screening for children with suspected abuse in selected US children's hospitals. Pediatrics. 2012;130:853860.
  15. Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J AHIMA. 2004;75:2226.
  16. Mongelluzzo J, Mohamad Z, Have TR, Shah SS. Corticosteroids and mortality in children with bacterial meningitis. JAMA. 2008;299:20482055.
  17. Kharbanda AB, Hall M, Shah SS, et al. Variation in resource utilization across a national sample of pediatric emergency departments. J Pediatr. 2013;163:230236.
  18. Feudtner C, Hays RM, Haynes G, Geyer JR, Neff JM, Koepsell TD. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics. 2001;107:E99.
  19. US Census Bureau. Geographic terms and concepts—census divisions and census regions. Available at: https://www.census.gov/geo/reference/gtc/gtc_census_divreg.html. Accessed September 10, 2014.
  20. Macy ML, Hall M, Shah SS, et al. Pediatric observation status: are we overlooking a growing population in children's hospitals? J Hosp Med. 2012;7:530536.
  21. Macy ML, Hall M, Shah SS, et al. Differences in designations of observation care in US freestanding children's hospitals: are they virtual or real? J Hosp Med. 2012;7:287293.
  22. Tieder JS, Hall M, Auger KA, et al. Accuracy of administrative billing codes to detect urinary tract infection hospitalizations. Pediatrics. 2011;128:323330.
  23. Williams DJ, Shah SS, Myers A, et al. Identifying pediatric community‐acquired pneumonia hospitalizations: accuracy of administrative billing codes. JAMA Pediatr. 2013;167:851858.
  24. Gordon JA, An LC, Hayward RA, Williams BC. Initial emergency department diagnosis and return visits: risk versus perception. Ann Emerg Med. 1998;32:569573.
  25. Cho CS, Shapiro DJ, Cabana MD, Maselli JH, Hersh AL. A national depiction of children with return visits to the emergency department within 72 hours, 2001–2007. Pediatr Emerg Care. 2012;28:606610.
  26. Healthcare Cost and Utilization Project. Cost‐to‐charge ratio files. Available at: http://www.hcup‐us.ahrq.gov/db/state/costtocharge.jsp. Accessed September 11, 2014.
  27. Levine DA, Platt SL, Dayan PS, et al. Risk of serious bacterial infection in young febrile infants with respiratory syncytial virus infections. Pediatrics. 2004;113:17281734.
  28. Parikh K, Hall M, Mittal V, et al. Establishing benchmarks for the hospitalized care of children with asthma, bronchiolitis, and pneumonia. Pediatrics. 2014;134:555562.
  29. Mintegi S, Benito J, Astobiza E, Capape S, Gomez B, Eguireun A. Well appearing young infants with fever without known source in the emergency department: are lumbar punctures always necessary? Eur J Emerg Med. 2010;17:167169.
  30. Baker MD, Bell LM. Unpredictability of serious bacterial illness in febrile infants from birth to 1 month of age. Arch Pediatr Adolesc Med. 1999;153:508511.
  31. Pantell RH, Newman TB, Bernzweig J, et al. Management and outcomes of care of fever in early infancy. JAMA. 2004;291:12031212.
Issue
Journal of Hospital Medicine - 10(6)
Issue
Journal of Hospital Medicine - 10(6)
Page Number
358-365
Page Number
358-365
Publications
Publications
Article Type
Display Headline
Association of clinical practice guidelines with emergency department management of febrile infants ≤56 days of age
Display Headline
Association of clinical practice guidelines with emergency department management of febrile infants ≤56 days of age
Sections
Article Source

© 2015 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Paul L. Aronson, MD, Section of Pediatric Emergency Medicine, Yale School of Medicine, 100 York Street, Suite 1F, New Haven, CT 06511; Telephone: 203–737‐7443; Fax: 203–737‐7447; E‐mail: paul.aronson@yale.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files