User login
ICD-9 codes were poor at picking out idiopathic pulmonary fibrosis patients from administrative databases for epidemiologic studies, but a new tool could improve diagnostic accuracy, according to Kaiser Permanente and University of California, San Francisco (UCSF), investigators.
“In the age of large administrative databases and electronic medical records, there is rich opportunity to conduct population-based studies” of disease behavior, outcomes, health care use, and other matters, but researchers first need to be able to accurately identify patients with idiopathic pulmonary fibrosis (IPF) in large data sets, said investigators led by Brett Ley, MD, an assistant professor of medicine at UCSF.
The research community has traditionally relied on claims for specific IPF diagnostic codes – ICD-9 code 516.3 or ICD-9-CM code 516.31 – to identify patients, but the approach had never been validated. To see how well it works, the investigators applied it to the nearly 5.4 million adults in the Kaiser Permanente Northern California system during 2000-2014. After patients with interstitial lung disease-associated codes entered on or after the day of the last IPF code were excluded, the algorithm identified 2,608 patients as having IPF (Ann Am Thorac Soc. 2017 Jun;14[6]:880-7).
Next, the investigators randomly selected 150 of those patients and examined their medical records, procedure codes, CTs, and other patient-level data to see how many of them really had IPF. The results weren’t good. The positive predictive value of the IPF code-based algorithm was only 42.2%, with a sensitivity 55.6%.
The widely used code-based IPF algorithm does “not generate accurate estimates of IPF incidence and prevalence. ... Over half of the patients identified as having IPF ... did not have IPF on case review. Alarmingly, whereas half of the misclassified cases had an alternative [interstitial lung disease] diagnosis, the other half had no clinical or radiologic evidence of ILD [interstitial lung disease] at all.” The algorithm also “likely misses a substantial proportion of patients who do have IPF,” Dr. Ley and his colleagues said.
“We can only speculate about the reasons. ... It seems likely to be due to a combination of misdiagnosis at the clinical level and miscoding at the administrative level,” they said.
To try to improve the situation, the team tweaked the algorithm to include only patients 50 years or older who had at least two 516.3 or 516.31 claims 1 month or more apart and a chest CT procedure code beforehand. They again excluded ILD-associated claims on or after the day of the last IPF code.
Although the sensitivity of the modified algorithm was lower than the original, it had a more robust positive predictive value of 70.4% in the derivation cohort and 61.8% in the validation cohort, both derived from the 150 patients used to validate the original algorithm.
“By making a few simple, empirically derived changes to the IPF algorithm,” it’s possible to “more reliably identif[y] patients” with IPF. “We believe the modified IPF algorithm will be useful for population-based studies of IPF ... that require high diagnostic certainty,” the investigators concluded.
The traditional algorithm found an incidence of 6.8 cases per 100,000 person-years, which was on the low end of previous reports, perhaps because of the relative health and youth of the 5.4 million patient pool. As in past studies, IPF incidence increased with older age and was highest in white patients and men.
“Whether the more specific codes provided by the ICD-10 system will allow for improved case classification of IPF requires further study,” the investigators noted.
The work was funded by the National Institutes of Health. Dr. Ley reported speaker’s fees from Genentech, and one of the authors was an employee of the company. The senior author Harold Collard, MD, an associate professor in UCSF’s Division of Pulmonary and Critical Care Medicine, reported personal fees from Takeda, ImmuneWorks, Parexel, Pharma Capital Partners, and others.
This study glaringly displays potential problems with using ICD codes for research purposes and calls into question results from a handful of studies that yielded epidemiological estimates for idiopathic pulmonary fibrosis. We are reminded that practitioner-generated diagnostic codes of IPF recorded in the medical record are subject to inaccuracies, which can be illuminated by the “gold standard” – multidisciplinary adjudication.
Moving forward, particularly as longitudinal, nationwide IPF registries come online, patient-level case validation should be employed. As we move into the era of ICD-10, the study should serve as a call to improve IPF case ascertainment accuracy for any investigators choosing to use large data analytic strategies. Doing so will mute the background noise and allow us to better hear the signals of this complex disease.
Evans R. Fernandez Perez, MD, is a pulmonologist at National Jewish Health, Denver. He made his comments in an editorial, and reported speaker’s fees from Boehringer Ingelheim and Genentech (Ann Am Thorac Soc. 2017 Jun;14[6]:829-30).
This study glaringly displays potential problems with using ICD codes for research purposes and calls into question results from a handful of studies that yielded epidemiological estimates for idiopathic pulmonary fibrosis. We are reminded that practitioner-generated diagnostic codes of IPF recorded in the medical record are subject to inaccuracies, which can be illuminated by the “gold standard” – multidisciplinary adjudication.
Moving forward, particularly as longitudinal, nationwide IPF registries come online, patient-level case validation should be employed. As we move into the era of ICD-10, the study should serve as a call to improve IPF case ascertainment accuracy for any investigators choosing to use large data analytic strategies. Doing so will mute the background noise and allow us to better hear the signals of this complex disease.
Evans R. Fernandez Perez, MD, is a pulmonologist at National Jewish Health, Denver. He made his comments in an editorial, and reported speaker’s fees from Boehringer Ingelheim and Genentech (Ann Am Thorac Soc. 2017 Jun;14[6]:829-30).
This study glaringly displays potential problems with using ICD codes for research purposes and calls into question results from a handful of studies that yielded epidemiological estimates for idiopathic pulmonary fibrosis. We are reminded that practitioner-generated diagnostic codes of IPF recorded in the medical record are subject to inaccuracies, which can be illuminated by the “gold standard” – multidisciplinary adjudication.
Moving forward, particularly as longitudinal, nationwide IPF registries come online, patient-level case validation should be employed. As we move into the era of ICD-10, the study should serve as a call to improve IPF case ascertainment accuracy for any investigators choosing to use large data analytic strategies. Doing so will mute the background noise and allow us to better hear the signals of this complex disease.
Evans R. Fernandez Perez, MD, is a pulmonologist at National Jewish Health, Denver. He made his comments in an editorial, and reported speaker’s fees from Boehringer Ingelheim and Genentech (Ann Am Thorac Soc. 2017 Jun;14[6]:829-30).
ICD-9 codes were poor at picking out idiopathic pulmonary fibrosis patients from administrative databases for epidemiologic studies, but a new tool could improve diagnostic accuracy, according to Kaiser Permanente and University of California, San Francisco (UCSF), investigators.
“In the age of large administrative databases and electronic medical records, there is rich opportunity to conduct population-based studies” of disease behavior, outcomes, health care use, and other matters, but researchers first need to be able to accurately identify patients with idiopathic pulmonary fibrosis (IPF) in large data sets, said investigators led by Brett Ley, MD, an assistant professor of medicine at UCSF.
The research community has traditionally relied on claims for specific IPF diagnostic codes – ICD-9 code 516.3 or ICD-9-CM code 516.31 – to identify patients, but the approach had never been validated. To see how well it works, the investigators applied it to the nearly 5.4 million adults in the Kaiser Permanente Northern California system during 2000-2014. After patients with interstitial lung disease-associated codes entered on or after the day of the last IPF code were excluded, the algorithm identified 2,608 patients as having IPF (Ann Am Thorac Soc. 2017 Jun;14[6]:880-7).
Next, the investigators randomly selected 150 of those patients and examined their medical records, procedure codes, CTs, and other patient-level data to see how many of them really had IPF. The results weren’t good. The positive predictive value of the IPF code-based algorithm was only 42.2%, with a sensitivity 55.6%.
The widely used code-based IPF algorithm does “not generate accurate estimates of IPF incidence and prevalence. ... Over half of the patients identified as having IPF ... did not have IPF on case review. Alarmingly, whereas half of the misclassified cases had an alternative [interstitial lung disease] diagnosis, the other half had no clinical or radiologic evidence of ILD [interstitial lung disease] at all.” The algorithm also “likely misses a substantial proportion of patients who do have IPF,” Dr. Ley and his colleagues said.
“We can only speculate about the reasons. ... It seems likely to be due to a combination of misdiagnosis at the clinical level and miscoding at the administrative level,” they said.
To try to improve the situation, the team tweaked the algorithm to include only patients 50 years or older who had at least two 516.3 or 516.31 claims 1 month or more apart and a chest CT procedure code beforehand. They again excluded ILD-associated claims on or after the day of the last IPF code.
Although the sensitivity of the modified algorithm was lower than the original, it had a more robust positive predictive value of 70.4% in the derivation cohort and 61.8% in the validation cohort, both derived from the 150 patients used to validate the original algorithm.
“By making a few simple, empirically derived changes to the IPF algorithm,” it’s possible to “more reliably identif[y] patients” with IPF. “We believe the modified IPF algorithm will be useful for population-based studies of IPF ... that require high diagnostic certainty,” the investigators concluded.
The traditional algorithm found an incidence of 6.8 cases per 100,000 person-years, which was on the low end of previous reports, perhaps because of the relative health and youth of the 5.4 million patient pool. As in past studies, IPF incidence increased with older age and was highest in white patients and men.
“Whether the more specific codes provided by the ICD-10 system will allow for improved case classification of IPF requires further study,” the investigators noted.
The work was funded by the National Institutes of Health. Dr. Ley reported speaker’s fees from Genentech, and one of the authors was an employee of the company. The senior author Harold Collard, MD, an associate professor in UCSF’s Division of Pulmonary and Critical Care Medicine, reported personal fees from Takeda, ImmuneWorks, Parexel, Pharma Capital Partners, and others.
ICD-9 codes were poor at picking out idiopathic pulmonary fibrosis patients from administrative databases for epidemiologic studies, but a new tool could improve diagnostic accuracy, according to Kaiser Permanente and University of California, San Francisco (UCSF), investigators.
“In the age of large administrative databases and electronic medical records, there is rich opportunity to conduct population-based studies” of disease behavior, outcomes, health care use, and other matters, but researchers first need to be able to accurately identify patients with idiopathic pulmonary fibrosis (IPF) in large data sets, said investigators led by Brett Ley, MD, an assistant professor of medicine at UCSF.
The research community has traditionally relied on claims for specific IPF diagnostic codes – ICD-9 code 516.3 or ICD-9-CM code 516.31 – to identify patients, but the approach had never been validated. To see how well it works, the investigators applied it to the nearly 5.4 million adults in the Kaiser Permanente Northern California system during 2000-2014. After patients with interstitial lung disease-associated codes entered on or after the day of the last IPF code were excluded, the algorithm identified 2,608 patients as having IPF (Ann Am Thorac Soc. 2017 Jun;14[6]:880-7).
Next, the investigators randomly selected 150 of those patients and examined their medical records, procedure codes, CTs, and other patient-level data to see how many of them really had IPF. The results weren’t good. The positive predictive value of the IPF code-based algorithm was only 42.2%, with a sensitivity 55.6%.
The widely used code-based IPF algorithm does “not generate accurate estimates of IPF incidence and prevalence. ... Over half of the patients identified as having IPF ... did not have IPF on case review. Alarmingly, whereas half of the misclassified cases had an alternative [interstitial lung disease] diagnosis, the other half had no clinical or radiologic evidence of ILD [interstitial lung disease] at all.” The algorithm also “likely misses a substantial proportion of patients who do have IPF,” Dr. Ley and his colleagues said.
“We can only speculate about the reasons. ... It seems likely to be due to a combination of misdiagnosis at the clinical level and miscoding at the administrative level,” they said.
To try to improve the situation, the team tweaked the algorithm to include only patients 50 years or older who had at least two 516.3 or 516.31 claims 1 month or more apart and a chest CT procedure code beforehand. They again excluded ILD-associated claims on or after the day of the last IPF code.
Although the sensitivity of the modified algorithm was lower than the original, it had a more robust positive predictive value of 70.4% in the derivation cohort and 61.8% in the validation cohort, both derived from the 150 patients used to validate the original algorithm.
“By making a few simple, empirically derived changes to the IPF algorithm,” it’s possible to “more reliably identif[y] patients” with IPF. “We believe the modified IPF algorithm will be useful for population-based studies of IPF ... that require high diagnostic certainty,” the investigators concluded.
The traditional algorithm found an incidence of 6.8 cases per 100,000 person-years, which was on the low end of previous reports, perhaps because of the relative health and youth of the 5.4 million patient pool. As in past studies, IPF incidence increased with older age and was highest in white patients and men.
“Whether the more specific codes provided by the ICD-10 system will allow for improved case classification of IPF requires further study,” the investigators noted.
The work was funded by the National Institutes of Health. Dr. Ley reported speaker’s fees from Genentech, and one of the authors was an employee of the company. The senior author Harold Collard, MD, an associate professor in UCSF’s Division of Pulmonary and Critical Care Medicine, reported personal fees from Takeda, ImmuneWorks, Parexel, Pharma Capital Partners, and others.
FROM THE ANNALS OF THE AMERICAN THORACIC SOCIETY
Key clinical point:
Major finding: The positive predictive value of the traditional IPF code-based algorithm was only 42.2%, with a sensitivity of 55.6%.
Data source: A study including almost 5.4 million patients at Kaiser Permanente Northern California.
Disclosures: The work was funded by the National Institutes of Health. One of the investigators was a Genentech employee. Others reported speaker’s and personal fees from Genentech and other companies.