New Quality Metric Proposed for Colonoscopy

CHICAGO – A novel quality indicator called Adenomas Under the Curve might have important advantages over the established metric, the Adenoma Detection Rate, for evaluation of endoscopists’ performance, said Dr. Hank S. Wang at the annual Digestive Disease Week.

The adenoma detection rate (ADR) was developed by the U.S. Multisociety Task Force on Colorectal Cancer and is widely used. However, while it may have strengths, it also has glaring weaknesses – most notably the lack of distinction between detection of one adenoma, and detection of more than one, in a given patient, Dr. Wang said.

"An endoscopist who consistently finds one adenoma per case receives the same credit towards ADR as somebody who consistently finds more than one," said Dr. Wang, a gastroenterology fellow in the division of digestive diseases, University of California, Los Angeles, Training Program.

"Clearly these [two] endoscopists are performing differently, and over a large enough case series, they may even be providing different rates of protection against colorectal cancer. Yet by ADR, they would appear to be identical," he explained.

Use of ADR as a quality measure might actually discourage the search for additional adenomas, said Dr. Wang, or even promote "gaming the quality indicator" to maximize efficiency while still ensuring that credit is received for ADR. Some endoscopists may become less vigilant once they have found the first adenoma, and thus the use of ADR may create a subtle disincentive, he said.

The new metric developed by Dr. Wang and his colleagues, Adenomas Under the Curve (AUC), incorporates ADR data and provides additional quality data on the total number of adenomas detected. Thus AUC extends ADR by capturing data on incremental adenomas beyond the first one detected, said Dr. Wang.

The abstract presented by Dr. Wang compared ADR to AUC in two groups of physicians – academic and community – who were providing screening services for the same pool of patients. Screening colonoscopies were performed over a 4-year period (2005-2009) by two types of practices: a Veterans Affairs teaching hospital endoscopy unit, and a group of three nonteaching, fee-for-service facilities, all serving same population of veterans. The choice of facility was made by the veterans, and the analysis was adjusted for selection bias.

ADR was calculated by dividing the number of cases with one or more adenoma(s) by the total number of cases performed.

In contrast, calculating the AUC involved plotting a histogram showing the frequency of cases and the number of adenomas detected by case. To account for weighting, case frequency was multiplied by the number of adenomas detected per case. When these data were combined, they yielded the measure Adenomas Under the Curve.

A bivariate analysis was performed to evaluate comparability between the two groups of patients. ADR was compared between groups using bivariate and multivariable logistic regression. Relative difference in ADR vs. AUC, and between the two cohorts, was also calculated.

Of the total population of 3,302 patients, 1,216 patients presented to the teaching facility (there were 6 academic gastroenterologists) and 2,086 presented to the nonteaching facilities (12 community-based gastroenterologists). Patients at the nonteaching facilities were on average 1 year younger, and the endoscopists there were slightly more experienced. All patients in both arms presented for screening, and none had a known history of polyps.

The mean ADR at the teaching site was 28.9%, compared with 25.9% at the nonteaching sites, a small difference that did not reach statistical significance (P = .056). However, when the model was adjusted for differences such as age, sedation, furthest point seen, bowel prep type and quality, experience, and the presence of a fellow, the odds of adenoma detection were 43% higher at the teaching facility, a difference that was highly statistically significant (adjusted odds ratio, 1.43 [1.1-1.8], P = .003).

A plot of the AUC for the teaching site superimposed on the AUC for the nonteaching sites clearly showed that, after detection of the first adenoma at the nonteaching facilities, there was a steep drop-off in the number of cases in which two, three, or four adenomas were detected.

"The difference in AUC between the two sites is largely driven by this difference in the frequency of cases where two, three, and four adenomas are detected per case. We calculated AUC of our teaching facility; it was 56.4 units, compared to 42.7 units at the nonteaching site. This difference was highly statistically significant," (P less than .001), said Dr. Wang. The relative difference in ADR was 10.6%, but the relative AUC difference was 25%.

"As we move toward an era of greater accountability in medicine, our performance measures must become more, and not less, accurate. And I believe the AUC is a step in that direction," said Dr. Wang.

The study was sponsored by an NIH GI Training Grant and CURE: Digestive Diseases Research Center.

CHICAGO – A novel quality indicator called Adenomas Under the Curve might have important advantages over the established metric, the Adenoma Detection Rate, for evaluation of endoscopists’ performance, said Dr. Hank S. Wang at the annual Digestive Disease Week.

The adenoma detection rate (ADR) was developed by the U.S. Multisociety Task Force on Colorectal Cancer and is widely used. However, while it may have strengths, it also has glaring weaknesses – most notably the lack of distinction between detection of one adenoma, and detection of more than one, in a given patient, Dr. Wang said.

"An endoscopist who consistently finds one adenoma per case receives the same credit towards ADR as somebody who consistently finds more than one," said Dr. Wang, a gastroenterology fellow in the division of digestive diseases, University of California, Los Angeles, Training Program.

"Clearly these [two] endoscopists are performing differently, and over a large enough case series, they may even be providing different rates of protection against colorectal cancer. Yet by ADR, they would appear to be identical," he explained.

Use of ADR as a quality measure might actually discourage the search for additional adenomas, said Dr. Wang, or even promote "gaming the quality indicator" to maximize efficiency while still ensuring that credit is received for ADR. Some endoscopists may become less vigilant once they have found the first adenoma, and thus the use of ADR may create a subtle disincentive, he said.

The new metric developed by Dr. Wang and his colleagues, Adenomas Under the Curve (AUC), incorporates ADR data and provides additional quality data on the total number of adenomas detected. Thus AUC extends ADR by capturing data on incremental adenomas beyond the first one detected, said Dr. Wang.

The abstract presented by Dr. Wang compared ADR to AUC in two groups of physicians – academic and community – who were providing screening services for the same pool of patients. Screening colonoscopies were performed over a 4-year period (2005-2009) by two types of practices: a Veterans Affairs teaching hospital endoscopy unit, and a group of three nonteaching, fee-for-service facilities, all serving same population of veterans. The choice of facility was made by the veterans, and the analysis was adjusted for selection bias.

ADR was calculated by dividing the number of cases with one or more adenoma(s) by the total number of cases performed.

In contrast, calculating the AUC involved plotting a histogram showing the frequency of cases and the number of adenomas detected by case. To account for weighting, case frequency was multiplied by the number of adenomas detected per case. When these data were combined, they yielded the measure Adenomas Under the Curve.

A bivariate analysis was performed to evaluate comparability between the two groups of patients. ADR was compared between groups using bivariate and multivariable logistic regression. Relative difference in ADR vs. AUC, and between the two cohorts, was also calculated.

Of the total population of 3,302 patients, 1,216 patients presented to the teaching facility (there were 6 academic gastroenterologists) and 2,086 presented to the nonteaching facilities (12 community-based gastroenterologists). Patients at the nonteaching facilities were on average 1 year younger, and the endoscopists there were slightly more experienced. All patients in both arms presented for screening, and none had a known history of polyps.

The mean ADR at the teaching site was 28.9%, compared with 25.9% at the nonteaching sites, a small difference that did not reach statistical significance (P = .056). However, when the model was adjusted for differences such as age, sedation, furthest point seen, bowel prep type and quality, experience, and the presence of a fellow, the odds of adenoma detection were 43% higher at the teaching facility, a difference that was highly statistically significant (adjusted odds ratio, 1.43 [1.1-1.8], P = .003).

A plot of the AUC for the teaching site superimposed on the AUC for the nonteaching sites clearly showed that, after detection of the first adenoma at the nonteaching facilities, there was a steep drop-off in the number of cases in which two, three, or four adenomas were detected.

"The difference in AUC between the two sites is largely driven by this difference in the frequency of cases where two, three, and four adenomas are detected per case. We calculated AUC of our teaching facility; it was 56.4 units, compared to 42.7 units at the nonteaching site. This difference was highly statistically significant," (P less than .001), said Dr. Wang. The relative difference in ADR was 10.6%, but the relative AUC difference was 25%.

"As we move toward an era of greater accountability in medicine, our performance measures must become more, and not less, accurate. And I believe the AUC is a step in that direction," said Dr. Wang.

The study was sponsored by an NIH GI Training Grant and CURE: Digestive Diseases Research Center.

CHICAGO – A novel quality indicator called Adenomas Under the Curve might have important advantages over the established metric, the Adenoma Detection Rate, for evaluation of endoscopists’ performance, said Dr. Hank S. Wang at the annual Digestive Disease Week.

The adenoma detection rate (ADR) was developed by the U.S. Multisociety Task Force on Colorectal Cancer and is widely used. However, while it may have strengths, it also has glaring weaknesses – most notably the lack of distinction between detection of one adenoma, and detection of more than one, in a given patient, Dr. Wang said.

"An endoscopist who consistently finds one adenoma per case receives the same credit towards ADR as somebody who consistently finds more than one," said Dr. Wang, a gastroenterology fellow in the division of digestive diseases, University of California, Los Angeles, Training Program.

"Clearly these [two] endoscopists are performing differently, and over a large enough case series, they may even be providing different rates of protection against colorectal cancer. Yet by ADR, they would appear to be identical," he explained.

Use of ADR as a quality measure might actually discourage the search for additional adenomas, said Dr. Wang, or even promote "gaming the quality indicator" to maximize efficiency while still ensuring that credit is received for ADR. Some endoscopists may become less vigilant once they have found the first adenoma, and thus the use of ADR may create a subtle disincentive, he said.

The new metric developed by Dr. Wang and his colleagues, Adenomas Under the Curve (AUC), incorporates ADR data and provides additional quality data on the total number of adenomas detected. Thus AUC extends ADR by capturing data on incremental adenomas beyond the first one detected, said Dr. Wang.

The abstract presented by Dr. Wang compared ADR to AUC in two groups of physicians – academic and community – who were providing screening services for the same pool of patients. Screening colonoscopies were performed over a 4-year period (2005-2009) by two types of practices: a Veterans Affairs teaching hospital endoscopy unit, and a group of three nonteaching, fee-for-service facilities, all serving same population of veterans. The choice of facility was made by the veterans, and the analysis was adjusted for selection bias.

ADR was calculated by dividing the number of cases with one or more adenoma(s) by the total number of cases performed.

In contrast, calculating the AUC involved plotting a histogram showing the frequency of cases and the number of adenomas detected by case. To account for weighting, case frequency was multiplied by the number of adenomas detected per case. When these data were combined, they yielded the measure Adenomas Under the Curve.

A bivariate analysis was performed to evaluate comparability between the two groups of patients. ADR was compared between groups using bivariate and multivariable logistic regression. Relative difference in ADR vs. AUC, and between the two cohorts, was also calculated.

Of the total population of 3,302 patients, 1,216 patients presented to the teaching facility (there were 6 academic gastroenterologists) and 2,086 presented to the nonteaching facilities (12 community-based gastroenterologists). Patients at the nonteaching facilities were on average 1 year younger, and the endoscopists there were slightly more experienced. All patients in both arms presented for screening, and none had a known history of polyps.

The mean ADR at the teaching site was 28.9%, compared with 25.9% at the nonteaching sites, a small difference that did not reach statistical significance (P = .056). However, when the model was adjusted for differences such as age, sedation, furthest point seen, bowel prep type and quality, experience, and the presence of a fellow, the odds of adenoma detection were 43% higher at the teaching facility, a difference that was highly statistically significant (adjusted odds ratio, 1.43 [1.1-1.8], P = .003).

A plot of the AUC for the teaching site superimposed on the AUC for the nonteaching sites clearly showed that, after detection of the first adenoma at the nonteaching facilities, there was a steep drop-off in the number of cases in which two, three, or four adenomas were detected.

"The difference in AUC between the two sites is largely driven by this difference in the frequency of cases where two, three, and four adenomas are detected per case. We calculated AUC of our teaching facility; it was 56.4 units, compared to 42.7 units at the nonteaching site. This difference was highly statistically significant," (P less than .001), said Dr. Wang. The relative difference in ADR was 10.6%, but the relative AUC difference was 25%.

"As we move toward an era of greater accountability in medicine, our performance measures must become more, and not less, accurate. And I believe the AUC is a step in that direction," said Dr. Wang.

The study was sponsored by an NIH GI Training Grant and CURE: Digestive Diseases Research Center.