Ana Wilson, MD

In 1993, the National Polyp Study (NPS) was published in the New England Journal of Medicine and demonstrated the power of colonoscopy and polypectomy to reduce subsequent risk of colorectal cancer. The protocol for NPS included removal of all visible lesions and all were sent for pathological review. This process became standard practice in gastroenterology. Recently, the concept of “optical biopsy,” where hyperplastic polyps can be accurately identified and discarded in lieu of pathologic analysis might be safely accomplished, thus reducing costs without compromising patient health outcomes. Issues related to accuracy of optical biopsies, potential liability, and practice reimbursement have all been barriers to widespread implementation. In this month’s column, Dr. Kaltenbach and colleagues outline a process to standardize studies, training, and classification of optical biopsies; a needed step in the evolution of our colonoscopy practice.

John I Allen, M.D. , MBA, AGAF,
Special Section Editor

The potential application of optical diagnosis for diminutive colorectal polyp is at a crossroads. Recent studies have shown its feasibility; the diagnostic operating characteristics for the real-time diagnosis of diminutive colorectal polyps are similar to those of pathologists. These studies showed 93% concordance between the surveillance interval recommendations that are based on optical and pathologic diagnoses and ≥90% negative predictive value for polyps in the rectosigmoid colon.¹ These findings may open the applications of optical diagnosis for diminutive colorectal polyps in practice, which in turn may lead to improved cost-effectiveness of colonoscopy for colorectal cancer screening.²

However, some recent reports of optical diagnosis conducted beyond the academic setting did not reproduce the high levels of accuracy, eliciting reservation on the generalizability of optical diagnosis in practice. A variety of explanations could account for or contribute to these results. These studies (as well as some studies from academia) have not followed the key steps for a system redesign, the underlying basis for implementation of optical diagnosis. Because of the recent pattern of results, we propose a set of recommendations to be considered by investigators in the design of future studies. Our objective is to share the lessons learned from successful optical diagnosis studies¹ and thereby to suggest a framework in which to conduct and report such studies.

Designing an optical diagnosis study

General framework

The implementation of optical diagnosis, a system redesign, should be evidence-based and adopt a quality improvement model. It requires participants to recognize that learning is experiential: “a cyclic process of doing, noticing, questioning, reflecting, exploring concepts and models (evidence), and then doing again – only doing it better the next time (PDSA cycle)” [Supplementary Figure 1; www.cghjournal.org/article/S1542-3565(14)01469-4/fulltext].³ The iterative process of “checking” the correlation of endoscopic diagnosis to pathology findings is important. Without it, the study participants miss a significant opportunity to continuously improve the quality of their optical diagnoses.

Studies should commence once there is consistency in the ability to provide optical and pathology diagnoses, and this ability should be periodically checked, following the Plan-Do-Study-Act cycle. A study can only be successful when the participants remain interested in learning, engaged, and committed to the process. Published guidelines on the general framework for study conduct and standards for the reporting of results can be very useful. As such, the research team should be deliberate to include the key elements of diagnostic studies before, during, and after the study [Supplementary Table 1; www.cghjournal.org/article/S1542-3565(14)01469-4/fulltext]. These standards are necessary to minimize biased results from incompletely designed, conducted, or analyzed diagnostic studies.

Optical diagnosis specific framework

Training

The knowledge and skills required to perform optical diagnosis are not innate but can be learned by people with varying levels of expertise. As such, training modules have been developed and studied. In an early report of training of a short teaching session on optical diagnosis for the endoscopic differentiation of colorectal polyp histology, Raghavendra et al⁴ showed attainment of high accuracy (90.8%) and good interobserver agreement (kappa = 0.69) by using high-definition still photographs of polyps. Ignjatovic et al⁵ assessed the construct and content validity of a still image–based teaching module on the basic principles of narrow band imaging (NBI), the microvessel patterns, and the role of NBI in differentiating between adenomas and hyperplastic polyps. After training, they found improved accuracy and specificity of optical diagnosis in novices, trainees, and experts with moderate agreement (kappa = 0.56, 0.70, and 0.54, respectively). Rastogi et al⁶ showed the importance of active feedback to achieve high performance. After a 20-minute training module, community and academic practitioners reviewed 80 short clips of diminutive polyps, with feedback provided after each video. They made significant improvements in accuracy and the proportion of high confidence predictions as they progressed through consecutive video blocks of 20. Although none of the studies used consecutively collected images or video content and none assessed durability of performance after the training in real-time in vivo setting, their findings underscore the importance of learning before engagement in a formal study or the practice of optical diagnosis.

A teaching video entitled “Optical Diagnosis of Colorectal Polyps” is available through the American Society for Gastrointestinal Endoscopy On-line Learning Center. The program outlines the steps necessary to practice the technique. It provides a review of the concepts of optical diagnosis and numerous illustrative case examples.

Documentation of competence

The documentation of successful completion of training is important. The formal training should be based on a validated tool, should be periodic, and should include an in vivo component. Ex vivo competency should be assessed before evaluation of clinical performance. After achievement of ex vivo performance thresholds, study participants should then be evaluated in real time to ensure sustained performance before study initiation. Finally, and consistent with the Plan-Do-Study-Act quality improvement model, participants should undergo additional ex vivo testing periodically throughout the study to ensure sustained performance and evaluate the need for further training. By using this approach of regular self-training and a robust teaching tool, we observed no significant difference in a group of experienced endoscopists between performance in the first and second halves of the study. Agreement in surveillance interval recommendations between optical-based and pathology-based strategies exceeded 95% in both halves of the study.^7,8

Standardized optical diagnostic criteria

When feasible, investigators should use validated criteria for the endoscopic diagnosis of colorectal polyps. An example is the Narrow Band Imaging International Colorectal Endoscopic (NICE) classification by using NBI, which describes real-time differentiation of non-neoplastic (type 1) and neoplastic (type 2) colorectal polyps,⁹ as well as for lesions with deep submucosal invasion (type 3). Other endoscopic classifications of colorectal polyps by using NBI, i-Scan, or chromoendoscopy have been described with and without optical magnification but have not yet been validated.

Although sessile serrated adenoma/polyps exhibit features of non-neoplastic lesions, their distinction from hyperplastic polyps is challenging because of the variations in pathologic diagnoses. Until such endoscopic and pathologic distinctions are further described, investigated, and reproducible, it may be necessary to remove and submit to pathology all proximal and/or large NICE type 1 polyps.

Training should include a working definition of the application of confidence levels. The principle of confidence levels (high vs. low) is easily understood but lacks a formal operational definition. A recent study suggested that the speed of the diagnostic determination correlated with confidence, with a cutoff of 5 seconds predicting high confidence. Five expert endoscopists performed optical diagnosis of 1309 polyps in 558 patients. The average time to diagnosis was 20 seconds, and this was an independent predictor of accuracy. An optical diagnosis made in 5 seconds or less had an accuracy greater than 90%, with 90% high-confidence determinations; those made in 6–60 seconds had an accuracy of 85%, with 77% high confidence, and diagnoses that took more than 60 seconds had an accuracy of 68%, with only 64% high confidence.

Standardized pathologic diagnostic criteria

It is critically important to use a diagnostic standard for the colorectal polyp histopathology. The World Health Organization criteria have been the most used. It is key to recognize that even standardized pathology is not a gold standard but rather a reference standard, which is often less than 100% accurate. In clinical practice, many diminutive polyps are not retrieved after polypectomy (6%–13%), are unsuitable for analysis because of diathermy artifacts (7%–19%), or may be misclassified because of incorrect orientation or limited sectioning. Furthermore, errors in differentiating conventional adenomas from hyperplastic/serrated lesions by pathologists can be as high as 10%. This may be because the normal surrounding tissue in a polypectomy specimen of a diminutive lesion has been sectioned and interpreted. Efforts to strengthen pathology as a reference standard in clinical trials must be taken, including the need to have centralized blinded pathology reading and the need to perform re-cuts of the individual polyp specimens, especially those that are interpreted by pathologists as normal tissue.

Standardized outcomes

The ASGE Preservation and Incorporation of Valuable Endoscopic Innovations working group has established a priori diagnostic thresholds for real-time endoscopic assessment of the histology of diminutive colorectal polyps.¹⁰ These thresholds are meant to define clinically important roles for imaging technology and acceptable thresholds of performance for which ASGE could support their use as an alternative paradigm for management of diminutive polyps in clinical practice. These are the two proposed thresholds for optical diagnosis. (1) For colorectal polyps ≤5 mm in size to be resected and discarded without pathologic assessment, endoscopic technology (when used with high confidence) and histopathologic assessment of polyps ≤5 mm in size should provide ≥90% agreement in assignment of post-polypectomy surveillance intervals when compared with decisions that are based on pathology assessment of all identified polyps. (2) For a technology to be used to guide the decision to leave suspected rectosigmoid hyperplastic polyps ≤5 mm in size in place (without resection), the technology should provide ≥90% negative predictive value (when used with high confidence) for adenomatous histology.

We emphasize the importance of confidence levels in making an optical diagnosis. The use of confidence levels allows calibration and standardization between endoscopists with varying levels of diagnostic ability and reduces interobserver variation. Thus, if a polyp lacks clear endoscopic features precluding confident endoscopic assignment of histology, the endoscopist could still resect and submit it for pathologic assessment (Figure 1).

Documentation

Photo documentation and archiving are a key component in both the study and clinical implementation of optical diagnosis for accreditation and quality assurance.

The first step is to optimize the processor, monitor, and capture settings to display and capture high-quality representative images of the polyp. The endoscope manufacturer may assist to set optimal image parameters on the endoscope processor. Digital integration of the polyp image and optical diagnosis into the endoscopy reporting system is necessary for efficient real-time relay of information as well as reliability for review. Such archiving would permit both self and formal audits.

Limitations

Other factors may hinder the outcomes of a study of optical diagnosis. A lack of academic interest or lack of financial incentive may influence the commitment or performance during a study. Physicians with direct or indirect ownership in a pathology facility should at a minimum declare a potential conflict of interest.

Discussion

Our comments are directed only as an effort to make results of published trials more consistent. Whether and to what extent specific factors unique to community physicians may contribute to the recent less promising results remain uncertain. These and other as yet unconsidered factors will likely also affect academic physicians who do not have a special interest in endoscopy or in this endoscopic issue. Such variable performance in optical diagnosis could be interpreted to mean that optical diagnosis can and should be implemented in the context of a credentialing program. Thus, because many studies have met proposed thresholds and some have not, accurate optical diagnosis is possible, but individual physicians need to prove their skill to start the practice. Such a policy could be implemented in any practice whether academic or community-based.

The actual implementation of optical diagnosis must also address other obstacles. For example, there are often institutional policies requiring submission of resected tissue to pathology. Furthermore, adenoma detection rate (ADR) has emerged as the most important quality indicator in colonoscopy. In a resect and discard policy, ADR would have to be measured by photography, which would require endoscope manufacturers to provide image storage with quality that reproduces the image seen in real time and that can be easily audited by experts to verify ADR. Such image storage would also be necessary to provide medical-legal protection for endoscopists. Finally, the current fee-for-service reimbursement model does not result in optimal financial incentives to drive a resect and discard policy forward. However, certain reimbursement models under consideration such as bundled payment and reference payment could make resect and discard more attractive to endoscopists.

Conclusion

We hope that the framework we describe will be useful in improving the accuracy, completeness of reporting, and meta-analysis of future studies of the diagnostic characteristics of optical diagnosis, with the ultimate goal of incorporating this paradigm shift into routine day-to-day clinical practice.

Supplementary Material

Supplementary Figure 1

References

1. McGill, S.K., Evangelou, E., Ioannidis, J.P., et al. Narrow band imaging to differentiate neoplastic and non-neoplastic colorectal polyps in real time: a meta-analysis of diagnostic operating characteristics. Gut 2013;62:1704-13.

2 . Hassan, C., Pickhardt, P.J., Rex, D.K. A resect and discard strategy would improve cost-effectiveness of colorectal cancer screening. Clin. Gastroenterol. Hepatol. 2010;8:865-9.

3. Glasziou, P., Ogrinc, G., Goodman, S. Can evidence-based medicine and clinical quality improvement learn from each other? BMJ Qual. Saf. 2011;20: i13-i17.

4. Raghavendra, M., Hewett, D.G., Rex, D.K. Differentiating adenomas from hyperplastic colorectal polyps: narrow-band imaging can be learned in 20 minutes. Gastrointest. Endosc. 2010;72:572-6.

5. Ignjatovic, A., Thomas-Gibson, S., East, J.E., et al. Development and validation of a training module on the use of narrow-band imaging in differentiation of small adenomas from hyperplastic colorectal polyps. Gastrointest. Endosc. 2011;73:128-33.

6. Rastogi, A., Rao, S.D., Gupta, N., et al. Impact of a computer-based teaching module on characterization of diminutive colon polyps by using narrow band imaging by non-experts in academic and community practice: a video-based study. Gastrointest. Endosc. 2014;79:390-8.

7. McGill, S.K., Soetikno, R., Rastogi, A., et al. Endoscopists can sustain high performance for the optical diagnosis of colorectal polyps following standardized and continued training. Endoscopy 2014 Sep 29; (Epub ahead of print).

8. Kaltenbach, T., Rastogi, R., Rouse, R.V., et al. Real-time optical diagnosis of diminutive colorectal polyps using narrow band imaging: The VALID randomised clinical trial. Gut 2014 Nov 11; (Epub ahead of print).

9. Hewett, D.G., Kaltenbach, T., Sano, Y., et al. Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging. Gastroenterology 2012; 143: 599-607.

10. Rex, D.K., Kahi, C., O’Brien, M., et al. The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest. Endosc. 2011;73:419-22.

Dr. Kaltenbach is a gastroenterologist at Veterans Affairs in Palo Alto and Stanford University School of Medicine, Calif; she has received consulting fees and research support from Olympus America Corporation; Dr. Rex is Dr. Douglas K. Rex is a distinguished professor of medicine at Indiana University School of Medicine, Chancellor’s Professor at Indiana University Purdue University Indianapolis, and director of endoscopy at Indiana University Hospital in Indianapolis; he has received consulting fees and research support from Olympus America Corporation; Dr. Wilson is a consultant gastroenterologist and endoscopist, St. Mark’s Hospital and Academic Institute, Harrow, U.K.; she has received research support from Olympus Medical Corporation, United Kingdom; Dr. Hewett is an associate professor at the University of Queensland, Brisbane, Australia, and he has received consulting fees from Olympus Medical Systems Corporation, Tokyo and Boston Scientific, Sydney. Dr. Sanduleanu is a gastroenterologist at Maastricht University, and she received research support from Pentax Medical Corporation. Dr. Rastogi is a gastroenterologist at the Kansas City Veterans Affairs Medical Center, he received consulting fees and research support from Olympus America Corporation. Dr. Wallace is a gastroenterologist affiliated with Mayo Clinic in Jacksonville, Fla., and has received research support from Olympus America Corporation and Ninepoint Medical. Dr. Soeitkno is a gastroenterologist at Veterans Affairs in Palo Alto and Stanford University School of Medicine, Calif; he received consulting fees and research support from Olympus America Corporation.

[email protected]

In 1993, the National Polyp Study (NPS) was published in the New England Journal of Medicine and demonstrated the power of colonoscopy and polypectomy to reduce subsequent risk of colorectal cancer. The protocol for NPS included removal of all visible lesions and all were sent for pathological review. This process became standard practice in gastroenterology. Recently, the concept of “optical biopsy,” where hyperplastic polyps can be accurately identified and discarded in lieu of pathologic analysis might be safely accomplished, thus reducing costs without compromising patient health outcomes. Issues related to accuracy of optical biopsies, potential liability, and practice reimbursement have all been barriers to widespread implementation. In this month’s column, Dr. Kaltenbach and colleagues outline a process to standardize studies, training, and classification of optical biopsies; a needed step in the evolution of our colonoscopy practice.

John I Allen, M.D. , MBA, AGAF,
Special Section Editor

The potential application of optical diagnosis for diminutive colorectal polyp is at a crossroads. Recent studies have shown its feasibility; the diagnostic operating characteristics for the real-time diagnosis of diminutive colorectal polyps are similar to those of pathologists. These studies showed 93% concordance between the surveillance interval recommendations that are based on optical and pathologic diagnoses and ≥90% negative predictive value for polyps in the rectosigmoid colon.¹ These findings may open the applications of optical diagnosis for diminutive colorectal polyps in practice, which in turn may lead to improved cost-effectiveness of colonoscopy for colorectal cancer screening.²

However, some recent reports of optical diagnosis conducted beyond the academic setting did not reproduce the high levels of accuracy, eliciting reservation on the generalizability of optical diagnosis in practice. A variety of explanations could account for or contribute to these results. These studies (as well as some studies from academia) have not followed the key steps for a system redesign, the underlying basis for implementation of optical diagnosis. Because of the recent pattern of results, we propose a set of recommendations to be considered by investigators in the design of future studies. Our objective is to share the lessons learned from successful optical diagnosis studies¹ and thereby to suggest a framework in which to conduct and report such studies.

Designing an optical diagnosis study

General framework

The implementation of optical diagnosis, a system redesign, should be evidence-based and adopt a quality improvement model. It requires participants to recognize that learning is experiential: “a cyclic process of doing, noticing, questioning, reflecting, exploring concepts and models (evidence), and then doing again – only doing it better the next time (PDSA cycle)” [Supplementary Figure 1; www.cghjournal.org/article/S1542-3565(14)01469-4/fulltext].³ The iterative process of “checking” the correlation of endoscopic diagnosis to pathology findings is important. Without it, the study participants miss a significant opportunity to continuously improve the quality of their optical diagnoses.

Studies should commence once there is consistency in the ability to provide optical and pathology diagnoses, and this ability should be periodically checked, following the Plan-Do-Study-Act cycle. A study can only be successful when the participants remain interested in learning, engaged, and committed to the process. Published guidelines on the general framework for study conduct and standards for the reporting of results can be very useful. As such, the research team should be deliberate to include the key elements of diagnostic studies before, during, and after the study [Supplementary Table 1; www.cghjournal.org/article/S1542-3565(14)01469-4/fulltext]. These standards are necessary to minimize biased results from incompletely designed, conducted, or analyzed diagnostic studies.

Optical diagnosis specific framework

Training

The knowledge and skills required to perform optical diagnosis are not innate but can be learned by people with varying levels of expertise. As such, training modules have been developed and studied. In an early report of training of a short teaching session on optical diagnosis for the endoscopic differentiation of colorectal polyp histology, Raghavendra et al⁴ showed attainment of high accuracy (90.8%) and good interobserver agreement (kappa = 0.69) by using high-definition still photographs of polyps. Ignjatovic et al⁵ assessed the construct and content validity of a still image–based teaching module on the basic principles of narrow band imaging (NBI), the microvessel patterns, and the role of NBI in differentiating between adenomas and hyperplastic polyps. After training, they found improved accuracy and specificity of optical diagnosis in novices, trainees, and experts with moderate agreement (kappa = 0.56, 0.70, and 0.54, respectively). Rastogi et al⁶ showed the importance of active feedback to achieve high performance. After a 20-minute training module, community and academic practitioners reviewed 80 short clips of diminutive polyps, with feedback provided after each video. They made significant improvements in accuracy and the proportion of high confidence predictions as they progressed through consecutive video blocks of 20. Although none of the studies used consecutively collected images or video content and none assessed durability of performance after the training in real-time in vivo setting, their findings underscore the importance of learning before engagement in a formal study or the practice of optical diagnosis.

A teaching video entitled “Optical Diagnosis of Colorectal Polyps” is available through the American Society for Gastrointestinal Endoscopy On-line Learning Center. The program outlines the steps necessary to practice the technique. It provides a review of the concepts of optical diagnosis and numerous illustrative case examples.

Documentation of competence

The documentation of successful completion of training is important. The formal training should be based on a validated tool, should be periodic, and should include an in vivo component. Ex vivo competency should be assessed before evaluation of clinical performance. After achievement of ex vivo performance thresholds, study participants should then be evaluated in real time to ensure sustained performance before study initiation. Finally, and consistent with the Plan-Do-Study-Act quality improvement model, participants should undergo additional ex vivo testing periodically throughout the study to ensure sustained performance and evaluate the need for further training. By using this approach of regular self-training and a robust teaching tool, we observed no significant difference in a group of experienced endoscopists between performance in the first and second halves of the study. Agreement in surveillance interval recommendations between optical-based and pathology-based strategies exceeded 95% in both halves of the study.^7,8

Standardized optical diagnostic criteria

When feasible, investigators should use validated criteria for the endoscopic diagnosis of colorectal polyps. An example is the Narrow Band Imaging International Colorectal Endoscopic (NICE) classification by using NBI, which describes real-time differentiation of non-neoplastic (type 1) and neoplastic (type 2) colorectal polyps,⁹ as well as for lesions with deep submucosal invasion (type 3). Other endoscopic classifications of colorectal polyps by using NBI, i-Scan, or chromoendoscopy have been described with and without optical magnification but have not yet been validated.

Although sessile serrated adenoma/polyps exhibit features of non-neoplastic lesions, their distinction from hyperplastic polyps is challenging because of the variations in pathologic diagnoses. Until such endoscopic and pathologic distinctions are further described, investigated, and reproducible, it may be necessary to remove and submit to pathology all proximal and/or large NICE type 1 polyps.

Training should include a working definition of the application of confidence levels. The principle of confidence levels (high vs. low) is easily understood but lacks a formal operational definition. A recent study suggested that the speed of the diagnostic determination correlated with confidence, with a cutoff of 5 seconds predicting high confidence. Five expert endoscopists performed optical diagnosis of 1309 polyps in 558 patients. The average time to diagnosis was 20 seconds, and this was an independent predictor of accuracy. An optical diagnosis made in 5 seconds or less had an accuracy greater than 90%, with 90% high-confidence determinations; those made in 6–60 seconds had an accuracy of 85%, with 77% high confidence, and diagnoses that took more than 60 seconds had an accuracy of 68%, with only 64% high confidence.

Standardized pathologic diagnostic criteria

It is critically important to use a diagnostic standard for the colorectal polyp histopathology. The World Health Organization criteria have been the most used. It is key to recognize that even standardized pathology is not a gold standard but rather a reference standard, which is often less than 100% accurate. In clinical practice, many diminutive polyps are not retrieved after polypectomy (6%–13%), are unsuitable for analysis because of diathermy artifacts (7%–19%), or may be misclassified because of incorrect orientation or limited sectioning. Furthermore, errors in differentiating conventional adenomas from hyperplastic/serrated lesions by pathologists can be as high as 10%. This may be because the normal surrounding tissue in a polypectomy specimen of a diminutive lesion has been sectioned and interpreted. Efforts to strengthen pathology as a reference standard in clinical trials must be taken, including the need to have centralized blinded pathology reading and the need to perform re-cuts of the individual polyp specimens, especially those that are interpreted by pathologists as normal tissue.

Standardized outcomes

The ASGE Preservation and Incorporation of Valuable Endoscopic Innovations working group has established a priori diagnostic thresholds for real-time endoscopic assessment of the histology of diminutive colorectal polyps.¹⁰ These thresholds are meant to define clinically important roles for imaging technology and acceptable thresholds of performance for which ASGE could support their use as an alternative paradigm for management of diminutive polyps in clinical practice. These are the two proposed thresholds for optical diagnosis. (1) For colorectal polyps ≤5 mm in size to be resected and discarded without pathologic assessment, endoscopic technology (when used with high confidence) and histopathologic assessment of polyps ≤5 mm in size should provide ≥90% agreement in assignment of post-polypectomy surveillance intervals when compared with decisions that are based on pathology assessment of all identified polyps. (2) For a technology to be used to guide the decision to leave suspected rectosigmoid hyperplastic polyps ≤5 mm in size in place (without resection), the technology should provide ≥90% negative predictive value (when used with high confidence) for adenomatous histology.

We emphasize the importance of confidence levels in making an optical diagnosis. The use of confidence levels allows calibration and standardization between endoscopists with varying levels of diagnostic ability and reduces interobserver variation. Thus, if a polyp lacks clear endoscopic features precluding confident endoscopic assignment of histology, the endoscopist could still resect and submit it for pathologic assessment (Figure 1).

Documentation

Photo documentation and archiving are a key component in both the study and clinical implementation of optical diagnosis for accreditation and quality assurance.

The first step is to optimize the processor, monitor, and capture settings to display and capture high-quality representative images of the polyp. The endoscope manufacturer may assist to set optimal image parameters on the endoscope processor. Digital integration of the polyp image and optical diagnosis into the endoscopy reporting system is necessary for efficient real-time relay of information as well as reliability for review. Such archiving would permit both self and formal audits.

Limitations

Other factors may hinder the outcomes of a study of optical diagnosis. A lack of academic interest or lack of financial incentive may influence the commitment or performance during a study. Physicians with direct or indirect ownership in a pathology facility should at a minimum declare a potential conflict of interest.

Discussion

Our comments are directed only as an effort to make results of published trials more consistent. Whether and to what extent specific factors unique to community physicians may contribute to the recent less promising results remain uncertain. These and other as yet unconsidered factors will likely also affect academic physicians who do not have a special interest in endoscopy or in this endoscopic issue. Such variable performance in optical diagnosis could be interpreted to mean that optical diagnosis can and should be implemented in the context of a credentialing program. Thus, because many studies have met proposed thresholds and some have not, accurate optical diagnosis is possible, but individual physicians need to prove their skill to start the practice. Such a policy could be implemented in any practice whether academic or community-based.

The actual implementation of optical diagnosis must also address other obstacles. For example, there are often institutional policies requiring submission of resected tissue to pathology. Furthermore, adenoma detection rate (ADR) has emerged as the most important quality indicator in colonoscopy. In a resect and discard policy, ADR would have to be measured by photography, which would require endoscope manufacturers to provide image storage with quality that reproduces the image seen in real time and that can be easily audited by experts to verify ADR. Such image storage would also be necessary to provide medical-legal protection for endoscopists. Finally, the current fee-for-service reimbursement model does not result in optimal financial incentives to drive a resect and discard policy forward. However, certain reimbursement models under consideration such as bundled payment and reference payment could make resect and discard more attractive to endoscopists.

Conclusion

We hope that the framework we describe will be useful in improving the accuracy, completeness of reporting, and meta-analysis of future studies of the diagnostic characteristics of optical diagnosis, with the ultimate goal of incorporating this paradigm shift into routine day-to-day clinical practice.

Supplementary Material

Supplementary Figure 1

References

1. McGill, S.K., Evangelou, E., Ioannidis, J.P., et al. Narrow band imaging to differentiate neoplastic and non-neoplastic colorectal polyps in real time: a meta-analysis of diagnostic operating characteristics. Gut 2013;62:1704-13.

2 . Hassan, C., Pickhardt, P.J., Rex, D.K. A resect and discard strategy would improve cost-effectiveness of colorectal cancer screening. Clin. Gastroenterol. Hepatol. 2010;8:865-9.

3. Glasziou, P., Ogrinc, G., Goodman, S. Can evidence-based medicine and clinical quality improvement learn from each other? BMJ Qual. Saf. 2011;20: i13-i17.

4. Raghavendra, M., Hewett, D.G., Rex, D.K. Differentiating adenomas from hyperplastic colorectal polyps: narrow-band imaging can be learned in 20 minutes. Gastrointest. Endosc. 2010;72:572-6.

5. Ignjatovic, A., Thomas-Gibson, S., East, J.E., et al. Development and validation of a training module on the use of narrow-band imaging in differentiation of small adenomas from hyperplastic colorectal polyps. Gastrointest. Endosc. 2011;73:128-33.

6. Rastogi, A., Rao, S.D., Gupta, N., et al. Impact of a computer-based teaching module on characterization of diminutive colon polyps by using narrow band imaging by non-experts in academic and community practice: a video-based study. Gastrointest. Endosc. 2014;79:390-8.

7. McGill, S.K., Soetikno, R., Rastogi, A., et al. Endoscopists can sustain high performance for the optical diagnosis of colorectal polyps following standardized and continued training. Endoscopy 2014 Sep 29; (Epub ahead of print).

8. Kaltenbach, T., Rastogi, R., Rouse, R.V., et al. Real-time optical diagnosis of diminutive colorectal polyps using narrow band imaging: The VALID randomised clinical trial. Gut 2014 Nov 11; (Epub ahead of print).

9. Hewett, D.G., Kaltenbach, T., Sano, Y., et al. Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging. Gastroenterology 2012; 143: 599-607.

10. Rex, D.K., Kahi, C., O’Brien, M., et al. The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest. Endosc. 2011;73:419-22.

Dr. Kaltenbach is a gastroenterologist at Veterans Affairs in Palo Alto and Stanford University School of Medicine, Calif; she has received consulting fees and research support from Olympus America Corporation; Dr. Rex is Dr. Douglas K. Rex is a distinguished professor of medicine at Indiana University School of Medicine, Chancellor’s Professor at Indiana University Purdue University Indianapolis, and director of endoscopy at Indiana University Hospital in Indianapolis; he has received consulting fees and research support from Olympus America Corporation; Dr. Wilson is a consultant gastroenterologist and endoscopist, St. Mark’s Hospital and Academic Institute, Harrow, U.K.; she has received research support from Olympus Medical Corporation, United Kingdom; Dr. Hewett is an associate professor at the University of Queensland, Brisbane, Australia, and he has received consulting fees from Olympus Medical Systems Corporation, Tokyo and Boston Scientific, Sydney. Dr. Sanduleanu is a gastroenterologist at Maastricht University, and she received research support from Pentax Medical Corporation. Dr. Rastogi is a gastroenterologist at the Kansas City Veterans Affairs Medical Center, he received consulting fees and research support from Olympus America Corporation. Dr. Wallace is a gastroenterologist affiliated with Mayo Clinic in Jacksonville, Fla., and has received research support from Olympus America Corporation and Ninepoint Medical. Dr. Soeitkno is a gastroenterologist at Veterans Affairs in Palo Alto and Stanford University School of Medicine, Calif; he received consulting fees and research support from Olympus America Corporation.

[email protected]

In 1993, the National Polyp Study (NPS) was published in the New England Journal of Medicine and demonstrated the power of colonoscopy and polypectomy to reduce subsequent risk of colorectal cancer. The protocol for NPS included removal of all visible lesions and all were sent for pathological review. This process became standard practice in gastroenterology. Recently, the concept of “optical biopsy,” where hyperplastic polyps can be accurately identified and discarded in lieu of pathologic analysis might be safely accomplished, thus reducing costs without compromising patient health outcomes. Issues related to accuracy of optical biopsies, potential liability, and practice reimbursement have all been barriers to widespread implementation. In this month’s column, Dr. Kaltenbach and colleagues outline a process to standardize studies, training, and classification of optical biopsies; a needed step in the evolution of our colonoscopy practice.

John I Allen, M.D. , MBA, AGAF,
Special Section Editor

The potential application of optical diagnosis for diminutive colorectal polyp is at a crossroads. Recent studies have shown its feasibility; the diagnostic operating characteristics for the real-time diagnosis of diminutive colorectal polyps are similar to those of pathologists. These studies showed 93% concordance between the surveillance interval recommendations that are based on optical and pathologic diagnoses and ≥90% negative predictive value for polyps in the rectosigmoid colon.¹ These findings may open the applications of optical diagnosis for diminutive colorectal polyps in practice, which in turn may lead to improved cost-effectiveness of colonoscopy for colorectal cancer screening.²

However, some recent reports of optical diagnosis conducted beyond the academic setting did not reproduce the high levels of accuracy, eliciting reservation on the generalizability of optical diagnosis in practice. A variety of explanations could account for or contribute to these results. These studies (as well as some studies from academia) have not followed the key steps for a system redesign, the underlying basis for implementation of optical diagnosis. Because of the recent pattern of results, we propose a set of recommendations to be considered by investigators in the design of future studies. Our objective is to share the lessons learned from successful optical diagnosis studies¹ and thereby to suggest a framework in which to conduct and report such studies.

Designing an optical diagnosis study

General framework

The implementation of optical diagnosis, a system redesign, should be evidence-based and adopt a quality improvement model. It requires participants to recognize that learning is experiential: “a cyclic process of doing, noticing, questioning, reflecting, exploring concepts and models (evidence), and then doing again – only doing it better the next time (PDSA cycle)” [Supplementary Figure 1; www.cghjournal.org/article/S1542-3565(14)01469-4/fulltext].³ The iterative process of “checking” the correlation of endoscopic diagnosis to pathology findings is important. Without it, the study participants miss a significant opportunity to continuously improve the quality of their optical diagnoses.

Studies should commence once there is consistency in the ability to provide optical and pathology diagnoses, and this ability should be periodically checked, following the Plan-Do-Study-Act cycle. A study can only be successful when the participants remain interested in learning, engaged, and committed to the process. Published guidelines on the general framework for study conduct and standards for the reporting of results can be very useful. As such, the research team should be deliberate to include the key elements of diagnostic studies before, during, and after the study [Supplementary Table 1; www.cghjournal.org/article/S1542-3565(14)01469-4/fulltext]. These standards are necessary to minimize biased results from incompletely designed, conducted, or analyzed diagnostic studies.

Optical diagnosis specific framework

Training

The knowledge and skills required to perform optical diagnosis are not innate but can be learned by people with varying levels of expertise. As such, training modules have been developed and studied. In an early report of training of a short teaching session on optical diagnosis for the endoscopic differentiation of colorectal polyp histology, Raghavendra et al⁴ showed attainment of high accuracy (90.8%) and good interobserver agreement (kappa = 0.69) by using high-definition still photographs of polyps. Ignjatovic et al⁵ assessed the construct and content validity of a still image–based teaching module on the basic principles of narrow band imaging (NBI), the microvessel patterns, and the role of NBI in differentiating between adenomas and hyperplastic polyps. After training, they found improved accuracy and specificity of optical diagnosis in novices, trainees, and experts with moderate agreement (kappa = 0.56, 0.70, and 0.54, respectively). Rastogi et al⁶ showed the importance of active feedback to achieve high performance. After a 20-minute training module, community and academic practitioners reviewed 80 short clips of diminutive polyps, with feedback provided after each video. They made significant improvements in accuracy and the proportion of high confidence predictions as they progressed through consecutive video blocks of 20. Although none of the studies used consecutively collected images or video content and none assessed durability of performance after the training in real-time in vivo setting, their findings underscore the importance of learning before engagement in a formal study or the practice of optical diagnosis.

A teaching video entitled “Optical Diagnosis of Colorectal Polyps” is available through the American Society for Gastrointestinal Endoscopy On-line Learning Center. The program outlines the steps necessary to practice the technique. It provides a review of the concepts of optical diagnosis and numerous illustrative case examples.

Documentation of competence

The documentation of successful completion of training is important. The formal training should be based on a validated tool, should be periodic, and should include an in vivo component. Ex vivo competency should be assessed before evaluation of clinical performance. After achievement of ex vivo performance thresholds, study participants should then be evaluated in real time to ensure sustained performance before study initiation. Finally, and consistent with the Plan-Do-Study-Act quality improvement model, participants should undergo additional ex vivo testing periodically throughout the study to ensure sustained performance and evaluate the need for further training. By using this approach of regular self-training and a robust teaching tool, we observed no significant difference in a group of experienced endoscopists between performance in the first and second halves of the study. Agreement in surveillance interval recommendations between optical-based and pathology-based strategies exceeded 95% in both halves of the study.^7,8

Standardized optical diagnostic criteria

When feasible, investigators should use validated criteria for the endoscopic diagnosis of colorectal polyps. An example is the Narrow Band Imaging International Colorectal Endoscopic (NICE) classification by using NBI, which describes real-time differentiation of non-neoplastic (type 1) and neoplastic (type 2) colorectal polyps,⁹ as well as for lesions with deep submucosal invasion (type 3). Other endoscopic classifications of colorectal polyps by using NBI, i-Scan, or chromoendoscopy have been described with and without optical magnification but have not yet been validated.

Although sessile serrated adenoma/polyps exhibit features of non-neoplastic lesions, their distinction from hyperplastic polyps is challenging because of the variations in pathologic diagnoses. Until such endoscopic and pathologic distinctions are further described, investigated, and reproducible, it may be necessary to remove and submit to pathology all proximal and/or large NICE type 1 polyps.

Training should include a working definition of the application of confidence levels. The principle of confidence levels (high vs. low) is easily understood but lacks a formal operational definition. A recent study suggested that the speed of the diagnostic determination correlated with confidence, with a cutoff of 5 seconds predicting high confidence. Five expert endoscopists performed optical diagnosis of 1309 polyps in 558 patients. The average time to diagnosis was 20 seconds, and this was an independent predictor of accuracy. An optical diagnosis made in 5 seconds or less had an accuracy greater than 90%, with 90% high-confidence determinations; those made in 6–60 seconds had an accuracy of 85%, with 77% high confidence, and diagnoses that took more than 60 seconds had an accuracy of 68%, with only 64% high confidence.

Standardized pathologic diagnostic criteria

It is critically important to use a diagnostic standard for the colorectal polyp histopathology. The World Health Organization criteria have been the most used. It is key to recognize that even standardized pathology is not a gold standard but rather a reference standard, which is often less than 100% accurate. In clinical practice, many diminutive polyps are not retrieved after polypectomy (6%–13%), are unsuitable for analysis because of diathermy artifacts (7%–19%), or may be misclassified because of incorrect orientation or limited sectioning. Furthermore, errors in differentiating conventional adenomas from hyperplastic/serrated lesions by pathologists can be as high as 10%. This may be because the normal surrounding tissue in a polypectomy specimen of a diminutive lesion has been sectioned and interpreted. Efforts to strengthen pathology as a reference standard in clinical trials must be taken, including the need to have centralized blinded pathology reading and the need to perform re-cuts of the individual polyp specimens, especially those that are interpreted by pathologists as normal tissue.

Standardized outcomes

The ASGE Preservation and Incorporation of Valuable Endoscopic Innovations working group has established a priori diagnostic thresholds for real-time endoscopic assessment of the histology of diminutive colorectal polyps.¹⁰ These thresholds are meant to define clinically important roles for imaging technology and acceptable thresholds of performance for which ASGE could support their use as an alternative paradigm for management of diminutive polyps in clinical practice. These are the two proposed thresholds for optical diagnosis. (1) For colorectal polyps ≤5 mm in size to be resected and discarded without pathologic assessment, endoscopic technology (when used with high confidence) and histopathologic assessment of polyps ≤5 mm in size should provide ≥90% agreement in assignment of post-polypectomy surveillance intervals when compared with decisions that are based on pathology assessment of all identified polyps. (2) For a technology to be used to guide the decision to leave suspected rectosigmoid hyperplastic polyps ≤5 mm in size in place (without resection), the technology should provide ≥90% negative predictive value (when used with high confidence) for adenomatous histology.

We emphasize the importance of confidence levels in making an optical diagnosis. The use of confidence levels allows calibration and standardization between endoscopists with varying levels of diagnostic ability and reduces interobserver variation. Thus, if a polyp lacks clear endoscopic features precluding confident endoscopic assignment of histology, the endoscopist could still resect and submit it for pathologic assessment (Figure 1).

Documentation

Photo documentation and archiving are a key component in both the study and clinical implementation of optical diagnosis for accreditation and quality assurance.

The first step is to optimize the processor, monitor, and capture settings to display and capture high-quality representative images of the polyp. The endoscope manufacturer may assist to set optimal image parameters on the endoscope processor. Digital integration of the polyp image and optical diagnosis into the endoscopy reporting system is necessary for efficient real-time relay of information as well as reliability for review. Such archiving would permit both self and formal audits.

Limitations

Other factors may hinder the outcomes of a study of optical diagnosis. A lack of academic interest or lack of financial incentive may influence the commitment or performance during a study. Physicians with direct or indirect ownership in a pathology facility should at a minimum declare a potential conflict of interest.

Discussion

Our comments are directed only as an effort to make results of published trials more consistent. Whether and to what extent specific factors unique to community physicians may contribute to the recent less promising results remain uncertain. These and other as yet unconsidered factors will likely also affect academic physicians who do not have a special interest in endoscopy or in this endoscopic issue. Such variable performance in optical diagnosis could be interpreted to mean that optical diagnosis can and should be implemented in the context of a credentialing program. Thus, because many studies have met proposed thresholds and some have not, accurate optical diagnosis is possible, but individual physicians need to prove their skill to start the practice. Such a policy could be implemented in any practice whether academic or community-based.

The actual implementation of optical diagnosis must also address other obstacles. For example, there are often institutional policies requiring submission of resected tissue to pathology. Furthermore, adenoma detection rate (ADR) has emerged as the most important quality indicator in colonoscopy. In a resect and discard policy, ADR would have to be measured by photography, which would require endoscope manufacturers to provide image storage with quality that reproduces the image seen in real time and that can be easily audited by experts to verify ADR. Such image storage would also be necessary to provide medical-legal protection for endoscopists. Finally, the current fee-for-service reimbursement model does not result in optimal financial incentives to drive a resect and discard policy forward. However, certain reimbursement models under consideration such as bundled payment and reference payment could make resect and discard more attractive to endoscopists.

Conclusion

We hope that the framework we describe will be useful in improving the accuracy, completeness of reporting, and meta-analysis of future studies of the diagnostic characteristics of optical diagnosis, with the ultimate goal of incorporating this paradigm shift into routine day-to-day clinical practice.

Supplementary Material

Supplementary Figure 1

References

1. McGill, S.K., Evangelou, E., Ioannidis, J.P., et al. Narrow band imaging to differentiate neoplastic and non-neoplastic colorectal polyps in real time: a meta-analysis of diagnostic operating characteristics. Gut 2013;62:1704-13.

2 . Hassan, C., Pickhardt, P.J., Rex, D.K. A resect and discard strategy would improve cost-effectiveness of colorectal cancer screening. Clin. Gastroenterol. Hepatol. 2010;8:865-9.

3. Glasziou, P., Ogrinc, G., Goodman, S. Can evidence-based medicine and clinical quality improvement learn from each other? BMJ Qual. Saf. 2011;20: i13-i17.

4. Raghavendra, M., Hewett, D.G., Rex, D.K. Differentiating adenomas from hyperplastic colorectal polyps: narrow-band imaging can be learned in 20 minutes. Gastrointest. Endosc. 2010;72:572-6.

5. Ignjatovic, A., Thomas-Gibson, S., East, J.E., et al. Development and validation of a training module on the use of narrow-band imaging in differentiation of small adenomas from hyperplastic colorectal polyps. Gastrointest. Endosc. 2011;73:128-33.

6. Rastogi, A., Rao, S.D., Gupta, N., et al. Impact of a computer-based teaching module on characterization of diminutive colon polyps by using narrow band imaging by non-experts in academic and community practice: a video-based study. Gastrointest. Endosc. 2014;79:390-8.

7. McGill, S.K., Soetikno, R., Rastogi, A., et al. Endoscopists can sustain high performance for the optical diagnosis of colorectal polyps following standardized and continued training. Endoscopy 2014 Sep 29; (Epub ahead of print).

8. Kaltenbach, T., Rastogi, R., Rouse, R.V., et al. Real-time optical diagnosis of diminutive colorectal polyps using narrow band imaging: The VALID randomised clinical trial. Gut 2014 Nov 11; (Epub ahead of print).

9. Hewett, D.G., Kaltenbach, T., Sano, Y., et al. Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging. Gastroenterology 2012; 143: 599-607.

10. Rex, D.K., Kahi, C., O’Brien, M., et al. The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest. Endosc. 2011;73:419-22.

Dr. Kaltenbach is a gastroenterologist at Veterans Affairs in Palo Alto and Stanford University School of Medicine, Calif; she has received consulting fees and research support from Olympus America Corporation; Dr. Rex is Dr. Douglas K. Rex is a distinguished professor of medicine at Indiana University School of Medicine, Chancellor’s Professor at Indiana University Purdue University Indianapolis, and director of endoscopy at Indiana University Hospital in Indianapolis; he has received consulting fees and research support from Olympus America Corporation; Dr. Wilson is a consultant gastroenterologist and endoscopist, St. Mark’s Hospital and Academic Institute, Harrow, U.K.; she has received research support from Olympus Medical Corporation, United Kingdom; Dr. Hewett is an associate professor at the University of Queensland, Brisbane, Australia, and he has received consulting fees from Olympus Medical Systems Corporation, Tokyo and Boston Scientific, Sydney. Dr. Sanduleanu is a gastroenterologist at Maastricht University, and she received research support from Pentax Medical Corporation. Dr. Rastogi is a gastroenterologist at the Kansas City Veterans Affairs Medical Center, he received consulting fees and research support from Olympus America Corporation. Dr. Wallace is a gastroenterologist affiliated with Mayo Clinic in Jacksonville, Fla., and has received research support from Olympus America Corporation and Ninepoint Medical. Dr. Soeitkno is a gastroenterologist at Veterans Affairs in Palo Alto and Stanford University School of Medicine, Calif; he received consulting fees and research support from Olympus America Corporation.

[email protected]