Home / Specialties / Ophthalmology / Counterpoint: The Pros and Cons of AI-based “Diagnosis” of Diabetic Retinopathy

Counterpoint: The Pros and Cons of AI-based “Diagnosis” of Diabetic Retinopathy

Apr 28, 2018
 

By A. Paul Chous, MA, OD, FAAO, CDE

The FDA just gave first approval to an artificial intelligence (AI) algorithm for the detection of diabetic retinopathy in the offices of non-ophthalmic health care practitioners [1]. Dubbed the IDx-DR (IDx, LLC, Coralville, Iowa), and paired with a Topcon NW400 non-mydriatic retinal camera, captured images are sent to a cloud-based server that utilizes the IDx-DR software and a ‘deep learning’ algorithm to detect retinal findings consistent with diabetic retinopathy based on autonomous comparison with a large dataset of representative fundus images. The FDA statement says that if captured images “are of sufficient quality, the software provides the doctor with one of two results: (1) “more than mild diabetic retinopathy detected: refer to an eye care professional” or (2) “negative for more than mild diabetic retinopathy; rescreen in 12 months.”

Further guidance from the FDA states that, “If a positive result is detected, patients should see an eye care provider for further diagnostic evaluation and possible treatment as soon as possible.”

Approval was based on submission of a 900-subject study using IDx-DR in a primary care setting (10 sites) with automated image analysis of two, 45 degree digital images per eye (one centered on the macula and one on the optic nerve). This was compared against stereo, wide-field fundus imaging interpreted by the Wisconsin Fundus Photograph Reading Center (FPRC) based on the Early Treatment Diabetic Retinopathy Study Severity Scale (ETDRS) and wide-field stereo photographs combined with macular optical coherence tomography (OCT) for detection of diabetic macular edema (DME) [2]. Ninety-six percent of images acquired in primary care were deemed of sufficient quality for algorithmic assessment after human operators received 4-hours of training with the Topcon-IDx-DR system.

The technology was 87% sensitive and 90% specific for detecting more than mild diabetic retinopathy. Of note, the algorithm correctly identified 100% of subjects with ETDRS level 43 or higher diabetic retinopathy (moderate non-proliferative disease or worse).

The public health argument for wide-spread, autonomous detection of diabetes-related eye disease centers on the fact that many patients do not receive dilated retinal examinations by eye care practitioners (ECPs -optometrists or ophthalmologists) at recommended intervals [3]. Moreover, detecting referable or treatable disease in patients who would not otherwise receive eye examinations for whatever reason (lack of knowledge, inaccessibility of ECPs, cost, etc.) is anticipated to save both vision and money, and allows PCPs to boost their quality measures, protect their income, and may represent a new revenue stream depending on how third party payers decide to handle reimbursement for AI services. Additionally, the marginal costs of operating AI are almost nil once devices//technology has been acquired by PCPs or health networks [4]. Another argument in favor of AI is a failure of ECPs to send a diabetes exam report to the PCP (though, interestingly, correspondence from the PCP to the ECP has been shown to be more important for compliance with dilated eye examinations.) [5]

However, there are some downsides that are critically important for PCPs, endocrinologists and diabetes educators to understand about the current iteration of AI for diabetes eye care.

(1) Diabetes patients frequently have ocular disease other than diabetic retinopathy (glaucoma, age-related macular degeneration, cataract, dry eye just to name the most prevalent) and these require a comprehensive (dilated) eye examination for proper diagnosis and management – a ‘negative’ AI-based finding may give PCPs and patients a false sense of security about the totality of their ocular status.

(2) The relatively high false positive rate reported in the aforementioned clinical trial (13%) means that some patients already have or will soon progress to sight-threatening eye disease. Individual patient risk of disease progression and vision loss is predicated on disease duration, degree, and consistency of metabolic control and accurate staging of diabetic retinopathy – factors that determine appropriate follow-up intervals and patient-specific education. Moreover, emerging evidence from the Joslin Diabetes Center shows that diabetic retinopathy lesions in the peripheral retina are highly predictive of which patients will develop proliferative diabetic retinopathy (DRCR.net Protocol AA is under way to substantiate this), the truly blinding form of the disease. The system employed by IDx-DR does not capture the peripheral retina and, as such, may not correctly stage the disease, again giving both patients and PCPs a false sense of security.

(3) The LEADING cause of vision loss in diabetes is diabetic macular edema (DME), the gold-standard for detection of which is stereoscopic macular examination coupled with spectral domain optical coherence tomography (sdOCT). Though all subjects with ETDRS level 43 or higher DR were detected via IDx-DR, we have to wonder how many cases of subtle DME were missed as this specific data was not reported by study investigators. I also have concern that AI algorithms designed to pass or fail individual patients above any specific level of disease severity may hinder appropriate education and targeted intervention for patients with milder DR. Mounting evidence demonstrates that diabetes induces structural and functional retinal/visual abnormalities long before the appearance of the classic retinal vascular findings associated with DR [6] and patients exhibiting early changes deserve correct diagnosis and early intervention.

(5) The US currently has about 58,000 eye care providers (optometrists, ophthalmologists, retinal specialists) who are specifically trained to diagnose and manage the spectrum of diabetes-related eye disease and who have committed substantial resources in education and instrumentation to help us identify patients earlier and educate appropriately based on individualized risk factors and stage of disease. As such, it makes far more economic sense for non-ophthalmic providers to work collaboratively with eye care providers in their communities to ensure that patients receive appropriate diagnoses and care, rather than spend tens of thousands of dollars to utilize a system that identifies a fraction of at-risk patients. As I recently told a primary care physician sitting in my exam chair, “If your HEDIS scores are down due to inadequate patient adherence to dilated eye examinations, the solution is to encourage a diabetes-savvy eye care provider to open a practice adjacent to or within yours! Work collaboratively and proactively with optometrists and ophthalmologists; we are ready, willing and able to assist you.”

AI will almost certainly play an increasingly important role in diabetes care and autonomous detection of diabetes complications has merit in underserved populations where providers are not plentiful. That is not the case, however, in the United States. Patients with diabetes in our country deserve a real eye examination, not a partially adequate decision tool that tells them if they, in fact, need an eye examination by a knowledgeable and experienced eye care provider.

References

[1]  FDA News Release, April 11, 2018. Accessed at https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm604357.htm, April 15, 2018.

[2] Abramoff M. Artificial intelligence for automated detection of diabetic retinopathy in primary care. Paper presented at: Macular Society; February 22, 2018; Beverly Hills, Calif. Available at: http://webeye.ophth.uiowa.edu/abramoff/MDA-MacSocAbst-2018-02-22.pdf

[3] Arraya Paksin-Hall, Michelle L. Dent, Frank Dong & Elizabeth Ablah (2013) Factors Contributing to Diabetes Patients Not Receiving Annual Dilated Eye Examinations, Ophthalmic Epidemiology, 20:5, 281-287.

[4] Jiang F, Jiang Y, Zhi H, et al. Artificial intelligence in healthcare: past, present and future. Stroke and Vascular Neurology. 2017;2(4):230-243. doi:10.1136/svn-2017-000101.

[5] Storey PP1, Murchison AP, Pizzi LT, Hark LA, Dai Y, Leiby BE, Haller JA. IMPACT OF PHYSICIAN COMMUNICATION ON DIABETIC EYE EXAMINATION ADHERENCE: Results From a Retrospective Cohort Analysis. Retina. 2016 Jan;36(1):20-7.

[6] Joltikov KA, de Castro VM, Davila JR, et al. Multidimensional Functional and Structural Evaluation Reveals Neuroretinal Impairment in Early Diabetic Retinopathy. Investigative Ophthalmology & Visual Science. 2017;58(6).