James Requa1, Tyler Dao2, Andrew Ninh2, William Karnes, MD3
1University of California Irvine Medical Center, Orange, CA; 2Docbot, Irvine, CA; 3University of California Irvine, Orange, CA
Polyp detection using neural networks (NNs) during colonoscopy shows promise for reducing adenoma miss rate. NNs are also being developed to document polyp features, including their pathology, size and shape, as well as procedure quality measures, including cecal intubation, withdrawal time and prep quality. Counting polyps of predicted size, shape and pathology is also critical for determining appropriate surveillance intervals at point-of-care. The latter requires NNs capable of "remembering" each polyp to avoid recounting at each re-encounter.
The polyp-tracking system includes 3 components: 1) Polyp Detection/Localization
: YOLO-based Convolutional Neural Network architecture which identifies & localizes new polyps in the colon in real-time during colonoscopy procedure; 2) Polyp Future Positioning
: Recurrent Neural Network Bi-directional LSTM (Long Short-Term Memory) that predicts polyp’s future position based on temporal features across sequential video frames and 3) Polyp Re-Identification/Matching
: one-shot learning Siamese Neural Network that outputs a “similarity score” expressing the likelihood that two polyps are the same.
Results: YOLO-based polyp detection algorithm was trained on 7000 polyp images (unique polyps), 7000 non-polyp images and can achieve a detection rate of over 99% on validation set of 3000 images. Both LSTM polyp future positioning algorithm & Siamese Net polyp re-identification algorithms were trained on sequences of detected polyps in video frames from 400 colonoscopy video clips and validated on 100 clips. For the LSTM, with objective to predict center-point coordinates for polyp in the next (future) frame, it achieved < 0.001 validation loss (mean-squared-error) compared to ground truth center-point coordinates and can process inference at 1000FPS. For polyp re-identification, we achieved 94.7% validation accuracy at re-identifying the same polyp across frames.
Polyp tracking will provide many benefits to AI-assisted colonoscopy, including correct polyp counting, more accurate rendering of predictions of optical pathology, polyp size and polyp shape. Theoretically, the technology could be modified to assist in finding a lost polyp, in alerting the colonoscopist to unseen surfaces, and defining finding locations.
Citation: James Requa; Tyler Dao; Andrew Ninh; William Karnes, MD. P0237 - BEYOND POLYP DETECTION: NOVEL METHOD TO TRACK INDIVIDUAL POLYPS USING NEURAL NETWORKS. Program No. P0237. ACG 2019 Annual Scientific Meeting Abstracts. San Antonio, Texas: American College of Gastroenterology.