Neural network-based colonoscopic diagnosis using on-line learning and differential evolution

In this paper, on-line training of neural networks is investigated in the context of computer-assisted colonoscopic diagnosis. A memory-based adaptation of the learning rate for the on-line back-propagation (BP) is proposed and used to seed an on-line evolution process that applies a differential evolution (DE) strategy to (re-) adapt the neural network to modified environmental conditions. Our approach looks at on-line training from the perspective of tracking the changing location of an approximate solution of a pattern-based, and thus, dynamically changing, error function. The proposed hybrid strategy is compared with other standard training methods that have traditionally been used for training neural networks off-line. Results in interpreting colonoscopy images and frames of video sequences are promising and suggest that networks trained with this strategy detect malignant regions of interest with accuracy.

[1]  Cornelius T. Leondes,et al.  Neural network systems techniques and applications , 1998 .

[2]  K. Haruma,et al.  Pit pattern diagnosis of early colorectal carcinoma by magnifying colonoscopy: clinical and histological implications. , 2000, International journal of oncology.

[3]  G. D. Magoulas,et al.  Image recognition and neuronal networks: Intelligent systems for the improvement of imaging information , 2000, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[4]  George D. Magoulas,et al.  Neural network supervised training based on a dimension reducing method , 1997 .

[5]  Carlos Eduardo Pedreira,et al.  An Online Learning Approach: A Methodology for Time Varying Applications , 2001, Neural Computing & Applications.

[6]  George D. Magoulas,et al.  Learning Rate Adaptation in Stochastic Gradient Descent , 2001 .

[7]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[8]  B. L. Davies,et al.  Automation of colonoscopy. II. Visual control aspects , 1998 .

[9]  Nicol N. Schraudolph,et al.  Local Gain Adaptation in Stochastic Gradient Descent , 1999 .

[10]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[11]  George D. Magoulas,et al.  SUPERVISED TRAINING USING GLOBAL SEARCH METHODS , 2001 .

[12]  Thibault Langlois,et al.  Parameter adaptation in stochastic optimization , 1999 .

[13]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[14]  Richard S. Sutton,et al.  Online Learning with Random Representations , 1993, ICML.

[15]  Trevor P Martin,et al.  EUNITE 2001 - European Symposium on Intelligent Technologies, Hybrid Systems and their implementation on Smart Adaptive Systems, Tenerife , 2001 .

[16]  George D. Magoulas,et al.  Improving the Convergence of the Backpropagation Algorithm Using Learning Rate Adaptation Methods , 1999, Neural Computation.

[17]  Martin H. Floch.  Methods in Disease: Investigating the Gastrointestinal Tract , 1999 .

[18]  Peter J. Angeline,et al.  Tracking Extrema in Dynamic Environments , 1997, Evolutionary Programming.

[19]  Cornelius T. Leondes.  Image processing and pattern recognition , 1998 .

[20]  I M Chen,et al.  Automation of colonoscopy. Part II: Visual-control aspects. , 1998, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[21]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[22]  Robert Ivor John,et al.  Application of the fuzzy ART/MAP and MinMax/MAP neural network models to radiographic image classification , 1997, Artif. Intell. Medicine.

[23]  P M Delaney,et al.  Fibre optic confocal imaging (FOCI) for subsurface microscopy of the colon in vivo. , 1994, Journal of anatomy.

[24]  Ralf Salomon,et al.  Adaptation on the Evolutionary Time Scale: A Working Hypothesis and Basic Experiments , 1997, Artificial Evolution.

[25]  George D. Magoulas,et al.  Evaluation of textural feature extraction schemes for neural network-based interpretation of regions in medical images , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[26]  H. Kazmierczak,et al.  Image processing and pattern recognition , 1968, IFIP Congress.

[27]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[28]  M.,et al.  Statistical and Structural Approaches to Texture , 2022 .

[29]  Shin-ei Kudo,et al.  Colonoscopic Diagnosis and Management of Nonpolypoid Early Colorectal Cancer , 2000, World Journal of Surgery.

[30]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[31]  Carl G. Looney,et al.  Pattern recognition using neural networks , 1997 .

[32]  Chee Keong Kwoh,et al.  Probabilistic reasoning from correlated objective data , 1995 .

[33]  Smart Adaptive Systems : State of the Art and Future Directions of Research , 2001 .

[34]  Terence C. Fogarty,et al.  A Comparative Study of Steady State and Generational Genetic Algorithms , 1996, Evolutionary Computing, AISB Workshop.

[35]  M. N. Vrahatis,et al.  Adaptive stepsize algorithms for on-line training of neural networks , 2001 .

[36]  Stefanos D. Kollias,et al.  On-line retrainable neural networks: improving the performance of neural networks in image analysis problems , 2000, IEEE Trans. Neural Networks Learn. Syst..

[37]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[38]  Nicol N. Schraudolph,et al.  Online Local Gain Adaptation for Multi-Layer Perceptrons , 1998 .

[39]  A. K. Rigler,et al.  Accelerating the convergence of the back-propagation method , 1988, Biological Cybernetics.