Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization

In the last decade, recurrent neural networks (RNNs) have attracted more efforts in inferring genetic regulatory networks (GRNs), using time series gene expression data from microarray experiments. This is critically important for revealing fundamental cellular processes, investigating gene functions, and understanding their relations. However, RNNs are well known for training difficulty. Traditional gradient descent-based methods are easily stuck in local minima and the computation of the derivatives is also not always possible. Here, the performance of three evolutionary-swarm computation technology-based methods, known as differential evolution (DE), particle swarm optimization (PSO), and the hybrid of DE and PSO (DEPSO), in training RNNs is investigated. Furthermore, the gene networks are reconstructed via the identification of the gene interactions, which are explained through corresponding connection weight matrices. The experimental results on two data sets studied in this paper demonstrate that the DEPSO algorithm performs better in RNN training. Also, the RNN-based model can provide meaningful insight in capturing the nonlinear dynamics of genetic networks and revealing genetic regulatory interactions.

[1]  Salman Mohagheghi,et al.  Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems , 2008, IEEE Transactions on Evolutionary Computation.

[2]  Marcel J. T. Reinders,et al.  Linear Modeling of Genetic Networks from Experimental Data , 2000, ISMB.

[3]  John A. Hertz,et al.  Modeling Genetic Regulatory Dynamics in Neural Development , 2002, J. Comput. Biol..

[4]  V. Anne Smith,et al.  Evaluating functional network inference using simulations of complex biological systems , 2002, ISMB.

[5]  P. Brown,et al.  DNA arrays for analysis of gene expression. , 1999, Methods in enzymology.

[6]  Marcel J. T. Reinders,et al.  Genetic network models: a comparative study , 2001, SPIE BiOS.

[7]  Donald C. Wunsch,et al.  Hybrid PSO-EA Algorithm for Training Feedforward and Recurrent Neural Networks for Challenging Problems , 2006 .

[8]  Edward R. Dougherty,et al.  From Boolean to probabilistic Boolean networks as models of genetic regulatory networks , 2002, Proc. IEEE.

[9]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[10]  Gary D. Stormo,et al.  Modeling Regulatory Networks with Weight Matrices , 1998, Pacific Symposium on Biocomputing.

[11]  Paul T. Jackway,et al.  Network Motifs, Feedback Loops and the Dynamics of Genetic Regulatory Networks , 2005, 2005 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology.

[12]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[13]  Bernd Reusch.  Computational Intelligence, Theory and Applications , 1997 .

[14]  Aurélien Mazurie,et al.  Gene networks inference using dynamic Bayesian networks , 2003, ECCB.

[15]  U. Alon,et al.  Assigning numbers to the arrows: Parameterizing a gene regulation network by using accurate expression kinetics , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[16]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[17]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[18]  John F. Kolen,et al.  Dynamical Recurrent Networks , 2001 .

[19]  Ronald W. Davis,et al.  Quantitative Monitoring of Gene Expression Patterns with a Complementary DNA Microarray , 1995, Science.

[20]  Patrik D'haeseleer,et al.  Linear Modeling of mRNA Expression Levels During CNS Development and Injury , 1998, Pacific Symposium on Biocomputing.

[21]  J. Vohradský.  Neural network model of gene expression , 2001, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[22]  Patrik D'haeseleer,et al.  Genetic network inference: from co-expression clustering to reverse engineering , 2000, Bioinform..

[23]  Edward Keedwell,et al.  Discovering Gene Networks with a Neural-Genetic Hybrid , 2005, TCBB.

[24]  Stefan Janaqi,et al.  Generalization of the strategies in differential evolution , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[25]  Stephanie Forrest,et al.  Reconstructing gene networks from large scale gene expression data , 2000 .

[26]  Michal Linial,et al.  Using Bayesian Networks to Analyze Expression Data , 2000, J. Comput. Biol..

[27]  S. P. Fodor,et al.  High density synthetic oligonucleotide arrays , 1999, Nature Genetics.

[28]  Ziv Bar-Joseph,et al.  Analyzing time series gene expression data , 2004, Bioinform..

[29]  Geoffrey J. McLachlan,et al.  Analyzing Microarray Gene Expression Data , 2004 .

[30]  Satoru Miyano,et al.  Estimating gene networks from gene expression data by combining Bayesian network model with promoter element detection , 2003, ECCB.

[31]  Terence Soule,et al.  Comparison of Genetic Algorithm and Particle Swarm Optimizer When Evolving a Recurrent Neural Network , 2003, GECCO.

[32]  Pierre Baldi,et al.  A Bayesian framework for the analysis of microarray expression data: regularized t -test and statistical inferences of gene changes , 2001, Bioinform..

[33]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[34]  Kevin Murphy,et al.  Modelling Gene Expression Data using Dynamic Bayesian Networks , 2006 .

[35]  Hidde de Jong,et al.  Modeling and Simulation of Genetic Regulatory Systems: A Literature Review , 2002, J. Comput. Biol..

[36]  M Wahde,et al.  Coarse-grained reverse engineering of genetic regulatory networks. , 2000, Bio Systems.

[37]  John P. Vanden Heuvel,et al.  Regulation of gene expression I , 1995 .

[38]  Marco Grzegorczyk,et al.  Comparative evaluation of reverse engineering gene regulatory networks with relevance networks, graphical gaussian models and bayesian networks , 2006, Bioinform..

[39]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[40]  Ganesh K. Venayagamoorthy,et al.  Optimal PSO for collective robotic search applications , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[41]  D. Latchman.  Gene Regulation: A Eukaryotic Perspective , 1990 .

[42]  S Fuhrman,et al.  Reveal, a general reverse engineering algorithm for inference of genetic network architectures. , 1998, Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing.

[43]  Ka Yee Yeung,et al.  From co-expression to co-regulation , 2004 .

[44]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[45]  Herbert Jaeger,et al.  A tutorial on training recurrent neural networks , covering BPPT , RTRL , EKF and the " echo state network " approach - Semantic Scholar , 2005 .

[46]  Paul P. Wang,et al.  Advances to Bayesian network inference for generating causal networks from observational biological data , 2004, Bioinform..

[47]  Chia-Feng Juang,et al.  A hybrid of genetic algorithm and particle swarm optimization for recurrent network design , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[48]  Xiao-Feng Xie,et al.  DEPSO: hybrid particle swarm with differential evolution operator , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[49]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[50]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[51]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[52]  Dirk Husmeier,et al.  Sensitivity and specificity of inferring genetic regulatory interactions from microarray experiments with dynamic Bayesian networks , 2003, Bioinform..

[53]  Marcel J. T. Reinders,et al.  ROBUST GENETIC NETWORK MODELING BY ADDING NOISY DATA , 2001 .

[54]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[55]  Donald C. Wunsch,et al.  A Study of Particle Swarm Optimization in Gene Regulatory Networks Inference , 2006, ISNN.