Extractive single document summarization using multi-objective optimization: Exploring self-organized differential evolution, grey wolf optimizer and water cycle algorithm

Abstract Text summarization techniques become paramount in extracting relevant information from large databa-ses. Current paper attempts to build some extractive single document text summarization (ESDS) systems using multi-objective optimization (MOO) frameworks. Three techniques are proposed: (1) first is an integration of self-organizing map (SOM) and multi-objective differential evolution (MODE) (named as ESDS_SMODE) (2) second is based on multi-objective grey wolf optimizer (ESDS_MGWO) and (3) third is based on multi-objective water cycle algorithm (ESDS_MWCA). The sentences present in the document are first clustered utilizing the concept of multi-objective clustering. Two objective functions measuring compactness and separation of the sentence clusters in two different ways are optimized simultaneously using MOO framework. The proposed approach is able to automatically detect the number of sentence clusters present in a document and then representative sentences are selected from different clusters using some sentence scoring features to generate the summary. The experiments were conducted on two benchmark datasets, DUC2001, and DUC2002, and the obtained results are compared with various state-of-the-art techniques using ROUGE measures. Results illustrate the superiority of our approach in comparison to state-of-the-art techniques in terms of ROUGE − 2 score for both datasets. Code of the developed approach ESDS_SMODE is available online at https://drive.google.com/open?id=1WagTeIDLgphttPrKHpnF_eO7QHWJxXxK .

[1]  Seyed Taghi Akhavan Niaki,et al.  A hybrid vendor managed inventory and redundancy allocation optimization problem in supply chain management: An NSGA-II with tuned parameters , 2014, Comput. Oper. Res..

[2]  Pushpak Bhattacharyya,et al.  A Self Organizing Map Based Multi-objective Framework for Automatic Evolution of Clusters , 2017, ICONIP.

[3]  Pushpak Bhattacharyya,et al.  Sophisticated SOM based genetic operators in multi-objective clustering framework , 2018, Applied Intelligence.

[4]  Francine Chen,et al.  A trainable document summarizer , 1995, SIGIR '95.

[5]  Pushpak Bhattacharyya,et al.  Incorporation of Neighborhood Concept in Enhancing SOM Based Multi-label Classification , 2019, PReMI.

[6]  Sanghamitra Bandyopadhyay,et al.  A Point Symmetry-Based Clustering Technique for Automatic Evolution of Clusters , 2008, IEEE Transactions on Knowledge and Data Engineering.

[7]  Sanghamitra Bandyopadhyay,et al.  A new multiobjective clustering technique based on the concepts of stability and symmetry , 2010, Knowledge and Information Systems.

[8]  Soheyl Khalilpourazari,et al.  Optimization of production time in the multi-pass milling process via a Robust Grey Wolf Optimizer , 2018, Neural Computing and Applications.

[9]  Sriparna Saha,et al.  Textual Entailment--Based Figure Summarization for Biomedical Articles , 2020, ACM Trans. Multim. Comput. Commun. Appl..

[10]  Ujjwal Maulik,et al.  A Simulated Annealing-Based Multiobjective Optimization Algorithm: AMOSA , 2008, IEEE Transactions on Evolutionary Computation.

[11]  Lucy Vanderwende,et al.  Enhancing Single-Document Summarization by Combining RankNet and Third-Party Sources , 2007, EMNLP.

[12]  Xiaojun Wan,et al.  Manifold-Ranking Based Topic-Focused Multi-Document Summarization , 2007, IJCAI.

[13]  Leandro dos Santos Coelho,et al.  Multi-objective grey wolf optimizer: A novel algorithm for multi-criterion optimization , 2016, Expert Syst. Appl..

[14]  Rada Mihalcea,et al.  Graph-based Ranking Algorithms for Sentence Extraction, Applied to Text Summarization , 2004, ACL.

[15]  Ardeshir Bahreininejad,et al.  Water cycle algorithm with evaporation rate for solving constrained and unconstrained optimization problems , 2015, Appl. Soft Comput..

[16]  Ardeshir Bahreininejad,et al.  Water cycle algorithm for solving multi-objective optimization problems , 2014, Soft Computing.

[17]  Adam Lipowski,et al.  Roulette-wheel selection via stochastic acceptance , 2011, ArXiv.

[18]  Ujjwal Maulik,et al.  Multiobjective Genetic Clustering with Ensemble Among Pareto Front Solutions: Application to MRI Brain Image Segmentation , 2009, 2009 Seventh International Conference on Advances in Pattern Recognition.

[19]  Hua Li,et al.  Document Summarization Using Conditional Random Fields , 2007, IJCAI.

[20]  Ardeshir Bahreininejad,et al.  Water cycle algorithm - A novel metaheuristic optimization method for solving constrained engineering optimization problems , 2012 .

[21]  Wei Song,et al.  Fuzzy evolutionary optimization modeling and its applications to unsupervised categorization and extractive summarization , 2011, Expert Syst. Appl..

[22]  Wen-Lian Hsu,et al.  Exploring Word Mover's Distance and Semantic-Aware Embedding Techniques for Extractive Broadcast News Summarization , 2016, INTERSPEECH.

[23]  Soheyl Khalilpourazari,et al.  Multi-objective optimization of multi-item EOQ model with partial backordering and defective batches and stochastic constraints using MOWCA and MOGWO , 2020, Oper. Res..

[24]  G. Ruxton.  The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test , 2006 .

[25]  Leila Sharif Hassanabadi,et al.  Text summarization with harmony search algorithm-based sentence extraction , 2008, CSTST.

[26]  Pushpak Bhattacharyya,et al.  Automatic Scientific Document Clustering Using Self-organized Multi-objective Differential Evolution , 2018, Cognitive Computation.

[27]  Pushpak Bhattacharyya,et al.  Cascaded SOM: An Improved Technique for Automatic Email Classification , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[28]  Sriparna Saha,et al.  Figure Summarization: A Multiobjective Optimization-Based Approach , 2019, IEEE Intelligent Systems.

[29]  Sriparna Saha,et al.  Multiobjective-Based Approach for Microblog Summarization , 2019, IEEE Transactions on Computational Social Systems.

[30]  Michael Wurst,et al.  Multi-objective frequent termset clustering , 2011, Knowledge and Information Systems.

[31]  Dianne P. O'Leary,et al.  QCS: A system for querying, clustering and summarizing documents , 2007, Inf. Process. Manag..

[32]  Xiaojun Wan,et al.  Towards a Unified Approach to Simultaneous Single-Document and Multi-Document Summarizations , 2010, COLING.

[33]  Kalyanmoy Deb,et al.  Omni-optimizer: A generic evolutionary algorithm for single and multi-objective optimization , 2008, Eur. J. Oper. Res..

[34]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[35]  Joshua D. Knowles,et al.  An Evolutionary Approach to Multiobjective Clustering , 2007, IEEE Transactions on Evolutionary Computation.

[36]  Pushpak Bhattacharyya,et al.  Multi-document Summarization Using Adaptive Composite Differential Evolution , 2019, ICONIP.

[37]  Gary G. Yen,et al.  Visualization and Performance Metric in Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[38]  Sriparna Saha,et al.  Extractive single document summarization using binary differential evolution: Optimization of different sentence quality measures , 2019, PloS one.

[39]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.

[40]  Hae-Sang Park,et al.  A simple and fast algorithm for K-medoids clustering , 2009, Expert Syst. Appl..

[41]  Ramiz M. Aliguliyev,et al.  A new sentence similarity measure and sentence based extractive technique for automatic text summarization , 2009, Expert Syst. Appl..

[42]  Qingfu Zhang,et al.  A Self-Organizing Multiobjective Evolutionary Algorithm , 2016, IEEE Transactions on Evolutionary Computation.

[43]  Partha Pakray,et al.  Aggregation of multi-objective fuzzy symmetry-based clustering techniques for improving gene and cancer classification , 2017, Soft Computing.

[44]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[45]  Ruijun Dong,et al.  Differential Evolution Versus Particle Swarm Optimization for PID Controller Design , 2009, 2009 Fifth International Conference on Natural Computation.

[46]  Seyed Taghi Akhavan Niaki,et al.  Optimization of multi-product economic production quantity model with partial backordering and physical constraints: SQP, SFS, SA, and WCA , 2016, Appl. Soft Comput..

[47]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[48]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[49]  Ajith Abraham,et al.  Data Clustering Using Multi-objective Differential Evolution Algorithms , 2009, Fundam. Informaticae.

[50]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[51]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[52]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[53]  Matt J. Kusner,et al.  From Word Embeddings To Document Distances , 2015, ICML.

[54]  René Thomsen,et al.  A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[55]  Bin Wei,et al.  Comparison between differential evolution and particle swarm optimization algorithms , 2014, 2014 IEEE International Conference on Mechatronics and Automation.

[56]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[57]  Ben Niu,et al.  Multi-objective bacterial foraging optimization , 2013, Neurocomputing.

[58]  Sriparna Saha,et al.  A generalized automatic clustering algorithm in a multiobjective framework , 2013, Appl. Soft Comput..

[59]  Swagatam Das,et al.  Automatic Clustering Using an Improved Differential Evolution Algorithm , 2007 .

[60]  Ujjwal Maulik,et al.  Validity index for crisp and fuzzy clusters , 2004, Pattern Recognit..

[61]  Seyed Taghi Akhavan Niaki,et al.  Two parameter tuned multi-objective evolutionary algorithms for a bi-objective vendor managed inventory model with trapezoidal fuzzy demand , 2015, Appl. Soft Comput..

[62]  Wei-Pang Yang,et al.  Text summarization using a trainable summarizer and latent semantic analysis , 2005, Inf. Process. Manag..

[63]  Elizabeth León Guzman,et al.  Extractive single-document summarization based on genetic operators and guided local search , 2014, Expert Syst. Appl..

[64]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[65]  Sanghamitra Bandyopadhyay,et al.  A symmetry based multiobjective clustering technique for automatic evolution of clusters , 2010, Pattern Recognit..

[66]  Paul M. B. Vitányi,et al.  The Google Similarity Distance , 2004, IEEE Transactions on Knowledge and Data Engineering.