Information entropy of humpback whale songs.

The structure of humpback whale (Megaptera novaeangliae) songs was examined using information theory techniques. The song is an ordered sequence of individual sound elements separated by gaps of silence. Song samples were converted into sequences of discrete symbols by both human and automated classifiers. This paper analyzes the song structure in these symbol sequences using information entropy estimators and autocorrelation estimators. Both parametric and nonparametric entropy estimators are applied to the symbol sequences representing the songs. The results provide quantitative evidence consistent with the hierarchical structure proposed for these songs by Payne and McVay [Science 173, 587-597 (1971)]. Specifically, this analysis demonstrates that: (1) There is a strong structural constraint, or syntax, in the generation of the songs, and (2) the structural constraints exhibit periodicities with periods of 6-8 and 180-400 units. This implies that no empirical Markov model is capable of representing the songs' structure. The results are robust to the choice of either human or automated song-to-symbol classifiers. In addition, the entropy estimates indicate that the maximum amount of information that could be communicated by the sequence of sounds made is less than 1 bit per second.

[1]  Noam Chomsky,et al.  Language and problems of knowledge : the Managua lectures , 1990 .

[2]  Thomas M. Cover,et al.  A convergent gambling estimate of the entropy of English , 1978, IEEE Trans. Inf. Theory.

[3]  VINCENT M. JANIK,et al.  Pitfalls in the categorization of behaviour: a comparison of dolphin whistle classification methods , 1999, Animal Behaviour.

[4]  G. A. Miller,et al.  Finitary models of language users , 1963 .

[5]  Paul C. Shields,et al.  Correction: "Entropy and the consistent estimation of joint distributions" , 1994 .

[6]  K. Marton,et al.  Entropy and the Consistent Estimation of Joint Distributions , 1993, Proceedings. IEEE International Symposium on Information Theory.

[7]  Robert K. Peet,et al.  The Measurement of Species Diversity , 1974 .

[8]  Ioannis Kontoyiannis The complexity and entropy of literary styles , 1997 .

[9]  I. Good THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS , 1953 .

[10]  Richard Lippmann,et al.  A comparison of signal processing front ends for automatic word recognition , 1995, IEEE Trans. Speech Audio Process..

[11]  Aaron D. Wyner,et al.  On the Role of Pattern Matching in Information Theory , 1998, IEEE Trans. Inf. Theory.

[12]  S. Savage-Rumbaugh,et al.  Spontaneous symbol acquisition and communicative use by pygmy chimpanzees (Pan paniscus). , 1986, Journal of experimental psychology. General.

[13]  H. E. Winn,et al.  The song of the humpback whale Megaptera novaeangliae in the West Indies , 1978 .

[14]  P. J. B. Slater,et al.  Describing Sequences of Behavior , 1973 .

[15]  Claude E. Shannon,et al.  Prediction and Entropy of Printed English , 1951 .

[16]  Abraham Lempel,et al.  On the Complexity of Finite Sequences , 1976, IEEE Trans. Inf. Theory.

[17]  Benjamin Weiss,et al.  Entropy and data compression schemes , 1993, IEEE Trans. Inf. Theory.

[18]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[19]  M. D. Beecher Signalling systems for individual recognition: an information theory approach , 1989, Animal Behaviour.

[20]  P. Slater,et al.  Vocal Learning in Mammals , 1997 .

[21]  Peter L. Tyack,et al.  Acoustic Communication Under the Sea , 1998 .

[22]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[23]  L. Breiman The Individual Ergodic Theorem of Information Theory , 1957 .

[24]  Nicholas Mitsakakis,et al.  Singing maps: Classification of whalesong units using a self-organizing feature mapping algorithm , 1996 .

[25]  G. Patil,et al.  Diversity as a Concept and its Measurement , 1982 .

[26]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[27]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[28]  Noam Chomsky,et al.  The faculty of language: what is it, who has it, and how did it evolve? , 2002, Science.

[29]  Yuri M. Suhov,et al.  Nonparametric Entropy Estimation for Stationary Processesand Random Fields, with Applications to English Text , 1998, IEEE Trans. Inf. Theory.

[30]  W. Fitch,et al.  Computational Constraints on Syntactic Processing in a Nonhuman Primate , 2004, Science.

[31]  D. Cato,et al.  Cultural revolution in whale songs , 2000, Nature.

[32]  E. Abe,et al.  Alloys: A stable binary quasicrystal , 2000, Nature.

[33]  D. Ketten STRUCTURE AND FUNCTION IN WHALE EARS , 1997 .

[34]  Jun Muramatsu,et al.  Almost-Sure Variable-Length Source Coding Theorems for General Sources , 1999, IEEE Trans. Inf. Theory.

[35]  A. D. Wyner,et al.  The sliding-window Lempel-Ziv algorithm is asymptotically optimal , 1994, Proc. IEEE.

[36]  S. H. Hulse,et al.  Perceptual mechanisms for individual vocal recognition in European starlings,Sturnus vulgaris , 1998, Animal Behaviour.

[37]  이영식 Communication 으로서의 영어교육 , 1986 .

[38]  B. McMillan The Basic Theorems of Information Theory , 1953 .

[39]  Jacob Ziv,et al.  Coding theorems for individual sequences , 1978, IEEE Trans. Inf. Theory.

[40]  LAURANCE R. DOYLE,et al.  Quantitative tools for comparing animal communication systems: information theory applied to bottlenose dolphin whistle repertoires , 1999, Animal Behaviour.

[41]  P. Tyack,et al.  Interactions between singing Hawaiian humpback whales and conspecifics nearby , 1981, Behavioral Ecology and Sociobiology.

[42]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[43]  Aaron D. Wyner,et al.  Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression , 1989, IEEE Trans. Inf. Theory.

[44]  L. Breiman Correction Notes: Correction to "The Individual Ergodic Theorem of Information Theory" , 1960 .

[45]  Lev B. Levitin,et al.  Entropy of natural languages: Theory and experiment , 1994 .

[46]  Biing-Hwang Juang,et al.  Time-frequency analysis and auditory modeling for automatic recognition of speech , 1996, Proc. IEEE.

[47]  G. Basharin On a Statistical Estimate for the Entropy of a Sequence of Independent Random Variables , 1959 .

[48]  Sergio Verdú,et al.  The role of the asymptotic equipartition property in noiseless source coding , 1997, IEEE Trans. Inf. Theory.

[49]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[50]  Peter L. Tyack,et al.  Whale songs lengthen in response to sonar , 2000, Nature.

[51]  Aaron D. Wyner,et al.  Improved redundancy of a version of the Lempel-Ziv algorithm , 1995, IEEE Trans. Inf. Theory.

[52]  R. Payne,et al.  Songs of Humpback Whales , 1971, Science.

[53]  Ioannis Kontoyiannis,et al.  Prefixes and the entropy rate for long-range sources , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[54]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[55]  Peter L. Tyack,et al.  Social influences on vocal development: Vocal learning in cetaceans , 1997 .