Sensor-Fusion in Neural Networks

The problem of sensor-fusion arises in many applications. We have studied the problem primarily in the context of cognitive robotics. An autonomous robot has to create a meaningful representation of its position in the world and, more generally, the situation in which it has to perform an appropriate action. Such robots usually have several different sensors (sonars, laser range finders, cameras, microphones, infrared sensors, bumpers) and they have to combine the different kinds of information provided by these sensors. Neural networks seem to be particularly well suited for the combination of inputs from completely different sources. So we will be using them and the corresponding learning and training strategies for these problems. In the context of pattern recognition, a community has emerged that is analyzing these problems and has accumulated a considerable amount of practical and theoretical knowledge in the theoretical framework of multiple classifier systems. There is a strong overlap between this community and our neural networks community, and we have also contributed to the application of multiple classifier systems built from neural networks for sensor-fusion, for example, for the classification of multivariate biological time series. One question to be addressed here is the use of uncertainty measures for the combination of classifiers. Here we use the framework of multiple classifier systems to compare several methods of sensor- or information-fusion.

[1]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[2]  Didier Dubois,et al.  Fuzzy sets and systems ' . Theory and applications , 2007 .

[3]  James C. Bezdek,et al.  Decision templates for multiple classifier fusion: an experimental comparison , 2001, Pattern Recognit..

[4]  Friedhelm Schwenker,et al.  Classification of bioacoustic time series based on the combination of global and local decisions , 2004, Pattern Recognit..

[5]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[6]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[7]  Günther Palm,et al.  Decision templates for the classification of bioacoustic time series , 2003, Inf. Fusion.

[8]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Didier Dubois,et al.  Possibility Theory - An Approach to Computerized Processing of Uncertainty , 1988 .

[10]  Fabio Roli,et al.  Fusion of multiple classifiers for intrusion detection in computer networks , 2003, Pattern Recognit. Lett..

[11]  Luc Vandendorpe,et al.  Decision Level Fusion of Intramodal Personal Identity Verification Experts , 2002, Multiple Classifier Systems.

[12]  Friedhelm Schwenker,et al.  Three learning phases for radial-basis-function networks , 2001, Neural Networks.

[13]  Günther Palm,et al.  Radial basis function neural networks and temporal fusion for the classification of bioacoustic time series , 2003, Neurocomputing.