Redundancy and Perception

In the situations that confront communications engineers, it is relatively easy to isolate and identify the component parts to which information theory can be applied, namely a set of messages which are then encoded and transmitted down a communication channel. One can show that the statistical properties of the ensemble of messages have definite calculable effects on the efficiency with which they can be passed down the channel; if the ensemble of messages, after encoding, is highly redundant, much less information can be transmitted. Redundant messages are also less perturbed by errors in transmission but this is a good deal more complicated, and there are problems enough in applying the simpler concepts of information theory to perception.