Communication Theory

The properties of signals and transmission systems are discussed, and it is shown that the Fourier transform greatly simplifies the handling of commonly-occurring physical systems. Moreover, it explains the concepts of sampling and filtering. The description is then extended to unknown signals and noise. It is shown that known properties are well-described by probabilities. Further, probabilities define how to optimally decide under an observation, and lead to the concepts of communication source and channel. Building on these concepts, analogue signals are constructed such that a transmission of digital messages is possible. The Shannon measure of binary information is used to prove that a message source can be compressed up to its entropy, and that one may maximally transmit up to the channel capacity without error. The results are based on the identification of typical sets, i.e., precise sets of possibilities. Codes for the transmission of messages are described, and an efficient decoding method for good codes is presented. This decoder is based on residual sets, which are defined in a manner similar to typical sets.