Information Theory
Information theory is the reduction of uncertainty. Information theory is the branch of mathematics that describes how uncertainty should be quantified, manipulated and represented. Claude Shannon is the famous person in this field since 1949. The fundamental of information theory had far reaching implications for almost every field of science and technology. Information theory is important in shaping theories of perception, cognition and neutral computation.
Coding Theory
Coding theory is the branch of mathematics concerned with transmitting data across noisy channels and recovering the message. Coding theory is about making messages easy to read. We assume that our message is in the form of binary digits or bits, strings of 0 or 1. We have to transmit these bits along a channel (such as a telephone line) in which errors occur randomly, but at a predictable overall rate. To compensate for the errors we need to transmit more bits than there are in the original message. Figure shows the basic communication system which is coding theory involved at the encoder and decoder of the system.
