Thx!!

. As discussed in lecture Information Theory is critical to understanding and developing communication systems. One particularly important application s the efficient encoding of information, for instance to minimize the total number of bits required to send an electronic message or save a file on a hard drive. The mapping of symbols from your original message (i.e., in alpha-numeric) to symbols in a target alphabet (e.g., binary) is called an encoding, and a variety of algorithms exist. You will learn one such encoding, defined by the Shannon-Fano algorithm, which is used in some zip file algorithms. Before doing so, some basic (Shannon) information theory concepts will need to be introduced information: represents the certainty about the probability of the outcome of an event. The event itself will have several possible outcomes that can be modeled by a random variable X. Each of the n possible outcomes -1,... EX has a probability P(Xpi of occurring. information content: can be viewed as how much useful information is contained in r E X. Shannon derived the measure I(zi logb where b corresponds the basis units in which information is measured (e.g., b-2 means information is measured in bits) entropy: is essentially the expected amount of information from an event, and can be calculated as