Question 1.

a) Four symbols, A, B, C and D, occur with probabilities 1/2 , 1/4 , 1/8 , and 1/8 respectively.

Assume the symbols are statistically independent. Determine the information contained in the message:

i) y = BDA ii) y = CDB

b) Find the entropy of a binary source which produces independent symbols 0 and 1 with equal probability

c) Determine the entropy of a code that uses the symbols, A, B, C and D, which i) occur with probabilities 1/2 , 1/4 , 1/8 , and 1/8 respectively. ii) are equiprobable