These questions use a combination of discrete math and cryptography to figure out the solution. For example the first one would be a probability and entropy problem. It would be helpful if you could show out the work to understand how each of these problems are done.

Let E1 and E2 be two independent flips of a fair coin. Find the entropy H(E1) and joint entropy H(E1,E2). Why is H(E1,E2)=H(E1)+H(E2)?

Consider modifying the experiment in 5 above so that either 1) the coin is loaded OR 2) you only flip a second time when the first flip is heads, when tails you assign the value of the flip E2 to be tails. Which modification of the experiment (1 or 2) will make it so that H(E1,E2) < H(E1)+H(E2) ? Will one of them make it so H(E1,E2) > H(E1)+H(E2) ?

Let X be a random variable that takes on integer values. The probability is that X will be in the range [0,2^4-1] with all values in that range being equally likely. The rest of the time, it will be in the range [2^4,2^8-1], again with uniform probability. What is the entropy H(X)? Estimation is OK.

A bag contains 7 red balls, 2 green balls, and 1 black ball.

a. You choose 2 balls from the bag with replacement (i.e. you put the first ball back in and shake before drawing the second). What is the entropy of the experiment?

b. You choose 2 balls from the bag without replacement (i.e. you keep the first ball while drawing the second). How is your uncertainty affected now that you know the color of the first ball before drawing the second? What is the entropy of this new experiment given the first ball you choose is red?