The entropy \(H(X)\) of a discrete random variable \(X\) measures the uncertainty about predicting the value of \(X\) (Cover \(\&\) Thomas, 2006). If \(X\) has the probability distribution \(p_{X}(x)\), its entropy is determined by

\[H(X)=-\sum_{x} p_{X}(x) \log _{b} p_{X}(x)\]

where the base \(b\) of the logarithm determines the entropy unit. If \(b=2\), for instance, the entropy is measured in bits/symbol. Determine in bits/symbol the entropy of a random variable \(X\) characterized by the discrete uniform distribution \(u_{X, \mathrm{~d}}(x)\) given in Equation (1.208).

Transcribed Image Text:

ux,d(x): = M 0, 8(x-x), xx1,x2x (1.208) otherwise