3. Compare the different parameter settings for the game of Example 12.2. In particular compare the following situations

(a) α varies, and the Q-values are initialized to 0.0

(b) α varies, and the Q-values are initialized to 5.0

(c) α is fixed to 0.1, and the Q-values are initialized to 0.0

(d) α is fixed to 0.1, and the Q-values are initialized to 5.0

(e) Some other parameter settings.

For each of these, carry out multiple runs and compare

(a) the distributions of minimum values

(b) the zero crossing

(c) the asymptotic slope for the policy that includes exploration

(d) the asymptotic slope for the policy that does not include exploration. To test this, after the algorithm has explored, set the exploitation parameter to 100% and run additional steps.

Which of these settings would you recommend? Why?