Consider a multi$-$arm bandit problem with k $=5$ actions, denoted $1,2,3,4,$ and $5.$ Consider applying to this problem a bandit algorithm using E$-$greedy action selection, sample$-$average action$-$value estimates, and initial estimates of Q$1($a$)=0$ for all a$.$ Suppose the initial sequence of actions and rewards is A$1=1,$ R$1=2,$ A$2=2,$ R$2=3,$ A$3=3,$ R$3=1,$ A$4=2,$ R$4=2,$ A$5=3,$ R$5=0,$ A$6=4,$ R$6=5.$ On some of these time steps the E case may have occurred causing an action to be selected at random. On which time steps did this definitely occur? On which time steps could this possibly have occurred?