Consider the car domain above $($without knowing the $T$ or $R)$ and given the following experiences:

Episode $1$:

cool, fast, warm, $+2$

warm, fast, overheated, $-10$

Episode $2$:

cool, slow, cool, $+1$

cool, slow, cool, $+1$

cool, fast, cool, $+2$

cool, fast, cool, $+2$

cool, fast, warm, $+2$

warm, fast, overheated, $-10$

Episode $3$:

cool, fast, warm, $+2$

warm, slow, cool, $+1$

cool, slow, cool, $+1$

cool, fast, cool, $+2$

cool, fast, warm, $+2$

warm, fast, overheated, $-10$

a$.(3$pt$)$ Estimating the parameters for $T$ and $R$ for model$-$based reinforcement learning.

b$.(3$pt$)$ Use MC reinforcement learning method $($direct evaluation$)$ to estimate the Q function, assuming $=1\mathrm{.}0.$ Count all occurrences of a state in each episode.