The Gambler has arrived, and pitched his tent near ours. He wants to gamble with us tonight.

The Gambler has a weird shaped coin, with heads on one side and tails on the other.

He lets us see the coin, and flip it 20 times to practice. Tonight we will engage in gambling, proposing bets such as "You pay me $1000 if the next flip is heads, but otherwise I pay you $872. Or even more complex bets such as "You pay me $10000 if the next two flips are heads, but otherwise I pay you $3000." We will need to choose wisely which bets to accept.

The Gambler leaves, taking the coin with him and disappearing into his tent.

Obviously a very simple problem for us, since we know about models and data - we will build an AI to make predictions and use that in our gambling endeavors. Fortunately we wrote down the results of our 20 flips(They were HTHHHTHTHTHTTHTTHHHH).The Gambler probably won't even notice our computer in the light of the flickering campfire - we will catch him unawares!

The first approach we might try is a non-stochastic model of a coin flip that makes a point prediction.

0) We usually want a data representation for heads/tails data that is numerical. Choose the representation tails=0, heads=1

1) We decide on using the affine family of functions for our model. Explain what that means quantitatively using equation 8.4 for this problem. What are the dimensions of all the symbols. For this model are there parameters and how many?

2) So now we train our model using minimization of risk.

Here is an illegal loss function loss(y^,y) = y^-y. Why is it illegal?

Use this loss function: loss(y^,y) = |y^-y| to train the model. Explain what we do to train the model, and tell what is the result of training in this case.

Should we be worried about overfitting?

3) So now we can use our model to make point predictions. What does that mean - what does a prediction tell us for this model?

========================

After some thought, we decide this is not a sufficient approach. Instead, lets make a probabilistic model and use Bayesian inference to produce predictions that are not just point predictions.

4) We (again) decide on using the affine family of functions for our model. But we also will need a prior (based on our understanding of the geometry of the coin and also on rumors from the east about the Gambler and his coin). Give a specific simple example of a plausible prior.

5) Should we still train our model? Explain what mathematics will be used in training the model, and what will be the result of training. 8.22, 8.23 applied to this problem.

6) So now we are ready to use our model and become rich while destroying the Gambler. What kind of non-point predictions will our model give us and how might we use them?

7) BAD NEWS - just before the gambling begins we catch a glimpse through the open flap of the Gambler's tent - he has a computer just like ours! The Gambler knows how to build his own AI! Why would we suspect that the Gambler's AI will be better than ours?