Q1: Please briefly and precisely answer the following questions (a short paragraph or a couple of sentences shall do):

1. What are the two most common supervised tasks?

2. Can you name four common unsupervised tasks?

3. Which Linear Regression training algorithm can you use if you have a training set with millions of features?

4. Suppose the features in your training set have very different scales. Which algorithms might suffer from this, and how? What can you do about it?

5. Do all Gradient Descent algorithms lead to the same model, provided you let them run long enough?

6. Suppose you use Batch Gradient Descent and you plot the validation error at every epoch. If you notice that the validation error consistently goes up, what is likely going on? How can you fix this?

7. Suppose you are using Ridge Regression and you notice that the training error and the validation error are almost equal and fairly high. Would you say that the model suffers from high bias or high variance?

8. What is a support vector?

9. Can an SVM classifier output a confidence score when it classifies an instance? What about a probability?

10. Say youve trained an SVM classifier with an RBF kernel, but it seems to underfit the training set. Should you increase or decrease (gamma)? What about C?

Q2: Computation (summarize your results) Load the MNIST data (introduced in Chapter 3), and split it into a training set, a validation set, and a test set (e.g., use 50,000 instances for training, 10,000 for validation, and 10,000 for testing). Then train the following classifiers, (1) a logistics regression classifier and (2) an SVM classifier. Please compare the performance (computation time, accuracy, balance between variance and bias, and AUC) of these two classifiers on this dataset.

Load the MNIST data (introduced in Chapter 3), and split it into a training set, a validation set, and a test set (e.g., use 50,000 instances for training, 10,000 for validation, and 10,000 for testing). Then train the following classifiers, (1) a logistics regression classifier and (2) an SVM classifier. Please compare the performance (computation time, accuracy, confusion matrix, and error analysis) of these two classifiers on this dataset.