Consider a data set for a classifleation problem. The problem has thres decision boundariss, there are five

descriptive features and two class values. A neunal network has been traind to separate the two clases,

and contairs enough input units, one output unit, and a hidden layer with five hidden units. The data set

also contains noise, and has $1500$ instanos. After training the neural network, the prodiction accuracy

on the training set is $95\%$ and the prodiction ancuracy on the test set is $67\%.$ Which of the following

statements are true for this predictive model? Incoervet answers will be penalized. Write down only the

lecter$($s$)$ of your answer$($s$).$

$($a$)$ The model overfits the training set

$($b$)$ Aceunacy on the test set can be improwed by reducing the number of hidden units

$($c$)$ Aocuracy on the test set can be improwed by incressing the number of hidden units

$($d$)$ Aceurncy will be improwed if the lesurning rate is incressod

$($e$)$ Aocuracy can be improved by uaing an active lesarning approsch

$($f$)$ Accuracy on the test set will improwe if the network is trainod for longer

$($g$)$ Aceurnacy on the test sat may improwe if ragularixation is applised

Suppose that the sigmaid activation function is used in the hidden and output layers of the nounal network,

and that gradient desoent is used to adjust the weights.

$($a$)$ Is normalixation$/$scaling of the target frature nocsewry? Motivate your answer.

$($b$)$ Will it be prudent to normalieg$/$scale the input festures? Motivate your answer.

$($c$)$ Why is it not a good strategy to initialize weights and bises to large abeolute values?

Assume again that gradient desoent is usod, explain why a momentum term is noessary for stochasti

lesarning.

Is the following statement true or false: A single artiffeial neuron can only separate linearly sepanable

classes. Motivate your answer.

Compared to grodient descent, what are the main advantagrs of using scaled conjugate gradient to train

a neunal network?

What are the main advantages in using population$-$based meta$-$beuristis to tnain neural networks?

What is the rationale behind setive lesarning in neural networks?

Is the following statement troe or false? If the validation errur is much wone than the training error, then

the neural network averfits. Motivate your answer

Discuss the implications of using regularization, such as wcight docay, when training a foedforward neural

network.

For each of the following statements, indieate if it is true or false. Cive a clas motivation for each answer.

$($a$)$ Cindient descent can be usod to adjust the wcights of a neunal notwork if the following objective

function is usad:

$E-{\displaystyle \underset{p=1}{\overset{N_v}{}}}{\displaystyle \underset{k=1}{\overset{K}{}}}|k_{k_p}-o_{kp}|$

where $R_T$ is the number of training instanoes, $K$ is the total number of output units, $t_{kg},$ is the target

$($expected$)$ output of the $k-$th output unit of instanoe $p,$ and ${}_{k,p}$ is the setual output provided by

the neural network.

$($b$)$ An advantage of neural networks is that thry have expellent generalization abilities.

Consider the elessification problem depicted in the figure below. Can logiestic regression be used to separate

the two claeses? Motivate your answer.