$(25$ points$)$ the minimax loss for $L_G$ suffers from vanishing gradient

problem. In terms of the discriminator's logits$,$ the minimax loss is

$L_G^{\text{m}\text{i}\text{n}\text{i}\text{m}\text{a}\text{x}}($;$)=E_{zN(0,I)}[log(1-(h_{}(G_{}(z))))]$

Show that the derivative of $L_G^{\text{m}\text{i}\text{n}\text{i}\text{m}\text{a}\text{x}}$ with respect to $$ is approximately $0$ if

$D(G_{}(z))$~~$0,$ or equivalently, if $h_{}(G_{}(z))0.$ You may use the fact that

${}^{\text{'}}(x)=(x)(1-(x)).$ Why is this problematic for the training of the generator

when the discriminator successfully identifies a fake sample $G_{}(z)?$

$(25$ points$)$ To solve this vanishing gradient problem, we usually replace

$L_G^{\text{m}\text{i}\text{n}\text{i}\text{m}\text{a}\text{x}}$ with other loss functions such as non$-$saturating loss $L_G^{\text{n}\text{s}\text{g}\text{a}\text{n}}[1]$ and

more other forms of loss functions can be found in

$[2].$ You may plot differ$-$

ent loss functions including minimax loss and non$-$saturating loss to show the

contrast. You also need to explain why non$-$saturating loss can avoid vanishing

gradient problem.

$L_G^{\text{n}\text{s}\text{g}\text{a}\text{n}}($;$)=-E_{zN(0,I)}[log{D}_{}(G_{}(z))]$