Suppose that the output structure of the neural network in Exercise 9 is changed so that there are k-dimensional outputs o1 . . . ot in each layer, and the overall loss is L = t i=1 L(oi). The output recurrence is op = Uhp. All other recurrences remain the same. Show that the backpropagation recurrence of the hidden layers changes as follows:

∂L

∂ht

= UT ∂L(ot)

∂ot

∂L

∂hp−1

= WTΔp−1

∂L

∂hp

+ UT ∂L(op−1)

∂op−1

∀p ∈ {2 . . . t}