Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Problem 3.1 (5 pts) Consider a 2-layer fully-connected NN, where we have input x = R1, hidden feature x2 = Rmx1, output x3 E

 

Problem 3.1 (5 pts) Consider a 2-layer fully-connected NN, where we have input x = R1, hidden feature x2 = Rmx1, output x3 E Rkx1 and weights and bias W = Rmxn, W2 Rkxm, b = Rmx1, b2 E Rkx1 of the two layers. The hidden features and outputs are computed as follows x2 = Sigmoid (Wx1 + b) x3 = W2x2 + b +b2 (1) (2) A MSE loss function L = (t x3)(t - x3) is applied in the end, where t = Rk1 is the target value. Following the chain rule, derive the gradient w' 2' ab' b L L aL in a vectorized format. L Problem 3.2 (5 pts) Replace the Sigmoid function with ReLU function. Given a data x = target value t = [1, 2], weights and bias at this iteration are 3 -1 , b1 21] - 2 - 1 -2 , b -3 1 " == W1 W2 = = = L L L L Following the results in Problem 3.1, calculate the values of L, 1' ' ' b [0, 1, 2], (3) (4)

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Microeconomics An Intuitive Approach with Calculus

Authors: Thomas Nechyba

1st edition

538453257, 978-0538453257

More Books

Students also viewed these Programming questions

Question

4. How do you learn best?

Answered: 1 week ago

Question

5. What would you like to improve about how you learn?

Answered: 1 week ago