Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Is this implementation of L - BFGS method correct? def l _ bfgs ( w _ 0 , data, labels, m = 1 0 ,

Is this implementation of L-BFGS method correct?
def l_bfgs(w_0, data, labels, m=10, pred_f=prediction, grad_f=gradient, loss_f=logloss, max_iter=100, tol=0.0001):
w = w_0
n = len(w)
s_list =[]
y_list =[]
rho_list =[]
alpha_list =[]
grad_w = grad_f(n, w, data, labels)
for i in range(max_iter):
q = grad_w
alpha_list.clear()
for j in range(len(s_list)-1,-1,-1):
s = s_list[j]
y = y_list[j]
rho = rho_list[j]
alpha = rho * np.dot(s, q)
alpha_list.append(alpha)
q = q - alpha * y
if len(s_list)>0:
gamma = np.dot(s_list[-1], y_list[-1])/ np.dot(y_list[-1], y_list[-1])
H0= gamma * np.eye(n)
else:
H0= np.eye(n)
z = np.dot(H0, q)
for j in range(len(s_list)):
s = s_list[j]
y = y_list[j]
rho = rho_list[j]
beta = rho * np.dot(y, z)
alpha = alpha_list[len(s_list)-1- j]
z = z + s *(alpha - beta)
p =-z
alpha = wolfe(w, p, data, labels)
s = alpha * p
w_new = w + s
grad_new = grad_f(n, w_new, data, labels)
if np.linalg.norm(grad_new - grad_w)< tol:
break
y = grad_new - grad_w
rho =1.0/ np.dot(y, s)
if len(s_list)== m:
s_list.pop(0)
y_list.pop(0)
rho_list.pop(0)
s_list.append(s)
y_list.append(y)
rho_list.append(rho)
grad_w = grad_new
predictions = pred_f(w, data)
loss = loss_f(predictions, labels)
print("iter:", i," loss:", loss)
w = w_new
return w

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

C++ Database Development

Authors: Al Stevens

1st Edition

1558283579, 978-1558283572

More Books

Students also viewed these Databases questions

Question

Do you think physicians should have unions? Why or why not?

Answered: 1 week ago