Answered step by step
Verified Expert Solution
Question
1 Approved Answer
This question is about convergence. All the unconstrained optimization methods we have seen are iterative schemes that can be defined as follows: xk+1=G(xk) for some
This question is about convergence. All the unconstrained optimization methods we have seen are iterative schemes that can be defined as follows: xk+1=G(xk) for some function G:RnRn. For example, for the steepest descent method with fixed step size , we have G(xk)=xk+(f(xk)). A fixed-point of such an iterative scheme is any point x satisfying x=G(x) We define the error as we have seen in class: e(xk)=xkx=G(xk1)x (as usual, the norm is the 2-norm). In this question we are interested in the question of the convergence of such a scheme to a fixed-point, or equivalently the convergence of e to 0 . We first look (questions 1,2,3 ) at what happens when n=1, i.e. x is actually a scalar x and G:RR; you have seen such iterative schemes in Calc I, for example the Newton method to find the zero of f. 2. Using Taylor expansion (you can disregard the o(.)term)aroundx, find which condition G(x) needs to satisfy in order that we have linear convergence. What is the rate of convergence then? This question is about convergence. All the unconstrained optimization methods we have seen are iterative schemes that can be defined as follows: xk+1=G(xk) for some function G:RnRn. For example, for the steepest descent method with fixed step size , we have G(xk)=xk+(f(xk)). A fixed-point of such an iterative scheme is any point x satisfying x=G(x) We define the error as we have seen in class: e(xk)=xkx=G(xk1)x (as usual, the norm is the 2-norm). In this question we are interested in the question of the convergence of such a scheme to a fixed-point, or equivalently the convergence of e to 0 . We first look (questions 1,2,3 ) at what happens when n=1, i.e. x is actually a scalar x and G:RR; you have seen such iterative schemes in Calc I, for example the Newton method to find the zero of f. 2. Using Taylor expansion (you can disregard the o(.)term)aroundx, find which condition G(x) needs to satisfy in order that we have linear convergence. What is the rate of convergence then
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started