1. Root-finding algorithms

A root of a function g(theta) is a solution to the equation g(theta) = 0. Root-finding is an important computational problem that arises commonly across many disciplines. For example, in statistics, root-finding can be used in maximum likelihood estimation. Here you are required to find a maximum likelihood estimate for the Cauchy distribution with the help of root-finding algorithms implemented in C++ with Rcpp. The Cauchy distribution with unknown location parameter theta and scale parameter set to gamma = 1 is a continuous probability distribution, with density function:

pdf (x) = 1/pi {1 + (x+theta)^2] -> 1

, For n independent observations {xi, x2, , xn} from this Cauchy distribution, the log likelihood is calculated as:

l(theta) = n log(pi) Sum( log(1 + (xi theta)^2) ) -> 2 Consider the following numeric vector in R storing observations generated from a Cauchy distribution with location parameter theta and scale parameter gamma = 1:

x <- c( 12.262307, 10.281078, 10.287090, 12.734039, 3 11.731881, 8.861998, 12.246609, 11.244818, 4 9.696278, 11.667672, 11.112631. 10.660190, 5 9.018438, 10.704774, 9.616617, 10.003247, 8 10.278362, 9.709630, 10.963906, 17.314814)

A researcher wishes to estimate the maximum likelihood estimate theta hat for the Cauchy distribution from which the data in x were generated. A maximum likelihood estimate for theta can be found by differentiating the log likelihood, equating to zero, and solving for theta i.e. we can find a maximum likelihood estimate for the above Cauchy distribution by finding a root of l'(theta), the first derivative of the log likelihood. Taking the first derivative and equating to zero, gives the following equation

l'(theta) = 2.Sum(xi theta)/1 + (xi theta)^2) = 0 -> 3

A maximum likelihood estimate can be obtained by solving the equation l'(thet) = 0 for theta i.e. by finding a root of the function l'(theta). There are several numerical optimization algorithms available for finding the roots of continuous univariate functions such as l'(theta), including the: Bisection method

Newton's method

Secant method

For this assignment, implement each of the above three root finding algorithms in C/C++ with Rcpp, to find the MLE of theta for the data x.

Each function should take the vector x as input from R (along with any other required inputs), and return the maximum likelihood estimate theta hat to R. Ensure that all code submitted is well commented.

Briefly describe each implementation in your submitted pdf document.

Evaluate and discuss the behaviour of each implemention with different starting values.

Benchmark your implementations against each other.

Discuss their advantages and limitations.

Comment on the suitability and efficiency of each implementation.

Finally, propose an optimal implementation of a root-finding algorithm for estimating the MLE.

The efficiency, accuracy and readability of your implementations will be considered in your grade.