Consider the case that, in building linear regression models, there is a concern that some data points may be more important (or more trustable).

Q1: Consider the case that, in building linear regression models, there is a concern that some data points may be more important (or more trustable). For these cases, it is not uncommon to assign a weight to each data point. Denote the weight for the ith data point as w. An example is shown in the data table below, as the last column, e.g., W1 = 1, W2 = 2, W5 = 3. \"In. ---El We still want to estimate the regression parameters in the least-squares framework. Follow the process of the derivation of the least-squares estimator as shown in Chapter 2, and propose your new estimator of the regression parameters. P y = 30 + Zimi + a. (13) i=1 To fit this multivariate linear regression model with p predictors, we collect N data points, denoted as 191 1 9311 3321 $131 :92 1 $12 $22 $132 y = a X : ZUN 1 $1N $2N 113pN where y E RNX1 denotes for the N measurements of the outcome variable, and X E RNX(p+1) denotes for the data matrix that includes the N measurements of the p input variables and the intercept term, ['30, i.e., the first column of X corresponds to 50.18 To estimate the regression coefficients in Eq. (13), again, we use the least squares estimation method. The first step is to calculate the sum of the squared of the vertical derivations of the observed data points from \"the line\" 19. Following Eq. (7), we can define the residual as P in]. Then, following Eq. (8), the sum of the squared of the vertical derivations ofthe observed data points from \"the line\" is N 1(p0,...,5p) = Zea. (15) n=1 This is again an unconstrained continuous optimization problem, that could be solved by the same procedure we have done for the simple linear regression model. Here, we show how a vector-/matrixbased representation of this derivation process could make things easier. Let's write up the regression coefficients and residuals in vector forms as W Q m Q B: . ,m52 , p 5N Here, )6 E R(p+1)X1 denotes for the regression parameters and e E RNX1 denotes for the N residuals which are assumed to follow a normal distribution with mean as zero and variance as a2 8. Then, based on Eq. (14), we rewrite e as s = y XB. Eq. (15) could be rewritten as 1(3) = (y 103)le Xi?)- (15) To estimate is to solve the optimization problem ngno X)T(y X3). To solve this problem, we can take the gradients of the objective function regarding [9 and set them to be zero 8(3) X)T(y X3) 2 as 0' which gives rise to the equation XT(y X3) = 0. This leads to the least squares estimator of as {3' = (XTx) '1XTy. (17)