Weighted Least Squares

5. (Weighted Least Squares) Consider the simple linear regression model 1%: = 50 + 51% + 5i where E03,) 2 0 and the errors are independent, but Var(5,) = 02/19,, where the w, are known constants, so the errors do not have equal variance. Because the variances are not equal, the standard theory does not apply. Intuitively, it seems that the observations with large variability should inuence the estimates of [30 and )31 less than the observations with small variability. The problem can be transformed by multiplying y,- by $3,, so we have a new model \\/'wr'. :2 = \\f'wz' + \\f'wa'w'i + \\/wi5i Zn: = our + 23101: + 6i where Z}; = 1111),: \"Hg 2 \"103311,; 2 vat-:12\" and 5,; = Milka. (a) Show that the new model satises the assumptions of the standard simple linear regression model. (b) Show that the least squares estimates for 50 and ,81 are given by . A . 2104552: iwxyz' 17w) o=w15w and 31: 211, 2 212,012,; 551132 21 where 53,, and 3),, are the weighted means E wax: Z \"wt-ya _ 1:1 _ 1221 w = n and 3;,\" = n X we: 2 ma. 3:1 :1 The estimates [3'0 and Bl are known as the weighted least squares estimates. (c) Show that performing a least squares analysis on the new model is equivalent to minimizing 2: mi (1% 230 2313302- i=1 This is a criterion of the weighted least squares method: The observations with large variances are weighted less. ((1) Optional Challenge: Find the variances of the estimates in part (b)