To develop a feel as to why the estimators should be obtained by minimizing the weighted sum of squares rather than the ordinary sum of squares, consider the following situation. Suppose that X1, . . . , Xn are independent normal random variables each having mean μ and variance σ2. Suppose further that the Xi are not directly observable but rather only Y1 and Y2, defined by

are directly observable. Based on Y1 and Y2, how should we estimate μ?
Whereas the best estimator of μ is clearly X = ni =1 Xi = (Y1 + Y2), let us see what the ordinary least squares estimator would be. Since

Transcribed Image Text:

Y = X + + X Y =X+++X k < n