Constrained least squares. Suppose we wish to find the least-squares estimator of $boldsymbol{beta}$ in the model $mathbf{y}=mathbf{X}
Question:
Constrained least squares. Suppose we wish to find the least-squares estimator of $\boldsymbol{\beta}$ in the model $\mathbf{y}=\mathbf{X} \boldsymbol{\beta}+\boldsymbol{\varepsilon}$ subject to a set of equality constraints on $\boldsymbol{\beta}$, say $\mathbf{T} \boldsymbol{\beta}=\mathbf{c}$. Show that the estimator is
\[\tilde{\boldsymbol{\beta}}=\hat{\boldsymbol{\beta}}+\left(\mathbf{X}^{\prime} \mathbf{X}\right)^{-1} \mathbf{T}^{\prime}\left[\mathbf{T}\left(\mathbf{X}^{\prime} \mathbf{X}\right)^{-1} \mathbf{T}\right]^{-1}(\mathbf{c}-\mathbf{T} \hat{\boldsymbol{\beta}})\]
where $\hat{\boldsymbol{\beta}}=\left(\mathbf{X}^{\prime} \mathbf{X}\right)^{-1} \mathbf{X}^{\prime} \mathbf{y}$. Discuss situations in which this constrained estimator might be appropriate. Find the residual sum of squares for the constrained estimator. Is it larger or smaller than the residual sum of squares in the unconstrained case?
Step by Step Answer:
Introduction To Linear Regression Analysis
ISBN: 9781119578727
6th Edition
Authors: Douglas C. Montgomery, Elizabeth A. Peck, G. Geoffrey Vining