Let \(\mathbf{f}(\boldsymbol{\theta})\) be a \(n \times 1\) and \(\mathbf{g}(\boldsymbol{\theta})\) a \(1 \times m\) vector-valued function of \(\boldsymbol{\theta}=\left(\theta_{1}, \ldots, \theta_{q}\right)\). The derivatives of \(\frac{\partial}{\partial \boldsymbol{\theta}} \mathbf{f}\) and \(\frac{\partial}{\partial \boldsymbol{\theta}} \mathbf{g}\) are defined as follows:

\[\frac{\partial}{\partial \boldsymbol{\theta}} \mathbf{f}=\left(\begin{array}{ccc}\frac{\partial f_{1}}{\partial \theta_{1}} & \cdots & \frac{\partial f_{n}}{\partial \theta_{1}} \tag{1.30}\\\vdots & \ddots & \vdots \\\frac{\partial f_{1}}{\partial \theta_{q}} & \cdots & \frac{\partial f_{n}}{\partial \theta_{q}}\end{array}\right)_{q \times n} \quad, \quad \frac{\partial}{\partial \theta} \mathbf{g}=\left(\begin{array}{ccc}\frac{\partial g_{1}}{\partial \theta_{1}} & \cdots & \frac{\partial g_{1}}{\partial \theta_{q}} \\\vdots & \ddots & \vdots \\\frac{\partial g_{m}}{\partial \theta_{1}} & \cdots & \frac{\partial g_{m}}{\partial \theta_{q}}\end{array}\right)_{m \times q}\]

Thus, \(\frac{\partial}{\partial \theta} \mathbf{g}=\left(\frac{\partial}{\partial \theta} \mathbf{g}^{\top}\right)^{\top}\). As a special case, if \(f(\boldsymbol{\theta})\) is a scalar function, it follows from (1.30) that \(\frac{\partial f}{\partial \theta}=\left(\frac{\partial}{\partial \theta_{1}}

f, \ldots, \frac{\partial}{\partial \theta_{q}} f\right)^{\top}\) is a \(q \times 1\) column vector. Let \(A\) be a \(m \times n\) matrix of constants, \(\mathbf{g}(\boldsymbol{\theta})\) a \(m \times 1\) vector-valued function of \(\boldsymbol{\theta}\), and \(h(\boldsymbol{\theta})\) a function of \(\boldsymbol{\theta}\). Then, we have the following:

(a) \(\frac{\partial}{\partial \boldsymbol{\theta}}(A \mathbf{f})=\left(\frac{\partial}{\partial \boldsymbol{\theta}} \mathbf{f}\right) A^{\top}\),

(b) \(\frac{\partial}{\partial \boldsymbol{\theta}}(h \mathbf{f})=\left(\frac{\partial}{\partial \boldsymbol{\theta}} h\right) \mathbf{f}^{\top}+h \frac{\partial}{\partial \theta} \mathbf{f}\),

(c) \(\frac{\partial}{\partial \theta}\left(\mathbf{g}^{\top} A \mathbf{f}\right)=\left(\frac{\partial}{\partial \theta} \mathbf{g}\right) A \mathbf{f}+\left(\frac{\partial}{\partial \theta} \mathbf{f}\right) A^{\top} \mathbf{g}\).