Questions and Answers of Pattern Recognition And Machine Learning

8.29 ( ) www Show that if the sum-product algorithm is run on a factor graph with a tree structure (no loops), then after a finite number of messages have been sent, there will be no pending messages.
8.28 ( ) www The concept of a pending message in the sum-product algorithm for a factor graph was defined in Section 8.4.7. Show that if the graph has one or more cycles, there will always be at
8.27 ( ) Consider two discrete variables x and y each having three possible states, for example x, y ∈ {0, 1, 2}. Construct a joint distribution p(x, y) over these variables having the property
8.26 () Consider a tree-structured factor graph over discrete variables, and suppose we wish to evaluate the joint distribution p(xa, xb) associated with two variables xa and xb that do not belong
8.25 ( ) In (8.86), we verified that the sum-product algorithm run on the graph in Figure 8.51 with node x3 designated as the root node gives the correct marginal for x2. Show that the correct
8.24 ( ) Show that the marginal distribution for the variables xs in a factor fs(xs) in a tree-structured factor graph, after running the sum-product message passing algorithm, can be written as the
8.23 ( ) www In Section 8.4.4, we showed that the marginal distribution p(xi) for a variable node xi in a factor graph is given by the product of the messages arriving at this node from neighbouring
8.22 () Consider a tree-structured factor graph, in which a given subset of the variable nodes form a connected subgraph (i.e., any variable node of the subset is connected to at least one of the
8.21 ( ) www Show that the marginal distributions p(xs) over the sets of variables xs associated with each of the factors fx(xs) in a factor graph can be found by first running the sum-product
8.20 () www Consider the message passing protocol for the sum-product algorithm on a tree-structured factor graph in which messages are first propagated from the leaves to an arbitrarily chosen root
8.19 ( ) Apply the sum-product algorithm derived in Section 8.4.4 to the chain-ofnodes model discussed in Section 8.4.1 and show that the results (8.54), (8.55), and(8.57) are recovered as a special
8.18 ( ) www Show that a distribution represented by a directed tree can trivially be written as an equivalent distribution over the corresponding undirected tree. Also show that a distribution
8.17 ( ) Consider a graph of the form shown in Figure 8.38 having N = 5 nodes, in which nodes x3 and x5 are observed. Use d-separation to show that x2 ⊥⊥ x5 | x3.Show that if the message passing
8.16 ( ) Consider the inference problem of evaluating p(xn|xN) for the graph shown in Figure 8.38, for all nodes n ∈ {1, . . . , N − 1}. Show that the message passing algorithm discussed in
8.15 ( ) www Show that the joint distribution p(xn−1, xn) for two neighbouring nodes in the graph shown in Figure 8.38 is given by an expression of the form (8.58).
8.14 () Consider a particular case of the energy function given by (8.42) in which the coefficients β = h = 0. Show that the most probable configuration of the latent variables is given by xi = yi
8.13 () Consider the use of iterated conditional modes (ICM) to minimize the energy function given by (8.42). Write down an expression for the difference in the values of the energy associated with
8.12 () www Show that there are 2M(M−1)/2 distinct undirected graphs over a set of M distinct random variables. Draw the 8 possibilities for the case of M = 3.
8.11 ( ) Consider the example of the car fuel system shown in Figure 8.21, and suppose that instead of observing the state of the fuel gauge G directly, the gauge is seen by the driver D who reports
8.10 () Consider the directed graph shown in Figure 8.54 in which none of the variables is observed. Show that a ⊥⊥ b | ∅. Suppose we now observe the variabled. Show that in general a⊥ ⊥ b
8.9 () www Using the d-separation criterion, show that the conditional distribution for a node x in a directed graph, conditioned on all of the nodes in the Markov blanket, is independent of the
8.8 () www Show that a ⊥⊥b, c | d implies a ⊥⊥ b | d.
8.7 ( ) Using the recursion relations (8.15) and (8.16), show that the mean and covariance of the joint distribution for the graph shown in Figure 8.14 are given by (8.17)and (8.18), respectively.
8.6 () For the model shown in Figure 8.13, we have seen that the number of parameters required to specify the conditional distribution p(y|x1, . . . , xM), where xi ∈ {0, 1}, could be reduced from
8.5 () www Draw a directed probabilistic graphical model corresponding to the relevance vector machine described by (7.79) and (7.80).
8.4 ( ) Evaluate the distributions p(a), p(b|c), and p(c|a) corresponding to the joint distribution given in Table 8.2. Hence show by direct evaluation that p(a,b, c) =p(a)p(c|a)p(b|c). Draw the
8.3 ( ) Consider three binary variablesa, b, c ∈ {0, 1} having the joint distribution given in Table 8.2. Show by direct evaluation that this distribution has the property that a and b are
8.2 () www Show that the property of there being no directed cycles in a directed graph follows from the statement that there exists an ordered numbering of the nodes such that for each node there
8.1 () www By marginalizing out the variables in order, show that the representation(8.5) for the joint distribution of a directed graph is correctly normalized, provided each of the conditional
7.19 ( ) Verify that maximization of the approximate log marginal likelihood function(7.114) for the classification relevance vector machine leads to the result (7.116) for re-estimation of the
7.18 () www Show that the gradient vector and Hessian matrix of the log posterior distribution (7.109) for the classification relevance vector machine are given by(7.110) and (7.111).
7.17 ( ) Using (7.83) and (7.86), together with the matrix identity (C.7), show that the quantities Sn and Qn defined by (7.102) and (7.103) can be written in the form(7.106) and (7.107).
7.16 () By taking the second derivative of the log marginal likelihood (7.97) for the regression RVM with respect to the hyperparameter αi, show that the stationary point given by (7.101) is a
7.15 ( ) www Using the results (7.94) and (7.95), show that the marginal likelihood(7.85) can be written in the form (7.96), where λ(αn) is defined by (7.97) and the sparsity and quality factors
7.14 ( ) Derive the result (7.90) for the predictive distribution in the relevance vector machine for regression. Show that the predictive variance is given by (7.91).
7.13 ( ) In the evidence framework for RVM regression, we obtained the re-estimation formulae (7.87) and (7.88) by maximizing the marginal likelihood given by (7.85).Extend this approach by inclusion
7.12 ( ) www Show that direct maximization of the log marginal likelihood (7.85) for the regression relevance vector machine leads to the re-estimation equations (7.87)and (7.88) where γi is defined
7.11 ( ) Repeat the above exercise, but this time make use of the general result (2.115).
7.10 ( ) www Derive the result (7.85) for the marginal likelihood function in the regression RVM, by performing the Gaussian integral over w in (7.84) using the technique of completing the square in
7.9 () Verify the results (7.82) and (7.83) for the mean and covariance of the posterior distribution over weights in the regression RVM.
7.8 () www For the regression support vector machine considered in Section 7.1.4, show that all training data points for which ξn > 0 will have an = C, and similarly all points for whichξn > 0
7.7 () Consider the Lagrangian (7.56) for the regression support vector machine. By setting the derivatives of the Lagrangian with respect to w,b, ξn, andξn to zero and then back substituting to
7.6 () Consider the logistic regression model with a target variable t ∈ {−1, 1}. If we define p(t = 1|y) = σ(y) where y(x) is given by (7.1), show that the negative log likelihood, with the
7.5 ( ) Show that the values of ρ and {an} in the previous exercise also satisfy 1ρ2 = 2L(a) (7.124)whereL(a) is defined by (7.10). Similarly, show that 1ρ2 = w2. (7.125)
7.4 ( ) www Show that the value ρ of the margin for the maximum-margin hyperplane is given by 1ρ2 =N n=1 an (7.123)where {an} are given by maximizing (7.10) subject to the constraints (7.11)
7.3 ( ) Show that, irrespective of the dimensionality of the data space, a data set consisting of just two data points, one from each class, is sufficient to determine the location of the
7.2 () Show that, if the 1 on the right-hand side of the constraint (7.5) is replaced by some arbitrary constant γ > 0, the solution for the maximum margin hyperplane is unchanged.
7.1 ( ) www Suppose we have a data set of input vectors {xn} with corresponding target values tn ∈ {−1, 1}, and suppose that we model the density of input vectors within each class separately
6.27 ( ) Derive the result (6.90) for the log likelihood function in the Laplace approximation framework for Gaussian process classification. Similarly, derive the results(6.91), (6.92), and (6.94)
6.26 () Using the result (2.115), derive the expressions (6.87) and (6.88) for the mean and variance of the posterior distribution p(aN+1|tN) in the Gaussian process classification model.
6.25 () www Using the Newton-Raphson formula (4.92), derive the iterative update formula (6.83) for finding the mode aN of the posterior distribution in the Gaussian process classification model.
6.24 () Show that a diagonal matrixWwhose elements satisfy 0
6.23 ( ) www Consider a Gaussian process regression model in which the target variable t has dimensionality D. Write down the conditional distribution of tN+1 for a test input vector xN+1, given a
6.22 ( ) Consider a regression problem with N training set input vectors x1, . . . , xN and L test set input vectors xN+1, . . . , xN+L, and suppose we define a Gaussian process prior over functions
6.21 ( ) www Consider a Gaussian process regression model in which the kernel function is defined in terms of a fixed set of nonlinear basis functions. Show that the predictive distribution is
6.20 ( ) www Verify the results (6.66) and (6.67).
6.19 ( ) Another viewpoint on kernel regression comes from a consideration of regression problems in which the input variables as well as the target variables are corrupted with additive noise.
6.18 () Consider a Nadaraya-Watson model with one input variable x and one target variable t having Gaussian components with isotropic covariances, so that the covariance matrix is given by σ2I
6.17 ( ) www Consider the sum-of-squares error function (6.39) for data having noisy inputs, where ν(ξ) is the distribution of the noise. Use the calculus of variations to minimize this error
6.16 ( ) Consider a parametric model governed by the parameter vector w together with a data set of input values x1, . . . , xN and a nonlinear feature mapping φ(x).Suppose that the dependence of
6.15 () By considering the determinant of a 2 × 2 Gram matrix, show that a positivedefinite kernel function k(x, x) satisfies the Cauchy-Schwartz inequality k(x1, x2)2 k(x1, x1)k(x2, x2). (6.96)
6.14 () www Write down the form of the Fisher kernel, defined by (6.33), for the case of a distribution p(x|μ) = N(x|μ, S) that is Gaussian with mean μ and fixed covariance S.
6.13 () Show that the Fisher kernel, defined by (6.33), remains invariant if we make a nonlinear transformation of the parameter vector θ → ψ(θ), where the functionψ(·) is invertible and
6.12 ( ) www Consider the space of all possible subsets A of a given fixed set D.Show that the kernel function (6.27) corresponds to an inner product in a feature space of dimensionality 2|D| defined
6.11 () By making use of the expansion (6.25), and then expanding the middle factor as a power series, show that the Gaussian kernel (6.23) can be expressed as the inner product of an
6.10 () Show that an excellent choice of kernel for learning a function f(x) is given by k(x, x) = f(x)f(x) by showing that a linear learning machine based on this kernel will always find a
6.9 () Verify the results (6.21) and (6.22) for constructing valid kernels.
6.8 () Verify the results (6.19) and (6.20) for constructing valid kernels.
6.7 () www Verify the results (6.17) and (6.18) for constructing valid kernels.
6.6 () Verify the results (6.15) and (6.16) for constructing valid kernels.
6.5 () www Verify the results (6.13) and (6.14) for constructing valid kernels.
6.4 () In Appendix C, we give an example of a matrix that has positive elements but that has a negative eigenvalue and hence that is not positive definite. Find an example of the converse property,
6.3 () The nearest-neighbour classifier (Section 2.5.2) assigns a new input vector x to the same class as that of the nearest input vector xn from the training set, where in the simplest case, the
6.2 ( ) In this exercise, we develop a dual formulation of the perceptron learning algorithm. Using the perceptron learning rule (4.55), show that the learned weight vector w can be written as a
6.1 ( ) www Consider the dual formulation of the least squares linear regression problem given in Section 6.1. Show that the solution for the components an of the vector a can be expressed as a
5.41 ( ) By following analogous steps to those given in Section 5.7.1 for regression networks, derive the result (5.183) for the marginal likelihood in the case of a network having a cross-entropy
5.40 () www Outline the modifications needed to the framework for Bayesian neural networks, discussed in Section 5.7.3, to handle multiclass problems using networks having softmax output-unit
5.39 () www Make use of the Laplace approximation result (4.135) to show that the evidence function for the hyperparameters α and β in the Bayesian neural network model can be approximated by
5.38 () Using the general result (2.115), derive the predictive distribution (5.172) for the Laplace approximation to the Bayesian neural network model.
5.37 () Verify the results (5.158) and (5.160) for the conditional mean and variance of the mixture density network model.
5.36 () Derive the result (5.157) for the derivative of the error function with respect to the network output activations controlling the component variances in the mixture density network.
5.35 () Derive the result (5.156) for the derivative of the error function with respect to the network output activations controlling the component means in the mixture density network.
5.34 () www Derive the result (5.155) for the derivative of the error function with respect to the network output activations controlling the mixing coefficients in the mixture density network.
5.33 () Write down a pair of equations that express the Cartesian coordinates (x1, x2)for the robot arm shown in Figure 5.18 in terms of the joint angles θ1 and θ2 and the lengths L1 and L2 of the
5.32 ( ) Show that the derivatives of the mixing coefficients {πk}, defined by (5.146), with respect to the auxiliary parameters {ηj} are given by∂πk∂ηj= δjkπj − πjπk. (5.208)Hence, by
5.31 () Verify the result (5.143).
5.30 () Verify the result (5.142).
5.29 () www Verify the result (5.141).
5.28 () www Consider a neural network, such as the convolutional network discussed in Section 5.5.6, in which multiple weights are constrained to have the same value.Discuss how the standard
5.27 ( ) www Consider the framework for training with transformed data in the special case in which the transformation consists simply of the addition of random noise x → x + ξ where ξ has a
5.26 ( ) Consider a multilayer perceptron with arbitrary feed-forward topology, which is to be trained by minimizing the tangent propagation error function (5.127) in which the regularizing function
5.25 ( ) www Consider a quadratic error function of the form E = E0 +1 2(w − w)TH(w − w) (5.195)where w represents the minimum, and the Hessian matrixHis positive definite and constant.
5.24 () Verify that the network function defined by (5.113) and (5.114) is invariant under the transformation (5.115) applied to the inputs, provided the weights and biases are simultaneously
5.23 ( ) Extend the results of Section 5.4.5 for the exact Hessian of a two-layer network to include skip-layer connections that go directly from inputs to outputs.
5.22 ( ) Derive the results (5.93), (5.94), and (5.95) for the elements of the Hessian matrix of a two-layer feed-forward network by application of the chain rule of calculus.
5.21 ( ) Extend the expression (5.86) for the outer product approximation of the Hessian matrix to the case ofK > 1 output units. Hence, derive a recursive expression analogous to (5.87) for
5.20 () Derive an expression for the outer product approximation to the Hessian matrix for a network having K outputs with a softmax output-unit activation function and a cross-entropy error
5.19 () www Derive the expression (5.85) for the outer product approximation to the Hessian matrix for a network having a single output with a logistic sigmoid output-unit activation function and a
5.18 () Consider a two-layer network of the form shown in Figure 5.1 with the addition of extra parameters corresponding to skip-layer connections that go directly from the inputs to the outputs. By
5.17 () Consider a squared loss function of the form E =1 2{y(x,w) − t}2 p(x, t) dx dt (5.193)where y(x,w) is a parametric function such as a neural network. The result (1.89)shows that the

Showing 1 - 100 of 269