The convolution layer inside of a CNN is intrinsically an affine trans$-$

formation: A vector is received as input and is multiplied with a matrix

to produce an output $($to which a bias vector is usually added before

passing the result through a nonlinearity$).$ This operation can be rep$-$

resented as $y=Ax,$ in which A describes the affine transformation.

We will first revisit the convolution layer as discussed in the class.

Consider a convolution layer with a $33$ kernel $W,$ operated on a single

input channel $x,$ represented as:

$W=\left[\begin{array}{ccc}{w}_{(0,0)}& w_{(0,1)}& w_{(0,2)}\\ w_{(1,0)}& w_{(1,1)}& w_{(1,2)}\\ w_{(2,0)}& w_{(2,1)}& w_{(2,2)}\end{array}\right],x=\left[\begin{array}{ccc}{x}_{(0,0)}& x_{(0,1)}& x_{(0,2)}\\ x_{(1,0)}& x_{(1,1)}& x_{(1,2)}\\ x_{(2,0)}& x_{(2,1)}& x_{(2,2)}\end{array}\right]$

Using this example, let us work out a stride$-4$ convolution layer, with

zero padding size of $2.$ Consider 'flattening' the input tensor $x$ in

row$-$major order as:

$x=[x_{(0,0)},x_{(0,1)},x_{(0,2)},$dots,$x_{(2,0)},x_{(2,1)},x_{(2,2)}{]}^{TT}$

Write down this convolution as a matrix operation A such that: $Y=$

$Ax.$ Output $Y$ is also flattened in row$-$major order. NOTE: For this

problem we are referring to a convolution in the deep learning context

$($i$.$e$.$ do not flip the kernel$).$

$($a$)$ What are the dimensions of $A?$

$($b$)$ Write down the entries in this matrix A$.$