The convolution layer inside of a CNN is intrinsically an affine transformation: A vector is received as input and is multiplied with a matrix to produce an output $($to which a bias vector is usually added before passing the result through a nonlinearity$).$ This operation can be represented as y$=$Ax in which A describes the affine transformation.

We will first revisit the convolution layer as discussed in the class. Consider a convolution layer with a $3$ x $3$ kernel W operated on a single input channel X represented as: