Implement a simple Transformer neural network that is composed of the following layers:

$*$ Use BERT as feature extractor for each token.

$*$ A few of transformer encoder layers, hidden dimension $768.$ You need to determine how many layers to use between $1$~$3.$

$*$ A few of transformer decoder layers, hidden dimension $768.$ You need to determine how many layers to use between $1$~$3.$

$*1$ hidden layer with size $512.$

$*$ The final output layer with one cell for binary classification to predict whether two inputs are related or not.

Note that each input for this model should be a concatenation of a positive pair $($i$.$e$.$ question $+$ one answer$)$ or a negative pair $($i$.$e$.$ question $+$ not related sentence$).$ The format is usually like $[$CLS$]+$ question $+[$SEP$]+$ a positive$/$negative sentence.

Train the model with the training data, use the dev$\_$test set to determine a good size of the transformer layers, and report the final results using the test set. Again, remember to use the test set only after you have determined the optimal parameters of the transformer layers.