In Elman$(1990),$ the hidden layer activation vectors of a trained simple recurrent network were found to cluster according to word classes such as : no

the current word presented at the network input

the next word, not uet presented the network iinut

the word class of teh current word presented at the netweork input