There is a mathematical challenge of learning long-term dependencies in recurrent networks: gradients propagated over many stages tend to either vanish (most of the time) or explode (rarely, but with much damage to the optimization). To discuss this problem for simplicity, consider the following recurrence relation without a nonlinear activation function and input x: ht=Wht-1 (4) Where W is a weight sharing matrix for recurrent relation at any time t. Let A,..., An be the eigenvalues of the weight matrix WE Cnxn. Its spectral radius p(W) is defined as: p(W) = max{|A|, ..., |An|} Refering to the long-range relationship between the hidden states, discuss the convergence behavior of this RNN in equation 4 with respect to weight matrix W.