Title: Complex  unitary recurrent neural networks using scaled Cayley transform

Abstract: Recurrent neural networks (RNNs) have been successfully used in wide range of sequential problems. Despite this success, RNNs suffer from the vanishing or exploding gradients problem. One recent method ''scaled Cayley orthogonal recurrent neural network'' (scoRNN) addresses this issue by maintaining an orthogonal recurrent weight matrix by parametrizing a skew-symmetric matrix through a scaled Cayley transform. The initial implementation of scoRNN used an orthogonal recurrent matrix and we extend the idea to the complex case using unitary matrices. We discuss the advantage the complex scoRNN has over the traditional scoRNN and implementation issues.