Initialization of the hidden state and the memory cell

[shuaitang]

Hi all,

I have been using cudnn.torch binding for cudnn-rnn for quite a long time, and an issue confused me a lot.

when I set the input sequence to be input = torch.zeros(1,5,1), and pass it thru the cudnn.GRU module, the outputs are not a bunch of 0s, which should be if the initialization of the hidden state is 0 at the very beginning.

Any thoughts about the issue?

Thank you!

[guillaumekln]

Hi,

The biases are most certainly non zero.

[shuaitang]

Thank you for answering my question!

But I also tried to set the bias to zeros, and then forward the cudnn.GRU module with all-zero input, the results are still non-zero.

Any thoughts?

Thank you!