1

Less popular alternatives include adding layers to the connections from input to the hidden state, between hidden states, or from the hidden state to the output. These designs employ skip connections to avoid a situation where the shortest path between time steps increases and training becomes more difficult.

This is from page 597 of the book Machine Learning for Algorithmic Trading. And this chapter is about RNN.

I don't quite understand what does 'These designs employ skip connections to avoid a situation where the shortest path between time steps increases' means. I haven't seen any materials about how to use skip connections in RNN.

nbro
  • 39,006
  • 12
  • 98
  • 176
user900476
  • 121
  • 3
  • My guess (based on my separate knowledge of RNNs and skip connections, so take this comment with a grain of salt) is that skip connections in RNNs have a similar purpose as skip connections in other neural networks. Here, by "shortest path between time steps increases", I think they mean that it's harder to keep track of longer dependencies between words (chars, or whatever is the unit of a sequence, in your case probably stocks) of the sequence as those dependencies are farther away from each other. So, I guess skip connections would help to overcome that to some extent. – nbro Jan 01 '22 at 21:08

0 Answers0