3

Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)?

image

If so could you give some names or links to research papers?

Shayan Shafiq
  • 350
  • 1
  • 4
  • 12
GKozinski
  • 1,240
  • 8
  • 19
  • This seems to be a duplicate of https://ai.stackexchange.com/q/17190/2444. Well, at least, I answer your question there too. – nbro Oct 29 '20 at 09:17
  • @nbro Well, I don't think this question is a duplicate, but yes there is an answer to my question in the linked one. Thanks! – GKozinski Oct 29 '20 at 09:41

1 Answers1

2

This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: ResNet, SkipRNN. In addition here you can find a paper that empirically explores the skip connections for sequential tagging, or this one for speech enhancement.

razvanc92
  • 1,108
  • 7
  • 18