For questions related to layers of neural networks, so you can use this tag if your question involves the input, output, hidden, feedforward, recurrent, convolutional, pooling (aka subsampling), upsampling, etc., layers.
Questions tagged [layers]
7 questions
33
votes
4 answers
How to find the optimal number of neurons per layer?
When you're writing your algorithm, how do you know how many neurons you need per single layer? Are there any methods for finding the optimal number of them, or is it a rule of thumb?
kenorb
- 10,423
- 3
- 43
- 91
5
votes
3 answers
Can you confirm that the transformer works strictly deterministically and there is no randomness inside or between the attention layers?
On a high-level temperature and randomness affect the output of a generative language model:
Lower temperature: Produces more focused, conservative, and consistent responses.
Moderate temperature: Strikes a balance between creativity and…
Hans-Peter Stricker
- 811
- 1
- 8
- 20
5
votes
2 answers
In a neural network, by how much does the number of neurons typically vary from layer to layer?
In a neural network, by how much does the number of neurons typically vary from layer to layer?
Note that I am NOT asking how to find the optimal number of neurons per layer.
As a hardware design engineer with no practical experience programming…
Angela Johnson
- 51
- 1
2
votes
1 answer
What are the major layers in a Vision Transformer?
Currently, I am studying deepfake detection using deep learning methods. Convolution neural networks, recurrent neural networks, long-short term memory networks, and vision transformers are famous deep learning-based methods that are used in…
Pawara Siriwardhane
- 143
- 6
2
votes
0 answers
Has the idea of using different learning rates for different layers been explored in the literature?
I wonder whether there are heuristic rules for the optimal selection of learning rates for different layers. I expect that there is no general recipe, but probably there are some choices that may be beneficial.
The common strategy uses the same…
spiridon_the_sun_rotator
- 2,454
- 8
- 16
1
vote
1 answer
Formally, what are the layers in an Artificial Neural Network?
You may not believe it, but I am an ANN expert. Perhaps, for that reason, I am unable to grasp completely what the layers are in a Deep Forward Artificial Neural Network (DFANN).
According to the Deep Learning "bible", p. 164 "the model is…
neoglez
- 11
- 2
0
votes
0 answers
Layer Questions regarding Bidirectional VAE (D3VAE)
I am currently trying to figure out how D3VAE are working, but I can't seem to understand the network architecture given. The paper can be found here:
https://openreview.net/pdf?id=rG0jm74xtx
The things I don't understand in the picture can be…
