2

I noticed that the TensorFlow library includes a use_bias parameter for the Dense layer, which is set to True by default, but allows you to disable it. At first glance, it seems unfavorable to turn off the biases, as this may negatively affect data fitting and prediction.

What is the purpose of layers without biases?

nbro
  • 39,006
  • 12
  • 98
  • 176
mark mark
  • 753
  • 4
  • 23
  • Before anything, having option is not bad. Besides I think some applications do not need bias. For example if if I'm not wrong, in CBOW (Continuous Bag Of Words for word embedding) you don't need bias. – amin Dec 06 '20 at 08:56
  • @amin You say "Besides I think some applications do not need bias.", but I think the question is really: "**Why** do some applications not use/need bias". If you don't have the answer to this question, besides providing the usual explanation of why the bias can be useful, then I think you should upvote this question/post, so that maybe it attracts someone that knows the answer. Personally, I don't know the answer of when you would strictly not need the bias (and thus when you can disable it), so it would be nice to have some guidance on when to disable it (because not really necessary). – nbro Dec 06 '20 at 12:17
  • @nbro, is this the right stackexchange to ask questions like this, or it is more datascience? Where is the line drawn? – mark mark Dec 09 '20 at 22:52
  • @markmark This is a theoretical ML question, so it is perfectly suitable for AI SE, and I think it's a question that other people could also have, so I think it's useful. Unfortunately, it has not received too much attention, but this is more or less normal (especially, in the last months): sadly, there aren't many people contributing to the site as much as I would like. In any case, I think this question would also be on-topic on Data Science SE and Cross Validated SE. – nbro Dec 09 '20 at 23:00

1 Answers1

1

Bias is one of the hyperparameters in neural networks, which let you shift activation function. Disabling bias means setting bias to be zero.

Even though, in many cases, bias is a big help for successful learning, in some cases, you may want to add an extra constraint to your neural network in finding the objective function. For example, in the paper below, a zero-bias layer as the last layer helps increase the output's interpretability.

https://ieeexplore.ieee.org/abstract/document/9173537

kiarash_kiani
  • 51
  • 5
  • It would be nice if you could describe in which sense no bias makes the output more interpretable (according to that paper). – nbro Dec 06 '20 at 17:40