Questions tagged [bias]

12 questions
2
votes
3 answers

Why my classification results are correlated with the proportionality of my data?

I'm facing a problem. I'm working on mixed data model with NN (MLP & Word Embedding). My results are not pretty good. And I observed that the proportionality of my data are corelated with my classification results. I explain: As you can see, I have…
2
votes
0 answers

Why do neural networks have bias units?

Why do neural networks have bias units? Why is it sometimes okay to opt them out?
Dinamo
  • 21
  • 1
1
vote
0 answers

Does the number of biases in a Convolutional layer scale with the number of images?

I am convolving 32 grey scale images of size 28 x28 with 16 filters of size 5x5. Which of the following is the correct way to add biases to the convolution operation output? add 1 scalar value bias for each of the 16 filters. (Each of the 32 images…
The Ace
  • 11
  • 2
1
vote
1 answer

Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression

Below are the two tensors [ 73., 67., 43.] [ 91., 88., 64.], [ 87., 134., 58.], [102., 43., 37.], [ 69., 96., 70.] [ 56., 70.], [ 81., 101.], [119., 133.], [ 22., 37.], [103., 119.] These are the weight that are added Weights and…
ZKS
  • 121
  • 3
0
votes
0 answers

How can a bias in a large language model like GPT-4 depend on the languages used for interaction?

Large language models can have biases that depend on cultural biases present in the information they trained on. Some models, including GPT-4, are trained on input data in multiple languages. Some languages are used by people from many different…
Volker Siegel
  • 589
  • 1
  • 4
  • 17
0
votes
1 answer

Is there any evidence that the bias terms thelp in the attention mechanism of the transformers?

In the original transformer paper, the attention mechanism uses parameter matrices, but no bias terms. However, in more recent implementations I see people often using a bias term when computing "key", "query", and "value". For example, in Andrej…
Tarvoc
  • 1
0
votes
0 answers

Bias in predictions made by Mixture Density Network

I have a very simple Mixture Density Network architecture with 3 input neurons, 10 neurons in the (only) hidden layer and then just one output (which is 2 neurons, one for the mean and one for the std of the prediction). I use ELU as the activation…
JohnDoe122
  • 149
  • 2
0
votes
1 answer

Confusion about bias in McCulloch-Pitts neurons

I just have a quick question, maybe I am too nit picky here. We recently had an introductory lecture to AI in university and the professor talked about McCulloch-Pitts neurons, e.g. activation as soon as the sum of inputs is reaching a certain…
DerOeko
  • 13
  • 3
0
votes
1 answer

Is there a way to freeze training for weights, but not biases in PyTorch?

I'm constructing a neural network where the weights of my first hidden layer (connected to the input) are all 1 (identity matrix), but the biases are variable. Is there a way to "freeze" any updates/training to the weights in a specific layer, but…
Nova
  • 133
  • 4
0
votes
1 answer

Bias equal to 1 and neuron output equal to -1 in neural networks

I have read that bias in neural networks is used to avoid situation in which output of neuron is equal to 0. But what if the same output is equal to -1 and we add 1 to it? Isn't it the same issue as in case of zero output and no bias?
bridgemnc
  • 25
  • 3
0
votes
1 answer

Can we change bias and control the output of neural network?

I have read the use of Targeted Adversarial Attacks for making the model perform better. But can we change the bias of the neural networks and control the outcome of the network rather than changing the input. if yes, can you share some resources or…
harsha
  • 101
0
votes
0 answers

How is the bias added after the convolution in a CNN?

I'm having trouble understanding how bias is added to the feature extraction convolution. I've seen people either refer to the bias as a single number that changes per filter or the whole matrix that is the size of the output. Here is what I…