Is there a way to freeze training for weights, but not biases in PyTorch?

Question

I'm constructing a neural network where the weights of my first hidden layer (connected to the input) are all 1 (identity matrix), but the biases are variable.

Is there a way to "freeze" any updates/training to the weights in a specific layer, but continue to allow the biases in that specific layer to be updated?

Maybe this will help: https://discuss.pytorch.org/t/how-the-pytorch-freeze-network-in-some-layers-only-the-rest-of-the-training/7088 — GKozinski, Aug 19 '22 at 20:08

score 2 · Accepted Answer · answered Aug 19 '22 at 21:49

2

You can do something link this

model.linear1.weight.requires_grad = False
model.linear1.bias.requires_grad = False

for the pytorch model with linear1 defined as:

self.linear1 = nn.Linear(5, 5)

as in this code snippet

answered Aug 19 '22 at 21:49

Robert

51
5

Is there a way to freeze training for weights, but not biases in PyTorch?

1 Answers1