About freeze batch norm

[fyhfly]

Hi, sorry for bothering you.
I've been wondering for a long time why did you freeze bn when training GASDA using the pretrained F_s, F_t and CycleGAN.
If frozen batch norm, what parameters will be optimized in training?

[sshan-zhao]

Hi, sorry for bothering you.
I've been wondering for a long time why did you freeze bn when training GASDA using the pretrained F_s, F_t and CycleGAN.
If frozen batch norm, what parameters will be optimized in training?

1. Firstly we can reduce the required memory. Secondly, we use small batchsize when training GASDA. Thirdly, when training GASDA, all parts have been pre-trained, so the BN layer can be fixed.
2. Parameters in convolutional layers.

[fyhfly]

Hi, sorry for bothering you.
I've been wondering for a long time why did you freeze bn when training GASDA using the pretrained F_s, F_t and CycleGAN.
If frozen batch norm, what parameters will be optimized in training?

1. Firstly we can reduce the required memory. Secondly, we use small batchsize when training GASDA. Thirdly, when training GASDA, all parts have been pre-trained, so the BN layer can be fixed.
2. Parameters in convolutional layers.

Parameters in convolutional layers are weights and biases. If frozen bn, in my view, weights and biases will not change any more, and performance of net will not improve.
When running gasda_model.py, 2 task nets(depth generate), 2 trans nets(cycle), 4 discriminators are trained. You freeze bn in 2 task nets, does that mean these 2 nets won't be optimized?
That's what I don't understand.

[sshan-zhao]

Parameters in convolutional layers are weights and biases. If frozen bn, in my view, weights and biases will not change any more——why? The parameters in bn are fixed, but the parameters in conv could be changed. 2021年2月4日 +1100 AM2:39 sshan-zhao/GASDA <reply@reply.github.com>，写道：

…

Parameters in convolutional layers are weights and biases. If frozen bn, in my view, weights and biases will not change any more,