generate white images during training

[NikLi66]

Hi, I found the all of the saved reconstructed images are white images under the directory ./logs/ during training(shown below). Is there anyone can tell me the potential reason?

[ashwani-ver]

Hi, I found the all of the saved reconstructed images are white images under the directory ./logs/ during training(shown below). Is there anyone can tell me the potential reason?

did you face any issue in training like below?

[rank0]: Parameter indices which did not receive grad for rank 0: 338 339 340 341 342 343 344 345 346 347 348 349 350
[rank0]: In addition, you can set the environment variable TORCH_DISTRIBUTED_DEBUG to either INFO or DETAIL to print out information about which particular parameters did not receive gradient on this rank as part of this error