查询是否有不参与梯度下降的参数,当设置ddp为find_unused_parameters=True,

PHOTO EMBED

Mon Aug 29 2022 10:57:26 GMT+0000 (Coordinated Universal Time)

Saved by @Frank_xu #python

[name for name,para in model.named_parameters() if para.grad==None]
TORCH_DISTRIBUTED_DEBUG=DETAIL bash train.sh 
content_copyCOPY

第二条命令可以看到哪些参数没有用上