Question about iterative object box prediction

[XuyangBai]

Hi, thanks for your sharing. I find for each decoder layer, you use the cluster_xyz as the initial location instead of the updated base_xyz

Group-Free-3D/models/detector.py

Lines 221 to 227 in ef8b7bb

	base_xyz, base_size = self.prediction_heads[i](query,
	base_xyz=cluster_xyz,
	end_points=end_points,
	prefix=prefix)
	base_xyz = base_xyz.detach().clone()
	base_size = base_size.detach().clone()

My question is since each layer uses the box location of the previous layer to produce the spatial encoding, why does each layer predict the offset to the gt box location relative to the initial cluster center instead of the updated center of the previous layer? In another word, why not

base_xyz, base_size = self.prediction_heads[i](query,
                           base_xyz=base_xyz,                                               
                           end_points=end_points, 
                           prefix=prefix)

And under your setting, I think the "auxiliary loss" is necessary? The reason is that if no auxiliary loss is applied, the prediction head of the first N-1 decoder layers will not get supervision for the center_residual, the updated box prediction and spatial encoding for the next decoder layer will be meanless. Am I correct?

Best,
Xuyang