VisionLanguageFusionModule

[lzl2040]

I find in this module you use multiplication instead of addition or concatenation.

tgt2 = self.multihead_attn(query=self.with_pos_embed(tgt, query_pos),
                                   key=self.with_pos_embed(memory, pos),
                                   value=memory, attn_mask=None,
                                   key_padding_mask=memory_key_padding_mask)[0]
        tgt = tgt * tgt2
        return tgt

why not use addition or concatenation? What are the benefits of using multiplication？

[wjn922]

In our early experiments, we find that multiplication would have higher performance than addition for RVOS. So we consistently use multiplication all along the work.

But actually, the performance advantage is less than 1 point. So using multiplication, addition or concatenation may not significantly impact the results.