Vision Language Early fusion

[nero1342]

I saw that you did not mention anything about the early-fusion module that you used in your paper. However, in your code, that module is utilized before the Transformer module. I think this simple module contributes a lot to the result. Can you explain about this?

Thank you!

ReferFormer/models/referformer.py

Line 141 in 9c8f237

self.fusion_module = VisionLanguageFusionModule(d_model=hidden_dim, nhead=8)

ReferFormer/models/referformer.py

Line 243 in 9c8f237

src_proj_l = self.fusion_module(tgt=src_proj_l,

[basavaraj-hampiholi]

@nero1342 I hope this is helpful for you.
#27