Robust Deep Object Tracking against Adversarial Attacks

🌿 Robust Deep Object Tracking against Adversarial Attacks

International Journal of Computer Vision (IJCV)

Shuai Jia, Chao Ma, Yibing Song, Xiaokang Yang and Ming-Hsuan Yang

This work is based on

• Robust Tracking against Adversarial Attacks in ECCV2020 [Project]
• IoU Attack: Towards Temporally Coherent Black-Box Adversarial Attack for Visual Object Tracking in CVPR2021 [Project]

Introduction

Deep neural networks (DNNs) are vulnerable to adversarial attacks.

• We study the robustness of the state-of-the-art deep trackers against adversarial attacks under both white-box and black-box settings.
• We propose a defense method to subtract perturbations from the input frame, which eliminates performance drops caused by adversarial attacks.
• We craft universal adversarial perturbations to directly inject them into every frame of video sequences, leading to a higher attack speed.
• We choose five representative trackers, SiamRPN++, SiamCAR, RT-MDNet, DiMP and TransT.

Demo

🌿 More demos are available at [Video] .

Prerequisites

The environment follows the tracker you intend to attack：

• The specific setting and pretrained model for SiamPRN++ can refer to [Code_SiamRPN++].
• The specific setting and pretrained model for SiamCAR can refer to [Code_SiamCAR].
• The specific setting and pretrained model for RT-MDNet can refer to [Code_RT-MDNet].
• The specific setting and pretrained model for DiMP can refer to [Code_DiMP].
• The specific setting and pretrained model for TransT can refer to [Code_TransT].

Experiments

• Results on the OTB2015 dataset

• Results on the UAV123 dataset

• Results on the LaSOT dataset

🌿 All raw results are available. [Google_drive]

Quick Start

🌿 The code of adversarial attack and defense on SiamRPN++ is released.

• Please follow SiamRPN++ to finish the experimental setting, including dataset, model, environment, etc.
• First, put test_attack.py and test_defense.py into tools folder.
• Second, replace the original siamrpn_tracker.py in pysot/tracker with our new siamrpn_tracker.py.
• Note that the new siamrpn_tracker.py in this project consists of all original codes of SiamRPN++ and our new attack and defense code.

Test the original performance on OTB100 dataset, please using the follwing command.

cd experiments/siamrpn_r50_l234_dwxcorr_otb
python -u ../../tools/test.py 	\
	--snapshot model.pth 	\ # model path
	--dataset OTB100 	\ # dataset name
	--config config.yaml	  # config file

Test the adversarial attack performance on OTB100 dataset, please using the follwing command.

cd experiments/siamrpn_r50_l234_dwxcorr_otb
python -u ../../tools/test_attack.py 	\
	--snapshot model.pth     	\ # model path
	--dataset OTB100         	\ # dataset name
	--config config.yaml	          # config file

Test the adversarial defense performance on OTB100 dataset, please using the follwing command.

cd experiments/siamrpn_r50_l234_dwxcorr_otb
python -u ../../tools/test_defense.py 	\
	--snapshot model.pth     	\ # model path
	--dataset OTB100         	\ # dataset name
	--config config.yaml	          # config file

• The original/attack/defense results will be saved in the current directory(results/dataset/model_name/).
• --vis can be used to visualize the tracking results during attack and defense.

Acknowledgement

We sincerely thanks the authors of SiamRPN++, SiamCAR, RT-MDNet, DiMP and TransT, who provide the baseline trackers for our attack and defense.

License

Licensed under an MIT license.