Stop infinite retries of upload and clean up snapshot #532

lipingxue · 2023-05-18T17:22:01Z

What this PR does / why we need it:
Currently, we Infinite retry of upload in the Data Manager. When upload keeps failing , the local snapshot was not deleted. We need to delete snapshots if upload keeps failing.
Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

With this change, users can configure the "upload-cr-retry-max" in  configmap "velero-vsphere-plugin-config".

If this parameter is not specified, the default value 10 will be used.  Upload will retry until it reaches the 
maximum retry. 

After that, we will try to delete the local snapshot and mark the upload cr as "UploadFailedAfterRetry" and 
stop further retry.

lipingxue · 2023-05-18T17:29:52Z

Testing is done. See this
pr-test-result.txt

svcbot-qecnsdp · 2023-05-18T17:31:01Z

Sorry, some of the pre-check tests are failing for this PR !

Please check the Jenkins job for more details
https://container-dp.svc.eng.vmware.com/job/CNSDP-CI/805/

svcbot-qecnsdp · 2023-05-18T20:59:04Z

Sorry, some of the pre-check tests are failing for this PR !

Please check the Jenkins job for more details
https://container-dp.svc.eng.vmware.com/job/CNSDP-CI/806/

xing-yang

Please also update the README accordingly.

pkg/apis/backupdriver/v1alpha1/snapshot.go

pkg/apis/datamover/v1alpha1/upload.go

pkg/constants/constants.go

pkg/controller/upload_controller.go

svcbot-qecnsdp · 2023-05-19T21:59:11Z

Sorry, some of the pre-check tests are failing for this PR !

Please check the Jenkins job for more details
https://container-dp.svc.eng.vmware.com/job/CNSDP-CI/808/

deepakkinni

Can you add some unit tests as well.

pkg/apis/backupdriver/v1alpha1/snapshot.go

pkg/constants/constants.go

pkg/utils/utils.go

pkg/controller/upload_controller.go

svcbot-qecnsdp · 2023-05-23T23:58:19Z

Sorry, some of the pre-check tests are failing for this PR !

Please check the Jenkins job for more details
https://container-dp.svc.eng.vmware.com/job/CNSDP-CI/811/

lipingxue · 2023-05-24T00:05:23Z

Can you add some unit tests as well.

Yes. I added two unit tests. This is the unit test result.
unit-test-May-23.txt

pkg/constants/constants.go

svcbot-qecnsdp · 2023-05-30T17:56:23Z

Sorry, some of the pre-check tests are failing for this PR !

Please check the Jenkins job for more details
https://container-dp.svc.eng.vmware.com/job/CNSDP-CI/817/

xing-yang · 2023-05-30T18:44:00Z

LGTM

Signed-off-by: Liping Xue <lipingx@vmware.com>

lipingxue · 2023-05-30T20:57:03Z

@xing-yang I have squashed the commits, please help to merge it. Thanks!

svcbot-qecnsdp · 2023-05-30T21:06:34Z

Sorry, some of the pre-check tests are failing for this PR !

Please check the Jenkins job for more details
https://container-dp.svc.eng.vmware.com/job/CNSDP-CI/818/

lipingxue · 2023-06-01T20:39:04Z

@xing-yang I have squashed the commits, please help to merge it. Thanks!

@xing-yang Could you help to merge this MR? Thanks!

Signed-off-by: Liping Xue <lipingx@vmware.com>

* Update script. Signed-off-by: Liping Xue <lipingx@vmware.com> Signed-off-by: Liping Xue <lipingx@vmware.com> * Add warning message in compatibility table. (#534) * Add warning message in compatibility table. Signed-off-by: Liping Xue <lipingx@vmware.com> * Address comment from Xing. Signed-off-by: Liping Xue <lipingx@vmware.com> * Update link to vSphere documentation in supervisor-datamgr.md file (#536) * Update vSphere doc link in supervisor-datamgr.md. Signed-off-by: Liping Xue <lipingx@vmware.com> * Address comment from Xing. Signed-off-by: Liping Xue <lipingx@vmware.com> * Update link to data manager ova. (#537) Signed-off-by: Liping Xue <lipingx@vmware.com> * Fix upload retry. (#532) Signed-off-by: Liping Xue <lipingx@vmware.com> * Update support matrix for Vanilla, WCP and GC. (#538) Signed-off-by: Liping Xue <lipingx@vmware.com> * Update support matrix for WCP. Signed-off-by: Liping Xue lipingx@vmware.com Signed-off-by: Liping Xue <lipingx@vmware.com> * Update Snapshot and upload CRD status in doc. (#545) Signed-off-by: Liping Xue lipingx@vmware.com Signed-off-by: Liping Xue <lipingx@vmware.com> * Add document on Deploy Velero in Supervisor Cluster with Images from Private Registry. (#543) Signed-off-by: Liping Xue lipingx@vmware.com Signed-off-by: Liping Xue lipingx@vmware.com Signed-off-by: Liping Xue <lipingx@vmware.com> * Update support matrix for Vanilla. (#548) Signed-off-by: Liping Xue <lipingx@vmware.com> * Bump the golang.org/x/net version to v0.17.0 to address CVEs. (#552) Signed-off-by: Xun Jiang <jxun@vmware.com> Signed-off-by: Liping Xue <lipingx@vmware.com> --------- Signed-off-by: Liping Xue <lipingx@vmware.com> Signed-off-by: Liping Xue lipingx@vmware.com Signed-off-by: Xun Jiang <jxun@vmware.com> Co-authored-by: Xun Jiang/Bruce Jiang <59276555+blackpiglet@users.noreply.github.com>

lipingxue force-pushed the lipingx-upload-cr-retry branch 2 times, most recently from d5aa083 to 4e90441 Compare May 18, 2023 20:51

xing-yang reviewed May 19, 2023

View reviewed changes

deepakkinni reviewed May 20, 2023

View reviewed changes

xing-yang reviewed May 26, 2023

View reviewed changes

pkg/constants/constants.go Outdated Show resolved Hide resolved

deepakkinni approved these changes May 30, 2023

View reviewed changes

Fix upload retry.

1926986

Signed-off-by: Liping Xue <lipingx@vmware.com>

lipingxue force-pushed the lipingx-upload-cr-retry branch from ed9200b to 1926986 Compare May 30, 2023 20:55

xing-yang merged commit 4f710f3 into vmware-tanzu:main Jun 2, 2023

lipingxue added a commit to lipingxue/velero-plugin-for-vsphere that referenced this pull request Jun 27, 2023

Fix upload retry. (vmware-tanzu#532)

81d1ac2

Signed-off-by: Liping Xue <lipingx@vmware.com>

lipingxue mentioned this pull request Jul 24, 2023

Update Snapshot and upload CR status in doc. #545

Merged

lipingxue added a commit to lipingxue/velero-plugin-for-vsphere that referenced this pull request Oct 26, 2023

Fix upload retry. (vmware-tanzu#532)

268df55

Signed-off-by: Liping Xue <lipingx@vmware.com>

lipingxue mentioned this pull request Oct 26, 2023

Chery pick changes to release-1.5 #555

Merged

lipingxue added a commit to lipingxue/velero-plugin-for-vsphere that referenced this pull request Oct 27, 2023

Fix upload retry. (vmware-tanzu#532)

e180e78

Signed-off-by: Liping Xue <lipingx@vmware.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stop infinite retries of upload and clean up snapshot #532

Stop infinite retries of upload and clean up snapshot #532

lipingxue commented May 18, 2023 •

edited by xing-yang

Loading

lipingxue commented May 18, 2023

svcbot-qecnsdp commented May 18, 2023

svcbot-qecnsdp commented May 18, 2023

xing-yang left a comment

svcbot-qecnsdp commented May 19, 2023

deepakkinni left a comment

svcbot-qecnsdp commented May 23, 2023

lipingxue commented May 24, 2023

svcbot-qecnsdp commented May 30, 2023

xing-yang commented May 30, 2023

lipingxue commented May 30, 2023

svcbot-qecnsdp commented May 30, 2023

lipingxue commented Jun 1, 2023

Stop infinite retries of upload and clean up snapshot #532

Stop infinite retries of upload and clean up snapshot #532

Conversation

lipingxue commented May 18, 2023 • edited by xing-yang Loading

lipingxue commented May 18, 2023

svcbot-qecnsdp commented May 18, 2023

svcbot-qecnsdp commented May 18, 2023

xing-yang left a comment

Choose a reason for hiding this comment

svcbot-qecnsdp commented May 19, 2023

deepakkinni left a comment

Choose a reason for hiding this comment

svcbot-qecnsdp commented May 23, 2023

lipingxue commented May 24, 2023

svcbot-qecnsdp commented May 30, 2023

xing-yang commented May 30, 2023

lipingxue commented May 30, 2023

svcbot-qecnsdp commented May 30, 2023

lipingxue commented Jun 1, 2023

lipingxue commented May 18, 2023 •

edited by xing-yang

Loading