Backup Failure in Velero with Kopia Due to Suspected Repository Initialization Issue

[erfanmazraei]

What steps did you take and what happened:

I created a backup using Velero and Kopia, but in one of my clusters, I encountered the following error:

message: /pod volume backup failed: error to initialize data path: error to boost backup repository connection <KOPIA_REPO>: error to connect backup repo: error to connect to storage: error retrieving storage config from bucket "XXX": Get "XXXX.com": dial tcp X.X.X.X:443: connect: no route to host

It seems the Kopia repository wasn't initialized successfully:
➜ ~ kopia manifest list
56112d8e2005a4e113d0044b69408b43 239 2024-10-04 07:05:42 UTC type:maintenance
I don't see the usual policy type in each Kopia.

I don't think the issue is related to connectivity because I used the same configuration across all of my clusters, and it worked fine on the others. Additionally, I created a pod on the same worker node and successfully connected to S3 using s3cmd. I also added a container to the Velero pod, and it was able to connect without any issue.

What did you expect to happen:
I expected the backup to complete without any issues.

Environment:

• Velero version (use velero version): v1.14.0
• Kubernetes version (use kubectl version): v1.28.8+k3s1

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "I would like to see this bug fixed as soon as possible"
• 👎 for "There are more important bugs to focus on right now"

[Lyndon-Li]

no route to host indeed looks like a network connection problem.

Is it an intermittent problem or a consistent problem in the cluster?