Package on conda-forge/osx-64 missing

[wolfv]

This package is apparently missing from anaconda.org repositories, but it still appears in repodata_from_packages.json: https://conda.anaconda.org/conda-forge/osx-64/arpack-3.6.1-blas_openblash1f444ea_0.tar.bz2

It also appears twice on the website (https://anaconda.org/conda-forge/arpack/files?page=2):

[jakirkham]

cc @beckermr (as I believe we had discussed this issue and you wanted a more recent example)

[jakirkham]

AIUI the cause is there are two jobs that have a race condition and both try to upload the same package. If they both do so roughly at the same time, Anaconda.org will accept both packages (even though they are duplicates of one another). Have seen this issue outside of conda-forge.

The net result is one can't download either package. So attempting to install the package will fail.

The only solution I've found to work is to delete the duplicate packages. Marking broken or otherwise handling the issue doesn't work. In particular I normally delete the ones that have 0 download count.

That all being said, maybe we give some time for others to see this issue so they can dig into the context for how this happened in this instance

[jakirkham]

cc @jezdez

[dbast]

This is afaik still like @jakirkham described it... so the duplicated package with download number 0 has to be removed from anaconda.org. This then either resolves with the next sync or also the sync instance requires some intervention.

This is quite an old package... this should not happen for new uploads anymore.

Also pinging @barabo

[barabo]

I believe the uploaded package with 0 downloads was probably a corrupt upload from anaconda.org to our internal bucket. We have recently (Oct? 2022) added sha256 hash verification for uploads, so this shouldn't happen in the future.

[beckermr]

The explanation from @jakirkham is not right. The code we have that does have a race condition is from cfep-13's implementation. That happened about 2 years ago. This package is older than that.

[jakirkham]

Think we need to disambiguate things. There are two issues:

1. Whether conda-forge uploads run into a race condition
2. How Anaconda.org handles rapid, duplicate uploads

My explanation above is focused on 2 (not 1). AFAIK 2 is not resolved.

[beckermr]

Right. Agreed.

[wolfv]

I found a couple more with this symptomatic:

Retrying in 5 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/sundials-3.1.0-blas_openblash0edd121_202.tar.bz2
Successfully pushed ghcr.io/channel-mirrors/conda-forge/osx-64/aws-parallelcluster:3.0.0-py36h79c6626_0
Retrying in 3 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/vlfeat-0.9.20-h470a237_2.tar.bz2
Retrying in 29 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/arpack-3.6.1-blas_openblash1f444ea_0.tar.bz2
Retrying in 29 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/lammps-2018.03.16-.tar.bz2
Retrying in 5 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/vlfeat-0.9.20-h470a237_2.tar.bz2
Retrying in 11 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/sundials-3.1.0-blas_openblash0edd121_202.tar.bz2
Retrying in 11 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/vlfeat-0.9.20-h470a237_2.tar.bz2
Retrying in 29 seconds - error: 404 Client Error: Not 
Found for url: https://conda.anaconda.org/conda-forge/osx-64/sundials-3.1.0-blas_openblash0edd121_202.tar.bz2
Retrying in 29 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/vlfeat-0.9.20-h470a237_2.tar.bz2
Retrying in 83 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/arpack-3.6.1-blas_openblash1f444ea_0.tar.bz2
Retrying in 83 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/lammps-2018.03.16-.tar.bz2
Retrying in 83 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/sundials-3.1.0-blas_openblash0edd121_202.tar.bz2
Retrying in 83 seconds - error: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/osx-64/vlfeat-0.9.20-h470a237_2.tar.bz2

For some reason it seems to be mainly OSX-64 that is conerned by this.

Not sure what we should do. Maybe Anaconda can do some sort of sync, or we can remove them from the frontend?

[jezdez]

Oh, interesting, the plot thickens! 😆 Did that show up during the channel mirroring? Is there any easy way for me to try to reproduce this locally?

[wolfv]

You can click on the links 🙂

[wolfv]

Or you can check these logs: https://github.com/channel-mirrors/mirrormirror/actions/runs/4194077479/jobs/7272488261