cargo does not see Cargo.toml if .gitignore contains /*

[cospectrum]

Problem

If .gitignore is written in the style of "ignore everything except this", then cargo package --list for some reason does not see anything.
It seems to me that everything was fine on previous versions, I spent an hour investigating, did I miss something?

cargo version: 1.80.1 (376290515 2024-07-16)
git version 2.39.2 (Apple Git-143)
OS: mac

Cargo.toml:

[package]
name = "gitignore-bug"
version = "0.1.0"
edition = "2021"
license = "MIT"
description = "to reproduce bug"

[dependencies]

.gitignore:

/target
/*
!.gitignore
!Cargo.toml
!src

cargo package --list

warning: manifest has no documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
warning: no `Cargo.toml` file found when packaging `gitignore-bug` (note the case of the file name).
.cargo_vcs_info.json
Cargo.lock

Steps

cargo init
echo '\n/*\n\n!.gitignore\n!Cargo.toml\n!src' >> .gitignore
git add . && git commit -m "."
cargo package --list

Possible Solution(s)

No response

Notes

cargo build is working, cargo publish is not working

Version

cargo 1.80.1 (376290515 2024-07-16)
release: 1.80.1
commit-hash: 37629051518c3df9ac2c1744589362a02ecafa99
commit-date: 2024-07-16
host: aarch64-apple-darwin
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.7.1 (sys:0.4.72+curl-8.6.0 system ssl:(SecureTransport) LibreSSL/3.3.6)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Mac OS 13.6.8 [64-bit]

[weihanglo]

Thanks for the report, though I cannot reproduce it.

When you run git ls-files on that repository, what is the output?

Mine is

$ git ls-files
.gitignore
Cargo.toml
src/main.rs

cargo package --list queries git index to get the file list. If they are in the git index, they shall be packaged.

[weihanglo]

Sorry. Clicked the wrong button 🙇🏾‍♂️.

[cospectrum]

git ls-files gives the same output as yours

[weihanglo]

I did the correct list of files on macOS.

$ cargo +1.80.1 package --list
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo_vcs_info.json
.gitignore
Cargo.lock
Cargo.toml
Cargo.toml.orig
src/main.rs

Do you have any special Git configuration on your machine?
(~/.gitconfig or git/config under the repo)

It seems to me that everything was fine on previous versions, I spent an hour investigating, did I miss something?

Is it possible on your side to bisect which version starts failing? Note that Cargo switched to gitoxide for file listing in 1.79.0. You could try __CARGO_GITOXIDE_DISABLE_LIST_FILES=1 cargo package --list and see if it works.

[cospectrum]

__CARGO_GITOXIDE_DISABLE_LIST_FILES=1 cargo package --list is working correctly

[cospectrum]

toolchain version 1.78.0 is working, 1.79.0 is not

[weihanglo]

Do you have any special Git configuration on your machine?
(~/.gitconfig or git/config under the repo)

@cospectrum it might help if you can provide some information about this.

cc @Byron

[cospectrum]

I only have a short global ~/.gitconfig, but I think it had no effect, tried with it and without it, no effect.

[Byron]

I was also unable to reproduce the issue with v1.80.1 and v1.82.0.

My guess is that this gitoxide commit is the fix that was first introduced in gix v0.64 - maybe older cargo releases with prior versions of gix suffer from the issue, i.e. directory traversal would allow to skip the traversal root when everything was excluded, with only includes to go by.

I don't even have a question for @cospectrum as it's clear by now that we can't reproduce the issue, which probably means that some global git configuration is affecting this unexpectedly. It's actually kind of impossible right now to disable this effect without recompiling, unfortunately.

cargo/testing ( master) +5 -1 [$!⇡1] took 6s
❯ cargo package --list
warning: manifest has no description.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo_vcs_info.json
.gitignore
Cargo.lock
Cargo.toml
Cargo.toml.orig
src/main.rs

cargo/testing ( master) +5 -1 [$!⇡1]
❯ ../target/debug/cargo package --list
warning: manifest has no description.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo_vcs_info.json
.gitignore
Cargo.lock
Cargo.toml
Cargo.toml.orig
src/main.rs

cargo/testing ( master) +5 -1 [$!⇡1]
❯ cargo --version
cargo 1.80.1 (376290515 2024-07-16)

cargo/testing ( master) +5 -1 [$!⇡1]
❯ ../target/debug/cargo --version
cargo 1.82.0 (9a170d718 2024-08-15)

[flearc]

Follow the steps above and have the same problem

~/git-cargo  $ cargo init
    Creating binary (application) package
note: see more `Cargo.toml` keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
~/git-cargo  ‹master*› $ echo '\n/*\n\n!.gitignore\n!Cargo.toml\n!src' >> .gitignore
~/git-cargo  ‹master*› $ git add . && git commit -m "."
[master (root-commit) 7c9134f] .
 3 files changed, 16 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 Cargo.toml
 create mode 100644 src/main.rs
~/git-cargo  ‹master› $ cargo package --list
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
warning: no `Cargo.toml` file found when packaging `git-cargo` (note the case of the file name).
.cargo_vcs_info.json
Cargo.lock
~/git-cargo  ‹master› $ __CARGO_GITOXIDE_DISABLE_LIST_FILES=1 cargo package --list
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo_vcs_info.json
.gitignore
Cargo.lock
Cargo.toml
Cargo.toml.orig
src/main.rs
~/git-cargo  ‹master› $ cargo -V
cargo 1.80.1 (376290515 2024-07-16)
~/git-cargo  ‹master› $ git -v
git version 2.39.3 (Apple Git-146)

git config -l :

user.name=flea
user.email=myemail
core.bare=false
core.repositoryformatversion=0
core.filemode=true
core.logallrefupdates=true

[Byron]

That's great! Actually, I noticed that previously cargo init didn't initialize a repository in the newly created cargo project as I invoked it from within the Cargo repository.

Once that is fixed, I can reproduce the issue.

❯ cargo package -l
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
warning: no `Cargo.toml` file found when packaging `testing` (note the case of the file name).
.cargo_vcs_info.json
Cargo.lock

On it now.

[Byron]

Apologies for getting back to this with such a delay, but there is good news: It looks like the latest master already has a fix.

In the example output below, ../testing was initialized with the reproducer from this issue.

❯ git rev-parse @
88957a1b7f65b233118dcd4f59df2763043ef9dd

cargo ( master) [$] via 🐍
❯ cargo run -- package --manifest-path ../testing/Cargo.toml  -l
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/cargo package --manifest-path ../testing/Cargo.toml -l`
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo_vcs_info.json
.gitignore
Cargo.lock
Cargo.toml
Cargo.toml.orig
src/main.rs

The fix is part of gix v0.64, which comes with gix-dir v0.6.0 that is very likely the crate the contains the actual fix.

As a lower boundary, when compiling an older version (88957a1b7f65b233118dcd4f59df2763043ef9dd ~100), the issue can be reproduced in the test setup used here:

cargo (6d68563) [$] via 🐍
❯ cargo run -- package --manifest-path ../testing/Cargo.toml  -l
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.28s
     Running `target/debug/cargo package --manifest-path ../testing/Cargo.toml -l`
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
warning: no `Cargo.toml` file found when packaging `testing` (note the case of the file name).
.cargo_vcs_info.json
Cargo.lock

Thus, no action is needed unless a backport is desired. Then I'd have to try and port the fix to gix-dir v0.5.x and Cargo would need another patch release after updating to it.

[epage]

So that would mean #14332 is what resolved this which is scheduled for 1.82

For 1.80, we have the workaround of __CARGO_GITOXIDE_DISABLE_LIST_FILES=1. That was removed in 1.81 which means there is no working solution in that release under the same reproduction steps.

@Byron what would you say the risk profile is for the upgrade from gix 0.63 to 0.64? This can help us in determining whether we should cherrypick the upgrade into 1.81. Overall, I'm leaning towards "no" regardless because we have about 2 weeks left before release which is not much time left for testing, so I would be very hesitant to do the cherry pick

[Byron]

Thanks for the analysis!

gix 0.64 contains only bugfixes and probably also fixes for security advisories (via plumbing crates), so I'd think the risk of cherry-picking it is very low.
Maybe this means that 1.81 can be fixed before release.

[EliahKagan]

In support of #14411 (comment) and #14411 (comment), I believe this was the same underlying issue as in Byron/gitoxide#1458, so the fix for that in Byron/gitoxide@1e92d1e (Byron/gitoxide#1462) applies to this too.

[weihanglo]

#14431 has backported gix bump to beta-1.81, so when 1.81 is stable there should be no issue of this. Close as resolved and thanks!