Duplicated snapshots and os error 2 when running a workspace + root crate #396

dzfrias · 2023-07-29T19:42:06Z

What happened?

When running insta, I get duplicated snapshot reviews, as well as a crash at the end of the snapshot reviewing. In the original (non-minimal) case, I wasn't able to review some of my tests.

I should have reviewed two snapshots, but only was able to review one. The error was the dreaded "No such file or directory (os error 2)".

Reproduction steps

Create a new directory (not using cargo new --lib)
Create a Cargo.toml
Put this in your Cargo.toml:

[package]
name = "testing2"
version = "0.1.0"

[[test]]
name = "integration"
path = "tests/tests.rs"

[dev-dependencies]
insta = { workspace = true }

[workspace]
members = ["testing"]

[workspace.dependencies]
insta = "1.31"

Run cargo new testing --lib in the project root
Put this in testing/src/lib.rs

#[test]
fn it_works() {
    insta::assert_snapshot!("stuff...");
}

Create the tests/ directory in the project root
Create tests/tests.rs
Put this in tests/tests.rs

#[test]
fn works() {
    insta::assert_snapshot!("hr");
}

Run cargo insta test --review --all

Insta Version

1.31.0

rustc Version

1.71.0

What did you expect?

I expected to review two tests, and have no crash.

The text was updated successfully, but these errors were encountered:

TheLostLambda · 2024-02-22T02:10:16Z

I'm also struggling with this issue... I need to run things without --workspace, one-at-a-time, just to get insta to work!

regexident · 2024-07-15T15:07:26Z

Same here, which makes cargo insta barely usable in a larger workspace.

max-sixty · 2024-07-15T15:53:52Z

I've added some better error messages on missing files, currently only on master.

Could folks try that and see what that reports?

(I'm not familiar with insta not working on larger workspaces; we use it very successfully in PRQL. Is the distinction that nested crates have problems?)

regexident · 2024-07-16T19:54:49Z

[patch.crates-io]
insta = { git = "https://github.com/mitsuhiko/insta.git", branch = "master" }

@max-sixty no difference, still only getting this:

error: No such file or directory (os error 2)

cargo insta test --release --accept --workspace

     Running tests/snapshots.rs (target/release/deps/snapshots-eee0592393c0f811)

running 3 tests
test <SNIP> ... ok
stored new snapshot <SNIP>.snap.new
stored new snapshot <SNIP>.snap.new
test <SNIP> ... ok
test <SNIP> ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 7.28s

     Running unittests src/lib.rs (target/release/deps/snapshots-08c69356ac69b7db)

running 15 tests
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok
test <SNIP> ... ok

test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

...

error: No such file or directory (os error 2)

The workspace consists of a root bin package and 9 member lib crates, two of which contain the snapshot tests.

Running cargo insta test --release --accept on each of the crates individually works just fine.

Curiously running the cargo insta command with --workspace even failed when only one crate had snapshot tests.

max-sixty · 2024-07-16T23:18:07Z

still only getting this:

error: No such file or directory (os error 2)

Would you mind trying cargo install --git https://github.com/mitsuhiko/insta --bin=cargo-insta to ensure it's also running the latest cargo-insta?

(There may be an error source which doesn't offer more info. But I at least more info to some sources, including ones that look like this)

The workspace consists of a root bin package and 9 member lib crates, two of which contain the snapshot tests.

Thanks — I guess this is because of the root / nested package. I'm guessing if turn the root package off then it works OK (to the extent that's possible to test)?

regexident · 2024-07-17T10:23:22Z

still only getting this:

error: No such file or directory (os error 2)

Would you mind trying cargo install --git https://github.com/mitsuhiko/insta --bin=cargo-insta to ensure it's also running the latest cargo-insta?

(There may be an error source which doesn't offer more info. But I at least more info to some sources, including ones that look like this)

Now it does indeed provide a more useful error message:

error: File error for "/tests/snapshots/snapshots__.snap.new": No such file or directory (os error 2)

I deleted the contents of the tests/snapshots directory of one of the crates to trigger the error and looking at the contents of the directory the file above is the only one with .snap, while all other still are .snap.new, in case that's helpful.

The workspace consists of a root bin package and 9 member lib crates, two of which contain the snapshot tests.

Thanks — I guess this is because of the root / nested package. I'm guessing if turn the root package off then it works OK (to the extent that's possible to test)?

Commenting out [package] & Co in the root Cargo.toml did indeed make the error vanish!

max-sixty · 2024-07-17T18:23:05Z

OK nice, that's some progress.

I had a look through the code — I can't see exactly what's going on. I think two main possibilities:

Two tests — one in the root package and one in a nested package — are named the same, and the snapshot has been removed by the time we get to the second
Some variant of insta attempting to collect the tests twice

Could we try renaming the test that references "/tests/snapshots/snapshots__.snap.new" to something really unique and seeing whether the error still triggers?

Also if anyone has a minimally reproducible example — probably requires a link to a repo — that would be v useful...

regexident · 2024-07-18T07:57:20Z

Also if anyone has a minimally reproducible example — probably requires a link to a repo — that would be v useful...

This is all you need: Workspace with root package and single snapshot test in member crate. 💣💥

Create root cargo project:

cargo new "insta-bug-repro"
cd "insta-bug-repro"

Change contents of root Cargo.toml file to this:

# Cargo.toml

[workspace]
members = [
    "crates/member-crate",
]

[workspace.dependencies]
insta = "1.39.0"

[package]
name = "insta-bug-repro"
version = "0.1.0"
edition = "2021"

[dependencies]
insta = { workspace = true }

[patch.crates-io]
insta = { git = "https://github.com/mitsuhiko/insta.git", branch = "master" }

Create member crate cargo project:

mkdir "crates"
cd "crates"
cargo new "member-crate"

Change contents of member crate Cargo.toml file to this:

# crates/member-crate/Cargo.toml

[package]
name = "member-crate"
version = "0.0.0"
edition = "2021"

[dependencies]
insta = { workspace = true }

Add snapshot test to root package (doesn't matter if you add it to main.rs or to tests/test.rs)

// src/main.rs

fn main() {
    println!("Hello, world!");
}

#[cfg(test)]
mod tests {
    #[test]
    fn another_test() {
        insta::assert_debug_snapshot!(vec![1, 2, 3]);
    }
}

Run tests via cargo insta

cargo insta test --workspace --accept

   Compiling member-crate v0.0.0 (<SNIP>/insta-bug-repro/crates/member-crate)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.40s
     Running unittests src/main.rs (<SNIP>/insta-bug-repro/target/debug/deps/insta_bug_repro-74dd16e027693651)

running 1 test
stored new snapshot <SNIP>/insta-bug-repro/src/snapshots/insta_bug_repro__tests__test2.snap.new
test tests::test2 ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.13s

     Running unittests src/lib.rs (<SNIP>/insta-bug-repro/target/debug/deps/member_crate-2eb078fdc829f709)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests member_crate

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

insta review finished
accepted:
  src/main.rs (test2.snap)

Add snapshot test to member crate package (doesn't matter if you add it to lib.rs or to tests/test.rs)

// crates/member-crate/src/lib.rs

#[cfg(test)]
mod tests {
    #[test]
    fn test() {
        insta::assert_debug_snapshot!(vec![1, 2, 3]);
    }
}

Run tests via cargo insta

cargo insta test --workspace --accept

    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.03s
     Running unittests src/main.rs (<SNIP>/insta-bug-repro/target/debug/deps/insta_bug_repro-74dd16e027693651)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (<SNIP>/insta-bug-repro/target/debug/deps/member_crate-2eb078fdc829f709)

running 1 test
stored new snapshot <SNIP>/insta-bug-repro/crates/member-crate/src/snapshots/member_crate__tests__test.snap.new
test tests::test ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.13s

     Running tests/insta.rs (<SNIP>/insta-bug-repro/target/debug/deps/insta-e229e15f5cea934d)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests member_crate

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

error: File error for "<SNIP>/insta-bug-repro/crates/member-crate/src/snapshots/member_crate__tests__test.snap.new": No such file or directory (os error 2)

Disable root package:

[workspace]
members = [
    "crates/member-crate",
]

[workspace.dependencies]
insta = "1.39.0"

# [package]
# name = "insta-bug-repro"
# version = "0.1.0"
# edition = "2021"

# [dependencies]
# insta = { workspace = true }

[patch.crates-io]
insta = { git = "https://github.com/mitsuhiko/insta.git", branch = "master" }

Run tests via cargo insta

cargo insta test --workspace --accept

```plain
Compiling member-crate v0.0.0 (<SNIP>/insta-bug-repro/crates/member-crate)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.65s
    Running unittests src/lib.rs (<SNIP>/insta-bug-repro/target/debug/deps/member_crate-2eb078fdc829f709)

running 1 test
stored new snapshot <SNIP>/insta-bug-repro/crates/member-crate/src/snapshots/member_crate__tests__test.snap.new
test tests::test ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s

Doc-tests member_crate

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

insta review finished
accepted:
crates/member-crate/src/lib.rs (test.snap)
```

As evident from the stdout/stderr output above the issue doesn't seem to be related to name collisions among workspace crates, since even adding a single test to a single member crate triggers the error. The error however only gets triggered if the workspace also has a root package (that doesn't need to actively be using insta though).

max-sixty · 2024-07-18T17:00:20Z

@regexident thanks for the full example!

I obv fully believe it happens. Weirdly I can't repro locally — I'm guessing it's something about the order in which the crates run. I haven't tried your exact example — would you be able to push it as a git repo? Otherwise I can copy & paste the files.

regexident · 2024-07-19T06:50:53Z

Here is a repository: insta-bug-repro.zip

I've added a commit (with explanatory messages) for each of the steps described above and added ✅/❌ for the expected outcome of running cargo insta test --workspace --accept on the given commit's checkout.

Make sure to delete the snapshot directory in between runs.

regexident · 2024-07-19T06:54:04Z

I'm running the following in case it helps:

macOS Sonoma 14.6 Beta (23G5075b)

cargo 1.79.0 (ffa9cf99a 2024-06-03)
rustc 1.79.0 (129f3b996 2024-06-10)

cargo-insta 1.39.0

max-sixty · 2024-07-19T07:25:31Z

Thank you! I can repro with cargo insta test --workspace --accept; git clean -fd on 7fd61fb nicely. I'll have a look in the next day or two.

(+ ref #388, I think likely related)

Fixes mitsuhiko#396

max-sixty · 2024-07-21T20:31:57Z

Great @regexident , if you get a moment, would you mind testing whether #531 works? Otherwise we can presume it does; it solves your (excellent!) MCVE

cargo install --git=https://github.com/max-sixty/insta/ --branch=396

regexident · 2024-07-22T10:50:44Z

@max-sixty Just gave it a few test runs and got no errors. Awesome!

Thanks for looking into the issue! 🙏

max-sixty · 2024-09-11T17:21:53Z

Fixed by #532

dzfrias added the bug Something isn't working label Jul 29, 2023

max-sixty changed the title ~~Duplicated snapshots and os error 2 when running both integration and unit tests~~ Duplicated snapshots and os error 2 when running a workspace + root crate Jul 18, 2024

max-sixty added a commit to max-sixty/insta that referenced this issue Jul 21, 2024

Fix snapshots within a nested crate

2986754

Fixes mitsuhiko#396

max-sixty mentioned this issue Jul 21, 2024

Fix snapshots within a nested crate #531

Closed

max-sixty mentioned this issue Jul 21, 2024

Pending snapshot in nested crate is double-counted #388

Closed

max-sixty closed this as completed Sep 11, 2024

max-sixty mentioned this issue Sep 11, 2024

Regression running cargo-insta in workspace with crate in root #593

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Duplicated snapshots and os error 2 when running a workspace + root crate #396

Duplicated snapshots and os error 2 when running a workspace + root crate #396

dzfrias commented Jul 29, 2023

TheLostLambda commented Feb 22, 2024

regexident commented Jul 15, 2024

max-sixty commented Jul 15, 2024

regexident commented Jul 16, 2024

max-sixty commented Jul 16, 2024

regexident commented Jul 17, 2024

max-sixty commented Jul 17, 2024

regexident commented Jul 18, 2024 •

edited

Loading

max-sixty commented Jul 18, 2024

regexident commented Jul 19, 2024

regexident commented Jul 19, 2024

max-sixty commented Jul 19, 2024

max-sixty commented Jul 21, 2024

regexident commented Jul 22, 2024

max-sixty commented Sep 11, 2024

Duplicated snapshots and os error 2 when running a workspace + root crate #396

Duplicated snapshots and os error 2 when running a workspace + root crate #396

Comments

dzfrias commented Jul 29, 2023

What happened?

Reproduction steps

Insta Version

rustc Version

What did you expect?

TheLostLambda commented Feb 22, 2024

regexident commented Jul 15, 2024

max-sixty commented Jul 15, 2024

regexident commented Jul 16, 2024

max-sixty commented Jul 16, 2024

regexident commented Jul 17, 2024

max-sixty commented Jul 17, 2024

regexident commented Jul 18, 2024 • edited Loading

max-sixty commented Jul 18, 2024

regexident commented Jul 19, 2024

regexident commented Jul 19, 2024

max-sixty commented Jul 19, 2024

max-sixty commented Jul 21, 2024

regexident commented Jul 22, 2024

max-sixty commented Sep 11, 2024

regexident commented Jul 18, 2024 •

edited

Loading