Support glob patterns for crate arguments to --package and --exclude flags

[fitzgen]

Describe the problem you are trying to solve

In workspaces where there are many crates, I would like to only build/test the crates that share the "peepmatic-" name prefix. Or I would like to exclude all of the crates with the "wasmtime-" name prefix.

Describe the solution you'd like

Support glob-style * wildcards for the -p/--package and --exclude flags when building in a workspace, like

$ cargo build --exclude peepmatic-*

Notes

Would help cut down on monstrosities like this:

        cargo +nightly \
            -Zfeatures=all -Zpackage-features \
            test \
            --features test-programs/test_programs \
            --features experimental_x64 \
            --all \
            --exclude lightbeam \
            --exclude peepmatic \
            --exclude peepmatic-automata \
            --exclude peepmatic-fuzzing \
            --exclude peepmatic-macro \
            --exclude peepmatic-runtime \
            --exclude peepmatic-test

We could collapse all those excludes into just --exlude lightbeam --exclude peepmatic*.

[weihanglo]

I am interesting in this and wrote a proof-of-concept. However I need guidance on some design topics.

First, we know that cargo uses an onion model to determine the final compilation targets (borrows from @ehuss's comment):

workspace > package > target crate > modules > (cfg, attr, ...)

Supposed this issue is trying to deal with workspace level and we can just stick to the function Packages::to_package_id_specs where the actual opt-out works. The following are design issues that call for advice.

• The current opt-out (--exclude) mechanism compares packages as string but not PackageIdSpec, which does not match what command line help says (--exclude <SPEC>...). Not sure shall --exclude respect the Spec or not.
• The original approach using BTreeSet is very clever but my POC gets worse on time complexity (O(pkg * excludes)). Is there any space to improve? Is it need to resolve immediately?
• Do we need all the glob functionalities or just a simple * wildcard? Should we write simple wildcard from scratch?
• Bash on macOS expands glob args automatically. For example cargo build --workspace --exclude tokio-* is interpreted as cargo build --workspace --exclude tokio-test tokio-util tokio-macros, which is not we desire. By quoting around the arg we can solve it but not sure is this ergonomics enough.

Here is my proof-of-concept on opt-out process:

            Packages::OptOut(opt_out) => {
                let opt_out = opt_out
                    .iter()
                    .map(|pat| (glob::Pattern::new(pat).unwrap(), 0))
                    .collect::<Vec<_>>();
                let packages = ws
                    .members()
                    .filter(|pkg| opt_out.iter().any(|(pat, _)| pat.matches(pkg.name().as_str())))
                    .map(Package::package_id)
                    .map(PackageIdSpec::from_package_id)
                    .collect();
                    let not_match = opt_out
                        .iter()
                        .filter(|(_, count)| *count == 0)
                        .map(|(pat, _)| pat.as_str())
                        .collect::<Vec<_>>();
                if !not_match.is_empty() {
                    ws.config().shell().warn(format!(
                        "excluded package(s) {} not found in workspace `{}`",
                        not_match.join(", "),
                        ws.root().display(),
                    ))?;
                }
                packages
            }

[weihanglo]

@fitzgen I've created a pull request #8752 for this issue. Is that what you expect?

[fitzgen]

Based on the documentation changes, yes. I am not familiar enough with the cargo codebase to comment on the implementation.