treat benchmarks as normal tests

[tabokie]

Signed-off-by: tabokie xy.tao@outlook.com

Last I checked this behavior is consistent with cargo test. Bench cases that appear in test binary list should be considered a normal test.

[sunshowers]

Thanks for the PR!

What happens if a benchmark is invoked? Is it run just once or many times?

Note that the most popular benchmarking library criterion.rs doesn't currently work with nextest: bheisler/criterion.rs#562

[tabokie]

@sunshowers In the case of cargo test, all #[bench] benches will behave just like normal unit tests (run just once). They will be run even if --tests is passed. Cases managed by criterion.rs will not be run, because they are not contained in the test binary generated by cargo test.

[sunshowers]

Thanks -- in that case this makes sense. I'll add a note to the release notes explaining that criterion still won't work but this diff doesn't make the situation any worse.

I'll also add some integration tests (using a magic env var to use nightly features on stable rust, but just for the tests...)

[sunshowers]

This is now out as cargo-nextest 0.9.22 (binaries will be out shortly).