From 93f9023ea38ddac7f33b487d938f47cc1333d567 Mon Sep 17 00:00:00 2001 From: olp-cs <162949+olp-cs@users.noreply.github.com> Date: Thu, 22 Aug 2024 06:35:09 +0300 Subject: [PATCH] Add `hyperfine` installation instructions; update `hyperfine` code samples (#13034) ## Summary When following the step-by-step instructions to run the benchmarks in `CONTRIBUTING.md`, I encountered two errors: **Error 1:** `bash: hyperfine: command not found` **Solution**: I updated the instructions to include the step of installing the benchmark tool. **Error 2:** ```shell $ ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ error: `ruff ` has been removed. Use `ruff check ` instead. ``` **Solution**: I added `check`. ## Test Plan I tested it by running the benchmark-related commands in a new workspace within GitHub Codespaces. --- CONTRIBUTING.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b89d83325425a..6aa5646a6f45a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -397,12 +397,18 @@ which makes it a good target for benchmarking. git clone --branch 3.10 https://github.com/python/cpython.git crates/ruff_linter/resources/test/cpython ``` +Install `hyperfine`: + +```shell +cargo install hyperfine +``` + To benchmark the release build: ```shell cargo build --release && hyperfine --warmup 10 \ - "./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e" \ - "./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ -e" + "./target/release/ruff check ./crates/ruff_linter/resources/test/cpython/ --no-cache -e" \ + "./target/release/ruff check ./crates/ruff_linter/resources/test/cpython/ -e" Benchmark 1: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache Time (mean ± σ): 293.8 ms ± 3.2 ms [User: 2384.6 ms, System: 90.3 ms] @@ -421,7 +427,7 @@ To benchmark against the ecosystem's existing tools: ```shell hyperfine --ignore-failure --warmup 5 \ - "./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache" \ + "./target/release/ruff check ./crates/ruff_linter/resources/test/cpython/ --no-cache" \ "pyflakes crates/ruff_linter/resources/test/cpython" \ "autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython" \ "pycodestyle crates/ruff_linter/resources/test/cpython" \ @@ -467,7 +473,7 @@ To benchmark a subset of rules, e.g. `LineTooLong` and `DocLineTooLong`: ```shell cargo build --release && hyperfine --warmup 10 \ - "./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e --select W505,E501" + "./target/release/ruff check ./crates/ruff_linter/resources/test/cpython/ --no-cache -e --select W505,E501" ``` You can run `poetry install` from `./scripts/benchmarks` to create a working environment for the