Merge 38789bf into 2bb8c9a

.github/workflows/setup-nextjs-build.yml

@@ -126,7 +126,7 @@ jobs:
       # Note that we do not use upload / download artifacts for this -
       # it is too heavyweight for the purpose since we do not need to persist
       # the cache across multiple runs.
-      - name: Store next.js build cache with next-dev binary
+      - name: Store next.js build cache with next-swc
         uses: actions/cache/save@v3
         id: cache-build
         with:

.github/workflows/test.yml

@@ -641,7 +641,7 @@ jobs:
           (set -o pipefail && \
           cargo check \
             --config $BINDING --config $TASKS --config $TASKS_FS \
-            --all -p next-swc-api -p next-dev \
+            --all -p next-swc-api \
             --no-default-features --features cli,custom_allocator,rustls-tls \
             --message-format short --quiet \
           2>&1 | tee cargo_output.log)
@@ -975,7 +975,7 @@ jobs:
           # One of Turbopack with 1000 modules is benchmarked in every run
           # to create a baseline result for normalization (should the runners performance vary between jobs)
 
-          # This runs all the non next-dev benchmarks
+          # This runs all the non turbopack-bench benchmarks
           - name: generic
             cache_key: all
             args: --workspace --exclude turbopack-bench

CONTRIBUTING.md

@@ -258,7 +258,7 @@ Finally, the crate must be added to the Turborepo section of CODEOWNERS:
 
 Turbopack uses [Cargo workspaces][workspaces] in the Turbo monorepo. You'll find
 several workspaces inside the `crates/` directory. In order to run a particular
-crate, you can use the `cargo run -p [CRATE_NAME]` command. For example, to test the Next.js development server, run `cargo run -p next-dev`.
+crate, you can use the `cargo run -p [CRATE_NAME]` command.
 
 ### Turbopack Architecture
 
@@ -288,10 +288,6 @@ You can also create a little demo app and run
 cargo run -p node-file-trace -- print demo/index.js
 ```
 
-### Benchmarking Turbopack
-
-See [the benchmarking README for Turbopack](crates/next-dev/benches/README.md) for details.
-
 ### Profiling Turbopack
 
 See [the profiling docs for Turbopack](https://turbo.build/pack/docs/advanced/profiling) for details.

crates/turbopack-bench/README.md

@@ -0,0 +1,45 @@
+<!--
+    IMPORTANT: This document is linked to from https://turbo.build
+    DO NOT rename this file without updating the link in docs/pages/pack/docs/benchmarks.mdx
+-->
+
+# Benchmarking Turbopack
+
+The simplest way to run Turbopack's benchmark suite is with the command `cargo bench -p turbopack-bench`. This will benchmark Turbopack's Next.js development server in a variety of scenarios and it's what we use to track Turbopack's performance over time.
+
+`cargo bench -p turbopack-bench` accepts different options via environment variables. To vary the number of modules tested, set `TURBOPACK_BENCH_COUNTS`. For example, to test against an app with 5,000 modules instead of the default 1,000, run
+
+```sh
+TURBOPACK_BENCH_COUNTS=5000 cargo bench -p turbopack-bench
+```
+
+## Benchmarking Turbopack against other bundlers
+
+The benchmark numbers we share on [the Turbopack website](https://turbo.build/pack) are informed by running Turbopack's benchmark suite against Turbopack and other bundlers. These are run in a controlled environment prior to being published. We use the `bench_startup` and `bench_hmr_to_eval` benchmarks currently (see below).
+
+To run Turbopack benchmarks against other bundlers, run:
+
+```sh
+cargo bench -p turbopack-bench -p turbopack-cli
+```
+
+and optionally filter the benchmarks run to specific bundlers, such as:
+
+```sh
+cargo bench -p turbopack-bench -p turbopack-cli -- "hmr_to_eval/(Turbopack CSR|Vite)"
+```
+
+**Note**: The Turbopack benchmark suite includes a mix of server-side rendered and client-only rendered examples -- these are reflected in "CSR" or "SSR" in the benchmark name. Turbopack supports both, while some other bundlers only support client-rendered examples. Take that into account when comparing CSR results against SSR.
+
+**Hint**: These benchmarks take a long time to complete, since they try to capture at least 10 samples for every scenario. There is a `TURBOPACK_BENCH_PROGRESS=1` env var to show values while the benchmarks are running.
+
+## Benchmark Suite scenarios
+
+The benchmark suite runs Turbopack and other bundlers in a variety of scenarios. The tests use a real headless browser and perform a variety of common scenarios in web development, and wait for results to be reflected in the page.
+
+- **bench_startup:** Time from startup (without cache) until the app is rendered in the browser (it doesn't have to be interactive/hydrated for this.)
+- **bench_hydration:** Time from startup (without cache) until the app is interactive in the browser (it needs to be hydrated for that.) This metric is not captured for CSR since the first render is interactive.
+- **bench_hmr_to_eval:** Time from changing a file until the new code is evaluated in the browser. Evaluating the code does not mean the change is visible to the user yet. For instance, when a React component changes, it needs to be re-rendered in the browser. This mostly measures the time spent computing the update in the bundler itself and sending it to the client.
+- **bench_hmr_to_commit:** Time from changing a file until the change is reflected in the browser. We are using a `useEffect` hook within a React component to measure the time it takes for the updated React component to be committed to the DOM. This is a good measure of the end to end performance perceived by the user.
+- **bench_startup_cache:** Time from startup with persistent cache until the app is rendered in the browser (it doesn't have to be interactive/hydrated for this.). Turbopack doesn't include a persistent cache yet. (This benchmark is disabled by default and can be enabled with `TURBOPACK_BENCH_CACHED=1`)
+- **bench_hydration:** Time from startup with persistent cache until the app is interactive in the browser (it needs to be hydrated for that.) This metric is not captured for CSR since the first render is interactive. Turbopack doesn't include a persistent cache yet. (This benchmark is disabled by default and can be enabled with `TURBOPACK_BENCH_CACHED=1`)

docs/pages/blog/turbopack-benchmarks.mdx

@@ -99,7 +99,7 @@ Based on feedback and collaboration with the Vite team, we used the [SWC plugin]
 To run this benchmark yourself, clone [`vercel/turbo`](https://github.com/vercel/turbo) and then use this command from the root:
 
 ```sh
-TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p next-dev "startup/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)."
+TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p turbopack-bench "startup/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)."
 ```
 
 Here are the numbers we were able to produce on a 16” MacBook Pro 2021, M1 Max, 32GB RAM, macOS 13.0.1 (22A400):
@@ -176,7 +176,7 @@ For more info, visit the comparison docs for [Vite](/pack/docs/comparisons/vite)
 To run this benchmark yourself, clone [`vercel/turbo`](https://github.com/vercel/turbo) and then use this command from the root:
 
 ```
-TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p next-dev "hmr_to_commit/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)"
+TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p turbopack-bench "hmr_to_commit/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)"
 ```
 
 Here are the numbers we were able to produce on a 16” MacBook Pro 2021, M1 Max, 32GB RAM, macOS 13.0.1 (22A400):

docs/pages/pack/docs/advanced/profiling.mdx

@@ -19,12 +19,12 @@ Make sure you have all the [prerequisites](https://github.com/cmyr/cargo-instrum
 
 ### Run the profiler
 
-By default, the next-dev CLI will keep watching for changes to your application and never exit until you manually interrupt it. However, [`cargo-instruments`] waits for your program to exit before building and opening the trace file. For this purpose, we've added a `profile` feature to `next-dev` which exits the program if no updates are detected within a given time frame and there are no pending tasks.
+By default, `turbopack-cli dev` will keep watching for changes to your application and never exit until you manually interrupt it. However, [`cargo-instruments`] waits for your program to exit before building and opening the trace file. For this purpose, we've added a `profile` feature to `turbopack-cli` which exits the program if no updates are detected within a given time frame and there are no pending tasks.
 
-To profile `next-dev`, run the following command:
+To profile `turbopack-cli`, run the following command:
 
 ```sh
-cargo instruments -t time --bin next-dev --release --features profile [-- [...args]]
+cargo instruments -t time --bin turbopack-cli --release --features profile [-- [...args]]
 ```
 
 You can also run [other templates](https://github.com/cmyr/cargo-instruments#templates) than the time profiler.
@@ -56,14 +56,14 @@ Once the program exits, the profiler will open the trace file in Instruments. Re
 sudo apt install heaptrack heaptrack_gui
 
 # Compile with debug info but without the alternative allocator:
-CARGO_PROFILE_RELEASE_DEBUG=1 cargo build --bin next-dev --release --no-default-features --features cli
+CARGO_PROFILE_RELEASE_DEBUG=1 cargo build --bin turbopack-cli --release --no-default-features --features native-tls
 
 # Run the binary with heaptrack (it will be much slower than usual)
-heaptrack target/release/next-dev [...]
+heaptrack target/release/turbopack-cli [...]
 
 # Stop it anytime
 
-# Open the GUI and open the heaptrack.next-dev.XXX.gz file
+# Open the GUI and open the heaptrack.turbopack-cli.XXX.gz file
 heaptrack_gui
 ```
 

docs/pages/pack/docs/benchmarks.mdx

@@ -36,7 +36,7 @@ As the current state of the art, we're including [Vite](https://vitejs.dev/) alo
 Let's break down exactly what each of these metrics mean, and how they'll impact your day-to-day developer experience.
 
 <Callout type="info">
-  Curious about how these benchmarks are implemented, or want to run them yourself? Check out [our benchmark suite documentation in the Turbo monorepo](https://github.com/vercel/turbo/blob/main/crates/next-dev/benches/README.md).
+  Curious about how these benchmarks are implemented, or want to run them yourself? Check out [our benchmark suite documentation in the Turbo monorepo](https://github.com/vercel/turbo/blob/main/crates/turbopack-bench/README.md).
 </Callout>
 
 ### Cold startup time
@@ -66,7 +66,7 @@ This test measures how fast a local development server starts up on an applicati
 To run this benchmark yourself, clone [`vercel/turbo`](https://github.com/vercel/turbo) and then use this command from the root:
 
 ```sh
-TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p next-dev "startup/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)."
+TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p turbopack-bench "startup/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)."
 ```
 
 Here are the numbers we were able to produce on a 16” MacBook Pro 2021, M1 Max, 32GB RAM, macOS 13.0.1 (22A400):
@@ -145,7 +145,7 @@ We're excited by Turbopack's performance, but we also expect to make further spe
 To run this benchmark yourself, clone [`vercel/turbo`](https://github.com/vercel/turbo) and then use this command from the root:
 
 ```
-TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p next-dev "hmr_to_commit/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)"
+TURBOPACK_BENCH_COUNTS=1000,5000,10000,30000 TURBOPACK_BENCH_BUNDLERS=all cargo bench -p turbopack-bench "hmr_to_commit/(Turbopack SSR|Next.js 12 SSR|Next.js 11 SSR|Vite SWC CSR)"
 ```
 
 Here are the numbers we were able to produce on a 16” MacBook Pro 2021, M1 Max, 32GB RAM, macOS 13.0.1 (22A400):

pnpm-workspace.yaml

@@ -9,8 +9,6 @@ packages:
   # Intentionally exclude the `npm` `turbo` package from the workspaces.
   - "!packages/turbo"
   - "packages/turbo-tracing-next-plugin/test/with-mongodb-mongoose"
-  - "crates/next-dev-tests/test-harness"
-  - "crates/next-dev-tests/tests"
   - "crates/turbopack-tests/tests/execution"
   - "crates/*/js"
   - examples