Alternative package manager support for the module resolution

[XiNiHa]

Describe the feature

Since the currently implemented plugin resolve algorithm relies on the node_modules structure, it doesn't work with alternative module linking approaches like ones from pnpm and Yarn Berry.
While directly pointing at the WASM binary is implemented as a temporary workaround, this should be covered with a more elegant and user-friendly approach.

Initially discussed in the Discord channel

Babel plugin or link to the feature description

No response

Additional context

No response

[kwonoj]

This is largely about module resolution we use in swc. It is not limited to plugin itself but somewhat overlaps to bundler, or any other features needs module resolution in some way.

Two main questions

• We'll support npm as first class citizen. What else / and how? there are some esoteric implementation like yarn's pnp.
• What / how to provide way to explicitly specify some resolution? We already support polymorphic behavior of plugin config itself (path vs. module name). But we need better.

[charrondev]

We'll support npm as first class citizen. What else / and how? there are some esoteric implementation like yarn's pnp.

One really nice thing about yarn pnp is that there is one source of truth for how modules are resolved. Particularly with monorepo structures it can be really complicated if there are multiple things trying to resolve a module:

• Your IDE.
• Your bundler.
• Other tooling (type-checker, linting, etc).

[kwonoj]

One really nice thing about yarn pnp

I know good things about pnp and I'd like to use for some other places as well.

Main blocker in here is yarn's pnp resolution relies on its generated resolver in js, which swc's host cannot resolve by running it. This means it may requires to reimplement whole resolving logic by ourselves which I consider as heavy blocker to support.

[RiESAEX]

for plugin:
I think a postInstall would work,
or maybe we can put it in bin in package.json and try to parse generated .sh file in node_modules/.bin.
for bundler:
I think it support pnpm since we follow soft link. but no idea for yarn pnp.

[laclance]

any update?

[kwonoj]

We do not have any work in the progress / do not have visible plan for this. This issue is filed for tracking purpose.

[Ethorsen]

Support for Yarn 3.x PNP would be really appreciated. Thx

[noahnu]

Main blocker in here is yarn's pnp resolution relies on its generated resolver in js, which swc's host cannot resolve by running it.

Yarn supports a JSON data file. Perhaps that might be an easier pnp map to integrate with? You don't need to run any JS.

The preferred approach IMO would be to expose a plugin API for module resolution. Yarn could then maintain their own plugin.

[kwonoj]

The doc I was referencing was https://yarnpkg.com/features/pnp#initializing-pnp which explicitly says it creates cjs. Where does those json spec exists?

Also it doesn't mean reducing lot of work. The only difference is port cjs resolution logic to rust, while json we need to still deal with all others like zipfs implementation.

Opening up module resolution is huge refactoring work, and it still requires yarn implements above logic. If that's the case, someone can directly upstream pr still. In short, the issue is amount of the work and resources we need compare to the active userbase we currently support who should use pnp.

[noahnu]

It's opt-in. You have to set pnpEnableInlining to false to generate the json file: https://yarnpkg.com/configuration/yarnrc#pnpEnableInlining

[sargunv]

The spec for .pnp.data.json is here: https://yarnpkg.com/advanced/pnp-spec.

In addition to Yarn itself, it's also been implemented in ESBuild v0.15.0.

[kwonoj]

I'm marking this as open for contributions. Anyone would like to provide support welcome to review PR, however we do not have enough bandwidth to try this by our own.

[swwind]

pnpm only places packages declared explicitly in the package.json to the root node_modules directory. For secondary dependencies, pnpm stores them within the node_modules directory of their direct dependencies.

Actually I'm writing a tool using swc and some plugins. When I install this tool in other projects, pnpm actually install dependencies like this.

my-project/
└── node_modules/
    └── my-build-tools/
        └── node_modules/
            ├── @swc/core/...
            └── @swc/plugin-styled-jsx/...

And sadly when using my-build-tools inside my-project, swc cannot find the plugin @swc/plugin-styled-jsx and aborts.

[swwind]

I suggest that plugins export a URL pointing to the location of the wasm file. This approach can simplify the process of finding the wasm file for plugins.

// index.mjs
import { fileURLToPath } from "node:url";
export default fileURLToPath(new URL("./my-swc-plugin.wasm", import.meta.url))

You can then use it like this:

import plugin from "my-swc-plugin";
// ...
jsc: {
  experimental: {
    plugins: [[plugin, {}]]
  }
}

This should work with npm, pnpm, and yarn pnp.

However, this approach conflicts with the default "main": "wasm" strategy. If there are plans to change the current strategy, I would prefer adopting this solution.

[XiNiHa]

Utilizing oxc-resolver (or implementing the equivalent?) will solve the issue at least for pnpm users. Looks like PnP support is also lacking for them.

[XiNiHa]

Looks like the Rspack team just started to explore PnP support for oxc-resolver with a separate rspack-resolver crate.