Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: cache synchronous module jobs before linking #52868

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 25 additions & 14 deletions lib/internal/modules/esm/module_job.js
Original file line number Diff line number Diff line change
Expand Up @@ -295,22 +295,33 @@ class ModuleJobSync extends ModuleJobBase {
#loader = null;
constructor(loader, url, importAttributes, moduleWrap, isMain, inspectBrk) {
super(url, importAttributes, moduleWrap, isMain, inspectBrk, true);
assert(this.module instanceof ModuleWrap);
this.#loader = loader;
const moduleRequests = this.module.getModuleRequests();
// Specifiers should be aligned with the moduleRequests array in order.
const specifiers = Array(moduleRequests.length);
const modules = Array(moduleRequests.length);
const jobs = Array(moduleRequests.length);
for (let i = 0; i < moduleRequests.length; ++i) {
const { specifier, attributes } = moduleRequests[i];
const job = this.#loader.getModuleJobForRequire(specifier, url, attributes);
specifiers[i] = specifier;
modules[i] = job.module;
jobs[i] = job;

assert(this.module instanceof ModuleWrap);
anonrig marked this conversation as resolved.
Show resolved Hide resolved
// Store itself into the cache first before linking in case there are circular
// references in the linking.
loader.loadCache.set(url, importAttributes.type, this);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting in cache at

job = new ModuleJobSync(this, url, importAttributes, wrap, isMain, inspectBrk);
this.loadCache.set(url, importAttributes.type, job);
would be redundant but I think it could be a follow up work.

Copy link
Member Author

@joyeecheung joyeecheung May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was what I did initially but it would cache modules that failed to link and if that’s caught, subsequent attempts would not fail as expected - which is why I added comments explaining why it’s reset in the finally block (doing it inside the linking phase prevents a rethrow in the caller which looks less nice)

Copy link
Member

@legendecas legendecas May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, maybe we could transform the try-finally into a try-catch-rethrow and only delete in catch? In this way we don't need to set cache again in the caller.

Copy link
Member Author

@joyeecheung joyeecheung May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rethrow was exactly what I am trying to prevent - setting the cache again is fine as the overhead is trivial but having to rethrow in all callers of this makes the code harder to read IMO. It also makes all the source lines of the errors less readable, which shows in the stderr when they are not caught.


try {
const moduleRequests = this.module.getModuleRequests();
// Specifiers should be aligned with the moduleRequests array in order.
const specifiers = Array(moduleRequests.length);
const modules = Array(moduleRequests.length);
const jobs = Array(moduleRequests.length);
for (let i = 0; i < moduleRequests.length; ++i) {
const { specifier, attributes } = moduleRequests[i];
const job = this.#loader.getModuleJobForRequire(specifier, url, attributes);
specifiers[i] = specifier;
modules[i] = job.module;
jobs[i] = job;
}
this.module.link(specifiers, modules);
this.linked = jobs;
} finally {
// Restore it - if it succeeds, we'll reset in the caller; Otherwise it's
// not cached and if the error is caught, subsequent attempt would still fail.
loader.loadCache.delete(url, importAttributes.type);
}
this.module.link(specifiers, modules);
this.linked = jobs;
}

get modulePromise() {
Expand Down
6 changes: 6 additions & 0 deletions lib/internal/modules/esm/module_map.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,12 @@ class LoadCache extends SafeMap {
validateString(type, 'type');
return super.get(url)?.[type] !== undefined;
}
delete(url, type = kImplicitAssertType) {
const cached = super.get(url);
if (cached) {
cached[type] = undefined;
}
}
}

module.exports = {
Expand Down
14 changes: 14 additions & 0 deletions test/common/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ const net = require('net');
const path = require('path');
const { inspect } = require('util');
const { isMainThread } = require('worker_threads');
const { isModuleNamespaceObject } = require('util/types');

const tmpdir = require('./tmpdir');
const bits = ['arm64', 'loong64', 'mips', 'mipsel', 'ppc64', 'riscv64', 's390x', 'x64']
Expand Down Expand Up @@ -938,6 +939,18 @@ function getPrintedStackTrace(stderr) {
return result;
}

/**
* Check the exports of require(esm).
* TODO(joyeecheung): use it in all the test-require-module-* tests to minimize changes
* if/when we change the layout of the result returned by require(esm).
* @param {object} mod result returned by require()
* @param {object} expectation shape of expected namespace.
*/
function expectRequiredModule(mod, expectation) {
assert(isModuleNamespaceObject(mod));
anonrig marked this conversation as resolved.
Show resolved Hide resolved
assert.deepStrictEqual({ ...mod }, { ...expectation });
}

const common = {
allowGlobals,
buildType,
Expand All @@ -946,6 +959,7 @@ const common = {
createZeroFilledFile,
defaultAutoSelectFamilyAttemptTimeout,
expectsError,
expectRequiredModule,
expectWarning,
gcUntil,
getArrayBufferViews,
Expand Down
8 changes: 8 additions & 0 deletions test/es-module/test-require-module-cycle-cjs-esm-esm.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
// Flags: --experimental-require-module
'use strict';

// This tests that ESM <-> ESM cycle is allowed in a require()-d graph.
const common = require('../common');
const cycle = require('../fixtures/es-modules/cjs-esm-esm-cycle/c.cjs');

common.expectRequiredModule(cycle, { b: 5 });
1 change: 1 addition & 0 deletions test/fixtures/es-modules/cjs-esm-esm-cycle/a.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export { b } from './b.mjs';
2 changes: 2 additions & 0 deletions test/fixtures/es-modules/cjs-esm-esm-cycle/b.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
import './a.mjs'
export const b = 5;
1 change: 1 addition & 0 deletions test/fixtures/es-modules/cjs-esm-esm-cycle/c.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = require('./a.mjs');