module: revert checking for package.json before extensions #15015

bmeck · 2017-08-24T15:02:15Z

This PR reverts a regression from a change in behavior that deviated from the documented behavior of the module resolution algorithm. The deviation caused the module resolution algorithm to check for package.json files when traversing up to a parent directory before following the documented module resolution algorithm.

Link discussing the regression in greater depth

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

module

ljharb

would love this + a backport

mscdex · 2017-08-24T17:04:02Z

I'm not sure we need to use the word "revert" if the current behavior was added a very long time ago.

bmeck · 2017-08-24T17:06:53Z

@mscdex it has accidental side effects, I would prefer to keep that word.

ljharb · 2017-08-24T17:07:31Z

If it was unintentional and undocumented, it's a bug - I think the term isn't bound by recency.

targos · 2017-09-13T08:20:32Z

/cc @nodejs/tsc

I am in favor of this change

BridgeAR · 2017-12-05T03:15:14Z

I am also in favor of this. It seems like a more correct approach than the way it is currently working.

As there was no progress for a long time, I would like to have the @nodejs/tsc to have a look at this and to decide on how to move forward. There are already two TSC votes in favor.

BridgeAR

This needs a rebase. LGTM otherwise

BridgeAR · 2017-12-05T03:19:03Z

test/parallel/test-require-extension-over-directory.js

+  fixturesRequire,
+  require(fixtures.path('module-extension-over-directory', 'inner.js')),
+  'test-require-extension-over-directory failed to import fixture' +
+  ' requirements'


There is a new eslint rule in place that prevents messages without dynamic content (or it is at least planned). I do see that it makes sense to have a individual message here, so we should either explicitly accept the message here or maybe just change the inner.js export content to contain a property with a string, remove the message and use deepStrictEqual. With the latter the error message should be explicit enough.

@BridgeAR If that rule ever comes to pass, we can always disable it with a comment for the instances where a string literal makes sense. So this can stay as is, if the proposed lint rule is the only motivation for changing it.

BridgeAR · 2017-12-11T08:49:16Z

@bmeck I guess you actually want this to be looked at and it to get merged, right?

In that case it will need a rebase and the title should be changed accordingly.

fhinkel · 2017-12-13T10:59:00Z

@BridgeAR I don't see any -1. With the approvals, I think we can go ahead and merge it (after a rebase). Can we take this off the TSC agenda? Or am I missing something?

mcollina

LGTM with a better PR description and commit description

BridgeAR · 2017-12-15T19:23:33Z

@fhinkel sure, I just removed the labels. I just wanted to make sure it is properly looked at as it was originally controversial if that should be changed or not.

BridgeAR · 2017-12-28T23:44:30Z

Ping @bmeck

bmeck · 2017-12-29T00:12:51Z

I can make a better description tomorrow and I think we are approved beyond that.

jasnell · 2018-01-22T15:48:44Z

just double checking... this will land in v10

Should be plenty of time for that assuming it lands by end of March... :-)

jdalton · 2018-01-22T16:06:29Z

Anyone know about the treatment of the trailing /. In some places I see logic to handle \ (for Windows). I'm wondering if it's needed here too.

Update:

Okay confirmed. Windows supports paths ending in \. and \... I think the check should be done to support both or branch per platform as is common elsewhere in Node.

bmeck · 2018-01-22T17:46:23Z

@jdalton

I think the check should be done to support both or branch per platform as is common elsewhere in Node.

The surrounding code is only looking at / characters outside of this PR, it looks like \\ has been swapped by the point it gets to that otherwise there would be lots of bugs in module.js.

jdalton · 2018-01-22T17:58:22Z

It looks like it goes

`require` then
`Module.prototype.require` then
`Module._resolveFilename` then
`Module._findPath`, with array from `Module._resolveLookupPaths`, then
// this code

The request isn't normalized through those passes yet.
The bug is likely there with the initial trailingSlash check in Node today too.

Update:

Filed the bug: #18299.

jdalton · 2018-01-22T22:30:45Z

lib/module.js

-      } else if (rc === 1) {  // Directory.
-        if (exts === undefined)
-          exts = Object.keys(Module._extensions);
-        filename = tryPackage(basePath, exts, isMain);
      }


👆 at the moment because the trailing slash check is for forward-slash-only, on Windows paths ending in a backslash hit that code path (which is removed in this PR). This means it will now fall through to the tryExtensions call on line 234, instead of the tryPackage call on line 243.

Note: Assumes a fixed #18299

BridgeAR · 2018-02-07T02:51:39Z

CI https://ci.nodejs.org/job/node-test-pull-request/12986/

BridgeAR · 2018-02-09T02:53:57Z

@bmeck it seems like there is a lint error and a tests constantly fails:

not ok 1376 parallel/test-require-extensions-same-filename-as-dir
  ---
  duration_ms: 0.171
  severity: fail
  stack: |-
    assert.js:74
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: undefined strictEqual 'artischocko'
        at Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1010/test/parallel/test-require-extensions-same-filename-as-dir.js:31:8)
        at Module._compile (module.js:668:30)
        at Object.Module._extensions..js (module.js:679:10)
        at Module.load (module.js:580:32)
        at tryModuleLoad (module.js:520:12)
        at Function.Module._load (module.js:512:3)
        at Function.Module.runMain (module.js:709:10)
        at startup (bootstrap_node.js:189:16)
        at bootstrap_node.js:699:3

Would you be so kind and take a look? :-)

The documented resolution algorithm was regressed and started to search for package.json files prior to searching for file extensions when searching for a specifier. Oddly, it did not search for index files at same time it searched for package.json. This reverts that regression and restores the documented behavior of searching for file extensions prior to searching directories. Fixes: nodejs#14990 PR-URL: nodejs#15015 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

BridgeAR · 2018-02-17T05:29:43Z

I just rebased this, fixed the linter error and fixed the test issue. I am a bit confused that the old test is now working as before.

@bmeck can you please have a look at this?

BridgeAR · 2018-02-17T05:30:22Z

New CI https://ci.nodejs.org/job/node-test-pull-request/13236/

bmeck · 2018-02-17T05:36:45Z

@BridgeAR I'm dealing w/ family, won't get to this in next few days.

BridgeAR · 2018-03-02T03:40:10Z

Ping @bmeck. I hope everything is fine on your side :-)

bmeck · 2018-03-02T14:02:43Z

@BridgeAR I'm still pretty under water with workload due to things, nothing seems really apparent on why it would fail. I could try to revert and then git bisect if desired but won't get to it for a while.

BridgeAR · 2018-03-02T14:06:07Z

@bmeck I fixed the failure in 39f51aa but I wanted you to confirm that this is the right thing. To me it seems like a independent change made it possible to have this change without having a impact on any existing code.

BridgeAR · 2018-03-02T14:06:34Z

And I rebased and pushed the changes here. So this is good to go if you give your LG.

bmeck · 2018-03-02T14:22:27Z

@BridgeAR that test change looks correct, this PR should be adding trailing / effectively when going up a directory. So it should preserve upwards traversal looking at package.json with that test which uses ../.. for the require specifier. It would have had to have been ../../../module-stub for it to grab module-stub.json instead.

The documented resolution algorithm started to search for package.json files prior to searching for file extensions when searching for a specifier. Oddly, it did not search for index files at same time it searched for package.json. This restores the documented behavior of searching for file extensions prior to searching directories. PR-URL: nodejs#15015 Fixes: nodejs#14990 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Khaidi Chu <i@2333.moe>

BridgeAR · 2018-03-11T02:26:09Z

Landed in 1ed36ae 🎉

The documented resolution algorithm started to search for package.json files prior to searching for file extensions when searching for a specifier. Oddly, it did not search for index files at same time it searched for package.json. This restores the documented behavior of searching for file extensions prior to searching directories. PR-URL: nodejs#15015 Fixes: nodejs#14990 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Khaidi Chu <i@2333.moe>

nodejs-github-bot added the module Issues and PRs related to the module subsystem. label Aug 24, 2017

bmeck changed the title ~~[module] Revert checking for package.json before extensions~~ DO NOT MERGE: [module] Revert checking for package.json before extensions Aug 24, 2017

ljharb approved these changes Aug 24, 2017

View reviewed changes

jasnell added the wip Issues and PRs that are still a work in progress. label Aug 24, 2017

mscdex added the semver-major PRs that contain breaking changes and should be released in the next major version. label Aug 24, 2017

mcollina mentioned this pull request Sep 13, 2017

Regression: Resolution algorithm collision #14990

Closed

BridgeAR requested a review from a team September 27, 2017 22:11

BridgeAR added tsc-agenda Issues and PRs to discuss during the meetings of the TSC. tsc-review labels Dec 5, 2017

BridgeAR approved these changes Dec 5, 2017

View reviewed changes

MylesBorins force-pushed the master branch from b7405ab to 7f086dd Compare December 8, 2017 16:37

mhdawson mentioned this pull request Dec 11, 2017

Node.js Foundation Technical Steering Committee (TSC) Meeting 2017-12-13 nodejs/TSC#435

Closed

mcollina approved these changes Dec 15, 2017

View reviewed changes

BridgeAR removed tsc-agenda Issues and PRs to discuss during the meetings of the TSC. tsc-review labels Dec 15, 2017

BridgeAR changed the title ~~DO NOT MERGE: [module] Revert checking for package.json before extensions~~ module: revert checking for package.json before extensions Dec 15, 2017

ljharb mentioned this pull request Jan 9, 2018

When core modules are shadowed, behavior differs from require.resolve browserify/resolve#147

Closed

BridgeAR requested review from a team, jasnell and MylesBorins January 19, 2018 16:04

jasnell approved these changes Jan 22, 2018

View reviewed changes

jdalton mentioned this pull request Jan 22, 2018

require(".\\") doesn't resolve index.js on Windows #18299

Closed

jdalton reviewed Jan 22, 2018

View reviewed changes

XadillaX approved these changes Jan 23, 2018

View reviewed changes

maclover7 force-pushed the master branch from bb5575a to 993b716 Compare January 26, 2018 22:02

cjihrig force-pushed the master branch from 993b716 to 082f952 Compare January 26, 2018 22:36

bmeck and others added 2 commits February 17, 2018 05:54

fixup: address linter & test issue

39f51aa

BridgeAR force-pushed the issue-14990 branch from 28d83f0 to 39f51aa Compare February 17, 2018 05:29

fixup: remove added new line

c3e5b81

BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Mar 2, 2018

BridgeAR closed this Mar 11, 2018

richardlau mentioned this pull request Mar 29, 2018

require does not work as described in the documentation #18695

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

module: revert checking for package.json before extensions #15015

module: revert checking for package.json before extensions #15015

bmeck commented Aug 24, 2017 •

edited

Loading

ljharb left a comment

mscdex commented Aug 24, 2017

bmeck commented Aug 24, 2017

ljharb commented Aug 24, 2017 •

edited

Loading

targos commented Sep 13, 2017

BridgeAR commented Dec 5, 2017

BridgeAR left a comment

BridgeAR Dec 5, 2017

Trott Dec 5, 2017

BridgeAR commented Dec 11, 2017

fhinkel commented Dec 13, 2017

mcollina left a comment

BridgeAR commented Dec 15, 2017

BridgeAR commented Dec 28, 2017

bmeck commented Dec 29, 2017

jasnell commented Jan 22, 2018

jdalton commented Jan 22, 2018 •

edited

Loading

bmeck commented Jan 22, 2018

jdalton commented Jan 22, 2018 •

edited

Loading

jdalton Jan 22, 2018 •

edited

Loading

BridgeAR commented Feb 7, 2018

BridgeAR commented Feb 9, 2018

BridgeAR commented Feb 17, 2018

BridgeAR commented Feb 17, 2018

bmeck commented Feb 17, 2018

BridgeAR commented Mar 2, 2018 •

edited

Loading

bmeck commented Mar 2, 2018

BridgeAR commented Mar 2, 2018

BridgeAR commented Mar 2, 2018

bmeck commented Mar 2, 2018

BridgeAR commented Mar 11, 2018

module: revert checking for package.json before extensions #15015

module: revert checking for package.json before extensions #15015

Conversation

bmeck commented Aug 24, 2017 • edited Loading

Checklist

Affected core subsystem(s)

ljharb left a comment

Choose a reason for hiding this comment

mscdex commented Aug 24, 2017

bmeck commented Aug 24, 2017

ljharb commented Aug 24, 2017 • edited Loading

targos commented Sep 13, 2017

BridgeAR commented Dec 5, 2017

BridgeAR left a comment

Choose a reason for hiding this comment

BridgeAR Dec 5, 2017

Choose a reason for hiding this comment

Trott Dec 5, 2017

Choose a reason for hiding this comment

BridgeAR commented Dec 11, 2017

fhinkel commented Dec 13, 2017

mcollina left a comment

Choose a reason for hiding this comment

BridgeAR commented Dec 15, 2017

BridgeAR commented Dec 28, 2017

bmeck commented Dec 29, 2017

jasnell commented Jan 22, 2018

jdalton commented Jan 22, 2018 • edited Loading

bmeck commented Jan 22, 2018

jdalton commented Jan 22, 2018 • edited Loading

jdalton Jan 22, 2018 • edited Loading

Choose a reason for hiding this comment

BridgeAR commented Feb 7, 2018

BridgeAR commented Feb 9, 2018

BridgeAR commented Feb 17, 2018

BridgeAR commented Feb 17, 2018

bmeck commented Feb 17, 2018

BridgeAR commented Mar 2, 2018 • edited Loading

bmeck commented Mar 2, 2018

BridgeAR commented Mar 2, 2018

BridgeAR commented Mar 2, 2018

bmeck commented Mar 2, 2018

BridgeAR commented Mar 11, 2018

bmeck commented Aug 24, 2017 •

edited

Loading

ljharb commented Aug 24, 2017 •

edited

Loading

jdalton commented Jan 22, 2018 •

edited

Loading

jdalton commented Jan 22, 2018 •

edited

Loading

jdalton Jan 22, 2018 •

edited

Loading

BridgeAR commented Mar 2, 2018 •

edited

Loading