Commits on Apr 8, 2023

1. [utils] [perf] Performance of fullResolve …

   While looking at a larger code base https://gitlab.com/gitlab-org/gitlab,
   I've came to realize that `fullResolve` takes a lot of CPU cycles,
   particularly the `hashObject` calls inside it.
   
   I applied the following patch locally to see how often this is called and
   how many unique hashes were produced:
   
   ```diff
   diff --git a/utils/resolve.js b/utils/resolve.js
   index 4a35c6a..3c28324 100644
   --- a/utils/resolve.js
   +++ b/utils/resolve.js
   @@ -83,13 +83,28 @@ function relative(modulePath, sourceFile, settings) {
      return fullResolve(modulePath, sourceFile, settings).path;
    }
   
   +let prevSettings = null;
   +let nonEqualSettings = 0;
   +let totalCalls = 0;
   +let uniqueHashes = new Set();
    function fullResolve(modulePath, sourceFile, settings) {
      // check if this is a bonus core module
      const coreSet = new Set(settings['import/core-modules']);
      if (coreSet.has(modulePath)) return { found: true, path: null };
   
      const sourceDir = path.dirname(sourceFile);
   -  const cacheKey = sourceDir + hashObject(settings).digest('hex') + modulePath;
   +
   +  totalCalls+=1;
   +  const hash = hashObject(settings).digest('hex');
   +
   +  if(prevSettings !== settings){
   +    uniqueHashes.add(hash);
   +    prevSettings = settings;
   +    nonEqualSettings+=1;
   +    console.log(`fullResolve | Total calls:${totalCalls} | Non-Equal settings:${nonEqualSettings} | Unique hashes:${uniqueHashes.size} | dir:${sourceDir}`)
   +  }
   +
   +  const cacheKey = sourceDir + hash + modulePath;
   
      const cacheSettings = ModuleCache.getSettings(settings);
   ```
   
   For our code base, `fullResolve` is called more than 570 thousand times.
   The simple in-equality `!==` code path is taken 1090 times.
   Actually only _four_ unique hashes are produced, meaning we only have
   four unique settings across our code base.
   
   I assume that a full object equality comparison might not be needed,
   and a simple object comparison with `!==` already would reduce the amount
   of `hashObject` calls by 570x. This is what is implemented in this commit.
   
   Time spend in `fullResolve` was reduced by ~38%:
   - Before: 17% (19.10s) of our total execution time
   - After: 11% (11.86s) of our total execution time
   
   The effect might even be more pronounced on machines that are slower when
   calculating `sha256` hashes or that have less memory, as the `hashObject`
   method tends to create loads of small strings which need to be garbage
   collected.

   

   leipert authored and ljharb committed Apr 8, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 6cb9616
   • Browse repository at this point

   Copy the full SHA

   6cb9616 View commit details

   Browse the repository at this point in the history