pylance slow to start analysis after vscode start

[onet-git]

Environment data

• Language Server version: 2021.5.2
• OS and version: Ubuntu 18.04.5 LTS
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.6.9

Expected behaviour

Pylance can start providing language support within a few seconds of vscode starting.

Actual behaviour

On "fast" hardware Pylance takes 60+ seconds to start analysis of the first open file. On slow hardware it can take several minutes.

Logs

[Info  - 4:30:18 PM] No configuration file found.
[Info  - 4:30:18 PM] pyproject.toml file found at /home/mmonroe/av.
[Info  - 4:30:18 PM] Loading pyproject.toml file at /home/mmonroe/av/pyproject.toml
[Error - 4:30:18 PM] Pyproject file "/home/mmonroe/av/pyproject.toml" is missing "[tool.pyright] section.
[Info  - 4:30:18 PM] Setting pythonPath for service "av": "/usr/bin/python3"
[Warn  - 4:30:18 PM] stubPath /home/mmonroe/av/typings is not a valid directory.
[Info  - 4:30:18 PM] Assuming Python version 3.6
[Info  - 4:30:18 PM] Assuming Python platform Linux
Search paths for /home/mmonroe/av
  /home/mmonroe/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib
  /home/mmonroe/av
  /home/mmonroe/av
  /home/mmonroe/av/bazel-bin
  /home/mmonroe/av/bazel-out
  /home/mmonroe/av/typings
  /home/mmonroe/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/...
  /home/mmonroe/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs
  /usr/lib/python3.6
  /usr/lib/python3.6/lib-dynload
  /home/mmonroe/.local/lib/python3.6/site-packages
  /usr/local/lib/python3.6/dist-packages
  /usr/lib/python3/dist-packages
[Error - 4:30:18 PM] Exception received when installing recursive file system watcher
[Info  - 4:30:18 PM] Searching for source files
[Info  - 4:31:25 PM] Found 565581 source files

Code Snippet / Additional information

I suspect the issue is the large number of source files being found by pylance. The workspace its self only contains ~6k python files.

Looking at the search paths listed the bulk of the files seem to be in /usr/local/lib/python3.6/dist-packages. Yet there only appear to be ~100k .py or .pyi files across all of the directories so it is unclear where pylance is coming with its 565581 source files. I have also been unable to find a way to exclude the search directories included by pylance (e.g. pip directories), so isolate the issue.

[jakebailey]

Your workspace root is av; can you try and create a pyrightconfig.json file in the workspace root and exclude directories via the exclude config listed here? https://github.com/microsoft/pyright/blob/main/docs/configuration.md#main-pyright-config-options

Are there any directories that should be ignored? What's in this directory? The bazel output sounds suspect, but I'm not sure if that contains importable code or not.

[jakebailey]

We also support symlinks (and avoid recursion); I believe bazel might use symlinks so we might be picking up more files that way.

[onet-git]

Using pyrightconfig.json I was able to track down a symlink someone added to the repo, which links to a different repo with 400k+ python files. Excluding that directory dropped startup time to under 2 seconds.

Thanks, I guess this is either resolved or made into a feature request to provide visibility into the slow startup issues.

[jakebailey]

Feedback about the startup scan is certainly important; if we can speed this up somehow that would be great. It's one of the few things we have that isn't lazy.

[momodesuka]

I also meet this problem.
Pylance keep scan files in some dir such as /.vscode/extensions and other py files in $HOME
it takes a long long time to loading. And tell me Found more than 5600 source files.

Once I touch pyrightconfig.json in $HOME. It is an empty json file.
I don't know what happened, but Pylance no longer scan all .py files. It only takes a second. Much faster.
Anyway, it works just by making an empty json file.

@jakebailey Really appreciate your help.!

[heejaechang]

closing dup of this #1569