"RangeError: Maximum call stack size exceeded"

[whoahbot]

Environment data

• Language Server version: 2023.11.10
• OS and version: MacOS Ventura
• Python version: Python 3.11.2

Code Snippet

from bytewax.dataflow import Dataflow

flow = Dataflow()

Repro Steps

1. Install bytewax in a virtual environment pip install bytewax
2. Start VSCode and open a new .py file
3. Add the above code

Expected behavior

Pylance should show inline documentation.

Actual behavior

Pylance crashes.

Logs

2023-11-06 09:43:48.736 [info] [Info  - 9:43:48 AM] (841) Pylance language server 2023.11.10 (pyright 088ebaa5) starting
2023-11-06 09:43:48.736 [info] [Info  - 9:43:48 AM] (841) Server root directory: /Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist
2023-11-06 09:43:48.736 [info] [Info  - 9:43:48 AM] (841) Starting service instance "bytewax-test"
2023-11-06 09:43:48.751 [info] [Info  - 9:43:48 AM] (841) Setting pythonPath for service "bytewax-test": "/Users/whoahbot/.pyenv/versions/3.11.2/envs/test-bytewax/bin/python"
2023-11-06 09:43:48.751 [info] [Info  - 9:43:48 AM] (841) Setting environmentName for service "bytewax-test": "3.11.2 (test-bytewax venv)"
2023-11-06 09:43:48.869 [info] [Info  - 9:43:48 AM] (841) Assuming Python version 3.11
2023-11-06 09:43:48.916 [info] [Info  - 9:43:48 AM] (841) No source files found.
2023-11-06 09:45:41.582 [info] [Info  - 9:45:41 AM] (841) Found 1 source file
2023-11-06 09:46:15.015 [info] [Error - 9:46:15 AM] (841) An internal error occurred while parsing file "/Users/whoahbot/.pyenv/versions/3.11.2/envs/test-bytewax/lib/python3.11/site-packages/bytewax/bytewax.cpython-311-darwin.so": RangeError: Maximum call stack size exceeded
    at t.Tokenizer._skipToEndOfStringLiteral (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1441887)
    at t.Tokenizer._handleString (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1440447)
    at t.Tokenizer._handleCharacter (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1426514)
    at t.Tokenizer._addNextToken (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1426238)
    at t.Tokenizer.tokenize (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1424281)
    at I._startNewParse (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1353067)
    at /Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1351401
    at a.timeOperation (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1126304)
    at I.parseSourceFile (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1351376)
    at _0x520120._parseFile (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:511687)
    at /Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:498412
    at t.LogTracker.log (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1089150)
    at _0x520120.parse (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:497678)
    at N._parseFile (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:432471)
    at t.SourceMapper._boundSourceGetter (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:426950)
    at t.SourceMapper.findModules (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:517368)
    at g (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:546473)
    at _0x446ce3.resolveCompletionItem (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1185672)
    at _0x22ae58.<computed>.resolveCompletionItem (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js:1:569185)
    at /Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js:1:998142
    at /Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js:1:251718
    at /Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:419605
    at runWithCancellationToken (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:842748)
    at a.timeOperation (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:1126304)
    at N._runEvaluatorWithCancellationToken (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:424949)
    at N.run (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:419563)
    at k.run (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/pyright.bundle.js:1:467291)
    at _0x1905c1.<computed>._runCore (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js:1:251706)
    at _0x1905c1.<computed>.run (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js:1:250389)
    at _0x108e2b.onCompletionResolve (/Users/whoahbot/.vscode/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js:1:997498)

/cc bytewax/bytewax/issues/187

[erictraut]

I'm not able to repro the problem as reported. If someone on the pylance team is able to repro, please include repro steps and (if appropriate) move to the pyright project.

[rchiodo]

I can't reproduce the problem either. Typing in that code manually seems to work fine with Pylance 2023.11.10. On reload it still opens.

@whoahbot - The callstack seems to indicate a completion was being created.

Are there any more repro steps that are necessary? Like maybe you get the crash when typing the import?

[whoahbot]

I can't reproduce the problem either. Typing in that code manually seems to work fine with Pylance 2023.11.10. On reload it still opens.

@whoahbot - The callstack seems to indicate a completion was being created.

Are there any more repro steps that are necessary? Like maybe you get the crash when typing the import?

Hi, thanks for having a look at this.

I just tried reproducing the error again, and yes, you are correct. The error was triggered in my new session when I opened the code snippet I provided and tried adding a second import line after the first:

from bytewax.dataflow import Dataflow
from bytewax.i # <-- crash log appears here

The module imports is listed as an autocompletion source, but the crash appears in the log before I've accepted the autocomplete suggestion.

[rchiodo]

What's your version of bytewax. I don't see imports in the list of completions:

[whoahbot]

What's your version of bytewax. I don't see imports in the list of completions:

Sorry, that was a typo, it should be "inputs" as you have in your screenshot.

This is using bytewax 0.17.2.

[erictraut]

Thanks for the bug report.

This is due to a bug in the completion provider component. It was improperly allowing binary (native) library files to be treated as Python source files. An attempt was then made to read the contents of the binary and tokenize/parse it. In some cases, this could result in a crash. I've fixed the logic so it no longer attempts to treat these binaries as source files.

@rchiodo, I've posted a PR for your review. Apologies if you were already working on a fix. I just noticed that you had assigned this bug to yourself.

[rchiodo]

No worries, thanks for fixing it :)

[bschnurr]

This issue has been fixed in prerelease version 2023.11.12, which we've just released. You can find the changelog here: CHANGELOG.md