error squiggle when using %pip in interactive window

[amunger]

enter %pip install torch into the IW input box

[greazer]

We think this may be due to pylance not realizing that the IW input box is part of a "notebook". The "pylance lsp notebooks" setting was on.

[debonte]

When LSP notebooks are enabled, the _isNotebookCell check in NotebookHandler.handleOpen returns false and therefore we set ipythonMode to false for the interactive window document.

Should we be getting a notebookDocument/didOpen instead of a textDocument/didOpen?

[rchiodo]

No _isNotebookCell is actually wrong or should be updated to also include the interactive cell scheme.

[debonte]

lsp-notebook-concat's isNotebookCell implementation already returns true for the interactive window, which is why this works correctly when the LSP notebooks experiment is disabled. However, when the experiment is enabled we don't call that function. Instead, we assume that only notebook documents (i.e. notebookDocument/didOpen) are notebooks.

Maybe for textDocument/didOpen we should call lsp-notebook-concat's isNotebookCell to determine the right value for the parser's ipythonMode boolean.

[debonte]

@rchiodo, are you planning to have the interactive window be treated as a notebook document from LSP's perspective at some point in the future?

I could fix the %magic issue, but that's not the only problem when the LSP notebooks experiment is enabled. Since the interactive window cells are passed as individual text documents, Pylance doesn't chain them together as we do for notebook cells. And therefore IntelliSense, hover, semantic coloring, go to def, etc don't work when working with symbols defined in earlier cells.

[rchiodo]

Sounds like a bug in the LSP client if it isn't already behaving this way.

With the old middleware solution, the interactive window was treated just like a notebook. I concatenated all of the cells (including the special one at the bottom) into the concatenated document.

This would need to happen in the new way too.

[debonte]

Ok, I'll take a look at the LSP code.

Earlier I asked:

Should we be getting a notebookDocument/didOpen instead of a textDocument/didOpen?

And your reply was "No..." so I thought you were saying this was expected at the moment.

[rchiodo]

I would expect that the interactive window will have to be treated special in the language client. It's internally a notebook and separate text document, so unless Dirk pieced them together, the language client probably doesn't know about it.

[debonte]

so unless Dirk pieced them together, the language client probably doesn't know about it

Is the interactive window a Python-specific feature? I'm wondering if it is generic enough that Dirk would be willing to handle this in vscode-languageclient or if Pylance should special-case it.

For IW cells (not the input box), part of the problem in my screenshot above was that Pylance's notebookSelector only requested notifications about Jupyter notebooks. Fixing that got hover and semantic highlighting working, but go to definition is still broken (opens the target cell in a new tab).

[dbaeumer]

The notebook support in LSP syncs notebooks and not individual cells. Piecing this together might be complicated for me since currently a cell text document is referenced by notebook cell. when syncing things.

It sounds like that the interactive windows is visually part of a notebook. Why can't it be a notebook cell as well with a special language id. The you could simply use a special notebook selector (https://github.com/microsoft/vscode-languageserver-node/blob/12900cfb006ce56360e3f64feaeff9c0224854a1/protocol/src/common/protocol.notebook.ts#L312) and everything should work.

[rchiodo]

Why can't it be a notebook cell as well with a special language id.

It has a specific scheme associated with it. So I believe if the selector was updated to include the interactive input scheme it might include that document as part of a notebook.

Here's the two schemes in use for 'cells' in an interactive window:

export const InteractiveInputScheme = 'vscode-interactive-input';
export const InteractiveScheme = 'vscode-interactive';

The interactive window isn't tied to a specific language (we've just only used it for python so for).

[debonte]

It's not currently included because cells are expected to have the vscode-notebook-cell scheme. The LSP notebooks feature supports notebook documents with custom schemes, but not cells with custom schemes.

Here's the two schemes in use for 'cells' in an interactive window:

export const InteractiveInputScheme = 'vscode-interactive-input';
export const InteractiveScheme = 'vscode-interactive';

To clarify, vscode-interactive is the scheme of the notebook document. The non-input cells use vscode-notebook-cell.

[debonte]

@dbaeumer, here's a picture of the Interactive Window in case you're not familiar with it. When the user clicks the triangular "Execute Code" button to the left of the input box, the text in the input box is removed and placed in a new cell at the bottom of the notebook.

[dbaeumer]

LSP has no knowledge about the scheme that the editor uses to sync notebooks. Even the VS Code implementations has only little knowledge and mostly only for performance reasons (to not generate unnecessary events). This was done that way since IMO a client shouldn't make any assumption about the cell schema. VS Code should have the freedom to change it. And a server should never ever know which schema an editor uses to identify a notebook cell.

The way how LSP communicates with VS Code is basically by using the extended document selector syntax specified via the VS Code API (https://github.com/microsoft/vscode/blob/5bb16df4b0dea028da65676ef28045e70d151157/src/vscode-dts/vscode.d.ts#L2166). To make the interactive Window work I recommend that you use a standard cell and use a special language mode for the interactive cell. Then every code path in VS Code and LSP client / server will work as expected.

May I ask how you get that cell with a different schema into the editor in the first place. I was always under the assumption that VS Code manages all these cells?

[rchiodo]

May I ask how you get that cell with a different schema into the editor in the first place. I was always under the assumption that VS Code manages all these cells?

VS code does manage the entire interactive window. None of this is created in the extension. It's the one that knows about the schema. We can't change the language as we don't create it.

[dbaeumer]

OK. Now I see. Then I think either VS Code should use a consistent schema or the schema needs to be handles correctly in the document filter code.

[debonte]

We're still discussing how to properly fix this, but a temporary workaround will be available in our next release.

[debonte]

A more correct fix has been checked in and in coming days can be tested with:

• Jupyter 2022.7.1002051049 or later
• Python <tonight's build> or later
• Pylance 2022.7.51 or later (should be available Wednesday 7/27)

[heejaechang]

This issue has been fixed in version 2022.7.42, which we've just released. You can find the changelog here: CHANGELOG.md