Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect inferred type of returned value of Callable that has a generic Optional return type #1462

Closed
ZappD0S opened this issue Jun 17, 2021 · 6 comments
Labels
bug Something isn't working fixed in next version (main) A fix has been implemented and will appear in an upcoming version

Comments

@ZappD0S
Copy link

ZappD0S commented Jun 17, 2021

Environment data

  • Language Server version: 2021.6.2 (pyright dafa497d)
  • OS and version: Linux x64
  • Python version (& distribution if applicable, e.g. Anaconda): Python 3.9.4 Anaconda

Expected behaviour

The infered type of y shoud be U | None.

Actual behaviour

The inferred type of the local variable y in the function listed below is None. Because of this, whithin the if y is not None: the inferred type becomes Never and if I return y PyLance marks it as an error.

Logs

[Info  - 10:03:52 AM] Pylance language server 2021.6.2 (pyright dafa497d) starting
[Info  - 10:03:52 AM] Server root directory: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist
[Info  - 10:03:52 AM] No configuration file found.
[Info  - 10:03:52 AM] No pyproject.toml file found.
[Info  - 10:03:52 AM] Setting pythonPath for service "playground": "/home/zapp/.conda/envs/playground/bin/python"
[Warn  - 10:03:52 AM] stubPath /home/zapp/source/repos/playground/typings is not a valid directory.
[Info  - 10:03:52 AM] Assuming Python version 3.9
[Info  - 10:03:52 AM] Assuming Python platform Linux
Search paths for /home/zapp/source/repos/playground
  /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib
  /home/zapp/source/repos/playground
  /home/zapp/source/repos/playground/typings
  /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stubs/...
  /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/bundled/stubs
  /home/zapp/.conda/envs/playground/lib/python3.9
  /home/zapp/.conda/envs/playground/lib/python3.9/lib-dynload
  /home/zapp/.local/lib/python3.9/site-packages
  /home/zapp/.conda/envs/playground/lib/python3.9/site-packages
[Warn  - 10:03:53 AM] Exception received when installing recursive file system watcher: TypeError [ERR_FEATURE_UNAVAILABLE_ON_PLATFORM]: The feature watch recursively is unavailable on the current platform, which is being used to run Node.js
[Warn  - 10:03:53 AM] Exception received when installing recursive file system watcher: TypeError [ERR_FEATURE_UNAVAILABLE_ON_PLATFORM]: The feature watch recursively is unavailable on the current platform, which is being used to run Node.js
[Warn  - 10:03:53 AM] Exception received when installing recursive file system watcher: TypeError [ERR_FEATURE_UNAVAILABLE_ON_PLATFORM]: The feature watch recursively is unavailable on the current platform, which is being used to run Node.js
[Warn  - 10:03:53 AM] Exception received when installing recursive file system watcher: TypeError [ERR_FEATURE_UNAVAILABLE_ON_PLATFORM]: The feature watch recursively is unavailable on the current platform, which is being used to run Node.js
[Info  - 10:03:53 AM] Searching for source files
[Info  - 10:03:53 AM] Found 53 source files
[Info  - 10:03:53 AM] Background analysis(1) root directory: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist
[Info  - 10:03:53 AM] Background analysis(1) started
Background analysis message: setConfigOptions
Background analysis message: ensurePartialStubPackages
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: setFileOpened
Background analysis message: analyze
[BG(1)] analyzing: /home/zapp/source/repos/playground/tests/bug6.py ...
[BG(1)]   parsing: /home/zapp/source/repos/playground/tests/bug6.py (19ms)
[BG(1)]   parsing: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/builtins.pyi [fs read 2ms] (123ms)
[BG(1)]   binding: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/builtins.pyi (46ms)
[BG(1)]   binding: /home/zapp/source/repos/playground/tests/bug6.py (0ms)
[BG(1)]   checking: /home/zapp/source/repos/playground/tests/bug6.py ...
[BG(1)]     parsing: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/typing.pyi [fs read 2ms] (34ms)
[BG(1)]     binding: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/typing.pyi (18ms)
[BG(1)]     parsing: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi [fs read 1ms] (6ms)
[BG(1)]     binding: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi (3ms)
[BG(1)]     parsing: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/typing_extensions.pyi [fs read 0ms] (2ms)
[BG(1)]     binding: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/typing_extensions.pyi (1ms)
[BG(1)]     parsing: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/collections/abc.pyi [fs read 0ms] (2ms)
[BG(1)]     binding: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/collections/abc.pyi ...
[BG(1)]       parsing: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/_collections_abc.pyi [fs read 0ms] (0ms)
[BG(1)]       binding: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/_collections_abc.pyi (1ms)
[BG(1)]     binding: /home/zapp/.vscode/extensions/ms-python.vscode-pylance-2021.6.2/dist/typeshed-fallback/stdlib/collections/abc.pyi (2ms)
[BG(1)]   checking: /home/zapp/source/repos/playground/tests/bug6.py (121ms)
[BG(1)] analyzing: /home/zapp/source/repos/playground/tests/bug6.py (317ms)
Background analysis message: resumeAnalysis

Code Snippet / Additional information

from typing import Optional, TypeVar
from collections.abc import Callable


T = TypeVar("T")
U = TypeVar("U")


def g(f: Callable[[T], Optional[U]], x: T) -> U:
    y = f(x)

    if y is not None:
        return y

    raise ValueError()
@erictraut
Copy link
Contributor

Thanks for the bug report.

There is logic in the type evaluator that handles a union return type that includes both concrete types and one or more generic types that are not "solved" when the call expression is evaluated. This is important in cases like this:

def func(a: List[int | T]) -> int | T:
    ...

reveal_type(func(["hi"])) # int | str

# In this case, "T" remains unsolved, and we don't want the resulting
# type to be `int | T` because `T` has no meaning outside of the scope
# of the callee. For that reason, the `T` is dropped from the resulting type.
reveal_type(func([3])) # int

This logic resulted in the type of f(x) to be evaluated as just None because U was unsolved and was therefore removed from the type.

However, in this example, U is still in scope (because the code is within the body of g) and therefore still has meaning in this context. I've updated the logic to remove the unsolved type variable only in cases where the call expression is out of scope.

This will be fixed in the next release.

@erictraut erictraut added bug Something isn't working fixed in next version (main) A fix has been implemented and will appear in an upcoming version and removed triage labels Jun 18, 2021
@bschnurr
Copy link
Member

This issue has been fixed in version 2021.6.3, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/main/CHANGELOG.md#202163-23-june-2021

@ZappD0S
Copy link
Author

ZappD0S commented Jun 24, 2021

I'm afraid the bug is still not solved completely. The code that originally produced the bug is a little more complex than the sample I provided here. This other sample produces the same effect:

from typing import Optional, TypeVar
from collections.abc import Callable


T = TypeVar("T")
U = TypeVar("U")


def h(x: T, f: Callable[[T], Optional[U]]) -> U:
    def g() -> U:
        y = f(x)
        if y is not None:
            return y

        raise ValueError()

    return g()

@erictraut
Copy link
Contributor

Yep, my previous fix didn't account for nested scopes. I've updated it to handle that case as well.

@ZappD0S
Copy link
Author

ZappD0S commented Jun 24, 2021

Many thanks for your rapidity! Also, I have encountered another problem that might be related to this bug but I'm not totally sure. I was waiting for today's release to see if it would fix it but apparently it didn't. Let me know if you think I should open another issue. The snippet is this:

from typing import (
    Any,
    Callable,
    Optional,
    TypeVar,
    Union,
)


T = TypeVar("T")
U = TypeVar("U")


def g(f: Callable[[Any], Optional[T]]) -> T:
    ...


def h(t1: type[T], t2: type[U]) -> Union[T, U]:
    def f(x: Any) -> Union[T, U]:
        if isinstance(x, (t1, t2)):
            return x

        raise ValueError()

    x = g(f)
    return x

The type of x at the line x = g(f) is object but I would expect T | U. I'm saiyng this is related because if you replace
def g(f: Callable[[Any], Optional[T]]) -> T: with def g(f: Callable[[Any], T]) -> T: the inferred type of x is correct.

@erictraut
Copy link
Contributor

Please file a new issue in the pyright repo, and I'll take a look at it. On first inspection, mypy also evaluates the type as object in this case and returns the same error, but I can look into it in more detail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working fixed in next version (main) A fix has been implemented and will appear in an upcoming version
Projects
None yet
Development

No branches or pull requests

3 participants