Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: Incomplete Support for Union & NotRequired #121

Open
adamcunnington-mlg opened this issue Jul 16, 2024 · 2 comments
Open

[BUG]: Incomplete Support for Union & NotRequired #121

adamcunnington-mlg opened this issue Jul 16, 2024 · 2 comments

Comments

@adamcunnington-mlg
Copy link

adamcunnington-mlg commented Jul 16, 2024

Issues

There are 2 issues I've stumbled across when combining some semi-complex types:

  1. Low-impact as can be workaround by Using Union[...] or Optional: You can't use X | Y syntax < python3.10 with from __future__ import annotations if the types are not simple. In my case, it was when I was using a TypedDict(...) | None. I got crazy syntax error with incomplete terminations. E.g.:
meta: TypedDict("Meta", {"superset": NotRequired[ColumnMetaSuperset]}, total=False) | None = None
SyntaxError: TypedDict('Meta',{'superset':NotRequired['ColumnMetaSuperset']},Union[total=False),None]

Root cause suggestion from community member who helped me debug was:

My guess is that third party library created an override for type and not _TypedDictMeta
(which allows types to be or'ed in its annotations, but only if they do not have a metaclass)
Probably the same issue as https://github.com/rnag/dataclass-wizard/issues/118 , since StrEnum also uses a metaclass IIRC
  1. High-impact as there is no workaround: It looks like NotRequired can't be used in conjunction with Union/Optional
meta: Union[TypedDict("Meta", {"superset": NotRequired[ColumnMetaSuperset]}, total=False), None] = None
TypeError: issubclass() arg 1 must be a class

Again, suggestion from community member:
"I think it has to due with the Union[..., None]; the Optional parser assumes the first argument does not need a custom parser"

@adamcunnington-mlg
Copy link
Author

@rnag here's the traceback to help investigation of point 2. I'm trying to understand what fix is needed but this get_parser_for_annotation function is complex.

You can see from the traceback below that I am printing base type (I've inserted print(base_type) on L341 of loaders.py). It's the specific flow of an optional/union followed by a NotRequired (which is used within TypedDict). You handle NotRequired outside of optional context but not within. I'd be really grateful for a fix/workaround here.

typing.Union
typing_extensions.NotRequired
Traceback (most recent call last):
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 534, in fromdict
    load = _CLASS_TO_LOAD_FUNC[cls]
KeyError: <class '__main__.Common'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 273, in get_parser_for_annotation
    base_type = get_origin(ann_type, raise_=True)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/utils/typing_compat.py", line 345, in get_origin
    return _get_origin(cls, raise_=raise_)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/utils/typing_compat.py", line 177, in _get_origin
    return cls.__origin__
AttributeError: type object 'Metric' has no attribute '__origin__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 273, in get_parser_for_annotation
    base_type = get_origin(ann_type, raise_=True)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/utils/typing_compat.py", line 345, in get_origin
    return _get_origin(cls, raise_=raise_)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/utils/typing_compat.py", line 177, in _get_origin
    return cls.__origin__
AttributeError: type object 'TypeParams' has no attribute '__origin__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ac/projects/medialabgroup/misc/scripts/mart-linter.py", line 810, in <module>
    main()
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ac/projects/medialabgroup/misc/scripts/mart-linter.py", line 799, in main
    LinterWorkflow(bq_client, bq_dataset_name, prompter=questionary.prompt).run(mart_yaml_file_paths, fix)
  File "/home/ac/projects/medialabgroup/misc/scripts/mart-linter.py", line 747, in run
    self.common = Common.from_yaml_file(self.COMMON_FILE_PATH)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/wizard_mixins.py", line 147, in from_yaml_file
    return cls.from_yaml(in_file, decoder=decoder,
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/wizard_mixins.py", line 136, in from_yaml
    return fromdict(cls, o) if isinstance(o, dict) else fromlist(cls, o)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 536, in fromdict
    load = load_func_for_dataclass(cls)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 583, in load_func_for_dataclass
    field_to_parser = dataclass_field_to_load_parser(cls_loader, cls, config)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/class_helper.py", line 120, in dataclass_field_to_load_parser
    return _setup_load_config_for_cls(cls_loader, cls, config, save)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/class_helper.py", line 189, in _setup_load_config_for_cls
    name_to_parser[f.name] = cls_loader.get_parser_for_annotation(
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 373, in get_parser_for_annotation
    return MappingParser(
  File "<string>", line 5, in __init__
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/parsers.py", line 502, in __post_init__
    self.val_parser = get_parser(val_type, cls, extras)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 287, in get_parser_for_annotation
    load_hook = load_func_for_dataclass(
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 583, in load_func_for_dataclass
    field_to_parser = dataclass_field_to_load_parser(cls_loader, cls, config)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/class_helper.py", line 120, in dataclass_field_to_load_parser
    return _setup_load_config_for_cls(cls_loader, cls, config, save)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/class_helper.py", line 189, in _setup_load_config_for_cls
    name_to_parser[f.name] = cls_loader.get_parser_for_annotation(
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 353, in get_parser_for_annotation
    return OptionalParser(
  File "<string>", line 4, in __init__
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/parsers.py", line 156, in __post_init__
    self.parser: AbstractParser = get_parser(self.base_type, cls, extras)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 319, in get_parser_for_annotation
    return TypedDictParser(
  File "<string>", line 5, in __init__
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/parsers.py", line 545, in __post_init__
    self.key_to_parser: FieldToParser = {
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/parsers.py", line 546, in <dictcomp>
    k: get_parser(v, cls, extras)
  File "/home/ac/projects/medialabgroup/misc/scripts/.venv/lib/python3.9/site-packages/dataclass_wizard/loaders.py", line 364, in get_parser_for_annotation
    elif issubclass(base_type, defaultdict):
TypeError: issubclass() arg 1 must be a class

@adamcunnington-mlg
Copy link
Author

@rnag are you able to take a look at this? is this project still maintained?

claui added a commit to claui/dataclass-wizard that referenced this issue Aug 18, 2024
Previously, annotating a `TypedDict` field with one of the `Required`
and `NotRequired` wrappers introduced in Python 3.11, dataclass-wizard
would raise the following error:

> TypeError: issubclass() arg 1 must be a class

Fix that error by adding support for `Required` and `NotRequired`.

Partially addresses issue rnag#121. [1]

[1]: rnag#121
claui added a commit to claui/dataclass-wizard that referenced this issue Aug 18, 2024
Previously, annotating a `TypedDict` field with one of the `Required`
and `NotRequired` wrappers introduced in Python 3.11, dataclass-wizard
would raise the following error:

> TypeError: issubclass() arg 1 must be a class

Fix that error by adding support for `Required` and `NotRequired`.

Partially addresses issue rnag#121. [1]

[1]: rnag#121
claui added a commit to dataclass-cauldron/dataclass-witch that referenced this issue Aug 18, 2024
Previously, annotating a `TypedDict` field with one of the `Required`
and `NotRequired` wrappers introduced in Python 3.11, dataclass-wizard
would raise the following error:

> TypeError: issubclass() arg 1 must be a class

Fix that error by adding support for `Required` and `NotRequired`.

Partially addresses issue rnag#121. [1]

[1]: rnag#121
claui added a commit to dataclass-cauldron/dataclass-witch that referenced this issue Aug 18, 2024
Previously, annotating a `TypedDict` field with one of the `Required`
and `NotRequired` wrappers introduced in Python 3.11, the library would
raise the following error:

> TypeError: issubclass() arg 1 must be a class

Fix that error by adding support for `Required` and `NotRequired`.

Partially addresses issue rnag#121. [1]

[1]: rnag#121
claui added a commit to dataclass-cauldron/dataclass-witch that referenced this issue Aug 18, 2024
Previously, annotating a `TypedDict` field with one of the `Required`
and `NotRequired` wrappers introduced in Python 3.11, the library would
raise the following error:

> TypeError: issubclass() arg 1 must be a class

Fix that error by adding support for `Required` and `NotRequired`.

Partially addresses issue rnag#121. [1]

[1]: rnag#121
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant