-
Notifications
You must be signed in to change notification settings - Fork 789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(perf): Lazy rewrite of schemapi.py
, improve docs
#3547
base: main
Are you sure you want to change the base?
Commits on Aug 18, 2024
-
perf: Replace
_use_referencing_library()
with a constantEvery call was identical as it was based on an existing constant `jsonschema_version_str`
Configuration menu - View commit details
-
Copy full SHA for d95f332 - Browse repository at this point
Copy the full SHA d95f332View commit details -
docs: Move
_get_errors_from_spec
comment into docstringI'm going to do this a lot. Docstrings can be collapsed in all editors and can benefit from markdown. Everything here is already private, so using long comments has no benefit
Configuration menu - View commit details
-
Copy full SHA for f4a4e0e - Browse repository at this point
Copy the full SHA f4a4e0eView commit details -
fix(typing): Resolve
jsonschema
incomplete stubs issue`typeshed` disagrees with `jsonschema`, this is just enforcing what `jsonschema` says is true
Configuration menu - View commit details
-
Copy full SHA for 6dfe61b - Browse repository at this point
Copy the full SHA 6dfe61bView commit details -
Configuration menu - View commit details
-
Copy full SHA for e9a4beb - Browse repository at this point
Copy the full SHA e9a4bebView commit details -
perf: Replace
_prepare_references_in_schema
Produces the same result, but skips the upfront `deepcopy`. No longer modifying the copy inplace, new objects are created inside the iterator.
Configuration menu - View commit details
-
Copy full SHA for c534876 - Browse repository at this point
Copy the full SHA c534876View commit details -
Configuration menu - View commit details
-
Copy full SHA for e30cac6 - Browse repository at this point
Copy the full SHA e30cac6View commit details -
Configuration menu - View commit details
-
Copy full SHA for e702b26 - Browse repository at this point
Copy the full SHA e702b26View commit details
Commits on Aug 19, 2024
-
Configuration menu - View commit details
-
Copy full SHA for af878d0 - Browse repository at this point
Copy the full SHA af878d0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ca44df - Browse repository at this point
Copy the full SHA 5ca44dfView commit details -
perf: Redefine
_json_path
to be bound onjsonschema
versionPreviously, using a version below `4.0.1` would still always check first if there was a property. This would not change between checks. Defining in this style removes the need for as much documentation, since the version guards are very clear when each branch is used.
Configuration menu - View commit details
-
Copy full SHA for de07012 - Browse repository at this point
Copy the full SHA de07012View commit details -
Configuration menu - View commit details
-
Copy full SHA for 05976a6 - Browse repository at this point
Copy the full SHA 05976a6View commit details -
perf(DRAFT): Initial lazy validation
First non-failing version. Have left most of the original code in. Planning to migrate & adapt the comments before removing. #
Configuration menu - View commit details
-
Copy full SHA for af6e41c - Browse repository at this point
Copy the full SHA af6e41cView commit details -
Configuration menu - View commit details
-
Copy full SHA for b24ac45 - Browse repository at this point
Copy the full SHA b24ac45View commit details
Commits on Aug 20, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c261cb4 - Browse repository at this point
Copy the full SHA c261cb4View commit details -
Configuration menu - View commit details
-
Copy full SHA for bbf6f14 - Browse repository at this point
Copy the full SHA bbf6f14View commit details -
ci: Add temporary script for
hatch run validation
**Remove before review**. Using for quicker feedback loop, where running mypy and all tests are not beneficial
Configuration menu - View commit details
-
Copy full SHA for 2d20db7 - Browse repository at this point
Copy the full SHA 2d20db7View commit details -
Configuration menu - View commit details
-
Copy full SHA for ad06d08 - Browse repository at this point
Copy the full SHA ad06d08View commit details -
refactor(perf): Tidying up, more lazy ops, renaming
Note to self: Fill out comments re individual changes.
Configuration menu - View commit details
-
Copy full SHA for af17d44 - Browse repository at this point
Copy the full SHA af17d44View commit details
Commits on Aug 21, 2024
-
test(perf): Adds
test_chart_validation_benchmark
Temporary, will remove before review. Tried to isolate to a single function so that I can reproduce on main
Configuration menu - View commit details
-
Copy full SHA for 5483db8 - Browse repository at this point
Copy the full SHA 5483db8View commit details -
Configuration menu - View commit details
-
Copy full SHA for e77b9d1 - Browse repository at this point
Copy the full SHA e77b9d1View commit details -
fix: Invert default for
_SKIP_SLOW_BENCHMARKS
I renamed this from `_SLOW_BENCHMARKS` but forgot to invert the bool lol
Configuration menu - View commit details
-
Copy full SHA for f208066 - Browse repository at this point
Copy the full SHA f208066View commit details -
Configuration menu - View commit details
-
Copy full SHA for 996ea97 - Browse repository at this point
Copy the full SHA 996ea97View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5751132 - Browse repository at this point
Copy the full SHA 5751132View commit details -
Configuration menu - View commit details
-
Copy full SHA for cb1fa24 - Browse repository at this point
Copy the full SHA cb1fa24View commit details -
Configuration menu - View commit details
-
Copy full SHA for 09c83ae - Browse repository at this point
Copy the full SHA 09c83aeView commit details -
refactor(perf): Conditionally define compatibility code
Rather than checking a function/constant, a single set of behaviour is defined **once** - depending on `jsonschema` version. This makes the remaining functionality much easier to reason with. Also easier to avoid typing issues
Configuration menu - View commit details
-
Copy full SHA for a0a8978 - Browse repository at this point
Copy the full SHA a0a8978View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2e81592 - Browse repository at this point
Copy the full SHA 2e81592View commit details
Commits on Aug 22, 2024
-
refactor: Use language that more closely aligns with json schema
Provided links to `json-schema` reference for more info. Also some minor edits to docs
Configuration menu - View commit details
-
Copy full SHA for 49aeec1 - Browse repository at this point
Copy the full SHA 49aeec1View commit details -
perf: Cache the result of
referencing.jsonschema.specification_with
Currently, this will only ever return a single result, based on https://json-schema.org/draft-07/json-schema-release-notes
Configuration menu - View commit details
-
Copy full SHA for 61bf448 - Browse repository at this point
Copy the full SHA 61bf448View commit details -
Configuration menu - View commit details
-
Copy full SHA for c665bb1 - Browse repository at this point
Copy the full SHA c665bb1View commit details -
fix(typing): Address
None
propagation, Remove outdated docIf a `None` reached `_get_schema_dialect_uri`, it would cause a runtime error. `None` does not have a `get` method. Confident this is only a theoretical issue, but is now fixed at the source.
Configuration menu - View commit details
-
Copy full SHA for 69588b3 - Browse repository at this point
Copy the full SHA 69588b3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7fceab8 - Browse repository at this point
Copy the full SHA 7fceab8View commit details -
Configuration menu - View commit details
-
Copy full SHA for e6912c1 - Browse repository at this point
Copy the full SHA e6912c1View commit details -
refactor: Rename
_get_referencing_registry
->_registry
, improve docThe original name is misleading, as this is a factory function. That is, a new `Registry` is created on each call. I think this could be impacting performance. Discarding the registry every time doesn't utilise the immutable properties provided by https://referencing.readthedocs.io/en/stable/api/#referencing.Registry
Configuration menu - View commit details
-
Copy full SHA for 283b69e - Browse repository at this point
Copy the full SHA 283b69eView commit details -
refactor: Renaming, docs to align with
jsonschema
Now very clear what is being wrapped, and where to find more information
Configuration menu - View commit details
-
Copy full SHA for 56be984 - Browse repository at this point
Copy the full SHA 56be984View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4c4b322 - Browse repository at this point
Copy the full SHA 4c4b322View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3f8f9bb - Browse repository at this point
Copy the full SHA 3f8f9bbView commit details -
Configuration menu - View commit details
-
Copy full SHA for a3a148e - Browse repository at this point
Copy the full SHA a3a148eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5fd6787 - Browse repository at this point
Copy the full SHA 5fd6787View commit details -
perf: Experiment with more cache layers
All of this is intended to avoid repeating any work that has already been performed.
Configuration menu - View commit details
-
Copy full SHA for 4cc1619 - Browse repository at this point
Copy the full SHA 4cc1619View commit details -
Configuration menu - View commit details
-
Copy full SHA for 509b678 - Browse repository at this point
Copy the full SHA 509b678View commit details
Commits on Aug 23, 2024
-
Fixes: ``` 2024-08-23 16:23:02.509 [info] [Info - 4:23:02 PM] (15104) Loading pyproject.toml file at c:\Users\danie\Documents\GitHub\altair\pyproject.toml 2024-08-23 16:23:02.511 [info] [Info - 4:23:02 PM] (15104) No include entries specified; assuming c:\Users\danie\Documents\GitHub\altair 2024-08-23 16:23:04.953 [info] [Info - 4:23:04 PM] (15104) Found 8234 source files 2024-08-23 16:23:50.675 [info] [Warn - 4:23:50 PM] (15104) Workspace indexing has hit its upper limit: 5000 files ```
Configuration menu - View commit details
-
Copy full SHA for c594e55 - Browse repository at this point
Copy the full SHA c594e55View commit details
Commits on Aug 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5334a2c - Browse repository at this point
Copy the full SHA 5334a2cView commit details
Commits on Aug 25, 2024
-
fix: Use uniform docs in version-gated functions
Both versions of these functions can be targeted by an IDE. But hovering over the name (anywhere but the actual definition) displays *only* the final docstring on hover. This duplicates the docs, and adds a **NOTE** comment to be super clear when viewing the code itself.
Configuration menu - View commit details
-
Copy full SHA for 2751bb2 - Browse repository at this point
Copy the full SHA 2751bb2View commit details -
Configuration menu - View commit details
-
Copy full SHA for f235284 - Browse repository at this point
Copy the full SHA f235284View commit details -
Configuration menu - View commit details
-
Copy full SHA for bd1d580 - Browse repository at this point
Copy the full SHA bd1d580View commit details -
perf: Refactor
SchemaBase.from_dict
and coThere's quite a lot in here, so I've left my notes in `_subclasses` temporarily. - Removed unused `use_json=False` branch - Evaluate the hash table **once** and not every time `SchemaBase.from_dict` is called
Configuration menu - View commit details
-
Copy full SHA for 8ca4266 - Browse repository at this point
Copy the full SHA 8ca4266View commit details -
perf: Avoid expensive exceptions in
_FromDict.from_dict
Try running `hatch test --all` on main vs this. Locally, each version gets a 1.32-1.53x speedup. Somehow this also includes `3.11`, despite that being the version "zero-cost" exceptions were introduced.
Configuration menu - View commit details
-
Copy full SHA for 933c045 - Browse repository at this point
Copy the full SHA 933c045View commit details -
chore: Add todo for
__init_subclasses__
Thinking that moving the checks the the definition of the class (rather than the instance) could reduce a lot of calls.
Configuration menu - View commit details
-
Copy full SHA for 1e5993b - Browse repository at this point
Copy the full SHA 1e5993bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 67b7ae4 - Browse repository at this point
Copy the full SHA 67b7ae4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8725a6c - Browse repository at this point
Copy the full SHA 8725a6cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2ea0eb0 - Browse repository at this point
Copy the full SHA 2ea0eb0View commit details -
perf(ruff): Add & ignore some performance rules
Ignoring these *first* for visibility, since I'll be making changes that otherwise have no context
Configuration menu - View commit details
-
Copy full SHA for 6770177 - Browse repository at this point
Copy the full SHA 6770177View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0ddf19e - Browse repository at this point
Copy the full SHA 0ddf19eView commit details -
perf: Remove unreachable
ValidationError
exceptThis code path will not raise `jsonschema.ValidationError`
Configuration menu - View commit details
-
Copy full SHA for cace782 - Browse repository at this point
Copy the full SHA cace782View commit details -
Configuration menu - View commit details
-
Copy full SHA for ac99939 - Browse repository at this point
Copy the full SHA ac99939View commit details -
Configuration menu - View commit details
-
Copy full SHA for f776fcf - Browse repository at this point
Copy the full SHA f776fcfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 56c573f - Browse repository at this point
Copy the full SHA 56c573fView commit details
Commits on Aug 26, 2024
-
perf: Avoid an exception in
SchemaBase.__getattr__
Uses the default parameter of `getattr` instead
Configuration menu - View commit details
-
Copy full SHA for 53663e8 - Browse repository at this point
Copy the full SHA 53663e8View commit details -
refactor: Ensure every
VegaLiteSchema
has a._schema
Eventually, this will replace the need for checking `None` as frequently
Configuration menu - View commit details
-
Copy full SHA for bd31d7c - Browse repository at this point
Copy the full SHA bd31d7cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 06ca339 - Browse repository at this point
Copy the full SHA 06ca339View commit details -
test: Adds a mini suite for
_SchemaBasePEP487
Purely to demonstrate the differences to status quo. There would be no loss in functionality, this would simply be some small changes to generated code and the removal of instance-level checks
Configuration menu - View commit details
-
Copy full SHA for 56a43cb - Browse repository at this point
Copy the full SHA 56a43cbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6033d40 - Browse repository at this point
Copy the full SHA 6033d40View commit details -
Configuration menu - View commit details
-
Copy full SHA for 003c7fb - Browse repository at this point
Copy the full SHA 003c7fbView commit details
Commits on Aug 27, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 34e647e - Browse repository at this point
Copy the full SHA 34e647eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5fad088 - Browse repository at this point
Copy the full SHA 5fad088View commit details
Commits on Aug 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 1a93d0d - Browse repository at this point
Copy the full SHA 1a93d0dView commit details -
test: Add
SchemaBase.__init_subclass__
benchmark resultsLocally, did a full replacement of class hierarchy - but saw no observable performance improvement
Configuration menu - View commit details
-
Copy full SHA for 69d9f67 - Browse repository at this point
Copy the full SHA 69d9f67View commit details
Commits on Aug 29, 2024
-
refactor(typing): Widen internal
rootschema|schema
types from `dict……[str, Any]` No functional or user-facing change. This will be to support using types other than `dict`, which isn't strictly required here
Configuration menu - View commit details
-
Copy full SHA for 307cb9b - Browse repository at this point
Copy the full SHA 307cb9bView commit details -
perf: Adds
resolve_references_rpds
See docstring for notes and plan
Configuration menu - View commit details
-
Copy full SHA for 8fa8975 - Browse repository at this point
Copy the full SHA 8fa8975View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5e93933 - Browse repository at this point
Copy the full SHA 5e93933View commit details -
Configuration menu - View commit details
-
Copy full SHA for c322e79 - Browse repository at this point
Copy the full SHA c322e79View commit details -
Configuration menu - View commit details
-
Copy full SHA for 96eed9b - Browse repository at this point
Copy the full SHA 96eed9bView commit details -
perf: Remove unreachable
"oneOf"
keyword checkSince `v2` this keyword has not been part of any schema. It has only been a property of what became `FieldOneOfPredicate` https://github.com/vega/schema/blob/ef61166f3f95154465c4b3ebdca88e3c2d25b005/vega-lite/v2.0.0.json#L4060
Configuration menu - View commit details
-
Copy full SHA for 3258f56 - Browse repository at this point
Copy the full SHA 3258f56View commit details -
Configuration menu - View commit details
-
Copy full SHA for d6ce749 - Browse repository at this point
Copy the full SHA d6ce749View commit details
Commits on Aug 30, 2024
-
perf: Optimize
dict
branch in_FromDict.from_dict
- Bypass `dct.items()` when there are no properties -Reuse result of `props` lookup, rather than twice per hit
Configuration menu - View commit details
-
Copy full SHA for 89fcaf7 - Browse repository at this point
Copy the full SHA 89fcaf7View commit details -
Configuration menu - View commit details
-
Copy full SHA for c9f9d8a - Browse repository at this point
Copy the full SHA c9f9d8aView commit details -
Configuration menu - View commit details
-
Copy full SHA for e9c8f85 - Browse repository at this point
Copy the full SHA e9c8f85View commit details -
test: Add note on possibly outdated tests
If the current schema doesn't trigger branches, but the test schema(s) do - it may indicate the tests need to be updated to better reflect actual
Configuration menu - View commit details
-
Copy full SHA for 759a556 - Browse repository at this point
Copy the full SHA 759a556View commit details -
Configuration menu - View commit details
-
Copy full SHA for dace488 - Browse repository at this point
Copy the full SHA dace488View commit details -
Configuration menu - View commit details
-
Copy full SHA for 531aa7c - Browse repository at this point
Copy the full SHA 531aa7cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3277b43 - Browse repository at this point
Copy the full SHA 3277b43View commit details -
Configuration menu - View commit details
-
Copy full SHA for 96dde18 - Browse repository at this point
Copy the full SHA 96dde18View commit details -
Configuration menu - View commit details
-
Copy full SHA for ab20005 - Browse repository at this point
Copy the full SHA ab20005View commit details
Commits on Aug 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5bceda9 - Browse repository at this point
Copy the full SHA 5bceda9View commit details -
Configuration menu - View commit details
-
Copy full SHA for a419ede - Browse repository at this point
Copy the full SHA a419edeView commit details
Commits on Sep 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4a7cc4e - Browse repository at this point
Copy the full SHA 4a7cc4eView commit details -
chore: Remove
# FIXME
(s) that won't be addressedMaybe revisit lazy imports in the future, but for now this is too unrelated to the PR
Configuration menu - View commit details
-
Copy full SHA for 9bc3ccd - Browse repository at this point
Copy the full SHA 9bc3ccdView commit details
Commits on Sep 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for af480f5 - Browse repository at this point
Copy the full SHA af480f5View commit details -
Configuration menu - View commit details
-
Copy full SHA for dff817b - Browse repository at this point
Copy the full SHA dff817bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 133aa01 - Browse repository at this point
Copy the full SHA 133aa01View commit details -
refactor: Collapse
...
following:=
Not sure why I wrote this with `...` originally.
Configuration menu - View commit details
-
Copy full SHA for 141c8d1 - Browse repository at this point
Copy the full SHA 141c8d1View commit details -
Configuration menu - View commit details
-
Copy full SHA for fba3c46 - Browse repository at this point
Copy the full SHA fba3c46View commit details -
Configuration menu - View commit details
-
Copy full SHA for 44f5227 - Browse repository at this point
Copy the full SHA 44f5227View commit details -
refactor: Remove unused
resolve_references_rpds
Wasn't able to demonstrate a performance improvement
Configuration menu - View commit details
-
Copy full SHA for 5d9fb65 - Browse repository at this point
Copy the full SHA 5d9fb65View commit details -
Configuration menu - View commit details
-
Copy full SHA for a26b3f1 - Browse repository at this point
Copy the full SHA a26b3f1View commit details
Commits on Sep 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 1858bf5 - Browse repository at this point
Copy the full SHA 1858bf5View commit details -
chore: Add
_is_valid
toschemapi.__all__
Used in `Chart.from_dict`
Configuration menu - View commit details
-
Copy full SHA for 82a8106 - Browse repository at this point
Copy the full SHA 82a8106View commit details -
All were notes added earlier in PR, but not needed now
Configuration menu - View commit details
-
Copy full SHA for af783b2 - Browse repository at this point
Copy the full SHA af783b2View commit details -
refactor: Remove
_SchemaBasePEP487
& test suiteWIll put this on another branch, won't improve performance so not relevant here vega#3547 (comment)
Configuration menu - View commit details
-
Copy full SHA for 9a48448 - Browse repository at this point
Copy the full SHA 9a48448View commit details -
Configuration menu - View commit details
-
Copy full SHA for 67642f0 - Browse repository at this point
Copy the full SHA 67642f0View commit details -
test: Remove
test_chart_validation_benchmark
Feeling I've squeezed out all the performance I can for now. Will add in a collpased comment on the PR for reference
Configuration menu - View commit details
-
Copy full SHA for 8002dab - Browse repository at this point
Copy the full SHA 8002dabView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2feeeb7 - Browse repository at this point
Copy the full SHA 2feeeb7View commit details
Commits on Sep 4, 2024
-
Configuration menu - View commit details
-
Copy full SHA for ac45244 - Browse repository at this point
Copy the full SHA ac45244View commit details
Commits on Sep 6, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0838fef - Browse repository at this point
Copy the full SHA 0838fefView commit details -
Configuration menu - View commit details
-
Copy full SHA for 20fbdc4 - Browse repository at this point
Copy the full SHA 20fbdc4View commit details
Commits on Sep 9, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 1fdb791 - Browse repository at this point
Copy the full SHA 1fdb791View commit details
Commits on Sep 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2dd4c42 - Browse repository at this point
Copy the full SHA 2dd4c42View commit details -
Configuration menu - View commit details
-
Copy full SHA for 842da01 - Browse repository at this point
Copy the full SHA 842da01View commit details
Commits on Sep 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 75b35b4 - Browse repository at this point
Copy the full SHA 75b35b4View commit details
Commits on Sep 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0c829fb - Browse repository at this point
Copy the full SHA 0c829fbView commit details