Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix extractor misses local functions #3954

Merged
merged 20 commits into from
Jan 26, 2022

Conversation

liqunfu
Copy link
Collaborator

@liqunfu liqunfu commented Jan 19, 2022

Description

  • Fix issue 3938 extract a model with local functions
  • Add function test cases

Motivation and Context

  • bug fix
  • improve test coverage

@liqunfu liqunfu requested a review from a team as a code owner January 19, 2022 21:18
@liqunfu liqunfu changed the title (WIP) to repro issue 3938 extract model w/ local func To repro issue 3938 extract model w/ local func Jan 20, 2022
onnx/utils.py Outdated Show resolved Hide resolved
onnx/utils.py Outdated Show resolved Hide resolved
@liqunfu liqunfu changed the title To repro issue 3938 extract model w/ local func Bugfix extractor misses local functions Jan 24, 2022
liqunfu and others added 11 commits January 24, 2022 12:26
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
#3902)

* Fix typos (#3894)

Signed-off-by: Lewis Tunstall <lewis.c.tunstall@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Honor existing dim_param in shape inference (#3896)

* Honor existing dim_param

Before this change, existing dim_param was overwritten by generated
symbols like `unk__`. `NonZero` is used to test the behavior.

Signed-off-by: Shinichiro Hamaji <shinichiro.hamaji@gmail.com>

* Fix test expectation of symbolic_shape_test.py

Signed-off-by: Shinichiro Hamaji <shinichiro.hamaji@gmail.com>

* Keep the original check as suggested in review

Signed-off-by: Shinichiro Hamaji <shinichiro.hamaji@gmail.com>

* Simplify the logic of mergeShapesAndTypes

Signed-off-by: Shinichiro Hamaji <shinichiro.hamaji@gmail.com>

Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* for issue 3849 to confirm that type check is performed during checker.check_model call

Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* fix path to InferenceError

Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* replace whitelist by safelist (#3900)

Signed-off-by: Alexandre Eichenberger <alexe@us.ibm.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* add checker test case for input type mismatch conflict

Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* remove old numpy in ipynb (#3916)

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Shapeinf for functions (#3722)

* add shape inference for model local functions

Signed-off-by: Ashwini Khade <askhade@microsoft.com>

* Plus tests and more changes

Signed-off-by: Ashwini Khade <askhade@microsoft.com>

* fix typo

Signed-off-by: Ashwini Khade <askhade@microsoft.com>

* Plus updates

Signed-off-by: Ashwini Khade <askhade@microsoft.com>

* Plus updates per review

Signed-off-by: Ashwini Khade <askhade@microsoft.com>

* plus updates

Signed-off-by: Ashwini Khade <askhade@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Fix old ConvTranspose shape inference and softmax upgrader (#3893)

* Fixed ConvTranspose-1 shape inference

Brings change from #3188 (for ConvTranspose-11) into old shape inference
Signed-off-by: Matteo Salvarezza <matteo.salvarezza@gmail.com>

* Fix softmax adapter

Signed-off-by: Matteo Salvarezza <matteo.salvarezza@gmail.com>

Co-authored-by: G. Ramalingam <grama@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Fix Linux i686 Release CI failure due to the latest NumPy (#3918)

* 3.8 and 3.9 use numpy 1.21.5

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* 1.21.4

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* 6be8011a073feeca08d256ac64335a19fc8eee4c6312668b6781ace71db0de20

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* 2021-12-19-a4d7f5a

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* 1.21.5

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* 1.16.6

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* 1.16.6

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* do not check generated

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* remove unrelated numpy

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* if

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* remove generation

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* name

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* add back

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* for testing

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* no 1.16.6

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Remind release manager to remove old onnx-weekly packages after release (#3923)

* remind release manager to remove old onnx-weekly packages

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* add steps

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Simplify function definition of context-dependent functions (#3882)

* Simplify function definition of context-dependent functions

Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>

* Add missing parenthesis

Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>

* Fix errors in function defs

Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>

* Eliminate unused variable

Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>

* Add int64 type specifier to literal

Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>

Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Migration to using main branch (#3925)

* add warning announce

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

* Rename to main branch globally

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Fix the bug of shape in docs (#3927)

* fix the bug of shape

Signed-off-by: namasikanam <namasikanam@gmail.com>

* fix the bug of shape

Signed-off-by: namasikanam <namasikanam@gmail.com>

Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Append dim even both dim value and param are not set (#3828)

* Append dim even both dim value and param are not set

Signed-off-by: Joe <joe@preferred.jp>

* Copy dim

Signed-off-by: Joe <joe@preferred.jp>

* Revert "Copy dim"

This reverts commit 6623505.

Signed-off-by: Joe <joe@preferred.jp>

* Simplify code

Signed-off-by: Joe <joe@preferred.jp>

Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* bump to 10.15 in azp (#3941)

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Add explanation to run gtest (#3933)

Signed-off-by: Joe <joe@preferred.jp>

Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

* Update TreeEnsembleClassifier and TreeEnsembleRegressor to support tensor as attributes (#3897)

* update TreeEnsembleClassifier and TreeEnsembleRegressor

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* change the type of another attribute

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* add missing file

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* update documentation

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* eol

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* add field with _as_tensor

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* fix error messages

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* fix error message

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

* fix missing change

Signed-off-by: xavier dupré <xavier.dupre@gmail.com>

Co-authored-by: xavier dupré <xavier.dupre@gmail.com>
Co-authored-by: G. Ramalingam <grama@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
Co-authored-by: Shinichiro Hamaji <shinichiro.hamaji@gmail.com>
Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com>
Co-authored-by: Chun-Wei Chen <jacky82226@gmail.com>
Co-authored-by: Matteo Salvarezza <matteo.salvarezza@gmail.com>
Co-authored-by: G. Ramalingam <grama@microsoft.com>
Co-authored-by: Xingyu Xie <namasikanam@gmail.com>
Co-authored-by: Joe <joe@preferred.jp>
Co-authored-by: Xavier Dupré <xadupre@users.noreply.github.com>
Co-authored-by: xavier dupré <xavier.dupre@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
* Infer shape in ConstantOfShape via data propagation

Signed-off-by: Joe <joe@preferred.jp>

* Remove unnecessary lines

Signed-off-by: Joe <joe@preferred.jp>

* Fix comment

Signed-off-by: Joe <joe@preferred.jp>

* Add test

Signed-off-by: Joe <joe@preferred.jp>

Co-authored-by: Chun-Wei Chen <jacky82226@gmail.com>
Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
…nightly (#3953)

* Remove testing ort-nightly for Mac Python 3.6 due to unsupported packages

Signed-off-by: Liqun Fu <liqfu@microsoft.com>
* mypy: update to 0.760, remove protobuf stubs

- We can remove the stubs in favor of types-protobuf package
- mypy==0.600 doesn't install properly for me, updating to a somewhat newer version that does install but doesn't change much logic otherwise

Signed-off-by: Chris Hua <hua.christopher@gmail.com>

* mypy: use setattr

per python/mypy#2427

Signed-off-by: Chris Hua <hua.christopher@gmail.com>

* mypy: fix type signatures

Signed-off-by: Christopher Hua <chua@squareup.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
* Trim trailing space
* Use relative links. Absolute links were needed because of
  <dhimmel/obonet#12>, which is fixed now.
* Fix typos.
* master -> main.

Signed-off-by: Gary Miguel <garymiguel@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Python 2 is not supported anymore.

Signed-off-by: Gary Miguel <garymiguel@microsoft.com>

Co-authored-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
@liqunfu liqunfu force-pushed the liqun/extract_model_w_local_func_fails_3938 branch from faea61a to acb1c7d Compare January 24, 2022 20:27
@liqunfu liqunfu requested a review from a team as a code owner January 24, 2022 20:27
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
onnx/helper.py Outdated Show resolved Hide resolved
Copy link
Contributor

@gramalingam gramalingam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Just have a minor suggestion for make_function

Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
@liqunfu liqunfu merged commit 2e0ff9c into onnx:main Jan 26, 2022
@liqunfu liqunfu deleted the liqun/extract_model_w_local_func_fails_3938 branch January 26, 2022 00:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants